piecrust2: piecrust/page.py annotate

annotate piecrust/page.py @ 1145:e94737572542

serve: Fix an issue where false positive matches were rendered as the requested page. Now we try to render the page, but also try to detect for the most common "empty" pages.

author	Ludovic Chabant <ludovic@chabant.com>
date	Tue, 05 Jun 2018 22:08:51 -0700
parents	af57fbe9435a
children	e307f61d7034

rev	line source
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import re
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import json
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import datetime
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 49 diff changeset	6 import collections
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	7 from werkzeug.utils import cached_property
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	8 from piecrust.configuration import (
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	9 Configuration, ConfigurationError,
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	10 parse_config_header,
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	11 MERGE_PREPEND_LISTS)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17 class PageConfiguration(Configuration):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 def __init__(self, values=None, validate=True):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19 super(PageConfiguration, self).__init__(values, validate)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 def _validateAll(self, values):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22 values.setdefault('content_type', 'html')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 ppp = values.get('posts_per_page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 if ppp is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 values.setdefault('items_per_page', ppp)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 pf = values.get('posts_filters')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 if pf is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28 values.setdefault('items_filters', pf)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 return values
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	31
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	32 FLAG_NONE = 0
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	33 FLAG_RAW_CACHE_VALID = 2**0
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	34
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	35
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	36 class PageNotFoundError(Exception):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	37 pass
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	38
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	39
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	40 class Page:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	41 """ Represents a page that is text content with an optional YAML
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	42 front-matter, and that goes through the page pipeline.
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	43 """
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	44 def __init__(self, source, content_item):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	45 self.source = source
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	46 self.content_item = content_item
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 self._config = None
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	48 self._segments = None
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	49 self._flags = FLAG_NONE
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	50 self._datetime = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	52 @cached_property
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	53 def app(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	54 return self.source.app
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	55
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	56 @cached_property
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	57 def route(self):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	58 return self.source.route
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	59
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60 @property
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	61 def source_metadata(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	62 return self.content_item.metadata
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	63
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	64 @property
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	65 def content_spec(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	66 return self.content_item.spec
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	68 @cached_property
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	69 def content_mtime(self):
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	70 return self.source.getItemMtime(self.content_item)
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	71
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 @property
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	73 def flags(self):
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	74 return self._flags
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	75
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	76 @property
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	77 def config(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 self._load()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 return self._config
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 @property
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	82 def segments(self):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 self._load()
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	84 return self._segments
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	85
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87 def datetime(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 if self._datetime is None:
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	89 try:
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	90 self._datetime = _compute_datetime(self.source_metadata,
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	91 self.config)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	92 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	93 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	94 raise Exception(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	95 "Error computing time for page: %s" %
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	96 self.content_spec) from ex
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	97
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	98 if self._datetime is None:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	99 self._datetime = datetime.datetime.fromtimestamp(
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	100 self.content_mtime)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	101
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	102 return self._datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	103
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	104 @datetime.setter
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105 def datetime(self, value):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106 self._datetime = value
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	107
991 1857dbd4580f bake: Fix bugs introduced by bake optimizations, of course. Ludovic Chabant <ludovic@chabant.com> parents: 979 diff changeset	108 @property
1857dbd4580f bake: Fix bugs introduced by bake optimizations, of course. Ludovic Chabant <ludovic@chabant.com> parents: 979 diff changeset	109 def was_modified(self):
1857dbd4580f bake: Fix bugs introduced by bake optimizations, of course. Ludovic Chabant <ludovic@chabant.com> parents: 979 diff changeset	110 return (self._flags & FLAG_RAW_CACHE_VALID) == 0
1857dbd4580f bake: Fix bugs introduced by bake optimizations, of course. Ludovic Chabant <ludovic@chabant.com> parents: 979 diff changeset	111
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	112 def getUri(self, sub_num=1):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	113 route_params = self.source_metadata['route_params']
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	114 return self.route.getUri(route_params, sub_num=sub_num)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	115
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	116 def getSegment(self, name='content'):
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	117 return self.segments[name]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	118
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	119 def _load(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120 if self._config is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121 return
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	123 config, content, was_cache_valid = load_page(
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	124 self.source, self.content_item)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	125
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	126 extra_config = self.source_metadata.get('config')
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	127 if extra_config is not None:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	128 # Merge the source metadata configuration settings with the
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	129 # configuration settings from the page's contents. We only
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	130 # prepend to lists, i.e. we don't overwrite values because we
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	131 # want to keep what the user wrote in the file.
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	132 config.merge(extra_config, mode=MERGE_PREPEND_LISTS)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	133
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	134 self._config = config
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	135 self._segments = content
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	136 if was_cache_valid:
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	137 self._flags \|= FLAG_RAW_CACHE_VALID
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	138
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	139
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	140 def _compute_datetime(source_metadata, config):
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	141 # Get the date/time from the source.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	142 dt = source_metadata.get('datetime')
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	143 if dt is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	144 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	145
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	146 # Get the date from the source. Potentially get the
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	147 # time from the page config.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	148 page_date = source_metadata.get('date')
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	149 if page_date is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	150 dt = datetime.datetime(
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	151 page_date.year, page_date.month, page_date.day)
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	152
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	153 page_time = _parse_config_time(config.get('time'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	154 if page_time is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	155 dt += page_time
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	156
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	157 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	158
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	159 # Get the date from the page config, and maybe the
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	160 # time too.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	161 page_date = _parse_config_date(config.get('date'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	162 if page_date is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	163 dt = datetime.datetime(
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	164 page_date.year, page_date.month, page_date.day)
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	165
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	166 page_time = _parse_config_time(config.get('time'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	167 if page_time is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	168 dt += page_time
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	169
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	170 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	171
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	172 # No idea what the date/time for this page is.
1005 2e5c5d33d62c Merge changes. Ludovic Chabant <ludovic@chabant.com> parents: 991 995 diff changeset	173 return None
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	174
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	175
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	176 def _parse_config_date(page_date):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	177 if page_date is None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	178 return None
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	179
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	180 if isinstance(page_date, str):
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	181 import dateutil.parser
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	182 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	183 parsed_d = dateutil.parser.parse(page_date)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	184 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	185 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	186 raise ConfigurationError("Invalid date: %s" % page_date) from ex
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	187 return datetime.date(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	188 year=parsed_d.year,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	189 month=parsed_d.month,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	190 day=parsed_d.day)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	191
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	192 raise ConfigurationError("Invalid date: %s" % page_date)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	193
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	194
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	195 def _parse_config_time(page_time):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	196 if page_time is None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	197 return None
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	198
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	199 if isinstance(page_time, datetime.timedelta):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	200 return page_time
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	201
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	202 if isinstance(page_time, str):
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	203 import dateutil.parser
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	204 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	205 parsed_t = dateutil.parser.parse(page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	206 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	207 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	208 raise ConfigurationError("Invalid time: %s" % page_time) from ex
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	209 return datetime.timedelta(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	210 hours=parsed_t.hour,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	211 minutes=parsed_t.minute,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	212 seconds=parsed_t.second)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	213
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	214 if isinstance(page_time, int):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	215 # Total seconds... convert to a time struct.
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	216 return datetime.timedelta(seconds=page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	217
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	218 raise ConfigurationError("Invalid time: %s" % page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	219
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	220
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	221 class PageLoadingError(Exception):
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	222 def __init__(self, spec):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	223 super().__init__("Error loading page: %s" % spec)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	224
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	225
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	226 class ContentSegment(object):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	227 debug_render_func = 'debug_render'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	228
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	229 def __init__(self, content, fmt=None, offset=-1, line=-1):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	230 self.content = content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	231 self.fmt = fmt
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	232 self.offset = offset
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	233 self.line = line
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	234
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	235 def debug_render(self):
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	236 return '[%s] %s' % (self.fmt or '<none>', self.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	237
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	238
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	239 def json_load_segments(data):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	240 segments = {}
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	241 for key, sd in data.items():
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	242 seg = ContentSegment(sd['c'], sd['f'], sd['o'], sd['l'])
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	243 segments[key] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	244 return segments
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	245
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	246
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	247 def json_save_segments(segments):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	248 data = {}
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	249 for key, seg in segments.items():
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	250 seg_data = {
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	251 'c': seg.content, 'f': seg.fmt, 'o': seg.offset, 'l': seg.line}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	252 data[key] = seg_data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	253 return data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	254
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	255
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	256 def load_page(source, content_item):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	257 try:
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	258 with source.app.env.stats.timerScope('PageLoad'):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	259 return _do_load_page(source, content_item)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	260 except Exception as e:
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	261 logger.exception("Error loading page: %s" % content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	262 raise PageLoadingError(content_item.spec) from e
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	263
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	264
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	265 def _do_load_page(source, content_item):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	266 # Check the cache first.
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	267 app = source.app
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	268 cache = app.cache.getCache('pages')
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	269 cache_token = "%s@%s" % (source.name, content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	270 cache_path = hashlib.md5(cache_token.encode('utf8')).hexdigest() + '.json'
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	271 page_time = source.getItemMtime(content_item)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	272 if cache.isValid(cache_path, page_time):
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	273 cache_data = json.loads(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	274 cache.read(cache_path),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	275 object_pairs_hook=collections.OrderedDict)
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	276 config = PageConfiguration(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	277 values=cache_data['config'],
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	278 validate=False)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	279 content = json_load_segments(cache_data['content'])
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	280 return config, content, True
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	281
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	282 # Nope, load the page from the source file.
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	283 logger.debug("Loading page configuration from: %s" % content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	284 with source.openItem(content_item, 'r', encoding='utf-8') as fp:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	285 raw = fp.read()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	286 header, offset = parse_config_header(raw)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	287
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	288 config = PageConfiguration(header)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	289 content = parse_segments(raw, offset)
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	290 config.set('segments', list(content.keys()))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	291
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	292 # Save to the cache.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	293 cache_data = {
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	294 'config': config.getAll(),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	295 'content': json_save_segments(content)}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	296 cache.write(cache_path, json.dumps(cache_data))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	297
876 d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 856 diff changeset	298 app.env.stats.stepCounter('PageLoads')
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 856 diff changeset	299
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	300 return config, content, False
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	301
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	302
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	303 segment_pattern = re.compile(
1119 af57fbe9435a formatter: Fix the regex used to find page segments. Ludovic Chabant <ludovic@chabant.com> parents: 1090 diff changeset	304 r"^\-\-\-[ \t](?P<name>\w+)(\:(?P<fmt>\w+))?[ \t]\-\-\-[ \t]*$", re.M)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	305
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	306
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	307 def _count_lines(txt, start=0, end=-1):
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	308 cur = start
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	309 line_count = 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	310 while True:
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	311 nex = txt.find('\n', cur)
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	312 if nex < 0 or (end >= 0 and nex >= end):
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	313 break
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	314
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	315 cur = nex + 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	316 line_count += 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	317
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	318 if end >= 0 and cur >= end:
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	319 break
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	320
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	321 return line_count
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	322
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	323
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	324 def _string_needs_parsing(txt, offset):
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	325 txtlen = len(txt)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	326 index = txt.find('-', offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	327 while index >= 0 and index < txtlen - 8:
727 71309814e88f bug: Also look for format changes when determining if a page needs parsing. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	328 # Look for a potential `---segment---`
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	329 if (index > 0 and
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	330 txt[index - 1] == '\n' and
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	331 txt[index + 1] == '-' and txt[index + 2] == '-'):
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	332 return True
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	333 index = txt.find('-', index + 1)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	334 return False
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	335
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	336
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	337 def parse_segments(raw, offset=0):
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	338 # Get the number of lines in the header.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	339 header_lines = _count_lines(raw, 0, offset)
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	340 current_line = header_lines
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	341
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	342 # Figure out if we need any parsing.
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	343 do_parse = _string_needs_parsing(raw, offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	344 if not do_parse:
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	345 seg = ContentSegment(raw[offset:], None, offset, current_line)
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	346 return {'content': seg}
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	347
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	348 # Start parsing segments.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	349 matches = list(segment_pattern.finditer(raw, offset))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	350 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	351 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	352 contents = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	353
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	354 first_offset = matches[0].start()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	355 if first_offset > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	356 # There's some default content segment at the beginning.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	357 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	358 raw[offset:first_offset], None, offset, current_line)
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	359 current_line += _count_lines(seg.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	360 contents['content'] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	361
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	362 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	363 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	364 m2 = matches[i]
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	365
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	366 cur_seg_start = m1.end() + 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	367 cur_seg_end = m2.start()
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	368
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	369 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	370 raw[cur_seg_start:cur_seg_end],
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	371 m1.group('fmt'),
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	372 cur_seg_start,
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	373 current_line)
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	374 current_line += _count_lines(seg.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	375 contents[m1.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	376
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	377 # Handle text past the last match.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	378 lastm = matches[-1]
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	379
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	380 last_seg_start = lastm.end() + 1
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	381
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	382 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	383 raw[last_seg_start:],
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	384 lastm.group('fmt'),
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	385 last_seg_start,
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	386 current_line)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	387 contents[lastm.group('name')] = seg
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	388 # No need to count lines for the last one.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	389
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	390 return contents
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	391 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	392 # No segments, just content.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	393 seg = ContentSegment(raw[offset:], None, offset, current_line)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	394 return {'content': seg}

Mercurial > piecrust2

annotate piecrust/page.py @ 1145:e94737572542