piecrust2: piecrust/page.py annotate

annotate piecrust/page.py @ 1026:d85de09f40c7

bake: Fix bug with postponed pages due to them using other sources.

author	Ludovic Chabant <ludovic@chabant.com>
date	Mon, 25 Dec 2017 19:01:27 -0800
parents	2e5c5d33d62c
children	2e269c02ab97

rev	line source
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import re
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import json
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import datetime
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 49 diff changeset	6 import collections
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	7 from werkzeug.utils import cached_property
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	8 from piecrust.configuration import (
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	9 Configuration, ConfigurationError,
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	10 parse_config_header,
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	11 MERGE_PREPEND_LISTS)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17 class PageConfiguration(Configuration):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 def __init__(self, values=None, validate=True):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19 super(PageConfiguration, self).__init__(values, validate)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 def _validateAll(self, values):
957 84d8fadf9e67 page: Default to an empty title instead of 'Untitled Page'. Ludovic Chabant <ludovic@chabant.com> parents: 924 diff changeset	22 values.setdefault('title', '')
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 values.setdefault('content_type', 'html')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 ppp = values.get('posts_per_page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 if ppp is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 values.setdefault('items_per_page', ppp)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 pf = values.get('posts_filters')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28 if pf is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 values.setdefault('items_filters', pf)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30 return values
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	31
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	32
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	33 FLAG_NONE = 0
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	34 FLAG_RAW_CACHE_VALID = 2**0
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	35
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	36
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	37 class PageNotFoundError(Exception):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	38 pass
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	39
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	40
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	41 class Page:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	42 """ Represents a page that is text content with an optional YAML
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	43 front-matter, and that goes through the page pipeline.
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	44 """
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	45 def __init__(self, source, content_item):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46 self.source = source
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	47 self.content_item = content_item
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48 self._config = None
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	49 self._segments = None
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	50 self._flags = FLAG_NONE
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51 self._datetime = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	52
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	53 @cached_property
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	54 def app(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	55 return self.source.app
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	56
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	57 @cached_property
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	58 def route(self):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	59 return self.source.route
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 @property
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	62 def source_metadata(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	63 return self.content_item.metadata
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	64
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	65 @property
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	66 def content_spec(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	67 return self.content_item.spec
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	69 @cached_property
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	70 def content_mtime(self):
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	71 return self.source.getItemMtime(self.content_item)
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	72
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73 @property
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	74 def flags(self):
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	75 return self._flags
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	76
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	77 @property
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 def config(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 self._load()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 return self._config
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 @property
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	83 def segments(self):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84 self._load()
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	85 return self._segments
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 def datetime(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 if self._datetime is None:
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	90 try:
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	91 self._datetime = _compute_datetime(self.source_metadata,
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	92 self.config)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	93 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	94 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	95 raise Exception(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	96 "Error computing time for page: %s" %
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	97 self.content_spec) from ex
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	98
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	99 if self._datetime is None:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	100 self._datetime = datetime.datetime.fromtimestamp(
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	101 self.content_mtime)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	102
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	103 return self._datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	104
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105 @datetime.setter
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106 def datetime(self, value):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	107 self._datetime = value
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	108
991 1857dbd4580f bake: Fix bugs introduced by bake optimizations, of course. Ludovic Chabant <ludovic@chabant.com> parents: 979 diff changeset	109 @property
1857dbd4580f bake: Fix bugs introduced by bake optimizations, of course. Ludovic Chabant <ludovic@chabant.com> parents: 979 diff changeset	110 def was_modified(self):
1857dbd4580f bake: Fix bugs introduced by bake optimizations, of course. Ludovic Chabant <ludovic@chabant.com> parents: 979 diff changeset	111 return (self._flags & FLAG_RAW_CACHE_VALID) == 0
1857dbd4580f bake: Fix bugs introduced by bake optimizations, of course. Ludovic Chabant <ludovic@chabant.com> parents: 979 diff changeset	112
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	113 def getUri(self, sub_num=1):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	114 route_params = self.source_metadata['route_params']
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	115 return self.route.getUri(route_params, sub_num=sub_num)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	116
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	117 def getSegment(self, name='content'):
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	118 return self.segments[name]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	119
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120 def _load(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121 if self._config is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122 return
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	123
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	124 config, content, was_cache_valid = load_page(
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	125 self.source, self.content_item)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	126
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	127 extra_config = self.source_metadata.get('config')
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	128 if extra_config is not None:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	129 # Merge the source metadata configuration settings with the
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	130 # configuration settings from the page's contents. We only
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	131 # prepend to lists, i.e. we don't overwrite values because we
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	132 # want to keep what the user wrote in the file.
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	133 config.merge(extra_config, mode=MERGE_PREPEND_LISTS)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	134
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	135 self._config = config
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	136 self._segments = content
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	137 if was_cache_valid:
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	138 self._flags \|= FLAG_RAW_CACHE_VALID
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	139
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	140
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	141 def _compute_datetime(source_metadata, config):
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	142 # Get the date/time from the source.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	143 dt = source_metadata.get('datetime')
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	144 if dt is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	145 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	146
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	147 # Get the date from the source. Potentially get the
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	148 # time from the page config.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	149 page_date = source_metadata.get('date')
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	150 if page_date is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	151 dt = datetime.datetime(
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	152 page_date.year, page_date.month, page_date.day)
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	153
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	154 page_time = _parse_config_time(config.get('time'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	155 if page_time is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	156 dt += page_time
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	157
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	158 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	159
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	160 # Get the date from the page config, and maybe the
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	161 # time too.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	162 page_date = _parse_config_date(config.get('date'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	163 if page_date is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	164 dt = datetime.datetime(
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	165 page_date.year, page_date.month, page_date.day)
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	166
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	167 page_time = _parse_config_time(config.get('time'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	168 if page_time is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	169 dt += page_time
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	170
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	171 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	172
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	173 # No idea what the date/time for this page is.
1005 2e5c5d33d62c Merge changes. Ludovic Chabant <ludovic@chabant.com> parents: 991 995 diff changeset	174 return None
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	175
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	176
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	177 def _parse_config_date(page_date):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	178 if page_date is None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	179 return None
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	180
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	181 if isinstance(page_date, str):
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	182 import dateutil.parser
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	183 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	184 parsed_d = dateutil.parser.parse(page_date)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	185 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	186 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	187 raise ConfigurationError("Invalid date: %s" % page_date) from ex
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	188 return datetime.date(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	189 year=parsed_d.year,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	190 month=parsed_d.month,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	191 day=parsed_d.day)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	192
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	193 raise ConfigurationError("Invalid date: %s" % page_date)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	194
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	195
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	196 def _parse_config_time(page_time):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	197 if page_time is None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	198 return None
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	199
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	200 if isinstance(page_time, datetime.timedelta):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	201 return page_time
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	202
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	203 if isinstance(page_time, str):
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	204 import dateutil.parser
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	205 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	206 parsed_t = dateutil.parser.parse(page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	207 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	208 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	209 raise ConfigurationError("Invalid time: %s" % page_time) from ex
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	210 return datetime.timedelta(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	211 hours=parsed_t.hour,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	212 minutes=parsed_t.minute,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	213 seconds=parsed_t.second)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	214
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	215 if isinstance(page_time, int):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	216 # Total seconds... convert to a time struct.
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	217 return datetime.timedelta(seconds=page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	218
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	219 raise ConfigurationError("Invalid time: %s" % page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	220
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	221
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	222 class PageLoadingError(Exception):
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	223 def __init__(self, spec):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	224 super().__init__("Error loading page: %s" % spec)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	225
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	226
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	227 class ContentSegment(object):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	228 debug_render_func = 'debug_render'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	229
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	230 def __init__(self, content, fmt=None, offset=-1, line=-1):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	231 self.content = content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	232 self.fmt = fmt
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	233 self.offset = offset
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	234 self.line = line
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	235
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	236 def debug_render(self):
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	237 return '[%s] %s' % (self.fmt or '<none>', self.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	238
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	239
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	240 def json_load_segments(data):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	241 segments = {}
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	242 for key, sd in data.items():
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	243 seg = ContentSegment(sd['c'], sd['f'], sd['o'], sd['l'])
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	244 segments[key] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	245 return segments
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	246
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	247
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	248 def json_save_segments(segments):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	249 data = {}
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	250 for key, seg in segments.items():
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	251 seg_data = {
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	252 'c': seg.content, 'f': seg.fmt, 'o': seg.offset, 'l': seg.line}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	253 data[key] = seg_data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	254 return data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	255
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	256
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	257 def load_page(source, content_item):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	258 try:
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	259 with source.app.env.stats.timerScope('PageLoad'):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	260 return _do_load_page(source, content_item)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	261 except Exception as e:
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	262 logger.exception("Error loading page: %s" % content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	263 raise PageLoadingError(content_item.spec) from e
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	264
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	265
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	266 def _do_load_page(source, content_item):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	267 # Check the cache first.
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	268 app = source.app
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	269 cache = app.cache.getCache('pages')
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	270 cache_token = "%s@%s" % (source.name, content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	271 cache_path = hashlib.md5(cache_token.encode('utf8')).hexdigest() + '.json'
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	272 page_time = source.getItemMtime(content_item)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	273 if cache.isValid(cache_path, page_time):
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	274 cache_data = json.loads(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	275 cache.read(cache_path),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	276 object_pairs_hook=collections.OrderedDict)
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	277 config = PageConfiguration(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	278 values=cache_data['config'],
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	279 validate=False)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	280 content = json_load_segments(cache_data['content'])
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	281 return config, content, True
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	282
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	283 # Nope, load the page from the source file.
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	284 logger.debug("Loading page configuration from: %s" % content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	285 with source.openItem(content_item, 'r', encoding='utf-8') as fp:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	286 raw = fp.read()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	287 header, offset = parse_config_header(raw)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	288
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	289 config = PageConfiguration(header)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	290 content = parse_segments(raw, offset)
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	291 config.set('segments', list(content.keys()))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	292
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	293 # Save to the cache.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	294 cache_data = {
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	295 'config': config.getAll(),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	296 'content': json_save_segments(content)}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	297 cache.write(cache_path, json.dumps(cache_data))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	298
876 d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 856 diff changeset	299 app.env.stats.stepCounter('PageLoads')
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 856 diff changeset	300
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	301 return config, content, False
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	302
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	303
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	304 segment_pattern = re.compile(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	305 r"""^\-\-\-\s(?P<name>\w+)(\:(?P<fmt>\w+))?\s\-\-\-\s*$""",
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	306 re.M)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	307
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	308
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	309 def _count_lines(txt, start=0, end=-1):
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	310 cur = start
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	311 line_count = 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	312 while True:
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	313 nex = txt.find('\n', cur)
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	314 if nex < 0 or (end >= 0 and nex >= end):
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	315 break
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	316
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	317 cur = nex + 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	318 line_count += 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	319
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	320 if end >= 0 and cur >= end:
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	321 break
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	322
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	323 return line_count
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	324
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	325
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	326 def _string_needs_parsing(txt, offset):
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	327 txtlen = len(txt)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	328 index = txt.find('-', offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	329 while index >= 0 and index < txtlen - 8:
727 71309814e88f bug: Also look for format changes when determining if a page needs parsing. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	330 # Look for a potential `---segment---`
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	331 if (index > 0 and
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	332 txt[index - 1] == '\n' and
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	333 txt[index + 1] == '-' and txt[index + 2] == '-'):
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	334 return True
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	335 index = txt.find('-', index + 1)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	336 return False
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	337
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	338
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	339 def parse_segments(raw, offset=0):
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	340 # Get the number of lines in the header.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	341 header_lines = _count_lines(raw, 0, offset)
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	342 current_line = header_lines
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	343
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	344 # Figure out if we need any parsing.
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	345 do_parse = _string_needs_parsing(raw, offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	346 if not do_parse:
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	347 seg = ContentSegment(raw[offset:], None, offset, current_line)
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	348 return {'content': seg}
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	349
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	350 # Start parsing segments.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	351 matches = list(segment_pattern.finditer(raw, offset))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	352 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	353 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	354 contents = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	355
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	356 first_offset = matches[0].start()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	357 if first_offset > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	358 # There's some default content segment at the beginning.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	359 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	360 raw[offset:first_offset], None, offset, current_line)
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	361 current_line += _count_lines(seg.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	362 contents['content'] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	363
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	364 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	365 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	366 m2 = matches[i]
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	367
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	368 cur_seg_start = m1.end() + 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	369 cur_seg_end = m2.start()
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	370
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	371 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	372 raw[cur_seg_start:cur_seg_end],
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	373 m1.group('fmt'),
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	374 cur_seg_start,
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	375 current_line)
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	376 current_line += _count_lines(seg.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	377 contents[m1.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	378
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	379 # Handle text past the last match.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	380 lastm = matches[-1]
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	381
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	382 last_seg_start = lastm.end() + 1
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	383
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	384 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	385 raw[last_seg_start:],
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	386 lastm.group('fmt'),
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	387 last_seg_start,
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	388 current_line)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	389 contents[lastm.group('name')] = seg
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	390 # No need to count lines for the last one.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	391
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	392 return contents
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	393 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	394 # No segments, just content.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	395 seg = ContentSegment(raw[offset:], None, offset, current_line)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	396 return {'content': seg}

Mercurial > piecrust2

annotate piecrust/page.py @ 1026:d85de09f40c7