piecrust2: piecrust/page.py annotate

annotate piecrust/page.py @ 979:45ad976712ec

tests: Big push to get the tests to pass again. - Lots of fixes everywhere in the code. - Try to handle debug logging in the multiprocessing worker pool when running in pytest. Not perfect, but usable for now. - Replace all `.md` test files with `.html` since now a auto-format extension always sets the format. - Replace `out` with `outfiles` in most places since now blog archives are added to the bake output and I don't want to add expected outputs for blog archives everywhere.

author	Ludovic Chabant <ludovic@chabant.com>
date	Sun, 29 Oct 2017 22:51:57 -0700
parents	84d8fadf9e67
children	1857dbd4580f

rev	line source
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import re
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import json
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import datetime
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 49 diff changeset	6 import collections
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	7 from werkzeug.utils import cached_property
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	8 from piecrust.configuration import (
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	9 Configuration, ConfigurationError,
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	10 parse_config_header,
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	11 MERGE_PREPEND_LISTS)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17 class PageConfiguration(Configuration):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 def __init__(self, values=None, validate=True):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19 super(PageConfiguration, self).__init__(values, validate)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 def _validateAll(self, values):
957 84d8fadf9e67 page: Default to an empty title instead of 'Untitled Page'. Ludovic Chabant <ludovic@chabant.com> parents: 924 diff changeset	22 values.setdefault('title', '')
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 values.setdefault('content_type', 'html')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 ppp = values.get('posts_per_page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 if ppp is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 values.setdefault('items_per_page', ppp)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 pf = values.get('posts_filters')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28 if pf is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 values.setdefault('items_filters', pf)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30 return values
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	31
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	32
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	33 FLAG_NONE = 0
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	34 FLAG_RAW_CACHE_VALID = 2**0
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	35
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	36
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	37 class PageNotFoundError(Exception):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	38 pass
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	39
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	40
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	41 class Page:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	42 """ Represents a page that is text content with an optional YAML
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	43 front-matter, and that goes through the page pipeline.
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	44 """
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	45 def __init__(self, source, content_item):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	46 self.source = source
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	47 self.content_item = content_item
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48 self._config = None
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	49 self._segments = None
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	50 self._flags = FLAG_NONE
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51 self._datetime = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	52
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	53 @cached_property
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	54 def app(self):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	55 return self.source.app
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	56
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	57 @cached_property
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	58 def route(self):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	59 return self.source.route
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 @property
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	62 def source_metadata(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	63 return self.content_item.metadata
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	64
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	65 @property
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	66 def content_spec(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	67 return self.content_item.spec
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	68
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	69 @cached_property
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	70 def content_mtime(self):
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	71 return self.source.getItemMtime(self.content_item)
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	72
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73 @property
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	74 def flags(self):
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	75 return self._flags
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	76
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	77 @property
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 def config(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 self._load()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 return self._config
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 @property
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	83 def segments(self):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84 self._load()
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	85 return self._segments
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 def datetime(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 if self._datetime is None:
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	90 try:
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	91 self._datetime = _compute_datetime(self.source_metadata,
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	92 self.config)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	93 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	94 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	95 raise Exception(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	96 "Error computing time for page: %s" %
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	97 self.content_spec) from ex
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	98
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	99 if self._datetime is None:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	100 self._datetime = datetime.datetime.fromtimestamp(
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	101 self.content_mtime)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	102
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	103 return self._datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	104
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105 @datetime.setter
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106 def datetime(self, value):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	107 self._datetime = value
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	108
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	109 def getUri(self, sub_num=1):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	110 route_params = self.source_metadata['route_params']
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	111 return self.route.getUri(route_params, sub_num=sub_num)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	112
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	113 def getSegment(self, name='content'):
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	114 return self.segments[name]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	115
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	116 def _load(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	117 if self._config is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	118 return
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	119
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	120 config, content, was_cache_valid = load_page(
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	121 self.source, self.content_item)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	122
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	123 extra_config = self.source_metadata.get('config')
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	124 if extra_config is not None:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	125 # Merge the source metadata configuration settings with the
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	126 # configuration settings from the page's contents. We only
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	127 # prepend to lists, i.e. we don't overwrite values because we
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	128 # want to keep what the user wrote in the file.
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	129 config.merge(extra_config, mode=MERGE_PREPEND_LISTS)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	130
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	131 self._config = config
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	132 self._segments = content
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	133 if was_cache_valid:
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	134 self._flags \|= FLAG_RAW_CACHE_VALID
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	135
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	136
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	137 def _compute_datetime(source_metadata, config):
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	138 # Get the date/time from the source.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	139 dt = source_metadata.get('datetime')
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	140 if dt is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	141 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	142
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	143 # Get the date from the source. Potentially get the
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	144 # time from the page config.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	145 page_date = source_metadata.get('date')
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	146 if page_date is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	147 dt = datetime.datetime(
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	148 page_date.year, page_date.month, page_date.day)
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	149
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	150 page_time = _parse_config_time(config.get('time'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	151 if page_time is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	152 dt += page_time
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	153
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	154 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	155
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	156 # Get the date from the page config, and maybe the
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	157 # time too.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	158 page_date = _parse_config_date(config.get('date'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	159 if page_date is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	160 dt = datetime.datetime(
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	161 page_date.year, page_date.month, page_date.day)
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	162
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	163 page_time = _parse_config_time(config.get('time'))
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	164 if page_time is not None:
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	165 dt += page_time
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	166
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	167 return dt
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	168
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	169 # No idea what the date/time for this page is.
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	170 return datetime.datetime.fromtimestamp(0)
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	171
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 853 diff changeset	172
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	173 def _parse_config_date(page_date):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	174 if page_date is None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	175 return None
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	176
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	177 if isinstance(page_date, str):
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	178 import dateutil.parser
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	179 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	180 parsed_d = dateutil.parser.parse(page_date)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	181 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	182 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	183 raise ConfigurationError("Invalid date: %s" % page_date) from ex
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	184 return datetime.date(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	185 year=parsed_d.year,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	186 month=parsed_d.month,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	187 day=parsed_d.day)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	188
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	189 raise ConfigurationError("Invalid date: %s" % page_date)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	190
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	191
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	192 def _parse_config_time(page_time):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	193 if page_time is None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	194 return None
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	195
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	196 if isinstance(page_time, datetime.timedelta):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	197 return page_time
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	198
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	199 if isinstance(page_time, str):
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	200 import dateutil.parser
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	201 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	202 parsed_t = dateutil.parser.parse(page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	203 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	204 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	205 raise ConfigurationError("Invalid time: %s" % page_time) from ex
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	206 return datetime.timedelta(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	207 hours=parsed_t.hour,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	208 minutes=parsed_t.minute,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	209 seconds=parsed_t.second)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	210
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	211 if isinstance(page_time, int):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	212 # Total seconds... convert to a time struct.
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	213 return datetime.timedelta(seconds=page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	214
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	215 raise ConfigurationError("Invalid time: %s" % page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	216
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	217
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	218 class PageLoadingError(Exception):
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	219 def __init__(self, spec):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	220 super().__init__("Error loading page: %s" % spec)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	221
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	222
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	223 class ContentSegment(object):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	224 debug_render_func = 'debug_render'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	225
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	226 def __init__(self, content, fmt=None, offset=-1, line=-1):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	227 self.content = content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	228 self.fmt = fmt
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	229 self.offset = offset
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	230 self.line = line
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	231
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	232 def debug_render(self):
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	233 return '[%s] %s' % (self.fmt or '<none>', self.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	234
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	235
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	236 def json_load_segments(data):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	237 segments = {}
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	238 for key, sd in data.items():
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	239 seg = ContentSegment(sd['c'], sd['f'], sd['o'], sd['l'])
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	240 segments[key] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	241 return segments
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	242
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	243
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	244 def json_save_segments(segments):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	245 data = {}
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	246 for key, seg in segments.items():
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	247 seg_data = {
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	248 'c': seg.content, 'f': seg.fmt, 'o': seg.offset, 'l': seg.line}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	249 data[key] = seg_data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	250 return data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	251
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	252
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	253 def load_page(source, content_item):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	254 try:
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	255 with source.app.env.stats.timerScope('PageLoad'):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	256 return _do_load_page(source, content_item)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	257 except Exception as e:
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	258 logger.exception("Error loading page: %s" % content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	259 raise PageLoadingError(content_item.spec) from e
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	260
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	261
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	262 def _do_load_page(source, content_item):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	263 # Check the cache first.
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	264 app = source.app
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	265 cache = app.cache.getCache('pages')
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	266 cache_token = "%s@%s" % (source.name, content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	267 cache_path = hashlib.md5(cache_token.encode('utf8')).hexdigest() + '.json'
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	268 page_time = source.getItemMtime(content_item)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	269 if cache.isValid(cache_path, page_time):
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	270 cache_data = json.loads(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	271 cache.read(cache_path),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	272 object_pairs_hook=collections.OrderedDict)
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	273 config = PageConfiguration(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	274 values=cache_data['config'],
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	275 validate=False)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	276 content = json_load_segments(cache_data['content'])
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	277 return config, content, True
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	278
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	279 # Nope, load the page from the source file.
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	280 logger.debug("Loading page configuration from: %s" % content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	281 with source.openItem(content_item, 'r', encoding='utf-8') as fp:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	282 raw = fp.read()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	283 header, offset = parse_config_header(raw)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	284
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	285 config = PageConfiguration(header)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	286 content = parse_segments(raw, offset)
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	287 config.set('segments', list(content.keys()))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	288
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	289 # Save to the cache.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	290 cache_data = {
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	291 'config': config.getAll(),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	292 'content': json_save_segments(content)}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	293 cache.write(cache_path, json.dumps(cache_data))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	294
876 d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 856 diff changeset	295 app.env.stats.stepCounter('PageLoads')
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 856 diff changeset	296
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	297 return config, content, False
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	298
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	299
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	300 segment_pattern = re.compile(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	301 r"""^\-\-\-\s(?P<name>\w+)(\:(?P<fmt>\w+))?\s\-\-\-\s*$""",
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 824 diff changeset	302 re.M)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	303
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	304
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	305 def _count_lines(txt, start=0, end=-1):
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	306 cur = start
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	307 line_count = 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	308 while True:
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	309 nex = txt.find('\n', cur)
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	310 if nex < 0 or (end >= 0 and nex >= end):
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	311 break
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	312
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	313 cur = nex + 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	314 line_count += 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	315
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	316 if end >= 0 and cur >= end:
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	317 break
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	318
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	319 return line_count
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	320
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	321
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	322 def _string_needs_parsing(txt, offset):
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	323 txtlen = len(txt)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	324 index = txt.find('-', offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	325 while index >= 0 and index < txtlen - 8:
727 71309814e88f bug: Also look for format changes when determining if a page needs parsing. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	326 # Look for a potential `---segment---`
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	327 if (index > 0 and
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	328 txt[index - 1] == '\n' and
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	329 txt[index + 1] == '-' and txt[index + 2] == '-'):
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	330 return True
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	331 index = txt.find('-', index + 1)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	332 return False
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	333
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	334
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	335 def parse_segments(raw, offset=0):
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	336 # Get the number of lines in the header.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	337 header_lines = _count_lines(raw, 0, offset)
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	338 current_line = header_lines
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	339
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	340 # Figure out if we need any parsing.
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	341 do_parse = _string_needs_parsing(raw, offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	342 if not do_parse:
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	343 seg = ContentSegment(raw[offset:], None, offset, current_line)
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	344 return {'content': seg}
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	345
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	346 # Start parsing segments.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	347 matches = list(segment_pattern.finditer(raw, offset))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	348 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	349 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	350 contents = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	351
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	352 first_offset = matches[0].start()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	353 if first_offset > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	354 # There's some default content segment at the beginning.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	355 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	356 raw[offset:first_offset], None, offset, current_line)
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	357 current_line += _count_lines(seg.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	358 contents['content'] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	359
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	360 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	361 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	362 m2 = matches[i]
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	363
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	364 cur_seg_start = m1.end() + 1
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	365 cur_seg_end = m2.start()
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	366
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	367 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	368 raw[cur_seg_start:cur_seg_end],
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	369 m1.group('fmt'),
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	370 cur_seg_start,
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	371 current_line)
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	372 current_line += _count_lines(seg.content)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	373 contents[m1.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	374
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	375 # Handle text past the last match.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	376 lastm = matches[-1]
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	377
979 45ad976712ec tests: Big push to get the tests to pass again. Ludovic Chabant <ludovic@chabant.com> parents: 957 diff changeset	378 last_seg_start = lastm.end() + 1
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	379
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	380 seg = ContentSegment(
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	381 raw[last_seg_start:],
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	382 lastm.group('fmt'),
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	383 last_seg_start,
1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	384 current_line)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	385 contents[lastm.group('name')] = seg
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	386 # No need to count lines for the last one.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	387
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	388 return contents
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	389 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	390 # No segments, just content.
924 1bb704434ee2 formatting: Remove segment parts, you can use template tags instead. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	391 seg = ContentSegment(raw[offset:], None, offset, current_line)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	392 return {'content': seg}

Mercurial > piecrust2

annotate piecrust/page.py @ 979:45ad976712ec