piecrust2: piecrust/page.py annotate

annotate piecrust/page.py @ 840:7f3043f9f26f

internal: Don't check for a page repository, there's always one.

author	Ludovic Chabant <ludovic@chabant.com>
date	Wed, 15 Feb 2017 22:17:13 -0800
parents	e01473c3ea7e
children	4850f8c21b6e 601f4a236ee5

rev	line source
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import re
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import sys
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import json
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import os.path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	6 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	7 import datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	8 import dateutil.parser
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 49 diff changeset	9 import collections
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	10 from werkzeug.utils import cached_property
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	11 from piecrust.configuration import (
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	12 Configuration, ConfigurationError,
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13 parse_config_header)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19 class PageConfiguration(Configuration):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20 def __init__(self, values=None, validate=True):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 super(PageConfiguration, self).__init__(values, validate)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 def _validateAll(self, values):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 values.setdefault('title', 'Untitled Page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 values.setdefault('content_type', 'html')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 ppp = values.get('posts_per_page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 if ppp is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28 values.setdefault('items_per_page', ppp)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 pf = values.get('posts_filters')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30 if pf is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	31 values.setdefault('items_filters', pf)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	32 return values
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	33
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	34
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	35 FLAG_NONE = 0
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	36 FLAG_RAW_CACHE_VALID = 2**0
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	37
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	38
792 58ebf50235a5 routing: Simplify how routes are defined. Ludovic Chabant <ludovic@chabant.com> parents: 728 diff changeset	39 class Page(object):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	40 def __init__(self, source, source_metadata, rel_path):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	41 self.source = source
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	42 self.source_metadata = source_metadata
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	43 self.rel_path = rel_path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	44 self._config = None
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	45 self._segments = None
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	46 self._flags = FLAG_NONE
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 self._datetime = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	49 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	50 def app(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51 return self.source.app
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	52
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	53 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	54 def ref_spec(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	55 return '%s:%s' % (self.source.name, self.rel_path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	56
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	57 @cached_property
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	58 def path(self):
363 dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages. Ludovic Chabant <ludovic@chabant.com> parents: 336 diff changeset	59 path, _ = self.source.resolveRef(self.rel_path)
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages. Ludovic Chabant <ludovic@chabant.com> parents: 336 diff changeset	60 return path
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	61
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	62 @cached_property
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	63 def path_mtime(self):
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	64 return os.path.getmtime(self.path)
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	65
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66 @property
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	67 def flags(self):
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	68 return self._flags
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	69
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	70 @property
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71 def config(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 self._load()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73 return self._config
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	74
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	75 @property
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	76 def segments(self):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	77 self._load()
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	78 return self._segments
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 def datetime(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 if self._datetime is None:
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	83 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	84 if 'datetime' in self.source_metadata:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	85 # Get the date/time from the source.
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	86 self._datetime = self.source_metadata['datetime']
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	87 elif 'date' in self.source_metadata:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	88 # Get the date from the source. Potentially get the
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	89 # time from the page config.
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	90 page_date = self.source_metadata['date']
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	91 page_time = _parse_config_time(self.config.get('time'))
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	92 if page_time is not None:
108 b6ec402d32bb Properly add the config time to a page's datetime. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	93 self._datetime = datetime.datetime(
b6ec402d32bb Properly add the config time to a page's datetime. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	94 page_date.year,
b6ec402d32bb Properly add the config time to a page's datetime. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	95 page_date.month,
b6ec402d32bb Properly add the config time to a page's datetime. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	96 page_date.day) + page_time
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	97 else:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	98 self._datetime = datetime.datetime(
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	99 page_date.year, page_date.month, page_date.day)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	100 elif 'date' in self.config:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	101 # Get the date from the page config, and maybe the
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	102 # time too.
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	103 page_date = _parse_config_date(self.config.get('date'))
90 e293f08d954e PyYAML supports sexagesimal notation, so handle that for page times. Ludovic Chabant <ludovic@chabant.com> parents: 67 diff changeset	104 self._datetime = datetime.datetime(
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	105 page_date.year,
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	106 page_date.month,
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	107 page_date.day)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	108 page_time = _parse_config_time(self.config.get('time'))
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	109 if page_time is not None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	110 self._datetime += page_time
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	111 else:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	112 # No idea what the date/time for this page is.
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	113 self._datetime = datetime.datetime.fromtimestamp(0)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	114 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	115 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	116 raise Exception(
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	117 "Error computing time for page: %s" %
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	118 self.path) from ex
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	119 return self._datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121 @datetime.setter
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122 def datetime(self, value):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	123 self._datetime = value
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	124
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	125 def getSegment(self, name='content'):
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	126 return self.segments[name]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	127
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	128 def _load(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	129 if self._config is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	130 return
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	131
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	132 config, content, was_cache_valid = load_page(self.app, self.path,
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	133 self.path_mtime)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	134 if 'config' in self.source_metadata:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	135 config.merge(self.source_metadata['config'])
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	136
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	137 self._config = config
578 683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is. Ludovic Chabant <ludovic@chabant.com> parents: 458 diff changeset	138 self._segments = content
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	139 if was_cache_valid:
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	140 self._flags \|= FLAG_RAW_CACHE_VALID
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	141
824 e01473c3ea7e Allow page source to post-process page config at the end of page loading Ben Artin <ben@artins.org> parents: 792 diff changeset	142 self.source.finalizeConfig(self)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	143
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	144 def _parse_config_date(page_date):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	145 if page_date is None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	146 return None
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	147
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	148 if isinstance(page_date, str):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	149 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	150 parsed_d = dateutil.parser.parse(page_date)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	151 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	152 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	153 raise ConfigurationError("Invalid date: %s" % page_date) from ex
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	154 return datetime.date(
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	155 year=parsed_d.year,
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	156 month=parsed_d.month,
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	157 day=parsed_d.day)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	158
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	159 raise ConfigurationError("Invalid date: %s" % page_date)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	160
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	161
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	162 def _parse_config_time(page_time):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	163 if page_time is None:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	164 return None
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	165
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	166 if isinstance(page_time, datetime.timedelta):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	167 return page_time
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	168
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	169 if isinstance(page_time, str):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	170 try:
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	171 parsed_t = dateutil.parser.parse(page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	172 except Exception as ex:
711 ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	173 logger.exception(ex)
142 0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	174 raise ConfigurationError("Invalid time: %s" % page_time) from ex
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	175 return datetime.timedelta(
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	176 hours=parsed_t.hour,
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	177 minutes=parsed_t.minute,
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	178 seconds=parsed_t.second)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	179
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	180 if isinstance(page_time, int):
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	181 # Total seconds... convert to a time struct.
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	182 return datetime.timedelta(seconds=page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	183
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	184 raise ConfigurationError("Invalid time: %s" % page_time)
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	185
0bdd938d6b9f Better date/time handling for pages: Ludovic Chabant <ludovic@chabant.com> parents: 128 diff changeset	186
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	187 class PageLoadingError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	188 def __init__(self, path, inner=None):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	189 super(PageLoadingError, self).__init__(
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	190 "Error loading page: %s" % path,
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	191 inner)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	192
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	193
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	194 class ContentSegment(object):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	195 debug_render_func = 'debug_render'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	196
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	197 def __init__(self):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	198 self.parts = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	199
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	200 def debug_render(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	201 return '\n'.join([p.content for p in self.parts])
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	202
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	203
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	204 class ContentSegmentPart(object):
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	205 def __init__(self, content, fmt=None, offset=-1, line=-1):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	206 self.content = content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	207 self.fmt = fmt
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	208 self.offset = offset
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	209 self.line = line
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	210
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	211 def __str__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	212 return '%s [%s]' % (self.content, self.fmt or '<default>')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	213
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	214
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	215 def json_load_segments(data):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	216 segments = {}
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	217 for key, seg_data in data.items():
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	218 seg = ContentSegment()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	219 for p_data in seg_data:
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	220 part = ContentSegmentPart(p_data['c'], p_data['f'], p_data['o'],
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	221 p_data['l'])
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	222 seg.parts.append(part)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	223 segments[key] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	224 return segments
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	225
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	226
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	227 def json_save_segments(segments):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	228 data = {}
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	229 for key, seg in segments.items():
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	230 seg_data = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	231 for part in seg.parts:
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	232 p_data = {'c': part.content, 'f': part.fmt, 'o': part.offset,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	233 'l': part.line}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	234 seg_data.append(p_data)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	235 data[key] = seg_data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	236 return data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	237
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	238
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	239 def load_page(app, path, path_mtime=None):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	240 try:
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	241 with app.env.timerScope('PageLoad'):
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	242 return _do_load_page(app, path, path_mtime)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	243 except Exception as e:
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	244 logger.exception(
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	245 "Error loading page: %s" %
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	246 os.path.relpath(path, app.root_dir))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	247 _, __, traceback = sys.exc_info()
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	248 raise PageLoadingError(path, e).with_traceback(traceback)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	249
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	250
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	251 def _do_load_page(app, path, path_mtime):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	252 # Check the cache first.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	253 cache = app.cache.getCache('pages')
336 aa6b7ff6a193 internal: Use hashes for cache paths. Ludovic Chabant <ludovic@chabant.com> parents: 147 diff changeset	254 cache_path = hashlib.md5(path.encode('utf8')).hexdigest() + '.json'
49 fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	255 page_time = path_mtime or os.path.getmtime(path)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	256 if cache.isValid(cache_path, page_time):
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	257 cache_data = json.loads(
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	258 cache.read(cache_path),
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 49 diff changeset	259 object_pairs_hook=collections.OrderedDict)
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	260 config = PageConfiguration(
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	261 values=cache_data['config'],
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	262 validate=False)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	263 content = json_load_segments(cache_data['content'])
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	264 return config, content, True
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	265
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	266 # Nope, load the page from the source file.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	267 logger.debug("Loading page configuration from: %s" % path)
458 612ed0526afd internal: Just use the plain old standard function. Ludovic Chabant <ludovic@chabant.com> parents: 454 diff changeset	268 with open(path, 'r', encoding='utf-8') as fp:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	269 raw = fp.read()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	270 header, offset = parse_config_header(raw)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	271
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	272 if 'format' not in header:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	273 auto_formats = app.config.get('site/auto_formats')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	274 name, ext = os.path.splitext(path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	275 header['format'] = auto_formats.get(ext, None)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	276
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	277 config = PageConfiguration(header)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	278 content = parse_segments(raw, offset)
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	279 config.set('segments', list(content.keys()))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	280
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	281 # Save to the cache.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	282 cache_data = {
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	283 'config': config.getAll(),
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	284 'content': json_save_segments(content)}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	285 cache.write(cache_path, json.dumps(cache_data))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	286
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 90 diff changeset	287 return config, content, False
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	288
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	289
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	290 segment_pattern = re.compile(
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	291 r"""^\-\-\-\s(?P<name>\w+)(\:(?P<fmt>\w+))?\s\-\-\-\s*$""",
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	292 re.M)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	293 part_pattern = re.compile(
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	294 r"""^<\-\-\s(?P<fmt>\w+)\s\-\->\s*$""",
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	295 re.M)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	296
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	297
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	298 def _count_lines(s):
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	299 return len(s.split('\n'))
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	300
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	301
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	302 def _string_needs_parsing(txt, offset):
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	303 txtlen = len(txt)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	304 index = txt.find('-', offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	305 while index >= 0 and index < txtlen - 8:
727 71309814e88f bug: Also look for format changes when determining if a page needs parsing. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	306 # Look for a potential `<--format-->`
71309814e88f bug: Also look for format changes when determining if a page needs parsing. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	307 if index > 0 and txt[index - 1] == '<' and txt[index + 1] == '-':
71309814e88f bug: Also look for format changes when determining if a page needs parsing. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	308 return True
71309814e88f bug: Also look for format changes when determining if a page needs parsing. Ludovic Chabant <ludovic@chabant.com> parents: 703 diff changeset	309 # Look for a potential `---segment---`
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	310 if txt[index + 1] == '-' and txt[index + 2] == '-':
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	311 return True
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	312 index = txt.find('-', index + 1)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	313 return False
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	314
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	315
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	316 def parse_segments(raw, offset=0):
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	317 # Get the number of lines in the header.
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	318 header_lines = _count_lines(raw[:offset].rstrip())
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	319 current_line = header_lines
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	320
703 dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	321 # Figure out if we need any parsing.
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	322 do_parse = _string_needs_parsing(raw, offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	323 if not do_parse:
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	324 seg = ContentSegment()
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	325 seg.parts = [
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	326 ContentSegmentPart(raw[offset:], None, offset, current_line)]
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	327 return {'content': seg}
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	328
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments. Ludovic Chabant <ludovic@chabant.com> parents: 578 diff changeset	329 # Start parsing segments and parts.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	330 matches = list(segment_pattern.finditer(raw, offset))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	331 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	332 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	333 contents = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	334
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	335 first_offset = matches[0].start()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	336 if first_offset > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	337 # There's some default content segment at the beginning.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	338 seg = ContentSegment()
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	339 seg.parts, current_line = parse_segment_parts(
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	340 raw, offset, first_offset, current_line)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	341 contents['content'] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	342
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	343 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	344 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	345 m2 = matches[i]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	346 seg = ContentSegment()
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	347 seg.parts, current_line = parse_segment_parts(
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	348 raw, m1.end() + 1, m2.start(), current_line,
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	349 m1.group('fmt'))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	350 contents[m1.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	351
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	352 # Handle text past the last match.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	353 lastm = matches[-1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	354 seg = ContentSegment()
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	355 seg.parts, current_line = parse_segment_parts(
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	356 raw, lastm.end() + 1, len(raw), current_line,
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	357 lastm.group('fmt'))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	358 contents[lastm.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	359
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	360 return contents
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	361 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	362 # No segments, just content.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	363 seg = ContentSegment()
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	364 seg.parts, current_line = parse_segment_parts(
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	365 raw, offset, len(raw), current_line)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	366 return {'content': seg}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	367
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	368
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	369 def parse_segment_parts(raw, start, end, line_offset, first_part_fmt=None):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	370 matches = list(part_pattern.finditer(raw, start, end))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	371 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	372 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	373 parts = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	374
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	375 # First part, before the first format change.
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	376 part_text = raw[start:matches[0].start()]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	377 parts.append(
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	378 ContentSegmentPart(part_text, first_part_fmt, start,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	379 line_offset))
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	380 line_offset += _count_lines(part_text)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	381
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	382 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	383 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	384 m2 = matches[i]
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	385 part_text = raw[m1.end() + 1:m2.start()]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	386 parts.append(
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	387 ContentSegmentPart(
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	388 part_text, m1.group('fmt'), m1.end() + 1,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	389 line_offset))
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	390 line_offset += _count_lines(part_text)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	391
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	392 lastm = matches[-1]
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	393 part_text = raw[lastm.end() + 1:end]
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	394 parts.append(ContentSegmentPart(
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	395 part_text, lastm.group('fmt'), lastm.end() + 1,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	396 line_offset))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	397
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	398 return parts, line_offset
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	399 else:
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	400 part_text = raw[start:end]
454 96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	401 parts = [ContentSegmentPart(part_text, first_part_fmt, start,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	402 line_offset)]
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 112 diff changeset	403 return parts, line_offset
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	404

Mercurial > piecrust2

annotate piecrust/page.py @ 840:7f3043f9f26f