Mercurial > piecrust2

annotate piecrust/page.py @ 550:6f216c1ab6b1

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bake: Add a flag to know which record entries got collapsed from last run. This makes it possible to find entries for things that were actually baked during the current run, as opposed to skipped because they were "clean".
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 04 Aug 2015 21:22:30 -0700
parents 612ed0526afd
children 683be25cbdb2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import re
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import sys
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import json
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import os.path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7 import datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8 import dateutil.parser
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 49
diff changeset
9 import collections
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
10 from werkzeug.utils import cached_property
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
11 from piecrust.configuration import (
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
12 Configuration, ConfigurationError,
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13 parse_config_header)
147
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
14 from piecrust.routing import IRouteMetadataProvider
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20 class PageConfiguration(Configuration):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21 def __init__(self, values=None, validate=True):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
22 super(PageConfiguration, self).__init__(values, validate)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 def _validateAll(self, values):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25 values.setdefault('title', 'Untitled Page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26 values.setdefault('content_type', 'html')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27 ppp = values.get('posts_per_page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28 if ppp is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
29 values.setdefault('items_per_page', ppp)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30 pf = values.get('posts_filters')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
31 if pf is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
32 values.setdefault('items_filters', pf)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33 return values
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
35
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
36 FLAG_NONE = 0
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
37 FLAG_RAW_CACHE_VALID = 2**0
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
38
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
39
147
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
40 class Page(IRouteMetadataProvider):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
41 def __init__(self, source, source_metadata, rel_path):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
42 self.source = source
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
43 self.source_metadata = source_metadata
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44 self.rel_path = rel_path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
45 self._config = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
46 self._raw_content = None
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
47 self._flags = FLAG_NONE
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
48 self._datetime = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
49
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
50 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
51 def app(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
52 return self.source.app
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
53
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
54 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
55 def ref_spec(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
56 return '%s:%s' % (self.source.name, self.rel_path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
57
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
58 @cached_property
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
59 def path(self):
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 336
diff changeset
60 path, _ = self.source.resolveRef(self.rel_path)
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 336
diff changeset
61 return path
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
62
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
63 @cached_property
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
64 def path_mtime(self):
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
65 return os.path.getmtime(self.path)
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
66
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
67 @property
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
68 def flags(self):
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
69 return self._flags
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
70
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
71 @property
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
72 def config(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73 self._load()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 return self._config
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
75
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
76 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
77 def raw_content(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
78 self._load()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
79 return self._raw_content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82 def datetime(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
83 if self._datetime is None:
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
84 try:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
85 if 'datetime' in self.source_metadata:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
86 # Get the date/time from the source.
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
87 self._datetime = self.source_metadata['datetime']
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
88 elif 'date' in self.source_metadata:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
89 # Get the date from the source. Potentially get the
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
90 # time from the page config.
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
91 page_date = self.source_metadata['date']
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
92 page_time = _parse_config_time(self.config.get('time'))
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
93 if page_time is not None:
108
b6ec402d32bb Properly add the config time to a page's datetime.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
94 self._datetime = datetime.datetime(
b6ec402d32bb Properly add the config time to a page's datetime.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
95 page_date.year,
b6ec402d32bb Properly add the config time to a page's datetime.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
96 page_date.month,
b6ec402d32bb Properly add the config time to a page's datetime.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
97 page_date.day) + page_time
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
98 else:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
99 self._datetime = datetime.datetime(
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
100 page_date.year, page_date.month, page_date.day)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
101 elif 'date' in self.config:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
102 # Get the date from the page config, and maybe the
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
103 # time too.
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
104 page_date = _parse_config_date(self.config.get('date'))
90
e293f08d954e PyYAML supports sexagesimal notation, so handle that for page times.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
105 self._datetime = datetime.datetime(
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
106 page_date.year,
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
107 page_date.month,
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
108 page_date.day)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
109 page_time = _parse_config_time(self.config.get('time'))
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
110 if page_time is not None:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
111 self._datetime += page_time
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
112 else:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
113 # No idea what the date/time for this page is.
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
114 self._datetime = datetime.datetime.fromtimestamp(0)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
115 except Exception as ex:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
116 raise Exception(
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
117 "Error computing time for page: %s" %
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
118 self.path) from ex
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119 return self._datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
120
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
121 @datetime.setter
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
122 def datetime(self, value):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
123 self._datetime = value
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
124
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
125 def getSegment(self, name='content'):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
126 return self.raw_content[name]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
127
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
128 def _load(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
129 if self._config is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
130 return
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
131
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
132 config, content, was_cache_valid = load_page(self.app, self.path,
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
133 self.path_mtime)
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
134 if 'config' in self.source_metadata:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
135 config.merge(self.source_metadata['config'])
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
136
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
137 self._config = config
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
138 self._raw_content = content
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
139 if was_cache_valid:
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
140 self._flags |= FLAG_RAW_CACHE_VALID
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
141
147
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
142 def getRouteMetadata(self):
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
143 page_dt = self.datetime
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
144 return {
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
145 'year': page_dt.year,
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
146 'month': page_dt.month,
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
147 'day': page_dt.day}
ab6e7e0e9d44 Pass date information to routing when building URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 142
diff changeset
148
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
149
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
150 def _parse_config_date(page_date):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
151 if page_date is None:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
152 return None
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
153
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
154 if isinstance(page_date, str):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
155 try:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
156 parsed_d = dateutil.parser.parse(page_date)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
157 except Exception as ex:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
158 raise ConfigurationError("Invalid date: %s" % page_date) from ex
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
159 return datetime.date(
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
160 year=parsed_d.year,
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
161 month=parsed_d.month,
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
162 day=parsed_d.day)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
163
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
164 raise ConfigurationError("Invalid date: %s" % page_date)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
165
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
166
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
167 def _parse_config_time(page_time):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
168 if page_time is None:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
169 return None
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
170
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
171 if isinstance(page_time, datetime.timedelta):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
172 return page_time
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
173
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
174 if isinstance(page_time, str):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
175 try:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
176 parsed_t = dateutil.parser.parse(page_time)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
177 except Exception as ex:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
178 raise ConfigurationError("Invalid time: %s" % page_time) from ex
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
179 return datetime.timedelta(
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
180 hours=parsed_t.hour,
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
181 minutes=parsed_t.minute,
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
182 seconds=parsed_t.second)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
183
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
184 if isinstance(page_time, int):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
185 # Total seconds... convert to a time struct.
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
186 return datetime.timedelta(seconds=page_time)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
187
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
188 raise ConfigurationError("Invalid time: %s" % page_time)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
189
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
190
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
191 class PageLoadingError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
192 def __init__(self, path, inner=None):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
193 super(PageLoadingError, self).__init__(
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
194 "Error loading page: %s" % path,
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
195 inner)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
196
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
197
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
198 class ContentSegment(object):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
199 debug_render_func = 'debug_render'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
200
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
201 def __init__(self):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
202 self.parts = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
203
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
204 def debug_render(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
205 return '\n'.join([p.content for p in self.parts])
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
206
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
207
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
208 class ContentSegmentPart(object):
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
209 def __init__(self, content, fmt=None, offset=-1, line=-1):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
210 self.content = content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
211 self.fmt = fmt
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
212 self.offset = offset
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
213 self.line = line
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
214
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
215 def __str__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
216 return '%s [%s]' % (self.content, self.fmt or '<default>')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
217
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
218
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
219 def json_load_segments(data):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
220 segments = {}
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
221 for key, seg_data in data.items():
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
222 seg = ContentSegment()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
223 for p_data in seg_data:
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
224 part = ContentSegmentPart(p_data['c'], p_data['f'], p_data['o'],
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
225 p_data['l'])
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
226 seg.parts.append(part)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
227 segments[key] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
228 return segments
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
229
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
230
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
231 def json_save_segments(segments):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
232 data = {}
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
233 for key, seg in segments.items():
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
234 seg_data = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
235 for part in seg.parts:
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
236 p_data = {'c': part.content, 'f': part.fmt, 'o': part.offset,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
237 'l': part.line}
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
238 seg_data.append(p_data)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
239 data[key] = seg_data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
240 return data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
241
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
242
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
243 def load_page(app, path, path_mtime=None):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
244 try:
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
245 with app.env.timerScope('PageLoad'):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
246 return _do_load_page(app, path, path_mtime)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
247 except Exception as e:
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
248 logger.exception(
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
249 "Error loading page: %s" %
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
250 os.path.relpath(path, app.root_dir))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
251 _, __, traceback = sys.exc_info()
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
252 raise PageLoadingError(path, e).with_traceback(traceback)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
253
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
254
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
255 def _do_load_page(app, path, path_mtime):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
256 # Check the cache first.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
257 cache = app.cache.getCache('pages')
336
aa6b7ff6a193 internal: Use hashes for cache paths.
Ludovic Chabant <ludovic@chabant.com>
parents: 147
diff changeset
258 cache_path = hashlib.md5(path.encode('utf8')).hexdigest() + '.json'
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
259 page_time = path_mtime or os.path.getmtime(path)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
260 if cache.isValid(cache_path, page_time):
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
261 cache_data = json.loads(
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
262 cache.read(cache_path),
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 49
diff changeset
263 object_pairs_hook=collections.OrderedDict)
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
264 config = PageConfiguration(
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
265 values=cache_data['config'],
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
266 validate=False)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
267 content = json_load_segments(cache_data['content'])
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
268 return config, content, True
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
269
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
270 # Nope, load the page from the source file.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
271 logger.debug("Loading page configuration from: %s" % path)
458
612ed0526afd internal: Just use the plain old standard function.
Ludovic Chabant <ludovic@chabant.com>
parents: 454
diff changeset
272 with open(path, 'r', encoding='utf-8') as fp:
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
273 raw = fp.read()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
274 header, offset = parse_config_header(raw)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
275
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
276 if 'format' not in header:
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
277 auto_formats = app.config.get('site/auto_formats')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
278 name, ext = os.path.splitext(path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
279 header['format'] = auto_formats.get(ext, None)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
280
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
281 config = PageConfiguration(header)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
282 content = parse_segments(raw, offset)
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
283 config.set('segments', list(content.keys()))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
284
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
285 # Save to the cache.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
286 cache_data = {
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
287 'config': config.getAll(),
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
288 'content': json_save_segments(content)}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
289 cache.write(cache_path, json.dumps(cache_data))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
290
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
291 return config, content, False
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
292
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
293
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
294 segment_pattern = re.compile(
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
295 r"""^\-\-\-\s*(?P<name>\w+)(\:(?P<fmt>\w+))?\s*\-\-\-\s*$""",
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
296 re.M)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
297 part_pattern = re.compile(
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
298 r"""^<\-\-\s*(?P<fmt>\w+)\s*\-\->\s*$""",
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
299 re.M)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
300
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
301
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
302 def _count_lines(s):
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
303 return len(s.split('\n'))
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
304
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
305
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
306 def parse_segments(raw, offset=0):
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
307 # Get the number of lines in the header.
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
308 header_lines = _count_lines(raw[:offset].rstrip())
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
309 current_line = header_lines
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
310
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
311 # Start parsing.
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
312 matches = list(segment_pattern.finditer(raw, offset))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
313 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
314 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
315 contents = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
316
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
317 first_offset = matches[0].start()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
318 if first_offset > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
319 # There's some default content segment at the beginning.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
320 seg = ContentSegment()
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
321 seg.parts, current_line = parse_segment_parts(
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
322 raw, offset, first_offset, current_line)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
323 contents['content'] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
324
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
325 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
326 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
327 m2 = matches[i]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
328 seg = ContentSegment()
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
329 seg.parts, current_line = parse_segment_parts(
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
330 raw, m1.end() + 1, m2.start(), current_line,
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
331 m1.group('fmt'))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
332 contents[m1.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
333
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
334 # Handle text past the last match.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
335 lastm = matches[-1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
336 seg = ContentSegment()
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
337 seg.parts, current_line = parse_segment_parts(
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
338 raw, lastm.end() + 1, len(raw), current_line,
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
339 lastm.group('fmt'))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
340 contents[lastm.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
341
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
342 return contents
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
343 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
344 # No segments, just content.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
345 seg = ContentSegment()
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
346 seg.parts, current_line = parse_segment_parts(
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
347 raw, offset, len(raw), current_line)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
348 return {'content': seg}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
349
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
350
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
351 def parse_segment_parts(raw, start, end, line_offset, first_part_fmt=None):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
352 matches = list(part_pattern.finditer(raw, start, end))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
353 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
354 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
355 parts = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
356
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
357 # First part, before the first format change.
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
358 part_text = raw[start:matches[0].start()]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
359 parts.append(
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
360 ContentSegmentPart(part_text, first_part_fmt, start,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
361 line_offset))
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
362 line_offset += _count_lines(part_text)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
363
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
364 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
365 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
366 m2 = matches[i]
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
367 part_text = raw[m1.end() + 1:m2.start()]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
368 parts.append(
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
369 ContentSegmentPart(
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
370 part_text, m1.group('fmt'), m1.end() + 1,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
371 line_offset))
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
372 line_offset += _count_lines(part_text)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
373
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
374 lastm = matches[-1]
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
375 part_text = raw[lastm.end() + 1:end]
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
376 parts.append(ContentSegmentPart(
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
377 part_text, lastm.group('fmt'), lastm.end() + 1,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
378 line_offset))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
379
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
380 return parts, line_offset
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
381 else:
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
382 part_text = raw[start:end]
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
383 parts = [ContentSegmentPart(part_text, first_part_fmt, start,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
384 line_offset)]
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
385 return parts, line_offset
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
386