annotate piecrust/page.py @ 885:13e8b50a2113

publish: Fix publishers API and add a simple "copy" publisher.
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 20 Jun 2017 21:12:35 -0700
parents d1095774bfcf
children 1bb704434ee2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import re
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import json
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6 import dateutil.parser
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 49
diff changeset
7 import collections
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
8 from werkzeug.utils import cached_property
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
9 from piecrust.configuration import (
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
10 Configuration, ConfigurationError,
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
11 parse_config_header,
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
12 MERGE_PREPEND_LISTS)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18 class PageConfiguration(Configuration):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19 def __init__(self, values=None, validate=True):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20 super(PageConfiguration, self).__init__(values, validate)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
22 def _validateAll(self, values):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 values.setdefault('title', 'Untitled Page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 values.setdefault('content_type', 'html')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25 ppp = values.get('posts_per_page')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26 if ppp is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27 values.setdefault('items_per_page', ppp)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28 pf = values.get('posts_filters')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
29 if pf is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30 values.setdefault('items_filters', pf)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
31 return values
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
32
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
34 FLAG_NONE = 0
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
35 FLAG_RAW_CACHE_VALID = 2**0
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
36
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
37
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
38 class PageNotFoundError(Exception):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
39 pass
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
40
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
41
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
42 class Page:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
43 """ Represents a page that is text content with an optional YAML
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
44 front-matter, and that goes through the page pipeline.
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
45 """
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
46 def __init__(self, source, content_item):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
47 self.source = source
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
48 self.content_item = content_item
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
49 self._config = None
578
683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is.
Ludovic Chabant <ludovic@chabant.com>
parents: 458
diff changeset
50 self._segments = None
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
51 self._flags = FLAG_NONE
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
52 self._datetime = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
53
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
54 @cached_property
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
55 def app(self):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
56 return self.source.app
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
57
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
58 @cached_property
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
59 def route(self):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
60 return self.source.route
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
61
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
62 @property
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
63 def source_metadata(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
64 return self.content_item.metadata
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
65
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
66 @property
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
67 def content_spec(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
68 return self.content_item.spec
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
69
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
70 @cached_property
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
71 def content_mtime(self):
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
72 return self.source.getItemMtime(self.content_item)
49
fce061f8c2ed Fix cache validation issue with rendered segments, limit disk access.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
73
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 @property
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
75 def flags(self):
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
76 return self._flags
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
77
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
78 @property
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
79 def config(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80 self._load()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81 return self._config
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
83 @property
578
683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is.
Ludovic Chabant <ludovic@chabant.com>
parents: 458
diff changeset
84 def segments(self):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
85 self._load()
578
683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is.
Ludovic Chabant <ludovic@chabant.com>
parents: 458
diff changeset
86 return self._segments
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
87
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
88 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
89 def datetime(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
90 if self._datetime is None:
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
91 try:
856
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
92 self._datetime = _compute_datetime(self.source_metadata,
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
93 self.config)
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
94 except Exception as ex:
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 703
diff changeset
95 logger.exception(ex)
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
96 raise Exception(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
97 "Error computing time for page: %s" %
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
98 self.content_spec) from ex
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
99
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
100 if self._datetime is None:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
101 self._datetime = datetime.datetime.fromtimestamp(
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
102 self.content_mtime)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
103
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
104 return self._datetime
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
105
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
106 @datetime.setter
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
107 def datetime(self, value):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
108 self._datetime = value
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
109
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
110 def getUri(self, sub_num=1):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
111 route_params = self.source_metadata['route_params']
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
112 return self.route.getUri(route_params, sub_num=sub_num)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
113
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
114 def getSegment(self, name='content'):
578
683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is.
Ludovic Chabant <ludovic@chabant.com>
parents: 458
diff changeset
115 return self.segments[name]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
116
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
117 def _load(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
118 if self._config is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119 return
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
120
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
121 config, content, was_cache_valid = load_page(
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
122 self.source, self.content_item)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
123
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
124 extra_config = self.source_metadata.get('config')
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
125 if extra_config is not None:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
126 # Merge the source metadata configuration settings with the
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
127 # configuration settings from the page's contents. We only
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
128 # prepend to lists, i.e. we don't overwrite values because we
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
129 # want to keep what the user wrote in the file.
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
130 config.merge(extra_config, mode=MERGE_PREPEND_LISTS)
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
131
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
132 self._config = config
578
683be25cbdb2 internal: Rename `raw_content` to `segments` since it's what it is.
Ludovic Chabant <ludovic@chabant.com>
parents: 458
diff changeset
133 self._segments = content
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
134 if was_cache_valid:
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
135 self._flags |= FLAG_RAW_CACHE_VALID
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
136
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
137
856
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
138 def _compute_datetime(source_metadata, config):
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
139 # Get the date/time from the source.
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
140 dt = source_metadata.get('datetime')
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
141 if dt is not None:
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
142 return dt
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
143
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
144 # Get the date from the source. Potentially get the
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
145 # time from the page config.
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
146 page_date = source_metadata.get('date')
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
147 if page_date is not None:
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
148 dt = datetime.datetime(
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
149 page_date.year, page_date.month, page_date.day)
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
150
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
151 page_time = _parse_config_time(config.get('time'))
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
152 if page_time is not None:
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
153 dt += page_time
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
154
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
155 return dt
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
156
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
157 # Get the date from the page config, and maybe the
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
158 # time too.
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
159 page_date = _parse_config_date(config.get('date'))
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
160 if page_date is not None:
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
161 dt = datetime.datetime(
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
162 page_date.year, page_date.month, page_date.day)
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
163
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
164 page_time = _parse_config_time(config.get('time'))
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
165 if page_time is not None:
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
166 dt += page_time
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
167
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
168 return dt
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
169
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
170 # No idea what the date/time for this page is.
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
171 return datetime.datetime.fromtimestamp(0)
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
172
9bb22bbe093c refactor: Make the blog archives functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
173
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
174 def _parse_config_date(page_date):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
175 if page_date is None:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
176 return None
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
177
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
178 if isinstance(page_date, str):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
179 try:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
180 parsed_d = dateutil.parser.parse(page_date)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
181 except Exception as ex:
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 703
diff changeset
182 logger.exception(ex)
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
183 raise ConfigurationError("Invalid date: %s" % page_date) from ex
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
184 return datetime.date(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
185 year=parsed_d.year,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
186 month=parsed_d.month,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
187 day=parsed_d.day)
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
188
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
189 raise ConfigurationError("Invalid date: %s" % page_date)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
190
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
191
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
192 def _parse_config_time(page_time):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
193 if page_time is None:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
194 return None
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
195
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
196 if isinstance(page_time, datetime.timedelta):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
197 return page_time
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
198
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
199 if isinstance(page_time, str):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
200 try:
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
201 parsed_t = dateutil.parser.parse(page_time)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
202 except Exception as ex:
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 703
diff changeset
203 logger.exception(ex)
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
204 raise ConfigurationError("Invalid time: %s" % page_time) from ex
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
205 return datetime.timedelta(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
206 hours=parsed_t.hour,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
207 minutes=parsed_t.minute,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
208 seconds=parsed_t.second)
142
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
209
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
210 if isinstance(page_time, int):
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
211 # Total seconds... convert to a time struct.
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
212 return datetime.timedelta(seconds=page_time)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
213
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
214 raise ConfigurationError("Invalid time: %s" % page_time)
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
215
0bdd938d6b9f Better date/time handling for pages:
Ludovic Chabant <ludovic@chabant.com>
parents: 128
diff changeset
216
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
217 class PageLoadingError(Exception):
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
218 def __init__(self, spec):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
219 super().__init__("Error loading page: %s" % spec)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
220
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
221
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
222 class ContentSegment(object):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
223 debug_render_func = 'debug_render'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
224
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
225 def __init__(self):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
226 self.parts = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
227
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
228 def debug_render(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
229 return '\n'.join([p.content for p in self.parts])
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
230
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
231
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
232 class ContentSegmentPart(object):
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
233 def __init__(self, content, fmt=None, offset=-1, line=-1):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
234 self.content = content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
235 self.fmt = fmt
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
236 self.offset = offset
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
237 self.line = line
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
238
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
239 def __str__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
240 return '%s [%s]' % (self.content, self.fmt or '<default>')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
241
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
242
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
243 def json_load_segments(data):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
244 segments = {}
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
245 for key, seg_data in data.items():
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
246 seg = ContentSegment()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
247 for p_data in seg_data:
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
248 part = ContentSegmentPart(p_data['c'], p_data['f'], p_data['o'],
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
249 p_data['l'])
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
250 seg.parts.append(part)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
251 segments[key] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
252 return segments
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
253
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
254
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
255 def json_save_segments(segments):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
256 data = {}
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
257 for key, seg in segments.items():
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
258 seg_data = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
259 for part in seg.parts:
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
260 p_data = {'c': part.content, 'f': part.fmt, 'o': part.offset,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
261 'l': part.line}
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
262 seg_data.append(p_data)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
263 data[key] = seg_data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
264 return data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
265
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
266
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
267 def load_page(source, content_item):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
268 try:
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
269 with source.app.env.stats.timerScope('PageLoad'):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
270 return _do_load_page(source, content_item)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
271 except Exception as e:
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
272 logger.exception("Error loading page: %s" % content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
273 raise PageLoadingError(content_item.spec) from e
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
274
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
275
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
276 def _do_load_page(source, content_item):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
277 # Check the cache first.
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
278 app = source.app
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
279 cache = app.cache.getCache('pages')
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
280 cache_token = "%s@%s" % (source.name, content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
281 cache_path = hashlib.md5(cache_token.encode('utf8')).hexdigest() + '.json'
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
282 page_time = source.getItemMtime(content_item)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
283 if cache.isValid(cache_path, page_time):
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
284 cache_data = json.loads(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
285 cache.read(cache_path),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
286 object_pairs_hook=collections.OrderedDict)
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
287 config = PageConfiguration(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
288 values=cache_data['config'],
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
289 validate=False)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
290 content = json_load_segments(cache_data['content'])
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
291 return config, content, True
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
292
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
293 # Nope, load the page from the source file.
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
294 logger.debug("Loading page configuration from: %s" % content_item.spec)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
295 with source.openItem(content_item, 'r', encoding='utf-8') as fp:
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
296 raw = fp.read()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
297 header, offset = parse_config_header(raw)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
298
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
299 config = PageConfiguration(header)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
300 content = parse_segments(raw, offset)
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
301 config.set('segments', list(content.keys()))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
302
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
303 # Save to the cache.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
304 cache_data = {
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
305 'config': config.getAll(),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
306 'content': json_save_segments(content)}
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
307 cache.write(cache_path, json.dumps(cache_data))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
308
876
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters.
Ludovic Chabant <ludovic@chabant.com>
parents: 856
diff changeset
309 app.env.stats.stepCounter('PageLoads')
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters.
Ludovic Chabant <ludovic@chabant.com>
parents: 856
diff changeset
310
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 90
diff changeset
311 return config, content, False
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
312
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
313
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
314 segment_pattern = re.compile(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
315 r"""^\-\-\-\s*(?P<name>\w+)(\:(?P<fmt>\w+))?\s*\-\-\-\s*$""",
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
316 re.M)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
317 part_pattern = re.compile(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
318 r"""^<\-\-\s*(?P<fmt>\w+)\s*\-\->\s*$""",
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
319 re.M)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
320
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
321
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
322 def _count_lines(s):
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
323 return len(s.split('\n'))
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
324
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
325
703
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
326 def _string_needs_parsing(txt, offset):
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
327 txtlen = len(txt)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
328 index = txt.find('-', offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
329 while index >= 0 and index < txtlen - 8:
727
71309814e88f bug: Also look for format changes when determining if a page needs parsing.
Ludovic Chabant <ludovic@chabant.com>
parents: 703
diff changeset
330 # Look for a potential `<--format-->`
71309814e88f bug: Also look for format changes when determining if a page needs parsing.
Ludovic Chabant <ludovic@chabant.com>
parents: 703
diff changeset
331 if index > 0 and txt[index - 1] == '<' and txt[index + 1] == '-':
71309814e88f bug: Also look for format changes when determining if a page needs parsing.
Ludovic Chabant <ludovic@chabant.com>
parents: 703
diff changeset
332 return True
71309814e88f bug: Also look for format changes when determining if a page needs parsing.
Ludovic Chabant <ludovic@chabant.com>
parents: 703
diff changeset
333 # Look for a potential `---segment---`
703
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
334 if txt[index + 1] == '-' and txt[index + 2] == '-':
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
335 return True
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
336 index = txt.find('-', index + 1)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
337 return False
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
338
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
339
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
340 def parse_segments(raw, offset=0):
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
341 # Get the number of lines in the header.
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
342 header_lines = _count_lines(raw[:offset].rstrip())
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
343 current_line = header_lines
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
344
703
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
345 # Figure out if we need any parsing.
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
346 do_parse = _string_needs_parsing(raw, offset)
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
347 if not do_parse:
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
348 seg = ContentSegment()
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
349 seg.parts = [
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
350 ContentSegmentPart(raw[offset:], None, offset, current_line)]
703
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
351 return {'content': seg}
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
352
dab26ab3d533 internal: Don't run regexes for the 99% case of pages with no segments.
Ludovic Chabant <ludovic@chabant.com>
parents: 578
diff changeset
353 # Start parsing segments and parts.
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
354 matches = list(segment_pattern.finditer(raw, offset))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
355 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
356 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
357 contents = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
358
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
359 first_offset = matches[0].start()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
360 if first_offset > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
361 # There's some default content segment at the beginning.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
362 seg = ContentSegment()
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
363 seg.parts, current_line = parse_segment_parts(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
364 raw, offset, first_offset, current_line)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
365 contents['content'] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
366
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
367 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
368 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
369 m2 = matches[i]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
370 seg = ContentSegment()
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
371 seg.parts, current_line = parse_segment_parts(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
372 raw, m1.end() + 1, m2.start(), current_line,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
373 m1.group('fmt'))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
374 contents[m1.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
375
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
376 # Handle text past the last match.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
377 lastm = matches[-1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
378 seg = ContentSegment()
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
379 seg.parts, current_line = parse_segment_parts(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
380 raw, lastm.end() + 1, len(raw), current_line,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
381 lastm.group('fmt'))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
382 contents[lastm.group('name')] = seg
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
383
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
384 return contents
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
385 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
386 # No segments, just content.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
387 seg = ContentSegment()
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
388 seg.parts, current_line = parse_segment_parts(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
389 raw, offset, len(raw), current_line)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
390 return {'content': seg}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
391
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
392
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
393 def parse_segment_parts(raw, start, end, line_offset, first_part_fmt=None):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
394 matches = list(part_pattern.finditer(raw, start, end))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
395 num_matches = len(matches)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
396 if num_matches > 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
397 parts = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
398
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
399 # First part, before the first format change.
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
400 part_text = raw[start:matches[0].start()]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
401 parts.append(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
402 ContentSegmentPart(part_text, first_part_fmt, start,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
403 line_offset))
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
404 line_offset += _count_lines(part_text)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
405
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
406 for i in range(1, num_matches):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
407 m1 = matches[i - 1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
408 m2 = matches[i]
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
409 part_text = raw[m1.end() + 1:m2.start()]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
410 parts.append(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
411 ContentSegmentPart(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
412 part_text, m1.group('fmt'), m1.end() + 1,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
413 line_offset))
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
414 line_offset += _count_lines(part_text)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
415
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
416 lastm = matches[-1]
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
417 part_text = raw[lastm.end() + 1:end]
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
418 parts.append(ContentSegmentPart(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
419 part_text, lastm.group('fmt'), lastm.end() + 1,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 824
diff changeset
420 line_offset))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
421
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
422 return parts, line_offset
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
423 else:
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
424 part_text = raw[start:end]
454
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
425 parts = [ContentSegmentPart(part_text, first_part_fmt, start,
96d363e2da4b templating: Let Jinja2 cache the parsed template for page contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
426 line_offset)]
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 112
diff changeset
427 return parts, line_offset
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
428