Mercurial > piecrust2

annotate piecrust/sources/autoconfig.py @ 1188:a7c43131d871

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bake: Fix file write flushing problem with Python 3.8+ Writing the cache files fails in Python 3.8 because it looks like flushing behaviour has changed. We need to explicitly flush. And even then, in very rare occurrences, it looks like it can still run into racing conditions, so we do a very hacky and ugly "retry" loop when fetching cached data :(
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 15 Jun 2021 22:36:23 -0700
parents 3d71cd95f90a
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
1 import re
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import os.path
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import logging
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
5 from piecrust.configuration import ConfigurationError
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
6 from piecrust.sources.base import ContentItem
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
7 from piecrust.sources.default import DefaultContentSource
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10 logger = logging.getLogger(__name__)
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
13 class AutoConfigContentSourceBase(DefaultContentSource):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
14 """ Base class for content sources that automatically apply configuration
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
15 settings to their generated pages based on those pages' paths.
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
16 """
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 def __init__(self, app, name, config):
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
18 super().__init__(app, name, config)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
19
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
20 config.setdefault('data_type', 'page_iterator')
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
21
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
22 self.capture_mode = config.get('capture_mode', 'path')
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
23 if self.capture_mode not in ['path', 'dirname', 'filename']:
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
24 raise ConfigurationError("Capture mode in source '%s' must be "
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
25 "one of: path, dirname, filename" %
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
26 name)
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
28 def _finalizeContent(self, parent_group, items, groups):
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
29 super()._finalizeContent(parent_group, items, groups)
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
30
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
31 # If `capture_mode` is `dirname`, we don't need to recompute it
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
32 # for each filename, so we do it here.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
33 if self.capture_mode == 'dirname':
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
34 rel_dirpath = '.'
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
35 if parent_group is not None:
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
36 rel_dirpath = os.path.relpath(parent_group.spec,
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
37 self.fs_endpoint_path)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
38 config = self._extractConfigFragment(rel_dirpath)
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
39
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
40 for i in items:
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
41 # Compute the config for the other capture modes.
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
42 if self.capture_mode == 'path':
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
43 rel_path = os.path.relpath(i.spec, self.fs_endpoint_path)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
44 config = self._extractConfigFragment(rel_path)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
45 elif self.capture_mode == 'filename':
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
46 fname = os.path.basename(i.spec)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
47 config = self._extractConfigFragment(fname)
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
48
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
49 # Set the config on the content item's metadata.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
50 i.metadata.setdefault('config', {}).update(config)
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
51
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
52 def _extractConfigFragment(self, rel_path):
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
53 raise NotImplementedError()
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
54
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
55
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
56 class AutoConfigContentSource(AutoConfigContentSourceBase):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
57 """ Content source that extracts configuration settings from the sub-folders
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
58 each page resides in. This is ideal for setting tags or categories
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
59 on pages based on the folders they're in.
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
60 """
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
61 SOURCE_NAME = 'autoconfig'
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
62
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
63 def __init__(self, app, name, config):
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
64 config['capture_mode'] = 'dirname'
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
65 super().__init__(app, name, config)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
66
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
67 self.setting_name = config.get('setting_name', name)
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
68 self.only_single_values = config.get('only_single_values', False)
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
69 self.collapse_single_values = config.get('collapse_single_values',
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
70 False)
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
71
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
72 def _extractConfigFragment(self, rel_path):
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73 if rel_path == '.':
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 values = []
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
75 else:
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
76 values = rel_path.split(os.sep)
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
77
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
78 if self.only_single_values:
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
79 if len(values) > 1:
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
80 raise Exception("Only one folder level is allowed for pages "
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
81 "in source '%s'." % self.name)
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
82 elif len(values) == 1:
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
83 values = values[0]
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
84 else:
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
85 values = None
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
86
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
87 if self.collapse_single_values:
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
88 if len(values) == 1:
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
89 values = values[0]
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
90 elif len(values) == 0:
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
91 values = None
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
92
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
93 return {self.setting_name: values}
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
94
989
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 979
diff changeset
95 def findContentFromRoute(self, route_params):
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
96 # Pages from this source are effectively flattened, so we need to
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
97 # find pages using a brute-force kinda way.
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
98 route_slug = route_params.get('slug', '')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
99 if not route_slug:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
100 route_slug = '_index'
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
101
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
102 for dirpath, dirnames, filenames in os.walk(self.fs_endpoint_path):
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
103 for f in filenames:
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
104 slug, _ = os.path.splitext(f)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
105 if slug == route_slug:
1085
3d71cd95f90a sources: Fix index page slug for ordered page source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1054
diff changeset
106 path = os.path.join(dirpath, f)
1052
f8572df0756e serve: Fix crash with autoconfig source pages when serving.
Ludovic Chabant <ludovic@chabant.com>
parents: 989
diff changeset
107 metadata = self._createItemMetadata(path)
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
108 path = os.path.join(dirpath, f)
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
109 rel_path = os.path.relpath(path, self.fs_endpoint_path)
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
110 config = self._extractConfigFragment(rel_path)
1052
f8572df0756e serve: Fix crash with autoconfig source pages when serving.
Ludovic Chabant <ludovic@chabant.com>
parents: 989
diff changeset
111 metadata.setdefault('config', {}).update(config)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
112 return ContentItem(path, metadata)
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 360
diff changeset
113 return None
148
432cd534ce08 Add `autoconfig` page source.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
114
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
115 def _makeSlug(self, path):
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
116 slug = super()._makeSlug(path)
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
117 return os.path.basename(slug)
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
118
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
119
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
120 class OrderedContentSource(AutoConfigContentSourceBase):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
121 """ A content source that assigns an "order" to its pages based on a
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
122 numerical prefix in their filename. Page iterators will automatically
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
123 sort pages using that order.
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
124 """
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
125 SOURCE_NAME = 'ordered'
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
126
286
a2d283d1033d tests: Fixes for running on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 267
diff changeset
127 re_pattern = re.compile(r'(^|[/\\])(?P<num>\d+)_')
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
128
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
129 def __init__(self, app, name, config):
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
130 config['capture_mode'] = 'path'
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
131 super().__init__(app, name, config)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
132
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
133 self.setting_name = config.get('setting_name', 'order')
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
134 self.default_value = config.get('default_value', 0)
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
135
989
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 979
diff changeset
136 def findContentFromRoute(self, route_params):
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
137 uri_path = route_params.get('slug', '')
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
138 if uri_path == '':
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
139 uri_path = '_index'
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
140
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
141 path = self.fs_endpoint_path
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
142 uri_parts = uri_path.split('/')
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
143 for i, p in enumerate(uri_parts):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
144 if i == len(uri_parts) - 1:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
145 # Last part, this is the filename. We need to check for either
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
146 # the name, or the name with the prefix, but also handle a
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
147 # possible extension.
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
148 p_pat = r'(\d+_)?' + re.escape(p)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
149
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
150 _, ext = os.path.splitext(uri_path)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
151 if ext == '':
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
152 p_pat += r'\.[\w\d]+'
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
153
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
154 found = False
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
155 for name in os.listdir(path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
156 if re.match(p_pat, name):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
157 path = os.path.join(path, name)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
158 found = True
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
159 break
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
160 if not found:
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 360
diff changeset
161 return None
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
162 else:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
163 # Find each sub-directory. It can either be a directory with
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
164 # the name itself, or the name with a number prefix.
1054
75f1b4460491 sources: Fix ordering and filename confusion in the "ordered" source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1052
diff changeset
165 p_pat = r'(\d+_)?' + re.escape(p) + '$'
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
166 found = False
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
167 for name in os.listdir(path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
168 if re.match(p_pat, name):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
169 path = os.path.join(path, name)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
170 found = True
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
171 break
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
172 if not found:
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 360
diff changeset
173 return None
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
174
1052
f8572df0756e serve: Fix crash with autoconfig source pages when serving.
Ludovic Chabant <ludovic@chabant.com>
parents: 989
diff changeset
175 metadata = self._createItemMetadata(path)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
176 rel_path = os.path.relpath(path, self.fs_endpoint_path)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
177 config = self._extractConfigFragment(rel_path)
1052
f8572df0756e serve: Fix crash with autoconfig source pages when serving.
Ludovic Chabant <ludovic@chabant.com>
parents: 989
diff changeset
178 metadata.setdefault('config', {}).update(config)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
179 return ContentItem(path, metadata)
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
180
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
181 def getSorterIterator(self, it):
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
182 accessor = self.getSettingAccessor()
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
183 return OrderTrailSortIterator(it, self.setting_name + '_trail',
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
184 value_accessor=accessor)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
185
1054
75f1b4460491 sources: Fix ordering and filename confusion in the "ordered" source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1052
diff changeset
186 def _finalizeContent(self, parent_group, items, groups):
75f1b4460491 sources: Fix ordering and filename confusion in the "ordered" source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1052
diff changeset
187 super()._finalizeContent(parent_group, items, groups)
75f1b4460491 sources: Fix ordering and filename confusion in the "ordered" source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1052
diff changeset
188
75f1b4460491 sources: Fix ordering and filename confusion in the "ordered" source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1052
diff changeset
189 sn = self.setting_name
75f1b4460491 sources: Fix ordering and filename confusion in the "ordered" source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1052
diff changeset
190 items.sort(key=lambda i: i.metadata['config'][sn])
75f1b4460491 sources: Fix ordering and filename confusion in the "ordered" source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1052
diff changeset
191
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
192 def _extractConfigFragment(self, rel_path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
193 values = []
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
194 for m in self.re_pattern.finditer(rel_path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
195 val = int(m.group('num'))
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
196 values.append(val)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
197
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
198 if len(values) == 0:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
199 values.append(self.default_value)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
200
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
201 return {
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
202 self.setting_name: values[-1],
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
203 self.setting_name + '_trail': values}
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
204
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
205 def _makeSlug(self, path):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 792
diff changeset
206 slug = super()._makeSlug(path)
1085
3d71cd95f90a sources: Fix index page slug for ordered page source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1054
diff changeset
207 slug = self.re_pattern.sub(r'\1', slug)
3d71cd95f90a sources: Fix index page slug for ordered page source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1054
diff changeset
208 if slug == '_index':
3d71cd95f90a sources: Fix index page slug for ordered page source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1054
diff changeset
209 slug = ''
3d71cd95f90a sources: Fix index page slug for ordered page source.
Ludovic Chabant <ludovic@chabant.com>
parents: 1054
diff changeset
210 return slug
239
f43f19975671 sources: Refactor `autoconfig` source, add `OrderedPageSource`.
Ludovic Chabant <ludovic@chabant.com>
parents: 156
diff changeset
211
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
212
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
213 class OrderTrailSortIterator(object):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
214 def __init__(self, it, trail_name, value_accessor):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
215 self.it = it
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
216 self.trail_name = trail_name
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
217 self.value_accessor = value_accessor
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
218
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
219 def __iter__(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
220 return iter(sorted(self.it, key=self._key_getter))
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
221
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
222 def _key_getter(self, item):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
223 values = self.value_accessor(item, self.trail_name)
505
883a5544cd7f bug: Fix a crash with the `ordered` page source when sorting pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
224 key = ''.join(map(lambda v: str(v), values))
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
225 return key
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents: 239
diff changeset
226