annotate piecrust/sources/pageref.py @ 369:4b1019bb2533

serve: Giant refactor to change how we handle data when serving pages. * We need a distinction between source metadata and route metadata. In most cases they're the same, but in cases like taxonomy pages, route metadata contains more things that can't be in source metadata if we want to re-use cached pages. * Create a new `QualifiedPage` type which is a page with a specific route and route metadata. Pass this around in many places. * Instead of passing an URL around, use the route in the `QualifiedPage` to generate URLs. This is better since it removes the guess-work from trying to generate URLs for sub-pages. * Deep-copy app and page configurations before passing them around to things that could modify them, like data builders and such. * Exclude taxonomy pages from iterator data providers. * Properly nest iterator data providers for when the theme and user page sources are merged inside `site.pages`.
author Ludovic Chabant <ludovic@chabant.com>
date Sun, 03 May 2015 18:47:10 -0700
parents dd25bd3ce1f9
children e7b865f8f335
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import re
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os.path
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
3 import copy
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
4 from piecrust.sources.base import PageFactory
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7 page_ref_pattern = re.compile(r'(?P<src>[\w]+)\:(?P<path>.*?)(;|$)')
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10 class PageNotFoundError(Exception):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11 pass
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14 class PageRef(object):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 """ A reference to a page, with support for looking a page in different
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16 realms.
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 """
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
18 _INDEX_NEEDS_LOADING = -2
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
19 _INDEX_NOT_FOUND = -1
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
20
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
21 class _HitInfo(object):
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
22 def __init__(self, source_name, rel_path, path, metadata):
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
23 self.source_name = source_name
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
24 self.rel_path = rel_path
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
25 self.path = path
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
26 self.metadata = metadata
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
27
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28 def __init__(self, app, page_ref):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
29 self.app = app
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30 self._page_ref = page_ref
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
31 self._hits = None
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
32 self._first_valid_hit_index = self._INDEX_NEEDS_LOADING
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33 self._exts = list(app.config.get('site/auto_formats').keys())
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
35 @property
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
36 def exists(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
37 try:
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
38 self._checkHits()
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
39 return True
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
40 except PageNotFoundError:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
41 return False
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
42
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
43 @property
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44 def source_name(self):
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
45 self._checkHits()
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
46 return self._first_valid_hit.source_name
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
48 @property
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
49 def source(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
50 return self.app.getSource(self.source_name)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
51
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
52 @property
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
53 def rel_path(self):
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
54 self._checkHits()
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
55 return self._first_valid_hit.rel_path
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
56
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
57 @property
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
58 def path(self):
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
59 self._checkHits()
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
60 return self._first_valid_hit.path
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
61
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
62 @property
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
63 def metadata(self):
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
64 self._checkHits()
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
65 return self._first_valid_hit.metadata
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
66
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
67 @property
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
68 def possible_rel_paths(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
69 self._load()
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
70 return [h.rel_path for h in self._hits]
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
71
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
72 @property
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73 def possible_paths(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 self._load()
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
75 return [h.path for h in self._hits]
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
76
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
77 def getFactory(self):
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
78 return PageFactory(self.source, self.rel_path,
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
79 copy.deepcopy(self.metadata))
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
80
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
81 @property
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
82 def _first_valid_hit(self):
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
83 return self._hits[self._first_valid_hit_index]
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
84
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
85 def _load(self):
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
86 if self._hits is not None:
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
87 return
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
88
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
89 it = list(page_ref_pattern.finditer(self._page_ref))
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
90 if len(it) == 0:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
91 raise Exception("Invalid page ref: %s" % self._page_ref)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
92
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
93 self._hits = []
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
94 for m in it:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
95 source_name = m.group('src')
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
96 source = self.app.getSource(source_name)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
97 if source is None:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
98 raise Exception("No such source: %s" % source_name)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
99 rel_path = m.group('path')
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
100 if '%ext%' in rel_path:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
101 for e in self._exts:
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
102 cur_rel_path = rel_path.replace('%ext%', e)
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
103 path, metadata = source.resolveRef(cur_rel_path)
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
104 self._hits.append(self._HitInfo(
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
105 source_name, cur_rel_path, path, metadata))
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
106 else:
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
107 path, metadata = source.resolveRef(rel_path)
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
108 self._hits.append(
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
109 self._HitInfo(source_name, rel_path, path, metadata))
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
110
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
111 def _checkHits(self):
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
112 if self._first_valid_hit_index >= 0:
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
113 return
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
114 if self._first_valid_hit_index == self._INDEX_NOT_FOUND:
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
115 raise PageNotFoundError(
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
116 "No valid paths were found for page reference: %s" %
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
117 self._page_ref)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
118
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119 self._load()
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
120 self._first_valid_hit_index = self._INDEX_NOT_FOUND
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
121 for i, hit in enumerate(self._hits):
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
122 if os.path.isfile(hit.path):
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
123 self._first_valid_hit_index = i
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
124 break
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
125