Mercurial > piecrust2
annotate piecrust/data/assetor.py @ 415:0e9a94b7fdfa
bake: Improve bake record information.
* Store things in the bake record that require less interaction between the
master process and the workers. For instance, don't store the paginator
object in the render pass info -- instead, just store whether pagination
was used, and whether it had more items.
* Simplify information passing between workers and bake passes by saving the
rendering info to the JSON cache. This means the "render first sub" job
doesn't have to return anything except errors now.
* Add more performance counter info.
author | Ludovic Chabant <ludovic@chabant.com> |
---|---|
date | Sat, 20 Jun 2015 19:23:16 -0700 |
parents | 422052d2e978 |
children | dc78ade3f320 |
rev | line source |
---|---|
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
1 import os |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
2 import os.path |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
3 import logging |
415
0e9a94b7fdfa
bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents:
329
diff
changeset
|
4 from piecrust import ASSET_DIR_SUFFIX |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
5 from piecrust.uriutil import multi_replace |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
6 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
7 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
8 logger = logging.getLogger(__name__) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
9 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
10 |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
11 class UnsupportedAssetsError(Exception): |
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
12 pass |
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
13 |
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
14 |
32
43091c9837bf
Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents:
25
diff
changeset
|
15 def build_base_url(app, uri, rel_assets_path): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
16 base_url_format = app.env.base_asset_url_format |
32
43091c9837bf
Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents:
25
diff
changeset
|
17 rel_assets_path = rel_assets_path.replace('\\', '/') |
43091c9837bf
Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents:
25
diff
changeset
|
18 |
33
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
32
diff
changeset
|
19 # Remove any extension since we'll be copying assets into the 1st |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
32
diff
changeset
|
20 # sub-page's folder. |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
21 pretty = app.config.get('site/pretty_urls') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
22 if not pretty: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
23 uri, _ = os.path.splitext(uri) |
32
43091c9837bf
Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents:
25
diff
changeset
|
24 |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
25 base_url = multi_replace( |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
26 base_url_format, |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
27 { |
32
43091c9837bf
Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents:
25
diff
changeset
|
28 '%path%': rel_assets_path, |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
29 '%uri%': uri}) |
329
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
290
diff
changeset
|
30 |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
290
diff
changeset
|
31 return base_url.rstrip('/') + '/' |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
32 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
33 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
34 class Assetor(object): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
35 debug_render_doc = """Helps render URLs to files in the current page's |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
36 asset folder.""" |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
37 debug_render = [] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
38 debug_render_dynamic = ['_debugRenderAssetNames'] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
39 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
40 def __init__(self, page, uri): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
41 self._page = page |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
42 self._uri = uri |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
43 self._cache = None |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
44 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
45 def __getattr__(self, name): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
46 try: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
47 self._cacheAssets() |
25
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
48 return self._cache[name][0] |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
49 except KeyError: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
50 raise AttributeError() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
51 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
52 def __getitem__(self, key): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
53 self._cacheAssets() |
25
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
54 return self._cache[key][0] |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
55 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
56 def __iter__(self): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
57 self._cacheAssets() |
25
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
58 return map(lambda i: i[0], self._cache.values()) |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
59 |
415
0e9a94b7fdfa
bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents:
329
diff
changeset
|
60 def _getFilenames(self): |
0e9a94b7fdfa
bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents:
329
diff
changeset
|
61 assert self._cache is not None |
0e9a94b7fdfa
bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents:
329
diff
changeset
|
62 return map(lambda i: i[1], self._cache.values()) |
0e9a94b7fdfa
bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents:
329
diff
changeset
|
63 |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
64 def _debugRenderAssetNames(self): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
65 self._cacheAssets() |
5 | 66 return list(self._cache.keys()) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
67 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
68 def _cacheAssets(self): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
69 if self._cache is not None: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
70 return |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
71 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
72 self._cache = {} |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
73 name, ext = os.path.splitext(self._page.path) |
415
0e9a94b7fdfa
bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents:
329
diff
changeset
|
74 assets_dir = name + ASSET_DIR_SUFFIX |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
75 if not os.path.isdir(assets_dir): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
76 return |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
77 |
32
43091c9837bf
Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents:
25
diff
changeset
|
78 rel_assets_dir = os.path.relpath(assets_dir, self._page.app.root_dir) |
43091c9837bf
Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents:
25
diff
changeset
|
79 base_url = build_base_url(self._page.app, self._uri, rel_assets_dir) |
25
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
80 for fn in os.listdir(assets_dir): |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
81 full_fn = os.path.join(assets_dir, fn) |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
82 if not os.path.isfile(full_fn): |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
83 raise Exception("Skipping: %s" % full_fn) |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
84 continue |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
85 |
25
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
86 name, ext = os.path.splitext(fn) |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
87 if name in self._cache: |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
88 raise UnsupportedAssetsError( |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
89 "Multiple asset files are named '%s'." % name) |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
90 self._cache[name] = (base_url + fn, full_fn) |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
91 |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
92 cpi = self._page.app.env.exec_info_stack.current_page_info |
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
93 if cpi is not None: |
415
0e9a94b7fdfa
bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents:
329
diff
changeset
|
94 cpi.render_ctx.used_assets = True |
25
65ae19c4e8a3
Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
13
diff
changeset
|
95 |