Mercurial > piecrust2

annotate piecrust/data/assetor.py @ 854:08e02c2a2a1a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
core: Keep refactoring, this time to prepare for generator sources. - Make a few APIs simpler. - Content pipelines create their own jobs, so that generator sources can keep aborting in `getContents`, but rely on their pipeline to generate pages for baking.
author Ludovic Chabant <ludovic@chabant.com>
date Sun, 04 Jun 2017 23:34:28 -0700
parents f070a4fc033c
children fddaf43424e2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
6
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
1 import os
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os.path
827
570f89414b2c Assetor is now responsible for copying assets, to allow customization
Ben Artin <ben@artins.org>
parents: 809
diff changeset
3 import shutil
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import logging
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 329
diff changeset
5 from piecrust import ASSET_DIR_SUFFIX
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
6 from piecrust.sources.base import REL_ASSETS
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7 from piecrust.uriutil import multi_replace
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
6
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
13 class UnsupportedAssetsError(Exception):
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
14 pass
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
15
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
16
32
43091c9837bf Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
17 def build_base_url(app, uri, rel_assets_path):
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
18 base_url_format = app.config.get('site/base_asset_url_format')
32
43091c9837bf Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
19 rel_assets_path = rel_assets_path.replace('\\', '/')
43091c9837bf Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
20
33
62c7a97c8340 Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
21 # Remove any extension since we'll be copying assets into the 1st
62c7a97c8340 Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
22 # sub-page's folder.
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 pretty = app.config.get('site/pretty_urls')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 if not pretty:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25 uri, _ = os.path.splitext(uri)
32
43091c9837bf Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 25
diff changeset
26
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27 base_url = multi_replace(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
28 base_url_format,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
29 {
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
30 '%path%': rel_assets_path,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
31 '%uri%': uri})
329
422052d2e978 internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents: 290
diff changeset
32
422052d2e978 internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents: 290
diff changeset
33 return base_url.rstrip('/') + '/'
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
35
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
36 class Assetor:
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
37 debug_render_doc = """Helps render URLs to files in the current page's
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
38 asset folder."""
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
39 debug_render = []
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
40 debug_render_dynamic = ['_debugRenderAssetNames']
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
41
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
42 def __init__(self, page):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
43 self._page = page
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44 self._cache = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
45
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
46 def __getattr__(self, name):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47 try:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
48 self._cacheAssets()
25
65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 13
diff changeset
49 return self._cache[name][0]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
50 except KeyError:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
51 raise AttributeError()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
52
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
53 def __getitem__(self, key):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
54 self._cacheAssets()
25
65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 13
diff changeset
55 return self._cache[key][0]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
56
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
57 def __iter__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
58 self._cacheAssets()
25
65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 13
diff changeset
59 return map(lambda i: i[0], self._cache.values())
65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 13
diff changeset
60
809
22c6f6a3d0a0 admin: Add ability to upload page assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 472
diff changeset
61 def allNames(self):
22c6f6a3d0a0 admin: Add ability to upload page assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 472
diff changeset
62 self._cacheAssets()
22c6f6a3d0a0 admin: Add ability to upload page assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 472
diff changeset
63 return list(self._cache.keys())
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 329
diff changeset
64
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
65 def _debugRenderAssetNames(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
66 self._cacheAssets()
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
67 return list(self._cache.keys())
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
68
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
69 def _cacheAssets(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
70 if self._cache is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
71 return
837
ad8f48a31c62 assets: Fix crash when a page doesn't have assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 832
diff changeset
72
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
73 self._cache = self.findAssets() or {}
832
61878590bf96 Refactored Assetor into Assetor and AssetorBase
Ben Artin <ben@artins.org>
parents: 831
diff changeset
74
61878590bf96 Refactored Assetor into Assetor and AssetorBase
Ben Artin <ben@artins.org>
parents: 831
diff changeset
75 def findAssets(self):
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
76 content_item = self._page.content_item
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
77 source = content_item.source
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
78 assets = source.getRelatedContent(content_item, REL_ASSETS)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
79 if assets is None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
80 return {}
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
82 app = source.app
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
83 stack = app.env.render_ctx_stack
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
84 cur_ctx = stack.current_ctx
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
85 if cur_ctx is not None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
86 cur_ctx.current_pass_info.used_assets = True
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
87
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
88 # base_url = build_base_url(app, self._uri, rel_assets_dir)
837
ad8f48a31c62 assets: Fix crash when a page doesn't have assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 832
diff changeset
89
832
61878590bf96 Refactored Assetor into Assetor and AssetorBase
Ben Artin <ben@artins.org>
parents: 831
diff changeset
90 return assets
837
ad8f48a31c62 assets: Fix crash when a page doesn't have assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 832
diff changeset
91
831
18978cf6d1ac Removed pointless page argument from copyAssets
Ben Artin <ben@artins.org>
parents: 827
diff changeset
92 def copyAssets(self, dest_dir):
18978cf6d1ac Removed pointless page argument from copyAssets
Ben Artin <ben@artins.org>
parents: 827
diff changeset
93 page_pathname, _ = os.path.splitext(self._page.path)
827
570f89414b2c Assetor is now responsible for copying assets, to allow customization
Ben Artin <ben@artins.org>
parents: 809
diff changeset
94 in_assets_dir = page_pathname + ASSET_DIR_SUFFIX
570f89414b2c Assetor is now responsible for copying assets, to allow customization
Ben Artin <ben@artins.org>
parents: 809
diff changeset
95 for fn in os.listdir(in_assets_dir):
570f89414b2c Assetor is now responsible for copying assets, to allow customization
Ben Artin <ben@artins.org>
parents: 809
diff changeset
96 full_fn = os.path.join(in_assets_dir, fn)
570f89414b2c Assetor is now responsible for copying assets, to allow customization
Ben Artin <ben@artins.org>
parents: 809
diff changeset
97 if os.path.isfile(full_fn):
570f89414b2c Assetor is now responsible for copying assets, to allow customization
Ben Artin <ben@artins.org>
parents: 809
diff changeset
98 dest_ap = os.path.join(dest_dir, fn)
570f89414b2c Assetor is now responsible for copying assets, to allow customization
Ben Artin <ben@artins.org>
parents: 809
diff changeset
99 logger.debug(" %s -> %s" % (full_fn, dest_ap))
570f89414b2c Assetor is now responsible for copying assets, to allow customization
Ben Artin <ben@artins.org>
parents: 809
diff changeset
100 shutil.copy(full_fn, dest_ap)
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 837
diff changeset
101