piecrust2: piecrust/data/assetor.py annotate

annotate piecrust/data/assetor.py @ 852:4850f8c21b6e

core: Start of the big refactor for PieCrust 3.0. * Everything is a `ContentSource`, including assets directories. * Most content sources are subclasses of the base file-system source. * A source is processed by a "pipeline", and there are 2 built-in pipelines, one for assets and one for pages. The asset pipeline is vaguely functional, but the page pipeline is completely broken right now. * Rewrite the baking process as just running appropriate pipelines on each content item. This should allow for better parallelization.

author	Ludovic Chabant <ludovic@chabant.com>
date	Wed, 17 May 2017 00:11:48 -0700
parents	ad8f48a31c62
children	f070a4fc033c

rev	line source
6 f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	1 import os
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import os.path
827 570f89414b2c Assetor is now responsible for copying assets, to allow customization Ben Artin <ben@artins.org> parents: 809 diff changeset	3 import shutil
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import logging
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 329 diff changeset	5 from piecrust import ASSET_DIR_SUFFIX
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	6 from piecrust.sources.base import REL_ASSETS
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	7 from piecrust.uriutil import multi_replace
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	8
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	9
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12
6 f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	13 class UnsupportedAssetsError(Exception):
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	14 pass
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	15
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	16
32 43091c9837bf Fix problems with asset URLs. Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	17 def build_base_url(app, uri, rel_assets_path):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	18 base_url_format = app.config.get('site/base_asset_url_format')
32 43091c9837bf Fix problems with asset URLs. Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	19 rel_assets_path = rel_assets_path.replace('\\', '/')
43091c9837bf Fix problems with asset URLs. Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	20
33 62c7a97c8340 Get the un-paginated URL of a page early and pass that around. Ludovic Chabant <ludovic@chabant.com> parents: 32 diff changeset	21 # Remove any extension since we'll be copying assets into the 1st
62c7a97c8340 Get the un-paginated URL of a page early and pass that around. Ludovic Chabant <ludovic@chabant.com> parents: 32 diff changeset	22 # sub-page's folder.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 pretty = app.config.get('site/pretty_urls')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 if not pretty:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 uri, _ = os.path.splitext(uri)
32 43091c9837bf Fix problems with asset URLs. Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	26
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 base_url = multi_replace(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	28 base_url_format,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	29 {
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	30 '%path%': rel_assets_path,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	31 '%uri%': uri})
329 422052d2e978 internal: Try handling URLs in a consistent way. Ludovic Chabant <ludovic@chabant.com> parents: 290 diff changeset	32
422052d2e978 internal: Try handling URLs in a consistent way. Ludovic Chabant <ludovic@chabant.com> parents: 290 diff changeset	33 return base_url.rstrip('/') + '/'
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	34
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	35
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	36 class Assetor(object):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	37 debug_render_doc = """Helps render URLs to files in the current page's
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	38 asset folder."""
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	39 debug_render = []
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	40 debug_render_dynamic = ['_debugRenderAssetNames']
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	41
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	42 def __init__(self, page, uri):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	43 self._page = page
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	44 self._uri = uri
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	45 self._cache = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 def __getattr__(self, name):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48 try:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	49 self._cacheAssets()
25 65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets. Ludovic Chabant <ludovic@chabant.com> parents: 13 diff changeset	50 return self._cache[name][0]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51 except KeyError:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	52 raise AttributeError()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	53
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	54 def __getitem__(self, key):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	55 self._cacheAssets()
25 65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets. Ludovic Chabant <ludovic@chabant.com> parents: 13 diff changeset	56 return self._cache[key][0]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	57
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	58 def __iter__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	59 self._cacheAssets()
25 65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets. Ludovic Chabant <ludovic@chabant.com> parents: 13 diff changeset	60 return map(lambda i: i[0], self._cache.values())
65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets. Ludovic Chabant <ludovic@chabant.com> parents: 13 diff changeset	61
809 22c6f6a3d0a0 admin: Add ability to upload page assets. Ludovic Chabant <ludovic@chabant.com> parents: 472 diff changeset	62 def allNames(self):
22c6f6a3d0a0 admin: Add ability to upload page assets. Ludovic Chabant <ludovic@chabant.com> parents: 472 diff changeset	63 self._cacheAssets()
22c6f6a3d0a0 admin: Add ability to upload page assets. Ludovic Chabant <ludovic@chabant.com> parents: 472 diff changeset	64 return list(self._cache.keys())
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 329 diff changeset	65
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66 def _debugRenderAssetNames(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67 self._cacheAssets()
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	68 return list(self._cache.keys())
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	69
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	70 def _cacheAssets(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71 if self._cache is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 return
837 ad8f48a31c62 assets: Fix crash when a page doesn't have assets. Ludovic Chabant <ludovic@chabant.com> parents: 832 diff changeset	73
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	74 self._cache = self.findAssets() or {}
832 61878590bf96 Refactored Assetor into Assetor and AssetorBase Ben Artin <ben@artins.org> parents: 831 diff changeset	75
61878590bf96 Refactored Assetor into Assetor and AssetorBase Ben Artin <ben@artins.org> parents: 831 diff changeset	76 def findAssets(self):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	77 content_item = self._page.content_item
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	78 source = content_item.source
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	79 assets = source.getRelatedContent(content_item, REL_ASSETS)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	80 if assets is None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	81 return {}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	83 app = source.app
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	84 stack = app.env.render_ctx_stack
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	85 cur_ctx = stack.current_ctx
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	86 if cur_ctx is not None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	87 cur_ctx.current_pass_info.used_assets = True
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	89 # base_url = build_base_url(app, self._uri, rel_assets_dir)
837 ad8f48a31c62 assets: Fix crash when a page doesn't have assets. Ludovic Chabant <ludovic@chabant.com> parents: 832 diff changeset	90
832 61878590bf96 Refactored Assetor into Assetor and AssetorBase Ben Artin <ben@artins.org> parents: 831 diff changeset	91 return assets
837 ad8f48a31c62 assets: Fix crash when a page doesn't have assets. Ludovic Chabant <ludovic@chabant.com> parents: 832 diff changeset	92
831 18978cf6d1ac Removed pointless page argument from copyAssets Ben Artin <ben@artins.org> parents: 827 diff changeset	93 def copyAssets(self, dest_dir):
18978cf6d1ac Removed pointless page argument from copyAssets Ben Artin <ben@artins.org> parents: 827 diff changeset	94 page_pathname, _ = os.path.splitext(self._page.path)
827 570f89414b2c Assetor is now responsible for copying assets, to allow customization Ben Artin <ben@artins.org> parents: 809 diff changeset	95 in_assets_dir = page_pathname + ASSET_DIR_SUFFIX
570f89414b2c Assetor is now responsible for copying assets, to allow customization Ben Artin <ben@artins.org> parents: 809 diff changeset	96 for fn in os.listdir(in_assets_dir):
570f89414b2c Assetor is now responsible for copying assets, to allow customization Ben Artin <ben@artins.org> parents: 809 diff changeset	97 full_fn = os.path.join(in_assets_dir, fn)
570f89414b2c Assetor is now responsible for copying assets, to allow customization Ben Artin <ben@artins.org> parents: 809 diff changeset	98 if os.path.isfile(full_fn):
570f89414b2c Assetor is now responsible for copying assets, to allow customization Ben Artin <ben@artins.org> parents: 809 diff changeset	99 dest_ap = os.path.join(dest_dir, fn)
570f89414b2c Assetor is now responsible for copying assets, to allow customization Ben Artin <ben@artins.org> parents: 809 diff changeset	100 logger.debug(" %s -> %s" % (full_fn, dest_ap))
570f89414b2c Assetor is now responsible for copying assets, to allow customization Ben Artin <ben@artins.org> parents: 809 diff changeset	101 shutil.copy(full_fn, dest_ap)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 837 diff changeset	102

Mercurial > piecrust2

annotate piecrust/data/assetor.py @ 852:4850f8c21b6e