piecrust2: piecrust/baking/baker.py annotate

annotate piecrust/baking/baker.py @ 852:4850f8c21b6e

core: Start of the big refactor for PieCrust 3.0. * Everything is a `ContentSource`, including assets directories. * Most content sources are subclasses of the base file-system source. * A source is processed by a "pipeline", and there are 2 built-in pipelines, one for assets and one for pages. The asset pipeline is vaguely functional, but the page pipeline is completely broken right now. * Rewrite the baking process as just running appropriate pipelines on each content item. This should allow for better parallelization.

author	Ludovic Chabant <ludovic@chabant.com>
date	Wed, 17 May 2017 00:11:48 -0700
parents	9a92e2804562
children	f070a4fc033c

rev	line source
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import time
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import os.path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import logging
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	5 from piecrust.baking.worker import BakeJob
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	6 from piecrust.chefutil import (
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	7 format_timed_scope, format_timed)
687 61d606fbc313 bake: Change `show-timers` to `show-stats`, add stats. Ludovic Chabant <ludovic@chabant.com> parents: 666 diff changeset	8 from piecrust.environment import ExecutionStats
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	9 from piecrust.pipelines.base import PipelineContext
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	10 from piecrust.pipelines.records import (
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	11 MultiRecordHistory, MultiRecord, RecordEntry,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	12 load_records)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	13 from piecrust.sources.base import REALM_USER, REALM_THEME
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	19 def get_bake_records_path(app, out_dir):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	20 records_cache = app.cache.getCache('baker')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	21 records_id = hashlib.md5(out_dir.encode('utf8')).hexdigest()
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	22 records_name = records_id + '.record'
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	23 return records_cache.getCachePath(records_name)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	24
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	25
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 class Baker(object):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	27 def __init__(self, appfactory, app, out_dir,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	28 force=False, allowed_pipelines=None):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	29 self.appfactory = appfactory
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30 self.app = app
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 120 diff changeset	31 self.out_dir = out_dir
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	32 self.force = force
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	33
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	34 self._pipeline_classes = {}
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	35 for pclass in app.plugin_loader.getPipelines():
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	36 self._pipeline_classes[pclass.PIPELINE_NAME] = pclass
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	37
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	38 self.allowed_pipelines = allowed_pipelines
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	39 if allowed_pipelines is None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	40 self.allowed_pipelines = list(self._pipeline_classes.keys())
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	41
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	42 self._records = None
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	43
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	44 def bake(self):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	45 start_time = time.perf_counter()
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46 logger.debug(" Bake Output: %s" % self.out_dir)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 logger.debug(" Root URL: %s" % self.app.config.get('site/root'))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	49 # Get into bake mode.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	50 self.app.config.set('baker/is_baking', True)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	51 self.app.config.set('site/base_asset_url_format', '%uri')
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	52
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	53 # Make sure the output directory exists.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	54 if not os.path.isdir(self.out_dir):
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	55 os.makedirs(self.out_dir, 0o755)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	56
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	57 # Load/create the bake records.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	58 records_path = get_bake_records_path(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	59 self.app, self.out_dir)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	60 if not self.force and os.path.isfile(records_path):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	61 with format_timed_scope(logger, "loaded previous bake records",
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	62 level=logging.DEBUG, colored=False):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	63 previous_records = load_records(records_path)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	64 else:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	65 previous_records = MultiRecord()
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	66 self._records = MultiRecord()
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	68 # Figure out if we need to clean the cache because important things
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	69 # have changed.
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	70 is_cache_valid = self._handleCacheValidity(previous_records,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	71 self._records)
453 8351a77e13f5 bake: Don't pass the previous record entries to the workers. Ludovic Chabant <ludovic@chabant.com> parents: 451 diff changeset	72 if not is_cache_valid:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	73 previous_records = MultiRecord()
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	74
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	75 # Create the bake records history which tracks what's up-to-date
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	76 # or not since last time we baked to the given output folder.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	77 record_history = MultiRecordHistory(previous_records, self._records)
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	78
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	79 # Pre-create all caches.
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	80 for cache_name in ['app', 'baker', 'pages', 'renders']:
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	81 self.app.cache.getCache(cache_name)
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	82
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 # Gather all sources by realm -- we're going to bake each realm
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	84 # separately so we can handle "overriding" (i.e. one realm overrides
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	85 # another realm's pages, like the user realm overriding the theme
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	86 # realm).
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	87 #
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	88 # Also, create and initialize each pipeline for each source.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 sources_by_realm = {}
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	90 ppctx = PipelineContext(self.out_dir, record_history,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	91 force=self.force)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	92 for source in self.app.sources:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	93 pname = source.config['pipeline']
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	94 if pname in self.allowed_pipelines:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	95 srclist = sources_by_realm.setdefault(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	96 source.config['realm'], [])
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	97
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	98 pp = self._pipeline_classes[pname](source)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	99 pp.initialize(ppctx)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	100 srclist.append((source, pp))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	101 else:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	102 logger.debug(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	103 "Skip source '%s' because pipeline '%s' is ignored." %
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	104 (source.name, pname))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	106 # Create the worker processes.
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	107 pool = self._createWorkerPool(records_path)
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	108
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	109 # Bake the realms -- user first, theme second, so that a user item
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	110 # can override a theme item.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	111 realm_list = [REALM_USER, REALM_THEME]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	112 for realm in realm_list:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	113 srclist = sources_by_realm.get(realm)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	114 if srclist is not None:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	115 self._bakeRealm(record_history, pool, realm, srclist)
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	116
687 61d606fbc313 bake: Change `show-timers` to `show-stats`, add stats. Ludovic Chabant <ludovic@chabant.com> parents: 666 diff changeset	117 # All done with the workers. Close the pool and get reports.
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	118 pool_stats = pool.close()
687 61d606fbc313 bake: Change `show-timers` to `show-stats`, add stats. Ludovic Chabant <ludovic@chabant.com> parents: 666 diff changeset	119 total_stats = ExecutionStats()
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	120 for ps in pool_stats:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	121 if ps is not None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	122 total_stats.mergeStats(ps)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	123 record_history.current.stats = total_stats
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	124
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	125 # Shutdown the pipelines.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	126 for realm in realm_list:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	127 srclist = sources_by_realm.get(realm)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	128 if srclist is not None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	129 for _, pp in srclist:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	130 pp.shutdown(ppctx)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	131
333 91b07f9efdc1 bake: Use a rotating bake record. Ludovic Chabant <ludovic@chabant.com> parents: 324 diff changeset	132 # Backup previous records.
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	133 records_dir, records_fn = os.path.split(records_path)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	134 records_id, _ = os.path.splitext(records_fn)
333 91b07f9efdc1 bake: Use a rotating bake record. Ludovic Chabant <ludovic@chabant.com> parents: 324 diff changeset	135 for i in range(8, -1, -1):
91b07f9efdc1 bake: Use a rotating bake record. Ludovic Chabant <ludovic@chabant.com> parents: 324 diff changeset	136 suffix = '' if i == 0 else '.%d' % i
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	137 records_path_i = os.path.join(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	138 records_dir,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	139 '%s%s.record' % (records_id, suffix))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	140 if os.path.exists(records_path_i):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	141 records_path_next = os.path.join(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	142 records_dir,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	143 '%s.%s.record' % (records_id, i + 1))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	144 if os.path.exists(records_path_next):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	145 os.remove(records_path_next)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	146 os.rename(records_path_i, records_path_next)
333 91b07f9efdc1 bake: Use a rotating bake record. Ludovic Chabant <ludovic@chabant.com> parents: 324 diff changeset	147
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	148 # Save the bake record.
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	149 with format_timed_scope(logger, "saved bake records.",
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	150 level=logging.DEBUG, colored=False):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	151 record_history.current.bake_time = time.time()
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	152 record_history.current.out_dir = self.out_dir
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	153 record_history.current.save(records_path)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	154
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	155 # All done.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	156 self.app.config.set('baker/is_baking', False)
151 fd146f54bdaa Forgot this wasn't C++. Ludovic Chabant <ludovic@chabant.com> parents: 150 diff changeset	157 logger.debug(format_timed(start_time, 'done baking'))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	158
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	159 self._records = None
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	160 return record_history.current
217 1f4c3dae1fe8 bake: Better error handling for site baking. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	161
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	162 def _handleCacheValidity(self, previous_records, current_records):
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 371 diff changeset	163 start_time = time.perf_counter()
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	164
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	165 reason = None
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	166 if self.force:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	167 reason = "ordered to"
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	168 elif not self.app.config.get('__cache_valid'):
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	169 # The configuration file was changed, or we're running a new
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	170 # version of the app.
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	171 reason = "not valid anymore"
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	172 elif previous_records.invalidated:
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	173 # We have no valid previous bake record.
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	174 reason = "need bake record regeneration"
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	175 else:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	176 # Check if any template has changed since the last bake. Since
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	177 # there could be some advanced conditional logic going on, we'd
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	178 # better just force a bake from scratch if that's the case.
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	179 max_time = 0
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	180 for d in self.app.templates_dirs:
42 9e058e221108 Fix a crash when checking for timestamps on template files. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	181 for dpath, _, filenames in os.walk(d):
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	182 for fn in filenames:
42 9e058e221108 Fix a crash when checking for timestamps on template files. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	183 full_fn = os.path.join(dpath, fn)
9e058e221108 Fix a crash when checking for timestamps on template files. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	184 max_time = max(max_time, os.path.getmtime(full_fn))
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	185 if max_time >= previous_records.bake_time:
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	186 reason = "templates modified"
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	187
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	188 if reason is not None:
85 3471ffa059b2 Add a `BakeScheduler` to handle build dependencies. Add unit-tests. Ludovic Chabant <ludovic@chabant.com> parents: 50 diff changeset	189 # We have to bake everything from scratch.
707 5f552aedd918 bake: Don't clean the `baker` cache on a force bake. Ludovic Chabant <ludovic@chabant.com> parents: 691 diff changeset	190 self.app.cache.clearCaches(except_names=['app', 'baker'])
45 efd0d3bacc9e Don't recursively clean the cache. Ludovic Chabant <ludovic@chabant.com> parents: 42 diff changeset	191 self.force = True
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	192 current_records.incremental_count = 0
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	193 previous_records = MultiRecord()
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	194 logger.info(format_timed(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	195 start_time, "cleaned cache (reason: %s)" % reason))
453 8351a77e13f5 bake: Don't pass the previous record entries to the workers. Ludovic Chabant <ludovic@chabant.com> parents: 451 diff changeset	196 return False
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	197 else:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	198 current_records.incremental_count += 1
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	199 logger.debug(format_timed(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	200 start_time, "cache is assumed valid", colored=False))
453 8351a77e13f5 bake: Don't pass the previous record entries to the workers. Ludovic Chabant <ludovic@chabant.com> parents: 451 diff changeset	201 return True
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	202
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	203 def _bakeRealm(self, record_history, pool, realm, srclist):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	204 for source, pp in srclist:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	205 logger.debug("Queuing jobs for source '%s' using pipeline '%s'." %
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	206 (source.name, pp.PIPELINE_NAME))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	207 jobs = [BakeJob(source.name, item.spec, item.metadata)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	208 for item in source.getAllContents()]
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	209 pool.queueJobs(jobs)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	210 pool.wait()
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	211
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	212 def _logErrors(self, item_spec, errors):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	213 logger.error("Errors found in %s:" % item_spec)
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	214 for e in errors:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	215 logger.error(" " + e)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	216
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	217 def _createWorkerPool(self, previous_records_path):
447 aefe70229fdd bake: Commonize worker pool code between html and asset baking. Ludovic Chabant <ludovic@chabant.com> parents: 430 diff changeset	218 from piecrust.workerpool import WorkerPool
aefe70229fdd bake: Commonize worker pool code between html and asset baking. Ludovic Chabant <ludovic@chabant.com> parents: 430 diff changeset	219 from piecrust.baking.worker import BakeWorkerContext, BakeWorker
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	220
456 5e902e228053 bake: Correctly use the `num_worers` setting. Ludovic Chabant <ludovic@chabant.com> parents: 453 diff changeset	221 worker_count = self.app.config.get('baker/workers')
462 04abc97dd3b6 bake: Add CLI argument to specify job batch size. Ludovic Chabant <ludovic@chabant.com> parents: 456 diff changeset	222 batch_size = self.app.config.get('baker/batch_size')
456 5e902e228053 bake: Correctly use the `num_worers` setting. Ludovic Chabant <ludovic@chabant.com> parents: 453 diff changeset	223
447 aefe70229fdd bake: Commonize worker pool code between html and asset baking. Ludovic Chabant <ludovic@chabant.com> parents: 430 diff changeset	224 ctx = BakeWorkerContext(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	225 self.appfactory,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	226 self.out_dir,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	227 force=self.force,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	228 previous_records_path=previous_records_path,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	229 allowed_pipelines=self.allowed_pipelines)
447 aefe70229fdd bake: Commonize worker pool code between html and asset baking. Ludovic Chabant <ludovic@chabant.com> parents: 430 diff changeset	230 pool = WorkerPool(
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	231 worker_count=worker_count,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	232 batch_size=batch_size,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	233 worker_class=BakeWorker,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	234 initargs=(ctx,),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	235 callback=self._handleWorkerResult,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	236 error_callback=self._handleWorkerError)
447 aefe70229fdd bake: Commonize worker pool code between html and asset baking. Ludovic Chabant <ludovic@chabant.com> parents: 430 diff changeset	237 return pool
217 1f4c3dae1fe8 bake: Better error handling for site baking. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	238
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	239 def _handleWorkerResult(self, job, res):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	240 record_name = self._getRecordName(job)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	241 record = self._records.getRecord(record_name)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	242 record.entries.append(res.record)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	243 if not res.record.success:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	244 record.success = False
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	245 self._records.success = False
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	246 self._logErrors(job.item_spec, res.record.errors)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	247
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	248 def _handleWorkerError(self, job, exc_data):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	249 e = RecordEntry()
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	250 e.item_spec = job.item_spec
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	251 e.errors.append(str(exc_data))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	252
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	253 record_name = self._getRecordName(job)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	254 record = self._records.getRecord(record_name)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	255 record.entries.append(e)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	256
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	257 record.success = False
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	258 self._records.success = False
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	259
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	260 self._logErrors(job.item_spec, e.errors)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	261 if self.app.debug:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	262 logger.error(exc_data.traceback)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	263
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	264 def _getRecordName(self, job):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	265 sn = job.source_name
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	266 ppn = self.app.getSource(sn).config['pipeline']
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 753 diff changeset	267 return '%s@%s' % (sn, ppn)

Mercurial > piecrust2

annotate piecrust/baking/baker.py @ 852:4850f8c21b6e