piecrust2: piecrust/baking/baker.py annotate

annotate piecrust/baking/baker.py @ 199:aaf08277b96d

bake: Remove `--portable` option until it's (maybe) implemented.

author	Ludovic Chabant <ludovic@chabant.com>
date	Tue, 13 Jan 2015 20:41:44 -0800
parents	232989a6df36
children	e725af1d48fb

rev	line source
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import time
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import os.path
25 65ae19c4e8a3 Copy page assets to bake output, use correct slashes when serving assets. Ludovic Chabant <ludovic@chabant.com> parents: 23 diff changeset	3 import shutil
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	6 import threading
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	7 from piecrust.baking.records import (
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	8 TransitionalBakeRecord, BakeRecordPageEntry)
150 91dcbb5fe1e8 Split baking code in smaller files. Ludovic Chabant <ludovic@chabant.com> parents: 147 diff changeset	9 from piecrust.baking.scheduler import BakeScheduler
91dcbb5fe1e8 Split baking code in smaller files. Ludovic Chabant <ludovic@chabant.com> parents: 147 diff changeset	10 from piecrust.baking.single import (BakingError, PageBaker)
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	11 from piecrust.chefutil import format_timed, log_friendly_exception
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	12 from piecrust.sources.base import (
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	13 PageFactory,
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14 REALM_NAMES, REALM_USER, REALM_THEME)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20 class Baker(object):
199 aaf08277b96d bake: Remove `--portable` option until it's (maybe) implemented. Ludovic Chabant <ludovic@chabant.com> parents: 159 diff changeset	21 def __init__(self, app, out_dir, force=False,
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	22 no_assets=False, num_workers=4):
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 120 diff changeset	23 assert app and out_dir
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 self.app = app
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 120 diff changeset	25 self.out_dir = out_dir
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 self.force = force
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 self.no_assets = no_assets
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	28 self.num_workers = num_workers
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30 # Remember what taxonomy pages we should skip
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	31 # (we'll bake them repeatedly later with each taxonomy term)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	32 self.taxonomy_pages = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	33 logger.debug("Gathering taxonomy page paths:")
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	34 for tax in self.app.taxonomies:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	35 for src in self.app.sources:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	36 path = tax.resolvePagePath(src.name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	37 if path is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	38 self.taxonomy_pages.append(path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	39 logger.debug(" - %s" % path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	40
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	41 def bake(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	42 logger.debug(" Bake Output: %s" % self.out_dir)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	43 logger.debug(" Root URL: %s" % self.app.config.get('site/root'))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	44
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	45 # Get into bake mode.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46 start_time = time.clock()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 self.app.config.set('baker/is_baking', True)
32 43091c9837bf Fix problems with asset URLs. Ludovic Chabant <ludovic@chabant.com> parents: 25 diff changeset	48 self.app.env.base_asset_url_format = '%uri%'
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	49
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	50 # Make sure the output directory exists.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51 if not os.path.isdir(self.out_dir):
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	52 os.makedirs(self.out_dir, 0o755)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	53
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	54 # Load/create the bake record.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	55 record = TransitionalBakeRecord()
45 efd0d3bacc9e Don't recursively clean the cache. Ludovic Chabant <ludovic@chabant.com> parents: 42 diff changeset	56 record_cache = self.app.cache.getCache('baker')
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	57 record_name = (
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	58 hashlib.md5(self.out_dir.encode('utf8')).hexdigest() +
7 343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 6 diff changeset	59 '.record')
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60 if not self.force and record_cache.has(record_name):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 t = time.clock()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	62 record.loadPrevious(record_cache.getCachePath(record_name))
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	63 logger.debug(format_timed(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	64 t, 'loaded previous bake record',
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	65 colored=False))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	67 # Figure out if we need to clean the cache because important things
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	68 # have changed.
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	69 self._handleCacheValidity(record)
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	70
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71 # Gather all sources by realm -- we're going to bake each realm
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 # separately so we can handle "overlaying" (i.e. one realm overrides
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73 # another realm's pages).
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	74 sources_by_realm = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	75 for source in self.app.sources:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	76 srclist = sources_by_realm.setdefault(source.realm, [])
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	77 srclist.append(source)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 # Bake the realms.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 realm_list = [REALM_USER, REALM_THEME]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 for realm in realm_list:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 srclist = sources_by_realm.get(realm)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 if srclist is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84 self._bakeRealm(record, realm, srclist)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	85
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86 # Bake taxonomies.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87 self._bakeTaxonomies(record)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	89 # Delete files from the output.
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	90 self._handleDeletetions(record)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	91
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	92 # Save the bake record.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	93 t = time.clock()
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	94 record.current.bake_time = time.time()
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	95 record.current.out_dir = self.out_dir
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	96 record.collapseRecords()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	97 record.saveCurrent(record_cache.getCachePath(record_name))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	98 logger.debug(format_timed(t, 'saved bake record', colored=False))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	99
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	100 # All done.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	101 self.app.config.set('baker/is_baking', False)
151 fd146f54bdaa Forgot this wasn't C++. Ludovic Chabant <ludovic@chabant.com> parents: 150 diff changeset	102 logger.debug(format_timed(start_time, 'done baking'))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	103
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	104 def _handleCacheValidity(self, record):
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	105 start_time = time.clock()
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	106
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	107 reason = None
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	108 if self.force:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	109 reason = "ordered to"
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	110 elif not self.app.config.get('__cache_valid'):
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	111 # The configuration file was changed, or we're running a new
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	112 # version of the app.
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	113 reason = "not valid anymore"
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	114 elif (not record.previous.bake_time or
e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	115 not record.previous.hasLatestVersion()):
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	116 # We have no valid previous bake record.
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	117 reason = "need bake record regeneration"
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	118 else:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	119 # Check if any template has changed since the last bake. Since
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	120 # there could be some advanced conditional logic going on, we'd
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	121 # better just force a bake from scratch if that's the case.
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	122 max_time = 0
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	123 for d in self.app.templates_dirs:
42 9e058e221108 Fix a crash when checking for timestamps on template files. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	124 for dpath, _, filenames in os.walk(d):
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	125 for fn in filenames:
42 9e058e221108 Fix a crash when checking for timestamps on template files. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	126 full_fn = os.path.join(dpath, fn)
9e058e221108 Fix a crash when checking for timestamps on template files. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	127 max_time = max(max_time, os.path.getmtime(full_fn))
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	128 if max_time >= record.previous.bake_time:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	129 reason = "templates modified"
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	130
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	131 if reason is not None:
85 3471ffa059b2 Add a `BakeScheduler` to handle build dependencies. Add unit-tests. Ludovic Chabant <ludovic@chabant.com> parents: 50 diff changeset	132 # We have to bake everything from scratch.
105 7d2fdf43d7ca Property clean all caches when force baking, except the `app` cache. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	133 for cache_name in self.app.cache.getCacheNames(
7d2fdf43d7ca Property clean all caches when force baking, except the `app` cache. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	134 except_names=['app']):
7d2fdf43d7ca Property clean all caches when force baking, except the `app` cache. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	135 cache_dir = self.app.cache.getCacheDir(cache_name)
7d2fdf43d7ca Property clean all caches when force baking, except the `app` cache. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	136 if os.path.isdir(cache_dir):
7d2fdf43d7ca Property clean all caches when force baking, except the `app` cache. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	137 logger.debug("Cleaning baker cache: %s" % cache_dir)
7d2fdf43d7ca Property clean all caches when force baking, except the `app` cache. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	138 shutil.rmtree(cache_dir)
45 efd0d3bacc9e Don't recursively clean the cache. Ludovic Chabant <ludovic@chabant.com> parents: 42 diff changeset	139 self.force = True
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	140 record.incremental_count = 0
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 91 diff changeset	141 record.clearPrevious()
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	142 logger.info(format_timed(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	143 start_time,
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	144 "cleaned cache (reason: %s)" % reason))
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	145 else:
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	146 record.incremental_count += 1
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	147 logger.debug(format_timed(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	148 start_time, "cache is assumed valid",
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	149 colored=False))
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	150
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	151 def _bakeRealm(self, record, realm, srclist):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	152 # Gather all page factories from the sources and queue them
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	153 # for the workers to pick up. Just skip taxonomy pages for now.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	154 logger.debug("Baking realm %s" % REALM_NAMES[realm])
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	155 pool, queue, abort = self._createWorkerPool(record, self.num_workers)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	156
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	157 for source in srclist:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	158 factories = source.getPageFactories()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159 for fac in factories:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	160 if fac.path in self.taxonomy_pages:
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	161 logger.debug(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	162 "Skipping taxonomy page: %s:%s" %
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	163 (source.name, fac.ref_spec))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	164 continue
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	165
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	166 entry = BakeRecordPageEntry(fac)
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	167 record.addEntry(entry)
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	168
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	169 route = self.app.getRoute(source.name, fac.metadata)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	170 if route is None:
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	171 entry.errors.append(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	172 "Can't get route for page: %s" % fac.ref_spec)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	173 logger.error(entry.errors[-1])
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	174 continue
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	175
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	176 queue.addJob(BakeWorkerJob(fac, route, entry))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	177
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	178 self._waitOnWorkerPool(pool, abort)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	179
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	180 def _bakeTaxonomies(self, record):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	181 logger.debug("Baking taxonomies")
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	182
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	183 # Let's see all the taxonomy terms for which we must bake a
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	184 # listing page... first, pre-populate our big map of used terms.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	185 buckets = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	186 tax_names = [t.name for t in self.app.taxonomies]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	187 source_names = [s.name for s in self.app.sources]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	188 for sn in source_names:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	189 source_taxonomies = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	190 buckets[sn] = source_taxonomies
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	191 for tn in tax_names:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	192 source_taxonomies[tn] = set()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	193
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	194 # Now see which ones are 'dirty' based on our bake record.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	195 logger.debug("Gathering dirty taxonomy terms")
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	196 for prev_entry, cur_entry in record.transitions.values():
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	197 for tax in self.app.taxonomies:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	198 changed_terms = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	199 # Re-bake all taxonomy pages that include new or changed
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	200 # pages.
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	201 if (not prev_entry and cur_entry and
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	202 cur_entry.was_baked_successfully):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	203 changed_terms = cur_entry.config.get(tax.name)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	204 elif (prev_entry and cur_entry and
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	205 cur_entry.was_baked_successfully):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	206 changed_terms = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	207 prev_terms = prev_entry.config.get(tax.name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	208 cur_terms = cur_entry.config.get(tax.name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	209 if tax.is_multiple:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	210 if prev_terms is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	211 changed_terms += prev_terms
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	212 if cur_terms is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	213 changed_terms += cur_terms
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	214 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	215 if prev_terms is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	216 changed_terms.append(prev_terms)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	217 if cur_terms is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	218 changed_terms.append(cur_terms)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	219 if changed_terms is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	220 if not isinstance(changed_terms, list):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	221 changed_terms = [changed_terms]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	222 buckets[cur_entry.source_name][tax.name] \|= (
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	223 set(changed_terms))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	224
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	225 # Re-bake the combination pages for terms that are 'dirty'.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	226 known_combinations = set()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	227 logger.debug("Gathering dirty term combinations")
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	228 for prev_entry, cur_entry in record.transitions.values():
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	229 if cur_entry:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	230 known_combinations \|= cur_entry.used_taxonomy_terms
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	231 elif prev_entry:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	232 known_combinations \|= prev_entry.used_taxonomy_terms
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	233 for sn, tn, terms in known_combinations:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	234 changed_terms = buckets[sn][tn]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	235 if not changed_terms.isdisjoint(set(terms)):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	236 changed_terms.add(terms)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	237
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	238 # Start baking those terms.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	239 pool, queue, abort = self._createWorkerPool(record, self.num_workers)
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	240 for source_name, source_taxonomies in buckets.items():
474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	241 for tax_name, terms in source_taxonomies.items():
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	242 if len(terms) == 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	243 continue
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	244
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	245 logger.debug(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	246 "Baking '%s' for source '%s': %s" %
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	247 (tax_name, source_name, terms))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	248 tax = self.app.getTaxonomy(tax_name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	249 route = self.app.getTaxonomyRoute(tax_name, source_name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	250 tax_page_ref = tax.getPageRef(source_name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	251 if not tax_page_ref.exists:
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	252 logger.debug(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	253 "No taxonomy page found at '%s', skipping." %
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	254 tax.page_ref)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	255 continue
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	256
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	257 tax_page_source = tax_page_ref.source
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	258 tax_page_rel_path = tax_page_ref.rel_path
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	259 logger.debug(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	260 "Using taxonomy page: %s:%s" %
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	261 (tax_page_source.name, tax_page_rel_path))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	262
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	263 for term in terms:
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	264 fac = PageFactory(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	265 tax_page_source, tax_page_rel_path,
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	266 {tax.term_name: term})
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	267 logger.debug(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	268 "Queuing: %s [%s, %s]" %
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	269 (fac.ref_spec, tax_name, term))
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	270 entry = BakeRecordPageEntry(fac, tax_name, term)
e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	271 record.addEntry(entry)
85 3471ffa059b2 Add a `BakeScheduler` to handle build dependencies. Add unit-tests. Ludovic Chabant <ludovic@chabant.com> parents: 50 diff changeset	272 queue.addJob(
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	273 BakeWorkerJob(fac, route, entry, tax_name, term))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	274
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	275 self._waitOnWorkerPool(pool, abort)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	276
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	277 def _handleDeletetions(self, record):
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	278 for path, reason in record.getDeletions():
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	279 logger.debug("Removing '%s': %s" % (path, reason))
146 0609739169bd Don't fail if trying to clean up a file that has already been deleted. Ludovic Chabant <ludovic@chabant.com> parents: 133 diff changeset	280 try:
0609739169bd Don't fail if trying to clean up a file that has already been deleted. Ludovic Chabant <ludovic@chabant.com> parents: 133 diff changeset	281 os.remove(path)
0609739169bd Don't fail if trying to clean up a file that has already been deleted. Ludovic Chabant <ludovic@chabant.com> parents: 133 diff changeset	282 logger.info('[delete] %s' % path)
0609739169bd Don't fail if trying to clean up a file that has already been deleted. Ludovic Chabant <ludovic@chabant.com> parents: 133 diff changeset	283 except OSError:
0609739169bd Don't fail if trying to clean up a file that has already been deleted. Ludovic Chabant <ludovic@chabant.com> parents: 133 diff changeset	284 # Not a big deal if that file had already been removed
0609739169bd Don't fail if trying to clean up a file that has already been deleted. Ludovic Chabant <ludovic@chabant.com> parents: 133 diff changeset	285 # by the user.
0609739169bd Don't fail if trying to clean up a file that has already been deleted. Ludovic Chabant <ludovic@chabant.com> parents: 133 diff changeset	286 pass
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	287
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	288 def _createWorkerPool(self, record, pool_size=4):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	289 pool = []
85 3471ffa059b2 Add a `BakeScheduler` to handle build dependencies. Add unit-tests. Ludovic Chabant <ludovic@chabant.com> parents: 50 diff changeset	290 queue = BakeScheduler(record)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	291 abort = threading.Event()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	292 for i in range(pool_size):
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	293 ctx = BakeWorkerContext(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	294 self.app, self.out_dir, self.force,
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	295 record, queue, abort)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	296 worker = BakeWorker(i, ctx)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	297 pool.append(worker)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	298 return pool, queue, abort
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	299
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	300 def _waitOnWorkerPool(self, pool, abort):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	301 for w in pool:
85 3471ffa059b2 Add a `BakeScheduler` to handle build dependencies. Add unit-tests. Ludovic Chabant <ludovic@chabant.com> parents: 50 diff changeset	302 w.start()
159 232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	303
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	304 try:
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	305 for w in pool:
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	306 w.join()
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	307 except KeyboardInterrupt:
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	308 logger.warning("Bake aborted by user... "
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	309 "waiting for workers to stop.")
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	310 abort.set()
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	311 for w in pool:
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	312 w.join()
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	313 raise
232989a6df36 Add support for KeyboardInterrupt in bake process. Ludovic Chabant <ludovic@chabant.com> parents: 158 diff changeset	314
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	315 if abort.is_set():
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	316 excs = [w.abort_exception for w in pool
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	317 if w.abort_exception is not None]
85 3471ffa059b2 Add a `BakeScheduler` to handle build dependencies. Add unit-tests. Ludovic Chabant <ludovic@chabant.com> parents: 50 diff changeset	318 logger.error("Baking was aborted due to %s error(s):" % len(excs))
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	319 if self.app.debug:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	320 for e in excs:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	321 logger.exception(e)
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	322 else:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	323 for e in excs:
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	324 log_friendly_exception(logger, e)
105 7d2fdf43d7ca Property clean all caches when force baking, except the `app` cache. Ludovic Chabant <ludovic@chabant.com> parents: 96 diff changeset	325 raise BakingError("Baking was aborted due to errors.")
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	326
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	327
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	328 class BakeWorkerContext(object):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	329 def __init__(self, app, out_dir, force, record, work_queue,
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	330 abort_event):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	331 self.app = app
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	332 self.out_dir = out_dir
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	333 self.force = force
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	334 self.record = record
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	335 self.work_queue = work_queue
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	336 self.abort_event = abort_event
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	337
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	338
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	339 class BakeWorkerJob(object):
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	340 def __init__(self, factory, route, record_entry,
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	341 taxonomy_name=None, taxonomy_term=None):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	342 self.factory = factory
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	343 self.route = route
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	344 self.record_entry = record_entry
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	345 self.taxonomy_name = taxonomy_name
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	346 self.taxonomy_term = taxonomy_term
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	347
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	348 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	349 def source(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	350 return self.factory.source
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	351
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	352
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	353 class BakeWorker(threading.Thread):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	354 def __init__(self, wid, ctx):
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	355 super(BakeWorker, self).__init__(name=('worker%d' % wid))
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	356 self.wid = wid
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	357 self.ctx = ctx
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	358 self.abort_exception = None
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	359 self._page_baker = PageBaker(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	360 ctx.app, ctx.out_dir, ctx.force,
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	361 ctx.record)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	362
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	363 def run(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	364 while(not self.ctx.abort_event.is_set()):
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 91 diff changeset	365 try:
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 91 diff changeset	366 job = self.ctx.work_queue.getNextJob(wait_timeout=1)
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 91 diff changeset	367 if job is None:
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	368 logger.debug(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	369 "[%d] No more work... shutting down." %
96 0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 91 diff changeset	370 self.wid)
0445a2232de7 Improvements and fixes to incremental baking. Ludovic Chabant <ludovic@chabant.com> parents: 91 diff changeset	371 break
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	372 self._unsafeRun(job)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	373 logger.debug("[%d] Done with page." % self.wid)
85 3471ffa059b2 Add a `BakeScheduler` to handle build dependencies. Add unit-tests. Ludovic Chabant <ludovic@chabant.com> parents: 50 diff changeset	374 self.ctx.work_queue.onJobFinished(job)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	375 except Exception as ex:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	376 self.ctx.abort_event.set()
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	377 self.abort_exception = ex
2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 36 diff changeset	378 logger.debug("[%d] Critical error, aborting." % self.wid)
50 2fec3ee1298f Properly override pages between realms. Ludovic Chabant <ludovic@chabant.com> parents: 49 diff changeset	379 if self.ctx.app.debug:
2fec3ee1298f Properly override pages between realms. Ludovic Chabant <ludovic@chabant.com> parents: 49 diff changeset	380 logger.exception(ex)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	381 break
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	382
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	383 def _unsafeRun(self, job):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	384 start_time = time.clock()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	385
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	386 entry = job.record_entry
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	387 try:
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	388 self._page_baker.bake(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	389 job.factory, job.route, entry,
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	390 taxonomy_name=job.taxonomy_name,
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	391 taxonomy_term=job.taxonomy_term)
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	392 except BakingError as ex:
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	393 logger.debug("Got baking error. Adding it to the record.")
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	394 while ex:
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	395 entry.errors.append(str(ex))
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	396 ex = ex.__cause__
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	397
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	398 if entry.was_baked_successfully:
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	399 uri = entry.out_uris[0]
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	400 friendly_uri = uri if uri != '' else '[main page]'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	401 friendly_count = ''
91 e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	402 if entry.num_subs > 1:
e88e330eb8dc Improvements to incremental baking and cache invalidating. Ludovic Chabant <ludovic@chabant.com> parents: 85 diff changeset	403 friendly_count = ' (%d pages)' % entry.num_subs
158 1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	404 logger.info(format_timed(
1187739e5a19 Fix some indentation and line lengths. Ludovic Chabant <ludovic@chabant.com> parents: 151 diff changeset	405 start_time, '[%d] %s%s' %
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	406 (self.wid, friendly_uri, friendly_count)))
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	407 elif entry.errors:
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	408 for e in entry.errors:
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 105 diff changeset	409 logger.error(e)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	410

Mercurial > piecrust2

annotate piecrust/baking/baker.py @ 199:aaf08277b96d