piecrust2: piecrust/commands/builtin/baking.py annotate

annotate piecrust/commands/builtin/baking.py @ 549:7453baeb0839

bake: Set the flags, don't combine. We don't want to combine old flags with new ones, especially if something went different between the last bake and the current one.

author	Ludovic Chabant <ludovic@chabant.com>
date	Tue, 04 Aug 2015 21:21:08 -0700
parents	456db44dcc53
children	c7ac7dc2fe73

rev	line source
121 300eb1c2cb14 Fix stupid bug. Ludovic Chabant <ludovic@chabant.com> parents: 120 diff changeset	1 import time
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import os.path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import hashlib
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	5 import fnmatch
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	6 import datetime
421 4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	7 from colorama import Fore
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	8 from piecrust.baking.baker import Baker
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	9 from piecrust.baking.records import (
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	10 BakeRecord, BakeRecordEntry, SubPageBakeInfo)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	11 from piecrust.chefutil import format_timed
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12 from piecrust.commands.base import ChefCommand
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	13 from piecrust.processing.pipeline import ProcessorPipeline
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	14 from piecrust.processing.records import (
221 f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	15 ProcessorPipelineRecord,
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	16 FLAG_PREPARED, FLAG_PROCESSED,
221 f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	17 FLAG_BYPASSED_STRUCTURED_PROCESSING)
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	18 from piecrust.rendering import (
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	19 PASS_FORMATTING, PASS_RENDERING)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 class BakeCommand(ChefCommand):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 super(BakeCommand, self).__init__()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28 self.name = 'bake'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 self.description = "Bakes your website into static HTML files."
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	31 def setupParser(self, parser, app):
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	32 parser.add_argument(
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	33 '-o', '--output',
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	34 help="The directory to put all the baked HTML files into "
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	35 "(defaults to `_counter`)")
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	36 parser.add_argument(
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	37 '-f', '--force',
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	38 help="Force re-baking the entire website.",
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	39 action='store_true')
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	40 parser.add_argument(
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	41 '-w', '--workers',
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	42 help="The number of worker processes to spawn.",
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	43 type=int, default=-1)
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	44 parser.add_argument(
462 04abc97dd3b6 bake: Add CLI argument to specify job batch size. Ludovic Chabant <ludovic@chabant.com> parents: 421 diff changeset	45 '--batch-size',
04abc97dd3b6 bake: Add CLI argument to specify job batch size. Ludovic Chabant <ludovic@chabant.com> parents: 421 diff changeset	46 help="The number of jobs per batch.",
04abc97dd3b6 bake: Add CLI argument to specify job batch size. Ludovic Chabant <ludovic@chabant.com> parents: 421 diff changeset	47 type=int, default=-1)
04abc97dd3b6 bake: Add CLI argument to specify job batch size. Ludovic Chabant <ludovic@chabant.com> parents: 421 diff changeset	48 parser.add_argument(
220 84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	49 '--assets-only',
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	50 help="Only bake the assets (don't bake the web pages).",
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	51 action='store_true')
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	52 parser.add_argument(
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	53 '--html-only',
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	54 help="Only bake the pages (don't run the asset pipeline).",
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	55 action='store_true')
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	56 parser.add_argument(
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	57 '--show-timers',
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	58 help="Show detailed timing information.",
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	59 action='store_true')
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 def run(self, ctx):
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	62 out_dir = (ctx.args.output or
bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	63 os.path.join(ctx.app.root_dir, '_counter'))
bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	64
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	65 success = True
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	66 ctx.timers = {}
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	67 start_time = time.perf_counter()
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	68 try:
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	69 # Bake the site sources.
220 84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	70 if not ctx.args.assets_only:
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	71 success = success & self._bakeSources(ctx, out_dir)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	72
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	73 # Bake the assets.
220 84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	74 if not ctx.args.html_only:
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	75 success = success & self._bakeAssets(ctx, out_dir)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	76
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	77 # Show merged timers.
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	78 if ctx.args.show_timers:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	79 logger.info("-------------------")
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	80 logger.info("Timing information:")
421 4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	81 _show_timers(ctx.timers)
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	82
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	83 # All done.
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	84 logger.info('-------------------------')
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	85 logger.info(format_timed(start_time, 'done baking'))
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	86 return 0 if success else 1
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	87 except Exception as ex:
43 f14889d6b067 Proper debug logging. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	88 if ctx.app.debug:
f14889d6b067 Proper debug logging. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	89 logger.exception(ex)
f14889d6b067 Proper debug logging. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	90 else:
f14889d6b067 Proper debug logging. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	91 logger.error(str(ex))
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	92 return 1
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	93
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	94 def _bakeSources(self, ctx, out_dir):
411 e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	95 if ctx.args.workers > 0:
e7b865f8f335 bake: Enable multiprocess baking. Ludovic Chabant <ludovic@chabant.com> parents: 359 diff changeset	96 ctx.app.config.set('baker/workers', ctx.args.workers)
462 04abc97dd3b6 bake: Add CLI argument to specify job batch size. Ludovic Chabant <ludovic@chabant.com> parents: 421 diff changeset	97 if ctx.args.batch_size > 0:
04abc97dd3b6 bake: Add CLI argument to specify job batch size. Ludovic Chabant <ludovic@chabant.com> parents: 421 diff changeset	98 ctx.app.config.set('baker/batch_size', ctx.args.batch_size)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	99 baker = Baker(
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	100 ctx.app, out_dir,
466 456db44dcc53 bake: Pass the config variants and values from the CLI to the baker. Ludovic Chabant <ludovic@chabant.com> parents: 462 diff changeset	101 force=ctx.args.force,
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker. Ludovic Chabant <ludovic@chabant.com> parents: 462 diff changeset	102 applied_config_variant=ctx.config_variant,
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker. Ludovic Chabant <ludovic@chabant.com> parents: 462 diff changeset	103 applied_config_values=ctx.config_values)
217 1f4c3dae1fe8 bake: Better error handling for site baking. Ludovic Chabant <ludovic@chabant.com> parents: 215 diff changeset	104 record = baker.bake()
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	105 _merge_timers(record.timers, ctx.timers)
217 1f4c3dae1fe8 bake: Better error handling for site baking. Ludovic Chabant <ludovic@chabant.com> parents: 215 diff changeset	106 return record.success
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	107
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	108 def _bakeAssets(self, ctx, out_dir):
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	109 proc = ProcessorPipeline(
205 e725af1d48fb bake: Changes in how assets directories are configured. Ludovic Chabant <ludovic@chabant.com> parents: 201 diff changeset	110 ctx.app, out_dir,
e725af1d48fb bake: Changes in how assets directories are configured. Ludovic Chabant <ludovic@chabant.com> parents: 201 diff changeset	111 force=ctx.args.force)
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	112 record = proc.run()
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	113 _merge_timers(record.timers, ctx.timers)
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	114 return record.success
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	115
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	116
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	117 def _merge_timers(source, target):
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	118 if source is None:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	119 return
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	120
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	121 for name, val in source.items():
421 4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	122 if isinstance(val, float):
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	123 if name not in target:
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	124 target[name] = 0
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	125 target[name] += val
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	126 elif isinstance(val, dict):
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	127 if name not in target:
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	128 target[name] = {}
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	129 _merge_timers(val, target[name])
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	130
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	131
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	132 def _show_timers(timers, indent=''):
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	133 sub_timer_names = []
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	134 for name in sorted(timers.keys()):
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	135 if isinstance(timers[name], float):
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	136 val_str = '%8.1f s' % timers[name]
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	137 logger.info(
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	138 "%s[%s%s%s] %s" %
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	139 (indent, Fore.GREEN, val_str, Fore.RESET, name))
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	140 else:
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	141 sub_timer_names.append(name)
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	142
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	143 for name in sub_timer_names:
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	144 logger.info('%s:' % name)
4a43d7015b75 bake: Improve performance timers reports. Ludovic Chabant <ludovic@chabant.com> parents: 415 diff changeset	145 _show_timers(timers[name], indent + ' ')
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	146
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	147
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	148 class ShowRecordCommand(ChefCommand):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	149 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	150 super(ShowRecordCommand, self).__init__()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	151 self.name = 'showrecord'
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	152 self.description = ("Shows the bake record for a given output "
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	153 "directory.")
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	154
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	155 def setupParser(self, parser, app):
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	156 parser.add_argument(
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	157 '-o', '--output',
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	158 help="The output directory for which to show the bake record "
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159 "(defaults to `_counter`)",
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	160 nargs='?')
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	161 parser.add_argument(
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	162 '-p', '--path',
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	163 help="A pattern that will be used to filter the relative path "
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	164 "of entries to show.")
331 f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	165 parser.add_argument(
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	166 '-t', '--out',
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	167 help="A pattern that will be used to filter the output path "
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	168 "of entries to show.")
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	169 parser.add_argument(
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	170 '--last',
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	171 type=int,
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	172 default=0,
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	173 help="Show the last Nth bake record.")
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	174 parser.add_argument(
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	175 '--html-only',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	176 action='store_true',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	177 help="Only show records for pages (not from the asset "
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	178 "pipeline).")
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	179 parser.add_argument(
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	180 '--assets-only',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	181 action='store_true',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	182 help="Only show records for assets (not from pages).")
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	183
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	184 def run(self, ctx):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	185 out_dir = ctx.args.output or os.path.join(ctx.app.root_dir, '_counter')
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	186 record_id = hashlib.md5(out_dir.encode('utf8')).hexdigest()
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	187 suffix = '' if ctx.args.last == 0 else '.%d' % ctx.args.last
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	188 record_name = '%s%s.record' % (record_id, suffix)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	189
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	190 pattern = None
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	191 if ctx.args.path:
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	192 pattern = '%s' % ctx.args.path.strip('*')
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	193
331 f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	194 out_pattern = None
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	195 if ctx.args.out:
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	196 out_pattern = '%s' % ctx.args.out.strip('*')
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	197
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	198 if not ctx.args.assets_only:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	199 self._showBakeRecord(ctx, record_name, pattern, out_pattern)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	200 if not ctx.args.html_only:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	201 self._showProcessingRecord(ctx, record_name, pattern, out_pattern)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	202
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	203 def _showBakeRecord(self, ctx, record_name, pattern, out_pattern):
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	204 # Show the bake record.
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	205 record_cache = ctx.app.cache.getCache('baker')
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	206 if not record_cache.has(record_name):
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	207 raise Exception("No record has been created for this output path. "
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	208 "Did you bake there yet?")
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	209
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	210 record = BakeRecord.load(record_cache.getCachePath(record_name))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	211 logging.info("Bake record for: %s" % record.out_dir)
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	212 logging.info("From: %s" % record_name)
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	213 logging.info("Last baked: %s" %
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	214 datetime.datetime.fromtimestamp(record.bake_time))
218 10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	215 if record.success:
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	216 logging.info("Status: success")
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	217 else:
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	218 logging.error("Status: failed")
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	219 logging.info("Entries:")
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	220 for entry in record.entries:
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	221 if pattern and not fnmatch.fnmatch(entry.path, pattern):
331 f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	222 continue
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	223 if out_pattern and not (
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	224 any([o for o in entry.out_paths
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	225 if fnmatch.fnmatch(o, out_pattern)])):
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	226 continue
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	227
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	228 flags = _get_flag_descriptions(
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	229 entry.flags,
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	230 {
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	231 BakeRecordEntry.FLAG_NEW: 'new',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	232 BakeRecordEntry.FLAG_SOURCE_MODIFIED: 'modified',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	233 BakeRecordEntry.FLAG_OVERRIDEN: 'overriden'})
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	234
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	235 logging.info(" - ")
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	236
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	237 rel_path = os.path.relpath(entry.path, ctx.app.root_dir)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	238 logging.info(" path: %s" % rel_path)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	239 logging.info(" source: %s" % entry.source_name)
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	240 if entry.taxonomy_info:
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	241 ti = entry.taxonomy_info
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	242 logging.info(" taxonomy: %s = %s (in %s)" %
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	243 (ti.taxonomy_name, ti.term, ti.source_name))
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	244 else:
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	245 logging.info(" taxonomy: <none>")
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	246 logging.info(" flags: %s" % _join(flags))
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	247 logging.info(" config: %s" % entry.config)
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	248
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	249 if entry.errors:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	250 logging.error(" errors: %s" % entry.errors)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	251
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	252 logging.info(" %d sub-pages:" % len(entry.subs))
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	253 for sub in entry.subs:
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	254 sub_flags = _get_flag_descriptions(
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	255 sub.flags,
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	256 {
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	257 SubPageBakeInfo.FLAG_BAKED: 'baked',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	258 SubPageBakeInfo.FLAG_FORCED_BY_SOURCE:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	259 'forced by source',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	260 SubPageBakeInfo.FLAG_FORCED_BY_NO_PREVIOUS:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	261 'forced by missing previous record entry',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	262 SubPageBakeInfo.FLAG_FORCED_BY_PREVIOUS_ERRORS:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	263 'forced by previous errors',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	264 SubPageBakeInfo.FLAG_FORMATTING_INVALIDATED:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	265 'formatting invalidated'})
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	266
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	267 logging.info(" - ")
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	268 logging.info(" URL: %s" % sub.out_uri)
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	269 logging.info(" path: %s" % os.path.relpath(
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	270 sub.out_path, record.out_dir))
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	271 logging.info(" flags: %s" % _join(sub_flags))
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	272
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	273 if sub.render_info:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	274 pass_names = {
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	275 PASS_FORMATTING: 'formatting pass',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	276 PASS_RENDERING: 'rendering pass'}
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	277 for p, ri in sub.render_info.items():
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	278 logging.info(" - %s" % pass_names[p])
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	279 logging.info(" used sources: %s" %
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	280 _join(ri.used_source_names))
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	281 pgn_info = 'no'
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	282 if ri.used_pagination:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	283 pgn_info = 'yes'
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	284 if ri.pagination_has_more:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	285 pgn_info += ', has more'
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	286 logging.info(" used pagination: %s", pgn_info)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	287 logging.info(" used assets: %s",
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	288 'yes' if ri.used_assets else 'no')
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	289 logging.info(" used terms: %s" %
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	290 _join(
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	291 ['%s=%s (%s)' % (tn, t, sn)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	292 for sn, tn, t in
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	293 ri.used_taxonomy_terms]))
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	294 else:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	295 logging.info(" no render info")
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	296
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	297 if sub.errors:
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	298 logging.error(" errors: %s" % sub.errors)
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	299
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	300 def _showProcessingRecord(self, ctx, record_name, pattern, out_pattern):
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	301 record_cache = ctx.app.cache.getCache('proc')
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	302 if not record_cache.has(record_name):
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	303 raise Exception("No record has been created for this output path. "
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	304 "Did you bake there yet?")
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	305
218 10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	306 # Show the pipeline record.
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	307 record = ProcessorPipelineRecord.load(
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	308 record_cache.getCachePath(record_name))
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	309 logging.info("")
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	310 logging.info("Processing record for: %s" % record.out_dir)
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	311 logging.info("Last baked: %s" %
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	312 datetime.datetime.fromtimestamp(record.process_time))
218 10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	313 if record.success:
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	314 logging.info("Status: success")
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	315 else:
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	316 logging.error("Status: failed")
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	317 logging.info("Entries:")
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	318 for entry in record.entries:
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	319 rel_path = os.path.relpath(entry.path, ctx.app.root_dir)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	320 if pattern and not fnmatch.fnmatch(rel_path, pattern):
331 f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	321 continue
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	322 if out_pattern and not (
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	323 any([o for o in entry.rel_outputs
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	324 if fnmatch.fnmatch(o, out_pattern)])):
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	325 continue
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	326
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	327 flags = _get_flag_descriptions(
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	328 entry.flags,
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	329 {
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	330 FLAG_PREPARED: 'prepared',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	331 FLAG_PROCESSED: 'processed',
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	332 FLAG_BYPASSED_STRUCTURED_PROCESSING: 'external'})
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	333
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	334 logger.info(" - ")
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	335 logger.info(" path: %s" % rel_path)
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	336 logger.info(" out paths: %s" % entry.rel_outputs)
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	337 logger.info(" flags: %s" % _join(flags))
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	338 logger.info(" proc tree: %s" % _format_proc_tree(
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	339 entry.proc_tree, 14*' '))
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	340
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	341 if entry.errors:
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	342 logger.error(" errors: %s" % entry.errors)
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	343
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	344
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	345 def _join(items, sep=', ', text_if_none='none'):
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	346 if items:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	347 return sep.join(items)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	348 return text_if_none
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	349
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	350
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	351 def _get_flag_descriptions(flags, descriptions):
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	352 res = []
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	353 for k, v in descriptions.items():
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	354 if flags & k:
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	355 res.append(v)
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	356 return res
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	357
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	358
0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	359 def _format_proc_tree(tree, margin='', level=0):
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	360 name, children = tree
221 f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	361 res = '%s%s+ %s\n' % (margin if level > 0 else '', level * ' ', name)
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	362 if children:
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	363 for c in children:
415 0e9a94b7fdfa bake: Improve bake record information. Ludovic Chabant <ludovic@chabant.com> parents: 411 diff changeset	364 res += _format_proc_tree(c, margin, level + 1)
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	365 return res
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	366

Mercurial > piecrust2

annotate piecrust/commands/builtin/baking.py @ 549:7453baeb0839