piecrust2: piecrust/commands/builtin/baking.py annotate

annotate piecrust/commands/builtin/baking.py @ 402:043b9d8304c7 2.0.0a11

linker: Fix linker returning the wrong value for `is_dir` in some situations.

author	Ludovic Chabant <ludovic@chabant.com>
date	Mon, 18 May 2015 23:45:51 -0700
parents	2cd2b5d07129
children	e7b865f8f335

rev	line source
121 300eb1c2cb14 Fix stupid bug. Ludovic Chabant <ludovic@chabant.com> parents: 120 diff changeset	1 import time
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import os.path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import logging
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import hashlib
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	5 import fnmatch
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	6 import datetime
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	7 from piecrust.baking.baker import Baker
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	8 from piecrust.baking.records import (
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	9 BakeRecord, BakeRecordPageEntry, BakeRecordSubPageEntry)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	10 from piecrust.chefutil import format_timed
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11 from piecrust.commands.base import ChefCommand
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	12 from piecrust.processing.base import ProcessorPipeline
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	13 from piecrust.processing.records import (
221 f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	14 ProcessorPipelineRecord,
f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	15 FLAG_PREPARED, FLAG_PROCESSED, FLAG_OVERRIDEN,
f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	16 FLAG_BYPASSED_STRUCTURED_PROCESSING)
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	17 from piecrust.rendering import PASS_FORMATTING, PASS_RENDERING
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 class BakeCommand(ChefCommand):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 super(BakeCommand, self).__init__()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 self.name = 'bake'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 self.description = "Bakes your website into static HTML files."
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 def setupParser(self, parser, app):
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	30 parser.add_argument(
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	31 '-o', '--output',
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	32 help="The directory to put all the baked HTML files into "
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	33 "(defaults to `_counter`)")
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	34 parser.add_argument(
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	35 '-f', '--force',
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	36 help="Force re-baking the entire website.",
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	37 action='store_true')
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	38 parser.add_argument(
220 84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	39 '--assets-only',
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	40 help="Only bake the assets (don't bake the web pages).",
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	41 action='store_true')
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	42 parser.add_argument(
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	43 '--html-only',
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	44 help="Only bake HTML files (don't run the asset pipeline).",
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	45 action='store_true')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 def run(self, ctx):
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	48 out_dir = (ctx.args.output or
bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	49 os.path.join(ctx.app.root_dir, '_counter'))
bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	50
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	51 success = True
121 300eb1c2cb14 Fix stupid bug. Ludovic Chabant <ludovic@chabant.com> parents: 120 diff changeset	52 start_time = time.clock()
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	53 try:
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	54 # Bake the site sources.
220 84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	55 if not ctx.args.assets_only:
84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	56 success = success & self._bakeSources(ctx, out_dir)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	57
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	58 # Bake the assets.
220 84e2bc2d16cb bake: Change arguments to selectively bake to make them symmetrical. Ludovic Chabant <ludovic@chabant.com> parents: 218 diff changeset	59 if not ctx.args.html_only:
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	60 success = success & self._bakeAssets(ctx, out_dir)
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	61
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	62 # All done.
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	63 logger.info('-------------------------')
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	64 logger.info(format_timed(start_time, 'done baking'))
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	65 return 0 if success else 1
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	66 except Exception as ex:
43 f14889d6b067 Proper debug logging. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	67 if ctx.app.debug:
f14889d6b067 Proper debug logging. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	68 logger.exception(ex)
f14889d6b067 Proper debug logging. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	69 else:
f14889d6b067 Proper debug logging. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	70 logger.error(str(ex))
39 2f717f961996 Better error reporting and cache validation. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	71 return 1
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	73 def _bakeSources(self, ctx, out_dir):
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	74 baker = Baker(
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	75 ctx.app, out_dir,
205 e725af1d48fb bake: Changes in how assets directories are configured. Ludovic Chabant <ludovic@chabant.com> parents: 201 diff changeset	76 force=ctx.args.force)
217 1f4c3dae1fe8 bake: Better error handling for site baking. Ludovic Chabant <ludovic@chabant.com> parents: 215 diff changeset	77 record = baker.bake()
1f4c3dae1fe8 bake: Better error handling for site baking. Ludovic Chabant <ludovic@chabant.com> parents: 215 diff changeset	78 return record.success
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	79
127 bc63dc20baa0 Fix how we pass the out directory to the baking modules. Ludovic Chabant <ludovic@chabant.com> parents: 121 diff changeset	80 def _bakeAssets(self, ctx, out_dir):
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	81 proc = ProcessorPipeline(
205 e725af1d48fb bake: Changes in how assets directories are configured. Ludovic Chabant <ludovic@chabant.com> parents: 201 diff changeset	82 ctx.app, out_dir,
e725af1d48fb bake: Changes in how assets directories are configured. Ludovic Chabant <ludovic@chabant.com> parents: 201 diff changeset	83 force=ctx.args.force)
215 a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	84 record = proc.run()
a47580a0955b bake: Better error handling for the processing pipeline. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	85 return record.success
120 133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	86
133845647083 Better error management and removal support in baking/processing. Ludovic Chabant <ludovic@chabant.com> parents: 92 diff changeset	87
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 class ShowRecordCommand(ChefCommand):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	90 super(ShowRecordCommand, self).__init__()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	91 self.name = 'showrecord'
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	92 self.description = ("Shows the bake record for a given output "
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	93 "directory.")
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	94
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	95 def setupParser(self, parser, app):
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	96 parser.add_argument(
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	97 '-o', '--output',
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	98 help="The output directory for which to show the bake record "
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	99 "(defaults to `_counter`)",
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	100 nargs='?')
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	101 parser.add_argument(
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	102 '-p', '--path',
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	103 help="A pattern that will be used to filter the relative path "
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	104 "of entries to show.")
331 f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	105 parser.add_argument(
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	106 '-t', '--out',
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	107 help="A pattern that will be used to filter the output path "
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	108 "of entries to show.")
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	109 parser.add_argument(
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	110 '--last',
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	111 type=int,
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	112 default=0,
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	113 help="Show the last Nth bake record.")
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	114
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	115 def run(self, ctx):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	116 out_dir = ctx.args.output or os.path.join(ctx.app.root_dir, '_counter')
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	117 record_id = hashlib.md5(out_dir.encode('utf8')).hexdigest()
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	118 suffix = '' if ctx.args.last == 0 else '.%d' % ctx.args.last
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	119 record_name = '%s%s.record' % (record_id, suffix)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	121 pattern = None
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	122 if ctx.args.path:
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	123 pattern = '%s' % ctx.args.path.strip('*')
0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	124
331 f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	125 out_pattern = None
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	126 if ctx.args.out:
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	127 out_pattern = '%s' % ctx.args.out.strip('*')
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	128
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	129 record_cache = ctx.app.cache.getCache('baker')
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	130 if not record_cache.has(record_name):
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	131 raise Exception("No record has been created for this output path. "
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	132 "Did you bake there yet?")
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	133
218 10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	134 # Show the bake record.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	135 record = BakeRecord.load(record_cache.getCachePath(record_name))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	136 logging.info("Bake record for: %s" % record.out_dir)
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	137 logging.info("From: %s" % record_name)
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	138 logging.info("Last baked: %s" %
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	139 datetime.datetime.fromtimestamp(record.bake_time))
218 10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	140 if record.success:
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	141 logging.info("Status: success")
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	142 else:
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	143 logging.error("Status: failed")
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	144 logging.info("Entries:")
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	145 for entry in record.entries:
331 f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	146 if pattern and not fnmatch.fnmatch(entry.rel_path, pattern):
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	147 continue
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	148 if out_pattern and not (
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	149 any([o for o in entry.out_paths
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	150 if fnmatch.fnmatch(o, out_pattern)])):
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	151 continue
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	152
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	153 flags = []
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	154 if entry.flags & BakeRecordPageEntry.FLAG_OVERRIDEN:
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	155 flags.append('overriden')
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	156
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	157 passes = {PASS_RENDERING: 'render', PASS_FORMATTING: 'format'}
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	158
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159 logging.info(" - ")
192 4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	160 logging.info(" path: %s" % entry.rel_path)
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	161 logging.info(" spec: %s:%s" % (entry.source_name,
4c0ab0b044fe cosmetic: Fix some PEP8 issues. Ludovic Chabant <ludovic@chabant.com> parents: 127 diff changeset	162 entry.rel_path))
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	163 if entry.taxonomy_info:
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	164 tn, t, sn = entry.taxonomy_info
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	165 logging.info(" taxonomy: %s (%s:%s)" %
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	166 (t, sn, tn))
334 b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	167 else:
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	168 logging.info(" taxonomy: <none>")
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes. Ludovic Chabant <ludovic@chabant.com> parents: 331 diff changeset	169 logging.info(" flags: %s" % ', '.join(flags))
92 0dd43c5f5484 More options for the `showrecord` command. Ludovic Chabant <ludovic@chabant.com> parents: 45 diff changeset	170 logging.info(" config: %s" % entry.config)
338 938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	171
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	172 logging.info(" %d sub-pages:" % len(entry.subs))
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	173 for sub in entry.subs:
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	174 logging.info(" - ")
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	175 logging.info(" URL: %s" % sub.out_uri)
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	176 logging.info(" path: %s" % os.path.relpath(sub.out_path,
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	177 out_dir))
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	178 logging.info(" baked?: %s" % sub.was_baked)
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	179
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	180 sub_flags = []
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	181 if sub.flags & BakeRecordSubPageEntry.FLAG_FORCED_BY_SOURCE:
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	182 sub_flags.append('forced by source')
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	183 if sub.flags & BakeRecordSubPageEntry.FLAG_FORCED_BY_NO_PREVIOUS:
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	184 sub_flags.append('forced by missing previous record entry')
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	185 if sub.flags & BakeRecordSubPageEntry.FLAG_FORCED_BY_PREVIOUS_ERRORS:
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	186 sub_flags.append('forced by previous errors')
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	187 logging.info(" flags: %s" % ', '.join(sub_flags))
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	188
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	189 for p, pi in sub.render_passes.items():
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	190 logging.info(" %s pass:" % passes[p])
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	191 logging.info(" used srcs: %s" %
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	192 ', '.join(pi.used_source_names))
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	193 logging.info(" used terms: %s" %
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	194 ', '.join(
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	195 ['%s (%s:%s)' % (t, sn, tn)
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	196 for sn, tn, t in pi.used_taxonomy_terms]))
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	197
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	198 if sub.errors:
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	199 logging.error(" errors: %s" % sub.errors)
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	200
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs. Ludovic Chabant <ludovic@chabant.com> parents: 334 diff changeset	201 logging.info(" assets: %s" % ', '.join(entry.assets))
359 2cd2b5d07129 bake: Fix crash when handling bake errors. Ludovic Chabant <ludovic@chabant.com> parents: 338 diff changeset	202 if entry.errors:
2cd2b5d07129 bake: Fix crash when handling bake errors. Ludovic Chabant <ludovic@chabant.com> parents: 338 diff changeset	203 logging.error(" errors: %s" % entry.errors)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	204
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	205 record_cache = ctx.app.cache.getCache('proc')
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	206 if not record_cache.has(record_name):
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	207 return
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	208
218 10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	209 # Show the pipeline record.
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	210 record = ProcessorPipelineRecord.load(
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	211 record_cache.getCachePath(record_name))
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	212 logging.info("")
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	213 logging.info("Processing record for: %s" % record.out_dir)
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	214 logging.info("Last baked: %s" %
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	215 datetime.datetime.fromtimestamp(record.process_time))
218 10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	216 if record.success:
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	217 logging.info("Status: success")
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	218 else:
10f24c62b05b showrecord: Show the overall status (success/failed) of the bake. Ludovic Chabant <ludovic@chabant.com> parents: 217 diff changeset	219 logging.error("Status: failed")
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	220 logging.info("Entries:")
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	221 for entry in record.entries:
331 f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	222 if pattern and not fnmatch.fnmatch(entry.rel_input, pattern):
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	223 continue
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	224 if out_pattern and not (
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	225 any([o for o in entry.rel_outputs
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	226 if fnmatch.fnmatch(o, out_pattern)])):
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	227 continue
f0fc2a9d3191 showrecord: Add ability to filter on the output path. Ludovic Chabant <ludovic@chabant.com> parents: 221 diff changeset	228
221 f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	229 flags = []
f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	230 if entry.flags & FLAG_PREPARED:
f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	231 flags.append('prepared')
f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	232 if entry.flags & FLAG_PROCESSED:
f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	233 flags.append('processed')
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	234 if entry.flags & FLAG_OVERRIDEN:
221 f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	235 flags.append('overriden')
f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	236 if entry.flags & FLAG_BYPASSED_STRUCTURED_PROCESSING:
f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	237 flags.append('external')
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	238 logger.info(" - ")
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	239 logger.info(" path: %s" % entry.rel_input)
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	240 logger.info(" out paths: %s" % entry.rel_outputs)
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	241 logger.info(" flags: %s" % flags)
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	242 logger.info(" proc tree: %s" % format_proc_tree(
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	243 entry.proc_tree, 14*' '))
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	244 if entry.errors:
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	245 logger.error(" errors: %s" % entry.errors)
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	246
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	247
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	248 def format_proc_tree(tree, margin='', level=0):
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	249 name, children = tree
221 f82262f59600 bake: Fix processing record bugs and error logging for external processes. Ludovic Chabant <ludovic@chabant.com> parents: 220 diff changeset	250 res = '%s%s+ %s\n' % (margin if level > 0 else '', level * ' ', name)
194 5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	251 if children:
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	252 for c in children:
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	253 res += format_proc_tree(c, margin, level + 1)
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	254 return res
5d8351cb32d8 showrecord: Also show the pipeline record. Ludovic Chabant <ludovic@chabant.com> parents: 193 diff changeset	255

Mercurial > piecrust2

annotate piecrust/commands/builtin/baking.py @ 402:043b9d8304c7 2.0.0a11