Mercurial > piecrust2
annotate piecrust/commands/builtin/baking.py @ 196:154b8df04829
processing: Add Compass and Sass processors.
The Sass processor is similar to the Less processor, i.e. it tries to be
part of the structured pipeline processing by using the mapfile produced by
the Sass compiler in order to provide a list of dependencies.
The Compass processor is completely acting outside of the pipeline, so the
server won't know what's up to date and what's not. It's expected that the
user will run `compass watch` to keep things up to date. However, it will
require to pass the server's cache directory to put things in, so we'll need
to add some easy way to get that path for the user.
author | Ludovic Chabant <ludovic@chabant.com> |
---|---|
date | Sun, 11 Jan 2015 23:08:49 -0800 |
parents | 5d8351cb32d8 |
children | aaf08277b96d |
rev | line source |
---|---|
121 | 1 import time |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
2 import os.path |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
3 import logging |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
4 import hashlib |
92
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
5 import fnmatch |
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
6 import datetime |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
7 from piecrust.baking.baker import Baker |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
8 from piecrust.baking.records import BakeRecord |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
9 from piecrust.chefutil import format_timed |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
10 from piecrust.commands.base import ChefCommand |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
11 from piecrust.processing.base import ProcessorPipeline |
194
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
12 from piecrust.processing.records import ( |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
13 ProcessorPipelineRecord, FLAG_OVERRIDEN) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
14 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
15 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
16 logger = logging.getLogger(__name__) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
17 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
18 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
19 class BakeCommand(ChefCommand): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
20 def __init__(self): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
21 super(BakeCommand, self).__init__() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
22 self.name = 'bake' |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
23 self.description = "Bakes your website into static HTML files." |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
24 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
25 def setupParser(self, parser, app): |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
26 parser.add_argument( |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
27 '-o', '--output', |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
28 help="The directory to put all the baked HTML files into " |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
29 "(defaults to `_counter`)") |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
30 parser.add_argument( |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
31 '-f', '--force', |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
32 help="Force re-baking the entire website.", |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
33 action='store_true') |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
34 parser.add_argument( |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
35 '--portable', |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
36 help="Uses relative paths for all URLs.", |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
37 action='store_true') |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
38 parser.add_argument( |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
39 '--no-assets', |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
40 help="Don't process assets (only pages).", |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
41 action='store_true') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
42 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
43 def run(self, ctx): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
44 if ctx.args.portable: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
45 # Disable pretty URLs because there's likely not going to be |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
46 # a web server to handle serving default documents. |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
47 ctx.app.config.set('site/pretty_urls', False) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
48 |
127
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
49 out_dir = (ctx.args.output or |
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
50 os.path.join(ctx.app.root_dir, '_counter')) |
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
51 |
121 | 52 start_time = time.clock() |
39
2f717f961996
Better error reporting and cache validation.
Ludovic Chabant <ludovic@chabant.com>
parents:
3
diff
changeset
|
53 try: |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
54 # Bake the site sources. |
127
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
55 self._bakeSources(ctx, out_dir) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
56 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
57 # Bake the assets. |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
58 if not ctx.args.no_assets: |
127
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
59 self._bakeAssets(ctx, out_dir) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
60 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
61 # All done. |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
62 logger.info('-------------------------') |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
63 logger.info(format_timed(start_time, 'done baking')) |
39
2f717f961996
Better error reporting and cache validation.
Ludovic Chabant <ludovic@chabant.com>
parents:
3
diff
changeset
|
64 return 0 |
2f717f961996
Better error reporting and cache validation.
Ludovic Chabant <ludovic@chabant.com>
parents:
3
diff
changeset
|
65 except Exception as ex: |
43 | 66 if ctx.app.debug: |
67 logger.exception(ex) | |
68 else: | |
69 logger.error(str(ex)) | |
39
2f717f961996
Better error reporting and cache validation.
Ludovic Chabant <ludovic@chabant.com>
parents:
3
diff
changeset
|
70 return 1 |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
71 |
127
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
72 def _bakeSources(self, ctx, out_dir): |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
73 num_workers = ctx.app.config.get('baker/workers') or 4 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
74 baker = Baker( |
127
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
75 ctx.app, out_dir, |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
76 force=ctx.args.force, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
77 portable=ctx.args.portable, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
78 no_assets=ctx.args.no_assets, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
79 num_workers=num_workers) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
80 baker.bake() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
81 |
127
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
82 def _bakeAssets(self, ctx, out_dir): |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
83 mounts = ctx.app.assets_dirs |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
84 baker_params = ctx.app.config.get('baker') or {} |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
85 skip_patterns = baker_params.get('skip_patterns') |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
86 force_patterns = baker_params.get('force_patterns') |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
87 num_workers = ctx.app.config.get('baker/workers') or 4 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
88 proc = ProcessorPipeline( |
127
bc63dc20baa0
Fix how we pass the out directory to the baking modules.
Ludovic Chabant <ludovic@chabant.com>
parents:
121
diff
changeset
|
89 ctx.app, mounts, out_dir, |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
90 force=ctx.args.force, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
91 skip_patterns=skip_patterns, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
92 force_patterns=force_patterns, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
93 num_workers=num_workers) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
94 proc.run() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
95 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
96 |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
97 class ShowRecordCommand(ChefCommand): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
98 def __init__(self): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
99 super(ShowRecordCommand, self).__init__() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
100 self.name = 'showrecord' |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
101 self.description = ("Shows the bake record for a given output " |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
102 "directory.") |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
103 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
104 def setupParser(self, parser, app): |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
105 parser.add_argument( |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
106 '-o', '--output', |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
107 help="The output directory for which to show the bake record " |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
108 "(defaults to `_counter`)", |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
109 nargs='?') |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
110 parser.add_argument( |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
111 '-p', '--path', |
92
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
112 help="A pattern that will be used to filter the relative path " |
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
113 "of entries to show.") |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
114 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
115 def run(self, ctx): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
116 out_dir = ctx.args.output or os.path.join(ctx.app.root_dir, '_counter') |
194
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
117 record_name = (hashlib.md5(out_dir.encode('utf8')).hexdigest() + |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
118 '.record') |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
119 |
92
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
120 pattern = None |
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
121 if ctx.args.path: |
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
122 pattern = '*%s*' % ctx.args.path.strip('*') |
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
123 |
194
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
124 record_cache = ctx.app.cache.getCache('baker') |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
125 if not record_cache.has(record_name): |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
126 raise Exception("No record has been created for this output path. " |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
127 "Did you bake there yet?") |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
128 |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
129 record = BakeRecord.load(record_cache.getCachePath(record_name)) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
130 logging.info("Bake record for: %s" % record.out_dir) |
92
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
131 logging.info("Last baked: %s" % |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
132 datetime.datetime.fromtimestamp(record.bake_time)) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
133 logging.info("Entries:") |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
134 for entry in record.entries: |
92
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
135 if pattern: |
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
136 if not fnmatch.fnmatch(entry.rel_path, pattern): |
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
137 continue |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
138 logging.info(" - ") |
192
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
139 logging.info(" path: %s" % entry.rel_path) |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
140 logging.info(" spec: %s:%s" % (entry.source_name, |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
141 entry.rel_path)) |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
142 logging.info(" taxonomy: %s:%s" % (entry.taxonomy_name, |
4c0ab0b044fe
cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents:
127
diff
changeset
|
143 entry.taxonomy_term)) |
92
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
144 logging.info(" config: %s" % entry.config) |
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
145 logging.info(" out URLs: %s" % entry.out_uris) |
193
c2acf5f31936
showrecord: Show relative paths.
Ludovic Chabant <ludovic@chabant.com>
parents:
192
diff
changeset
|
146 logging.info(" out paths: %s" % [os.path.relpath(p, out_dir) |
c2acf5f31936
showrecord: Show relative paths.
Ludovic Chabant <ludovic@chabant.com>
parents:
192
diff
changeset
|
147 for p in entry.out_paths]) |
92
0dd43c5f5484
More options for the `showrecord` command.
Ludovic Chabant <ludovic@chabant.com>
parents:
45
diff
changeset
|
148 logging.info(" used srcs: %s" % entry.used_source_names) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
149 if entry.errors: |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
92
diff
changeset
|
150 logging.error(" errors: %s" % entry.errors) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
151 |
194
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
152 record_cache = ctx.app.cache.getCache('proc') |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
153 if not record_cache.has(record_name): |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
154 return |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
155 |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
156 record = ProcessorPipelineRecord.load( |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
157 record_cache.getCachePath(record_name)) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
158 logging.info("") |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
159 logging.info("Processing record for: %s" % record.out_dir) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
160 logging.info("Last baked: %s" % |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
161 datetime.datetime.fromtimestamp(record.process_time)) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
162 logging.info("Entries:") |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
163 for entry in record.entries: |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
164 if pattern: |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
165 if not fnmatch.fnmatch(entry.rel_input, pattern): |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
166 continue |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
167 flags = '' |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
168 if entry.flags & FLAG_OVERRIDEN: |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
169 flags += 'overriden' |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
170 logger.info(" - ") |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
171 logger.info(" path: %s" % entry.rel_input) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
172 logger.info(" out paths: %s" % entry.rel_outputs) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
173 logger.info(" flags: %s" % flags) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
174 logger.info(" proc tree: %s" % format_proc_tree( |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
175 entry.proc_tree, 14*' ')) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
176 if entry.errors: |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
177 logger.error(" errors: %s" % entry.errors) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
178 |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
179 |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
180 def format_proc_tree(tree, margin='', level=0): |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
181 name, children = tree |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
182 res = '%s%s%s' % (margin if level > 0 else '', level * ' ', name) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
183 if children: |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
184 for c in children: |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
185 res += format_proc_tree(c, margin, level + 1) |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
186 return res |
5d8351cb32d8
showrecord: Also show the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
193
diff
changeset
|
187 |