Mercurial > piecrust2
annotate piecrust/processing/base.py @ 196:154b8df04829
processing: Add Compass and Sass processors.
The Sass processor is similar to the Less processor, i.e. it tries to be
part of the structured pipeline processing by using the mapfile produced by
the Sass compiler in order to provide a list of dependencies.
The Compass processor is completely acting outside of the pipeline, so the
server won't know what's up to date and what's not. It's expected that the
user will run `compass watch` to keep things up to date. However, it will
require to pass the server's cache directory to put things in, so we'll need
to add some easy way to get that path for the user.
author | Ludovic Chabant <ludovic@chabant.com> |
---|---|
date | Sun, 11 Jan 2015 23:08:49 -0800 |
parents | 308d5180bf81 |
children | 0c9de41689bb |
rev | line source |
---|---|
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
1 import re |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
2 import time |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
3 import shutil |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
4 import os.path |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
5 import logging |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
6 import hashlib |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
7 import threading |
5 | 8 from queue import Queue, Empty |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
9 from piecrust.chefutil import format_timed |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
10 from piecrust.processing.records import ( |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
11 ProcessorPipelineRecordEntry, TransitionalProcessorPipelineRecord, |
191
308d5180bf81
processing: Add more information to the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
133
diff
changeset
|
12 FLAG_PROCESSED, FLAG_OVERRIDEN, FLAG_BYPASSED_STRUCTURED_PROCESSING) |
308d5180bf81
processing: Add more information to the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
133
diff
changeset
|
13 from piecrust.processing.tree import ( |
308d5180bf81
processing: Add more information to the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
133
diff
changeset
|
14 ProcessingTreeBuilder, ProcessingTreeRunner, ProcessingTreeError, |
308d5180bf81
processing: Add more information to the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
133
diff
changeset
|
15 STATE_DIRTY, |
308d5180bf81
processing: Add more information to the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
133
diff
changeset
|
16 print_node, get_node_name_tree) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
17 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
18 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
19 logger = logging.getLogger(__name__) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
20 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
21 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
22 PRIORITY_FIRST = -1 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
23 PRIORITY_NORMAL = 0 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
24 PRIORITY_LAST = 1 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
25 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
26 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
27 class Processor(object): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
28 PROCESSOR_NAME = None |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
29 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
30 def __init__(self): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
31 self.priority = PRIORITY_NORMAL |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
32 self.is_bypassing_structured_processing = False |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
33 self.is_delegating_dependency_check = True |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
34 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
35 def initialize(self, app): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
36 self.app = app |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
37 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
38 def onPipelineStart(self, pipeline): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
39 pass |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
40 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
41 def onPipelineEnd(self, pipeline): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
42 pass |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
43 |
117
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
44 def matches(self, path): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
45 return False |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
46 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
47 def getDependencies(self, path): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
48 return None |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
49 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
50 def getOutputFilenames(self, filename): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
51 return None |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
52 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
53 def process(self, path, out_dir): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
54 pass |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
55 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
56 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
57 class CopyFileProcessor(Processor): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
58 PROCESSOR_NAME = 'copy' |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
59 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
60 def __init__(self): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
61 super(CopyFileProcessor, self).__init__() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
62 self.priority = PRIORITY_LAST |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
63 |
117
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
64 def matches(self, path): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
65 return True |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
66 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
67 def getOutputFilenames(self, filename): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
68 return [filename] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
69 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
70 def process(self, path, out_dir): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
71 out_path = os.path.join(out_dir, os.path.basename(path)) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
72 logger.debug("Copying: %s -> %s" % (path, out_path)) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
73 shutil.copyfile(path, out_path) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
74 return True |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
75 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
76 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
77 class SimpleFileProcessor(Processor): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
78 def __init__(self, extensions=None): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
79 super(SimpleFileProcessor, self).__init__() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
80 self.extensions = extensions or {} |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
81 |
117
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
82 def matches(self, path): |
55
45828c4167ad
Processors can match on other things than just the extension.
Ludovic Chabant <ludovic@chabant.com>
parents:
39
diff
changeset
|
83 for ext in self.extensions: |
117
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
84 if path.endswith('.' + ext): |
55
45828c4167ad
Processors can match on other things than just the extension.
Ludovic Chabant <ludovic@chabant.com>
parents:
39
diff
changeset
|
85 return True |
45828c4167ad
Processors can match on other things than just the extension.
Ludovic Chabant <ludovic@chabant.com>
parents:
39
diff
changeset
|
86 return False |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
87 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
88 def getOutputFilenames(self, filename): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
89 basename, ext = os.path.splitext(filename) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
90 ext = ext.lstrip('.') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
91 out_ext = self.extensions[ext] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
92 return ['%s.%s' % (basename, out_ext)] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
93 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
94 def process(self, path, out_dir): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
95 _, in_name = os.path.split(path) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
96 out_name = self.getOutputFilenames(in_name)[0] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
97 out_path = os.path.join(out_dir, out_name) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
98 return self._doProcess(path, out_path) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
99 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
100 def _doProcess(self, in_path, out_path): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
101 raise NotImplementedError() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
102 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
103 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
104 class ProcessingContext(object): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
105 def __init__(self, base_dir, job_queue, record=None): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
106 self.base_dir = base_dir |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
107 self.job_queue = job_queue |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
108 self.record = record |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
109 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
110 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
111 class ProcessorPipeline(object): |
36
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
112 def __init__(self, app, mounts, out_dir, force=False, |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
113 skip_patterns=None, force_patterns=None, num_workers=4): |
126
e5cba2622d26
Check we don't give null values to the processing pipeline.
Ludovic Chabant <ludovic@chabant.com>
parents:
120
diff
changeset
|
114 assert app and out_dir |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
115 self.app = app |
36
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
116 self.mounts = mounts |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
117 tmp_dir = app.cache_dir |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
118 if not tmp_dir: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
119 import tempfile |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
120 tmp_dir = os.path.join(tempfile.gettempdir(), 'piecrust') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
121 self.tmp_dir = os.path.join(tmp_dir, 'proc') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
122 self.out_dir = out_dir |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
123 self.force = force |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
124 self.skip_patterns = skip_patterns or [] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
125 self.force_patterns = force_patterns or [] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
126 self.processors = app.plugin_loader.getProcessors() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
127 self.num_workers = num_workers |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
128 |
36
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
129 self.skip_patterns += ['_cache', '_counter', |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
130 'theme_info.yml', |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
131 '.DS_Store', 'Thumbs.db', |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
132 '.git*', '.hg*', '.svn'] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
133 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
134 self.skip_patterns = make_re(self.skip_patterns) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
135 self.force_patterns = make_re(self.force_patterns) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
136 |
196
154b8df04829
processing: Add Compass and Sass processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
191
diff
changeset
|
137 def addSkipPatterns(self, patterns): |
154b8df04829
processing: Add Compass and Sass processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
191
diff
changeset
|
138 self.skip_patterns += make_re(patterns) |
154b8df04829
processing: Add Compass and Sass processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
191
diff
changeset
|
139 |
35
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
140 def filterProcessors(self, authorized_names): |
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
141 self.processors = list(filter( |
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
142 lambda p: p.PROCESSOR_NAME in authorized_names, |
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
143 self.processors)) |
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
144 |
133
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
145 def run(self, src_dir_or_file=None, *, |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
146 new_only=False, delete=True, |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
147 previous_record=None, save_record=True): |
117
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
148 # Invoke pre-processors. |
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
149 for proc in self.processors: |
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
150 proc.onPipelineStart(self) |
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
151 |
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
152 # Sort our processors again in case the pre-process step involved |
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
153 # patching the processors with some new ones. |
6827dcc9d3fb
Changes to the asset processing pipeline:
Ludovic Chabant <ludovic@chabant.com>
parents:
55
diff
changeset
|
154 self.processors.sort(key=lambda p: p.priority) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
155 |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
156 # Create the pipeline record. |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
157 record = TransitionalProcessorPipelineRecord() |
133
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
158 record_cache = self.app.cache.getCache('proc') |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
159 record_name = ( |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
160 hashlib.md5(self.out_dir.encode('utf8')).hexdigest() + |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
161 '.record') |
133
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
162 if previous_record: |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
163 record.setPrevious(previous_record) |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
164 elif not self.force and record_cache.has(record_name): |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
165 t = time.clock() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
166 record.loadPrevious(record_cache.getCachePath(record_name)) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
167 logger.debug(format_timed(t, 'loaded previous bake record', |
133
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
168 colored=False)) |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
169 logger.debug("Got %d entries in process record." % |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
170 len(record.previous.entries)) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
171 |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
172 # Create the workers. |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
173 pool = [] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
174 queue = Queue() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
175 abort = threading.Event() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
176 pipeline_lock = threading.Lock() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
177 for i in range(self.num_workers): |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
178 ctx = ProcessingWorkerContext(self, record, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
179 queue, abort, pipeline_lock) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
180 worker = ProcessingWorker(i, ctx) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
181 worker.start() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
182 pool.append(worker) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
183 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
184 if src_dir_or_file is not None: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
185 # Process only the given path. |
36
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
186 # Find out what mount point this is in. |
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
187 for path in self.mounts: |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
188 if src_dir_or_file[:len(path)] == path: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
189 base_dir = path |
36
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
190 break |
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
191 else: |
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
192 raise Exception("Input path '%s' is not part of any known " |
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
193 "mount point: %s" % |
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
194 (src_dir_or_file, self.mounts)) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
195 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
196 ctx = ProcessingContext(base_dir, queue, record) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
197 logger.debug("Initiating processing pipeline on: %s" % src_dir_or_file) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
198 if os.path.isdir(src_dir_or_file): |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
199 self.processDirectory(ctx, src_dir_or_file, new_only) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
200 elif os.path.isfile(src_dir_or_file): |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
201 self.processFile(ctx, src_dir_or_file, new_only) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
202 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
203 else: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
204 # Process everything. |
36
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
205 for path in self.mounts: |
485682a6de50
New site layout support.
Ludovic Chabant <ludovic@chabant.com>
parents:
35
diff
changeset
|
206 ctx = ProcessingContext(path, queue, record) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
207 logger.debug("Initiating processing pipeline on: %s" % path) |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
208 self.processDirectory(ctx, path, new_only) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
209 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
210 # Wait on all workers. |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
211 for w in pool: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
212 w.join() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
213 if abort.is_set(): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
214 raise Exception("Worker pool was aborted.") |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
215 |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
216 # Handle deletions. |
133
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
217 if delete and not new_only: |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
218 for path, reason in record.getDeletions(): |
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
219 logger.debug("Removing '%s': %s" % (path, reason)) |
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
220 os.remove(path) |
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
221 logger.info('[delete] %s' % path) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
222 |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
223 # Invoke post-processors. |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
224 for proc in self.processors: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
225 proc.onPipelineEnd(self) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
226 |
133
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
227 # Finalize the process record. |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
228 record.current.process_time = time.time() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
229 record.current.out_dir = self.out_dir |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
230 record.collapseRecords() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
231 |
133
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
232 # Save the process record. |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
233 if save_record: |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
234 t = time.clock() |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
235 record.saveCurrent(record_cache.getCachePath(record_name)) |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
236 logger.debug(format_timed(t, 'saved bake record', colored=False)) |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
237 |
9e4c2e68a129
Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents:
129
diff
changeset
|
238 return record.detach() |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
239 |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
240 def processDirectory(self, ctx, start_dir, new_only=False): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
241 for dirpath, dirnames, filenames in os.walk(start_dir): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
242 rel_dirpath = os.path.relpath(dirpath, start_dir) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
243 dirnames[:] = [d for d in dirnames |
35
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
244 if not re_matchany(d, self.skip_patterns, rel_dirpath)] |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
245 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
246 for filename in filenames: |
35
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
247 if re_matchany(filename, self.skip_patterns, rel_dirpath): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
248 continue |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
249 self.processFile(ctx, os.path.join(dirpath, filename), |
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
250 new_only) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
251 |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
252 def processFile(self, ctx, path, new_only=False): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
253 logger.debug("Queuing: %s" % path) |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
254 job = ProcessingWorkerJob(ctx.base_dir, path, new_only) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
255 ctx.job_queue.put_nowait(job) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
256 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
257 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
258 class ProcessingWorkerContext(object): |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
259 def __init__(self, pipeline, record, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
260 work_queue, abort_event, pipeline_lock): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
261 self.pipeline = pipeline |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
262 self.record = record |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
263 self.work_queue = work_queue |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
264 self.abort_event = abort_event |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
265 self.pipeline_lock = pipeline_lock |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
266 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
267 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
268 class ProcessingWorkerJob(object): |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
269 def __init__(self, base_dir, path, new_only=False): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
270 self.base_dir = base_dir |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
271 self.path = path |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
272 self.new_only = new_only |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
273 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
274 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
275 class ProcessingWorker(threading.Thread): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
276 def __init__(self, wid, ctx): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
277 super(ProcessingWorker, self).__init__() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
278 self.wid = wid |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
279 self.ctx = ctx |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
280 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
281 def run(self): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
282 while(not self.ctx.abort_event.is_set()): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
283 try: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
284 job = self.ctx.work_queue.get(True, 0.1) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
285 except Empty: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
286 logger.debug("[%d] No more work... shutting down." % self.wid) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
287 break |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
288 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
289 try: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
290 self._unsafeRun(job) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
291 logger.debug("[%d] Done with file." % self.wid) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
292 self.ctx.work_queue.task_done() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
293 except Exception as ex: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
294 self.ctx.abort_event.set() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
295 logger.error("[%d] Critical error, aborting." % self.wid) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
296 logger.exception(ex) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
297 break |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
298 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
299 def _unsafeRun(self, job): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
300 start_time = time.clock() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
301 pipeline = self.ctx.pipeline |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
302 record = self.ctx.record |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
303 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
304 rel_path = os.path.relpath(job.path, job.base_dir) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
305 previous_entry = record.getPreviousEntry(rel_path) |
129
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
306 if job.new_only and previous_entry: |
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
307 return |
3080b6d02f40
Add ability for the processing pipeline to only process new assets.
Ludovic Chabant <ludovic@chabant.com>
parents:
126
diff
changeset
|
308 |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
309 record_entry = ProcessorPipelineRecordEntry(job.base_dir, rel_path) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
310 record.addEntry(record_entry) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
311 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
312 # Figure out if a previously processed file is overriding this one. |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
313 # This can happen if a theme file (processed via a mount point) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
314 # is overridden in the user's website. |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
315 if record.current.hasOverrideEntry(rel_path): |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
316 record_entry.flags |= FLAG_OVERRIDEN |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
317 logger.info(format_timed(start_time, |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
318 '%s [not baked, overridden]' % rel_path)) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
319 return |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
320 |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
321 try: |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
322 builder = ProcessingTreeBuilder(pipeline.processors) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
323 tree_root = builder.build(rel_path) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
324 except ProcessingTreeError as ex: |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
325 record_entry.errors.append(str(ex)) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
326 logger.error("Error processing %s: %s" % (rel_path, ex)) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
327 return |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
328 |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
329 print_node(tree_root, recursive=True) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
330 leaves = tree_root.getLeaves() |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
331 record_entry.rel_outputs = [l.path for l in leaves] |
191
308d5180bf81
processing: Add more information to the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
133
diff
changeset
|
332 record_entry.proc_tree = get_node_name_tree(tree_root) |
308d5180bf81
processing: Add more information to the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
133
diff
changeset
|
333 if tree_root.getProcessor().is_bypassing_structured_processing: |
308d5180bf81
processing: Add more information to the pipeline record.
Ludovic Chabant <ludovic@chabant.com>
parents:
133
diff
changeset
|
334 record_entry.flags |= FLAG_BYPASSED_STRUCTURED_PROCESSING |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
335 |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
336 force = (pipeline.force or previous_entry is None or |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
337 not previous_entry.was_processed_successfully) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
338 if not force: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
339 force = re_matchany(rel_path, pipeline.force_patterns) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
340 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
341 if force: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
342 tree_root.setState(STATE_DIRTY, True) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
343 |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
344 try: |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
345 runner = ProcessingTreeRunner( |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
346 job.base_dir, pipeline.tmp_dir, |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
347 pipeline.out_dir, self.ctx.pipeline_lock) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
348 if runner.processSubTree(tree_root): |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
349 record_entry.flags |= FLAG_PROCESSED |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
350 logger.info(format_timed(start_time, "[%d] %s" % (self.wid, rel_path))) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
351 except ProcessingTreeError as ex: |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
352 record_entry.errors.append(str(ex)) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
117
diff
changeset
|
353 logger.error("Error processing %s: %s" % (rel_path, ex)) |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
354 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
355 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
356 def make_re(patterns): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
357 re_patterns = [] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
358 for pat in patterns: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
359 if pat[0] == '/' and pat[-1] == '/' and len(pat) > 2: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
360 re_patterns.append(pat[1:-1]) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
361 else: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
362 escaped_pat = (re.escape(pat) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
363 .replace(r'\*', r'[^/\\]*') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
364 .replace(r'\?', r'[^/\\]')) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
365 re_patterns.append(escaped_pat) |
5 | 366 return [re.compile(p) for p in re_patterns] |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
367 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
368 |
35
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
369 def re_matchany(filename, patterns, dirname=None): |
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
370 if dirname and dirname != '.': |
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
371 filename = os.path.join(dirname, filename) |
e4c345dcf33c
More unit tests, fix a bug with the skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
22
diff
changeset
|
372 |
11
617191dec18e
Fixes for Windows, make `findPagePath` return a ref path.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
373 # skip patterns use a forward slash regardless of the platform. |
617191dec18e
Fixes for Windows, make `findPagePath` return a ref path.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
374 filename = filename.replace('\\', '/') |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
375 for pattern in patterns: |
22
df790a827d38
Correctly match skip patterns.
Ludovic Chabant <ludovic@chabant.com>
parents:
11
diff
changeset
|
376 if pattern.search(filename): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
377 return True |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
378 return False |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
379 |