Mercurial > piecrust2

annotate piecrust/baking/records.py @ 837:ad8f48a31c62

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
assets: Fix crash when a page doesn't have assets.
author Ludovic Chabant <ludovic@chabant.com>
date Sun, 05 Feb 2017 22:52:01 -0800
parents 9a92e2804562
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
1 import copy
91
e88e330eb8dc Improvements to incremental baking and cache invalidating.
Ludovic Chabant <ludovic@chabant.com>
parents: 85
diff changeset
2 import os.path
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
3 import hashlib
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import logging
120
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
5 from piecrust.records import Record, TransitionalRecord
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
11 def _get_transition_key(path, extra_key=None):
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
12 key = path
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
13 if extra_key:
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
14 key += '+%s' % extra_key
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
15 return hashlib.md5(key.encode('utf8')).hexdigest()
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18 class BakeRecord(Record):
753
9a92e2804562 bake: Add the list of deleted files to the bake/processing records.
Ludovic Chabant <ludovic@chabant.com>
parents: 721
diff changeset
19 RECORD_VERSION = 20
120
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
20
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
22 super(BakeRecord, self).__init__()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 self.out_dir = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 self.bake_time = None
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
25 self.baked_count = {}
691
9ae9390192da bake: Use standard pickle and queue for now to fix some small issues.
Ludovic Chabant <ludovic@chabant.com>
parents: 687
diff changeset
26 self.total_baked_count = {}
753
9a92e2804562 bake: Add the list of deleted files to the bake/processing records.
Ludovic Chabant <ludovic@chabant.com>
parents: 721
diff changeset
27 self.deleted = []
217
1f4c3dae1fe8 bake: Better error handling for site baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 120
diff changeset
28 self.success = True
91
e88e330eb8dc Improvements to incremental baking and cache invalidating.
Ludovic Chabant <ludovic@chabant.com>
parents: 85
diff changeset
29
e88e330eb8dc Improvements to incremental baking and cache invalidating.
Ludovic Chabant <ludovic@chabant.com>
parents: 85
diff changeset
30
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
31 class SubPageBakeInfo(object):
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
32 FLAG_NONE = 0
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
33 FLAG_BAKED = 2**0
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
34 FLAG_FORCED_BY_SOURCE = 2**1
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
35 FLAG_FORCED_BY_NO_PREVIOUS = 2**2
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
36 FLAG_FORCED_BY_PREVIOUS_ERRORS = 2**3
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
37 FLAG_FORMATTING_INVALIDATED = 2**4
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
38
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
39 def __init__(self, out_uri, out_path):
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
40 self.out_uri = out_uri
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
41 self.out_path = out_path
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
42 self.flags = self.FLAG_NONE
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
43 self.errors = []
698
33ab9badfd7a render: Change how we store render passes info.
Ludovic Chabant <ludovic@chabant.com>
parents: 691
diff changeset
44 self.render_info = [None, None] # Same length as RENDER_PASSES
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
45
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
46 @property
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
47 def was_clean(self):
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
48 return (self.flags & self.FLAG_BAKED) == 0 and len(self.errors) == 0
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
49
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
50 @property
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
51 def was_baked(self):
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
52 return (self.flags & self.FLAG_BAKED) != 0
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
53
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
54 @property
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
55 def was_baked_successfully(self):
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
56 return self.was_baked and len(self.errors) == 0
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
57
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
58 def anyPass(self, func):
698
33ab9badfd7a render: Change how we store render passes info.
Ludovic Chabant <ludovic@chabant.com>
parents: 691
diff changeset
59 for pinfo in self.render_info:
33ab9badfd7a render: Change how we store render passes info.
Ludovic Chabant <ludovic@chabant.com>
parents: 691
diff changeset
60 if pinfo and func(pinfo):
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
61 return True
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
62 return False
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
63
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
64 def copyRenderInfo(self):
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
65 return copy.deepcopy(self.render_info)
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
66
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
67
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
68 class BakeRecordEntry(object):
334
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 217
diff changeset
69 """ An entry in the bake record.
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 217
diff changeset
70 """
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
71 FLAG_NONE = 0
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
72 FLAG_NEW = 2**0
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
73 FLAG_SOURCE_MODIFIED = 2**1
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
74 FLAG_OVERRIDEN = 2**2
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
75
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
76 def __init__(self, source_name, path, extra_key=None):
334
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 217
diff changeset
77 self.source_name = source_name
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 217
diff changeset
78 self.path = path
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
79 self.extra_key = extra_key
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
80 self.flags = self.FLAG_NONE
50
2fec3ee1298f Properly override pages between realms.
Ludovic Chabant <ludovic@chabant.com>
parents: 23
diff changeset
81 self.config = None
721
234d0c7c02cf bake: Add the timestamp of the page to each record entry.
Ludovic Chabant <ludovic@chabant.com>
parents: 711
diff changeset
82 self.timestamp = None
359
2cd2b5d07129 bake: Fix crash when handling bake errors.
Ludovic Chabant <ludovic@chabant.com>
parents: 338
diff changeset
83 self.errors = []
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
84 self.subs = []
334
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 217
diff changeset
85
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 217
diff changeset
86 @property
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 217
diff changeset
87 def path_mtime(self):
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 217
diff changeset
88 return os.path.getmtime(self.path)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
89
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
90 @property
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
91 def was_overriden(self):
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
92 return (self.flags & self.FLAG_OVERRIDEN) != 0
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
93
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
94 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
95 def num_subs(self):
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
96 return len(self.subs)
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
97
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
98 @property
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
99 def was_any_sub_baked(self):
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
100 for o in self.subs:
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
101 if o.was_baked:
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
102 return True
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
103 return False
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
104
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
105 @property
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
106 def all_assets(self):
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
107 for sub in self.subs:
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
108 yield from sub.assets
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
109
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
110 @property
753
9a92e2804562 bake: Add the list of deleted files to the bake/processing records.
Ludovic Chabant <ludovic@chabant.com>
parents: 721
diff changeset
111 def all_out_paths(self):
9a92e2804562 bake: Add the list of deleted files to the bake/processing records.
Ludovic Chabant <ludovic@chabant.com>
parents: 721
diff changeset
112 for sub in self.subs:
9a92e2804562 bake: Add the list of deleted files to the bake/processing records.
Ludovic Chabant <ludovic@chabant.com>
parents: 721
diff changeset
113 yield sub.out_path
9a92e2804562 bake: Add the list of deleted files to the bake/processing records.
Ludovic Chabant <ludovic@chabant.com>
parents: 721
diff changeset
114
9a92e2804562 bake: Add the list of deleted files to the bake/processing records.
Ludovic Chabant <ludovic@chabant.com>
parents: 721
diff changeset
115 @property
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
116 def has_any_error(self):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
117 if len(self.errors) > 0:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
118 return True
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
119 for o in self.subs:
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
120 if len(o.errors) > 0:
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
121 return True
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
122 return False
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
123
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
124 def getSub(self, sub_index):
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
125 return self.subs[sub_index - 1]
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
126
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
127 def getAllErrors(self):
400
c12ee6936b8c bake: Return all errors from a bake record entry when asked for it.
Ludovic Chabant <ludovic@chabant.com>
parents: 359
diff changeset
128 yield from self.errors
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
129 for o in self.subs:
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
130 yield from o.errors
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
131
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
132 def getAllUsedSourceNames(self):
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
133 res = set()
415
0e9a94b7fdfa bake: Improve bake record information.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
134 for o in self.subs:
698
33ab9badfd7a render: Change how we store render passes info.
Ludovic Chabant <ludovic@chabant.com>
parents: 691
diff changeset
135 for pinfo in o.render_info:
33ab9badfd7a render: Change how we store render passes info.
Ludovic Chabant <ludovic@chabant.com>
parents: 691
diff changeset
136 if pinfo:
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
137 res |= pinfo.used_source_names
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
138 return res
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
139
23
923699e816d0 Don't try to get the name of a source that doesn't have one.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
140
120
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
141 class TransitionalBakeRecord(TransitionalRecord):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
142 def __init__(self, previous_path=None):
120
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
143 super(TransitionalBakeRecord, self).__init__(BakeRecord,
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
144 previous_path)
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
145 self.dirty_source_names = set()
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
146
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
147 def addEntry(self, entry):
91
e88e330eb8dc Improvements to incremental baking and cache invalidating.
Ludovic Chabant <ludovic@chabant.com>
parents: 85
diff changeset
148 if (self.previous.bake_time and
e88e330eb8dc Improvements to incremental baking and cache invalidating.
Ludovic Chabant <ludovic@chabant.com>
parents: 85
diff changeset
149 entry.path_mtime >= self.previous.bake_time):
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
150 entry.flags |= BakeRecordEntry.FLAG_SOURCE_MODIFIED
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
151 self.dirty_source_names.add(entry.source_name)
120
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
152 super(TransitionalBakeRecord, self).addEntry(entry)
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
153
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
154 def getTransitionKey(self, entry):
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
155 return _get_transition_key(entry.path, entry.extra_key)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
156
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
157 def getPreviousAndCurrentEntries(self, path, extra_key=None):
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
158 key = _get_transition_key(path, extra_key)
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
159 pair = self.transitions.get(key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
160 return pair
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
161
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
162 def getOverrideEntry(self, path, uri):
50
2fec3ee1298f Properly override pages between realms.
Ludovic Chabant <ludovic@chabant.com>
parents: 23
diff changeset
163 for pair in self.transitions.values():
2fec3ee1298f Properly override pages between realms.
Ludovic Chabant <ludovic@chabant.com>
parents: 23
diff changeset
164 cur = pair[1]
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
165 if cur and cur.path != path:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
166 for o in cur.subs:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
167 if o.out_uri == uri:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
168 return cur
50
2fec3ee1298f Properly override pages between realms.
Ludovic Chabant <ludovic@chabant.com>
parents: 23
diff changeset
169 return None
2fec3ee1298f Properly override pages between realms.
Ludovic Chabant <ludovic@chabant.com>
parents: 23
diff changeset
170
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
171 def getPreviousEntry(self, path, extra_key=None):
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
172 pair = self.getPreviousAndCurrentEntries(path, extra_key)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
173 if pair is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
174 return pair[0]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
175 return None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
176
711
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
177 def getCurrentEntry(self, path, extra_key=None):
ab5c6a8ae90a bake: Replace hard-coded taxonomy support with "generator" system.
Ludovic Chabant <ludovic@chabant.com>
parents: 698
diff changeset
178 pair = self.getPreviousAndCurrentEntries(path, extra_key)
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
179 if pair is not None:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
180 return pair[1]
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
181 return None
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
182
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
183 def collapseEntry(self, prev_entry):
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
184 cur_entry = copy.deepcopy(prev_entry)
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
185 cur_entry.flags = BakeRecordEntry.FLAG_NONE
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
186 for o in cur_entry.subs:
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
187 o.flags = SubPageBakeInfo.FLAG_NONE
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
188 self.addEntry(cur_entry)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
189
120
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
190 def getDeletions(self):
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
191 for prev, cur in self.transitions.values():
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
192 if prev and not cur:
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
193 for sub in prev.subs:
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
194 yield (sub.out_path, 'previous source file was removed')
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
195 elif prev and cur:
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
196 prev_out_paths = [o.out_path for o in prev.subs]
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
197 cur_out_paths = [o.out_path for o in cur.subs]
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
198 diff = set(prev_out_paths) - set(cur_out_paths)
120
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
199 for p in diff:
133845647083 Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
200 yield (p, 'source file changed outputs')
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
201
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
202 def _onNewEntryAdded(self, entry):
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 400
diff changeset
203 entry.flags |= BakeRecordEntry.FLAG_NEW
338
938be93215cb bake: Improve render context and bake record, fix incremental bake bugs.
Ludovic Chabant <ludovic@chabant.com>
parents: 334
diff changeset
204