Mercurial > piecrust2

annotate piecrust/environment.py @ 411:e7b865f8f335

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bake: Enable multiprocess baking. Baking is now done by running a worker per CPU, and sending jobs to them. This changes several things across the codebase: * Ability to not cache things related to pages other than the 'main' page (i.e. the page at the bottom of the execution stack). * Decouple the baking process from the bake records, so only the main process keeps track (and modifies) the bake record. * Remove the need for 'batch page getters' and loading a page directly from the page factories. There are various smaller changes too included here, including support for scope performance timers that are saved with the bake record and can be printed out to the console. Yes I got carried away. For testing, the in-memory 'mock' file-system doesn't work anymore, since we're spawning processes, so this is replaced by a 'tmpfs' file-system which is saved in temporary files on disk and deleted after tests have run.
author Ludovic Chabant <ludovic@chabant.com>
date Fri, 12 Jun 2015 17:09:19 -0700
parents c2ca72fb7f0b
children a1567766c83c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
1 import time
47
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
2 import json
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import logging
336
aa6b7ff6a193 internal: Use hashes for cache paths.
Ludovic Chabant <ludovic@chabant.com>
parents: 158
diff changeset
4 import hashlib
114
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
5 import contextlib
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 52
diff changeset
6 import collections
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
7 import repoze.lru
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10 logger = logging.getLogger(__name__)
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
47
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
13 def _make_fs_cache_key(key):
336
aa6b7ff6a193 internal: Use hashes for cache paths.
Ludovic Chabant <ludovic@chabant.com>
parents: 158
diff changeset
14 return hashlib.md5(key.encode('utf8')).hexdigest()
47
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
15
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
16
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
17 class MemCache(object):
47
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
18 """ Simple memory cache. It can be backed by a simple file-system
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
19 cache, but items need to be JSON-serializable to do this.
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
20 """
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
21 def __init__(self, size=2048):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
22 self.cache = repoze.lru.LRUCache(size)
47
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
23 self.fs_cache = None
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
24 self._last_access_hit = None
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
25 self._invalidated_fs_items = set()
114
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
26
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
27 @property
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
28 def last_access_hit(self):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
29 return self._last_access_hit
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
30
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
31 def invalidate(self, key):
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
32 logger.debug("Invalidating cache item '%s'." % key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
33 self.cache.invalidate(key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
34 if self.fs_cache:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
35 logger.debug("Invalidating FS cache item '%s'." % key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
36 fs_key = _make_fs_cache_key(key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
37 self._invalidated_fs_items.add(fs_key)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
38
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
39 def put(self, key, item, save_to_fs=True):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
40 self.cache.put(key, item)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
41 if self.fs_cache and save_to_fs:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
42 fs_key = _make_fs_cache_key(key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
43 item_raw = json.dumps(item)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
44 self.fs_cache.write(fs_key, item_raw)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
45
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
46 def get(self, key, item_maker, fs_cache_time=None, save_to_fs=True):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
47 self._last_access_hit = True
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
48 item = self.cache.get(key)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
49 if item is None:
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
50 if (self.fs_cache is not None and
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
51 fs_cache_time is not None):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
52 # Try first from the file-system cache.
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
53 fs_key = _make_fs_cache_key(key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
54 if (fs_key not in self._invalidated_fs_items and
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
55 self.fs_cache.isValid(fs_key, fs_cache_time)):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
56 logger.debug("'%s' found in file-system cache." %
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
57 key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
58 item_raw = self.fs_cache.read(fs_key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
59 item = json.loads(
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
60 item_raw,
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
61 object_pairs_hook=collections.OrderedDict)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
62 self.cache.put(key, item)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
63 return item
47
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
64
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
65 # Look into the mem-cache.
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
66 logger.debug("'%s' not found in cache, must build." % key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
67 item = item_maker()
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
68 self.cache.put(key, item)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
69 self._last_access_hit = False
47
558e3602be40 Cache rendered segments to disk.
Ludovic Chabant <ludovic@chabant.com>
parents: 32
diff changeset
70
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
71 # Save to the file-system if needed.
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
72 if self.fs_cache is not None and save_to_fs:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
73 item_raw = json.dumps(item)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
74 self.fs_cache.write(fs_key, item_raw)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
75
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
76 return item
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
77
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
78
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
79 class ExecutionInfo(object):
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
80 def __init__(self, page, render_ctx):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
81 self.page = page
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
82 self.render_ctx = render_ctx
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
83 self.was_cache_valid = False
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
84 self.start_time = time.clock()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
85
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
86
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
87 class ExecutionInfoStack(object):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
88 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
89 self._page_stack = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
90
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
91 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
92 def current_page_info(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
93 if len(self._page_stack) == 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
94 return None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
95 return self._page_stack[-1]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
96
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
97 @property
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
98 def is_main_page(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
99 return len(self._page_stack) == 1
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
100
12
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
101 def hasPage(self, page):
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
102 for ei in self._page_stack:
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
103 if ei.page == page:
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
104 return True
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
105 return False
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
106
96
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
107 def pushPage(self, page, render_ctx):
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
108 if len(self._page_stack) > 0:
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
109 top = self._page_stack[-1]
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
110 assert top.page is not page
0445a2232de7 Improvements and fixes to incremental baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 67
diff changeset
111 self._page_stack.append(ExecutionInfo(page, render_ctx))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
112
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
113 def popPage(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
114 del self._page_stack[-1]
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
115
114
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
116 def clear(self):
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
117 self._page_stack = []
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
118
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
120 class Environment(object):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
121 def __init__(self):
114
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
122 self.start_time = None
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
123 self.exec_info_stack = ExecutionInfoStack()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
124 self.was_cache_cleaned = False
114
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
125 self.base_asset_url_format = '%uri%'
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
126 self.page_repository = MemCache()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 1
diff changeset
127 self.rendered_segments_repository = MemCache()
111
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
128 self.fs_caches = {
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
129 'renders': self.rendered_segments_repository}
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
130 self.fs_cache_only_for_main_page = False
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
131 self._timers = {}
114
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
132
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
133 def initialize(self, app):
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
134 self.start_time = time.perf_counter()
114
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
135 self.exec_info_stack.clear()
371a6c879ab9 When possible, try and batch-load pages so we only lock once.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
136 self.was_cache_cleaned = False
32
43091c9837bf Fix problems with asset URLs.
Ludovic Chabant <ludovic@chabant.com>
parents: 12
diff changeset
137 self.base_asset_url_format = '%uri%'
371
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts.
Ludovic Chabant <ludovic@chabant.com>
parents: 336
diff changeset
138 self._onSubCacheDirChanged(app)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
139
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
140 def registerTimer(self, category):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
141 self._timers[category] = 0
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
142
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
143 @contextlib.contextmanager
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
144 def timerScope(self, category):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
145 start = time.perf_counter()
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
146 yield
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
147 self._timers[category] += time.perf_counter() - start
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
148
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
149 def stepTimer(self, category, value):
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
150 self._timers[category] += value
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 371
diff changeset
151
371
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts.
Ludovic Chabant <ludovic@chabant.com>
parents: 336
diff changeset
152 def _onSubCacheDirChanged(self, app):
111
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
153 for name, repo in self.fs_caches.items():
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
154 cache = app.cache.getCache(name)
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 96
diff changeset
155 repo.fs_cache = cache
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
156
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
157
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
158 class StandardEnvironment(Environment):
1
aaa8fb7c8918 Re-arranged modules to reduce dependencies to builtin stuff.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
159 def __init__(self):
aaa8fb7c8918 Re-arranged modules to reduce dependencies to builtin stuff.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
160 super(StandardEnvironment, self).__init__()
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
161