Mercurial > piecrust2

annotate piecrust/data/base.py @ 440:32c7c2d219d2

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
performance: Refactor how data is managed to reduce copying. * Make use of `collections.abc.Mapping` to better identify things that are supposed to look like dictionaries. * Instead of handling "overlay" of data in a dict tree in each different data object, make all objects `Mapping`s and handle merging at a higher level with the new `MergedMapping` object. * Since this new object is read-only, remove the need for deep-copying of app and page configurations. * Split data classes into separate modules.
author Ludovic Chabant <ludovic@chabant.com>
date Sun, 28 Jun 2015 08:22:39 -0700
parents 5be275137056
children d446029c9478
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
1 import collections.abc
381
4c9eab0e283b data: Fix problems with using non-existing metadata on a linked page.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
2
4c9eab0e283b data: Fix problems with using non-existing metadata on a linked page.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
3
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
4 class MergedMapping(collections.abc.Mapping):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
5 """ Provides a dictionary-like object that's really the aggregation of
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
6 multiple dictionary-like objects.
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7 """
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
8 def __init__(self, dicts, path=''):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
9 self._dicts = dicts
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
10 self._path = path
226
e9dc18a275ff data: Add ability for `IPaginationSource`s to specify how to get settings.
Ludovic Chabant <ludovic@chabant.com>
parents: 158
diff changeset
11
12
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 6
diff changeset
12 def __getattr__(self, name):
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 6
diff changeset
13 try:
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
14 return self[name]
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
15 except KeyError:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
16 raise AttributeError("No such attribute: %s" % self._subp(name))
12
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 6
diff changeset
17
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18 def __getitem__(self, name):
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
19 values = []
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
20 for d in self._dicts:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
21 try:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
22 val = d[name]
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
23 except KeyError:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
24 continue
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
25 values.append(val)
236
eaf18442bff8 internal: Add support for "wildcard" loader in `LazyPageConfigData`.
Ludovic Chabant <ludovic@chabant.com>
parents: 228
diff changeset
26
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
27 if len(values) == 0:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
28 raise KeyError("No such item: %s" % self._subp(name))
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
29 if len(values) == 1:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
30 return values[0]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
31
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
32 for val in values:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
33 if not isinstance(val, (dict, collections.abc.Mapping)):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
34 raise Exception(
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
35 "Template data for '%s' contains an incompatible mix "
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
36 "of data: %s" % (
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
37 self._subp(name),
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
38 ', '.join([str(type(v)) for v in values])))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
39
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
40 return MergedMapping(values, self._subp(name))
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
41
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
42 def __iter__(self):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
43 keys = set()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
44 for d in self._dicts:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
45 keys |= set(d.keys())
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
46 return iter(keys)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
48 def __len__(self):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
49 keys = set()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
50 for d in self._dicts:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
51 keys |= set(d.keys())
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
52 return len(keys)
12
30a42341cfa8 Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents: 6
diff changeset
53
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
54 def _subp(self, name):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
55 return '%s/%s' % (self._path, name)
6
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
56
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
57 def _prependMapping(self, d):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
58 self._dicts.insert(0, d)
6
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
59
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
60 def _appendMapping(self, d):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 433
diff changeset
61 self._dicts.append(d)
6
f5ca5c5bed85 More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
62