Mercurial > piecrust2

annotate piecrust/configuration.py @ 661:2f780b191541

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
internal: Fix a bug with registering taxonomy terms that are not strings. Some objects, like the blog data provider's taxnonomy entries, can render as strings, but are objects themselves. When registering them as "used terms", we need to use their string representation.
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 01 Mar 2016 22:26:09 -0800
parents 9ccc933ac2c7
children 81d9c3a3a0b5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import re
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
2 import logging
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
3 import collections
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
4 import collections.abc
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import yaml
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
6 from yaml.constructor import ConstructorError
444
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
7 try:
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
8 from yaml import CSafeLoader as SafeLoader
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
9 except ImportError:
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
10 from yaml import SafeLoader
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
13 logger = logging.getLogger(__name__)
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
14
431
bdeeee777f85 internal: Floats are also allowed in configurations, duh.
Ludovic Chabant <ludovic@chabant.com>
parents: 367
diff changeset
15 default_allowed_types = (dict, list, tuple, float, int, bool, str)
367
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
16
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
17
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
18 class ConfigurationError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
19 pass
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
20
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
21
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
22 class Configuration(collections.abc.MutableMapping):
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 def __init__(self, values=None, validate=True):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 if values is not None:
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
25 self.setAll(values, validate=validate)
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
26 else:
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
27 self._values = None
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28
138
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
29 def __getitem__(self, key):
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
30 self._ensureLoaded()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
31 bits = key.split('/')
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
32 cur = self._values
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
33 for b in bits:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
34 try:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
35 cur = cur[b]
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
36 except KeyError:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
37 raise KeyError("No such item: %s" % key)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
38 return cur
138
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
39
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
40 def __setitem__(self, key, value):
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
41 self._ensureLoaded()
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
42 value = self._validateValue(key, value)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
43 bits = key.split('/')
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44 bitslen = len(bits)
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
45 cur = self._values
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
46 for i, b in enumerate(bits):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47 if i == bitslen - 1:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
48 cur[b] = value
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
49 else:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
50 if b not in cur:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
51 cur[b] = {}
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
52 cur = cur[b]
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
53
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
54 def __delitem__(self, key):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
55 raise NotImplementedError()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
56
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
57 def __iter__(self):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
58 self._ensureLoaded()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
59 return iter(self._values)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
60
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
61 def __len__(self):
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
62 self._ensureLoaded()
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
63 return len(self._values)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
64
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
65 def has(self, key):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
66 return key in self
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
67
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
68 def set(self, key, value):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
69 self[key] = value
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
70
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
71 def setAll(self, values, validate=False):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
72 if validate:
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
73 values = self._validateAll(values)
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
74 self._values = values
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
75
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
76 def getAll(self):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
77 self._ensureLoaded()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
78 return self._values
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
79
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80 def merge(self, other):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81 self._ensureLoaded()
138
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
82
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
83 if isinstance(other, dict):
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
84 other_values = other
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
85 elif isinstance(other, Configuration):
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
86 other_values = other._values
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
87 else:
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
88 raise Exception(
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
89 "Unsupported value type to merge: %s" % type(other))
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
90
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
91 merge_dicts(self._values, other_values,
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
92 validator=self._validateValue)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
93
367
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
94 def validateTypes(self, allowed_types=default_allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
95 self._validateDictTypesRecursive(self._values, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
96
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
97 def _validateDictTypesRecursive(self, d, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
98 for k, v in d.items():
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
99 if not isinstance(k, str):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
100 raise ConfigurationError("Key '%s' is not a string." % k)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
101 self._validateTypeRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
102
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
103 def _validateListTypesRecursive(self, l, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
104 for v in l:
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
105 self._validateTypeRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
106
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
107 def _validateTypeRecursive(self, v, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
108 if v is None:
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
109 return
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
110 if not isinstance(v, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
111 raise ConfigurationError(
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
112 "Value '%s' is of forbidden type: %s" % (v, type(v)))
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
113 if isinstance(v, dict):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
114 self._validateDictTypesRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
115 elif isinstance(v, list):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
116 self._validateListTypesRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
117
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
118 def _ensureLoaded(self):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119 if self._values is None:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
120 self._load()
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
121
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
122 def _load(self):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
123 self._values = self._validateAll({})
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
124
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
125 def _validateAll(self, values):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
126 return values
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
127
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
128 def _validateValue(self, key_path, value):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
129 return value
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
130
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
131
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
132 def merge_dicts(source, merging, validator=None, *args):
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
133 _recurse_merge_dicts(source, merging, None, validator)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
134 for other in args:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
135 _recurse_merge_dicts(source, other, None, validator)
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
136 return source
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
137
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
138
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
139 def _recurse_merge_dicts(local_cur, incoming_cur, parent_path, validator):
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
140 for k, v in incoming_cur.items():
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
141 key_path = k
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
142 if parent_path is not None:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
143 key_path = parent_path + '/' + k
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
144
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
145 local_v = local_cur.get(k)
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
146 if local_v is not None:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
147 if isinstance(v, dict) and isinstance(local_v, dict):
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
148 _recurse_merge_dicts(local_v, v, key_path, validator)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
149 elif isinstance(v, list) and isinstance(local_v, list):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
150 local_cur[k] = v + local_v
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
151 else:
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
152 if validator is not None:
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
153 v = validator(key_path, v)
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
154 local_cur[k] = v
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
155 else:
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
156 if validator is not None:
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
157 v = validator(key_path, v)
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
158 local_cur[k] = v
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
159
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
160
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
161 def visit_dict(subject, visitor):
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
162 _recurse_visit_dict(subject, None, visitor)
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
163
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
164
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
165 def _recurse_visit_dict(cur, parent_path, visitor):
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
166 for k, v in cur.items():
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
167 key_path = k
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
168 if parent_path is not None:
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
169 key_path = parent_path + '/' + k
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
170
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
171 visitor(key_path, v, cur, k)
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
172 if isinstance(v, dict):
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
173 _recurse_visit_dict(v, key_path, visitor)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
174
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
175
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
176 header_regex = re.compile(
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
177 r'(---\s*\n)(?P<header>(.*\n)*?)^(---\s*\n)', re.MULTILINE)
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
178
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
179
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
180 def parse_config_header(text):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
181 m = header_regex.match(text)
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
182 if m is not None:
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
183 header = str(m.group('header'))
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
184 config = yaml.load(header, Loader=ConfigurationLoader)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
185 offset = m.end()
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
186 else:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
187 config = {}
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
188 offset = 0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
189 return config, offset
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
190
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
191
444
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
192 class ConfigurationLoader(SafeLoader):
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
193 """ A YAML loader that loads mappings into ordered dictionaries.
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
194 """
81
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
195 def __init__(self, *args, **kwargs):
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
196 super(ConfigurationLoader, self).__init__(*args, **kwargs)
81
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
197
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
198 self.add_constructor('tag:yaml.org,2002:map',
81
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
199 type(self).construct_yaml_map)
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
200 self.add_constructor('tag:yaml.org,2002:omap',
81
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
201 type(self).construct_yaml_map)
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
202 self.add_constructor('tag:yaml.org,2002:sexagesimal',
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
203 type(self).construct_yaml_time)
81
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
204
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
205 def construct_yaml_map(self, node):
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
206 data = collections.OrderedDict()
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
207 yield data
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
208 value = self.construct_mapping(node)
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
209 data.update(value)
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
210
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
211 def construct_mapping(self, node, deep=False):
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
212 if not isinstance(node, yaml.MappingNode):
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
213 raise ConstructorError(None, None,
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
214 "expected a mapping node, but found %s" % node.id,
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
215 node.start_mark)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
216 mapping = collections.OrderedDict()
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
217 for key_node, value_node in node.value:
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
218 key = self.construct_object(key_node, deep=deep)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
219 if not isinstance(key, collections.Hashable):
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
220 raise ConstructorError("while constructing a mapping", node.start_mark,
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
221 "found unhashable key", key_node.start_mark)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
222 value = self.construct_object(value_node, deep=deep)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
223 mapping[key] = value
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
224 return mapping
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
225
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
226 time_regexp = re.compile(
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
227 r'''^(?P<hour>[0-9][0-9]?)
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
228 :(?P<minute>[0-9][0-9])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
229 (:(?P<second>[0-9][0-9])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
230 (\.(?P<fraction>[0-9]+))?)?$''', re.X)
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
231
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
232 def construct_yaml_time(self, node):
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
233 self.construct_scalar(node)
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
234 match = self.time_regexp.match(node.value)
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
235 values = match.groupdict()
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
236 hour = int(values['hour'])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
237 minute = int(values['minute'])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
238 second = 0
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
239 if values['second']:
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
240 second = int(values['second'])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
241 usec = 0
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
242 if values['fraction']:
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
243 usec = float('0.' + values['fraction'])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
244 return second + minute * 60 + hour * 60 * 60 + usec
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
245
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
246
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
247 ConfigurationLoader.add_implicit_resolver(
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
248 'tag:yaml.org,2002:sexagesimal',
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
249 re.compile(r'''^[0-9][0-9]?:[0-9][0-9]
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
250 (:[0-9][0-9](\.[0-9]+)?)?$''', re.X),
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
251 list('0123456789'))
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
252
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
253
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
254 # We need to add our `sexagesimal` resolver before the `int` one, which
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
255 # already supports sexagesimal notation in YAML 1.1 (but not 1.2). However,
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
256 # because we know we pretty much always want it for representing time, we
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
257 # need a simple `12:30` to mean 45000, not 750. So that's why we override
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
258 # the default behaviour.
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
259 for ch in list('0123456789'):
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
260 ch_resolvers = ConfigurationLoader.yaml_implicit_resolvers[ch]
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
261 ch_resolvers.insert(0, ch_resolvers.pop())
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
262
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
263
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
264 class ConfigurationDumper(yaml.SafeDumper):
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
265 def represent_ordered_dict(self, data):
301
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents: 204
diff changeset
266 # Not a typo: we're using `map` and not `omap` because we don't want
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents: 204
diff changeset
267 # ugly type tags printed in the generated YAML markup, and because
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents: 204
diff changeset
268 # we always load maps into `OrderedDicts` anyway.
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents: 204
diff changeset
269 return self.represent_mapping('tag:yaml.org,2002:map', data)
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
270
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
271
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
272 ConfigurationDumper.add_representer(collections.OrderedDict,
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
273 ConfigurationDumper.represent_ordered_dict)
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
274