Mercurial > piecrust2
annotate piecrust/configuration.py @ 680:c2ea75e37540
serve: Fix some crashes introduced by recent refactor.
| author | Ludovic Chabant <ludovic@chabant.com> |
|---|---|
| date | Tue, 08 Mar 2016 01:05:39 -0800 |
| parents | 81d9c3a3a0b5 |
| children | ec384174b8b2 |
| rev | line source |
|---|---|
| 0 | 1 import re |
|
2
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
2 import logging |
|
584
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
3 import collections |
|
440
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
4 import collections.abc |
| 0 | 5 import yaml |
|
67
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
6 from yaml.constructor import ConstructorError |
|
444
1359b2b0cc73
performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents:
440
diff
changeset
|
7 try: |
|
1359b2b0cc73
performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents:
440
diff
changeset
|
8 from yaml import CSafeLoader as SafeLoader |
|
1359b2b0cc73
performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents:
440
diff
changeset
|
9 except ImportError: |
|
1359b2b0cc73
performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents:
440
diff
changeset
|
10 from yaml import SafeLoader |
| 0 | 11 |
| 12 | |
|
2
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
13 logger = logging.getLogger(__name__) |
|
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
14 |
|
431
bdeeee777f85
internal: Floats are also allowed in configurations, duh.
Ludovic Chabant <ludovic@chabant.com>
parents:
367
diff
changeset
|
15 default_allowed_types = (dict, list, tuple, float, int, bool, str) |
|
367
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
16 |
|
2
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
17 |
|
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
2
diff
changeset
|
18 class ConfigurationError(Exception): |
|
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
2
diff
changeset
|
19 pass |
|
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
2
diff
changeset
|
20 |
|
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
2
diff
changeset
|
21 |
|
440
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
22 class Configuration(collections.abc.MutableMapping): |
| 0 | 23 def __init__(self, values=None, validate=True): |
| 24 if values is not None: | |
|
440
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
25 self.setAll(values, validate=validate) |
|
2
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
26 else: |
|
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
27 self._values = None |
| 0 | 28 |
|
138
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
29 def __getitem__(self, key): |
|
440
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
30 self._ensureLoaded() |
|
666
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
31 try: |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
32 return get_dict_value(self._values, key) |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
33 except KeyError: |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
34 raise KeyError("No such item: %s" % key) |
|
138
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
35 |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
36 def __setitem__(self, key, value): |
| 0 | 37 self._ensureLoaded() |
|
440
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
38 value = self._validateValue(key, value) |
|
666
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
39 set_dict_value(self._values, key, value) |
| 0 | 40 |
|
440
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
41 def __delitem__(self, key): |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
42 raise NotImplementedError() |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
43 |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
44 def __iter__(self): |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
45 self._ensureLoaded() |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
46 return iter(self._values) |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
47 |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
48 def __len__(self): |
| 0 | 49 self._ensureLoaded() |
|
440
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
50 return len(self._values) |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
51 |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
52 def has(self, key): |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
53 return key in self |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
54 |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
55 def set(self, key, value): |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
56 self[key] = value |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
57 |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
58 def setAll(self, values, validate=False): |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
59 if validate: |
|
584
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
60 values = self._validateAll(values) |
|
440
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
61 self._values = values |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
62 |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
63 def getAll(self): |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
64 self._ensureLoaded() |
|
32c7c2d219d2
performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents:
431
diff
changeset
|
65 return self._values |
| 0 | 66 |
| 67 def merge(self, other): | |
| 68 self._ensureLoaded() | |
|
138
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
69 |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
70 if isinstance(other, dict): |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
71 other_values = other |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
72 elif isinstance(other, Configuration): |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
73 other_values = other._values |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
74 else: |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
75 raise Exception( |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
76 "Unsupported value type to merge: %s" % type(other)) |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
77 |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
78 merge_dicts(self._values, other_values, |
|
b540d431f2da
Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents:
107
diff
changeset
|
79 validator=self._validateValue) |
| 0 | 80 |
|
367
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
81 def validateTypes(self, allowed_types=default_allowed_types): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
82 self._validateDictTypesRecursive(self._values, allowed_types) |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
83 |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
84 def _validateDictTypesRecursive(self, d, allowed_types): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
85 for k, v in d.items(): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
86 if not isinstance(k, str): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
87 raise ConfigurationError("Key '%s' is not a string." % k) |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
88 self._validateTypeRecursive(v, allowed_types) |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
89 |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
90 def _validateListTypesRecursive(self, l, allowed_types): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
91 for v in l: |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
92 self._validateTypeRecursive(v, allowed_types) |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
93 |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
94 def _validateTypeRecursive(self, v, allowed_types): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
95 if v is None: |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
96 return |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
97 if not isinstance(v, allowed_types): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
98 raise ConfigurationError( |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
99 "Value '%s' is of forbidden type: %s" % (v, type(v))) |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
100 if isinstance(v, dict): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
101 self._validateDictTypesRecursive(v, allowed_types) |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
102 elif isinstance(v, list): |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
103 self._validateListTypesRecursive(v, allowed_types) |
|
734f2abf361c
config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents:
301
diff
changeset
|
104 |
| 0 | 105 def _ensureLoaded(self): |
| 106 if self._values is None: | |
| 107 self._load() | |
| 108 | |
| 109 def _load(self): | |
| 110 self._values = self._validateAll({}) | |
| 111 | |
| 112 def _validateAll(self, values): | |
| 113 return values | |
| 114 | |
| 115 def _validateValue(self, key_path, value): | |
| 116 return value | |
| 117 | |
| 118 | |
|
666
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
119 def get_dict_value(d, key): |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
120 bits = key.split('/') |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
121 cur = d |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
122 for b in bits: |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
123 cur = cur[b] |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
124 return cur |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
125 |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
126 |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
127 def set_dict_value(d, key, value): |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
128 bits = key.split('/') |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
129 bitslen = len(bits) |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
130 cur = d |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
131 for i, b in enumerate(bits): |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
132 if i == bitslen - 1: |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
133 cur[b] = value |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
134 else: |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
135 if b not in cur: |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
136 cur[b] = {} |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
137 cur = cur[b] |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
138 |
|
81d9c3a3a0b5
internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents:
584
diff
changeset
|
139 |
|
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
2
diff
changeset
|
140 def merge_dicts(source, merging, validator=None, *args): |
|
2
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
141 _recurse_merge_dicts(source, merging, None, validator) |
|
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
2
diff
changeset
|
142 for other in args: |
|
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
2
diff
changeset
|
143 _recurse_merge_dicts(source, other, None, validator) |
|
584
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
144 return source |
| 0 | 145 |
|
2
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
146 |
|
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
147 def _recurse_merge_dicts(local_cur, incoming_cur, parent_path, validator): |
| 5 | 148 for k, v in incoming_cur.items(): |
| 0 | 149 key_path = k |
| 150 if parent_path is not None: | |
| 151 key_path = parent_path + '/' + k | |
| 152 | |
| 153 local_v = local_cur.get(k) | |
| 154 if local_v is not None: | |
| 155 if isinstance(v, dict) and isinstance(local_v, dict): | |
|
2
40fa08b261b9
Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents:
0
diff
changeset
|
156 _recurse_merge_dicts(local_v, v, key_path, validator) |
| 0 | 157 elif isinstance(v, list) and isinstance(local_v, list): |
| 158 local_cur[k] = v + local_v | |
| 159 else: | |
|
584
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
160 if validator is not None: |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
161 v = validator(key_path, v) |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
162 local_cur[k] = v |
| 0 | 163 else: |
|
584
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
164 if validator is not None: |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
165 v = validator(key_path, v) |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
166 local_cur[k] = v |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
167 |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
168 |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
169 def visit_dict(subject, visitor): |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
170 _recurse_visit_dict(subject, None, visitor) |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
171 |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
172 |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
173 def _recurse_visit_dict(cur, parent_path, visitor): |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
174 for k, v in cur.items(): |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
175 key_path = k |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
176 if parent_path is not None: |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
177 key_path = parent_path + '/' + k |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
178 |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
179 visitor(key_path, v, cur, k) |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
180 if isinstance(v, dict): |
|
9ccc933ac2c7
internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents:
444
diff
changeset
|
181 _recurse_visit_dict(v, key_path, visitor) |
| 0 | 182 |
| 183 | |
| 184 header_regex = re.compile( | |
| 185 r'(---\s*\n)(?P<header>(.*\n)*?)^(---\s*\n)', re.MULTILINE) | |
| 186 | |
| 187 | |
| 188 def parse_config_header(text): | |
| 189 m = header_regex.match(text) | |
| 190 if m is not None: | |
| 5 | 191 header = str(m.group('header')) |
|
107
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
192 config = yaml.load(header, Loader=ConfigurationLoader) |
| 0 | 193 offset = m.end() |
| 194 else: | |
| 195 config = {} | |
| 196 offset = 0 | |
| 197 return config, offset | |
| 198 | |
|
67
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
199 |
|
444
1359b2b0cc73
performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents:
440
diff
changeset
|
200 class ConfigurationLoader(SafeLoader): |
|
67
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
201 """ A YAML loader that loads mappings into ordered dictionaries. |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
202 """ |
|
81
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
203 def __init__(self, *args, **kwargs): |
|
107
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
204 super(ConfigurationLoader, self).__init__(*args, **kwargs) |
|
81
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
205 |
|
107
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
206 self.add_constructor('tag:yaml.org,2002:map', |
|
81
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
207 type(self).construct_yaml_map) |
|
107
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
208 self.add_constructor('tag:yaml.org,2002:omap', |
|
81
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
209 type(self).construct_yaml_map) |
|
107
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
210 self.add_constructor('tag:yaml.org,2002:sexagesimal', |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
211 type(self).construct_yaml_time) |
|
81
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
212 |
|
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
213 def construct_yaml_map(self, node): |
|
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
214 data = collections.OrderedDict() |
|
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
215 yield data |
|
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
216 value = self.construct_mapping(node) |
|
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
217 data.update(value) |
|
d64e4703f5e6
Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents:
68
diff
changeset
|
218 |
|
67
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
219 def construct_mapping(self, node, deep=False): |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
220 if not isinstance(node, yaml.MappingNode): |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
221 raise ConstructorError(None, None, |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
222 "expected a mapping node, but found %s" % node.id, |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
223 node.start_mark) |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
224 mapping = collections.OrderedDict() |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
225 for key_node, value_node in node.value: |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
226 key = self.construct_object(key_node, deep=deep) |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
227 if not isinstance(key, collections.Hashable): |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
228 raise ConstructorError("while constructing a mapping", node.start_mark, |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
229 "found unhashable key", key_node.start_mark) |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
230 value = self.construct_object(value_node, deep=deep) |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
231 mapping[key] = value |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
232 return mapping |
|
563ce5dd02af
I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
233 |
|
107
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
234 time_regexp = re.compile( |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
235 r'''^(?P<hour>[0-9][0-9]?) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
236 :(?P<minute>[0-9][0-9]) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
237 (:(?P<second>[0-9][0-9]) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
238 (\.(?P<fraction>[0-9]+))?)?$''', re.X) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
239 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
240 def construct_yaml_time(self, node): |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
241 self.construct_scalar(node) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
242 match = self.time_regexp.match(node.value) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
243 values = match.groupdict() |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
244 hour = int(values['hour']) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
245 minute = int(values['minute']) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
246 second = 0 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
247 if values['second']: |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
248 second = int(values['second']) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
249 usec = 0 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
250 if values['fraction']: |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
251 usec = float('0.' + values['fraction']) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
252 return second + minute * 60 + hour * 60 * 60 + usec |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
253 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
254 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
255 ConfigurationLoader.add_implicit_resolver( |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
256 'tag:yaml.org,2002:sexagesimal', |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
257 re.compile(r'''^[0-9][0-9]?:[0-9][0-9] |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
258 (:[0-9][0-9](\.[0-9]+)?)?$''', re.X), |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
259 list('0123456789')) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
260 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
261 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
262 # We need to add our `sexagesimal` resolver before the `int` one, which |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
263 # already supports sexagesimal notation in YAML 1.1 (but not 1.2). However, |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
264 # because we know we pretty much always want it for representing time, we |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
265 # need a simple `12:30` to mean 45000, not 750. So that's why we override |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
266 # the default behaviour. |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
267 for ch in list('0123456789'): |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
268 ch_resolvers = ConfigurationLoader.yaml_implicit_resolvers[ch] |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
269 ch_resolvers.insert(0, ch_resolvers.pop()) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
270 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
271 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
272 class ConfigurationDumper(yaml.SafeDumper): |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
273 def represent_ordered_dict(self, data): |
|
301
45aba3cb7228
config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents:
204
diff
changeset
|
274 # Not a typo: we're using `map` and not `omap` because we don't want |
|
45aba3cb7228
config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents:
204
diff
changeset
|
275 # ugly type tags printed in the generated YAML markup, and because |
|
45aba3cb7228
config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents:
204
diff
changeset
|
276 # we always load maps into `OrderedDicts` anyway. |
|
45aba3cb7228
config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents:
204
diff
changeset
|
277 return self.represent_mapping('tag:yaml.org,2002:map', data) |
|
107
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
278 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
279 |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
280 ConfigurationDumper.add_representer(collections.OrderedDict, |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
281 ConfigurationDumper.represent_ordered_dict) |
|
10fc9c8bf682
Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents:
81
diff
changeset
|
282 |
