Mercurial > piecrust2

annotate piecrust/configuration.py @ 1097:2b2eaee96121

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
tests: Fix pipeline test after having added new processors.
author Ludovic Chabant <ludovic@chabant.com>
date Sat, 17 Feb 2018 11:51:45 -0800
parents 7e51d14097cb
children c0c00dc1eac7
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import re
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
2 import logging
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
3 import collections
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
4 import collections.abc
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import yaml
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
6 from yaml.constructor import ConstructorError
444
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
7 try:
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
8 from yaml import CSafeLoader as SafeLoader
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
9 except ImportError:
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
10 from yaml import SafeLoader
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
13 logger = logging.getLogger(__name__)
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
14
431
bdeeee777f85 internal: Floats are also allowed in configurations, duh.
Ludovic Chabant <ludovic@chabant.com>
parents: 367
diff changeset
15 default_allowed_types = (dict, list, tuple, float, int, bool, str)
367
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
16
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
17
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
18 MERGE_NEW_VALUES = 0
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
19 MERGE_OVERWRITE_VALUES = 1
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
20 MERGE_PREPEND_LISTS = 2
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
21 MERGE_APPEND_LISTS = 4
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
22 MERGE_ALL = MERGE_OVERWRITE_VALUES | MERGE_PREPEND_LISTS
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
23
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
24
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
25 class ConfigurationError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
26 pass
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
27
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
28
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
29 class Configuration(collections.abc.MutableMapping):
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30 def __init__(self, values=None, validate=True):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
31 if values is not None:
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
32 self.setAll(values, validate=validate)
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
33 else:
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
34 self._values = None
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
35
138
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
36 def __getitem__(self, key):
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
37 self._ensureLoaded()
666
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
38 try:
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
39 return get_dict_value(self._values, key)
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
40 except KeyError:
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
41 raise KeyError("No such item: %s" % key)
138
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
42
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
43 def __setitem__(self, key, value):
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44 self._ensureLoaded()
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
45 value = self._validateValue(key, value)
666
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
46 set_dict_value(self._values, key, value)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
48 def __delitem__(self, key):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
49 raise NotImplementedError()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
50
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
51 def __iter__(self):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
52 self._ensureLoaded()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
53 return iter(self._values)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
54
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
55 def __len__(self):
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
56 self._ensureLoaded()
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
57 return len(self._values)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
58
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
59 def has(self, key):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
60 return key in self
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
61
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
62 def set(self, key, value):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
63 self[key] = value
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
64
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
65 def setAll(self, values, validate=False):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
66 if validate:
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
67 values = self._validateAll(values)
440
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
68 self._values = values
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
69
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
70 def getAll(self):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
71 self._ensureLoaded()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying.
Ludovic Chabant <ludovic@chabant.com>
parents: 431
diff changeset
72 return self._values
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
74 def merge(self, other, mode=MERGE_ALL):
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
75 self._ensureLoaded()
138
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
76
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
77 if isinstance(other, dict):
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
78 other_values = other
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
79 elif isinstance(other, Configuration):
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
80 other_values = other._values
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
81 else:
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
82 raise Exception(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
83 "Unsupported value type to merge: %s" % type(other))
138
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
84
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
85 merge_dicts(self._values, other_values,
978
7e51d14097cb config: Properly pass the merge mode to the `merge_dicts` function.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
86 mode=mode,
138
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`.
Ludovic Chabant <ludovic@chabant.com>
parents: 107
diff changeset
87 validator=self._validateValue)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
88
367
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
89 def validateTypes(self, allowed_types=default_allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
90 self._validateDictTypesRecursive(self._values, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
91
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
92 def _validateDictTypesRecursive(self, d, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
93 for k, v in d.items():
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
94 if not isinstance(k, str):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
95 raise ConfigurationError("Key '%s' is not a string." % k)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
96 self._validateTypeRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
97
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
98 def _validateListTypesRecursive(self, l, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
99 for v in l:
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
100 self._validateTypeRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
101
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
102 def _validateTypeRecursive(self, v, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
103 if v is None:
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
104 return
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
105 if not isinstance(v, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
106 raise ConfigurationError(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
107 "Value '%s' is of forbidden type: %s" % (v, type(v)))
367
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
108 if isinstance(v, dict):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
109 self._validateDictTypesRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
110 elif isinstance(v, list):
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
111 self._validateListTypesRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents.
Ludovic Chabant <ludovic@chabant.com>
parents: 301
diff changeset
112
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
113 def _ensureLoaded(self):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
114 if self._values is None:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
115 self._load()
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
116
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
117 def _load(self):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
118 self._values = self._validateAll({})
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
120 def _validateAll(self, values):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
121 return values
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
122
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
123 def _validateValue(self, key_path, value):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
124 return value
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
125
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
126
666
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
127 def get_dict_value(d, key):
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
128 bits = key.split('/')
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
129 cur = d
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
130 for b in bits:
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
131 cur = cur[b]
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
132 return cur
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
133
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
134
805
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
135 def get_dict_values(*args):
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
136 for d, key in args:
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
137 try:
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
138 return get_dict_value(d, key)
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
139 except KeyError:
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
140 continue
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
141 raise KeyError()
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
142
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
143
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
144 def try_get_dict_value(d, key, *, default=None):
683
ec384174b8b2 internal: More work/fixes on how default/theme/user configs are merged.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
145 try:
ec384174b8b2 internal: More work/fixes on how default/theme/user configs are merged.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
146 return get_dict_value(d, key)
ec384174b8b2 internal: More work/fixes on how default/theme/user configs are merged.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
147 except KeyError:
ec384174b8b2 internal: More work/fixes on how default/theme/user configs are merged.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
148 return default
ec384174b8b2 internal: More work/fixes on how default/theme/user configs are merged.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
149
ec384174b8b2 internal: More work/fixes on how default/theme/user configs are merged.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
150
743
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
151 def try_get_dict_values(*args, default=None):
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
152 for d, key in args:
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
153 try:
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
154 return get_dict_value(d, key)
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
155 except KeyError:
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
156 continue
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
157 return default
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
158
7705b3d981ca templating: Make the 'categories' taxonomy use a 'pccaturl' function again.
Ludovic Chabant <ludovic@chabant.com>
parents: 683
diff changeset
159
666
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
160 def set_dict_value(d, key, value):
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
161 bits = key.split('/')
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
162 bitslen = len(bits)
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
163 cur = d
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
164 for i, b in enumerate(bits):
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
165 if i == bitslen - 1:
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
166 cur[b] = value
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
167 else:
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
168 if b not in cur:
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
169 cur[b] = {}
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
170 cur = cur[b]
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
171
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 584
diff changeset
172
805
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
173 def merge_dicts(source, merging, *args,
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
174 validator=None, mode=MERGE_ALL):
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
175 _recurse_merge_dicts(source, merging, None, validator, mode)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents: 2
diff changeset
176 for other in args:
805
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
177 _recurse_merge_dicts(source, other, None, validator, mode)
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
178 return source
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
179
2
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`.
Ludovic Chabant <ludovic@chabant.com>
parents: 0
diff changeset
180
805
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
181 def _recurse_merge_dicts(local_cur, incoming_cur, parent_path,
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
182 validator, mode):
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
183 for k, v in incoming_cur.items():
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
184 key_path = k
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
185 if parent_path is not None:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
186 key_path = parent_path + '/' + k
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
187
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
188 local_v = local_cur.get(k)
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
189 if local_v is not None:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
190 if isinstance(v, dict) and isinstance(local_v, dict):
805
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
191 _recurse_merge_dicts(local_v, v, key_path,
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
192 validator, mode)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
193 elif isinstance(v, list) and isinstance(local_v, list):
805
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
194 if mode & MERGE_PREPEND_LISTS:
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
195 local_cur[k] = v + local_v
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
196 elif mode & MERGE_APPEND_LISTS:
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
197 local_cur[k] = local_v + v
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
198 else:
805
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
199 if mode & MERGE_OVERWRITE_VALUES:
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
200 if validator is not None:
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
201 v = validator(key_path, v)
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
202 local_cur[k] = v
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
203 else:
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
204 if ((mode & (MERGE_PREPEND_LISTS | MERGE_APPEND_LISTS)) or
fd694f1297c7 config: Cleanup config loading code. Add support for a `local.yml` config.
Ludovic Chabant <ludovic@chabant.com>
parents: 743
diff changeset
205 not isinstance(v, list)):
584
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
206 if validator is not None:
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
207 v = validator(key_path, v)
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
208 local_cur[k] = v
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
209
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
210
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
211 def visit_dict(subject, visitor):
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
212 _recurse_visit_dict(subject, None, visitor)
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
213
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
214
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
215 def _recurse_visit_dict(cur, parent_path, visitor):
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
216 for k, v in cur.items():
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
217 key_path = k
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
218 if parent_path is not None:
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
219 key_path = parent_path + '/' + k
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
220
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
221 visitor(key_path, v, cur, k)
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
222 if isinstance(v, dict):
9ccc933ac2c7 internal: Refactor the app configuration class.
Ludovic Chabant <ludovic@chabant.com>
parents: 444
diff changeset
223 _recurse_visit_dict(v, key_path, visitor)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
224
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
225
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
226 header_regex = re.compile(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
227 r'(---\s*\n)(?P<header>(.*\n)*?)^(---\s*\n)', re.MULTILINE)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
228
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
229
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
230 def parse_config_header(text):
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
231 m = header_regex.match(text)
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
232 if m is not None:
5
474c9882decf Upgrade to Python 3.
Ludovic Chabant <ludovic@chabant.com>
parents: 3
diff changeset
233 header = str(m.group('header'))
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
234 config = yaml.load(header, Loader=ConfigurationLoader)
0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
235 offset = m.end()
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
236 else:
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
237 config = {}
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
238 offset = 0
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
239 return config, offset
a212a3f2e3ee Initial commit.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
240
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
241
444
1359b2b0cc73 performance: Use the fast YAML loader if available.
Ludovic Chabant <ludovic@chabant.com>
parents: 440
diff changeset
242 class ConfigurationLoader(SafeLoader):
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
243 """ A YAML loader that loads mappings into ordered dictionaries,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
244 and supports sexagesimal notations for timestamps.
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
245 """
81
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
246 def __init__(self, *args, **kwargs):
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
247 super(ConfigurationLoader, self).__init__(*args, **kwargs)
81
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
248
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
249 self.add_constructor('tag:yaml.org,2002:map',
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
250 type(self).construct_yaml_map)
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
251 self.add_constructor('tag:yaml.org,2002:omap',
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
252 type(self).construct_yaml_map)
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
253 self.add_constructor('tag:yaml.org,2002:sexagesimal',
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
254 type(self).construct_yaml_time)
81
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
255
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
256 def construct_yaml_map(self, node):
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
257 data = collections.OrderedDict()
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
258 yield data
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
259 value = self.construct_mapping(node)
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
260 data.update(value)
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML.
Ludovic Chabant <ludovic@chabant.com>
parents: 68
diff changeset
261
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
262 def construct_mapping(self, node, deep=False):
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
263 if not isinstance(node, yaml.MappingNode):
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
264 raise ConstructorError(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
265 None, None,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
266 "expected a mapping node, but found %s" % node.id,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
267 node.start_mark)
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
268 mapping = collections.OrderedDict()
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
269 for key_node, value_node in node.value:
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
270 key = self.construct_object(key_node, deep=deep)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
271 if not isinstance(key, collections.Hashable):
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
272 raise ConstructorError(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
273 "while constructing a mapping", node.start_mark,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
274 "found unhashable key", key_node.start_mark)
67
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
275 value = self.construct_object(value_node, deep=deep)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
276 mapping[key] = value
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
277 return mapping
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
278
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
279 time_regexp = re.compile(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
280 r'''^(?P<hour>[0-9][0-9]?)
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
281 :(?P<minute>[0-9][0-9])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
282 (:(?P<second>[0-9][0-9])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
283 (\.(?P<fraction>[0-9]+))?)?$''', re.X)
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
284
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
285 def construct_yaml_time(self, node):
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
286 self.construct_scalar(node)
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
287 match = self.time_regexp.match(node.value)
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
288 values = match.groupdict()
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
289 hour = int(values['hour'])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
290 minute = int(values['minute'])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
291 second = 0
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
292 if values['second']:
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
293 second = int(values['second'])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
294 usec = 0
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
295 if values['fraction']:
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
296 usec = float('0.' + values['fraction'])
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
297 return second + minute * 60 + hour * 60 * 60 + usec
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
298
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
299
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
300 ConfigurationLoader.add_implicit_resolver(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
301 'tag:yaml.org,2002:sexagesimal',
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
302 re.compile(r'''^[0-9][0-9]?:[0-9][0-9]
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
303 (:[0-9][0-9](\.[0-9]+)?)?$''', re.X),
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
304 list('0123456789'))
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
305
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
306
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
307 # We need to add our `sexagesimal` resolver before the `int` one, which
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
308 # already supports sexagesimal notation in YAML 1.1 (but not 1.2). However,
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
309 # because we know we pretty much always want it for representing time, we
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
310 # need a simple `12:30` to mean 45000, not 750. So that's why we override
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
311 # the default behaviour.
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
312 for ch in list('0123456789'):
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
313 ch_resolvers = ConfigurationLoader.yaml_implicit_resolvers[ch]
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
314 ch_resolvers.insert(0, ch_resolvers.pop())
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
315
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
316
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
317 class ConfigurationDumper(yaml.SafeDumper):
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
318 def represent_ordered_dict(self, data):
301
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents: 204
diff changeset
319 # Not a typo: we're using `map` and not `omap` because we don't want
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents: 204
diff changeset
320 # ugly type tags printed in the generated YAML markup, and because
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents: 204
diff changeset
321 # we always load maps into `OrderedDicts` anyway.
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps.
Ludovic Chabant <ludovic@chabant.com>
parents: 204
diff changeset
322 return self.represent_mapping('tag:yaml.org,2002:map', data)
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
323
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
324
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
325 ConfigurationDumper.add_representer(collections.OrderedDict,
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
326 ConfigurationDumper.represent_ordered_dict)
107
10fc9c8bf682 Better support for times in YAML interop.
Ludovic Chabant <ludovic@chabant.com>
parents: 81
diff changeset
327