piecrust2: piecrust/configuration.py annotate

annotate piecrust/configuration.py @ 549:7453baeb0839

bake: Set the flags, don't combine. We don't want to combine old flags with new ones, especially if something went different between the last bake and the current one.

author	Ludovic Chabant <ludovic@chabant.com>
date	Tue, 04 Aug 2015 21:21:08 -0700
parents	1359b2b0cc73
children	9ccc933ac2c7

rev	line source
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import re
2 40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	2 import logging
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	3 import collections.abc
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import yaml
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	5 from yaml.constructor import ConstructorError
444 1359b2b0cc73 performance: Use the fast YAML loader if available. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	6 try:
1359b2b0cc73 performance: Use the fast YAML loader if available. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	7 from yaml import CSafeLoader as SafeLoader
1359b2b0cc73 performance: Use the fast YAML loader if available. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	8 except ImportError:
1359b2b0cc73 performance: Use the fast YAML loader if available. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	9 from yaml import SafeLoader
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11
2 40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	12 logger = logging.getLogger(__name__)
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	13
431 bdeeee777f85 internal: Floats are also allowed in configurations, duh. Ludovic Chabant <ludovic@chabant.com> parents: 367 diff changeset	14 default_allowed_types = (dict, list, tuple, float, int, bool, str)
367 734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	15
2 40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	16
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	17 class ConfigurationError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	18 pass
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	19
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	20
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	21 class Configuration(collections.abc.MutableMapping):
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22 def __init__(self, values=None, validate=True):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 if values is not None:
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	24 self.setAll(values, validate=validate)
2 40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	25 else:
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	26 self._values = None
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27
138 b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	28 def __getitem__(self, key):
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	29 self._ensureLoaded()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	30 bits = key.split('/')
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	31 cur = self._values
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	32 for b in bits:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	33 try:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	34 cur = cur[b]
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	35 except KeyError:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	36 raise KeyError("No such item: %s" % key)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	37 return cur
138 b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	38
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	39 def __setitem__(self, key, value):
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	40 self._ensureLoaded()
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	41 value = self._validateValue(key, value)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	42 bits = key.split('/')
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	43 bitslen = len(bits)
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	44 cur = self._values
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	45 for i, b in enumerate(bits):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46 if i == bitslen - 1:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 cur[b] = value
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48 else:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	49 if b not in cur:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	50 cur[b] = {}
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51 cur = cur[b]
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	52
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	53 def __delitem__(self, key):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	54 raise NotImplementedError()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	55
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	56 def __iter__(self):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	57 self._ensureLoaded()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	58 return iter(self._values)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	59
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	60 def __len__(self):
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 self._ensureLoaded()
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	62 return len(self._values)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	63
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	64 def has(self, key):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	65 return key in self
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	66
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	67 def set(self, key, value):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	68 self[key] = value
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	69
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	70 def setAll(self, values, validate=False):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	71 if validate:
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	72 self._validateAll(values)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	73 self._values = values
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	74
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	75 def getAll(self):
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	76 self._ensureLoaded()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 431 diff changeset	77 return self._values
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 def merge(self, other):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 self._ensureLoaded()
138 b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	81
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	82 if isinstance(other, dict):
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	83 other_values = other
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	84 elif isinstance(other, Configuration):
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	85 other_values = other._values
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	86 else:
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	87 raise Exception(
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	88 "Unsupported value type to merge: %s" % type(other))
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	89
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	90 merge_dicts(self._values, other_values,
b540d431f2da Make configuration class more like `dict`, add support for merging `dicts`. Ludovic Chabant <ludovic@chabant.com> parents: 107 diff changeset	91 validator=self._validateValue)
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	92
367 734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	93 def validateTypes(self, allowed_types=default_allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	94 self._validateDictTypesRecursive(self._values, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	95
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	96 def _validateDictTypesRecursive(self, d, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	97 for k, v in d.items():
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	98 if not isinstance(k, str):
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	99 raise ConfigurationError("Key '%s' is not a string." % k)
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	100 self._validateTypeRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	101
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	102 def _validateListTypesRecursive(self, l, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	103 for v in l:
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	104 self._validateTypeRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	105
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	106 def _validateTypeRecursive(self, v, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	107 if v is None:
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	108 return
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	109 if not isinstance(v, allowed_types):
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	110 raise ConfigurationError(
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	111 "Value '%s' is of forbidden type: %s" % (v, type(v)))
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	112 if isinstance(v, dict):
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	113 self._validateDictTypesRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	114 elif isinstance(v, list):
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	115 self._validateListTypesRecursive(v, allowed_types)
734f2abf361c config: Add method to deep-copy a config and validate its contents. Ludovic Chabant <ludovic@chabant.com> parents: 301 diff changeset	116
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	117 def _ensureLoaded(self):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	118 if self._values is None:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	119 self._load()
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121 def _load(self):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122 self._values = self._validateAll({})
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	123
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	124 def _validateAll(self, values):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	125 return values
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	126
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	127 def _validateValue(self, key_path, value):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	128 return value
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	129
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	130
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	131 def merge_dicts(source, merging, validator=None, *args):
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	132 if validator is None:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	133 validator = lambda k, v: v
2 40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	134 _recurse_merge_dicts(source, merging, None, validator)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	135 for other in args:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: 2 diff changeset	136 _recurse_merge_dicts(source, other, None, validator)
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	137
2 40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	138
40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	139 def _recurse_merge_dicts(local_cur, incoming_cur, parent_path, validator):
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	140 for k, v in incoming_cur.items():
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	141 key_path = k
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	142 if parent_path is not None:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	143 key_path = parent_path + '/' + k
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	144
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	145 local_v = local_cur.get(k)
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	146 if local_v is not None:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	147 if isinstance(v, dict) and isinstance(local_v, dict):
2 40fa08b261b9 Added unit tests (using `py.test`) for `Configuration`. Ludovic Chabant <ludovic@chabant.com> parents: 0 diff changeset	148 _recurse_merge_dicts(local_v, v, key_path, validator)
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	149 elif isinstance(v, list) and isinstance(local_v, list):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	150 local_cur[k] = v + local_v
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	151 else:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	152 local_cur[k] = validator(key_path, v)
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	153 else:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	154 local_cur[k] = validator(key_path, v)
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	155
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	156
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	157 header_regex = re.compile(
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	158 r'(---\s\n)(?P<header>(.\n)?)^(---\s\n)', re.MULTILINE)
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	160
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	161 def parse_config_header(text):
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	162 m = header_regex.match(text)
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	163 if m is not None:
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	164 header = str(m.group('header'))
107 10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	165 config = yaml.load(header, Loader=ConfigurationLoader)
0 a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	166 offset = m.end()
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	167 else:
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	168 config = {}
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	169 offset = 0
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	170 return config, offset
a212a3f2e3ee Initial commit. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	171
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	172
444 1359b2b0cc73 performance: Use the fast YAML loader if available. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	173 class ConfigurationLoader(SafeLoader):
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	174 """ A YAML loader that loads mappings into ordered dictionaries.
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	175 """
81 d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	176 def __init__(self, args, *kwargs):
107 10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	177 super(ConfigurationLoader, self).__init__(args, *kwargs)
81 d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	178
107 10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	179 self.add_constructor('tag:yaml.org,2002:map',
81 d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	180 type(self).construct_yaml_map)
107 10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	181 self.add_constructor('tag:yaml.org,2002:omap',
81 d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	182 type(self).construct_yaml_map)
107 10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	183 self.add_constructor('tag:yaml.org,2002:sexagesimal',
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	184 type(self).construct_yaml_time)
81 d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	185
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	186 def construct_yaml_map(self, node):
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	187 data = collections.OrderedDict()
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	188 yield data
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	189 value = self.construct_mapping(node)
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	190 data.update(value)
d64e4703f5e6 Propertly create `OrderedDict`s when loading YAML. Ludovic Chabant <ludovic@chabant.com> parents: 68 diff changeset	191
67 563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	192 def construct_mapping(self, node, deep=False):
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	193 if not isinstance(node, yaml.MappingNode):
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	194 raise ConstructorError(None, None,
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	195 "expected a mapping node, but found %s" % node.id,
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	196 node.start_mark)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	197 mapping = collections.OrderedDict()
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	198 for key_node, value_node in node.value:
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	199 key = self.construct_object(key_node, deep=deep)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	200 if not isinstance(key, collections.Hashable):
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	201 raise ConstructorError("while constructing a mapping", node.start_mark,
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	202 "found unhashable key", key_node.start_mark)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	203 value = self.construct_object(value_node, deep=deep)
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	204 mapping[key] = value
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	205 return mapping
563ce5dd02af I don't care what the YAML spec says, ordered maps are the only sane way. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	206
107 10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	207 time_regexp = re.compile(
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	208 r'''^(?P<hour>[0-9][0-9]?)
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	209 :(?P<minute>[0-9][0-9])
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	210 (:(?P<second>[0-9][0-9])
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	211 (\.(?P<fraction>[0-9]+))?)?$''', re.X)
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	212
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	213 def construct_yaml_time(self, node):
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	214 self.construct_scalar(node)
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	215 match = self.time_regexp.match(node.value)
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	216 values = match.groupdict()
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	217 hour = int(values['hour'])
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	218 minute = int(values['minute'])
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	219 second = 0
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	220 if values['second']:
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	221 second = int(values['second'])
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	222 usec = 0
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	223 if values['fraction']:
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	224 usec = float('0.' + values['fraction'])
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	225 return second + minute * 60 + hour * 60 * 60 + usec
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	226
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	227
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	228 ConfigurationLoader.add_implicit_resolver(
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	229 'tag:yaml.org,2002:sexagesimal',
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	230 re.compile(r'''^[0-9][0-9]?:[0-9][0-9]
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	231 (:[0-9][0-9](\.[0-9]+)?)?$''', re.X),
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	232 list('0123456789'))
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	233
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	234
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	235 # We need to add our `sexagesimal` resolver before the `int` one, which
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	236 # already supports sexagesimal notation in YAML 1.1 (but not 1.2). However,
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	237 # because we know we pretty much always want it for representing time, we
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	238 # need a simple `12:30` to mean 45000, not 750. So that's why we override
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	239 # the default behaviour.
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	240 for ch in list('0123456789'):
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	241 ch_resolvers = ConfigurationLoader.yaml_implicit_resolvers[ch]
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	242 ch_resolvers.insert(0, ch_resolvers.pop())
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	243
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	244
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	245 class ConfigurationDumper(yaml.SafeDumper):
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	246 def represent_ordered_dict(self, data):
301 45aba3cb7228 config: Make YAML consider `omap` structures as normal maps. Ludovic Chabant <ludovic@chabant.com> parents: 204 diff changeset	247 # Not a typo: we're using `map` and not `omap` because we don't want
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps. Ludovic Chabant <ludovic@chabant.com> parents: 204 diff changeset	248 # ugly type tags printed in the generated YAML markup, and because
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps. Ludovic Chabant <ludovic@chabant.com> parents: 204 diff changeset	249 # we always load maps into `OrderedDicts` anyway.
45aba3cb7228 config: Make YAML consider `omap` structures as normal maps. Ludovic Chabant <ludovic@chabant.com> parents: 204 diff changeset	250 return self.represent_mapping('tag:yaml.org,2002:map', data)
107 10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	251
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	252
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	253 ConfigurationDumper.add_representer(collections.OrderedDict,
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	254 ConfigurationDumper.represent_ordered_dict)
10fc9c8bf682 Better support for times in YAML interop. Ludovic Chabant <ludovic@chabant.com> parents: 81 diff changeset	255

Mercurial > piecrust2

annotate piecrust/configuration.py @ 549:7453baeb0839