Mercurial > piecrust2

annotate piecrust/fastpickle.py @ 1188:a7c43131d871

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bake: Fix file write flushing problem with Python 3.8+ Writing the cache files fails in Python 3.8 because it looks like flushing behaviour has changed. We need to explicitly flush. And even then, in very rare occurrences, it looks like it can still run into racing conditions, so we do a very hacky and ugly "retry" loop when fetching cached data :(
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 15 Jun 2021 22:36:23 -0700
parents 8adc27285d93
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
989
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
1 import io
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import sys
697
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
3 import codecs
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import datetime
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import collections
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7
989
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
8 use_msgpack = False
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
9 use_marshall = False
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
989
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
12 if use_msgpack:
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
13 import msgpack
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
14
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
15 def _dumps_msgpack(obj, buf):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
16 msgpack.pack(obj, buf)
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
17
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
18 def _loads_msgpack(buf, bufsize):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
19 return msgpack.unpack(buf)
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
20
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
21 _dumps = _dumps_msgpack
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
22 _loads = _loads_msgpack
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
23
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
24 elif use_marshall:
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
25 import marshal
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
26
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
27 def _dumps_marshal(obj, buf):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
28 marshal.dump(obj, buf)
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
29
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
30 def _loads_marshal(buf, bufsize):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
31 return marshal.load(buf)
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
32
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
33 _dumps = _dumps_marshal
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
34 _loads = _loads_marshal
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
35
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
36 else:
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
37 import json
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
38
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
39 class _BufferWrapper:
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
40 def __init__(self, buf):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
41 self._buf = buf
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
42
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
43 def write(self, data):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
44 self._buf.write(data.encode('utf8'))
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
45
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
46 def read(self):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
47 return self._buf.read().decode('utf8')
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
48
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
49 def _dumps_json(obj, buf):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
50 buf = _BufferWrapper(buf)
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
51 json.dump(obj, buf, indent=None, separators=(',', ':'))
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
52
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
53 def _loads_json(buf, bufsize):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
54 buf = _BufferWrapper(buf)
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
55 return json.load(buf)
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
56
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
57 _dumps = _dumps_json
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
58 _loads = _loads_json
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
59
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
60
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
61 def pickle(obj):
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
62 with io.BytesIO() as buf:
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
63 pickle_intob(obj, buf)
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
64 return buf.getvalue()
717
fcfbe103cfd1 internal: Fix some bugs with the `fastpickle` module.
Ludovic Chabant <ludovic@chabant.com>
parents: 700
diff changeset
65
fcfbe103cfd1 internal: Fix some bugs with the `fastpickle` module.
Ludovic Chabant <ludovic@chabant.com>
parents: 700
diff changeset
66
697
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
67 def pickle_intob(obj, buf):
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
68 data = _pickle_object(obj)
989
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
69 _dumps(data, buf)
697
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
70
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
71
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
72 def unpickle(data):
989
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
73 with io.BytesIO(data) as buf:
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
74 data = _loads(buf, len(data))
697
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
75 return _unpickle_object(data)
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
76
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
77
9e5393fcfab2 bake: Re-enable faster serialization between processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 461
diff changeset
78 def unpickle_fromb(buf, bufsize):
989
8adc27285d93 bake: Big pass on bake performance.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
79 data = _loads(buf, bufsize)
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80 return _unpickle_object(data)
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
83 _PICKLING = 0
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
84 _UNPICKLING = 1
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
85
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
86 _identity_dispatch = object()
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
87
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
88
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
89 def _tuple_convert(obj, func, op):
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
90 if op == _PICKLING:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
91 return ['__type__:tuple'] + [func(c) for c in obj]
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
92 elif op == _UNPICKLING:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
93 return tuple([func(c) for c in obj[1:]])
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
94
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
95
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
96 def _list_convert(obj, func, op):
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
97 return [func(c) for c in obj]
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
98
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
99
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
100 def _dict_convert(obj, func, op):
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
101 res = {}
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
102 for k, v in obj.items():
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
103 res[k] = func(v)
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
104 return res
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
105
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
106
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
107 def _ordered_dict_convert(obj, func, op):
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
108 if op == _PICKLING:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
109 res = {'__type__': 'OrderedDict'}
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
110 for k, v in obj.items():
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
111 res[k] = func(v)
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
112 return res
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
113 elif op == _UNPICKLING:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
114 res = collections.OrderedDict()
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
115 for k, v in obj.items():
717
fcfbe103cfd1 internal: Fix some bugs with the `fastpickle` module.
Ludovic Chabant <ludovic@chabant.com>
parents: 700
diff changeset
116 if k != '__type__':
fcfbe103cfd1 internal: Fix some bugs with the `fastpickle` module.
Ludovic Chabant <ludovic@chabant.com>
parents: 700
diff changeset
117 res[k] = func(v)
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
118 return res
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
120
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
121 def _set_convert(obj, func, op):
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
122 if op == _PICKLING:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
123 return ['__type__:set'] + [func(c) for c in obj]
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
124 elif op == _UNPICKLING:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
125 return set([func(c) for c in obj[1:]])
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
126
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
127
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
128 def _date_convert(obj, func, op):
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
129 if op == _PICKLING:
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
130 return {'__class__': 'date',
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
131 'year': obj.year,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
132 'month': obj.month,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
133 'day': obj.day}
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
134 elif op == _UNPICKLING:
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
135 return datetime.date(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
136 obj['year'], obj['month'], obj['day'])
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
137
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
138
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
139 def _datetime_convert(obj, func, op):
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
140 if op == _PICKLING:
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
141 return {'__class__': 'datetime',
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
142 'year': obj.year,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
143 'month': obj.month,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
144 'day': obj.day,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
145 'hour': obj.hour,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
146 'minute': obj.minute,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
147 'second': obj.second,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
148 'microsecond': obj.microsecond}
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
149 elif op == _UNPICKLING:
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
150 return datetime.datetime(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
151 obj['year'], obj['month'], obj['day'],
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
152 obj['hour'], obj['minute'], obj['second'], obj['microsecond'])
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
153
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
154
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
155 def _time_convert(obj, func, op):
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
156 if op == _PICKLING:
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
157 return {'__class__': 'time',
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
158 'hour': obj.hour,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
159 'minute': obj.minute,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
160 'second': obj.second,
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
161 'microsecond': obj.microsecond}
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
162 elif op == _UNPICKLING:
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
163 return datetime.time(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
164 obj['hour'], obj['minute'], obj['second'], obj['microsecond'])
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
165
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
166
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
167 _type_convert = {
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
168 type(None): _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
169 bool: _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
170 int: _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
171 float: _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
172 str: _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
173 datetime.date: _date_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
174 datetime.datetime: _datetime_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
175 datetime.time: _time_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
176 tuple: _tuple_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
177 list: _list_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
178 dict: _dict_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
179 set: _set_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
180 collections.OrderedDict: _ordered_dict_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
181 }
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
182
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
183
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
184 _type_unconvert = {
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
185 type(None): _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
186 bool: _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
187 int: _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
188 float: _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
189 str: _identity_dispatch,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
190 'date': _date_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
191 'datetime': _datetime_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
192 'time': _time_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
193 }
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
194
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
195
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
196 _collection_unconvert = {
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
197 '__type__:tuple': _tuple_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
198 '__type__:set': _set_convert,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
199 }
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
200
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
201
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
202 _mapping_unconvert = {
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
203 'OrderedDict': _ordered_dict_convert
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 717
diff changeset
204 }
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
205
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
206
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
207 def _pickle_object(obj):
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
208 t = type(obj)
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
209 conv = _type_convert.get(t)
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
210
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
211 # Object doesn't need conversion?
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
212 if conv is _identity_dispatch:
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
213 return obj
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
214
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
215 # Object has special conversion?
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
216 if conv is not None:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
217 return conv(obj, _pickle_object, _PICKLING)
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
218
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
219 # Use instance dictionary, or a custom state.
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
220 getter = getattr(obj, '__getstate__', None)
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
221 if getter is not None:
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
222 state = getter()
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
223 else:
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
224 state = obj.__dict__
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
225
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
226 state = _dict_convert(state, _pickle_object, _PICKLING)
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
227 state['__class__'] = obj.__class__.__name__
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
228 state['__module__'] = obj.__class__.__module__
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
229
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
230 return state
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
231
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
232
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
233 def _unpickle_object(state):
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
234 t = type(state)
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
235 conv = _type_unconvert.get(t)
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
236
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
237 # Object doesn't need conversion?
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
238 if conv is _identity_dispatch:
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
239 return state
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
240
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
241 # Try collection or mapping conversion.
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
242 if t is list:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
243 try:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
244 col_type = state[0]
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
245 if not isinstance(col_type, str):
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
246 col_type = None
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
247 except IndexError:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
248 col_type = None
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
249 if col_type is not None:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
250 conv = _collection_unconvert.get(col_type)
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
251 if conv is not None:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
252 return conv(state, _unpickle_object, _UNPICKLING)
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
253 return _list_convert(state, _unpickle_object, _UNPICKLING)
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
254
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
255 assert t is dict
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
256
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
257 # Custom mapping type?
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
258 map_type = state.get('__type__')
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
259 if map_type:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
260 conv = _mapping_unconvert.get(map_type)
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
261 return conv(state, _unpickle_object, _UNPICKLING)
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
262
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
263 # Class instance or other custom type.
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
264 class_name = state.get('__class__')
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
265 if class_name is None:
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
266 return _dict_convert(state, _unpickle_object, _UNPICKLING)
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
267
459
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
268 conv = _type_unconvert.get(class_name)
2ef04e16f0b9 internal: Add support for fake pickling of date/time structures.
Ludovic Chabant <ludovic@chabant.com>
parents: 450
diff changeset
269 if conv is not None:
461
b015e38d4ee1 internal: Handle data serialization more under the hood.
Ludovic Chabant <ludovic@chabant.com>
parents: 459
diff changeset
270 return conv(state, _unpickle_object, _UNPICKLING)
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
271
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
272 mod_name = state['__module__']
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
273 mod = sys.modules[mod_name]
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
274 class_def = getattr(mod, class_name)
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
275 obj = class_def.__new__(class_def)
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
276
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
277 attr_names = list(state.keys())
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
278 for name in attr_names:
717
fcfbe103cfd1 internal: Fix some bugs with the `fastpickle` module.
Ludovic Chabant <ludovic@chabant.com>
parents: 700
diff changeset
279 if name == '__class__' or name == '__module__':
fcfbe103cfd1 internal: Fix some bugs with the `fastpickle` module.
Ludovic Chabant <ludovic@chabant.com>
parents: 700
diff changeset
280 continue
fcfbe103cfd1 internal: Fix some bugs with the `fastpickle` module.
Ludovic Chabant <ludovic@chabant.com>
parents: 700
diff changeset
281 obj.__dict__[name] = _unpickle_object(state[name])
450
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
282
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
283 return obj
298f8f46432a internal: Add a `fastpickle` module to help with multiprocess serialization.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
284