Mercurial > piecrust2

annotate tests/conftest.py @ 411:e7b865f8f335

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bake: Enable multiprocess baking. Baking is now done by running a worker per CPU, and sending jobs to them. This changes several things across the codebase: * Ability to not cache things related to pages other than the 'main' page (i.e. the page at the bottom of the execution stack). * Decouple the baking process from the bake records, so only the main process keeps track (and modifies) the bake record. * Remove the need for 'batch page getters' and loading a page directly from the page factories. There are various smaller changes too included here, including support for scope performance timers that are saved with the bake record and can be printed out to the console. Yes I got carried away. For testing, the in-memory 'mock' file-system doesn't work anymore, since we're spawning processes, so this is replaced by a 'tmpfs' file-system which is saved in temporary files on disk and deleted after tests have run.
author Ludovic Chabant <ludovic@chabant.com>
date Fri, 12 Jun 2015 17:09:19 -0700
parents a0724af26c12
children 2aa879d63133
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
1 import io
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import sys
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
3 import pprint
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
4 import os.path
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import logging
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
6 import pytest
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
7 import yaml
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
8 import colorama
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
9 from piecrust.configuration import merge_dicts
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
10 from .mockutil import mock_fs, mock_fs_scope
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13 def pytest_runtest_setup(item):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14 pass
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 def pytest_addoption(parser):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
18 parser.addoption(
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
19 '--log-debug',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
20 action='store_true',
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21 help="Sets the PieCrust logger to output debug info to stdout.")
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
22
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 def pytest_configure(config):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25 if config.getoption('--log-debug'):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26 hdl = logging.StreamHandler(stream=sys.stdout)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27 logging.getLogger('piecrust').addHandler(hdl)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28 logging.getLogger('piecrust').setLevel(logging.DEBUG)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
29
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
30
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
31 def pytest_collect_file(parent, path):
397
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
32 if path.ext == '.yaml' and path.basename.startswith("test"):
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
33 category = os.path.basename(path.dirname)
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
34 if category == 'bakes':
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
35 return BakeTestFile(path, parent)
397
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
36 elif category == 'cli':
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
37 return ChefTestFile(path, parent)
397
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
38 elif category == 'servings':
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
39 return ServeTestFile(path, parent)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
40
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
41
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
42 class YamlTestFileBase(pytest.File):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
43 def collect(self):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
44 spec = yaml.load_all(self.fspath.open())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
45 for i, item in enumerate(spec):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
46 name = '%s_%d' % (self.fspath.basename, i)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
47 if 'test_name' in item:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
48 name += '_%s' % item['test_name']
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
49 yield self.__item_class__(name, self, item)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
50
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
51
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
52 class YamlTestItemBase(pytest.Item):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
53 def __init__(self, name, parent, spec):
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
54 super(YamlTestItemBase, self).__init__(name, parent)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
55 self.spec = spec
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
56
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
57 def _prepareMockFs(self):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
58 fs = mock_fs()
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
59
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
60 # Website config.
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
61 config = {
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
62 'site': {
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
63 'default_format': 'none',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
64 'default_page_layout': 'none',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
65 'default_post_layout': 'none'}
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
66 }
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
67 test_config = self.spec.get('config')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
68 if test_config is not None:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
69 merge_dicts(config, test_config)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
70 fs.withConfig(config)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
71
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
72 # Input file-system.
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
73 input_files = self.spec.get('in')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
74 if input_files is not None:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
75 _add_mock_files(fs, '/kitchen', input_files)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
76
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
77 return fs
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
78
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
79
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
80 class ChefTestItem(YamlTestItemBase):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
81 __initialized_logging__ = False
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
82
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
83 def runtest(self):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
84 if not ChefTestItem.__initialized_logging__:
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
85 colorama.init()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
86 hdl = logging.StreamHandler(stream=sys.stdout)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
87 logging.getLogger().addHandler(hdl)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
88 logging.getLogger().setLevel(logging.INFO)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
89 ChefTestItem.__initialized_logging__ = True
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
90
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
91 fs = self._prepareMockFs()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
92
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
93 argv = self.spec['args']
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
94 if isinstance(argv, str):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
95 argv = argv.split(' ')
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
96
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
97 expected_code = self.spec.get('code', 0)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
98 expected_out = self.spec.get('out', '')
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
99
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
100 with mock_fs_scope(fs):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
101 memstream = io.StringIO()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
102 hdl = logging.StreamHandler(stream=memstream)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
103 logging.getLogger().addHandler(hdl)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
104 try:
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
105 from piecrust.main import PreParsedChefArgs, _run_chef
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
106 pre_args = PreParsedChefArgs(
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
107 root=fs.path('/kitchen'))
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
108 exit_code = _run_chef(pre_args, argv)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
109 finally:
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
110 logging.getLogger().removeHandler(hdl)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
111
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
112 assert expected_code == exit_code
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
113 assert expected_out == memstream.getvalue()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
114
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
115 def reportinfo(self):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
116 return self.fspath, 0, "bake: %s" % self.name
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
117
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
118 def repr_failure(self, excinfo):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
119 if isinstance(excinfo.value, ExpectedChefOutputError):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
120 return ('\n'.join(
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
121 ['Unexpected command output. Left is expected output, '
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
122 'right is actual output'] +
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
123 excinfo.value.args[0]))
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
124 return super(ChefTestItem, self).repr_failure(excinfo)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
125
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
126
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
127 class ExpectedChefOutputError(Exception):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
128 pass
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
129
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
130
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
131 class ChefTestFile(YamlTestFileBase):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
132 __item_class__ = ChefTestItem
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
133
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
134
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
135 class BakeTestItem(YamlTestItemBase):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
136 def runtest(self):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
137 fs = self._prepareMockFs()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
138
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
139 # Output file-system.
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
140 expected_output_files = self.spec.get('out')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
141 expected_partial_files = self.spec.get('outfiles')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
142
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
143 # Bake!
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
144 from piecrust.baking.baker import Baker
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
145 with mock_fs_scope(fs):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
146 out_dir = fs.path('kitchen/_counter')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
147 app = fs.getApp()
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
148 baker = Baker(app, out_dir)
398
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
149 record = baker.bake()
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
150
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
151 if not record.success:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
152 errors = []
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
153 for e in record.entries:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
154 errors += e.getAllErrors()
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
155 raise BakeError(errors)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
156
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
157 if expected_output_files:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
158 actual = fs.getStructure('kitchen/_counter')
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
159 error = _compare_dicts(expected_output_files, actual)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
160 if error:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
161 raise ExpectedBakeOutputError(error)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
162
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
163 if expected_partial_files:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
164 keys = list(sorted(expected_partial_files.keys()))
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
165 for key in keys:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
166 try:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
167 actual = fs.getFileEntry('kitchen/_counter/' +
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
168 key.lstrip('/'))
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
169 except Exception as e:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
170 raise ExpectedBakeOutputError([
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
171 "Can't access output file %s: %s" % (key, e)])
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
172
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
173 expected = expected_partial_files[key]
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
174 # HACK because for some reason PyYAML adds a new line for
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
175 # those and I have no idea why.
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
176 actual = actual.rstrip('\n')
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
177 expected = expected.rstrip('\n')
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
178 cmpres = _compare_str(expected, actual, key)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
179 if cmpres:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
180 raise ExpectedBakeOutputError(cmpres)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
181
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
182 def reportinfo(self):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
183 return self.fspath, 0, "bake: %s" % self.name
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
184
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
185 def repr_failure(self, excinfo):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
186 if isinstance(excinfo.value, ExpectedBakeOutputError):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
187 return ('\n'.join(
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
188 ['Unexpected bake output. Left is expected output, '
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
189 'right is actual output'] +
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
190 excinfo.value.args[0]))
398
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
191 elif isinstance(excinfo.value, BakeError):
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
192 return ('\n'.join(
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
193 ['Errors occured during bake:'] +
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
194 excinfo.value.args[0]))
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
195 return super(BakeTestItem, self).repr_failure(excinfo)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
196
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
197
398
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
198 class BakeError(Exception):
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
199 pass
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
200
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
201
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
202 class ExpectedBakeOutputError(Exception):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
203 pass
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
204
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
205
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
206 class BakeTestFile(YamlTestFileBase):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
207 __item_class__ = BakeTestItem
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
208
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
209
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
210 class ServeTestItem(YamlTestItemBase):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
211 class _TestApp(object):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
212 def __init__(self, server):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
213 self.server = server
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
214
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
215 def __call__(self, environ, start_response):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
216 return self.server._try_run_request(environ, start_response)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
217
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
218 def runtest(self):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
219 fs = self._prepareMockFs()
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
220
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
221 url = self.spec.get('url')
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
222 if url is None:
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
223 raise Exception("Missing URL in test spec.")
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
224
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
225 expected_status = self.spec.get('status', 200)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
226 expected_headers = self.spec.get('headers')
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
227 expected_output = self.spec.get('out')
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
228 expected_contains = self.spec.get('out_contains')
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
229
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
230 from werkzeug.test import Client
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
231 from werkzeug.wrappers import BaseResponse
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
232 from piecrust.serving.server import Server
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
233 with mock_fs_scope(fs):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
234 server = Server(fs.path('/kitchen'))
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
235 test_app = self._TestApp(server)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
236 client = Client(test_app, BaseResponse)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
237 resp = client.get(url)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
238 assert expected_status == resp.status_code
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
239
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
240 if expected_headers:
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
241 for k, v in expected_headers.items():
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
242 assert v == resp.headers.get(k)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
243
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
244 actual = resp.data.decode('utf8').rstrip()
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
245 if expected_output:
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
246 assert expected_output.rstrip() == actual
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
247
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
248 if expected_contains:
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
249 assert expected_contains.rstrip() in actual
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
250
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
251 def reportinfo(self):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
252 return self.fspath, 0, "serve: %s" % self.name
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
253
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
254
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
255 class ServeTestFile(YamlTestFileBase):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
256 __item_class__ = ServeTestItem
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
257
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
258
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
259 def _add_mock_files(fs, parent_path, spec):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
260 for name, subspec in spec.items():
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
261 path = os.path.join(parent_path, name)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
262 if isinstance(subspec, str):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
263 fs.withFile(path, subspec)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
264 elif isinstance(subspec, dict):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
265 _add_mock_files(fs, path, subspec)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
266
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
267
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
268 def _compare(left, right, path):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
269 if type(left) != type(right):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
270 return (["Different items: ",
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
271 "%s: %s" % (path, pprint.pformat(left)),
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
272 "%s: %s" % (path, pprint.pformat(right))])
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
273 if isinstance(left, str):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
274 return _compare_str(left, right, path)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
275 elif isinstance(left, dict):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
276 return _compare_dicts(left, right, path)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
277 elif isinstance(left, list):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
278 return _compare_lists(left, right, path)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
279 elif left != right:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
280 return (["Different items: ",
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
281 "%s: %s" % (path, pprint.pformat(left)),
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
282 "%s: %s" % (path, pprint.pformat(right))])
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
283
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
284
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
285 def _compare_dicts(left, right, basepath=''):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
286 key_diff = set(left.keys()) ^ set(right.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
287 if key_diff:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
288 extra_left = set(left.keys()) - set(right.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
289 if extra_left:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
290 return (["Left contains more items: "] +
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
291 ['- %s/%s' % (basepath, k) for k in extra_left])
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
292 extra_right = set(right.keys()) - set(left.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
293 if extra_right:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
294 return (["Right contains more items: "] +
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
295 ['- %s/%s' % (basepath, k) for k in extra_right])
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
296 return ["Unknown difference"]
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
297
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
298 for key in left.keys():
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
299 lv = left[key]
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
300 rv = right[key]
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
301 childpath = basepath + '/' + key
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
302 cmpres = _compare(lv, rv, childpath)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
303 if cmpres:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
304 return cmpres
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
305 return None
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
306
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
307
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
308 def _compare_lists(left, right, path):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
309 for i in range(min(len(left), len(right))):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
310 l = left[i]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
311 r = right[i]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
312 cmpres = _compare(l, r, path)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
313 if cmpres:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
314 return cmpres
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
315 if len(left) > len(right):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
316 return (["Left '%s' contains more items. First extra item: " % path,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
317 left[len(right)]])
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
318 if len(right) > len(left):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
319 return (["Right '%s' contains more items. First extra item: " % path,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
320 right[len(left)]])
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
321 return None
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
322
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
323
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
324 def _compare_str(left, right, path):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
325 if left == right:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
326 return None
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
327 for i in range(min(len(left), len(right))):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
328 if left[i] != right[i]:
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
329 start = max(0, i - 15)
399
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
330 l_end = min(len(left), i + 15)
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
331 r_end = min(len(right), i + 15)
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
332
399
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
333 l_str = ''
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
334 l_offset = 0
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
335 for j in range(start, l_end):
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
336 c = repr(left[j]).strip("'")
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
337 l_str += c
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
338 if j < i:
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
339 l_offset += len(c)
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
340
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
341 r_str = ''
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
342 r_offset = 0
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
343 for j in range(start, r_end):
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
344 c = repr(right[j]).strip("'")
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
345 r_str += c
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
346 if j < i:
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
347 r_offset += len(c)
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
348
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
349 return ["Items '%s' differ at index %d:" % (path, i), '',
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
350 "Left:", left, '',
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
351 "Right:", right, '',
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
352 "Difference:",
399
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
353 l_str, (' ' * l_offset + '^'),
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
354 r_str, (' ' * r_offset + '^')]
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
355 if len(left) > len(right):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
356 return ["Left is longer.",
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
357 "Left '%s': " % path, left,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
358 "Right '%s': " % path, right,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
359 "Extra items: %r" % left[len(right):]]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
360 if len(right) > len(left):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
361 return ["Right is longer.",
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
362 "Left '%s': " % path, left,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
363 "Right '%s': " % path, right,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
364 "Extra items: %r" % right[len(left):]]
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
365