Mercurial > piecrust2

annotate tests/conftest.py @ 365:a9929e0b8f66

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
tests: Changes to output report and hack for comparing outputs. * Still some problems with newlines and YAML so might as well strip both the actual and expected outputs. * More fancy output for comparing strings.
author Ludovic Chabant <ludovic@chabant.com>
date Sun, 03 May 2015 18:40:05 -0700
parents 1f22d4b10fef
children 2d5f2289885a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import sys
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
2 import pprint
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
3 import os.path
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import logging
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
5 import pytest
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
6 import yaml
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
7 from piecrust.configuration import merge_dicts
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
8 from .mockutil import mock_fs, mock_fs_scope
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11 def pytest_runtest_setup(item):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12 pass
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 def pytest_addoption(parser):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
16 parser.addoption(
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
17 '--log-debug',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
18 action='store_true',
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19 help="Sets the PieCrust logger to output debug info to stdout.")
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
22 def pytest_configure(config):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 if config.getoption('--log-debug'):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 hdl = logging.StreamHandler(stream=sys.stdout)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25 logging.getLogger('piecrust').addHandler(hdl)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26 logging.getLogger('piecrust').setLevel(logging.DEBUG)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
28
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
29 def pytest_collect_file(parent, path):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
30 if path.ext == ".bake" and path.basename.startswith("test"):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
31 return BakeTestFile(path, parent)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
32
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
33
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
34 class BakeTestFile(pytest.File):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
35 def collect(self):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
36 spec = yaml.load_all(self.fspath.open())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
37 for i, item in enumerate(spec):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
38 name = '%s_%d' % (self.fspath.basename, i)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
39 if 'test_name' in item:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
40 name += '_%s' % item['test_name']
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
41 yield BakeTestItem(name, self, item)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
42
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
43
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
44 class BakeTestItem(pytest.Item):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
45 def __init__(self, name, parent, spec):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
46 super(BakeTestItem, self).__init__(name, parent)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
47 self.spec = spec
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
48
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
49 def runtest(self):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
50 fs = mock_fs()
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
51
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
52 # Website config.
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
53 config = {
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
54 'site': {
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
55 'default_format': 'none',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
56 'default_page_layout': 'none',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
57 'default_post_layout': 'none'}
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
58 }
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
59 test_config = self.spec.get('config')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
60 if test_config is not None:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
61 merge_dicts(config, test_config)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
62 fs.withConfig(config)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
63
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
64 # Input file-system.
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
65 input_files = self.spec.get('in')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
66 if input_files is not None:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
67 _add_mock_files(fs, '/kitchen', input_files)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
68
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
69 # Output file-system.
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
70 expected_output_files = self.spec.get('out')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
71 expected_partial_files = self.spec.get('outfiles')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
72
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
73 # Bake!
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
74 from piecrust.baking.baker import Baker
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
75 with mock_fs_scope(fs):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
76 out_dir = fs.path('kitchen/_counter')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
77 app = fs.getApp()
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
78 baker = Baker(app, out_dir)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
79 baker.bake()
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
80
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
81 if expected_output_files:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
82 actual = fs.getStructure('kitchen/_counter')
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
83 error = _compare_dicts(expected_output_files, actual)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
84 if error:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
85 raise ExpectedBakeOutputError(error)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
86
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
87 if expected_partial_files:
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
88 keys = list(sorted(expected_partial_files.keys()))
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
89 for key in keys:
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
90 try:
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
91 actual = fs.getFileEntry('kitchen/_counter/' +
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
92 key.lstrip('/'))
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
93 except Exception as e:
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
94 raise ExpectedBakeOutputError([
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
95 "Can't access output file %s: %s" % (key, e)])
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
96
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
97 expected = expected_partial_files[key]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
98 # HACK because for some reason PyYAML adds a new line for those
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
99 # and I have no idea why.
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
100 actual = actual.rstrip('\n')
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
101 expected = expected.rstrip('\n')
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
102 cmpres = _compare_str(expected, actual, key)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
103 if cmpres:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
104 raise ExpectedBakeOutputError(cmpres)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
105
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
106 def reportinfo(self):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
107 return self.fspath, 0, "bake: %s" % self.name
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
108
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
109 def repr_failure(self, excinfo):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
110 if isinstance(excinfo.value, ExpectedBakeOutputError):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
111 return ('\n'.join(
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
112 ['Unexpected bake output. Left is expected output, '
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
113 'right is actual output'] +
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
114 excinfo.value.args[0]))
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
115 return super(BakeTestItem, self).repr_failure(excinfo)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
116
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
117
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
118 class ExpectedBakeOutputError(Exception):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
119 pass
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
120
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
121
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
122 def _add_mock_files(fs, parent_path, spec):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
123 for name, subspec in spec.items():
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
124 path = os.path.join(parent_path, name)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
125 if isinstance(subspec, str):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
126 fs.withFile(path, subspec)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
127 elif isinstance(subspec, dict):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
128 _add_mock_files(fs, path, subspec)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
129
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
130
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
131 def _compare(left, right, path):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
132 if type(left) != type(right):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
133 return (["Different items: ",
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
134 "%s: %s" % (path, pprint.pformat(left)),
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
135 "%s: %s" % (path, pprint.pformat(right))])
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
136 if isinstance(left, str):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
137 return _compare_str(left, right, path)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
138 elif isinstance(left, dict):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
139 return _compare_dicts(left, right, path)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
140 elif isinstance(left, list):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
141 return _compare_lists(left, right, path)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
142 elif left != right:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
143 return (["Different items: ",
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
144 "%s: %s" % (path, pprint.pformat(left)),
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
145 "%s: %s" % (path, pprint.pformat(right))])
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
146
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
147
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
148 def _compare_dicts(left, right, basepath=''):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
149 key_diff = set(left.keys()) ^ set(right.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
150 if key_diff:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
151 extra_left = set(left.keys()) - set(right.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
152 if extra_left:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
153 return (["Left contains more items: "] +
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
154 ['- %s/%s' % (basepath, k) for k in extra_left])
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
155 extra_right = set(right.keys()) - set(left.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
156 if extra_right:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
157 return (["Right contains more items: "] +
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
158 ['- %s/%s' % (basepath, k) for k in extra_right])
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
159 return ["Unknown difference"]
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
160
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
161 for key in left.keys():
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
162 lv = left[key]
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
163 rv = right[key]
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
164 childpath = basepath + '/' + key
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
165 cmpres = _compare(lv, rv, childpath)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
166 if cmpres:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
167 return cmpres
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
168 return None
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
169
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
170
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
171 def _compare_lists(left, right, path):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
172 for i in range(min(len(left), len(right))):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
173 l = left[i]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
174 r = right[i]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
175 cmpres = _compare(l, r, path)
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
176 if cmpres:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
177 return cmpres
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
178 if len(left) > len(right):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
179 return (["Left '%s' contains more items. First extra item: " % path,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
180 left[len(right)]])
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
181 if len(right) > len(left):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
182 return (["Right '%s' contains more items. First extra item: " % path,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
183 right[len(left)]])
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
184 return None
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
185
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
186
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
187 def _compare_str(left, right, path):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
188 if left == right:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
189 return None
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
190 for i in range(min(len(left), len(right))):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
191 if left[i] != right[i]:
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
192 start = max(0, i - 15)
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
193 marker_offset = min(15, (i - start)) + 3
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
194
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
195 lend = min(len(left), i + 15)
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
196 rend = min(len(right), i + 15)
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
197
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
198 return ["Items '%s' differ at index %d:" % (path, i), '',
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
199 "Left:", left, '',
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
200 "Right:", right, '',
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
201 "Difference:",
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
202 repr(left[start:lend]),
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
203 (' ' * marker_offset + '^'),
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
204 repr(right[start:rend]),
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
205 (' ' * marker_offset + '^')]
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
206 if len(left) > len(right):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
207 return ["Left is longer.",
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
208 "Left '%s': " % path, left,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
209 "Right '%s': " % path, right,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
210 "Extra items: %r" % left[len(right):]]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
211 if len(right) > len(left):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
212 return ["Right is longer.",
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
213 "Left '%s': " % path, left,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
214 "Right '%s': " % path, right,
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
215 "Extra items: %r" % right[len(left):]]
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
216