annotate tests/conftest.py @ 578:683be25cbdb2

internal: Rename `raw_content` to `segments` since it's what it is.
author Ludovic Chabant <ludovic@chabant.com>
date Wed, 30 Dec 2015 14:46:34 -0800
parents a65f04ddbea2
children d8d9e0424a72
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
1 import io
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import sys
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
3 import time
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
4 import pprint
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
5 import os.path
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6 import logging
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
7 import pytest
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
8 import yaml
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
9 import colorama
520
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
10 from werkzeug.exceptions import HTTPException
466
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
11 from piecrust.app import apply_variant_and_values
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
12 from piecrust.configuration import merge_dicts
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
13 from .mockutil import mock_fs, mock_fs_scope
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16 def pytest_runtest_setup(item):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 pass
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20 def pytest_addoption(parser):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
21 parser.addoption(
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
22 '--log-debug',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
23 action='store_true',
86
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 help="Sets the PieCrust logger to output debug info to stdout.")
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27 def pytest_configure(config):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28 if config.getoption('--log-debug'):
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
29 hdl = logging.StreamHandler(stream=sys.stdout)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30 logging.getLogger('piecrust').addHandler(hdl)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
31 logging.getLogger('piecrust').setLevel(logging.DEBUG)
1cd67680c38c Ability to output debug logging to `stdout` when running unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
32
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
33
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
34 def pytest_collect_file(parent, path):
397
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
35 if path.ext == '.yaml' and path.basename.startswith("test"):
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
36 category = os.path.basename(path.dirname)
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
37 if category == 'bakes':
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
38 return BakeTestFile(path, parent)
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
39 elif category == 'procs':
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
40 return PipelineTestFile(path, parent)
397
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
41 elif category == 'cli':
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
42 return ChefTestFile(path, parent)
397
879b6b5647a8 tests: Move all bakes/cli/servings tests files to have a YAML extension.
Ludovic Chabant <ludovic@chabant.com>
parents: 391
diff changeset
43 elif category == 'servings':
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
44 return ServeTestFile(path, parent)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
45
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
46
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
47 class YamlTestFileBase(pytest.File):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
48 def collect(self):
530
1f37f66204b8 tests: Help the Yaml loader figure out the encoding on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 520
diff changeset
49 spec = yaml.load_all(self.fspath.open(encoding='utf8'))
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
50 for i, item in enumerate(spec):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
51 name = '%s_%d' % (self.fspath.basename, i)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
52 if 'test_name' in item:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
53 name += '_%s' % item['test_name']
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
54 yield self.__item_class__(name, self, item)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
55
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
56
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
57 class YamlTestItemBase(pytest.Item):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
58 def __init__(self, name, parent, spec):
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
59 super(YamlTestItemBase, self).__init__(name, parent)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
60 self.spec = spec
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
61
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
62 def _prepareMockFs(self):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
63 fs = mock_fs()
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
64
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
65 # Website config.
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
66 config = {
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
67 'site': {
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
68 'default_format': 'none',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
69 'default_page_layout': 'none',
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
70 'default_post_layout': 'none'}
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
71 }
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
72 test_config = self.spec.get('config')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
73 if test_config is not None:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
74 merge_dicts(config, test_config)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
75 fs.withConfig(config)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
76
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
77 # Input file-system.
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
78 input_files = self.spec.get('in')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
79 if input_files is not None:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
80 _add_mock_files(fs, '/kitchen', input_files)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
81
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
82 return fs
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
83
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
84
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
85 def check_expected_outputs(spec, fs, error_type):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
86 cctx = CompareContext()
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
87 expected_output_files = spec.get('out')
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
88 if expected_output_files:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
89 actual = fs.getStructure('kitchen/_counter')
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
90 error = _compare_dicts(expected_output_files, actual, cctx)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
91 if error:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
92 raise error_type(error)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
93
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
94 expected_partial_files = spec.get('outfiles')
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
95 if expected_partial_files:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
96 keys = list(sorted(expected_partial_files.keys()))
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
97 for key in keys:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
98 try:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
99 actual = fs.getFileEntry('kitchen/_counter/' +
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
100 key.lstrip('/'))
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
101 except Exception as e:
559
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
102 lines = print_fs_tree(fs.path('kitchen/_counter'))
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
103 raise error_type([
559
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
104 "Can't access output file %s: %s" % (key, e),
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
105 "Got output directory:"] +
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
106 lines)
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
107
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
108 expected = expected_partial_files[key]
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
109 # HACK because for some reason PyYAML adds a new line for
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
110 # those and I have no idea why.
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
111 actual = actual.rstrip('\n')
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
112 expected = expected.rstrip('\n')
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
113 cctx.path = key
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
114 cmpres = _compare_str(expected, actual, cctx)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
115 if cmpres:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
116 raise error_type(cmpres)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
117
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
118
559
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
119 def print_fs_tree(rootpath):
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
120 import os
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
121 import os.path
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
122 lines = []
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
123 offset = len(rootpath)
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
124 for pathname, dirnames, filenames in os.walk(rootpath):
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
125 level = pathname[offset:].count(os.sep)
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
126 indent = ' ' * 4 * (level)
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
127 lines.append(indent + os.path.basename(pathname) + '/')
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
128 indent2 = ' ' * 4 * (level + 1)
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
129 for f in filenames:
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
130 lines.append(indent2 + f)
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
131 return lines
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
132
b95977d97652 tests: Print more information when a bake test fails to find an output file.
Ludovic Chabant <ludovic@chabant.com>
parents: 553
diff changeset
133
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
134 class ChefTestItem(YamlTestItemBase):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
135 __initialized_logging__ = False
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
136
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
137 def runtest(self):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
138 if not ChefTestItem.__initialized_logging__:
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
139 colorama.init()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
140 hdl = logging.StreamHandler(stream=sys.stdout)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
141 logging.getLogger().addHandler(hdl)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
142 logging.getLogger().setLevel(logging.INFO)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
143 ChefTestItem.__initialized_logging__ = True
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
144
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
145 fs = self._prepareMockFs()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
146
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
147 argv = self.spec['args']
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
148 if isinstance(argv, str):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
149 argv = argv.split(' ')
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
150
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
151 expected_code = self.spec.get('code', 0)
567
a65f04ddbea2 showconfig: Don't crash when the whole config should be shown.
Ludovic Chabant <ludovic@chabant.com>
parents: 559
diff changeset
152 expected_out = self.spec.get('out', None)
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
153
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
154 with mock_fs_scope(fs):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
155 memstream = io.StringIO()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
156 hdl = logging.StreamHandler(stream=memstream)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
157 logging.getLogger().addHandler(hdl)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
158 try:
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
159 from piecrust.main import PreParsedChefArgs, _run_chef
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
160 pre_args = PreParsedChefArgs(
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
161 root=fs.path('/kitchen'))
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
162 exit_code = _run_chef(pre_args, argv)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
163 finally:
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
164 logging.getLogger().removeHandler(hdl)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
165
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
166 assert expected_code == exit_code
493
95b77239c3b7 tests: Fix `find` tests on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 466
diff changeset
167
567
a65f04ddbea2 showconfig: Don't crash when the whole config should be shown.
Ludovic Chabant <ludovic@chabant.com>
parents: 559
diff changeset
168 if expected_out is not None:
a65f04ddbea2 showconfig: Don't crash when the whole config should be shown.
Ludovic Chabant <ludovic@chabant.com>
parents: 559
diff changeset
169 actual_out = memstream.getvalue()
a65f04ddbea2 showconfig: Don't crash when the whole config should be shown.
Ludovic Chabant <ludovic@chabant.com>
parents: 559
diff changeset
170 if self.spec.get('replace_out_path_sep'):
a65f04ddbea2 showconfig: Don't crash when the whole config should be shown.
Ludovic Chabant <ludovic@chabant.com>
parents: 559
diff changeset
171 expected_out = expected_out.replace('/', os.sep)
a65f04ddbea2 showconfig: Don't crash when the whole config should be shown.
Ludovic Chabant <ludovic@chabant.com>
parents: 559
diff changeset
172 assert expected_out == actual_out
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
173
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
174 def reportinfo(self):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
175 return self.fspath, 0, "bake: %s" % self.name
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
176
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
177 def repr_failure(self, excinfo):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
178 if isinstance(excinfo.value, ExpectedChefOutputError):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
179 return ('\n'.join(
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
180 ['Unexpected command output. Left is expected output, '
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
181 'right is actual output'] +
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
182 excinfo.value.args[0]))
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
183 return super(ChefTestItem, self).repr_failure(excinfo)
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
184
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
185
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
186 class ExpectedChefOutputError(Exception):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
187 pass
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
188
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
189
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
190 class ChefTestFile(YamlTestFileBase):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
191 __item_class__ = ChefTestItem
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
192
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
193
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
194 class BakeTestItem(YamlTestItemBase):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
195 def runtest(self):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
196 fs = self._prepareMockFs()
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
197
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
198 from piecrust.baking.baker import Baker
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
199 with mock_fs_scope(fs):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
200 out_dir = fs.path('kitchen/_counter')
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
201 app = fs.getApp()
466
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
202
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
203 variant = self.spec.get('config_variant')
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
204 values = self.spec.get('config_values')
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
205 if values is not None:
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
206 values = list(values.items())
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
207 apply_variant_and_values(app, variant, values)
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
208
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
209 baker = Baker(app, out_dir,
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
210 applied_config_variant=variant,
456db44dcc53 bake: Pass the config variants and values from the CLI to the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 439
diff changeset
211 applied_config_values=values)
398
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
212 record = baker.bake()
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
213
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
214 if not record.success:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
215 errors = []
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
216 for e in record.entries:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
217 errors += e.getAllErrors()
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 399
diff changeset
218 raise BakeError(errors)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
219
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
220 check_expected_outputs(self.spec, fs, ExpectedBakeOutputError)
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
221
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
222 def reportinfo(self):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
223 return self.fspath, 0, "bake: %s" % self.name
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
224
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
225 def repr_failure(self, excinfo):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
226 if isinstance(excinfo.value, ExpectedBakeOutputError):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
227 return ('\n'.join(
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
228 ['Unexpected bake output. Left is expected output, '
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
229 'right is actual output'] +
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
230 excinfo.value.args[0]))
398
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
231 elif isinstance(excinfo.value, BakeError):
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
232 return ('\n'.join(
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
233 ['Errors occured during bake:'] +
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
234 excinfo.value.args[0]))
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
235 return super(BakeTestItem, self).repr_failure(excinfo)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
236
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
237
398
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
238 class BakeError(Exception):
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
239 pass
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
240
af17c143b9ab tests: Fail bake tests with a proper error message when bake fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 397
diff changeset
241
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
242 class ExpectedBakeOutputError(Exception):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
243 pass
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
244
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
245
385
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
246 class BakeTestFile(YamlTestFileBase):
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
247 __item_class__ = BakeTestItem
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
248
2d5f2289885a tests: Add support for "Chef tests", which are direct CLI tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 365
diff changeset
249
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
250 class PipelineTestItem(YamlTestItemBase):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
251 def runtest(self):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
252 fs = self._prepareMockFs()
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
253
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
254 from piecrust.processing.pipeline import ProcessorPipeline
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
255 with mock_fs_scope(fs):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
256 out_dir = fs.path('kitchen/_counter')
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
257 app = fs.getApp()
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
258 pipeline = ProcessorPipeline(app, out_dir)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
259
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
260 proc_names = self.spec.get('processors')
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
261 if proc_names:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
262 pipeline.enabled_processors = proc_names
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
263
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
264 record = pipeline.run()
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
265
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
266 if not record.success:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
267 errors = []
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
268 for e in record.entries:
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
269 errors += e.errors
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
270 raise PipelineError(errors)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
271
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
272 check_expected_outputs(self.spec, fs, ExpectedPipelineOutputError)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
273
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
274 def reportinfo(self):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
275 return self.fspath, 0, "pipeline: %s" % self.name
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
276
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
277 def repr_failure(self, excinfo):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
278 if isinstance(excinfo.value, ExpectedPipelineOutputError):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
279 return ('\n'.join(
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
280 ['Unexpected pipeline output. Left is expected output, '
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
281 'right is actual output'] +
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
282 excinfo.value.args[0]))
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
283 elif isinstance(excinfo.value, PipelineError):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
284 return ('\n'.join(
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
285 ['Errors occured during processing:'] +
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
286 excinfo.value.args[0]))
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
287 return super(PipelineTestItem, self).repr_failure(excinfo)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
288
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
289
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
290 class PipelineError(Exception):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
291 pass
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
292
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
293
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
294 class ExpectedPipelineOutputError(Exception):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
295 pass
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
296
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
297
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
298 class PipelineTestFile(YamlTestFileBase):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
299 __item_class__ = PipelineTestItem
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
300
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
301
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
302 class ServeTestItem(YamlTestItemBase):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
303 class _TestApp(object):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
304 def __init__(self, server):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
305 self.server = server
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
306
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
307 def __call__(self, environ, start_response):
553
cc6f3dbe3048 serve: Extract some of the server's functionality into WSGI middlewares.
Ludovic Chabant <ludovic@chabant.com>
parents: 530
diff changeset
308 response = self.server._try_run_request(environ)
cc6f3dbe3048 serve: Extract some of the server's functionality into WSGI middlewares.
Ludovic Chabant <ludovic@chabant.com>
parents: 530
diff changeset
309 return response(environ, start_response)
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
310
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
311 def runtest(self):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
312 fs = self._prepareMockFs()
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
313
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
314 url = self.spec.get('url')
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
315 if url is None:
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
316 raise Exception("Missing URL in test spec.")
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
317
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
318 expected_status = self.spec.get('status', 200)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
319 expected_headers = self.spec.get('headers')
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
320 expected_output = self.spec.get('out')
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
321 expected_contains = self.spec.get('out_contains')
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
322
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
323 from werkzeug.test import Client
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
324 from werkzeug.wrappers import BaseResponse
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
325 from piecrust.serving.server import Server
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
326 with mock_fs_scope(fs):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
327 server = Server(fs.path('/kitchen'))
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
328 test_app = self._TestApp(server)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
329 client = Client(test_app, BaseResponse)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
330 resp = client.get(url)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
331 assert expected_status == resp.status_code
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
332
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
333 if expected_headers:
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
334 for k, v in expected_headers.items():
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
335 assert v == resp.headers.get(k)
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
336
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
337 actual = resp.data.decode('utf8').rstrip()
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
338 if expected_output:
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
339 assert expected_output.rstrip() == actual
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
340
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
341 if expected_contains:
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
342 assert expected_contains.rstrip() in actual
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
343
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
344 def reportinfo(self):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
345 return self.fspath, 0, "serve: %s" % self.name
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
346
520
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
347 def repr_failure(self, excinfo):
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
348 from piecrust.serving.server import MultipleNotFound
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
349 if isinstance(excinfo.value, MultipleNotFound):
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
350 return '\n'.join(
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
351 ["HTTP error 404 returned:",
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
352 excinfo.value.description] +
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
353 [e.description for e in excinfo.value._nfes])
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
354 elif isinstance(excinfo.value, HTTPException):
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
355 return '\n'.join(
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
356 ["HTTP error %s returned:" % excinfo.value.code,
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
357 excinfo.value.description])
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
358 return super(ServeTestItem, self).repr_failure(excinfo)
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
359
391
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
360
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
361 class ServeTestFile(YamlTestFileBase):
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
362 __item_class__ = ServeTestItem
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
363
3e4bb57d8506 tests: Add support for testing the Chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 385
diff changeset
364
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
365 def _add_mock_files(fs, parent_path, spec):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
366 for name, subspec in spec.items():
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
367 path = os.path.join(parent_path, name)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
368 if isinstance(subspec, str):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
369 fs.withFile(path, subspec)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
370 elif isinstance(subspec, dict):
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
371 _add_mock_files(fs, path, subspec)
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
372
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
373
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
374 class CompareContext(object):
439
c0700c6d9545 tests: Fix crash in processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 436
diff changeset
375 def __init__(self, path=None, t=None):
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
376 self.path = path or ''
439
c0700c6d9545 tests: Fix crash in processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 436
diff changeset
377 self.time = t or time.time()
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
378
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
379 def createChildContext(self, name):
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
380 ctx = CompareContext(
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
381 path='%s/%s' % (self.path, name),
439
c0700c6d9545 tests: Fix crash in processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 436
diff changeset
382 t=self.time)
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
383 return ctx
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
384
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
385
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
386 def _compare(left, right, ctx):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
387 if type(left) != type(right):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
388 return (["Different items: ",
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
389 "%s: %s" % (ctx.path, pprint.pformat(left)),
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
390 "%s: %s" % (ctx.path, pprint.pformat(right))])
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
391 if isinstance(left, str):
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
392 return _compare_str(left, right, ctx)
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
393 elif isinstance(left, dict):
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
394 return _compare_dicts(left, right, ctx)
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
395 elif isinstance(left, list):
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
396 return _compare_lists(left, right, ctx)
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
397 elif left != right:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
398 return (["Different items: ",
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
399 "%s: %s" % (ctx.path, pprint.pformat(left)),
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
400 "%s: %s" % (ctx.path, pprint.pformat(right))])
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
401
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
402
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
403 def _compare_dicts(left, right, ctx):
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
404 key_diff = set(left.keys()) ^ set(right.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
405 if key_diff:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
406 extra_left = set(left.keys()) - set(right.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
407 if extra_left:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
408 return (["Left contains more items: "] +
520
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
409 ['- %s/%s' % (ctx.path, k) for k in extra_left] +
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
410 ['Left:', ', '.join(left.keys())] +
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
411 ['Right:', ', '.join(right.keys())])
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
412 extra_right = set(right.keys()) - set(left.keys())
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
413 if extra_right:
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
414 return (["Right contains more items: "] +
520
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
415 ['- %s/%s' % (ctx.path, k) for k in extra_right] +
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
416 ['Left:', ', '.join(left.keys())] +
bab91fcef741 bake/serve: Improve support for unicode, add slugification options.
Ludovic Chabant <ludovic@chabant.com>
parents: 493
diff changeset
417 ['Right:', ', '.join(right.keys())])
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
418 return ["Unknown difference"]
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
419
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
420 for key in left.keys():
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
421 lv = left[key]
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
422 rv = right[key]
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
423 child_ctx = ctx.createChildContext(key)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
424 cmpres = _compare(lv, rv, child_ctx)
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
425 if cmpres:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
426 return cmpres
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
427 return None
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
428
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
429
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
430 def _compare_lists(left, right, ctx):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
431 for i in range(min(len(left), len(right))):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
432 l = left[i]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
433 r = right[i]
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
434 cmpres = _compare(l, r, ctx)
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
435 if cmpres:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
436 return cmpres
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
437 if len(left) > len(right):
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
438 return (["Left '%s' contains more items. First extra item: " %
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
439 ctx.path, left[len(right)]])
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
440 if len(right) > len(left):
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
441 return (["Right '%s' contains more items. First extra item: " %
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
442 ctx.path, right[len(left)]])
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
443 return None
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
444
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
445
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
446 def _compare_str(left, right, ctx):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
447 if left == right:
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
448 return None
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
449
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
450 test_time_iso8601 = time.strftime('%Y-%m-%dT%H:%M:%SZ',
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
451 time.gmtime(ctx.time))
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
452 replacements = {
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
453 '%test_time_iso8601%': test_time_iso8601}
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
454 for token, repl in replacements.items():
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
455 left = left.replace(token, repl)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
456 right = right.replace(token, repl)
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
457
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
458 for i in range(min(len(left), len(right))):
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
459 if left[i] != right[i]:
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
460 start = max(0, i - 15)
399
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
461 l_end = min(len(left), i + 15)
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
462 r_end = min(len(right), i + 15)
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
463
399
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
464 l_str = ''
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
465 l_offset = 0
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
466 for j in range(start, l_end):
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
467 c = repr(left[j]).strip("'")
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
468 l_str += c
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
469 if j < i:
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
470 l_offset += len(c)
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
471
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
472 r_str = ''
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
473 r_offset = 0
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
474 for j in range(start, r_end):
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
475 c = repr(right[j]).strip("'")
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
476 r_str += c
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
477 if j < i:
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
478 r_offset += len(c)
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
479
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
480 return ["Items '%s' differ at index %d:" % (ctx.path, i), '',
365
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
481 "Left:", left, '',
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
482 "Right:", right, '',
a9929e0b8f66 tests: Changes to output report and hack for comparing outputs.
Ludovic Chabant <ludovic@chabant.com>
parents: 351
diff changeset
483 "Difference:",
399
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
484 l_str, (' ' * l_offset + '^'),
a0724af26c12 tests: More accurate marker position for diff'ing strings.
Ludovic Chabant <ludovic@chabant.com>
parents: 398
diff changeset
485 r_str, (' ' * r_offset + '^')]
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
486 if len(left) > len(right):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
487 return ["Left is longer.",
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
488 "Left '%s': " % ctx.path, left,
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
489 "Right '%s': " % ctx.path, right,
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
490 "Extra items: %r" % left[len(right):]]
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
491 if len(right) > len(left):
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
492 return ["Right is longer.",
436
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
493 "Left '%s': " % ctx.path, left,
2aa879d63133 tests: Add pipeline processing tests.
Ludovic Chabant <ludovic@chabant.com>
parents: 411
diff changeset
494 "Right '%s': " % ctx.path, right,
351
1f22d4b10fef tests: Improve bake tests output, add support for partial output checks.
Ludovic Chabant <ludovic@chabant.com>
parents: 347
diff changeset
495 "Extra items: %r" % right[len(left):]]
347
76c838453dbe tests: Support for YAML-based baking tests. Convert old code-based ones.
Ludovic Chabant <ludovic@chabant.com>
parents: 86
diff changeset
496