Mercurial > piecrust2
annotate tests/test_baking_baker.py @ 415:0e9a94b7fdfa
bake: Improve bake record information.
* Store things in the bake record that require less interaction between the
master process and the workers. For instance, don't store the paginator
object in the render pass info -- instead, just store whether pagination
was used, and whether it had more items.
* Simplify information passing between workers and bake passes by saving the
rendering info to the JSON cache. This means the "render first sub" job
doesn't have to return anything except errors now.
* Add more performance counter info.
author | Ludovic Chabant <ludovic@chabant.com> |
---|---|
date | Sat, 20 Jun 2015 19:23:16 -0700 |
parents | e7b865f8f335 |
children | 6b6c5442c790 |
rev | line source |
---|---|
286
a2d283d1033d
tests: Fixes for running on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents:
262
diff
changeset
|
1 import time |
11
617191dec18e
Fixes for Windows, make `findPagePath` return a ref path.
Ludovic Chabant <ludovic@chabant.com>
parents:
6
diff
changeset
|
2 import os.path |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
3 import pytest |
411
e7b865f8f335
bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents:
347
diff
changeset
|
4 from piecrust.baking.baker import Baker |
e7b865f8f335
bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents:
347
diff
changeset
|
5 from piecrust.baking.single import PageBaker |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
6 from piecrust.baking.records import BakeRecord |
85
3471ffa059b2
Add a `BakeScheduler` to handle build dependencies. Add unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
29
diff
changeset
|
7 from .mockutil import get_mock_app, mock_fs, mock_fs_scope |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
8 |
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
9 |
262
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
10 @pytest.mark.parametrize('uri, pretty, expected', [ |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
11 # Pretty URLs |
262
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
12 ('', True, 'index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
13 ('2', True, '2/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
14 ('foo', True, 'foo/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
15 ('foo/2', True, 'foo/2/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
16 ('foo/bar', True, 'foo/bar/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
17 ('foo/bar/2', True, 'foo/bar/2/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
18 ('foo.ext', True, 'foo.ext/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
19 ('foo.ext/2', True, 'foo.ext/2/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
20 ('foo/bar.ext', True, 'foo/bar.ext/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
21 ('foo/bar.ext/2', True, 'foo/bar.ext/2/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
22 ('foo.bar.ext', True, 'foo.bar.ext/index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
23 ('foo.bar.ext/2', True, 'foo.bar.ext/2/index.html'), |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
24 # Ugly URLs |
262
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
25 ('', False, 'index.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
26 ('2.html', False, '2.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
27 ('foo.html', False, 'foo.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
28 ('foo/2.html', False, 'foo/2.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
29 ('foo/bar.html', False, 'foo/bar.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
30 ('foo/bar/2.html', False, 'foo/bar/2.html'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
31 ('foo.ext', False, 'foo.ext'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
32 ('foo/2.ext', False, 'foo/2.ext'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
33 ('foo/bar.ext', False, 'foo/bar.ext'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
34 ('foo/bar/2.ext', False, 'foo/bar/2.ext'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
35 ('foo.bar.ext', False, 'foo.bar.ext'), |
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
36 ('foo.bar/2.ext', False, 'foo.bar/2.ext') |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
37 ]) |
262
61145dcd56e0
routing: Better generate URLs according to the site configuration.
Ludovic Chabant <ludovic@chabant.com>
parents:
145
diff
changeset
|
38 def test_get_output_path(uri, pretty, expected): |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
39 app = get_mock_app() |
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
40 if pretty: |
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
41 app.config.set('site/pretty_urls', True) |
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
42 assert app.config.get('site/pretty_urls') == pretty |
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
43 |
329
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
324
diff
changeset
|
44 for site_root in ['/', '/whatever/']: |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
324
diff
changeset
|
45 app.config.set('site/root', site_root) |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
324
diff
changeset
|
46 baker = PageBaker(app, '/destination') |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
324
diff
changeset
|
47 path = baker.getOutputPath(site_root + uri) |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
324
diff
changeset
|
48 expected = os.path.normpath( |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
324
diff
changeset
|
49 os.path.join('/destination', expected)) |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
324
diff
changeset
|
50 assert expected == path |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
51 |
85
3471ffa059b2
Add a `BakeScheduler` to handle build dependencies. Add unit-tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
29
diff
changeset
|
52 |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
53 def test_removed(): |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
54 fs = (mock_fs() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
55 .withPage('pages/foo.md', {'layout': 'none', 'format': 'none'}, 'a foo page') |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
56 .withPage('pages/_index.md', {'layout': 'none', 'format': 'none'}, "something")) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
57 with mock_fs_scope(fs): |
145 | 58 out_dir = fs.path('kitchen/_counter') |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
59 app = fs.getApp() |
411
e7b865f8f335
bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents:
347
diff
changeset
|
60 app.config.set('baker/workers', 1) |
145 | 61 baker = Baker(app, out_dir) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
62 baker.bake() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
63 structure = fs.getStructure('kitchen/_counter') |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
64 assert structure == { |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
65 'foo.html': 'a foo page', |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
66 'index.html': 'something'} |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
67 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
68 os.remove(fs.path('kitchen/pages/foo.md')) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
69 app = fs.getApp() |
145 | 70 baker = Baker(app, out_dir) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
71 baker.bake() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
72 structure = fs.getStructure('kitchen/_counter') |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
73 assert structure == { |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
74 'index.html': 'something'} |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
75 |
324
65e6d72f3877
bake/serve: Fix how taxonomy index pages are setup and rendered.
Ludovic Chabant <ludovic@chabant.com>
parents:
286
diff
changeset
|
76 |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
77 def test_record_version_change(): |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
78 fs = (mock_fs() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
79 .withPage('pages/foo.md', {'layout': 'none', 'format': 'none'}, 'a foo page')) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
80 with mock_fs_scope(fs): |
145 | 81 out_dir = fs.path('kitchen/_counter') |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
82 app = fs.getApp() |
145 | 83 baker = Baker(app, out_dir) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
84 baker.bake() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
85 mtime = os.path.getmtime(fs.path('kitchen/_counter/foo.html')) |
286
a2d283d1033d
tests: Fixes for running on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents:
262
diff
changeset
|
86 time.sleep(1) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
87 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
88 app = fs.getApp() |
145 | 89 baker = Baker(app, out_dir) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
90 baker.bake() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
91 assert mtime == os.path.getmtime(fs.path('kitchen/_counter/foo.html')) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
92 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
93 BakeRecord.RECORD_VERSION += 1 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
94 try: |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
95 app = fs.getApp() |
145 | 96 baker = Baker(app, out_dir) |
120
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
97 baker.bake() |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
98 assert mtime < os.path.getmtime(fs.path('kitchen/_counter/foo.html')) |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
99 finally: |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
100 BakeRecord.RECORD_VERSION -= 1 |
133845647083
Better error management and removal support in baking/processing.
Ludovic Chabant <ludovic@chabant.com>
parents:
85
diff
changeset
|
101 |