Mercurial > piecrust2

annotate tests/test_pipelines_page.py @ 1188:a7c43131d871

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bake: Fix file write flushing problem with Python 3.8+ Writing the cache files fails in Python 3.8 because it looks like flushing behaviour has changed. We need to explicitly flush. And even then, in very rare occurrences, it looks like it can still run into racing conditions, so we do a very hacky and ugly "retry" loop when fetching cached data :(
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 15 Jun 2021 22:36:23 -0700
parents 45ad976712ec
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
974
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import time
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os.path
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import urllib.parse
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import pytest
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 from piecrust.pipelines.records import MultiRecord
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
6 from piecrust.pipelines._pagebaker import get_output_path
974
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7 from .mockutil import get_mock_app, mock_fs, mock_fs_scope
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10 @pytest.mark.parametrize('uri, pretty, expected', [
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11 # Pretty URLs
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12 ('', True, 'index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13 ('2', True, '2/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14 ('foo', True, 'foo/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 ('foo/2', True, 'foo/2/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16 ('foo/bar', True, 'foo/bar/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 ('foo/bar/2', True, 'foo/bar/2/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18 ('foo.ext', True, 'foo.ext/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19 ('foo.ext/2', True, 'foo.ext/2/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20 ('foo/bar.ext', True, 'foo/bar.ext/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21 ('foo/bar.ext/2', True, 'foo/bar.ext/2/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
22 ('foo.bar.ext', True, 'foo.bar.ext/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 ('foo.bar.ext/2', True, 'foo.bar.ext/2/index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24 # Ugly URLs
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25 ('', False, 'index.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26 ('2.html', False, '2.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27 ('foo.html', False, 'foo.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28 ('foo/2.html', False, 'foo/2.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
29 ('foo/bar.html', False, 'foo/bar.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30 ('foo/bar/2.html', False, 'foo/bar/2.html'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
31 ('foo.ext', False, 'foo.ext'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
32 ('foo/2.ext', False, 'foo/2.ext'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33 ('foo/bar.ext', False, 'foo/bar.ext'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34 ('foo/bar/2.ext', False, 'foo/bar/2.ext'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
35 ('foo.bar.ext', False, 'foo.bar.ext'),
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
36 ('foo.bar/2.ext', False, 'foo.bar/2.ext')
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
37 ])
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
38 def test_get_output_path(uri, pretty, expected):
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
39 app = get_mock_app()
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
40 if pretty:
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
41 app.config.set('site/pretty_urls', True)
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
42 assert app.config.get('site/pretty_urls') == pretty
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
43
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
44 out_dir = '/destination'
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
45
974
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
46 for site_root in ['/', '/whatever/', '/~johndoe/']:
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47 app.config.set('site/root', urllib.parse.quote(site_root))
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
48 path = get_output_path(app, out_dir,
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
49 urllib.parse.quote(site_root) + uri,
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
50 pretty)
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
51 expected = os.path.normpath(
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
52 os.path.join('/destination', expected))
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
53 assert expected == path
974
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
54
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
55
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
56 def test_removed():
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
57 fs = (mock_fs()
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
58 .withConfig()
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
59 .withPage('pages/foo.md', {'layout': 'none', 'format': 'none'},
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
60 "a foo page")
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
61 .withPage('pages/_index.md', {'layout': 'none', 'format': 'none'},
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
62 "something"))
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
63 with mock_fs_scope(fs):
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
64 fs.runChef('bake')
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
65 structure = fs.getStructure('kitchen/_counter')
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
66 assert structure == {
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
67 'foo.html': 'a foo page',
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
68 'index.html': 'something'}
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
69
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
70 os.remove(fs.path('kitchen/pages/foo.md'))
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
71 fs.runChef('bake')
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
72 structure = fs.getStructure('kitchen/_counter')
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73 assert structure == {
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 'index.html': 'something'}
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
75
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
76
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
77 def test_record_version_change():
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
78 fs = (mock_fs()
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
79 .withConfig()
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80 .withPage('pages/foo.md', {'layout': 'none', 'format': 'none'},
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81 'a foo page'))
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82 with mock_fs_scope(fs):
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
83 time.sleep(1)
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
84 fs.runChef('bake', '-o', fs.path('counter'))
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
85 time.sleep(0.1)
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
86 mtime = os.path.getmtime(fs.path('counter/foo.html'))
974
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
87
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
88 time.sleep(1)
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
89 fs.runChef('bake', '-o', fs.path('counter'))
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
90 time.sleep(0.1)
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
91 assert mtime == os.path.getmtime(fs.path('counter/foo.html'))
974
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
92
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
93 MultiRecord.RECORD_VERSION += 1
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
94 try:
979
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
95 time.sleep(1)
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
96 fs.runChef('bake', '-o', fs.path('counter'))
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
97 time.sleep(0.1)
45ad976712ec tests: Big push to get the tests to pass again.
Ludovic Chabant <ludovic@chabant.com>
parents: 974
diff changeset
98 assert mtime < os.path.getmtime(fs.path('counter/foo.html'))
974
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
99 finally:
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
100 MultiRecord.RECORD_VERSION -= 1
72f17534d58e tests: First pass on making unit tests work again.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
101