Mercurial > piecrust2
annotate piecrust/uriutil.py @ 415:0e9a94b7fdfa
bake: Improve bake record information.
* Store things in the bake record that require less interaction between the
master process and the workers. For instance, don't store the paginator
object in the render pass info -- instead, just store whether pagination
was used, and whether it had more items.
* Simplify information passing between workers and bake passes by saving the
rendering info to the JSON cache. This means the "render first sub" job
doesn't have to return anything except errors now.
* Add more performance counter info.
author | Ludovic Chabant <ludovic@chabant.com> |
---|---|
date | Sat, 20 Jun 2015 19:23:16 -0700 |
parents | 422052d2e978 |
children | d4321317beae |
rev | line source |
---|---|
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
1 import re |
33
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
2 import os.path |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
3 import string |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
4 import logging |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
5 import functools |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
6 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
7 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
8 logger = logging.getLogger(__name__) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
9 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
10 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
11 class UriError(Exception): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
12 def __init__(self, uri): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
13 super(UriError, self).__init__("Invalid URI: %s" % uri) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
14 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
15 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
16 @functools.total_ordering |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
17 class UriInfo(object): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
18 def __init__(self, uri, source, args, taxonomy=None, page_num=1): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
19 self.uri = uri |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
20 self.source = source |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
21 self.args = args |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
22 self.taxonomy = taxonomy |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
23 self.page_num = page_num |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
24 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
25 def __eq__(self, other): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
26 return ((self.uri, self.source, self.args, self.taxonomy, |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
27 self.page_num) == |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
28 (other.uri, other.source, other.args, other.taxonomy, |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
29 other.page_num)) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
30 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
31 def __lt__(self, other): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
32 return ((self.uri, self.source, self.args, self.taxonomy, |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
33 self.page_num) < |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
34 (other.uri, other.source, other.args, other.taxonomy, |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
35 other.page_num)) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
36 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
37 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
38 pagenum_pattern = re.compile(r'/(\d+)/?$') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
39 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
40 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
41 def parse_uri(routes, uri): |
6
f5ca5c5bed85
More Python 3 fixes, modularization, and new unit tests.
Ludovic Chabant <ludovic@chabant.com>
parents:
5
diff
changeset
|
42 if uri.find('..') >= 0: |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
43 raise UriError(uri) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
44 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
45 page_num = 1 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
46 match = pagenum_pattern.search(uri) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
47 if match is not None: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
48 uri = uri[:match.start()] |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
49 page_num = int(match.group(1)) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
50 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
51 uri = '/' + uri.strip('/') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
52 |
5 | 53 for rn, rc in routes.items(): |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
54 pattern = route_to_pattern(rn) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
55 m = re.match(pattern, uri) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
56 if m is not None: |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
57 args = m.groupdict() |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
58 return UriInfo(uri, rc['source'], args, rc.get('taxonomy'), |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
59 page_num) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
60 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
61 return None |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
62 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
63 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
64 r2p_pattern = re.compile(r'%(\w+)%') |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
65 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
66 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
67 def route_to_pattern(route): |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
68 return r2p_pattern.sub(r'(?P<\1>[\w\-]+)', route) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
69 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
70 |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
71 def multi_replace(text, replacements): |
5 | 72 reps = dict((re.escape(k), v) for k, v in replacements.items()) |
73 pattern = re.compile("|".join(list(reps.keys()))) | |
3
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
74 return pattern.sub(lambda m: reps[re.escape(m.group(0))], text) |
f485ba500df3
Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
75 |
12
30a42341cfa8
Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents:
6
diff
changeset
|
76 |
329
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
77 def split_uri(app, uri): |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
78 root = app.config.get('site/root') |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
79 uri_root = uri[:len(root)] |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
80 if uri_root != root: |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
81 raise Exception("URI '%s' is not a full URI." % uri) |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
82 uri = uri[len(root):] |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
83 return uri_root, uri |
12
30a42341cfa8
Define page slugs properly, avoid recursions with debug data.
Ludovic Chabant <ludovic@chabant.com>
parents:
6
diff
changeset
|
84 |
33
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
85 |
298
b7ab1b503510
data: Fix incorrect next/previous page URLs in pagination data.
Ludovic Chabant <ludovic@chabant.com>
parents:
33
diff
changeset
|
86 def split_sub_uri(app, uri): |
329
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
87 root = app.config.get('site/root') |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
88 if not uri.startswith(root): |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
89 raise Exception("URI '%s' is not a full URI." % uri) |
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
90 |
33
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
91 pretty_urls = app.config.get('site/pretty_urls') |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
92 if not pretty_urls: |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
93 uri, ext = os.path.splitext(uri) |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
94 |
298
b7ab1b503510
data: Fix incorrect next/previous page URLs in pagination data.
Ludovic Chabant <ludovic@chabant.com>
parents:
33
diff
changeset
|
95 page_num = 1 |
33
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
96 pgn_suffix_re = app.config.get('__cache/pagination_suffix_re') |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
97 m = re.search(pgn_suffix_re, uri) |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
98 if m: |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
99 uri = uri[:m.start()] |
298
b7ab1b503510
data: Fix incorrect next/previous page URLs in pagination data.
Ludovic Chabant <ludovic@chabant.com>
parents:
33
diff
changeset
|
100 if uri == '': |
b7ab1b503510
data: Fix incorrect next/previous page URLs in pagination data.
Ludovic Chabant <ludovic@chabant.com>
parents:
33
diff
changeset
|
101 uri = '/' |
b7ab1b503510
data: Fix incorrect next/previous page URLs in pagination data.
Ludovic Chabant <ludovic@chabant.com>
parents:
33
diff
changeset
|
102 page_num = int(m.group('num')) |
33
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
103 |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
104 if not pretty_urls: |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
105 uri += ext |
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
106 |
329
422052d2e978
internal: Try handling URLs in a consistent way.
Ludovic Chabant <ludovic@chabant.com>
parents:
298
diff
changeset
|
107 return uri, page_num |
33
62c7a97c8340
Get the un-paginated URL of a page early and pass that around.
Ludovic Chabant <ludovic@chabant.com>
parents:
12
diff
changeset
|
108 |