annotate piecrust/serving.py @ 369:4b1019bb2533

serve: Giant refactor to change how we handle data when serving pages. * We need a distinction between source metadata and route metadata. In most cases they're the same, but in cases like taxonomy pages, route metadata contains more things that can't be in source metadata if we want to re-use cached pages. * Create a new `QualifiedPage` type which is a page with a specific route and route metadata. Pass this around in many places. * Instead of passing an URL around, use the route in the `QualifiedPage` to generate URLs. This is better since it removes the guess-work from trying to generate URLs for sub-pages. * Deep-copy app and page configurations before passing them around to things that could modify them, like data builders and such. * Exclude taxonomy pages from iterator data providers. * Properly nest iterator data providers for when the theme and user page sources are merged inside `site.pages`.
author Ludovic Chabant <ludovic@chabant.com>
date Sun, 03 May 2015 18:47:10 -0700
parents dd25bd3ce1f9
children c2ca72fb7f0b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
1 import io
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
2 import os
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import re
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
4 import json
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import gzip
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6 import time
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
7 import queue
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8 import os.path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10 import logging
258
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
11 import datetime
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
12 import threading
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
13 from werkzeug.exceptions import (
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
14 NotFound, MethodNotAllowed, InternalServerError, HTTPException)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 from werkzeug.wrappers import Request, Response
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
16 from werkzeug.wsgi import ClosingIterator, wrap_file
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 from jinja2 import FileSystemLoader, Environment
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18 from piecrust.app import PieCrust
111
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 110
diff changeset
19 from piecrust.environment import StandardEnvironment
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20 from piecrust.processing.base import ProcessorPipeline
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
21 from piecrust.rendering import QualifiedPage, PageRenderingContext, render_page
11
617191dec18e Fixes for Windows, make `findPagePath` return a ref path.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
22 from piecrust.sources.base import PageFactory, MODE_PARSING
298
b7ab1b503510 data: Fix incorrect next/previous page URLs in pagination data.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
23 from piecrust.uriutil import split_sub_uri
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
28
224
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running.
Ludovic Chabant <ludovic@chabant.com>
parents: 223
diff changeset
29 _sse_abort = threading.Event()
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running.
Ludovic Chabant <ludovic@chabant.com>
parents: 223
diff changeset
30
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running.
Ludovic Chabant <ludovic@chabant.com>
parents: 223
diff changeset
31
111
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 110
diff changeset
32 class ServingEnvironment(StandardEnvironment):
113
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
33 pass
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
34
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
35
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
36 class ServeRecord(object):
111
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 110
diff changeset
37 def __init__(self):
113
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
38 self.entries = {}
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
39
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
40 def addEntry(self, entry):
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
41 key = self._makeKey(entry.uri, entry.sub_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
42 self.entries[key] = entry
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
43
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
44 def getEntry(self, uri, sub_num):
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
45 key = self._makeKey(uri, sub_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
46 return self.entries.get(key)
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
47
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
48 def _makeKey(self, uri, sub_num):
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
49 return "%s:%s" % (uri, sub_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
50
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
51
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
52 class ServeRecordPageEntry(object):
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
53 def __init__(self, uri, sub_num):
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
54 self.uri = uri
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
55 self.sub_num = sub_num
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
56 self.used_source_names = set()
111
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 110
diff changeset
57
208c652551a3 Quick fix for making the server correctly update referenced pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 110
diff changeset
58
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
59 class WsgiServerWrapper(object):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
60 def __init__(self, server):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
61 self.server = server
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
62
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
63 def __call__(self, environ, start_response):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
64 return self.server._run_request(environ, start_response)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
65
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
66
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
67 class Server(object):
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
68 def __init__(self, root_dir,
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
69 debug=False, use_reloader=False, static_preview=True):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
70 self.root_dir = root_dir
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
71 self.debug = debug
155
70b86e904b85 Properly use, or not, the debugging when using the chef server.
Ludovic Chabant <ludovic@chabant.com>
parents: 141
diff changeset
72 self.use_reloader = use_reloader
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73 self.static_preview = static_preview
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 self._out_dir = None
113
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
75 self._page_record = None
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
76 self._proc_loop = None
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
77 self._mimetype_map = load_mimetype_map()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
78
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
79 def getWsgiApp(self):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80 # Bake all the assets so we know what we have, and so we can serve
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81 # them to the client. We need a temp app for this.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82 app = PieCrust(root_dir=self.root_dir, debug=self.debug)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
83 self._out_dir = os.path.join(app.cache_dir, 'server')
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
84 self._page_record = ServeRecord()
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
85
214
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
86 if (not self.use_reloader or
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
87 os.environ.get('WERKZEUG_RUN_MAIN') == 'true'):
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
88 # We don't want to run the processing loop here if this isn't
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
89 # the actual process that does the serving. In most cases it is,
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
90 # but if we're using Werkzeug's reloader, then it won't be the
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
91 # first time we get there... it will only be the correct process
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
92 # the second time, when the reloading process is spawned, with the
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
93 # `WERKZEUG_RUN_MAIN` variable set.
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`.
Ludovic Chabant <ludovic@chabant.com>
parents: 209
diff changeset
94 pipeline = ProcessorPipeline(app, self._out_dir)
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
95 self._proc_loop = ProcessingLoop(pipeline)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
96 self._proc_loop.start()
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
97
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
98 # Run the WSGI app.
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
99 wsgi_wrapper = WsgiServerWrapper(self)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
100 return wsgi_wrapper
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
101
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
102 def _run_request(self, environ, start_response):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
103 try:
200
76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs.
Ludovic Chabant <ludovic@chabant.com>
parents: 190
diff changeset
104 return self._try_run_request(environ, start_response)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
105 except Exception as ex:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
106 if self.debug:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
107 raise
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
108 return self._handle_error(ex, environ, start_response)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
109
200
76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs.
Ludovic Chabant <ludovic@chabant.com>
parents: 190
diff changeset
110 def _try_run_request(self, environ, start_response):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
111 request = Request(environ)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
112
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
113 # We don't support anything else than GET requests since we're
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
114 # previewing something that will be static later.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
115 if self.static_preview and request.method != 'GET':
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
116 logger.error("Only GET requests are allowed, got %s" %
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
117 request.method)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
118 raise MethodNotAllowed()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
120 # Handle special requests right away.
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
121 response = self._try_special_request(environ, request)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
122 if response is not None:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
123 return response(environ, start_response)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
124
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
125 # Also handle requests to a pipeline-built asset right away.
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
126 response = self._try_serve_asset(environ, request)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
127 if response is not None:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
128 return response(environ, start_response)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
129
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
130 # Create the app for this request.
223
7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`.
Ludovic Chabant <ludovic@chabant.com>
parents: 219
diff changeset
131 app = PieCrust(root_dir=self.root_dir, debug=self.debug)
53
73956224eb67 Setup the server better.
Ludovic Chabant <ludovic@chabant.com>
parents: 39
diff changeset
132 app.config.set('site/root', '/')
73956224eb67 Setup the server better.
Ludovic Chabant <ludovic@chabant.com>
parents: 39
diff changeset
133 app.config.set('server/is_serving', True)
223
7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`.
Ludovic Chabant <ludovic@chabant.com>
parents: 219
diff changeset
134 if (app.config.get('site/enable_debug_info') and
7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`.
Ludovic Chabant <ludovic@chabant.com>
parents: 219
diff changeset
135 '!debug' in request.args):
7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`.
Ludovic Chabant <ludovic@chabant.com>
parents: 219
diff changeset
136 app.config.set('site/show_debug_info', True)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
137
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
138 # We'll serve page assets directly from where they are.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
139 app.env.base_asset_url_format = '/_asset/%path%'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
140
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
141 # Let's see if it can be a page asset.
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
142 response = self._try_serve_page_asset(app, environ, request)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
143 if response is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
144 return response(environ, start_response)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
145
133
9e4c2e68a129 Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents: 130
diff changeset
146 # Nope. Let's see if it's an actual page.
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
147 try:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
148 response = self._try_serve_page(app, environ, request)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
149 return response(environ, start_response)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
150 except (RouteNotFoundError, SourceNotFoundError) as ex:
229
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
151 raise NotFound(str(ex)) from ex
133
9e4c2e68a129 Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents: 130
diff changeset
152 except HTTPException:
9e4c2e68a129 Optimize server for files that already exist.
Ludovic Chabant <ludovic@chabant.com>
parents: 130
diff changeset
153 raise
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
154 except Exception as ex:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
155 if app.debug:
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
156 logger.exception(ex)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
157 raise
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
158 msg = str(ex)
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
159 logger.error(msg)
229
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
160 raise InternalServerError(msg) from ex
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
161
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
162 def _try_special_request(self, environ, request):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
163 static_mount = '/__piecrust_static/'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
164 if request.path.startswith(static_mount):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
165 rel_req_path = request.path[len(static_mount):]
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
166 mount = os.path.join(
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
167 os.path.dirname(__file__),
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
168 'resources', 'server')
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
169 full_path = os.path.join(mount, rel_req_path)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
170 try:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
171 response = self._make_wrapped_file_response(
258
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
172 environ, request, full_path)
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
173 return response
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
174 except OSError:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
175 pass
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
176
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
177 debug_mount = '/__piecrust_debug/'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
178 if request.path.startswith(debug_mount):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
179 rel_req_path = request.path[len(debug_mount):]
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
180 if rel_req_path == 'pipeline_status':
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
181 provider = PipelineStatusServerSideEventProducer(
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
182 self._proc_loop.status_queue)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
183 it = ClosingIterator(provider.run(), [provider.close])
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
184 response = Response(it)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
185 response.headers['Cache-Control'] = 'no-cache'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
186 if 'text/event-stream' in request.accept_mimetypes:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
187 response.mimetype = 'text/event-stream'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
188 response.direct_passthrough = True
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
189 response.implicit_sequence_conversion = False
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
190 return response
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
191
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
192 return None
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
193
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
194 def _try_serve_asset(self, environ, request):
11
617191dec18e Fixes for Windows, make `findPagePath` return a ref path.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
195 rel_req_path = request.path.lstrip('/').replace('/', os.sep)
240
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 232
diff changeset
196 if request.path.startswith('/_cache/'):
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 232
diff changeset
197 # Some stuff needs to be served directly from the cache directory,
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 232
diff changeset
198 # like LESS CSS map files.
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 232
diff changeset
199 full_path = os.path.join(self.root_dir, rel_req_path)
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 232
diff changeset
200 else:
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 232
diff changeset
201 full_path = os.path.join(self._out_dir, rel_req_path)
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 232
diff changeset
202
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
203 try:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
204 response = self._make_wrapped_file_response(
258
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
205 environ, request, full_path)
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
206 return response
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
207 except OSError:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
208 pass
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
209 return None
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
210
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
211 def _try_serve_page_asset(self, app, environ, request):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
212 if not request.path.startswith('/_asset/'):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
213 return None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
214
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
215 full_path = os.path.join(app.root_dir, request.path[len('/_asset/'):])
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
216 if not os.path.isfile(full_path):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
217 return None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
218
258
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
219 return self._make_wrapped_file_response(environ, request, full_path)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
220
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
221 def _try_serve_page(self, app, environ, request):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
222 # Try to find what matches the requested URL.
298
b7ab1b503510 data: Fix incorrect next/previous page URLs in pagination data.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
223 req_path, page_num = split_sub_uri(app, request.path)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
224
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
225 routes = find_routes(app.routes, req_path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
226 if len(routes) == 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
227 raise RouteNotFoundError("Can't find route for: %s" % req_path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
228
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
229 taxonomy = None
334
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 330
diff changeset
230 tax_terms = None
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
231 for route, route_metadata in routes:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
232 source = app.getSource(route.source_name)
334
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 330
diff changeset
233 if route.taxonomy_name is None:
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 343
diff changeset
234 factory = source.findPageFactory(route_metadata, MODE_PARSING)
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 343
diff changeset
235 if factory is not None:
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
236 break
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
237 else:
334
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 330
diff changeset
238 taxonomy = app.getTaxonomy(route.taxonomy_name)
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 330
diff changeset
239 route_terms = route_metadata.get(taxonomy.term_name)
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 330
diff changeset
240 if route_terms is not None:
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
241 tax_page_ref = taxonomy.getPageRef(source.name)
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 343
diff changeset
242 factory = tax_page_ref.getFactory()
334
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 330
diff changeset
243 tax_terms = route.unslugifyTaxonomyTerm(route_terms)
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 343
diff changeset
244 factory.metadata[taxonomy.term_name] = tax_terms
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
245 break
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
246 else:
229
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
247 raise SourceNotFoundError(
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
248 "Can't find path for: %s (looked in: %s)" %
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
249 (req_path, [r.source_name for r, _ in routes]))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
250
113
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
251 # Build the page.
363
dd25bd3ce1f9 serve: Refactoring and fixes to be able to serve taxonomy pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 343
diff changeset
252 page = factory.buildPage()
174
e9a3d405e18f serve: Always force render the page being previewed.
Ludovic Chabant <ludovic@chabant.com>
parents: 173
diff changeset
253 # We force the rendering of the page because it could not have
e9a3d405e18f serve: Always force render the page being previewed.
Ludovic Chabant <ludovic@chabant.com>
parents: 173
diff changeset
254 # changed, but include pages that did change.
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
255 qp = QualifiedPage(page, route, route_metadata)
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
256 render_ctx = PageRenderingContext(qp,
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 363
diff changeset
257 page_num=page_num,
174
e9a3d405e18f serve: Always force render the page being previewed.
Ludovic Chabant <ludovic@chabant.com>
parents: 173
diff changeset
258 force_render=True)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
259 if taxonomy is not None:
334
b034f6f15e22 bake: Several bug taxonomy-related fixes for incorrect incremental bakes.
Ludovic Chabant <ludovic@chabant.com>
parents: 330
diff changeset
260 render_ctx.setTaxonomyFilter(taxonomy, tax_terms)
113
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
261
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
262 # See if this page is known to use sources. If that's the case,
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
263 # just don't use cached rendered segments for that page (but still
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
264 # use them for pages that are included in it).
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
265 entry = self._page_record.getEntry(req_path, page_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
266 if (taxonomy is not None or entry is None or
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
267 entry.used_source_names):
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
268 cache_key = '%s:%s' % (req_path, page_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
269 app.env.rendered_segments_repository.invalidate(cache_key)
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
270
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
271 # Render the page.
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
272 rendered_page = render_page(render_ctx)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
273 rp_content = rendered_page.content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
274
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
275 if taxonomy is not None:
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
276 paginator = rendered_page.data.get('pagination')
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
277 if (paginator and paginator.is_loaded and
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
278 len(paginator.items) == 0):
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
279 message = ("This URL matched a route for taxonomy '%s' but "
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
280 "no pages have been found to have it. This page "
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
281 "won't be generated by a bake." % taxonomy.name)
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
282 raise NotFound(message)
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
283
113
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
284 if entry is None:
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
285 entry = ServeRecordPageEntry(req_path, page_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
286 self._page_record.addEntry(entry)
343
741e97e63048 serve: Don't access the current render pass info after rendering is done.
Ludovic Chabant <ludovic@chabant.com>
parents: 338
diff changeset
287 for p, pinfo in render_ctx.render_passes.items():
741e97e63048 serve: Don't access the current render pass info after rendering is done.
Ludovic Chabant <ludovic@chabant.com>
parents: 338
diff changeset
288 entry.used_source_names |= pinfo.used_source_names
113
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
289
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
290 # Profiling.
279
980bbbd0705e serve: Correctly show timing info even when not in debug mode.
Ludovic Chabant <ludovic@chabant.com>
parents: 258
diff changeset
291 if app.config.get('site/show_debug_info'):
7
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
292 now_time = time.clock()
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
293 timing_info = ('%8.1f ms' %
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
294 ((now_time - app.env.start_time) * 1000.0))
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
295 rp_content = rp_content.replace('__PIECRUST_TIMING_INFORMATION__',
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
296 timing_info)
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
297
113
de257cc40ce1 Re-enable proper caching of rendered segments in server.
Ludovic Chabant <ludovic@chabant.com>
parents: 111
diff changeset
298 # Build the response.
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
299 response = Response()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
300
7
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
301 etag = hashlib.md5(rp_content.encode('utf8')).hexdigest()
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
302 if not app.debug and etag in request.if_none_match:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
303 response.status_code = 304
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
304 return response
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
305
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
306 response.set_etag(etag)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
307 response.content_md5 = etag
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
308
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
309 cache_control = response.cache_control
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
310 if app.debug:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
311 cache_control.no_cache = True
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
312 cache_control.must_revalidate = True
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
313 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
314 cache_time = (page.config.get('cache_time') or
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
315 app.config.get('site/cache_time'))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
316 if cache_time:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
317 cache_control.public = True
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
318 cache_control.max_age = cache_time
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
319
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
320 content_type = page.config.get('content_type')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
321 if content_type and '/' not in content_type:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
322 mimetype = content_type_map.get(content_type, content_type)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
323 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
324 mimetype = content_type
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
325 if mimetype:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
326 response.mimetype = mimetype
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
327
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
328 if ('gzip' in request.accept_encodings and
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
329 app.config.get('site/enable_gzip')):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
330 try:
7
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
331 with io.BytesIO() as gzip_buffer:
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
332 with gzip.open(gzip_buffer, mode='wt',
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
333 encoding='utf8') as gzip_file:
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
334 gzip_file.write(rp_content)
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
335 rp_content = gzip_buffer.getvalue()
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes.
Ludovic Chabant <ludovic@chabant.com>
parents: 5
diff changeset
336 response.content_encoding = 'gzip'
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
337 except Exception:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
338 logger.exception("Error compressing response, "
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
339 "falling back to uncompressed.")
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
340 response.set_data(rp_content)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
341
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
342 return response
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
343
258
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
344 def _make_wrapped_file_response(self, environ, request, path):
190
430ee5b80962 serve: Make the server find assets generated by external tools.
Ludovic Chabant <ludovic@chabant.com>
parents: 176
diff changeset
345 logger.debug("Serving %s" % path)
258
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
346
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
347 # Check if we can return a 304 status code.
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
348 mtime = os.path.getmtime(path)
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
349 etag_str = '%s$$%s' % (path, mtime)
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
350 etag = hashlib.md5(etag_str.encode('utf8')).hexdigest()
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
351 if etag in request.if_none_match:
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
352 response = Response()
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
353 response.status_code = 304
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
354 return response
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
355
190
430ee5b80962 serve: Make the server find assets generated by external tools.
Ludovic Chabant <ludovic@chabant.com>
parents: 176
diff changeset
356 wrapper = wrap_file(environ, open(path, 'rb'))
430ee5b80962 serve: Make the server find assets generated by external tools.
Ludovic Chabant <ludovic@chabant.com>
parents: 176
diff changeset
357 response = Response(wrapper)
430ee5b80962 serve: Make the server find assets generated by external tools.
Ludovic Chabant <ludovic@chabant.com>
parents: 176
diff changeset
358 _, ext = os.path.splitext(path)
258
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
359 response.set_etag(etag)
7ec06ec14247 serve: Use Etags and 304 responses for assets.
Ludovic Chabant <ludovic@chabant.com>
parents: 256
diff changeset
360 response.last_modified = datetime.datetime.fromtimestamp(mtime)
190
430ee5b80962 serve: Make the server find assets generated by external tools.
Ludovic Chabant <ludovic@chabant.com>
parents: 176
diff changeset
361 response.mimetype = self._mimetype_map.get(
430ee5b80962 serve: Make the server find assets generated by external tools.
Ludovic Chabant <ludovic@chabant.com>
parents: 176
diff changeset
362 ext.lstrip('.'), 'text/plain')
430ee5b80962 serve: Make the server find assets generated by external tools.
Ludovic Chabant <ludovic@chabant.com>
parents: 176
diff changeset
363 return response
430ee5b80962 serve: Make the server find assets generated by external tools.
Ludovic Chabant <ludovic@chabant.com>
parents: 176
diff changeset
364
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
365 def _handle_error(self, exception, environ, start_response):
200
76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs.
Ludovic Chabant <ludovic@chabant.com>
parents: 190
diff changeset
366 code = 500
128
28444014ce7d Fix error reporting and counting of lines.
Ludovic Chabant <ludovic@chabant.com>
parents: 113
diff changeset
367 if isinstance(exception, HTTPException):
200
76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs.
Ludovic Chabant <ludovic@chabant.com>
parents: 190
diff changeset
368 code = exception.code
229
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
369
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
370 path = 'error'
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
371 if isinstance(exception, NotFound):
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
372 path += '404'
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
373
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
374 descriptions = self._get_exception_descriptions(exception)
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
375
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
376 env = Environment(loader=ErrorMessageLoader())
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
377 template = env.get_template(path)
229
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
378 context = {'details': descriptions}
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
379 response = Response(template.render(context), mimetype='text/html')
200
76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs.
Ludovic Chabant <ludovic@chabant.com>
parents: 190
diff changeset
380 response.status_code = code
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
381 return response(environ, start_response)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
382
229
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
383 def _get_exception_descriptions(self, exception):
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
384 desc = []
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
385 while exception is not None:
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
386 if isinstance(exception, HTTPException):
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
387 desc.append(exception.description)
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
388 else:
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
389 desc.append(str(exception))
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
390
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
391 inner_ex = exception.__cause__
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
392 if inner_ex is None:
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
393 inner_ex = exception.__context__
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
394 exception = inner_ex
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
395 return desc
a951cd4ef361 serve: Print nested exception messages in the dev server.
Ludovic Chabant <ludovic@chabant.com>
parents: 224
diff changeset
396
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
397
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
398 class RouteNotFoundError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
399 pass
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
400
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
401
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
402 class SourceNotFoundError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
403 pass
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
404
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
405
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
406 content_type_map = {
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
407 'html': 'text/html',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
408 'xml': 'text/xml',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
409 'txt': 'text/plain',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
410 'text': 'text/plain',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
411 'css': 'text/css',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
412 'xhtml': 'application/xhtml+xml',
232
e534d2bc657c cosmetic: Fix PEP8 spacing.
Ludovic Chabant <ludovic@chabant.com>
parents: 231
diff changeset
413 'atom': 'application/atom+xml', # or 'text/xml'?
e534d2bc657c cosmetic: Fix PEP8 spacing.
Ludovic Chabant <ludovic@chabant.com>
parents: 231
diff changeset
414 'rss': 'application/rss+xml', # or 'text/xml'?
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
415 'json': 'application/json'}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
416
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
417
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
418 def find_routes(routes, uri):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
419 res = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
420 for route in routes:
176
d47d9493bb0a routes: When matching URIs, return metadata directly instead of the match object.
Ludovic Chabant <ludovic@chabant.com>
parents: 174
diff changeset
421 metadata = route.matchUri(uri)
231
137bcd498ef9 serve: Fix a bug where empty route metadata is not the same as invalid route.
Ludovic Chabant <ludovic@chabant.com>
parents: 229
diff changeset
422 if metadata is not None:
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
423 res.append((route, metadata))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
424 return res
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
425
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
426
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
427 class ErrorMessageLoader(FileSystemLoader):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
428 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
429 base_dir = os.path.join(os.path.dirname(__file__), 'resources',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
430 'messages')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
431 super(ErrorMessageLoader, self).__init__(base_dir)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
432
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
433 def get_source(self, env, template):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
434 template += '.html'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
435 return super(ErrorMessageLoader, self).get_source(env, template)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
436
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
437
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
438 def load_mimetype_map():
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
439 mimetype_map = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
440 sep_re = re.compile(r'\s+')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
441 path = os.path.join(os.path.dirname(__file__), 'mime.types')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
442 with open(path, 'r') as f:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
443 for line in f:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
444 tokens = sep_re.split(line)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
445 if len(tokens) > 1:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
446 for t in tokens[1:]:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
447 mimetype_map[t] = tokens[0]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
448 return mimetype_map
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
449
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
450
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
451 class PipelineStatusServerSideEventProducer(object):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
452 def __init__(self, status_queue):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
453 self.status_queue = status_queue
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
454 self.interval = 2
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
455 self.timeout = 60*10
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
456 self._start_time = 0
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
457
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
458 def run(self):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
459 logger.debug("Starting pipeline status SSE.")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
460 self._start_time = time.time()
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
461
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
462 outstr = 'event: ping\ndata: started\n\n'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
463 yield bytes(outstr, 'utf8')
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
464
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
465 count = 0
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
466 while True:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
467 if time.time() > self.timeout + self._start_time:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
468 logger.debug("Closing pipeline status SSE, timeout reached.")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
469 outstr = 'event: pipeline_timeout\ndata: bye\n\n'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
470 yield bytes(outstr, 'utf8')
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
471 break
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
472
224
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running.
Ludovic Chabant <ludovic@chabant.com>
parents: 223
diff changeset
473 if _sse_abort.is_set():
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running.
Ludovic Chabant <ludovic@chabant.com>
parents: 223
diff changeset
474 break
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running.
Ludovic Chabant <ludovic@chabant.com>
parents: 223
diff changeset
475
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
476 try:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
477 logger.debug("Polling pipeline status queue...")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
478 count += 1
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
479 data = self.status_queue.get(True, self.interval)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
480 except queue.Empty:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
481 if count < 3:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
482 continue
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
483 data = {'type': 'ping', 'message': 'ping'}
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
484 count = 0
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
485
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
486 event_type = data['type']
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
487 outstr = 'event: %s\ndata: %s\n\n' % (
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
488 event_type, json.dumps(data))
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
489 logger.debug("Sending pipeline status SSE.")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
490 yield bytes(outstr, 'utf8')
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
491
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
492 def close(self):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
493 logger.debug("Closing pipeline status SSE.")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
494
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
495
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
496 class ProcessingLoop(threading.Thread):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
497 def __init__(self, pipeline):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
498 super(ProcessingLoop, self).__init__(
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
499 name='pipeline-reloader', daemon=True)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
500 self.pipeline = pipeline
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
501 self.status_queue = queue.Queue()
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
502 self.interval = 1
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
503 self._paths = set()
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
504 self._record = None
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
505 self._last_bake = 0
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
506
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
507 def run(self):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
508 # Build the first list of known files and run the pipeline once.
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
509 app = self.pipeline.app
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
510 roots = [os.path.join(app.root_dir, r)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
511 for r in self.pipeline.mounts.keys()]
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
512 for root in roots:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
513 for dirpath, dirnames, filenames in os.walk(root):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
514 self._paths |= set([os.path.join(dirpath, f)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
515 for f in filenames])
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
516 self._last_bake = time.time()
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
517 self._record = self.pipeline.run(save_record=False)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
518
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
519 while True:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
520 for root in roots:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
521 # For each mount root we try to find the first new or
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
522 # modified file. If any, we just run the pipeline on
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
523 # that mount.
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
524 found_new_or_modified = False
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
525 for dirpath, dirnames, filenames in os.walk(root):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
526 for filename in filenames:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
527 path = os.path.join(dirpath, filename)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
528 if path not in self._paths:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
529 logger.debug("Found new asset: %s" % path)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
530 self._paths.add(path)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
531 found_new_or_modified = True
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
532 break
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
533 if os.path.getmtime(path) > self._last_bake:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
534 logger.debug("Found modified asset: %s" % path)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
535 found_new_or_modified = True
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
536 break
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
537
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
538 if found_new_or_modified:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
539 break
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
540
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
541 if found_new_or_modified:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
542 self._runPipeline(root)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
543
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
544 time.sleep(self.interval)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
545
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
546 def _runPipeline(self, root):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
547 self._last_bake = time.time()
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
548 try:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
549 self._record = self.pipeline.run(
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
550 root,
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
551 previous_record=self._record,
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
552 save_record=False)
219
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
553
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
554 # Update the status queue.
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
555 # (we need to clear it because there may not be a consumer
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
556 # on the other side, if the user isn't running with the
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
557 # debug window active)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
558 while True:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
559 try:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
560 self.status_queue.get_nowait()
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
561 except queue.Empty:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
562 break
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
563
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
564 if self._record.success:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
565 item = {
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
566 'type': 'pipeline_success'}
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
567 self.status_queue.put_nowait(item)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
568 else:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
569 item = {
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
570 'type': 'pipeline_error',
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
571 'assets': []}
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
572 for entry in self._record.entries:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
573 if entry.errors:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
574 asset_item = {
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
575 'path': entry.rel_input,
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
576 'errors': list(entry.errors)}
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
577 item['assets'].append(asset_item)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window.
Ludovic Chabant <ludovic@chabant.com>
parents: 214
diff changeset
578 self.status_queue.put_nowait(item)
209
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
579 except:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
580 pass
7a5a7a7e8cee serve: Run the asset pipeline asynchronously.
Ludovic Chabant <ludovic@chabant.com>
parents: 205
diff changeset
581