piecrust2: piecrust/serving.py annotate

annotate piecrust/serving.py @ 371:c2ca72fb7f0b 2.0.0a8

caching: Use separate caches for config variants and other contexts. * The `_cache` directory is now organized in multiple "sub-caches" for different contexts. * A new context is created when config variants or overrides are applied. * `serve` context uses a different context that the other commends, to prevent the `bake` command's output from messing up the preview server (e.g. with how asset URLs are generated differently between the two). * Fix a few places where the cache directory was referenced directly.

author	Ludovic Chabant <ludovic@chabant.com>
date	Sun, 03 May 2015 23:59:46 -0700
parents	4b1019bb2533
children

rev	line source
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	1 import io
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	2 import os
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import re
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	4 import json
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import gzip
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	6 import time
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	7 import queue
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	8 import os.path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	9 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10 import logging
258 7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	11 import datetime
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	12 import threading
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	13 from werkzeug.exceptions import (
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	14 NotFound, MethodNotAllowed, InternalServerError, HTTPException)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15 from werkzeug.wrappers import Request, Response
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	16 from werkzeug.wsgi import ClosingIterator, wrap_file
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17 from jinja2 import FileSystemLoader, Environment
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 from piecrust.app import PieCrust
111 208c652551a3 Quick fix for making the server correctly update referenced pages. Ludovic Chabant <ludovic@chabant.com> parents: 110 diff changeset	19 from piecrust.environment import StandardEnvironment
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20 from piecrust.processing.base import ProcessorPipeline
369 4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages. Ludovic Chabant <ludovic@chabant.com> parents: 363 diff changeset	21 from piecrust.rendering import QualifiedPage, PageRenderingContext, render_page
11 617191dec18e Fixes for Windows, make `findPagePath` return a ref path. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	22 from piecrust.sources.base import PageFactory, MODE_PARSING
298 b7ab1b503510 data: Fix incorrect next/previous page URLs in pagination data. Ludovic Chabant <ludovic@chabant.com> parents: 280 diff changeset	23 from piecrust.uriutil import split_sub_uri
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28
224 4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running. Ludovic Chabant <ludovic@chabant.com> parents: 223 diff changeset	29 _sse_abort = threading.Event()
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running. Ludovic Chabant <ludovic@chabant.com> parents: 223 diff changeset	30
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running. Ludovic Chabant <ludovic@chabant.com> parents: 223 diff changeset	31
111 208c652551a3 Quick fix for making the server correctly update referenced pages. Ludovic Chabant <ludovic@chabant.com> parents: 110 diff changeset	32 class ServingEnvironment(StandardEnvironment):
113 de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	33 pass
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	34
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	35
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	36 class ServeRecord(object):
111 208c652551a3 Quick fix for making the server correctly update referenced pages. Ludovic Chabant <ludovic@chabant.com> parents: 110 diff changeset	37 def __init__(self):
113 de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	38 self.entries = {}
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	39
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	40 def addEntry(self, entry):
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	41 key = self._makeKey(entry.uri, entry.sub_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	42 self.entries[key] = entry
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	43
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	44 def getEntry(self, uri, sub_num):
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	45 key = self._makeKey(uri, sub_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	46 return self.entries.get(key)
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	47
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	48 def _makeKey(self, uri, sub_num):
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	49 return "%s:%s" % (uri, sub_num)
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	50
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	51
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	52 class ServeRecordPageEntry(object):
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	53 def __init__(self, uri, sub_num):
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	54 self.uri = uri
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	55 self.sub_num = sub_num
de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	56 self.used_source_names = set()
111 208c652551a3 Quick fix for making the server correctly update referenced pages. Ludovic Chabant <ludovic@chabant.com> parents: 110 diff changeset	57
208c652551a3 Quick fix for making the server correctly update referenced pages. Ludovic Chabant <ludovic@chabant.com> parents: 110 diff changeset	58
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	59 class WsgiServerWrapper(object):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	60 def __init__(self, server):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	61 self.server = server
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	62
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	63 def __call__(self, environ, start_response):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	64 return self.server._run_request(environ, start_response)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	65
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	66
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67 class Server(object):
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	68 def __init__(self, root_dir,
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	69 debug=False, sub_cache_dir=None,
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	70 use_reloader=False, static_preview=True):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71 self.root_dir = root_dir
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 self.debug = debug
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	73 self.sub_cache_dir = sub_cache_dir
155 70b86e904b85 Properly use, or not, the debugging when using the chef server. Ludovic Chabant <ludovic@chabant.com> parents: 141 diff changeset	74 self.use_reloader = use_reloader
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	75 self.static_preview = static_preview
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	76 self._out_dir = None
113 de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	77 self._page_record = None
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	78 self._proc_loop = None
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 self._mimetype_map = load_mimetype_map()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	81 def getWsgiApp(self):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 # Bake all the assets so we know what we have, and so we can serve
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 # them to the client. We need a temp app for this.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84 app = PieCrust(root_dir=self.root_dir, debug=self.debug)
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	85 app._useSubCacheDir(self.sub_cache_dir)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	86 self._out_dir = os.path.join(app.sub_cache_dir, 'server')
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	87 self._page_record = ServeRecord()
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	88
214 09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	89 if (not self.use_reloader or
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	90 os.environ.get('WERKZEUG_RUN_MAIN') == 'true'):
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	91 # We don't want to run the processing loop here if this isn't
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	92 # the actual process that does the serving. In most cases it is,
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	93 # but if we're using Werkzeug's reloader, then it won't be the
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	94 # first time we get there... it will only be the correct process
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	95 # the second time, when the reloading process is spawned, with the
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	96 # `WERKZEUG_RUN_MAIN` variable set.
09e350db7f8f serve: Don't have 2 processing loops running when using `--use-reloader`. Ludovic Chabant <ludovic@chabant.com> parents: 209 diff changeset	97 pipeline = ProcessorPipeline(app, self._out_dir)
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	98 self._proc_loop = ProcessingLoop(pipeline)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	99 self._proc_loop.start()
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	100
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	101 # Run the WSGI app.
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	102 wsgi_wrapper = WsgiServerWrapper(self)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	103 return wsgi_wrapper
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	104
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105 def _run_request(self, environ, start_response):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106 try:
200 76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs. Ludovic Chabant <ludovic@chabant.com> parents: 190 diff changeset	107 return self._try_run_request(environ, start_response)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	108 except Exception as ex:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	109 if self.debug:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	110 raise
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	111 return self._handle_error(ex, environ, start_response)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	112
200 76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs. Ludovic Chabant <ludovic@chabant.com> parents: 190 diff changeset	113 def _try_run_request(self, environ, start_response):
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	114 request = Request(environ)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	115
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	116 # We don't support anything else than GET requests since we're
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	117 # previewing something that will be static later.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	118 if self.static_preview and request.method != 'GET':
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	119 logger.error("Only GET requests are allowed, got %s" %
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	120 request.method)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121 raise MethodNotAllowed()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	123 # Handle special requests right away.
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	124 response = self._try_special_request(environ, request)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	125 if response is not None:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	126 return response(environ, start_response)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	127
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	128 # Also handle requests to a pipeline-built asset right away.
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	129 response = self._try_serve_asset(environ, request)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	130 if response is not None:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	131 return response(environ, start_response)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	132
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	133 # Create the app for this request.
223 7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`. Ludovic Chabant <ludovic@chabant.com> parents: 219 diff changeset	134 app = PieCrust(root_dir=self.root_dir, debug=self.debug)
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	135 app._useSubCacheDir(self.sub_cache_dir)
53 73956224eb67 Setup the server better. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	136 app.config.set('site/root', '/')
73956224eb67 Setup the server better. Ludovic Chabant <ludovic@chabant.com> parents: 39 diff changeset	137 app.config.set('server/is_serving', True)
223 7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`. Ludovic Chabant <ludovic@chabant.com> parents: 219 diff changeset	138 if (app.config.get('site/enable_debug_info') and
7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`. Ludovic Chabant <ludovic@chabant.com> parents: 219 diff changeset	139 '!debug' in request.args):
7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`. Ludovic Chabant <ludovic@chabant.com> parents: 219 diff changeset	140 app.config.set('site/show_debug_info', True)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	141
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	142 # We'll serve page assets directly from where they are.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	143 app.env.base_asset_url_format = '/_asset/%path%'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	144
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	145 # Let's see if it can be a page asset.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	146 response = self._try_serve_page_asset(app, environ, request)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	147 if response is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	148 return response(environ, start_response)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	149
133 9e4c2e68a129 Optimize server for files that already exist. Ludovic Chabant <ludovic@chabant.com> parents: 130 diff changeset	150 # Nope. Let's see if it's an actual page.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	151 try:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	152 response = self._try_serve_page(app, environ, request)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	153 return response(environ, start_response)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	154 except (RouteNotFoundError, SourceNotFoundError) as ex:
229 a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	155 raise NotFound(str(ex)) from ex
133 9e4c2e68a129 Optimize server for files that already exist. Ludovic Chabant <ludovic@chabant.com> parents: 130 diff changeset	156 except HTTPException:
9e4c2e68a129 Optimize server for files that already exist. Ludovic Chabant <ludovic@chabant.com> parents: 130 diff changeset	157 raise
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	158 except Exception as ex:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159 if app.debug:
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 113 diff changeset	160 logger.exception(ex)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	161 raise
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 113 diff changeset	162 msg = str(ex)
28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 113 diff changeset	163 logger.error(msg)
229 a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	164 raise InternalServerError(msg) from ex
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	165
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	166 def _try_special_request(self, environ, request):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	167 static_mount = '/__piecrust_static/'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	168 if request.path.startswith(static_mount):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	169 rel_req_path = request.path[len(static_mount):]
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	170 mount = os.path.join(
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	171 os.path.dirname(__file__),
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	172 'resources', 'server')
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	173 full_path = os.path.join(mount, rel_req_path)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	174 try:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	175 response = self._make_wrapped_file_response(
258 7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	176 environ, request, full_path)
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	177 return response
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	178 except OSError:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	179 pass
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	180
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	181 debug_mount = '/__piecrust_debug/'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	182 if request.path.startswith(debug_mount):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	183 rel_req_path = request.path[len(debug_mount):]
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	184 if rel_req_path == 'pipeline_status':
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	185 provider = PipelineStatusServerSideEventProducer(
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	186 self._proc_loop.status_queue)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	187 it = ClosingIterator(provider.run(), [provider.close])
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	188 response = Response(it)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	189 response.headers['Cache-Control'] = 'no-cache'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	190 if 'text/event-stream' in request.accept_mimetypes:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	191 response.mimetype = 'text/event-stream'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	192 response.direct_passthrough = True
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	193 response.implicit_sequence_conversion = False
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	194 return response
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	195
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	196 return None
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	197
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	198 def _try_serve_asset(self, environ, request):
11 617191dec18e Fixes for Windows, make `findPagePath` return a ref path. Ludovic Chabant <ludovic@chabant.com> parents: 7 diff changeset	199 rel_req_path = request.path.lstrip('/').replace('/', os.sep)
240 c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file. Ludovic Chabant <ludovic@chabant.com> parents: 232 diff changeset	200 if request.path.startswith('/_cache/'):
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file. Ludovic Chabant <ludovic@chabant.com> parents: 232 diff changeset	201 # Some stuff needs to be served directly from the cache directory,
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file. Ludovic Chabant <ludovic@chabant.com> parents: 232 diff changeset	202 # like LESS CSS map files.
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file. Ludovic Chabant <ludovic@chabant.com> parents: 232 diff changeset	203 full_path = os.path.join(self.root_dir, rel_req_path)
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file. Ludovic Chabant <ludovic@chabant.com> parents: 232 diff changeset	204 else:
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file. Ludovic Chabant <ludovic@chabant.com> parents: 232 diff changeset	205 full_path = os.path.join(self._out_dir, rel_req_path)
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file. Ludovic Chabant <ludovic@chabant.com> parents: 232 diff changeset	206
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	207 try:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	208 response = self._make_wrapped_file_response(
258 7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	209 environ, request, full_path)
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	210 return response
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	211 except OSError:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	212 pass
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	213 return None
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	214
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	215 def _try_serve_page_asset(self, app, environ, request):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	216 if not request.path.startswith('/_asset/'):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	217 return None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	218
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	219 full_path = os.path.join(app.root_dir, request.path[len('/_asset/'):])
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	220 if not os.path.isfile(full_path):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	221 return None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	222
258 7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	223 return self._make_wrapped_file_response(environ, request, full_path)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	224
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	225 def _try_serve_page(self, app, environ, request):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	226 # Try to find what matches the requested URL.
298 b7ab1b503510 data: Fix incorrect next/previous page URLs in pagination data. Ludovic Chabant <ludovic@chabant.com> parents: 280 diff changeset	227 req_path, page_num = split_sub_uri(app, request.path)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	228
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	229 routes = find_routes(app.routes, req_path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	230 if len(routes) == 0:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	231 raise RouteNotFoundError("Can't find route for: %s" % req_path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	232
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	233 rendered_page = None
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	234 first_not_found = None
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	235 for route, route_metadata in routes:
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	236 try:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	237 logger.debug("Trying to render match from source '%s'." %
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	238 route.source_name)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	239 rendered_page = self._try_render_page(
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	240 app, route, route_metadata, page_num, req_path)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	241 if rendered_page is not None:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	242 break
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	243 except NotFound as nfe:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	244 if first_not_found is None:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	245 first_not_found = nfe
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	246 else:
229 a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	247 raise SourceNotFoundError(
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	248 "Can't find path for: %s (looked in: %s)" %
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	249 (req_path, [r.source_name for r, _ in routes]))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	250
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	251 # If we haven't found any good match, raise whatever exception we
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	252 # first got. Otherwise, raise a generic exception.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	253 if rendered_page is None:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	254 first_not_found = first_not_found or NotFound(
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	255 "This page couldn't be found.")
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	256 raise first_not_found
113 de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	257
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	258 # Start doing stuff.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	259 page = rendered_page.page
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	260 rp_content = rendered_page.content
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	261
113 de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	262 # Profiling.
279 980bbbd0705e serve: Correctly show timing info even when not in debug mode. Ludovic Chabant <ludovic@chabant.com> parents: 258 diff changeset	263 if app.config.get('site/show_debug_info'):
7 343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	264 now_time = time.clock()
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	265 timing_info = ('%8.1f ms' %
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	266 ((now_time - app.env.start_time) * 1000.0))
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	267 rp_content = rp_content.replace('__PIECRUST_TIMING_INFORMATION__',
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	268 timing_info)
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	269
113 de257cc40ce1 Re-enable proper caching of rendered segments in server. Ludovic Chabant <ludovic@chabant.com> parents: 111 diff changeset	270 # Build the response.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	271 response = Response()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	272
7 343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	273 etag = hashlib.md5(rp_content.encode('utf8')).hexdigest()
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	274 if not app.debug and etag in request.if_none_match:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	275 response.status_code = 304
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	276 return response
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	277
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	278 response.set_etag(etag)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	279 response.content_md5 = etag
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	280
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	281 cache_control = response.cache_control
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	282 if app.debug:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	283 cache_control.no_cache = True
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	284 cache_control.must_revalidate = True
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	285 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	286 cache_time = (page.config.get('cache_time') or
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	287 app.config.get('site/cache_time'))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	288 if cache_time:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	289 cache_control.public = True
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	290 cache_control.max_age = cache_time
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	291
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	292 content_type = page.config.get('content_type')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	293 if content_type and '/' not in content_type:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	294 mimetype = content_type_map.get(content_type, content_type)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	295 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	296 mimetype = content_type
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	297 if mimetype:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	298 response.mimetype = mimetype
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	299
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	300 if ('gzip' in request.accept_encodings and
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	301 app.config.get('site/enable_gzip')):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	302 try:
7 343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	303 with io.BytesIO() as gzip_buffer:
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	304 with gzip.open(gzip_buffer, mode='wt',
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	305 encoding='utf8') as gzip_file:
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	306 gzip_file.write(rp_content)
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	307 rp_content = gzip_buffer.getvalue()
343d08ef5668 More PieCrust 3 fixes, and a couple of miscellaneous bug fixes. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	308 response.content_encoding = 'gzip'
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	309 except Exception:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	310 logger.exception("Error compressing response, "
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	311 "falling back to uncompressed.")
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	312 response.set_data(rp_content)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	313
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	314 return response
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	315
371 c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	316 def _try_render_page(self, app, route, route_metadata, page_num, req_path):
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	317 # Match the route to an actual factory.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	318 taxonomy_info = None
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	319 source = app.getSource(route.source_name)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	320 if route.taxonomy_name is None:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	321 factory = source.findPageFactory(route_metadata, MODE_PARSING)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	322 if factory is None:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	323 return None
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	324 else:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	325 taxonomy = app.getTaxonomy(route.taxonomy_name)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	326 route_terms = route_metadata.get(taxonomy.term_name)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	327 if route_terms is None:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	328 return None
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	329
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	330 tax_page_ref = taxonomy.getPageRef(source.name)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	331 factory = tax_page_ref.getFactory()
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	332 tax_terms = route.unslugifyTaxonomyTerm(route_terms)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	333 route_metadata[taxonomy.term_name] = tax_terms
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	334 taxonomy_info = (taxonomy, tax_terms)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	335
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	336 # Build the page.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	337 page = factory.buildPage()
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	338 # We force the rendering of the page because it could not have
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	339 # changed, but include pages that did change.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	340 qp = QualifiedPage(page, route, route_metadata)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	341 render_ctx = PageRenderingContext(qp,
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	342 page_num=page_num,
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	343 force_render=True)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	344 if taxonomy_info is not None:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	345 taxonomy, tax_terms = taxonomy_info
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	346 render_ctx.setTaxonomyFilter(taxonomy, tax_terms)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	347
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	348 # See if this page is known to use sources. If that's the case,
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	349 # just don't use cached rendered segments for that page (but still
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	350 # use them for pages that are included in it).
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	351 uri = qp.getUri()
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	352 assert uri == req_path
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	353 entry = self._page_record.getEntry(uri, page_num)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	354 if (taxonomy_info is not None or entry is None or
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	355 entry.used_source_names):
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	356 cache_key = '%s:%s' % (uri, page_num)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	357 app.env.rendered_segments_repository.invalidate(cache_key)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	358
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	359 # Render the page.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	360 rendered_page = render_page(render_ctx)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	361
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	362 # Check if this page is a taxonomy page that actually doesn't match
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	363 # anything.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	364 if taxonomy_info is not None:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	365 paginator = rendered_page.data.get('pagination')
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	366 if (paginator and paginator.is_loaded and
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	367 len(paginator.items) == 0):
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	368 taxonomy = taxonomy_info[0]
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	369 message = ("This URL matched a route for taxonomy '%s' but "
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	370 "no pages have been found to have it. This page "
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	371 "won't be generated by a bake." % taxonomy.name)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	372 raise NotFound(message)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	373
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	374 # Remember stuff for next time.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	375 if entry is None:
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	376 entry = ServeRecordPageEntry(req_path, page_num)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	377 self._page_record.addEntry(entry)
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	378 for p, pinfo in render_ctx.render_passes.items():
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	379 entry.used_source_names \|= pinfo.used_source_names
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	380
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	381 # Ok all good.
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	382 return rendered_page
c2ca72fb7f0b caching: Use separate caches for config variants and other contexts. Ludovic Chabant <ludovic@chabant.com> parents: 369 diff changeset	383
258 7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	384 def _make_wrapped_file_response(self, environ, request, path):
190 430ee5b80962 serve: Make the server find assets generated by external tools. Ludovic Chabant <ludovic@chabant.com> parents: 176 diff changeset	385 logger.debug("Serving %s" % path)
258 7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	386
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	387 # Check if we can return a 304 status code.
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	388 mtime = os.path.getmtime(path)
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	389 etag_str = '%s$$%s' % (path, mtime)
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	390 etag = hashlib.md5(etag_str.encode('utf8')).hexdigest()
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	391 if etag in request.if_none_match:
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	392 response = Response()
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	393 response.status_code = 304
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	394 return response
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	395
190 430ee5b80962 serve: Make the server find assets generated by external tools. Ludovic Chabant <ludovic@chabant.com> parents: 176 diff changeset	396 wrapper = wrap_file(environ, open(path, 'rb'))
430ee5b80962 serve: Make the server find assets generated by external tools. Ludovic Chabant <ludovic@chabant.com> parents: 176 diff changeset	397 response = Response(wrapper)
430ee5b80962 serve: Make the server find assets generated by external tools. Ludovic Chabant <ludovic@chabant.com> parents: 176 diff changeset	398 _, ext = os.path.splitext(path)
258 7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	399 response.set_etag(etag)
7ec06ec14247 serve: Use Etags and 304 responses for assets. Ludovic Chabant <ludovic@chabant.com> parents: 256 diff changeset	400 response.last_modified = datetime.datetime.fromtimestamp(mtime)
190 430ee5b80962 serve: Make the server find assets generated by external tools. Ludovic Chabant <ludovic@chabant.com> parents: 176 diff changeset	401 response.mimetype = self._mimetype_map.get(
430ee5b80962 serve: Make the server find assets generated by external tools. Ludovic Chabant <ludovic@chabant.com> parents: 176 diff changeset	402 ext.lstrip('.'), 'text/plain')
430ee5b80962 serve: Make the server find assets generated by external tools. Ludovic Chabant <ludovic@chabant.com> parents: 176 diff changeset	403 return response
430ee5b80962 serve: Make the server find assets generated by external tools. Ludovic Chabant <ludovic@chabant.com> parents: 176 diff changeset	404
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	405 def _handle_error(self, exception, environ, start_response):
200 76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs. Ludovic Chabant <ludovic@chabant.com> parents: 190 diff changeset	406 code = 500
128 28444014ce7d Fix error reporting and counting of lines. Ludovic Chabant <ludovic@chabant.com> parents: 113 diff changeset	407 if isinstance(exception, HTTPException):
200 76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs. Ludovic Chabant <ludovic@chabant.com> parents: 190 diff changeset	408 code = exception.code
229 a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	409
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	410 path = 'error'
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	411 if isinstance(exception, NotFound):
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	412 path += '404'
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	413
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	414 descriptions = self._get_exception_descriptions(exception)
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	415
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	416 env = Environment(loader=ErrorMessageLoader())
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	417 template = env.get_template(path)
229 a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	418 context = {'details': descriptions}
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	419 response = Response(template.render(context), mimetype='text/html')
200 76e459d48c43 serve: Correctly pass on the HTTP status code when an error occurs. Ludovic Chabant <ludovic@chabant.com> parents: 190 diff changeset	420 response.status_code = code
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	421 return response(environ, start_response)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	422
229 a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	423 def _get_exception_descriptions(self, exception):
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	424 desc = []
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	425 while exception is not None:
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	426 if isinstance(exception, HTTPException):
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	427 desc.append(exception.description)
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	428 else:
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	429 desc.append(str(exception))
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	430
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	431 inner_ex = exception.__cause__
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	432 if inner_ex is None:
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	433 inner_ex = exception.__context__
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	434 exception = inner_ex
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	435 return desc
a951cd4ef361 serve: Print nested exception messages in the dev server. Ludovic Chabant <ludovic@chabant.com> parents: 224 diff changeset	436
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	437
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	438 class RouteNotFoundError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	439 pass
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	440
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	441
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	442 class SourceNotFoundError(Exception):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	443 pass
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	444
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	445
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	446 content_type_map = {
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	447 'html': 'text/html',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	448 'xml': 'text/xml',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	449 'txt': 'text/plain',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	450 'text': 'text/plain',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	451 'css': 'text/css',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	452 'xhtml': 'application/xhtml+xml',
232 e534d2bc657c cosmetic: Fix PEP8 spacing. Ludovic Chabant <ludovic@chabant.com> parents: 231 diff changeset	453 'atom': 'application/atom+xml', # or 'text/xml'?
e534d2bc657c cosmetic: Fix PEP8 spacing. Ludovic Chabant <ludovic@chabant.com> parents: 231 diff changeset	454 'rss': 'application/rss+xml', # or 'text/xml'?
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	455 'json': 'application/json'}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	456
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	457
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	458 def find_routes(routes, uri):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	459 res = []
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	460 for route in routes:
176 d47d9493bb0a routes: When matching URIs, return metadata directly instead of the match object. Ludovic Chabant <ludovic@chabant.com> parents: 174 diff changeset	461 metadata = route.matchUri(uri)
231 137bcd498ef9 serve: Fix a bug where empty route metadata is not the same as invalid route. Ludovic Chabant <ludovic@chabant.com> parents: 229 diff changeset	462 if metadata is not None:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	463 res.append((route, metadata))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	464 return res
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	465
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	466
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	467 class ErrorMessageLoader(FileSystemLoader):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	468 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	469 base_dir = os.path.join(os.path.dirname(__file__), 'resources',
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	470 'messages')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	471 super(ErrorMessageLoader, self).__init__(base_dir)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	472
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	473 def get_source(self, env, template):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	474 template += '.html'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	475 return super(ErrorMessageLoader, self).get_source(env, template)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	476
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	477
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	478 def load_mimetype_map():
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	479 mimetype_map = {}
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	480 sep_re = re.compile(r'\s+')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	481 path = os.path.join(os.path.dirname(__file__), 'mime.types')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	482 with open(path, 'r') as f:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	483 for line in f:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	484 tokens = sep_re.split(line)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	485 if len(tokens) > 1:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	486 for t in tokens[1:]:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	487 mimetype_map[t] = tokens[0]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	488 return mimetype_map
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	489
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	490
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	491 class PipelineStatusServerSideEventProducer(object):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	492 def __init__(self, status_queue):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	493 self.status_queue = status_queue
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	494 self.interval = 2
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	495 self.timeout = 60*10
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	496 self._start_time = 0
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	497
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	498 def run(self):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	499 logger.debug("Starting pipeline status SSE.")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	500 self._start_time = time.time()
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	501
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	502 outstr = 'event: ping\ndata: started\n\n'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	503 yield bytes(outstr, 'utf8')
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	504
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	505 count = 0
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	506 while True:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	507 if time.time() > self.timeout + self._start_time:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	508 logger.debug("Closing pipeline status SSE, timeout reached.")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	509 outstr = 'event: pipeline_timeout\ndata: bye\n\n'
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	510 yield bytes(outstr, 'utf8')
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	511 break
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	512
224 4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running. Ludovic Chabant <ludovic@chabant.com> parents: 223 diff changeset	513 if _sse_abort.is_set():
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running. Ludovic Chabant <ludovic@chabant.com> parents: 223 diff changeset	514 break
4f00bb99400e serve: Fix exiting the server with `CTRL+C` when the SSE response is running. Ludovic Chabant <ludovic@chabant.com> parents: 223 diff changeset	515
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	516 try:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	517 logger.debug("Polling pipeline status queue...")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	518 count += 1
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	519 data = self.status_queue.get(True, self.interval)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	520 except queue.Empty:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	521 if count < 3:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	522 continue
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	523 data = {'type': 'ping', 'message': 'ping'}
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	524 count = 0
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	525
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	526 event_type = data['type']
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	527 outstr = 'event: %s\ndata: %s\n\n' % (
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	528 event_type, json.dumps(data))
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	529 logger.debug("Sending pipeline status SSE.")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	530 yield bytes(outstr, 'utf8')
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	531
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	532 def close(self):
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	533 logger.debug("Closing pipeline status SSE.")
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	534
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	535
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	536 class ProcessingLoop(threading.Thread):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	537 def __init__(self, pipeline):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	538 super(ProcessingLoop, self).__init__(
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	539 name='pipeline-reloader', daemon=True)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	540 self.pipeline = pipeline
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	541 self.status_queue = queue.Queue()
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	542 self.interval = 1
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	543 self._paths = set()
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	544 self._record = None
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	545 self._last_bake = 0
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	546
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	547 def run(self):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	548 # Build the first list of known files and run the pipeline once.
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	549 app = self.pipeline.app
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	550 roots = [os.path.join(app.root_dir, r)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	551 for r in self.pipeline.mounts.keys()]
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	552 for root in roots:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	553 for dirpath, dirnames, filenames in os.walk(root):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	554 self._paths \|= set([os.path.join(dirpath, f)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	555 for f in filenames])
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	556 self._last_bake = time.time()
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	557 self._record = self.pipeline.run(save_record=False)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	558
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	559 while True:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	560 for root in roots:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	561 # For each mount root we try to find the first new or
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	562 # modified file. If any, we just run the pipeline on
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	563 # that mount.
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	564 found_new_or_modified = False
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	565 for dirpath, dirnames, filenames in os.walk(root):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	566 for filename in filenames:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	567 path = os.path.join(dirpath, filename)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	568 if path not in self._paths:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	569 logger.debug("Found new asset: %s" % path)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	570 self._paths.add(path)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	571 found_new_or_modified = True
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	572 break
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	573 if os.path.getmtime(path) > self._last_bake:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	574 logger.debug("Found modified asset: %s" % path)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	575 found_new_or_modified = True
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	576 break
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	577
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	578 if found_new_or_modified:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	579 break
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	580
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	581 if found_new_or_modified:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	582 self._runPipeline(root)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	583
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	584 time.sleep(self.interval)
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	585
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	586 def _runPipeline(self, root):
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	587 self._last_bake = time.time()
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	588 try:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	589 self._record = self.pipeline.run(
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	590 root,
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	591 previous_record=self._record,
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	592 save_record=False)
219 d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	593
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	594 # Update the status queue.
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	595 # (we need to clear it because there may not be a consumer
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	596 # on the other side, if the user isn't running with the
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	597 # debug window active)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	598 while True:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	599 try:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	600 self.status_queue.get_nowait()
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	601 except queue.Empty:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	602 break
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	603
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	604 if self._record.success:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	605 item = {
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	606 'type': 'pipeline_success'}
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	607 self.status_queue.put_nowait(item)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	608 else:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	609 item = {
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	610 'type': 'pipeline_error',
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	611 'assets': []}
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	612 for entry in self._record.entries:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	613 if entry.errors:
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	614 asset_item = {
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	615 'path': entry.rel_input,
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	616 'errors': list(entry.errors)}
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	617 item['assets'].append(asset_item)
d7a548ebcd58 serve: Add server sent events for showing pipeline errors in the debug window. Ludovic Chabant <ludovic@chabant.com> parents: 214 diff changeset	618 self.status_queue.put_nowait(item)
209 7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	619 except:
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	620 pass
7a5a7a7e8cee serve: Run the asset pipeline asynchronously. Ludovic Chabant <ludovic@chabant.com> parents: 205 diff changeset	621

Mercurial > piecrust2

annotate piecrust/serving.py @ 371:c2ca72fb7f0b 2.0.0a8