piecrust2: piecrust/serving/wrappers.py annotate

annotate piecrust/serving/wrappers.py @ 552:9612cfc6455a

serve: Rewrite of the Server-Sent Event code for build notifications. At the moment the server monitors the asset directories, and notifies the browser when an asset has changed and has been re-processed. * Fix issues around long-running requests/threads which mess up the ability to shutdown the server correctly with `CTRL-C` (see comments in code). * Move the notification queue to each SSE producer, to support having multiple pages open in a browser. * Add JS/CSS for showing quick notifications about re-processed assets. * Add support for hot-reloading CSS and pictures that have been re-processed.

author	Ludovic Chabant <ludovic@chabant.com>
date	Sat, 08 Aug 2015 16:12:04 -0700
parents	fa3ee8a8ee2d
children	cc6f3dbe3048

rev	line source
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import os
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	2 import logging
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	3 import threading
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	4 import urllib.request
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 from piecrust.serving.server import Server
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	6
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	7
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	8 logger = logging.getLogger(__name__)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	9
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11 def run_werkzeug_server(root_dir, host, port,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12 debug_piecrust=False, sub_cache_dir=None,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13 use_debugger=False, use_reloader=False):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14 from werkzeug.serving import run_simple
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16 def _run_sse_check():
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17 # We don't want to run the processing loop here if this isn't
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 # the actual process that does the serving. In most cases it is,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19 # but if we're using Werkzeug's reloader, then it won't be the
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20 # first time we get there... it will only be the correct process
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 # the second time, when the reloading process is spawned, with the
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22 # `WERKZEUG_RUN_MAIN` variable set.
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 return (not use_reloader or
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 os.environ.get('WERKZEUG_RUN_MAIN') == 'true')
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 app = _get_piecrust_server(root_dir,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 debug=debug_piecrust,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28 sub_cache_dir=sub_cache_dir,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 run_sse_check=_run_sse_check)
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	30
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	31 # We need to run Werkzeug in a background thread because we may have some
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	32 # SSE responses running. In theory we should be using a proper async
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	33 # server for this kind of stuff, but I'd rather avoid additional
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	34 # dependencies on stuff that's not necessarily super portable.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	35 # Anyway we run the server in multi-threading mode, but the request
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	36 # threads are not set to `daemon` mode (and there's no way to set that
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	37 # flag without re-implementing `run_simple` apparently). So instead we
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	38 # run the server in a background thread so we keep the main thread to
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	39 # ourselves here, which means we can trap `KeyboardInterrupt`, and set
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	40 # a global flag that will kill all the long-running SSE threads and make
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	41 # this whole thing exit cleanly and properly (hopefully).
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	42 def _inner():
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	43 run_simple(host, port, app,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	44 threaded=True,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	45 use_debugger=use_debugger,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46 use_reloader=use_reloader)
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	47
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	48 t = threading.Thread(name='WerkzeugServer', target=_inner)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	49 t.start()
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	50 try:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	51 while t.is_alive():
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	52 t.join(0.5)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	53 except KeyboardInterrupt:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	54 shutdown_url = 'http://%s:%s/__piecrust_debug/werkzeug_shutdown' % (
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	55 host, port)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	56 logger.info("")
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	57 logger.info("Shutting down server...")
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	58 urllib.request.urlopen(shutdown_url)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	59 finally:
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	60 logger.debug("Terminating push notifications...")
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	61 from piecrust.serving import procloop
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	62 procloop.server_shutdown = True
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	63
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	64
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	65 def run_gunicorn_server(root_dir,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66 debug_piecrust=False, sub_cache_dir=None,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67 gunicorn_options=None):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68 from gunicorn.app.base import BaseApplication
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	69
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	70 class PieCrustGunicornApplication(BaseApplication):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71 def __init__(self, app, options):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 self.app = app
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73 self.options = options
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	74 super(PieCrustGunicornApplication, self).__init__()
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	75
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	76 def load_config(self):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	77 for k, v in self.options.items():
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 if k in self.cfg.settings and v is not None:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 self.cfg.set(k, v)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 def load(self):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 return self.app
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84 app = _get_piecrust_server(root_dir,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	85 debug=debug_piecrust,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86 sub_cache_dir=sub_cache_dir)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 gunicorn_options = gunicorn_options or {}
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 app_wrapper = PieCrustGunicornApplication(app, gunicorn_options)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	90 app_wrapper.run()
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	91
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	92
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	93 def _get_piecrust_server(root_dir, **kwargs):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	94 server = Server(root_dir, **kwargs)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	95 app = server.getWsgiApp()
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	96 return app
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	97

Mercurial > piecrust2

annotate piecrust/serving/wrappers.py @ 552:9612cfc6455a