piecrust2: piecrust/serving/procloop.py annotate

annotate piecrust/serving/procloop.py @ 552:9612cfc6455a

serve: Rewrite of the Server-Sent Event code for build notifications. At the moment the server monitors the asset directories, and notifies the browser when an asset has changed and has been re-processed. * Fix issues around long-running requests/threads which mess up the ability to shutdown the server correctly with `CTRL-C` (see comments in code). * Move the notification queue to each SSE producer, to support having multiple pages open in a browser. * Add JS/CSS for showing quick notifications about re-processed assets. * Add support for hot-reloading CSS and pictures that have been re-processed.

author	Ludovic Chabant <ludovic@chabant.com>
date	Sat, 08 Aug 2015 16:12:04 -0700
parents	fa3ee8a8ee2d
children	cc6f3dbe3048

rev	line source
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import os
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import os.path
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import time
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import json
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import queue
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	6 import logging
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	7 import itertools
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	8 import threading
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	9
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11 logger = logging.getLogger(__name__)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	13 # This flag is for cancelling all long running requests like SSEs.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	14 server_shutdown = False
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	15
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	17 class PipelineStatusServerSentEventProducer(object):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	18 """ The producer for Server-Sent Events (SSE) notifying the front-end
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	19 about useful things like assets having been re-processed in the
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	20 background.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	21 Each has its own queue because the user could have multiple pages
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	22 open, each having to display notifications coming from the server.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	23 """
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	24 def __init__(self, proc_loop):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	25 self._proc_loop = proc_loop
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	26 self._queue = queue.Queue()
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 self._start_time = 0
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	28 self._poll_interval = 0.5
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	29 self._ping_interval = 30
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	30 self._time_between_pings = 0
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	31 self._running = 0
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	32
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	33 def addBuildEvent(self, item):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	34 self._queue.put_nowait(item)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	35
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	36 def run(self):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	37 logger.debug("Starting pipeline status SSE.")
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	38 self._proc_loop.addObserver(self)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	39 self._start_time = time.time()
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	40 self._running = 1
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	41
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	42 outstr = 'event: ping\ndata: started\n\n'
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	43 yield bytes(outstr, 'utf8')
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	44
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	45 while self._running == 1 and not server_shutdown:
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46 try:
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	47 # We use a short poll interval (less than a second) because
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	48 # we need to catch `server_shutdown` going `True` as soon as
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	49 # possible to exit this thread when the user hits `CTRL+C`.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	50 data = self._queue.get(True, self._poll_interval)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51 except queue.Empty:
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	52 # Not exact timing but close enough.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	53 self._time_between_pings += self._poll_interval
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	54 if self._time_between_pings >= self._ping_interval:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	55 self._time_between_pings = 0
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	56 logger.debug("Sending ping/heartbeat event.")
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	57 outstr = 'event: ping\ndata: 1\n\n'
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	58 yield bytes(outstr, 'utf8')
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	59 continue
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 logger.debug("Sending pipeline status SSE.")
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	62 outstr = (('event: %s\n' % data['type']) +
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	63 ('id: %s\n' % data['id']) +
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	64 ('data: %s\n\n' % json.dumps(data)))
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	65 self._queue.task_done()
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66 yield bytes(outstr, 'utf8')
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68 def close(self):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	69 logger.debug("Closing pipeline status SSE.")
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	70 self._proc_loop.removeObserver(self)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	71 self._running = 2
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	74 class ProcessingLoop(threading.Thread):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	75 def __init__(self, pipeline):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	76 super(ProcessingLoop, self).__init__(
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	77 name='pipeline-reloader', daemon=True)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 self.pipeline = pipeline
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	79 self.last_status_id = 0
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 self.interval = 1
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 self._paths = set()
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 self._record = None
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 self._last_bake = 0
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	84 self._obs = []
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	85 self._obs_lock = threading.Lock()
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	86
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	87 def addObserver(self, obs):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	88 with self._obs_lock:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	89 self._obs.append(obs)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	90
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	91 def removeObserver(self, obs):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	92 with self._obs_lock:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	93 self._obs.remove(obs)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	94
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	95 def run(self):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	96 # Build the first list of known files and run the pipeline once.
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	97 app = self.pipeline.app
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	98 roots = [os.path.join(app.root_dir, r)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	99 for r in self.pipeline.mounts.keys()]
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	100 for root in roots:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	101 for dirpath, dirnames, filenames in os.walk(root):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	102 self._paths \|= set([os.path.join(dirpath, f)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	103 for f in filenames])
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	104 self._last_bake = time.time()
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105 self._record = self.pipeline.run()
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	107 while True:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	108 for root in roots:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	109 # For each mount root we try to find the first new or
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	110 # modified file. If any, we just run the pipeline on
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	111 # that mount.
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	112 found_new_or_modified = False
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	113 for dirpath, dirnames, filenames in os.walk(root):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	114 for filename in filenames:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	115 path = os.path.join(dirpath, filename)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	116 if path not in self._paths:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	117 logger.debug("Found new asset: %s" % path)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	118 self._paths.add(path)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	119 found_new_or_modified = True
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120 break
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121 if os.path.getmtime(path) > self._last_bake:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122 logger.debug("Found modified asset: %s" % path)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	123 found_new_or_modified = True
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	124 break
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	125
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	126 if found_new_or_modified:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	127 break
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	128
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	129 if found_new_or_modified:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	130 self._runPipeline(root)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	131
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	132 time.sleep(self.interval)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	133
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	134 def _runPipeline(self, root):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	135 self._last_bake = time.time()
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	136 try:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	137 self._record = self.pipeline.run(
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	138 root,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	139 previous_record=self._record,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	140 save_record=False)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	141
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	142 status_id = self.last_status_id + 1
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	143 self.last_status_id += 1
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	144
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	145 if self._record.success:
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	146 changed = filter(
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	147 lambda i: not i.was_collapsed_from_last_run,
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	148 self._record.entries)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	149 changed = itertools.chain.from_iterable(
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	150 map(lambda i: i.rel_outputs, changed))
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	151 changed = list(changed)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	152 item = {
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	153 'id': status_id,
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	154 'type': 'pipeline_success',
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	155 'assets': changed}
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	156
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	157 self._notifyObservers(item)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	158 else:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159 item = {
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	160 'id': status_id,
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	161 'type': 'pipeline_error',
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	162 'assets': []}
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	163 for entry in self._record.entries:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	164 if entry.errors:
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	165 asset_item = {
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	166 'path': entry.rel_input,
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	167 'errors': list(entry.errors)}
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	168 item['assets'].append(asset_item)
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	169
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	170 self._notifyObservers(item)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	171 except Exception as ex:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	172 logger.exception(ex)
374 fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	173
552 9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	174 def _notifyObservers(self, item):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	175 with self._obs_lock:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	176 observers = list(self._obs)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	177 for obs in observers:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	178 obs.addBuildEvent(item)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications. Ludovic Chabant <ludovic@chabant.com> parents: 374 diff changeset	179

Mercurial > piecrust2

annotate piecrust/serving/procloop.py @ 552:9612cfc6455a