Mercurial > piecrust2

annotate piecrust/serving/procloop.py @ 1185:24413a2963b9

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
admin: Add hidden system information on publish page for troubleshooting.
author Ludovic Chabant <ludovic@chabant.com>
date Thu, 01 Oct 2020 10:58:33 -0700
parents 292e3a1316d8
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import os
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os.path
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import time
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import json
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import queue
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6 import logging
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
7 import itertools
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8 import threading
663
3ceeca7bb71c themes: Add support for a `--theme` argument to `chef`.
Ludovic Chabant <ludovic@chabant.com>
parents: 570
diff changeset
9 from piecrust import CONFIG_PATH, THEME_CONFIG_PATH
880
342e3ea24b5d serve: Fix asset processing loop.
Ludovic Chabant <ludovic@chabant.com>
parents: 876
diff changeset
10 from piecrust.chefutil import format_timed_scope
1037
89d94955b818 serve: Fix infinite loop in asset processing when a change is detected.
Ludovic Chabant <ludovic@chabant.com>
parents: 918
diff changeset
11 from piecrust.pipelines.records import MultiRecord
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14 logger = logging.getLogger(__name__)
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
16 # This flag is for cancelling all long running requests like SSEs.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
17 server_shutdown = False
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
20 class PipelineStatusServerSentEventProducer(object):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
21 """ The producer for Server-Sent Events (SSE) notifying the front-end
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
22 about useful things like assets having been re-processed in the
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
23 background.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
24 Each has its own queue because the user could have multiple pages
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
25 open, each having to display notifications coming from the server.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
26 """
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
27 def __init__(self, proc_loop):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
28 self._proc_loop = proc_loop
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
29 self._queue = queue.Queue()
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30 self._start_time = 0
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
31 self._poll_interval = 0.5
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
32 self._ping_interval = 30
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
33 self._time_between_pings = 0
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
34 self._running = 0
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
35
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
36 def addBuildEvent(self, item):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
37 self._queue.put_nowait(item)
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
38
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
39 def run(self):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
40 logger.debug("Starting pipeline status SSE.")
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
41 self._proc_loop.addObserver(self)
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
42 self._start_time = time.time()
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
43 self._running = 1
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
45 outstr = 'event: ping\ndata: started\n\n'
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
46 yield bytes(outstr, 'utf8')
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
48 while self._running == 1 and not server_shutdown:
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
49 try:
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
50 # We use a short poll interval (less than a second) because
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
51 # we need to catch `server_shutdown` going `True` as soon as
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
52 # possible to exit this thread when the user hits `CTRL+C`.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
53 data = self._queue.get(True, self._poll_interval)
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
54 except queue.Empty:
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
55 # Not exact timing but close enough.
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
56 self._time_between_pings += self._poll_interval
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
57 if self._time_between_pings >= self._ping_interval:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
58 self._time_between_pings = 0
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
59 logger.debug("Sending ping/heartbeat event.")
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
60 outstr = 'event: ping\ndata: 1\n\n'
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
61 yield bytes(outstr, 'utf8')
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
62 continue
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
63
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
64 logger.debug("Sending pipeline status SSE.")
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
65 outstr = (('event: %s\n' % data['type']) +
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
66 ('id: %s\n' % data['id']) +
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
67 ('data: %s\n\n' % json.dumps(data)))
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
68 self._queue.task_done()
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
69 yield bytes(outstr, 'utf8')
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
70
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
71 def close(self):
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
72 logger.debug("Closing pipeline status SSE.")
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
73 self._proc_loop.removeObserver(self)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
74 self._running = 2
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
75
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
76
854
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
77 class _AssetProcessingInfo:
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
78 def __init__(self, source):
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
79 self.source = source
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
80 self.paths = set()
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
81 self.last_bake_time = time.time()
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
82
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
83
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
84 class ProcessingLoopBase:
680
c2ea75e37540 serve: Fix some crashes introduced by recent refactor.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
85 def __init__(self, appfactory, out_dir):
666
81d9c3a3a0b5 internal: Get rid of the whole "sub cache" business.
Ludovic Chabant <ludovic@chabant.com>
parents: 663
diff changeset
86 self.appfactory = appfactory
680
c2ea75e37540 serve: Fix some crashes introduced by recent refactor.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
87 self.out_dir = out_dir
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
88 self.last_status_id = 0
854
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
89 self._app = None
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
90 self._obs = []
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
91 self._obs_lock = threading.Lock()
680
c2ea75e37540 serve: Fix some crashes introduced by recent refactor.
Ludovic Chabant <ludovic@chabant.com>
parents: 666
diff changeset
92 config_name = (
854
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
93 THEME_CONFIG_PATH if appfactory.theme_site else CONFIG_PATH)
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
94 self.config_path = os.path.join(appfactory.root_dir, config_name)
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
95
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
96 def addObserver(self, obs):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
97 with self._obs_lock:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
98 self._obs.append(obs)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
99
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
100 def removeObserver(self, obs):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
101 with self._obs_lock:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
102 self._obs.remove(obs)
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
103
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
104 def getApp(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
105 return self._app
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
106
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
107 def initialize(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
108 self._app = self.appfactory.create()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
109 self.onInitialize()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
110
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
111 def onInitialize(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
112 pass
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
113
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
114 def start(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
115 logger.info("Starting processing loop with output: %s" %
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
116 self.out_dir)
860
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
117 try:
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
118 self.initialize()
860
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
119 except Exception as ex:
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
120 logger.error("Error initializing processing loop:")
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
121 logger.exception(ex)
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
122 return
570
7dabfdd056a1 serve: Fix corner cases where the pipeline doesn't run correctly.
Ludovic Chabant <ludovic@chabant.com>
parents: 565
diff changeset
123
860
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
124 logger.debug("Doing initial processing loop bake...")
1045
2f39ffa601a9 serve: Re-enable baking assets when running the server.
Ludovic Chabant <ludovic@chabant.com>
parents: 1043
diff changeset
125 self.runPipelines()
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
126
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
127 self.onStart()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
128
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
129 def onStart(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
130 raise NotImplementedError()
570
7dabfdd056a1 serve: Fix corner cases where the pipeline doesn't run correctly.
Ludovic Chabant <ludovic@chabant.com>
parents: 565
diff changeset
131
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
132 def getSources(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
133 for src in self._app.sources:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
134 if src.config.get('pipeline') != 'asset':
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
135 continue
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
136 yield src
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
137
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
138 def runPipelines(self, only_for_source=None):
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
139 try:
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
140 self._doRunPipelines(only_for_source)
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
141 except Exception as ex:
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
142 logger.error("Error while running asset pipeline:")
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
143 logger.exception(ex)
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
144
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
145 def _doRunPipelines(self, only_for_source):
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
146 from piecrust.baking.baker import Baker
570
7dabfdd056a1 serve: Fix corner cases where the pipeline doesn't run correctly.
Ludovic Chabant <ludovic@chabant.com>
parents: 565
diff changeset
147
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
148 allowed_sources = None
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
149 if only_for_source:
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
150 allowed_sources = [only_for_source.name]
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
151 baker = Baker(
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
152 self.appfactory, self._app, self.out_dir,
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
153 allowed_pipelines=['asset'],
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
154 allowed_sources=allowed_sources,
1051
971b4d67e82a serve: Fix problems with assets disappearing between servings.
Ludovic Chabant <ludovic@chabant.com>
parents: 1045
diff changeset
155 rotate_bake_records=False,
971b4d67e82a serve: Fix problems with assets disappearing between servings.
Ludovic Chabant <ludovic@chabant.com>
parents: 1045
diff changeset
156 keep_unused_records=(allowed_sources is not None))
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
157 records = baker.bake()
854
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
158
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
159 self._onPipelinesRun(records)
570
7dabfdd056a1 serve: Fix corner cases where the pipeline doesn't run correctly.
Ludovic Chabant <ludovic@chabant.com>
parents: 565
diff changeset
160
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
161 def _onPipelinesRun(self, records):
854
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
162 self.last_status_id += 1
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
163
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
164 if records.success:
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
165 for rec in records.records:
860
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
166 changed = filter(
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
167 lambda i: not i.was_collapsed_from_last_run,
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
168 rec.getEntries())
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
169 changed = itertools.chain.from_iterable(
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
170 map(lambda i: i.out_paths, changed))
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
171 changed = list(changed)
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
172 item = {
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
173 'id': self.last_status_id,
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
174 'type': 'pipeline_success',
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
175 'assets': changed}
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
176
860
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
177 self._notifyObservers(item)
854
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
178 else:
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
179 item = {
860
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
180 'id': self.last_status_id,
854
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
181 'type': 'pipeline_error',
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
182 'assets': []}
918
7f1da7e7b154 internal: The processing loop for the server is now using the baker.
Ludovic Chabant <ludovic@chabant.com>
parents: 880
diff changeset
183 for rec in records.records:
860
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
184 for entry in rec.getEntries():
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
185 if entry.errors:
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
186 asset_item = {
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
187 'path': entry.item_spec,
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
188 'errors': list(entry.errors)}
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
189 item['assets'].append(asset_item)
374
fa3ee8a8ee2d serve: Split the server code in a couple modules inside a `serving` package.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
190
860
c71472e6537f refactor: Get the processing loop in the server functional again.
Ludovic Chabant <ludovic@chabant.com>
parents: 855
diff changeset
191 self._notifyObservers(item)
854
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources.
Ludovic Chabant <ludovic@chabant.com>
parents: 680
diff changeset
192
552
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
193 def _notifyObservers(self, item):
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
194 with self._obs_lock:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
195 observers = list(self._obs)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
196 for obs in observers:
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
197 obs.addBuildEvent(item)
9612cfc6455a serve: Rewrite of the Server-Sent Event code for build notifications.
Ludovic Chabant <ludovic@chabant.com>
parents: 374
diff changeset
198
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
199
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
200 try:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
201 from watchdog.observers import Observer
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
202 from watchdog.events import FileSystemEventHandler
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
203 _has_watchdog = True
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
204 except ImportError:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
205 _has_watchdog = False
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
206
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
207
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
208 if _has_watchdog:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
209 class _AssetFileEventHandler(FileSystemEventHandler):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
210 def __init__(self, proc_loop, source):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
211 self._proc_loop = proc_loop
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
212 self._source = source
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
213
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
214 def on_any_event(self, event):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
215 if event.is_directory:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
216 return
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
217
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
218 pl = self._proc_loop
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
219 with pl._lock:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
220 pl._ops.append({
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
221 'op': 'bake',
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
222 'source': self._source,
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
223 'path': event.src_path,
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
224 'change': event.event_type,
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
225 'time': time.time()})
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
226 pl._event.set()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
227
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
228
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
229 class _SiteConfigEventHandler(FileSystemEventHandler):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
230 def __init__(self, proc_loop, path):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
231 self._proc_loop = proc_loop
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
232 self._path = path
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
233
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
234 def on_modified(self, event):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
235 if event.src_path != self._path:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
236 return
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
237
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
238 pl = self._proc_loop
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
239 with pl._lock:
1059
292e3a1316d8 serve: Fix crash when editing `config.yml` while serving.
Ludovic Chabant <ludovic@chabant.com>
parents: 1051
diff changeset
240 pl._ops.append({'op': 'reinit', 'time': time.time()})
1043
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
241 pl._event.set()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
242
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
243
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
244 class WatchdogProcessingLoop(ProcessingLoopBase):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
245 def __init__(self, appfactory, out_dir):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
246 ProcessingLoopBase.__init__(self, appfactory, out_dir)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
247 self._op_thread = threading.Thread(
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
248 name='watchdog-operations',
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
249 target=self._runOpThread,
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
250 daemon=True)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
251 self._lock = threading.Lock()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
252 self._event = threading.Event()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
253 self._ops = []
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
254 self._last_op_time = 0
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
255
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
256 def onStart(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
257 logger.debug("Running watchdog monitor on:")
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
258 observer = Observer()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
259
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
260 event_handler = _SiteConfigEventHandler(self, self.config_path)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
261 observer.schedule(event_handler, os.path.dirname(self.config_path))
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
262 logger.debug(" - %s" % self.config_path)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
263
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
264 for src in self.getSources():
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
265 path = getattr(src, 'fs_endpoint_path', None)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
266 if not path:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
267 logger.warn("Skipping source '%s' -- it doesn't have "
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
268 "a file-system endpoint." % src.name)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
269 continue
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
270 if not os.path.isdir(path):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
271 continue
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
272
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
273 logger.debug(" - %s" % path)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
274 event_handler = _AssetFileEventHandler(self, src)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
275 observer.schedule(event_handler, path, recursive=True)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
276
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
277 observer.start()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
278 self._op_thread.start()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
279
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
280 def _runOpThread(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
281 while not server_shutdown:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
282 try:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
283 self._event.wait()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
284 with self._lock:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
285 ops = self._ops
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
286 self._ops = []
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
287 self._event.clear()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
288
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
289 orig_len = len(ops)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
290 lot = self._last_op_time
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
291 ops = list(filter(lambda o: o['time'] > lot, ops))
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
292 logger.debug("Got %d ops, with %d that happened after "
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
293 "our last operation." % (orig_len, len(ops)))
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
294 if len(ops) == 0:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
295 continue
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
296
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
297 if any(filter(lambda o: o['op'] == 'reinit', ops)):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
298 logger.info("Site configuration changed, "
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
299 "reloading pipeline.")
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
300 self.initialize()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
301 self.runPipelines()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
302 continue
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
303
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
304 sources = set()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
305 ops = list(filter(lambda o: o['op'] == 'bake', ops))
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
306 for op in ops:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
307 logger.info("Detected file-system change: "
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
308 "%s [%s]" %
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
309 (op['path'], op['change']))
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
310 sources.add(op['source'])
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
311
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
312 logger.debug("Processing: %s" % [s.name for s in sources])
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
313 for s in sources:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
314 self.runPipelines(s)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
315
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
316 self._last_op_time = time.time()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
317
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
318 except (KeyboardInterrupt, SystemExit):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
319 break
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
320
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
321 ProcessingLoop = WatchdogProcessingLoop
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
322
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
323 else:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
324 class LegacyProcessingLoop(ProcessingLoopBase, threading.Thread):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
325 def __init__(self, appfactory, out_dir):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
326 ProcessingLoopBase.__init__(self, appfactory, out_dir)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
327 threading.Thread.__init__(self, name='pipeline-reloader',
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
328 daemon=True)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
329 self.interval = 1
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
330 self._proc_infos = None
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
331 self._last_config_mtime = 0
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
332
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
333 def onInitialize(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
334 self._proc_infos = {}
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
335 for src in self.getSources():
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
336 procinfo = _AssetProcessingInfo(src)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
337 self._proc_infos[src.name] = procinfo
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
338
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
339 # Build the list of initial asset files.
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
340 for item in src.getAllContents():
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
341 procinfo.paths.add(item.spec)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
342
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
343 def onStart(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
344 self._last_config_mtime = os.path.getmtime(self.config_path)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
345 threading.Thread.start(self)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
346
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
347 def run(self):
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
348 while True:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
349 cur_config_time = os.path.getmtime(self.config_path)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
350 if self._last_config_mtime < cur_config_time:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
351 logger.info("Site configuration changed, reloading pipeline.")
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
352 self._last_config_mtime = cur_config_time
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
353 self.initialize()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
354 self.runPipelines()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
355 continue
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
356
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
357 for procinfo in self._proc_infos.values():
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
358 # For each assets folder we try to find the first new or
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
359 # modified file. If any, we just run the pipeline on
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
360 # that source.
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
361 found_new_or_modified = False
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
362 for item in procinfo.source.getAllContents():
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
363 path = item.spec
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
364 if path not in procinfo.paths:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
365 logger.debug("Found new asset: %s" % path)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
366 procinfo.paths.add(path)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
367 found_new_or_modified = True
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
368 break
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
369 if os.path.getmtime(path) > procinfo.last_bake_time:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
370 logger.debug("Found modified asset: %s" % path)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
371 found_new_or_modified = True
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
372 break
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
373 if found_new_or_modified:
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
374 logger.info("change detected, reprocessed '%s'." %
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
375 procinfo.source.name)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
376 self.runPipelines(procinfo.source)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
377 procinfo.last_bake_time = time.time()
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
378
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
379 time.sleep(self.interval)
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
380
54eb8ad9e809 serve: Use `watchdog` for a more efficient monitoring of assets files.
Ludovic Chabant <ludovic@chabant.com>
parents: 1037
diff changeset
381 ProcessingLoop = LegacyProcessingLoop