Mercurial > piecrust2

annotate piecrust/publishing/base.py @ 1188:a7c43131d871

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bake: Fix file write flushing problem with Python 3.8+ Writing the cache files fails in Python 3.8 because it looks like flushing behaviour has changed. We need to explicitly flush. And even then, in very rare occurrences, it looks like it can still run into racing conditions, so we do a very hacky and ugly "retry" loop when fetching cached data :(
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 15 Jun 2021 22:36:23 -0700
parents d709429f02eb
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
1 import os.path
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
2 import time
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
3 import logging
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
4 from piecrust.chefutil import format_timed
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
5
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
6
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
7 logger = logging.getLogger(__name__)
613
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
10 FILE_MODIFIED = 1
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
11 FILE_DELETED = 2
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
12
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
13
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
14 class PublisherConfigurationError(Exception):
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
15 pass
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
16
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
17
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
18 class PublishingContext:
613
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19 def __init__(self):
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
20 self.bake_out_dir = None
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
21 self.bake_records = None
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
22 self.processing_record = None
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
23 self.was_baked = False
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
24 self.preview = False
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
25 self.args = None
613
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
26
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
27
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
28 class Publisher:
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
29 PUBLISHER_NAME = 'undefined'
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
30 PUBLISHER_SCHEME = None
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
31
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
32 def __init__(self, app, target, config):
613
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33 self.app = app
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34 self.target = target
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
35 self.config = config
613
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
36 self.log_file_path = None
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
37
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
38 def setupPublishParser(self, parser, app):
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
39 return
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
40
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
41 def parseUrlTarget(self, url):
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
42 raise NotImplementedError()
613
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
43
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44 def run(self, ctx):
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
45 raise NotImplementedError()
e2e955a3bb25 publish: Add publish command.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
46
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
47 def getBakedFiles(self, ctx):
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
48 for rec in ctx.bake_records.records:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
49 for e in rec.getEntries():
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
50 paths = e.getAllOutputPaths()
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
51 if paths is not None:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
52 yield from paths
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
53
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
54 def getDeletedFiles(self, ctx):
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
55 for rec in ctx.bake_records.records:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
56 yield from rec.deleted_out_paths
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
57
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
58
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
59 class InvalidPublishTargetError(Exception):
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
60 pass
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
61
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
62
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
63 class PublishingError(Exception):
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
64 pass
758
6abb436fea5b publish: Make publisher more powerful and better exposed on the command line.
Ludovic Chabant <ludovic@chabant.com>
parents: 621
diff changeset
65
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
66
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
67 class PublishingManager:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
68 def __init__(self, appfactory, app):
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
69 self.appfactory = appfactory
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
70 self.app = app
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
71
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
72 def run(self, target,
954
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
73 force=False, preview=False, extra_args=None,
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
74 log_file=None, log_debug_info=False, append_log_file=False):
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
75 start_time = time.perf_counter()
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
76
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
77 # Get publisher for this target.
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
78 pub = self.app.getPublisher(target)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
79 if pub is None:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
80 raise InvalidPublishTargetError(
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
81 "No such publish target: %s" % target)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
82
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
83 # Will we need to bake first?
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
84 bake_first = pub.config.get('bake', True)
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
85
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
86 # Setup logging stuff.
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
87 hdlr = None
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
88 root_logger = logging.getLogger()
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
89 if log_file and not preview:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
90 logger.debug("Adding file handler for: %s" % log_file)
954
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
91 mode = 'w'
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
92 if append_log_file:
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
93 mode = 'a'
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
94 hdlr = logging.FileHandler(log_file, mode=mode, encoding='utf8')
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
95 root_logger.addHandler(hdlr)
954
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
96
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
97 if log_debug_info:
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
98 _log_debug_info(target, force, preview, extra_args)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
99
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
100 if not preview:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
101 logger.info("Deploying to %s" % target)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
102 else:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
103 logger.info("Previewing deployment to %s" % target)
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
104
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
105 # Bake first is necessary.
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
106 records = None
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
107 was_baked = False
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
108 bake_out_dir = os.path.join(self.app.root_dir, '_pub', target)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
109 if bake_first:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
110 if not preview:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
111 bake_start_time = time.perf_counter()
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
112 logger.debug("Baking first to: %s" % bake_out_dir)
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
113
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
114 from piecrust.baking.baker import Baker
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
115 baker = Baker(
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
116 self.appfactory, self.app, bake_out_dir, force=force)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
117 records = baker.bake()
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
118 was_baked = True
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
119
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
120 if not records.success:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
121 raise Exception(
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
122 "Error during baking, aborting publishing.")
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
123 logger.info(format_timed(bake_start_time, "Baked website."))
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
124 else:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
125 logger.info("Would bake to: %s" % bake_out_dir)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
126
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
127 # Publish!
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
128 logger.debug(
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
129 "Running publish target '%s' with publisher: %s" %
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
130 (target, pub.PUBLISHER_NAME))
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
131 pub_start_time = time.perf_counter()
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
132
954
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
133 success = False
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
134 ctx = PublishingContext()
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
135 ctx.bake_out_dir = bake_out_dir
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
136 ctx.bake_records = records
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
137 ctx.was_baked = was_baked
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
138 ctx.preview = preview
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
139 ctx.args = extra_args
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
140 try:
954
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
141 success = pub.run(ctx)
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
142 except Exception as ex:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
143 raise PublishingError(
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
144 "Error publishing to target: %s" % target) from ex
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
145 finally:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
146 if hdlr:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
147 root_logger.removeHandler(hdlr)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
148 hdlr.close()
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
149
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
150 logger.info(format_timed(
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
151 pub_start_time, "Ran publisher %s" % pub.PUBLISHER_NAME))
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
152
954
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
153 if success:
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
154 logger.info(format_timed(start_time, 'Deployed to %s' % target))
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
155 return 0
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
156 else:
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
157 logger.error(format_timed(start_time, 'Failed to deploy to %s' %
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
158 target))
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
159 return 1
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
160
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
161
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
162 def find_publisher_class(app, name, is_scheme=False):
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
163 attr_name = 'PUBLISHER_SCHEME' if is_scheme else 'PUBLISHER_NAME'
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
164 for pub_cls in app.plugin_loader.getPublishers():
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
165 pub_sch = getattr(pub_cls, attr_name, None)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
166 if pub_sch == name:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
167 return pub_cls
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
168 return None
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
169
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
170
885
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
171 def find_publisher_name(app, scheme):
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
172 pub_cls = find_publisher_class(app, scheme, True)
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
173 if pub_cls:
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
174 return pub_cls.PUBLISHER_NAME
13e8b50a2113 publish: Fix publishers API and add a simple "copy" publisher.
Ludovic Chabant <ludovic@chabant.com>
parents: 805
diff changeset
175 return None
621
8f9c0bdb3724 publish: Polish/refactor the publishing workflows.
Ludovic Chabant <ludovic@chabant.com>
parents: 613
diff changeset
176
954
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
177
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
178 def _log_debug_info(target, force, preview, extra_args):
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
179 import os
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
180 import sys
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
181
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
182 logger.info("---- DEBUG INFO START ----")
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
183 logger.info("System:")
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
184 logger.info(" sys.argv=%s" % sys.argv)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
185 logger.info(" sys.base_exec_prefix=%s" % sys.base_exec_prefix)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
186 logger.info(" sys.base_prefix=%s" % sys.base_prefix)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
187 logger.info(" sys.exec_prefix=%s" % sys.exec_prefix)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
188 logger.info(" sys.executable=%s" % sys.executable)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
189 logger.info(" sys.path=%s" % sys.path)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
190 logger.info(" sys.platform=%s" % sys.platform)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
191 logger.info(" sys.prefix=%s" % sys.prefix)
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
192 logger.info("Environment:")
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
193 logger.info(" cwd=%s" % os.getcwd())
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
194 logger.info(" pid=%s" % os.getpid())
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
195 logger.info("Variables:")
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
196 for k, v in os.environ.items():
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
197 logger.info(" %s=%s" % (k, v))
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
198 logger.info("---- DEBUG INFO END ----")
d709429f02eb publish: Add more options for logging, better feedback when it fails.
Ludovic Chabant <ludovic@chabant.com>
parents: 885
diff changeset
199