Mercurial > piecrust2

annotate piecrust/processing/compressors.py @ 380:f33712c4cfab

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
routing: Fix bugs with matching URLs with correct route but missing metadata. When matching a route like `/foo/%slug%` against an URL like `/foo`, the route will (correctly) return a match, but it will be completely missing the `slug` metadata, resulting in problems elsewhere. This change makes it so that any missing route metadata will be filled in with an empty string. And because this means generated URLs may differ from the incoming URL when using trailing slashes (`/foo/` _vs._ `/foo`), we make the assert in the chef server handle those discrepancies.
author Ludovic Chabant <ludovic@chabant.com>
date Sun, 10 May 2015 00:34:21 -0700
parents 0ab712eab0fb
children f070a4fc033c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import os
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os.path
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import logging
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import platform
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import subprocess
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
6 from piecrust.processing.base import Processor, SimpleFileProcessor
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9 logger = logging.getLogger(__name__)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
12 class CleanCssProcessor(Processor):
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13 PROCESSOR_NAME = 'cleancss'
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 def __init__(self):
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
16 super(CleanCssProcessor, self).__init__()
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 self._conf = None
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
19 def matches(self, path):
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
20 return path.endswith('.css')
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
21
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
22 def getOutputFilenames(self, filename):
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 self._ensureInitialized()
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
24 basename, _ = os.path.splitext(filename)
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
25 return ['%s%s' % (basename, self._conf['out_ext'])]
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
26
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
27 def process(self, path, out_dir):
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
28 self._ensureInitialized()
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
29
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
30 _, in_name = os.path.split(path)
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
31 out_name = self.getOutputFilenames(in_name)[0]
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
32 out_path = os.path.join(out_dir, out_name)
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34 args = [self._conf['bin'], '-o', out_path]
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
35 args += self._conf['options']
339
0ab712eab0fb cleancss: Fix stupid bug.
Ludovic Chabant <ludovic@chabant.com>
parents: 315
diff changeset
36 args.append(path)
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
37 logger.debug("Cleaning CSS file: %s" % args)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
38
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
39 # On Windows, we need to run the process in a shell environment
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
40 # otherwise it looks like `PATH` isn't taken into account.
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
41 shell = (platform.system() == 'Windows')
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
42 try:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
43 retcode = subprocess.call(args, shell=shell)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44 except FileNotFoundError as ex:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
45 logger.error("Tried running CleanCSS processor with command: %s" %
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
46 args)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47 raise Exception("Error running CleanCSS processor. "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
48 "Did you install it?") from ex
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
49 if retcode != 0:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
50 raise Exception("Error occured in CleanCSS. Please check "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
51 "log messages above for more information.")
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
52 return True
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
53
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
54 def _ensureInitialized(self):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
55 if self._conf is not None:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
56 return
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
57
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
58 self._conf = self.app.config.get('cleancss') or {}
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
59 self._conf.setdefault('bin', 'cleancss')
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
60 self._conf.setdefault('options', ['--skip-rebase'])
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
61 self._conf.setdefault('out_ext', '.css')
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
62 if len(self._conf['out_ext']) > 0 and self._conf['out_ext'][0] != '.':
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
63 self._conf['out_ext'] = '.' + self._conf['out_ext']
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
64 if not isinstance(self._conf['options'], list):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
65 raise Exception("The `cleancss/options` configuration setting "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
66 "must be an array of arguments.")
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
67
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
68
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
69 class UglifyJSProcessor(SimpleFileProcessor):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
70 PROCESSOR_NAME = 'uglifyjs'
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
71
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
72 def __init__(self):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73 super(UglifyJSProcessor, self).__init__({'js': 'js'})
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 self._conf = None
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
75
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
76 def _doProcess(self, in_path, out_path):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
77 self._ensureInitialized()
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
78
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
79 args = [self._conf['bin'], in_path, '-o', out_path]
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80 args += self._conf['options']
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81 logger.debug("Uglifying JS file: %s" % args)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
83 # On Windows, we need to run the process in a shell environment
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
84 # otherwise it looks like `PATH` isn't taken into account.
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
85 shell = (platform.system() == 'Windows')
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
86 try:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
87 retcode = subprocess.call(args, shell=shell)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
88 except FileNotFoundError as ex:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
89 logger.error("Tried running UglifyJS processor with command: %s" %
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
90 args)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
91 raise Exception("Error running UglifyJS processor. "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
92 "Did you install it?") from ex
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
93 if retcode != 0:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
94 raise Exception("Error occured in UglifyJS. Please check "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
95 "log messages above for more information.")
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
96 return True
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
97
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
98 def _ensureInitialized(self):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
99 if self._conf is not None:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
100 return
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
101
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
102 self._conf = self.app.config.get('uglifyjs') or {}
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
103 self._conf.setdefault('bin', 'uglifyjs')
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
104 self._conf.setdefault('options', ['--compress'])
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
105 if not isinstance(self._conf['options'], list):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
106 raise Exception("The `uglify/options` configuration setting "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
107 "must be an array of arguments.")
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
108