Mercurial > piecrust2

annotate piecrust/processing/compressors.py @ 1163:ed308313bcda

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
data: Allow combining different data providers.
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 01 Oct 2019 07:35:50 -0700
parents a3dec0fbd9ce
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import os
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os.path
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import logging
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import platform
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 import subprocess
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
6 from piecrust.processing.base import Processor, SimpleFileProcessor
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
8
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9 logger = logging.getLogger(__name__)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
12 class CleanCssProcessor(Processor):
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13 PROCESSOR_NAME = 'cleancss'
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 def __init__(self):
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
16 super(CleanCssProcessor, self).__init__()
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 self._conf = None
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
19 def matches(self, path):
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 339
diff changeset
20 return path.endswith('.css') and not path.endswith('.min.css')
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
21
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
22 def getOutputFilenames(self, filename):
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 self._ensureInitialized()
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
24 basename, _ = os.path.splitext(filename)
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
25 return ['%s%s' % (basename, self._conf['out_ext'])]
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
26
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
27 def process(self, path, out_dir):
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
28 self._ensureInitialized()
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
29
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
30 _, in_name = os.path.split(path)
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
31 out_name = self.getOutputFilenames(in_name)[0]
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
32 out_path = os.path.join(out_dir, out_name)
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34 args = [self._conf['bin'], '-o', out_path]
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
35 args += self._conf['options']
339
0ab712eab0fb cleancss: Fix stupid bug.
Ludovic Chabant <ludovic@chabant.com>
parents: 315
diff changeset
36 args.append(path)
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
37 logger.debug("Cleaning CSS file: %s" % args)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
38
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
39 try:
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
40 retcode = subprocess.call(args)
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
41 except FileNotFoundError as ex:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
42 logger.error("Tried running CleanCSS processor with command: %s" %
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
43 args)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
44 raise Exception("Error running CleanCSS processor. "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
45 "Did you install it?") from ex
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
46 if retcode != 0:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
47 raise Exception("Error occured in CleanCSS. Please check "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
48 "log messages above for more information.")
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
49 return True
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
50
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
51 def _ensureInitialized(self):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
52 if self._conf is not None:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
53 return
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
54
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
55 bin_name = 'cleancss'
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
56 if platform.system() == 'Windows':
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
57 bin_name += '.cmd'
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
58
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
59 self._conf = self.app.config.get('cleancss') or {}
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
60 self._conf.setdefault('bin', bin_name)
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
61 self._conf.setdefault('options', ['--skip-rebase'])
315
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
62 self._conf.setdefault('out_ext', '.css')
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
63 if len(self._conf['out_ext']) > 0 and self._conf['out_ext'][0] != '.':
d1490028e211 cleancss: Add option to specify an output extension, like `.min.css`.
Ludovic Chabant <ludovic@chabant.com>
parents: 206
diff changeset
64 self._conf['out_ext'] = '.' + self._conf['out_ext']
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
65 if not isinstance(self._conf['options'], list):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
66 raise Exception("The `cleancss/options` configuration setting "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
67 "must be an array of arguments.")
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
68
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
69
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
70 class UglifyJSProcessor(SimpleFileProcessor):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
71 PROCESSOR_NAME = 'uglifyjs'
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
72
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73 def __init__(self):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 super(UglifyJSProcessor, self).__init__({'js': 'js'})
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
75 self._conf = None
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
76
853
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 339
diff changeset
77 def matches(self, path):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 339
diff changeset
78 return path.endswith('.js') and not path.endswith('.min.js')
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 339
diff changeset
79
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80 def _doProcess(self, in_path, out_path):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81 self._ensureInitialized()
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
83 args = [self._conf['bin'], in_path, '-o', out_path]
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
84 args += self._conf['options']
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
85 logger.debug("Uglifying JS file: %s" % args)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
86
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
87 try:
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
88 retcode = subprocess.call(args)
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
89 except FileNotFoundError as ex:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
90 logger.error("Tried running UglifyJS processor with command: %s" %
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
91 args)
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
92 raise Exception("Error running UglifyJS processor. "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
93 "Did you install it?") from ex
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
94 if retcode != 0:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
95 raise Exception("Error occured in UglifyJS. Please check "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
96 "log messages above for more information.")
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
97 return True
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
98
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
99 def _ensureInitialized(self):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
100 if self._conf is not None:
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
101 return
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
102
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
103 bin_name = 'uglifyjs'
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
104 if platform.system() == 'Windows':
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
105 bin_name += '.cmd'
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
106
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
107 self._conf = self.app.config.get('uglifyjs') or {}
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 853
diff changeset
108 self._conf.setdefault('bin', bin_name)
206
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
109 self._conf.setdefault('options', ['--compress'])
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
110 if not isinstance(self._conf['options'], list):
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
111 raise Exception("The `uglify/options` configuration setting "
cba781477bd0 processing: Add `concat`, `uglifyjs` and `cleancss` processors.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
112 "must be an array of arguments.")