Mercurial > piecrust2

annotate piecrust/processing/less.py @ 1155:fac4483867a5

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
less: Fix issues moving the map file on Windows. Again.
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 11 Jun 2019 15:11:39 -0700
parents a3dec0fbd9ce
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import os
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os.path
221
f82262f59600 bake: Fix processing record bugs and error logging for external processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 195
diff changeset
3 import sys
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 import json
1155
fac4483867a5 less: Fix issues moving the map file on Windows. Again.
Ludovic Chabant <ludovic@chabant.com>
parents: 1112
diff changeset
5 import shutil
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6 import hashlib
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7 import logging
17
de6a296744f7 The LESS compiler must be launched in a shell on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
8 import platform
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9 import subprocess
221
f82262f59600 bake: Fix processing record bugs and error logging for external processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 195
diff changeset
10 from piecrust.processing.base import (
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
11 SimpleFileProcessor, ExternalProcessException, FORCE_BUILD)
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17 class LessProcessor(SimpleFileProcessor):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18 PROCESSOR_NAME = 'less'
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20 def __init__(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21 super(LessProcessor, self).__init__({'less': 'css'})
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
22 self._conf = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 self._map_dir = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
25 def onPipelineStart(self, ctx):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
26 self._map_dir = os.path.join(ctx.tmp_dir, 'less')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
27 if (ctx.is_main_process and
414
c4b3a7fd2f87 bake: Make pipeline processing multi-process.
Ludovic Chabant <ludovic@chabant.com>
parents: 240
diff changeset
28 not os.path.isdir(self._map_dir)):
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
29 os.makedirs(self._map_dir)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
30
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
31 def getDependencies(self, path):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
32 map_path = self._getMapPath(path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
33 try:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
34 with open(map_path, 'r') as f:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
35 dep_map = json.load(f)
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
36 except OSError:
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
37 # Map file not found... rebuild.
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
38 logger.debug("No map file found for LESS file '%s' at '%s'. "
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
39 "Rebuilding" % (path, map_path))
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
40 return FORCE_BUILD
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
41
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
42 # Check the version, since the `sources` list has changed
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
43 # meanings over time.
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
44 if dep_map.get('version') != 3:
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
45 logger.warning("Unknown LESS map version. Force rebuilding.")
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
46 return FORCE_BUILD
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
47
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
48 # Get the sources, but make all paths absolute.
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
49 sources = dep_map.get('sources')
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
50 path_dir = os.path.dirname(path)
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
51
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
52 def _makeAbs(p):
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
53 return os.path.join(path_dir, p)
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
54 deps = list(map(_makeAbs, sources))
1048
63be34ce6e65 bake: Don't add outputs as dependencies, duh.
Ludovic Chabant <ludovic@chabant.com>
parents: 852
diff changeset
55 return deps
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
56
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
57 def _doProcess(self, in_path, out_path):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
58 self._ensureInitialized()
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
59
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
60 map_path = self._getMapPath(in_path)
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
61 map_url = '/' + os.path.relpath(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
62 map_path, self.app.root_dir).replace('\\', '/')
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
63
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
64 # On Windows, it looks like LESSC is confused with paths when the
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
65 # map file is not to be created in the same directory as the input
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
66 # file (it ends up writing invalid dependencies in the map file, with
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
67 # a mix of relative and absolute paths stuck together).
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
68 # So create it there and move it afterwards... :(
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
69 temp_map_path = os.path.join(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
70 os.path.dirname(in_path),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
71 os.path.basename(map_path))
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
72
240
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 221
diff changeset
73 args = [self._conf['bin'],
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
74 '--source-map=%s' % temp_map_path,
240
c1d4e86a3918 less: Generate a proper, available URL for the LESS CSS map file.
Ludovic Chabant <ludovic@chabant.com>
parents: 221
diff changeset
75 '--source-map-url=%s' % map_url]
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
76 args += self._conf['options']
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
77 args.append(in_path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
78 args.append(out_path)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
79 logger.debug("Processing LESS file: %s" % args)
17
de6a296744f7 The LESS compiler must be launched in a shell on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
80
de6a296744f7 The LESS compiler must be launched in a shell on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
81 try:
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 1048
diff changeset
82 proc = subprocess.Popen(args, stderr=subprocess.PIPE)
221
f82262f59600 bake: Fix processing record bugs and error logging for external processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 195
diff changeset
83 stdout_data, stderr_data = proc.communicate()
17
de6a296744f7 The LESS compiler must be launched in a shell on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
84 except FileNotFoundError as ex:
de6a296744f7 The LESS compiler must be launched in a shell on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
85 logger.error("Tried running LESS processor with command: %s" %
195
b4724e577a8c cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents: 119
diff changeset
86 args)
17
de6a296744f7 The LESS compiler must be launched in a shell on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
87 raise Exception("Error running LESS processor. "
de6a296744f7 The LESS compiler must be launched in a shell on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 7
diff changeset
88 "Did you install it?") from ex
221
f82262f59600 bake: Fix processing record bugs and error logging for external processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 195
diff changeset
89 if proc.returncode != 0:
f82262f59600 bake: Fix processing record bugs and error logging for external processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 195
diff changeset
90 raise ExternalProcessException(
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
91 stderr_data.decode(sys.stderr.encoding))
221
f82262f59600 bake: Fix processing record bugs and error logging for external processes.
Ludovic Chabant <ludovic@chabant.com>
parents: 195
diff changeset
92
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
93 logger.debug("Moving map file: %s -> %s" % (temp_map_path, map_path))
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
94 if os.path.exists(map_path):
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
95 os.remove(map_path)
1155
fac4483867a5 less: Fix issues moving the map file on Windows. Again.
Ludovic Chabant <ludovic@chabant.com>
parents: 1112
diff changeset
96 shutil.move(temp_map_path, map_path)
498
35d4c172e5a6 less: Fix issues with the map file on Windows.
Ludovic Chabant <ludovic@chabant.com>
parents: 414
diff changeset
97
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
98 return True
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
99
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
100 def _ensureInitialized(self):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
101 if self._conf is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
102 return
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
103
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 1048
diff changeset
104 bin_name = 'lessc'
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 1048
diff changeset
105 if platform.system() == 'Windows':
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 1048
diff changeset
106 bin_name += '.cmd'
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 1048
diff changeset
107
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
108 self._conf = self.app.config.get('less') or {}
1112
a3dec0fbd9ce bake: Fix bug on Windows where shim scripts of NodeJS tools couldn't run.
Ludovic Chabant <ludovic@chabant.com>
parents: 1048
diff changeset
109 self._conf.setdefault('bin', bin_name)
195
b4724e577a8c cosmetic: Fix some PEP8 issues.
Ludovic Chabant <ludovic@chabant.com>
parents: 119
diff changeset
110 self._conf.setdefault('options', ['--compress'])
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
111 if not isinstance(self._conf['options'], list):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
112 raise Exception("The `less/options` configuration setting "
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
113 "must be an array of arguments.")
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
114
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
115 def _getMapPath(self, path):
119
0811f92cbdc7 Slightly more robust dependency handling for the LESS processor.
Ludovic Chabant <ludovic@chabant.com>
parents: 116
diff changeset
116 map_name = "%s_%s.map" % (
852
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
117 os.path.basename(path),
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0.
Ludovic Chabant <ludovic@chabant.com>
parents: 498
diff changeset
118 hashlib.md5(path.encode('utf8')).hexdigest())
3
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119 map_path = os.path.join(self._map_dir, map_name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
120 return map_path
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
121