Mercurial > piecrust2

annotate piecrust/sources/mixins.py @ 411:e7b865f8f335

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
bake: Enable multiprocess baking. Baking is now done by running a worker per CPU, and sending jobs to them. This changes several things across the codebase: * Ability to not cache things related to pages other than the 'main' page (i.e. the page at the bottom of the execution stack). * Decouple the baking process from the bake records, so only the main process keeps track (and modifies) the bake record. * Remove the need for 'batch page getters' and loading a page directly from the page factories. There are various smaller changes too included here, including support for scope performance timers that are saved with the bake record and can be printed out to the console. Yes I got carried away. For testing, the in-memory 'mock' file-system doesn't work anymore, since we're spawning processes, so this is replaced by a 'tmpfs' file-system which is saved in temporary files on disk and deleted after tests have run.
author Ludovic Chabant <ludovic@chabant.com>
date Fri, 12 Jun 2015 17:09:19 -0700
parents 4b1019bb2533
children 32c7c2d219d2
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 import os
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2 import os.path
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 import logging
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 from piecrust.data.base import PaginationData
280
8c0c53a315ae data: Correctly build pagination filters when we know items are pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 260
diff changeset
5 from piecrust.data.filters import PaginationFilter, page_value_accessor
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6 from piecrust.sources.base import PageFactory
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
7 from piecrust.sources.interfaces import IPaginationSource, IListableSource
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
8 from piecrust.sources.pageref import PageRef
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
9
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
10
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
11 logger = logging.getLogger(__name__)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
12
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
13
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
14 class SourceFactoryIterator(object):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
15 def __init__(self, source):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
16 self.source = source
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
17
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
18 # This is to permit recursive traversal of the
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
19 # iterator chain. It acts as the end.
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
20 self.it = None
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
21
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
22 def __iter__(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
23 return self.source.getPages()
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
24
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
25
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
26 class SourceFactoryWithoutTaxonomiesIterator(object):
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
27 def __init__(self, source):
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
28 self.source = source
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
29 self._taxonomy_pages = None
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
30 # See comment above.
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
31 self.it = None
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
32
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
33 def __iter__(self):
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
34 self._cacheTaxonomyPages()
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
35 for p in self.source.getPages():
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
36 if p.rel_path in self._taxonomy_pages:
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
37 continue
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
38 yield p
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
39
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
40 def _cacheTaxonomyPages(self):
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
41 if self._taxonomy_pages is not None:
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
42 return
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
43
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
44 app = self.source.app
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
45 self._taxonomy_pages = set()
411
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
46 for src in app.sources:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
47 for tax in app.taxonomies:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
48 ref_spec = src.getTaxonomyPageRef(tax.name)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
49 page_ref = PageRef(app, ref_spec)
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
50 for sn, rp in page_ref.possible_split_ref_specs:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
51 if sn == self.source.name:
e7b865f8f335 bake: Enable multiprocess baking.
Ludovic Chabant <ludovic@chabant.com>
parents: 369
diff changeset
52 self._taxonomy_pages.add(rp)
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
53
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
54
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
55 class DateSortIterator(object):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
56 def __init__(self, it, reverse=True):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
57 self.it = it
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
58 self.reverse = reverse
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
59
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
60 def __iter__(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
61 return iter(sorted(self.it,
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
62 key=lambda x: x.datetime, reverse=self.reverse))
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
63
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
64
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
65 class PaginationDataBuilderIterator(object):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
66 def __init__(self, it):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
67 self.it = it
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
68
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
69 def __iter__(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
70 for page in self.it:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
71 if page is None:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
72 yield None
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
73 else:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
74 yield PaginationData(page)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
75
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
76
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
77 class SimplePaginationSourceMixin(IPaginationSource):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
78 """ Implements the `IPaginationSource` interface in a standard way that
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
79 should fit most page sources.
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
80 """
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
81 def getItemsPerPage(self):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
82 return self.config['items_per_page']
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
83
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
84 def getSourceIterator(self):
369
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
85 if self.config.get('iteration_includes_taxonomies', False):
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
86 return SourceFactoryIterator(self)
4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 280
diff changeset
87 return SourceFactoryWithoutTaxonomiesIterator(self)
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
88
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
89 def getSorterIterator(self, it):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
90 return DateSortIterator(it)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
91
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
92 def getTailIterator(self, it):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
93 return PaginationDataBuilderIterator(it)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
94
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
95 def getPaginationFilter(self, page):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
96 conf = (page.config.get('items_filters') or
260
07b4b8484c0a sources: Use `posts_*` and `items_*` settings more appropriately.
Ludovic Chabant <ludovic@chabant.com>
parents: 242
diff changeset
97 self.config.get('items_filters'))
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
98 if conf == 'none' or conf == 'nil' or conf == '':
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
99 conf = None
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
100 if conf is not None:
280
8c0c53a315ae data: Correctly build pagination filters when we know items are pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 260
diff changeset
101 f = PaginationFilter(value_accessor=page_value_accessor)
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
102 f.addClausesFromConfig(conf)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
103 return f
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
104 return None
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
105
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
106 def getSettingAccessor(self):
280
8c0c53a315ae data: Correctly build pagination filters when we know items are pages.
Ludovic Chabant <ludovic@chabant.com>
parents: 260
diff changeset
107 return page_value_accessor
242
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
108
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
109
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
110 class SimpleListableSourceMixin(IListableSource):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
111 """ Implements the `IListableSource` interface for sources that map to
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
112 simple file-system structures.
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
113 """
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
114 def listPath(self, rel_path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
115 rel_path = rel_path.lstrip('\\/')
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
116 path = self._getFullPath(rel_path)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
117 names = self._sortFilenames(os.listdir(path))
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
118
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
119 items = []
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
120 for name in names:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
121 if os.path.isdir(os.path.join(path, name)):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
122 if self._filterPageDirname(name):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
123 rel_subdir = os.path.join(rel_path, name)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
124 items.append((True, name, rel_subdir))
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
125 else:
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
126 if self._filterPageFilename(name):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
127 slug = self._makeSlug(os.path.join(rel_path, name))
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
128 metadata = {'slug': slug}
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
129
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
130 fac_path = name
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
131 if rel_path != '.':
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
132 fac_path = os.path.join(rel_path, name)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
133 fac_path = fac_path.replace('\\', '/')
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
134
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
135 self._populateMetadata(fac_path, metadata)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
136 fac = PageFactory(self, fac_path, metadata)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
137
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
138 name, _ = os.path.splitext(name)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
139 items.append((False, name, fac))
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
140 return items
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
141
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
142 def getDirpath(self, rel_path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
143 return os.path.dirname(rel_path)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
144
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
145 def getBasename(self, rel_path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
146 filename = os.path.basename(rel_path)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
147 name, _ = os.path.splitext(filename)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
148 return name
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
149
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
150 def _getFullPath(self, rel_path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
151 return os.path.join(self.fs_endpoint_path, rel_path)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
152
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
153 def _sortFilenames(self, names):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
154 return sorted(names)
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
155
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
156 def _filterPageDirname(self, name):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
157 return True
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
158
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
159 def _filterPageFilename(self, name):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
160 return True
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
161
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
162 def _makeSlug(self, rel_path):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
163 return rel_path.replace('\\', '/')
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
164
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
165 def _populateMetadata(self, rel_path, metadata, mode=None):
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
166 pass
f130365568ff internal: Code reorganization to put less stuff in `sources.base`.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
167