piecrust2: piecrust/data/builder.py annotate

annotate piecrust/data/builder.py @ 852:4850f8c21b6e

core: Start of the big refactor for PieCrust 3.0. * Everything is a `ContentSource`, including assets directories. * Most content sources are subclasses of the base file-system source. * A source is processed by a "pipeline", and there are 2 built-in pipelines, one for assets and one for pages. The asset pipeline is vaguely functional, but the page pipeline is completely broken right now. * Rewrite the baking process as just running appropriate pipelines on each content item. This should allow for better parallelization.

author	Ludovic Chabant <ludovic@chabant.com>
date	Wed, 17 May 2017 00:11:48 -0700
parents	71c4f43d8fc1
children	f070a4fc033c

rev	line source
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import logging
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	2 from piecrust.data.base import MergedMapping
247 d9d5c5de02a8 data: Add a top level wrapper for `Linker`. Ludovic Chabant <ludovic@chabant.com> parents: 237 diff changeset	3 from piecrust.data.linker import PageLinkerData
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	4 from piecrust.data.pagedata import PageData
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 from piecrust.data.paginator import Paginator
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	6 from piecrust.data.piecrustdata import PieCrustData
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	7 from piecrust.data.providersdata import DataProvidersData
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	8 from piecrust.routing import RouteFunction
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	9
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14 class DataBuildingContext(object):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	15 def __init__(self, qualified_page):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	16 self.qualified_page = qualified_page
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17 self.pagination_source = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 self.pagination_filter = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 def build_page_data(ctx):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	22 qpage = ctx.qualified_page
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	23 page = qpage.page
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	24 app = page.app
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	25 pgn_source = ctx.pagination_source or get_default_pagination_source(page)
723 606f6d57b5df routing: Cleanup URL routing and improve page matching. Ludovic Chabant <ludovic@chabant.com> parents: 556 diff changeset	26 first_uri = ctx.page.getUri(1)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27
12 30a42341cfa8 Define page slugs properly, avoid recursions with debug data. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	28 pc_data = PieCrustData()
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	29 config_data = PageData(page, ctx)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	30 paginator = Paginator(qpage, pgn_source,
369 4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages. Ludovic Chabant <ludovic@chabant.com> parents: 352 diff changeset	31 pgn_filter=ctx.pagination_filter)
830 a12ad254176e Renamed buildPageAssetor to buildAssetor Ben Artin <ben@artins.org> parents: 826 diff changeset	32 assetor = page.source.buildAssetor(page, first_uri)
247 d9d5c5de02a8 data: Add a top level wrapper for `Linker`. Ludovic Chabant <ludovic@chabant.com> parents: 237 diff changeset	33 linker = PageLinkerData(page.source, page.rel_path)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	34 data = {
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	35 'piecrust': pc_data,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	36 'page': config_data,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	37 'assets': assetor,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	38 'pagination': paginator,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	39 'family': linker
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	40 }
420 f1b759c188b0 internal: Optimize page data building. Ludovic Chabant <ludovic@chabant.com> parents: 406 diff changeset	41
802 0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	42 for route in app.routes:
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	43 name = route.func_name
847 71c4f43d8fc1 data: Don't add route functions or data providers that happen to be null. Ludovic Chabant <ludovic@chabant.com> parents: 830 diff changeset	44 if not name:
71c4f43d8fc1 data: Don't add route functions or data providers that happen to be null. Ludovic Chabant <ludovic@chabant.com> parents: 830 diff changeset	45 continue
71c4f43d8fc1 data: Don't add route functions or data providers that happen to be null. Ludovic Chabant <ludovic@chabant.com> parents: 830 diff changeset	46
802 0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	47 func = data.get(name)
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	48 if func is None:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	49 data[name] = RouteFunction(route)
802 0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	50 else:
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	51 raise Exception("Route function '%s' collides with an "
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	52 "existing function or template data." %
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	53 name)
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	54
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	55 # TODO: handle slugified taxonomy terms.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	56
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	57 site_data = app.config.getAll()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	58 providers_data = DataProvidersData(page)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	59 data = MergedMapping([data, providers_data, site_data])
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 # Do this at the end because we want all the data to be ready to be
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	62 # displayed in the debugger window.
223 7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`. Ludovic Chabant <ludovic@chabant.com> parents: 222 diff changeset	63 if (app.config.get('site/show_debug_info') and
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	64 not app.config.get('baker/is_baking')):
556 93b656f0af54 serve: Improve debug information in the preview server. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	65 pc_data.enableDebugInfo(page)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67 return data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	69
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	70 def add_layout_data(page_data, contents):
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	71 for name, txt in contents.items():
369 4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages. Ludovic Chabant <ludovic@chabant.com> parents: 352 diff changeset	72 if name in page_data:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73 logger.warning("Content segment '%s' will hide existing data." %
369 4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages. Ludovic Chabant <ludovic@chabant.com> parents: 352 diff changeset	74 name)
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	75 page_data._prependMapping(contents)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	76
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	77
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 def get_default_pagination_source(page):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 app = page.app
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 source_name = page.config.get('source') or page.config.get('blog')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 if source_name is None:
323 412537e91e45 pagination: Add support for `site/default_pagination_source`. Ludovic Chabant <ludovic@chabant.com> parents: 298 diff changeset	82 source_name = app.config.get('site/default_pagination_source')
412537e91e45 pagination: Add support for `site/default_pagination_source`. Ludovic Chabant <ludovic@chabant.com> parents: 298 diff changeset	83 if source_name is None:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84 blog_names = app.config.get('site/blogs')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	85 if blog_names is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86 source_name = blog_names[0]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 source_name = app.sources[0].name
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 source = app.getSource(source_name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	90 return source
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	91

Mercurial > piecrust2

annotate piecrust/data/builder.py @ 852:4850f8c21b6e