piecrust2: piecrust/data/builder.py annotate

annotate piecrust/data/builder.py @ 853:f070a4fc033c

core: Continue PieCrust3 refactor, simplify pages. The asset pipeline is still the only function pipeline at this point. * No more `QualifiedPage`, and several other pieces of code deleted. * Data providers are simpler and more focused. For instance, the page iterator doesn't try to support other types of items. * Route parameters are proper known source metadata to remove the confusion between the two. * Make the baker and pipeline more correctly manage records and record histories. * Add support for record collapsing and deleting stale outputs in the asset pipeline.

author	Ludovic Chabant <ludovic@chabant.com>
date	Sun, 21 May 2017 00:06:59 -0700
parents	4850f8c21b6e
children	08e02c2a2a1a

rev	line source
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import logging
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	2 from piecrust.data.assetor import Assetor
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	3 from piecrust.data.base import MergedMapping
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	4 # from piecrust.data.linker import PageLinkerData
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	5 from piecrust.data.pagedata import PageData
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	6 from piecrust.data.paginator import Paginator
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	7 from piecrust.data.piecrustdata import PieCrustData
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	8 from piecrust.data.providersdata import DataProvidersData
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	9 from piecrust.routing import RouteFunction
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12 logger = logging.getLogger(__name__)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	15 class DataBuildingContext:
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	16 def __init__(self, page, sub_num):
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	17 self.page = page
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	18 self.sub_num = sub_num
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19 self.pagination_source = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20 self.pagination_filter = None
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 def build_page_data(ctx):
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	24 page = ctx.page
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	25 sub_num = ctx.sub_num
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	26 app = page.app
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	27
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	28 pgn_source = ctx.pagination_source or get_default_pagination_source(page)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29
12 30a42341cfa8 Define page slugs properly, avoid recursions with debug data. Ludovic Chabant <ludovic@chabant.com> parents: 5 diff changeset	30 pc_data = PieCrustData()
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	31 config_data = PageData(page, ctx)
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	32 paginator = Paginator(pgn_source, page, sub_num,
369 4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages. Ludovic Chabant <ludovic@chabant.com> parents: 352 diff changeset	33 pgn_filter=ctx.pagination_filter)
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	34 assetor = Assetor(page)
f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	35 # linker = PageLinkerData(page.source, page.rel_path)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	36 data = {
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	37 'piecrust': pc_data,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	38 'page': config_data,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	39 'assets': assetor,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	40 'pagination': paginator,
853 f070a4fc033c core: Continue PieCrust3 refactor, simplify pages. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	41 # 'family': linker
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	42 }
420 f1b759c188b0 internal: Optimize page data building. Ludovic Chabant <ludovic@chabant.com> parents: 406 diff changeset	43
802 0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	44 for route in app.routes:
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	45 name = route.func_name
847 71c4f43d8fc1 data: Don't add route functions or data providers that happen to be null. Ludovic Chabant <ludovic@chabant.com> parents: 830 diff changeset	46 if not name:
71c4f43d8fc1 data: Don't add route functions or data providers that happen to be null. Ludovic Chabant <ludovic@chabant.com> parents: 830 diff changeset	47 continue
71c4f43d8fc1 data: Don't add route functions or data providers that happen to be null. Ludovic Chabant <ludovic@chabant.com> parents: 830 diff changeset	48
802 0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	49 func = data.get(name)
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	50 if func is None:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	51 data[name] = RouteFunction(route)
802 0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	52 else:
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	53 raise Exception("Route function '%s' collides with an "
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	54 "existing function or template data." %
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	55 name)
0da1207472d3 templating: Put the routing functions in the data, not the template engine. Ludovic Chabant <ludovic@chabant.com> parents: 723 diff changeset	56
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	57 # TODO: handle slugified taxonomy terms.
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	58
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	59 site_data = app.config.getAll()
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	60 providers_data = DataProvidersData(page)
32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	61 data = MergedMapping([data, providers_data, site_data])
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	62
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	63 # Do this at the end because we want all the data to be ready to be
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	64 # displayed in the debugger window.
223 7decf00eee47 serve: Don't expose the debug info right away when running with `--debug`. Ludovic Chabant <ludovic@chabant.com> parents: 222 diff changeset	65 if (app.config.get('site/show_debug_info') and
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66 not app.config.get('baker/is_baking')):
556 93b656f0af54 serve: Improve debug information in the preview server. Ludovic Chabant <ludovic@chabant.com> parents: 440 diff changeset	67 pc_data.enableDebugInfo(page)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	69 return data
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	70
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: 847 diff changeset	72 def add_layout_data(page_data, contents):
5 474c9882decf Upgrade to Python 3. Ludovic Chabant <ludovic@chabant.com> parents: 3 diff changeset	73 for name, txt in contents.items():
369 4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages. Ludovic Chabant <ludovic@chabant.com> parents: 352 diff changeset	74 if name in page_data:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	75 logger.warning("Content segment '%s' will hide existing data." %
369 4b1019bb2533 serve: Giant refactor to change how we handle data when serving pages. Ludovic Chabant <ludovic@chabant.com> parents: 352 diff changeset	76 name)
440 32c7c2d219d2 performance: Refactor how data is managed to reduce copying. Ludovic Chabant <ludovic@chabant.com> parents: 420 diff changeset	77 page_data._prependMapping(contents)
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 def get_default_pagination_source(page):
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 app = page.app
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 source_name = page.config.get('source') or page.config.get('blog')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 if source_name is None:
323 412537e91e45 pagination: Add support for `site/default_pagination_source`. Ludovic Chabant <ludovic@chabant.com> parents: 298 diff changeset	84 source_name = app.config.get('site/default_pagination_source')
412537e91e45 pagination: Add support for `site/default_pagination_source`. Ludovic Chabant <ludovic@chabant.com> parents: 298 diff changeset	85 if source_name is None:
3 f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86 blog_names = app.config.get('site/blogs')
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87 if blog_names is not None:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 source_name = blog_names[0]
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 else:
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	90 source_name = app.sources[0].name
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	91 source = app.getSource(source_name)
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	92 return source
f485ba500df3 Gigantic change to basically make PieCrust 2 vaguely functional. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	93

Mercurial > piecrust2

annotate piecrust/data/builder.py @ 853:f070a4fc033c