piecrust2: piecrust/sources/taxonomy.py annotate

annotate piecrust/sources/taxonomy.py @ 877:d6d35b2efd04

bake: Rename "pass" to "step" and make the page pipeline use different steps. That pipeline is now first loading all pages, and then rendering full pages unless they trigger a sub-render.

author	Ludovic Chabant <ludovic@chabant.com>
date	Thu, 15 Jun 2017 22:16:23 -0700
parents	d1095774bfcf
children	acd9c3e8533f

rev	line source
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import re
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import logging
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import unidecode
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 from piecrust.configuration import ConfigurationError
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 from piecrust.data.filters import (
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	6 PaginationFilter, SettingFilterClause)
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	7 from piecrust.page import Page
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	8 from piecrust.pipelines._pagebaker import PageBaker
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	9 from piecrust.pipelines._pagerecords import PagePipelineRecordEntry
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	10 from piecrust.pipelines.base import (
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	11 ContentPipeline, get_record_name_for_source)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	12 from piecrust.pipelines.records import RecordHistory
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13 from piecrust.routing import RouteParameter
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 855 diff changeset	14 from piecrust.sources.base import ContentItem
9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 855 diff changeset	15 from piecrust.sources.generator import GeneratorSourceBase
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 logger = logging.getLogger(__name__)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 SLUGIFY_ENCODE = 1
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22 SLUGIFY_TRANSLITERATE = 2
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 SLUGIFY_LOWERCASE = 4
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 SLUGIFY_DOT_TO_DASH = 8
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 SLUGIFY_SPACE_TO_DASH = 16
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28 re_first_dot_to_dash = re.compile(r'^\.+')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	29 re_dot_to_dash = re.compile(r'\.+')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30 re_space_to_dash = re.compile(r'\s+')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	31
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	32
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	33 class Taxonomy(object):
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	34 """ Describes a taxonomy.
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	35 """
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	36 def __init__(self, name, config):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	37 self.name = name
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	38 self.config = config
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	39 self.term_name = config.get('term', name)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	40 self.is_multiple = bool(config.get('multiple', False))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	41 self.separator = config.get('separator', '/')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	42 self.page_ref = config.get('page')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	43
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	44 @property
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	45 def setting_name(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46 if self.is_multiple:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 return self.name
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48 return self.term_name
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	49
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	50
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	51 _taxonomy_index = """---
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	52 layout: %(template)s
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	53 ---
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	54 """
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	55
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	56
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 855 diff changeset	57 class TaxonomySource(GeneratorSourceBase):
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	58 """ A content source that generates taxonomy listing pages.
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	59 """
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60 SOURCE_NAME = 'taxonomy'
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	61 DEFAULT_PIPELINE_NAME = 'taxonomy'
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	62
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	63 def __init__(self, app, name, config):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	64 super().__init__(app, name, config)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	65
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66 tax_name = config.get('taxonomy')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67 if tax_name is None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68 raise ConfigurationError(
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	69 "Taxonomy source '%s' requires a taxonomy name." % name)
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	70 self.taxonomy = _get_taxonomy(app, tax_name)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 sm = config.get('slugify_mode')
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	73 self.slugifier = _get_slugifier(app, self.taxonomy, sm)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	74
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	75 tpl_name = config.get('template', '_%s.html' % tax_name)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	76 self._raw_item = _taxonomy_index % {'template': tpl_name}
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	77
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 def getSupportedRouteParameters(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 name = self.taxonomy.term_name
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 param_type = (RouteParameter.TYPE_PATH if self.taxonomy.is_multiple
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 else RouteParameter.TYPE_STRING)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 return [RouteParameter(name, param_type,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 variadic=self.taxonomy.is_multiple)]
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	85 def findContent(self, route_params):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	86 slugified_term = route_params[self.taxonomy.term_name]
876 d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	87 spec = '_index'
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	88 metadata = {'term': slugified_term,
876 d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	89 'record_entry_spec': '_index[%s]' % slugified_term,
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	90 'route_params': {
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	91 self.taxonomy.term_name: slugified_term}
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	92 }
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	93 return ContentItem(spec, metadata)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	94
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	95 def slugify(self, term):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	96 return self.slugifier.slugify(term)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	97
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	98 def slugifyMultiple(self, terms):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	99 return self.slugifier.slugifyMultiple(terms)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	100
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	101 def prepareRenderContext(self, ctx):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	102 # Set the pagination source as the source we're generating for.
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	103 ctx.pagination_source = self.inner_source
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	104
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105 # Get the taxonomy terms from the route metadata... this can come from
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106 # the browser's URL (while serving) or from the baking (see `bake`
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	107 # method below). In both cases, we expect to have the slugified
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	108 # version of the term, because we're going to set a filter that also
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	109 # slugifies the terms found on each page.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	110 #
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	111 # This is because:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	112 # * while serving, we get everything from the request URL, so we only
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	113 # have the slugified version.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	114 # * if 2 slightly different terms "collide" into the same slugified
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	115 # term, we'll get a merge of the 2 on the listing page, which is
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	116 # what the user expects.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	117 #
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	118 route_params = ctx.page.source_metadata['route_params']
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	119 tax_terms, is_combination = self._getTaxonomyTerms(route_params)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120 self._setTaxonomyFilter(ctx, tax_terms, is_combination)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122 # Add some custom data for rendering.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	123 ctx.custom_data.update({
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	124 self.taxonomy.term_name: tax_terms,
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	125 'is_multiple_%s' % self.taxonomy.term_name: is_combination})
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	126 # Add some "plural" version of the term... so for instance, if this
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	127 # is the "tags" taxonomy, "tag" will have one term most of the time,
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	128 # except when it's a combination. Here, we add "tags" as something that
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	129 # is always a tuple, even when it's not a combination.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	130 if (self.taxonomy.is_multiple and
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	131 self.taxonomy.name != self.taxonomy.term_name):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	132 mult_val = tax_terms
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	133 if not is_combination:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	134 mult_val = (mult_val,)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	135 ctx.custom_data[self.taxonomy.name] = mult_val
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	136
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	137 def _getTaxonomyTerms(self, route_params):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	138 # Get the individual slugified terms from the route metadata.
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	139 all_values = route_params.get(self.taxonomy.term_name)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	140 if all_values is None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	141 raise Exception("'%s' values couldn't be found in route metadata" %
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	142 self.taxonomy.term_name)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	143
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	144 # If it's a "multiple" taxonomy, we need to potentially split the
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	145 # route value into the individual terms (_e.g._ when listing all pages
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	146 # that have 2 given tags, we need to get each of those 2 tags).
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	147 if self.taxonomy.is_multiple:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	148 sep = self.taxonomy.separator
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	149 if sep in all_values:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	150 return tuple(all_values.split(sep)), True
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	151 # Not a "multiple" taxonomy, so there's only the one value.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	152 return all_values, False
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	153
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	154 def _setTaxonomyFilter(self, ctx, term_value, is_combination):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	155 # Set up the filter that will check the pages' terms.
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	156 flt = PaginationFilter()
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	157 flt.addClause(HasTaxonomyTermsFilterClause(
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	158 self.taxonomy, self.slugifier.mode, term_value, is_combination))
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159 ctx.pagination_filter = flt
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	160
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	161 def onRouteFunctionUsed(self, route_params):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	162 # Get the values, and slugify them appropriately.
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	163 values = route_params[self.taxonomy.term_name]
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	164 if self.taxonomy.is_multiple:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	165 # TODO: here we assume the route has been properly configured.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	166 slugified_values = self.slugifyMultiple((str(v) for v in values))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	167 route_val = self.taxonomy.separator.join(slugified_values)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	168 else:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	169 slugified_values = self.slugify(str(values))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	170 route_val = slugified_values
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	171
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	172 # We need to register this use of a taxonomy term.
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	173 rcs = self.app.env.render_ctx_stack
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	174 cpi = rcs.current_ctx.current_pass_info
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	175 if cpi:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	176 utt = cpi.getCustomInfo('used_taxonomy_terms', [], True)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	177 utt.append(slugified_values)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	178
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	179 # Put the slugified values in the route metadata so they're used to
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	180 # generate the URL.
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	181 route_params[self.taxonomy.term_name] = route_val
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	182
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	183
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	184 class HasTaxonomyTermsFilterClause(SettingFilterClause):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	185 def __init__(self, taxonomy, slugify_mode, value, is_combination):
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	186 super().__init__(taxonomy.setting_name, value)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	187 self._taxonomy = taxonomy
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	188 self._is_combination = is_combination
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	189 self._slugifier = _Slugifier(taxonomy, slugify_mode)
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	190 if taxonomy.is_multiple:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	191 self.pageMatches = self._pageMatchesAny
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	192 else:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	193 self.pageMatches = self._pageMatchesSingle
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	194
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	195 def _pageMatchesAny(self, fil, page):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	196 # Multiple taxonomy, i.e. it supports multiple terms, like tags.
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	197 page_values = page.config.get(self.name)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	198 if page_values is None or not isinstance(page_values, list):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	199 return False
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	200
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	201 page_set = set(map(self._slugifier.slugify, page_values))
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	202 if self._is_combination:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	203 # Multiple taxonomy, and multiple terms to match. Check that
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	204 # the ones to match are all in the page's terms.
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	205 value_set = set(self.value)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	206 return value_set.issubset(page_set)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	207 else:
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	208 # Multiple taxonomy, one term to match.
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	209 return self.value in page_set
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	210
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	211 def _pageMatchesSingle(self, fil, page):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	212 # Single taxonomy. Just compare the values.
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	213 page_value = page.config.get(self.name)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	214 if page_value is None:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	215 return False
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	216 page_value = self._slugifier.slugify(page_value)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	217 return page_value == self.value
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	218
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	219
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	220 def _get_taxonomy(app, tax_name):
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	221 tax_config = app.config.get('site/taxonomies/' + tax_name)
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	222 if tax_config is None:
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	223 raise ConfigurationError("No such taxonomy: %s" % tax_name)
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	224 return Taxonomy(tax_name, tax_config)
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	225
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	226
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	227 def _get_slugifier(app, taxonomy, slugify_mode=None):
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	228 if slugify_mode is None:
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	229 slugify_mode = app.config.get('site/slugify_mode', 'encode')
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	230 sm = _parse_slugify_mode(slugify_mode)
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	231 return _Slugifier(taxonomy, sm)
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	232
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	233
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	234 class TaxonomyPipelineRecordEntry(PagePipelineRecordEntry):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	235 def __init__(self):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	236 super().__init__()
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	237 self.term = None
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	238
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	239
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	240 class TaxonomyPipeline(ContentPipeline):
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	241 PIPELINE_NAME = 'taxonomy'
877 d6d35b2efd04 bake: Rename "pass" to "step" and make the page pipeline use different steps. Ludovic Chabant <ludovic@chabant.com> parents: 876 diff changeset	242 PASS_NUM = 10
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	243 RECORD_ENTRY_CLASS = TaxonomyPipelineRecordEntry
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	244
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	245 def __init__(self, source, ctx):
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	246 if not isinstance(source, TaxonomySource):
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	247 raise Exception("The taxonomy pipeline only supports taxonomy "
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	248 "content sources.")
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	249
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	250 super().__init__(source, ctx)
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	251 self.inner_source = source.inner_source
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	252 self.taxonomy = source.taxonomy
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	253 self.slugifier = source.slugifier
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	254 self._tpl_name = source.config['template']
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	255 self._analyzer = None
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	256 self._pagebaker = None
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	257
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	258 def initialize(self):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	259 self._pagebaker = PageBaker(self.app,
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	260 self.ctx.out_dir,
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	261 force=self.ctx.force)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	262 self._pagebaker.startWriterQueue()
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	263
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	264 def shutdown(self):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	265 self._pagebaker.stopWriterQueue()
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	266
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	267 def createJobs(self, ctx):
876 d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	268 logger.debug("Caching template page for taxonomy '%s'." %
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	269 self.taxonomy.name)
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	270 page = self.app.getPage(self.source, ContentItem('_index', {}))
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	271 page._load()
d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	272
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	273 logger.debug("Building '%s' taxonomy pages for source: %s" %
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	274 (self.taxonomy.name, self.inner_source.name))
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	275 self._analyzer = _TaxonomyTermsAnalyzer(self, ctx.record_histories)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	276 self._analyzer.analyze()
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	277
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	278 logger.debug("Queuing %d '%s' jobs." %
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	279 (len(self._analyzer.dirty_slugified_terms),
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	280 self.taxonomy.name))
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	281 jobs = []
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	282 for slugified_term in self._analyzer.dirty_slugified_terms:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	283 item = ContentItem(
876 d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	284 '_index',
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	285 {'term': slugified_term,
876 d1095774bfcf refactor: Fix some issues with record/cache entry collisions, add counters. Ludovic Chabant <ludovic@chabant.com> parents: 871 diff changeset	286 'record_entry_spec': '_index[%s]' % slugified_term,
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	287 'route_params': {
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	288 self.taxonomy.term_name: slugified_term}
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	289 })
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	290 jobs.append(self.createJob(item))
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	291 if len(jobs) > 0:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	292 return jobs
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	293 return None
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	294
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	295 def run(self, job, ctx, result):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	296 content_item = job.content_item
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	297 logger.debug("Rendering '%s' page: %s" %
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	298 (self.taxonomy.name, content_item.metadata['term']))
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	299
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	300 page = Page(self.source, job.content_item)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	301 prev_entry = ctx.previous_entry
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	302 cur_entry = result.record_entry
856 9bb22bbe093c refactor: Make the blog archives functional again. Ludovic Chabant <ludovic@chabant.com> parents: 855 diff changeset	303 cur_entry.term = content_item.metadata['term']
871 504ddb370df8 refactor: Fixing some issues with baking assets. Ludovic Chabant <ludovic@chabant.com> parents: 856 diff changeset	304 self._pagebaker.bake(page, prev_entry, cur_entry)
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	305
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	306 def postJobRun(self, ctx):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	307 # We create bake entries for all the terms that were not dirty.
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	308 # This is because otherwise, on the next incremental bake, we wouldn't
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	309 # find any entry for those things, and figure that we need to delete
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	310 # their outputs.
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	311 analyzer = self._analyzer
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	312 for prev, cur in ctx.record_history.diffs:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	313 # Only consider entries that don't have any current version
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	314 # (i.e. they weren't baked just now).
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	315 if prev and not cur:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	316 t = prev.term
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	317 if analyzer.isKnownSlugifiedTerm(t):
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	318 logger.debug("Creating unbaked entry for '%s' term: %s" %
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	319 (self.taxonomy.name, t))
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	320 cur.term = t
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	321 cur.out_paths = list(prev.out_paths)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	322 cur.errors = list(prev.errors)
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	323 else:
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	324 logger.debug("Term '%s' in '%s' isn't used anymore." %
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	325 (t, self.taxonomy.name))
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	326
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	327
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	328 class _TaxonomyTermsAnalyzer(object):
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	329 def __init__(self, pipeline, record_histories):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	330 self.pipeline = pipeline
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	331 self.record_histories = record_histories
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	332 self._all_terms = {}
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	333 self._single_dirty_slugified_terms = set()
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	334 self._all_dirty_slugified_terms = None
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	335
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	336 @property
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	337 def dirty_slugified_terms(self):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	338 """ Returns the slugified terms that have been 'dirtied' during
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	339 this bake.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	340 """
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	341 return self._all_dirty_slugified_terms
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	342
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	343 def isKnownSlugifiedTerm(self, term):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	344 """ Returns whether the given slugified term has been seen during
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	345 this bake.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	346 """
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	347 return term in self._all_terms
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	348
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	349 def analyze(self):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	350 # Build the list of terms for our taxonomy, and figure out which ones
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	351 # are 'dirty' for the current bake.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	352 #
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	353 # Remember all terms used.
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	354 source = self.pipeline.inner_source
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	355 taxonomy = self.pipeline.taxonomy
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	356 slugifier = self.pipeline.slugifier
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	357
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	358 record_name = get_record_name_for_source(source)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	359 current_records = self.record_histories.current
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	360 cur_rec = current_records.getRecord(record_name)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	361 for cur_entry in cur_rec.getEntries():
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	362 if not cur_entry.was_overriden:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	363 cur_terms = cur_entry.config.get(taxonomy.setting_name)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	364 if cur_terms:
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	365 if not taxonomy.is_multiple:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	366 self._addTerm(
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	367 slugifier, cur_entry.item_spec, cur_terms)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	368 else:
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	369 self._addTerms(
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	370 slugifier, cur_entry.item_spec, cur_terms)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	371
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	372 # Re-bake all taxonomy terms that include new or changed pages, by
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	373 # marking them as 'dirty'.
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	374 previous_records = self.record_histories.previous
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	375 prev_rec = previous_records.getRecord(record_name)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	376 history = RecordHistory(prev_rec, cur_rec)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	377 history.build()
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	378 for prev_entry, cur_entry in history.diffs:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	379 entries = [cur_entry]
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	380 if prev_entry:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	381 entries.append(prev_entry)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	382
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	383 for e in entries:
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	384 entry_terms = e.config.get(taxonomy.setting_name)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	385 if entry_terms:
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	386 if not taxonomy.is_multiple:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	387 self._single_dirty_slugified_terms.add(
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	388 slugifier.slugify(entry_terms))
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	389 else:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	390 self._single_dirty_slugified_terms.update(
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	391 (slugifier.slugify(t)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	392 for t in entry_terms))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	393
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	394 self._all_dirty_slugified_terms = list(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	395 self._single_dirty_slugified_terms)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	396 logger.debug("Gathered %d dirty taxonomy terms",
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	397 len(self._all_dirty_slugified_terms))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	398
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	399 # Re-bake the combination pages for terms that are 'dirty'.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	400 # We make all terms into tuple, even those that are not actual
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	401 # combinations, so that we have less things to test further down the
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	402 # line.
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	403 #
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	404 # Add the combinations to that list. We get those combinations from
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	405 # wherever combinations were used, so they're coming from the
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	406 # `onRouteFunctionUsed` method.
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	407 if taxonomy.is_multiple:
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	408 known_combinations = set()
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	409 for cur_entry in cur_rec.getEntries():
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	410 used_terms = _get_all_entry_taxonomy_terms(cur_entry)
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	411 for terms in used_terms:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	412 if len(terms) > 1:
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	413 known_combinations.add(terms)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	414
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	415 dcc = 0
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	416 for terms in known_combinations:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	417 if not self._single_dirty_slugified_terms.isdisjoint(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	418 set(terms)):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	419 self._all_dirty_slugified_terms.append(
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	420 taxonomy.separator.join(terms))
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	421 dcc += 1
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	422 logger.debug("Gathered %d term combinations, with %d dirty." %
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	423 (len(known_combinations), dcc))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	424
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	425 def _addTerms(self, slugifier, item_spec, terms):
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	426 for t in terms:
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	427 self._addTerm(slugifier, item_spec, t)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	428
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	429 def _addTerm(self, slugifier, item_spec, term):
448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	430 st = slugifier.slugify(term)
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	431 orig_terms = self._all_terms.setdefault(st, [])
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	432 if orig_terms and orig_terms[0] != term:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	433 logger.warning(
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	434 "Term '%s' in '%s' is slugified to '%s' which conflicts with "
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	435 "previously existing '%s'. The two will be merged." %
855 448710d84121 refactor: Get the taxonomy support back to a functional state. Ludovic Chabant <ludovic@chabant.com> parents: 854 diff changeset	436 (term, item_spec, st, orig_terms[0]))
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	437 orig_terms.append(term)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	438
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	439
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	440 def _get_all_entry_taxonomy_terms(entry):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	441 res = set()
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	442 for o in entry.subs:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	443 for pinfo in o.render_info:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	444 if pinfo:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	445 terms = pinfo.getCustomInfo('used_taxonomy_terms')
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	446 if terms:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	447 res \|= set(terms)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	448 return res
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	449
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	450
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	451 class _Slugifier(object):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	452 def __init__(self, taxonomy, mode):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	453 self.taxonomy = taxonomy
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	454 self.mode = mode
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	455
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	456 def slugifyMultiple(self, terms):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	457 return tuple(map(self.slugify, terms))
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	458
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	459 def slugify(self, term):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	460 if self.mode & SLUGIFY_TRANSLITERATE:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	461 term = unidecode.unidecode(term)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	462 if self.mode & SLUGIFY_LOWERCASE:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	463 term = term.lower()
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	464 if self.mode & SLUGIFY_DOT_TO_DASH:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	465 term = re_first_dot_to_dash.sub('', term)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	466 term = re_dot_to_dash.sub('-', term)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	467 if self.mode & SLUGIFY_SPACE_TO_DASH:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	468 term = re_space_to_dash.sub('-', term)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	469 return term
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	470
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	471
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	472 def _parse_slugify_mode(value):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	473 mapping = {
854 08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	474 'encode': SLUGIFY_ENCODE,
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	475 'transliterate': SLUGIFY_TRANSLITERATE,
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	476 'lowercase': SLUGIFY_LOWERCASE,
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	477 'dot_to_dash': SLUGIFY_DOT_TO_DASH,
08e02c2a2a1a core: Keep refactoring, this time to prepare for generator sources. Ludovic Chabant <ludovic@chabant.com> parents: 852 diff changeset	478 'space_to_dash': SLUGIFY_SPACE_TO_DASH}
852 4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	479 mode = 0
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	480 for v in value.split(','):
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	481 f = mapping.get(v.strip())
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	482 if f is None:
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	483 if v == 'iconv':
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	484 raise Exception("'iconv' is not supported as a slugify mode "
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	485 "in PieCrust2. Use 'transliterate'.")
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	486 raise Exception("Unknown slugify flag: %s" % v)
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	487 mode \|= f
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	488 return mode
4850f8c21b6e core: Start of the big refactor for PieCrust 3.0. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	489

Mercurial > piecrust2

annotate piecrust/sources/taxonomy.py @ 877:d6d35b2efd04