piecrust2: piecrust/importing/jekyll.py annotate

annotate piecrust/importing/jekyll.py @ 857:d231a10d18f9

refactor: Make the data providers and blog archives source functional. Also, because of a behaviour change in Jinja, the blog archives sources is now offering monthly archives by itself.

author	Ludovic Chabant <ludovic@chabant.com>
date	Thu, 08 Jun 2017 08:49:33 -0700
parents	28b021be74eb
children	727110ea112a

rev	line source
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import os
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import os.path
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import re
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import shutil
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import yaml
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	6 import logging
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	7 from piecrust.configuration import parse_config_header
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	8 from piecrust.importing.base import FileWalkingImporter
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	9 from piecrust.uriutil import multi_replace
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12 logger = logging.getLogger(__name__)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	15 class JekyllImporter(FileWalkingImporter):
297 2823ea40cfac import: Put importer metadata on the class, and allow return values. Ludovic Chabant <ludovic@chabant.com> parents: 62 diff changeset	16 name = 'jekyll'
2823ea40cfac import: Put importer metadata on the class, and allow return values. Ludovic Chabant <ludovic@chabant.com> parents: 62 diff changeset	17 description = "Imports content from a Jekyll or Octopress blog."
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19 def setupParser(self, parser, app):
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	20 super(JekyllImporter, self).setupParser(parser, app)
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 parser.add_argument('root_dir',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	22 help="The root directory of the Jekyll or Octopress website.")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 def importWebsite(self, app, args):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25 logger.debug("Importing Jekyll site from: %s" % args.root_dir)
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	26 self._startWalk(args.root_dir, args.exclude, app)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	27 logger.info("The Jekyll website was successfully imported.")
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	28
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	29 def _importFile(self, full_fn, rel_fn, app):
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	30 logger.debug("- %s" % rel_fn)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	31 if rel_fn == '_config.yml':
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	32 self.convertConfig(app, full_fn)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	33 elif rel_fn.startswith('_layouts'):
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	34 self.convertLayout(app, full_fn, rel_fn[len('_layouts/'):])
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	35 elif rel_fn.startswith('_includes'):
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	36 self.convertInclude(app, full_fn, rel_fn[len('_includes/'):])
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	37 elif rel_fn.startswith('_posts'):
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	38 self.convertPost(app, full_fn, rel_fn[len('_posts/'):])
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	39 else:
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	40 with open(full_fn, 'rb') as fp:
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	41 firstline = fp.read(3)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	42 if firstline == '---':
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	43 self.convertPage(app, full_fn, rel_fn)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	44 else:
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	45 self.convertStatic(app, full_fn, rel_fn)
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	46
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	47 def convertConfig(self, app, src_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48 logger.debug(" Converting configuration file.")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	49 with open(src_path, 'r', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	50 config = yaml.load(fp)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	52 if 'site' not in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	53 config['site'] = {}
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	54 config['site']['related_posts'] = []
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	55 config['site']['posts_fs'] = 'flat'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	56 config['site']['templates_dirs'] = ['includes', 'layouts']
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	57 config['site']['tag_url'] = 'tags/%tag%'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	58 if 'permalink' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	59 permalink = config['permalink']
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60 if permalink == 'date':
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 permalink = '/:categories/:year/:month/:day/:title.html'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	62 elif permalink == 'pretty':
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	63 permalink = '/:categories/:year/:month/:day/:title/'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	64 elif permalink == 'none':
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	65 permalink = '/:categories/:title.html'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67 # TODO: handle `:categories` token.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68 post_url = multi_replace(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	69 permalink,
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	70 {':year': '%year%', ':month': '%month%', ':day': '%day%',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71 ':title': '%slug%', ':categories': ''})
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 post_url = post_url.replace('//', '/').strip('/')
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73 config['site']['post_url'] = post_url
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	74 if 'exclude' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	75 if 'baker' not in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	76 config['baker'] = {}
355 28b021be74eb import: Use the proper baker setting in the Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: 297 diff changeset	77 config['baker']['ignore'] = list(map(
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 lambda i: '^/_%s/' % re.escape(i)))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 if 'jinja' not in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 config['jinja'] = {}
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 config['jinja']['auto_escape'] = False
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 if 'markdown' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 if not isinstance(config['markdown'], dict):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84 logger.warning("Discarding markdown setting: %s" %
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	85 config['markdown'])
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86 del config['markdown']
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 with open(os.path.join(app.root_dir, 'config.yml'), 'w') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89 yaml.dump(config, stream=fp)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	90
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	91 def convertPage(self, app, path, rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	92 logger.debug(" Converting page: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	93 is_index = False
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	94 is_static = False
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	95 _, ext = os.path.splitext(rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	96 if re.search(r'^index\.(html?\|textile\|markdown)$', rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	97 out_path = os.path.join(app.root_dir, 'pages', '_index' + ext)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	98 is_index = True
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	99 else:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	100 out_path = os.path.join(app.root_dir, 'pages', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	101
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	102 if ext not in ['htm', 'html', 'textile', 'markdown']:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	103 # There could be static files (SCSS or Less files) that look like
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	104 # pages because they have a YAML front matter.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105 is_static = True
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106 out_path = os.path.join(app.root_dir, 'assets', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	107
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	108 if is_static:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	109 logger.debug(" Actually a static file... forwarding converstion.")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	110 self.convertStatic(app, path, rel_path, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	111 return
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	112
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	113 self._doConvertPage(app, path, out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	114 if is_index:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	115 shutil.copy2(out_path, os.path.join(app.root_dir, 'pages', '_tag.%s' % ext))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	116
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	117 def convertPost(self, app, path, rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	118 logger.debug(" Converting post: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	119 out_path = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120 r'(\d{4}\-\d{2}\-\d{2})\-(.*)$',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121 r'\1_\2',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122 rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	123 out_path = os.path.join(app.root_dir, 'posts', out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	124 self._doConvertPage(app, path, out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	125
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	126 def convertLayout(self, app, path, rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	127 logger.debug(" Converting layout: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	128 out_path = os.path.join(app.root_dir, 'layouts', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	129 self._doConvertPage(app, path, out_path, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	130
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	131 def convertInclude(self, app, path, rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	132 logger.debug(" Converting include: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	133 out_path = os.path.join(app.root_dir, 'includes', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	134 self._doConvertPage(app, path, out_path, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	135
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	136 def convertStatic(self, app, path, rel_path, strip_header=False):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	137 logger.debug(" Converting static: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	138 out_path = os.path.join(app.root_dir, 'assets', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	139 logger.debug(" %s -> %s" % (path, out_path))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	140 os.makedirs(os.path.dirname(out_path), 0o755, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	141
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	142 if strip_header:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	143 with open(path, 'r', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	144 content = fp.write()
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	145 config, offset = parse_config_header(content)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	146 content = content[offset:]
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	147 with open(out_path, 'w', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	148 fp.write(content)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	149 return
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	150
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	151 shutil.copy2(path, out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	152
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	153 def _doConvertPage(self, app, path, out_path, is_template=False):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	154 logger.debug(" %s -> %s" % (path, out_path))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	155 os.makedirs(os.path.dirname(out_path), 0o755, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	156
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	157 with open(path, 'r', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	158 contents = fp.read()
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	160 config, offset = parse_config_header(contents)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	161 text = contents[offset:]
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	162 text_before = text
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	163
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	164 wrap_content_tag = True
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	165
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	166 if is_template:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	167 if 'layout' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	168 # Liquid doesn't support template inheritance but
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	169 # Jinja does.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	170 text = ("{%% extends '%s.html' %%}\n\n"
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	171 "{%% block jekyllcontent %%}\n"
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	172 "%s\n"
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	173 "{%% endblock %%}\n" % (config['layout'], text))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	174 wrap_content_tag = False
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	175 else:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	176 if 'layout' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	177 if config['layout'] == 'nil':
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	178 config['layout'] = 'none'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	179
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	180 # Convert the template stuff we can:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	181 # - content tag may have to be wrapped in a `jekyllcontent`
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	182 # because Jekyll uses implicit layout inheritance
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	183 # placements.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	184 if wrap_content_tag:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	185 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	186 r'{{\scontent\s}}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	187 r'{% block jekyllcontent %}{{ content }}{% endblock %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	188 text)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	189 # - list of posts
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	190 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	191 '(?<=\{%\|\{\{)([^\}]*)site.posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	192 '\\1blog.posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	193 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	194 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	195 '(?<=\{%\|\{\{)([^\}]*)paginator.posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	196 '\\1pagination.posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	197 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	198 # - list of categories or tags
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	199 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	200 '(?<=\{%\|\{\{)([^\}]*)site.categories',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	201 '\\1blog.categories',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	202 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	203 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	204 '(?<=\{%\|\{\{)([^\}]*)site.tags',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	205 '\\1blog.tags',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	206 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	207 # - list of related posts
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	208 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	209 '(?<=\{%\|\{\{)(?<!%\})site.related_posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	210 '\\1pagination.related_posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	211 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	212 # - enumeration limits
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	213 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	214 '{%\sfor\s+([^}]+)\s+limit\:\s(\d+)',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	215 '{% for \\1[:\\2]',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	216 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	217 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	218 '{%\sfor\s+([^}]+)\s+offset\:\s(\d+)',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	219 '{% for \\1[\\2:]',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	220 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	221 # - code highlighting
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	222 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	223 '{%\shighlight\s+([\w\d]+)\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	224 '{% geshi \'\\1\' %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	225 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	226 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	227 '{%\sendhighlight\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	228 '{% endgeshi %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	229 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	230 # - unless tag
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	231 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	232 '{%\sunless\s+([^}]+)\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	233 '{% if not \\1 %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	234 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	235 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	236 '{%\sendunless\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	237 '{% endif %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	238 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	239 # - variable assignment
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	240 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	241 '\{%\s*assign\s+',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	242 '{% set ',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	243 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	244 # - include tag
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	245 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	246 '\{%\sinclude\s+([\w\d\.\-_]+)\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	247 '{% include "\\1" %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	248 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	249 # - truncate filter
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	250 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	251 '\\|\struncate\:\s(\d+)',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	252 '\|truncate(\\1)',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	253 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	254 # - date filter
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	255 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	256 '\\|\sdate\:\s"([^"]+)"',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	257 '\|date("\\1")',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	258 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	259 # - some filters we don't need
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	260 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	261 '\\|\s*date_to_string',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	262 '',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	263 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	264
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	265 if text != text_before:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	266 # We changed the text, so create a backup.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	267 shutil.copy2(path, '%s.orig' % out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	268
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	269 with open(out_path, 'w', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	270 if not is_template:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	271 fp.write("---\n")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	272 fp.write(yaml.dump(config))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	273 fp.write("---\n")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	274 fp.write(text)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	275

Mercurial > piecrust2

annotate piecrust/importing/jekyll.py @ 857:d231a10d18f9