piecrust2: piecrust/importing/jekyll.py annotate

annotate piecrust/importing/jekyll.py @ 111:208c652551a3

Quick fix for making the server correctly update referenced pages. Disable the file-system cache for rendered segments when in server mode. We can bring this optimization back when we're actually using the baking record in the server too in order to know dependencies.

author	Ludovic Chabant <ludovic@chabant.com>
date	Thu, 16 Oct 2014 17:03:42 -0700
parents	52e4d9a1f917
children	2823ea40cfac

rev	line source
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	1 import os
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	2 import os.path
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	3 import re
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	4 import shutil
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	5 import yaml
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	6 import logging
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	7 from piecrust.configuration import parse_config_header
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	8 from piecrust.importing.base import FileWalkingImporter
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	9 from piecrust.uriutil import multi_replace
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	10
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	11
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	12 logger = logging.getLogger(__name__)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	13
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	14
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	15 class JekyllImporter(FileWalkingImporter):
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	16 def __init__(self):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	17 super(JekyllImporter, self).__init__()
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	18 self.name = 'jekyll'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	19 self.description = "Imports content from a Jekyll or Octopress blog."
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	20
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	21 def setupParser(self, parser, app):
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	22 super(JekyllImporter, self).setupParser(parser, app)
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	23 parser.add_argument('root_dir',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	24 help="The root directory of the Jekyll or Octopress website.")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	25
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	26 def importWebsite(self, app, args):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	27 logger.debug("Importing Jekyll site from: %s" % args.root_dir)
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	28 self._startWalk(args.root_dir, args.exclude, app)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	29 logger.info("The Jekyll website was successfully imported.")
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	30
62 52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	31 def _importFile(self, full_fn, rel_fn, app):
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	32 logger.debug("- %s" % rel_fn)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	33 if rel_fn == '_config.yml':
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	34 self.convertConfig(app, full_fn)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	35 elif rel_fn.startswith('_layouts'):
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	36 self.convertLayout(app, full_fn, rel_fn[len('_layouts/'):])
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	37 elif rel_fn.startswith('_includes'):
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	38 self.convertInclude(app, full_fn, rel_fn[len('_includes/'):])
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	39 elif rel_fn.startswith('_posts'):
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	40 self.convertPost(app, full_fn, rel_fn[len('_posts/'):])
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	41 else:
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	42 with open(full_fn, 'rb') as fp:
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	43 firstline = fp.read(3)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	44 if firstline == '---':
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	45 self.convertPage(app, full_fn, rel_fn)
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	46 else:
52e4d9a1f917 Simple importer for PieCrust 1 websites. Ludovic Chabant <ludovic@chabant.com> parents: 60 diff changeset	47 self.convertStatic(app, full_fn, rel_fn)
60 6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	48
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	49 def convertConfig(self, app, src_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	50 logger.debug(" Converting configuration file.")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	51 with open(src_path, 'r', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	52 config = yaml.load(fp)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	53
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	54 if 'site' not in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	55 config['site'] = {}
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	56 config['site']['related_posts'] = []
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	57 config['site']['posts_fs'] = 'flat'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	58 config['site']['templates_dirs'] = ['includes', 'layouts']
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	59 config['site']['tag_url'] = 'tags/%tag%'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	60 if 'permalink' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	61 permalink = config['permalink']
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	62 if permalink == 'date':
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	63 permalink = '/:categories/:year/:month/:day/:title.html'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	64 elif permalink == 'pretty':
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	65 permalink = '/:categories/:year/:month/:day/:title/'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	66 elif permalink == 'none':
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	67 permalink = '/:categories/:title.html'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	68
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	69 # TODO: handle `:categories` token.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	70 post_url = multi_replace(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	71 permalink,
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	72 {':year': '%year%', ':month': '%month%', ':day': '%day%',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	73 ':title': '%slug%', ':categories': ''})
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	74 post_url = post_url.replace('//', '/').strip('/')
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	75 config['site']['post_url'] = post_url
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	76 if 'exclude' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	77 if 'baker' not in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	78 config['baker'] = {}
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	79 config['baker']['skip_patterns'] = list(map(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	80 lambda i: '^/_%s/' % re.escape(i)))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	81 if 'jinja' not in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	82 config['jinja'] = {}
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	83 config['jinja']['auto_escape'] = False
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	84 if 'markdown' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	85 if not isinstance(config['markdown'], dict):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	86 logger.warning("Discarding markdown setting: %s" %
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	87 config['markdown'])
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	88 del config['markdown']
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	89
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	90 with open(os.path.join(app.root_dir, 'config.yml'), 'w') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	91 yaml.dump(config, stream=fp)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	92
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	93 def convertPage(self, app, path, rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	94 logger.debug(" Converting page: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	95 is_index = False
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	96 is_static = False
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	97 _, ext = os.path.splitext(rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	98 if re.search(r'^index\.(html?\|textile\|markdown)$', rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	99 out_path = os.path.join(app.root_dir, 'pages', '_index' + ext)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	100 is_index = True
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	101 else:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	102 out_path = os.path.join(app.root_dir, 'pages', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	103
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	104 if ext not in ['htm', 'html', 'textile', 'markdown']:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	105 # There could be static files (SCSS or Less files) that look like
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	106 # pages because they have a YAML front matter.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	107 is_static = True
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	108 out_path = os.path.join(app.root_dir, 'assets', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	109
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	110 if is_static:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	111 logger.debug(" Actually a static file... forwarding converstion.")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	112 self.convertStatic(app, path, rel_path, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	113 return
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	114
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	115 self._doConvertPage(app, path, out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	116 if is_index:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	117 shutil.copy2(out_path, os.path.join(app.root_dir, 'pages', '_tag.%s' % ext))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	118
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	119 def convertPost(self, app, path, rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	120 logger.debug(" Converting post: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	121 out_path = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	122 r'(\d{4}\-\d{2}\-\d{2})\-(.*)$',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	123 r'\1_\2',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	124 rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	125 out_path = os.path.join(app.root_dir, 'posts', out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	126 self._doConvertPage(app, path, out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	127
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	128 def convertLayout(self, app, path, rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	129 logger.debug(" Converting layout: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	130 out_path = os.path.join(app.root_dir, 'layouts', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	131 self._doConvertPage(app, path, out_path, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	132
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	133 def convertInclude(self, app, path, rel_path):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	134 logger.debug(" Converting include: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	135 out_path = os.path.join(app.root_dir, 'includes', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	136 self._doConvertPage(app, path, out_path, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	137
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	138 def convertStatic(self, app, path, rel_path, strip_header=False):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	139 logger.debug(" Converting static: %s" % rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	140 out_path = os.path.join(app.root_dir, 'assets', rel_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	141 logger.debug(" %s -> %s" % (path, out_path))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	142 os.makedirs(os.path.dirname(out_path), 0o755, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	143
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	144 if strip_header:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	145 with open(path, 'r', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	146 content = fp.write()
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	147 config, offset = parse_config_header(content)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	148 content = content[offset:]
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	149 with open(out_path, 'w', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	150 fp.write(content)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	151 return
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	152
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	153 shutil.copy2(path, out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	154
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	155 def _doConvertPage(self, app, path, out_path, is_template=False):
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	156 logger.debug(" %s -> %s" % (path, out_path))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	157 os.makedirs(os.path.dirname(out_path), 0o755, True)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	158
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	159 with open(path, 'r', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	160 contents = fp.read()
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	161
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	162 config, offset = parse_config_header(contents)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	163 text = contents[offset:]
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	164 text_before = text
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	165
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	166 wrap_content_tag = True
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	167
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	168 if is_template:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	169 if 'layout' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	170 # Liquid doesn't support template inheritance but
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	171 # Jinja does.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	172 text = ("{%% extends '%s.html' %%}\n\n"
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	173 "{%% block jekyllcontent %%}\n"
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	174 "%s\n"
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	175 "{%% endblock %%}\n" % (config['layout'], text))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	176 wrap_content_tag = False
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	177 else:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	178 if 'layout' in config:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	179 if config['layout'] == 'nil':
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	180 config['layout'] = 'none'
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	181
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	182 # Convert the template stuff we can:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	183 # - content tag may have to be wrapped in a `jekyllcontent`
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	184 # because Jekyll uses implicit layout inheritance
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	185 # placements.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	186 if wrap_content_tag:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	187 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	188 r'{{\scontent\s}}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	189 r'{% block jekyllcontent %}{{ content }}{% endblock %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	190 text)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	191 # - list of posts
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	192 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	193 '(?<=\{%\|\{\{)([^\}]*)site.posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	194 '\\1blog.posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	195 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	196 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	197 '(?<=\{%\|\{\{)([^\}]*)paginator.posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	198 '\\1pagination.posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	199 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	200 # - list of categories or tags
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	201 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	202 '(?<=\{%\|\{\{)([^\}]*)site.categories',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	203 '\\1blog.categories',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	204 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	205 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	206 '(?<=\{%\|\{\{)([^\}]*)site.tags',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	207 '\\1blog.tags',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	208 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	209 # - list of related posts
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	210 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	211 '(?<=\{%\|\{\{)(?<!%\})site.related_posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	212 '\\1pagination.related_posts',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	213 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	214 # - enumeration limits
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	215 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	216 '{%\sfor\s+([^}]+)\s+limit\:\s(\d+)',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	217 '{% for \\1[:\\2]',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	218 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	219 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	220 '{%\sfor\s+([^}]+)\s+offset\:\s(\d+)',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	221 '{% for \\1[\\2:]',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	222 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	223 # - code highlighting
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	224 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	225 '{%\shighlight\s+([\w\d]+)\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	226 '{% geshi \'\\1\' %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	227 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	228 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	229 '{%\sendhighlight\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	230 '{% endgeshi %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	231 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	232 # - unless tag
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	233 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	234 '{%\sunless\s+([^}]+)\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	235 '{% if not \\1 %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	236 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	237 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	238 '{%\sendunless\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	239 '{% endif %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	240 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	241 # - variable assignment
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	242 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	243 '\{%\s*assign\s+',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	244 '{% set ',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	245 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	246 # - include tag
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	247 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	248 '\{%\sinclude\s+([\w\d\.\-_]+)\s%}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	249 '{% include "\\1" %}',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	250 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	251 # - truncate filter
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	252 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	253 '\\|\struncate\:\s(\d+)',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	254 '\|truncate(\\1)',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	255 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	256 # - date filter
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	257 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	258 '\\|\sdate\:\s"([^"]+)"',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	259 '\|date("\\1")',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	260 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	261 # - some filters we don't need
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	262 text = re.sub(
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	263 '\\|\s*date_to_string',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	264 '',
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	265 text);
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	266
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	267 if text != text_before:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	268 # We changed the text, so create a backup.
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	269 shutil.copy2(path, '%s.orig' % out_path)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	270
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	271 with open(out_path, 'w', encoding='utf8') as fp:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	272 if not is_template:
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	273 fp.write("---\n")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	274 fp.write(yaml.dump(config))
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	275 fp.write("---\n")
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	276 fp.write(text)
6e60e0fef2be Add `import` command, Jekyll importer. Ludovic Chabant <ludovic@chabant.com> parents: diff changeset	277

Mercurial > piecrust2

annotate piecrust/importing/jekyll.py @ 111:208c652551a3