Mercurial > piecrust2

annotate docs/assets/robots.txt @ 912:cd236a6af9f6

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
data: Add access to route metadata in the templating data. It's available under `route`. Also, rename `slug` to `rel_url`, because it's confusing when `slug` could be the route metadata of the same name as provided by the default page source. The old `slug` is still there for backwards compatibility though.
author Ludovic Chabant <ludovic@chabant.com>
date Fri, 28 Apr 2017 10:05:14 -0700
parents 26e59f837558
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
243
26e59f837558 docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
1 # www.robotstxt.org/
26e59f837558 docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
2
26e59f837558 docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
3 # Allow crawling of all content
26e59f837558 docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
4 User-agent: *
26e59f837558 docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
5 Disallow:
26e59f837558 docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff changeset
6