Mercurial > piecrust2
annotate docs/assets/robots.txt @ 363:dd25bd3ce1f9
serve: Refactoring and fixes to be able to serve taxonomy pages.
* Page sources' `findPagePath` is renamed to `findPageFactory`, so that it
also returns source metadata.
* Page refs now store possible hits more properly, and use the previous point
to also store metadata. As a result, they can also return a proper factory.
* Implement `findPageFactory` correctly in all built-in sources.
* When the Chef server matches a taxonomy page, get the source metadata from
the page ref in order to make a more proper page.
* Make the `getRoute(s)` functions explicitely say if they want taxonomy routes
or not.
author | Ludovic Chabant <ludovic@chabant.com> |
---|---|
date | Sun, 26 Apr 2015 15:07:40 -0700 |
parents | 26e59f837558 |
children |
rev | line source |
---|---|
243
26e59f837558
docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
1 # www.robotstxt.org/ |
26e59f837558
docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
2 |
26e59f837558
docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
3 # Allow crawling of all content |
26e59f837558
docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
4 User-agent: * |
26e59f837558
docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
5 Disallow: |
26e59f837558
docs: Add embryo of a documentation website.
Ludovic Chabant <ludovic@chabant.com>
parents:
diff
changeset
|
6 |