view docs/assets/robots.txt @ 262:61145dcd56e0

routing: Better generate URLs according to the site configuration. This fixes problems now that even the chef server generates URLs that are appropriate given the `pretty_urls` and `trailing_slash` settings. Add some tests.
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 24 Feb 2015 08:06:25 -0800
parents 26e59f837558
children
line wrap: on
line source

# www.robotstxt.org/

# Allow crawling of all content
User-agent: *
Disallow: