Mercurial > piecrust2
view docs/assets/robots.txt @ 1156:4ea64255eadf
sources: Add ability to force a realm on a source.
author | Ludovic Chabant <ludovic@chabant.com> |
---|---|
date | Tue, 11 Jun 2019 15:12:47 -0700 |
parents | 26e59f837558 |
children |
line wrap: on
line source
# www.robotstxt.org/ # Allow crawling of all content User-agent: * Disallow: