view docs/assets/robots.txt @ 583:1eda551ee681

cli: More proper argument parsing for the main/root arguments.
author Ludovic Chabant <ludovic@chabant.com>
date Wed, 30 Dec 2015 20:42:45 -0800
parents 26e59f837558
children
line wrap: on
line source

# www.robotstxt.org/

# Allow crawling of all content
User-agent: *
Disallow: