view docs/assets/robots.txt @ 876:d1095774bfcf

refactor: Fix some issues with record/cache entry collisions, add counters.
author Ludovic Chabant <ludovic@chabant.com>
date Thu, 15 Jun 2017 07:33:40 -0700
parents 26e59f837558
children
line wrap: on
line source

# www.robotstxt.org/

# Allow crawling of all content
User-agent: *
Disallow: