view docs/assets/robots.txt @ 973:8419daaa7a0e

internal: Make the page serializer thread daemon. This avoids problems where the main process hangs if there's a critical failure.
author Ludovic Chabant <ludovic@chabant.com>
date Tue, 17 Oct 2017 01:04:10 -0700
parents 26e59f837558
children
line wrap: on
line source

# www.robotstxt.org/

# Allow crawling of all content
User-agent: *
Disallow: