git.gnu.io has moved to IP address 209.51.188.249 -- please double check where you are logging in.

Commit a3c5ef59 authored by mmn's avatar mmn

Fix merge #101 by replacing a non-working Yahoo! link with Wikipedia

The link was meant to describe robots.txt crawl-delay info
parent 52274838
......@@ -674,7 +674,7 @@ Web crawlers. See http://www.robotstxt.org/ for more information
on the format of this file.
crawldelay: if non-empty, this value is provided as the Crawl-Delay:
for the robots.txt file. see https://help.yahoo.com/kb/search
for the robots.txt file. see <https://en.wikipedia.org/wiki/Robots_exclusion_standard#Crawl-delay_directive>
for more information. Default is zero, no explicit delay.
disallow: Array of (virtual) directories to disallow. Default is 'main',
'search', 'message', 'settings', 'admin'. Ignored when site
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment