# robots.txt # Yahoo! Slurp crawlers #http://help.yahoo.com/l/us/yahoo/search/webcrawler/slurp-03.html User-agent: Slurp Crawl-delay: 10 User-agent: Yandex Crawl-delay: 30