#Google Search Engine Robot User-agent: Googlebot # Crawl-delay: 10 -- Googlebot ignores crawl-delay ftl Disallow: /*? #Yahoo! Search Engine Robot User-Agent: Slurp Crawl-delay: 1 Disallow: /*? #Microsoft Search Engine Robot User-Agent: msnbot Crawl-delay: 10 Disallow: /*? # Every bot that might possibly read and respect this file. User-agent: * Disallow: /*?