User-Agent: * Disallow: /api Disallow: /assessment Crawl-delay: 10 User-Agent: bingbot Disallow: /api Disallow: /assessment Crawl-delay: 20 User-Agent: Slurp User-Agent: Yeti User-Agent: Baiduspider User-Agent: Linespider User-Agent: ia_archiver User-Agent: archive.org_bot User-Agent: special_archiver Disallow: /api Disallow: /assessment Crawl-delay: 30 User-Agent: Amazonbot User-Agent: GPTBot User-Agent: TTD-Content User-Agent: proximic User-Agent: ias_crawler User-Agent: ias- User-Agent: admantx-ussy User-Agent: meta-externalagent Disallow: /