Free robots.txt checker and tester
Enter your site, and optionally a page path. We'll fetch robots.txt, check it for common mistakes and test whether Googlebot, Bingbot and other crawlers may fetch the page.
Enter your site, and optionally a page path. We'll fetch robots.txt, check it for common mistakes and test whether Googlebot, Bingbot and other crawlers may fetch the page.