Free tools / Robots.txt checker
See what your crawl rules say.
Fetch the public robots.txt file and inspect user-agent groups, rules and sitemap declarations.
← All free toolsFrom result to action
Want to know what to work on first?
Get the toolkit through a few short business questions, or bring this result into a business review. We recommend the next step after understanding the problem.