Robots.txt Analyzer

Inspect crawler rules, security exposures, and sitemap declarations

Input
🔒 Fetches robots.txt via AllOrigins proxy. The target domain will see a request from the AllOrigins server.
Overview
Risk Score
User-Agent Groups
No user-agent groups found.
Sitemaps
No sitemaps declared.
All Disallowed Paths
No disallowed paths found.
▶ Raw robots.txt