robots.txt Parser / Validator
Inspect /robotsParse pasted robots.txt content locally and flag malformed or unknown directives.
Parsing updates automatically as the content changes.
- 1 directive user-agent: *
- 2 directive disallow: /admin
- 3 directive sitemap: https://recica.dev/sitemap.xml
- 4 error Bad line Missing ":" separator.
3 directives · 1 errors 4 lines 79 B local only
How it works
Parse pasted robots.txt content and surface directives, comments, and malformed lines.
Use it to inspect a robots.txt draft for recognizable directives and malformed lines.
Example
Parse a User-agent group with Disallow and Sitemap directives before deployment.
Notes
- This tool validates pasted content only. It does not fetch remote URLs.
- Unknown directives and malformed rows are marked in the gutter and listed on the right.
- Useful for quick inspection before publishing robots.txt changes.
Constraints
- Checks syntax and grouping but does not simulate every crawler implementation.
- Limit: 1 MiB input. Nothing is uploaded or silently truncated.
Common errors
- Missing colon
- Unknown directive
- Assuming syntax guarantees crawler behavior
When to use it
Use it to inspect a robots.txt draft for recognizable directives and malformed lines.
Formats: RFC 9309 directives, Comments, Sitemap and common extension directives.