gorobots - robots.txt recon & path discovery in Go. Structured parsing, 29-category sensitivity classification, concurrent path probing, bulk domain scanning. - View it on GitHub