WWW::RobotRules - database of robots.txt-derived permissions - View it on GitHub