A pure-Python robots.txt parser with support for modern conventions. - View it on GitHub