A scalable web crawler framework. - View it on GitHub