A collection of awesome web crawler,spider in different languages - View it on GitHub