A collection of awesome web crawler,spider in different language - View it on GitHub