Pholcus is a distributed high-concurrency crawler software written in pure golang - View it on GitHub