Continuous scalable web crawler built on top of Flink and crawler-commons - View it on GitHub