Gecko crawler supports distributed by redis - View it on GitHub