Sometimes sites make crawling hard. Selenium-crawler uses selenium automation to fix that. - View it on GitHub