Distributed web-crawler frontier and rescheduler in Go: a polite, freshness-aware crawl schedule in compact single-file .meguri partitions - View it on GitHub