Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188) - View it on GitHub