Official implementation for NeurIPS 2024 paper "On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capability". - View it on GitHub