Code for the paper "Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension" - View it on GitHub