This repository is for studying how to efficiently fine-tune long-context LLMs (e.g., Llama 3.1 8B for a 128K context length) on commodity machines (e.g., 8x V100). - View it on GitHub