Fast BPE tokenizer with guaranteed perfect text reconstruction. 2x faster than tiktoken at encoding/decoding while preserving all whitespace and formatting. - View it on GitHub