Quartz & AENEA Release QT_V.2 Tokenizer Family
Quartz and AENEA have jointly open-sourced the second-generation tokenizer family, featuring 64K, 96K, and 114K Code variants. Validated across 204 languages on the rigorous FLORES-200 benchmark, the QT_V.2 suite achieves the fewest total tokens in its class while demonstrating a 4× improvement in cross-lingual equity compared to industry standards.
Read Announcement