https://lucalp.dev/bitter-lesson-tokenization-and-blt/
The Bitter Lesson is coming for Tokenization
lucalp
2025/06/24
“…one of the documented bottlenecks in the text transformer world that has received less optimisation effort is the very mechanism that shapes its world view - tokenization.”