https://ayushtambde.com/blog/matrix-orthogonalization-improves-memory-in-recurrent-models/
Matrix Orthogonalization Improves Memory in Recurrent Models
Ayush Tambde
06-30-2026
"But for some domains, we can't afford the quadratic-attention overhead of transformers.... we need to make recurrent neural networks work, but don't want to give up on associative recall."