https://alexshtf.github.io/2025/08/19/Orthogonality.html
Paying attention to feature distribution alignment
Alex Shtoff
Aug 19, 2025
“…its orthogonality is, in some sense informativeness. This is because it orthogonal bases produce features, and hence each basis function, in some sense, carries information that the other basis functions do not.”