What I Read: Delta Attention

Posted on 2026-10-27 :: Tags: attention, transformer, linear algebra, large language model, matrix, vector, loss, compression, recurrent, memory

https://blog.doubleword.ai/you-could-have-come-up-with-kimi-delta-attention
You Could Have Come Up With Kimi Delta Attention
Jamie Dborin
July 27, 2026
"...we are going to walk through the DeltaNet family of linear attention variants, two of which are used by the latest Qwen and Kimi model families..."