What I Read: History, Large Language Models

Posted on 2026-01-14 :: Tags: machine learning, natural language processing, large language model, neural network, transformer, attention, sequence, reinforcement learning, embedding, probability, loss, vector, matrix, linear algebra, autoregression, Markov, computer vision, bag of words, skip-gram, recurrent, convolution, encoder, decoder, long short-term memory, fine tuning, alignment, scalability

https://gregorygundersen.com/blog/2025/10/01/large-language-models/
A History of Large Language Models
Gregory Gundersen
01 October 2025
"I trace an academic history of some of the core ideas behind large language models, such as distributed representations, transducers, attention, the transformer, and generative pre-training."