https://howtoscalenn.github.io/
How To Scale
Seunghyun Seo
May 2, 2025
“However, scaling law papers never tell us how to set the lr or bsz for a given computing budget, C. This is a non-trivial issue, because “more compute always leads to better performance” is not guaranteed…”