Andrew Fairless, Ph.D.
blue-green banner
What I Read: Evaluating language models.
Posted on :: Tags: large language model, natural language processing, evaluation, metric, rank, question answering, fine tuning, logistic, optimization, reinforcement learning, reward, preference optimization, generative, generalization, alignment

https://mlbenchmarks.org/11-evaluating-language-models.html
Evaluating language models
Moritz Hardt
"Applying tune-before-test, rankings enjoy greater agreement across different benchmarks.... tune-before-test also aligns perplexity rankings with downstream task benchmarks."