Tag: memory
All the articles with the tag "memory".
How much do language models memorize?
Published:Explores the concept of memorization in large language models (LLMs), introducing a novel method to quantify it and distinguish it from generalization. The authors define model capacity and investigate its relationship with dataset size, training dynamics, and membership inference.