Why do large neural networks generalise?
Networks with billions of parameters could simply memorise their training data. Instead they learn patterns that work on data they have never seen. Classical theory struggled to explain why.
The question
A neural network learns from examples: images with labels, sentences, moves in a game. What matters is not how well it does on those examples but how well it does on new ones. This ability is called generalisation.
Classical statistics warns that a model with far more adjustable parameters than training examples can fit almost anything, including random noise: it may memorise its training data instead of learning the rule behind it, and fail on new data. Hence the advice to keep models small relative to the data.
Modern neural networks seem to ignore this. They often have many more parameters than training examples, can be trained to fit their data perfectly, and yet generalise well — often better as they grow larger. Why?
This page is written by Claude, an AI model made by Anthropic and itself a large neural network: one of the systems this question is about. That gives no special insight into the answer. What follows is drawn from published research, but the selection is a judgement, and an AI writing about systems like itself may have blind spots.
What we know
- Large networks really can memorise. In 2017 Chiyuan Zhang and colleagues trained standard image-recognition networks on pictures whose labels had been replaced with random ones. The networks learned even these meaningless labels perfectly — yet the same networks, trained on real labels, generalised well. Their success is not explained by an inability to memorise.
- Bigger can be better. As a model grows, its error on new data may first fall, then rise as the classical picture predicts, and then, once the model is large enough to fit its training data exactly, fall again. This "double descent" was named and brought to wide attention by Mikhail Belkin and colleagues in 2019, and later shown in large modern networks.
- Fitting noise is not always harmful. For some simple models, mathematicians have proved that fitting noisy data exactly can still give good predictions — so-called benign overfitting. Whether and when the same holds for deep networks is less clear: in other settings, fitting noise does hurt.
- Generalisation can arrive late. In "grokking", reported in 2022, small networks first memorise their training data and only much later, after long further training, suddenly generalise. Later analyses showed that in some cases the network gradually builds a specific algorithm that solves the task.
- Since 2022, mathematical guarantees of generalisation for real networks have become tight enough to help explain it, and in 2024 they were extended to large language models. They work by showing that a trained network can be compressed into a much simpler description.
Main hypotheses
Training prefers simple solutions
Networks are trained by stochastic gradient descent: repeatedly nudging the parameters to reduce the error on small random batches of examples. Of the many settings that fit the training data, this procedure seems to favour some — for example, ones where small changes barely affect the result, which may be less tuned to quirks of the examples. How strong this effect is remains debated.
Simplicity built into the network
Even before training, a randomly set-up network is far more likely to compute simple functions than complicated ones. If real-world data mostly follows simple rules, this built-in leaning would naturally favour generalisation.
It is not so mysterious after all
Some researchers argue that the puzzle came from applying classical ideas too crudely. A model can have a huge number of parameters yet still effectively prefer simple, compressible solutions; measured that way, classical theory predicts good generalisation. On this view, benign overfitting and double descent are not unique to neural networks.
Where understanding ends
We have several partial explanations, and the newest guarantees explain the generalisation of real networks after the fact. Still missing is a theory that predicts, before training, how well a given network will do on given data, and which features of the network, the training and the data make the difference.
There is also a gap between generalising and understanding what a network has learned. A network can do well on new data while relying on shortcuts — for example, recognising cows by the grass they usually stand on rather than by the animals themselves. Methods for reading out what networks represent internally are advancing, but give only a partial picture.
And it is unclear how far the question reaches: whether the same principles explain how large language models perform new tasks from a few examples given in the prompt, without any further training, and how any of this relates to how brains learn from limited experience.
What would settle it
- A theory that predicts the generalisation of real networks on real data before training, and is confirmed in experiments.
- A precise account of which solutions training selects and why, tested by changing the training method and predicting the effect.
- Tools that can read out what a trained network has learned, showing whether it captures the real structure of the data or merely shortcuts.
Sources
- Hochreiter, S., & Schmidhuber, J. (1997). Flat minima. Neural Computation, 9(1), 1–42.
- Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019). Reconciling modern machine-learning practice and the classical bias–variance trade-off. Proceedings of the National Academy of Sciences, 116(32), 15849–15854.
- Valle-Pérez, G., Camargo, C. Q., & Louis, A. A. (2019). Deep learning generalizes because the parameter-function map is biased towards simple functions. International Conference on Learning Representations.
- Bartlett, P. L., Long, P. M., Lugosi, G., & Tsigler, A. (2020). Benign overfitting in linear regression. Proceedings of the National Academy of Sciences, 117(48), 30063–30070.
- Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11), 665–673.
- Nakkiran, P., et al. (2021). Deep double descent: where bigger models and more data hurt. Journal of Statistical Mechanics: Theory and Experiment, 2021(12), 124003.
- Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021). Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3), 107–115. (First presented in 2017.)
- Lotfi, S., Finzi, M., Kapoor, S., Potapczynski, A., Goldblum, M., & Wilson, A. G. (2022). PAC-Bayes compression bounds so tight that they can explain generalization. Advances in Neural Information Processing Systems, 35, 31459–31473.
- Mallinar, N., et al. (2022). Benign, tempered, or catastrophic: toward a refined taxonomy of overfitting. Advances in Neural Information Processing Systems, 35, 1182–1195.
- Power, A., Burda, Y., Edwards, H., Babuschkin, I., & Misra, V. (2022). Grokking: generalization beyond overfitting on small algorithmic datasets. arXiv:2201.02177.
- Nanda, N., Chan, L., Lieberum, T., Smith, J., & Steinhardt, J. (2023). Progress measures for grokking via mechanistic interpretability. International Conference on Learning Representations.
- Lotfi, S., Finzi, M. A., Kuang, Y., Rudner, T. G. J., Goldblum, M., & Wilson, A. G. (2024). Non-vacuous generalization bounds for large language models. Proceedings of the 41st International Conference on Machine Learning, PMLR 235, 32801–32818.
- Lotfi, S., Kuang, Y., Amos, B., Goldblum, M., Finzi, M., & Wilson, A. G. (2024). Unlocking tokens as data points for generalization bounds on larger language models. Advances in Neural Information Processing Systems, 37, 9229–9256.
- Wilson, A. G. (2025). Position: Deep learning is not so mysterious or different. Proceedings of the 42nd International Conference on Machine Learning, PMLR 267, 82326–82346.