What exactly does word2vec learn?
Lab/vendor primary source
#217
- Topic
- Practitioner Notes
- First seen
- 2026-07-16 19:08:00
- Last seen
- 2026-07-16 19:08:00
Source raw items (1)
- Blog / Newsletter2026-07-16 19:07:20What exactly does word2vec learn?
<!-- twitter --> <p>What exactly does <code class="language-plaintext highlighter-rouge">word2vec</code> learn, and how? Answering this question amounts to understanding representation learning in a minimal yet interesting language modeling task. Despite the fact that <code class="language-plaintext highlighter-rouge">word2vec</code> is a well-known precursor to modern language models, for many years, researchers lacked a quantitative and predictive theory describing its learning process. In our new <a href="https://arxiv.org/abs/2502.09863">paper</a>, we finally provide such a theory. We prove that there are realistic, practical regimes in which the learning problem reduces to <em>unweighted least-squares matrix factorization</em>. We solve the gradient flow dynamics in closed form; the final learned representations are simply given by PCA.</p> <div style="width: 100%; margin: 0 auto; text-align: center;"> <p style="text-align: center;"> <img src="https://bair.berkeley.ed