Show HN: Every number in a tiny GPT, from one forward pass to one step of RLHF
Hacker News · 7 Oct 2026, 21:44 AEST
A new interactive tool provides a detailed walkthrough of a toy GPT-2 model's token generation process. It allows users to see every computation live in the browser, enhancing understanding of the model's functioning.
Key facts
- It's a step by step walkthrough of a toy GPT-2 like model.
- Every matrix fits on screen, every number is computed live in the browser, and hovering over any cell shows the exact math of that step.
- It covers inference -- following the generation of one token from tokenization to sampling, one training step of pretraining, and one PPO iteration of RLHF for post-training.
Read aloud
A new interactive tool provides a detailed walkthrough of a toy GPT-two model's token generation process. It allows users to see every computation live in the browser, enhancing understanding of the model's functioning. It's a step by step walkthrough of a toy GPT-two like model. Every matrix fits on screen, every number is computed live in the browser, and hovering over any cell shows the exact math of that step. It covers inference -- following the generation of one token from tokenization to sampling, one training step of pretraining, and one PPO iteration of RLHF for post-training.
Sources in this cluster
- Show HN: Every number in a tiny GPT, from one forward pass to one step of RLHF
Hacker News · 7 Oct 2026, 21:44 AEST