Interactive Lab · How models learn
It rolls downhill. The model sits on a loss curve (how wrong it is) and takes steps toward the bottom. One knob, the learning ratedecides the step size, and it makes all the difference between reaching the bottom, bouncing forever, or flying off the map. Drag it and watch.
🎬 Watch the 45-second explainer, then turn the knobs yourself below ↓
▶ The playground, set the step size, then run
Gradient descent is how every neural network on earth is trained, the loop you just steered is what runs for weeks across thousands of GPUs to make a GPT. The learning-rate dilemma you felt is a live production knob: LR schedules, warmup, and decay are among the most-tuned hyperparameters in any training run, and a wrong one wastes a million-dollar job.
The code that's actually running
Set the learning rate to 0.05 and Run, how many steps to reach the bottom? Now try 0.85, then 1.8. Before each: predict whether it will settle smoothly, zig-zag in, or blow up. Which regime would you pick for training a real model, and why not "as big as possible"?
The takeaway