Build the autograd engine
Based on Andrej Karpathy’s course.
Python · ~60 min · tests verify your work
Start building
Use the template, open it in your editor, and tell your AI tutor to take it from there.
Use this templateFree · no signup to start · you keep the code
Just want to read the code first?
Clone it locally - no GitHub account needed. Your changes stay on your machine, so use the template above to keep your progress.
What you’ll build
A working autodiff engine, built from scratch. By the end, a real trainable neuron whose gradients you can prove are correct, using the same machinery PyTorch runs at scale.
What you’ll learn
By the end, you’ll be able to:
- Implement a Value class that records every operation into a computation graph
- Write the backward pass for reverse-mode autodiff so gradients fill in automatically
- Derive and apply the chain rule by hand through +, *, and tanh
- Check your gradients against numerical estimates, so you know they're correct
- Wire it into a single neuron and watch the gradients flow end to end
Before you start: Comfortable writing Python classes. A faint memory of derivatives is plenty; you re-derive the chain rule here by building it.
Your AI tutor
It won't hand you the answer. It asks the one question that makes the next line obvious, then checks your work by running the tests.