Video lesson 7

Let's build GPT: from scratch, in code, spelled out.

The lecture the whole course was building toward. Build a GPT - a decoder-only Transformer, the same architecture as GPT-2 and GPT-3 - from empty files: the character pipeline, the bigram baseline, attention from its central trick outward, the block, the full assembly, the training harness. Train it on the complete works of Shakespeare until it speaks.