Fix your init & build BatchNorm

What you’ll build

Your Module 3 MLP worked - and it was quietly sick. It opened training at loss 27 when a network with no idea should score 3.3, burned thousands of steps un-learning its own confident garbage, and most of a tanh layer arrived dead - the loss curve never told you any of it. This project teaches you to notice: you fix the init down to Kaiming's actual formula, build BatchNorm from scratch including the running statistics that make single-example inference possible, and rebuild everything as your own mini torch.nn - six layers, 47,024 parameters - instrumented with the plots professionals actually watch. The finale prints a first loss of 3.3, ends past your Module 3 number, and hands you a diagnostic dashboard for every network you'll ever train.

What you’ll learn

By the end, you’ll be able to:

• Diagnose why Module 3's first loss was ~27 when an unsure network scores -log(1/27) = 3.3 - and fix it with two scale changes
• Measure tanh saturation, map your first dead neurons, and bring the hidden layer back to life
• Replace eyeballed init factors with Kaiming's formula - gain / sqrt(fan_in) - and know where the 5/3 comes from
• Build BatchNorm's forward pass by hand: center, scale, then let the network undo it with trainable gain and bias
• Keep running statistics so a batch of ONE works - the exact reason model.train() / model.eval() exist
• Write your own torch.nn - Linear, BatchNorm1d, Tanh classes - and stack them into a 6-layer, 47,024-parameter network
• Read training telemetry like a professional: per-layer activation histograms, gradient stats, and the update-to-data ratio's -3 line

Before you start: Do Module 3 (the MLP) first - this project starts from that exact network (its naive init ships as the 'before' picture) and its finale beats your own 2.2564. PyTorch is the one install; the companion Jupyter notebook is optional everywhere else but earns its keep here - this module is about seeing.

Your AI tutor

It won't fix the network for you. It asks the question that cracks the case - what should an untrained network score? - then runs the tests to check. And it knows where this lecture bites: the confidently-wrong logits, the batch coupling, the forgotten .eval().