Study chapter

AI

Models, data, and the thinking behind intelligent products.

The AI chapter builds a practical foundation in machine learning and applied AI, the same skills behind OZmind, our in-house AI engine.

What you'll learn

1
Sep, week 2
Member introductions
Getting to know each other and checking where everyone's skills are.
2
Sep, week 3
Kickoff & setup
Setting up Colab, GitHub, Jupyter, and PyTorch, and how we'll work through ARENA / miniARENA.
3
Sep, week 4
PyTorch & tensors
Hands-on with tensors, broadcasting, and einops / einsum basics.
4
Oct, week 1
Intro to neural networks
MLPs, torch.nn.Module, the training loop, and loss functions.
5
Oct, week 2
CNNs, ResNets & training basics
A light look at CNNs and residual connections, plus a review of how deep learning code is structured.
6
Oct, week 4
Transformer from scratch 1
Tokenization, embeddings, and the inputs and outputs of attention.
7
Oct, week 5
Transformer from scratch 2
Implementing attention, the MLP, the residual stream, and layer norm yourself.
8
Nov, week 1
Sampling & GPT-2
How sampling turns a Transformer's output into text.
9
Nov, week 2
Intro to mechanistic interpretability
TransformerLens, activation caching, and analyzing attention patterns.
10
Nov, week 3
Induction heads & hooks
What induction heads are, and intervening on activations with hooks.
11
Dec, week 1
Hands-on interpretability project
Pick one: function vectors, model steering, SAEs, IOI, or toy models.
12
Dec, week 2
Project wrap-up
Finishing projects and sharing results.

Ready to collect your first brick?

No prior experience required. Pick AI as an interest in your application.