Skip to content

RL, from scratch
to LLMs

Part I Foundations

01What is RL?
02Bandits
03MDPs
04Dynamic programming
05MC & TD learning

Part II Deep RL

06DQN
07Policy gradients
08Actor-critic & GAE
09PPO
10The wider landscape

Part III RL for LLMs

11LLM as a policy
12RLHF
13DPO
14Reward sources
15RLVR & GRPO
16Algorithm zoo
17Process rewards
18Agentic RL
19Failure modes
20RL systems
21The frontier

∑Notation
✎Exercises & setup
¶Paper map

Paper map

The papers behind each chapter, in reading order.

This page is assembled once the chapters are written.