If you are new to this site, the archive can feel large. This page groups the main books, articles, courses, and reference pages into a few learning paths so you can pick the route that matches what you want to understand next.

Build LLMs from scratch Architecture, tokenization, training, and PyTorch implementation. Reasoning models and advanced training Reinforcement learning, inference-time scaling, distillation, and evaluation. Compare LLM architectures Attention variants, MoE, KV cache, and modern model families. Learn practical ML and PyTorch Applied machine learning, deep learning, and course material.

Learning Paths

1

Build LLMs from scratch

Best if you want the bottom-up route: tokenization, attention, GPT-style models, pretraining, and finetuning.

1. Start with Build a Large Language Model (From Scratch) for the book and course links.
2. Use the LLMs-from-scratch code repository as the main implementation companion.
3. Read the BPE tokenizer article before diving into tokenization details.
4. Work through self-attention from scratch to build the core mental model.
5. Add the KV cache article once you start thinking about inference efficiency.

2

Reasoning models and advanced LLM training

Best if you already know the LLM basics and want to understand reinforcement learning for LLMs, inference-time scaling, distillation, and other post-training methods behind modern reasoning models.

1. Read Understanding Reasoning LLMs for the high-level map of reasoning methods.
2. Read the inference-time scaling overview for test-time compute methods.
3. Read the RL for reasoning article for GRPO, reinforcement learning, and advanced post-training ideas.
4. Start with Build a Reasoning Model (From Scratch) for the book and repository links.
5. Use the reasoning-from-scratch repository as the main implementation companion.

3

Compare modern LLM architectures

Best if you want to understand how today's open-weight models differ in attention, MoE, normalization, and inference tradeoffs.

1. Browse the LLM Architecture Gallery for a visual overview of model families.
2. Read The Big LLM Architecture Comparison for the main architecture narrative.
3. Read the visual guide to attention variants for MHA, GQA, MLA, sparse attention, and hybrids.
4. Read the workflow article to see how to inspect new open-weight releases.
5. Read the gallery announcement for context on how the reference collection is organized.

4

Learn practical ML and PyTorch

Best if you want a broader applied machine learning route before or alongside modern LLM material.

1. Use Machine Learning with PyTorch and Scikit-Learn for a full applied path.
2. Try PyTorch in One Hour for a compact refresher.
3. Study the university deep learning course for a longer lecture series and university material.
4. Use the deep learning resources page for older but still useful model notebooks and references.
5. Read Machine Learning Q and AI for concise explanations of common modern ML and AI concepts.

Shortcuts

Latest blog and notes archive All books LLM Architecture Gallery Courses More resources