rohitg00/ai-engineering-from-scratch

From the creator of Agent Memory - #1 Persistent memory ⭐ which naturally works with any agents or chat assistants.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Start here: choose what you want to build

You do not need to scan 523 lessons before beginning. Pick one goal. Each link opens the same curriculum on GitHub or the website, and both versions use the same lesson code.

Not sure where you fit? Use the start-learning placement tutor or the website prerequisites guide.

Compare four core domains and six career routes in the AI Engineering Learning Paths.

Sponsors

Use every lesson the same way

1. 1Read docs/en.md and explain the core idea in your own words.
2. 2Type and build the important code instead of treating the code block as decoration.
3. 3Run the lesson command from the repository root, the directory containing README.md and phases/.
4. 4Keep evidence: the command, working directory, exit code, meaningful output, and the artifact you changed or produced.
5. 5Continue only when you can explain the output and make one small change without guessing.

Commands in lesson pages are paths from the repository root unless the lesson explicitly says to change directories. If a lesson offers several languages, run the implementation for the language you are learning.

Clone it and produce your first evidence

git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python3 phases/00-setup-and-tooling/01-dev-environment/code/verify.py --route beginner
python3 phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py

The preflight separates requirements needed now from tools needed later. Every required failure includes the detected reason and a corrective command. The second command is a dependency-free lesson and ends by showing that a matrix times a vector is the operation inside a neural network layer. Save that terminal output as your first evidence.

Add the AI tutor in 30 seconds

If Node.js, npx, and a skill-capable coding agent are already installed, your coding agent can become your tutor in two commands. A repository clone is not needed to install or read the tutor. Runnable focused-path labs need python3. Agent Skills host labs also need a selected host and a writable user or project skill scope.

Check the local requirements first:

node --version
npx --version
python3 --version

Then install the curriculum skills and choose the host and scope you intend to use when the installer asks:

npx skills add rohitg00/ai-engineering-from-scratch

Invocation syntax belongs to the host, not to the portable SKILL.md format:

A ten-question placement quiz maps what you already know to a starting phase and saves a personalized study plan to LEARNING.md. From there, the learn skill teaches one lesson per session: concept, math, code, quiz. It streams lessons straight from this repo, and the course-guide skill jumps you to the exact lesson that covers anything you are stuck on. In Codex, invoke these skills with learn and course-guide; in Claude Code, use /learn and /course-guide; in other compatible hosts, ask to use the skill by name.

Only want Model Context Protocol (MCP)? Use the MCP invocation for your host. It creates MCP-LEARNING.md and follows one 17-lesson route through stateless requests, transports, bidirectional work, security, reliability, registry governance, and conformance evidence. The exact order and checkpoints live in the Model Context Protocol (MCP) manifest.

Only want Agent Skills? Use the Agent Skills invocation for your host. It creates AGENT-SKILLS-LEARNING.md and follows one coherent five-lesson route: contract, discovery, invocation, sandbox boundaries, then release evals and real-host portability. Start on the web with the Agent Skills path.

The installer lists the hosts it can configure and asks where to install. If you do not have Node.js, npx, python3, a supported host, or a writable scope yet, use the website or read docs/en.md manually. That path teaches the concepts, but real-host discovery, invocation, script, and uninstall evidence remains pending until the preflight is available. Read the lessons at aiengineeringfromscratch.com.

How this works

Most AI material teaches in scattered pieces. A paper here, a fine-tuning post there, a flashy agent demo somewhere else. The pieces rarely line up. You ship a chatbot but can't explain its loss curve. You hook a function to an agent but can't say what attention does inside the model that's calling it.

This curriculum is the spine. 20 phases, 523 lessons, four languages: Python, TypeScript, Rust, Julia. Linear algebra at one end, autonomous swarms at the other. Every algorithm gets built from raw math first. Backprop. Tokenizer. Attention. Agent loop. By the time PyTorch shows up, you already know what it's doing under the hood.

Each lesson runs the same loop: read the problem, derive the math, write the code, run the test, keep the artifact. No five-minute videos, no copy-paste deploys, no hand-holding. Free, open source, and built to run on your own laptop.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

The shape of the curriculum

Twenty phases stack on top of each other. Math is the floor. Agents and production are the roof. Skip ahead if you already know the lower layers, but don't skip and then wonder why something at the top is breaking.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#fafaf5','primaryTextColor':'#1a1a1a','primaryBorderColor':'#3553ff','lineColor':'#3553ff','fontFamily':'JetBrains Mono','fontSize':'12px'}}}%%
flowchart TB
  P0["Phase 0 — Setup & Tooling"] --> P1["Phase 1 — Math Foundations"]
  P1 --> P2["Phase 2 — ML Fundamentals"]
  P2 --> P3["Phase 3 — Deep Learning Core"]
  P3 --> P4["Phase 4 — Vision"]
  P3 --> P5["Phase 5 — NLP"]
  P3 --> P6["Phase 6 — Speech & Audio"]
  P3 --> P9["Phase 9 — RL"]
  P5 --> P7["Phase 7 — Transformers"]
  P7 --> P8["Phase 8 — GenAI"]
  P7 --> P10["Phase 10 — LLMs from Scratch"]
  P10 --> P11["Phase 11 — LLM Engineering"]
  P10 --> P12["Phase 12 — Multimodal"]
  P11 --> P13["Phase 13 — Tools & Protocols"]
  P13 --> P14["Phase 14 — Agent Engineering"]
  P14 --> P15["Phase 15 — Autonomous Systems"]
  P15 --> P16["Phase 16 — Multi-Agent & Swarms"]
  P14 --> P17["Phase 17 — Infrastructure & Production"]
  P15 --> P18["Phase 18 — Ethics & Alignment"]
  P16 --> P19["Phase 19 — Capstone Projects"]
  P17 --> P19
  P18 --> P19

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

The shape of a lesson

Each lesson lives in its own folder, with the same structure across the entire curriculum:

phases/<NN>-<phase-name>/<NN>-<lesson-name>/
├── code/      runnable implementations (Python, TypeScript, Rust, Julia)
├── docs/
│   └── en.md  lesson narrative
└── outputs/   prompts, skills, agents, or MCP servers this lesson produces

Every lesson follows six beats. The Build It / Use It split is the spine — you implement the algorithm from scratch first, then run the same thing through the production library. You understand what the framework is doing because you wrote the smaller version yourself.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#fafaf5','primaryTextColor':'#1a1a1a','primaryBorderColor':'#3553ff','lineColor':'#3553ff','fontFamily':'JetBrains Mono','fontSize':'13px'}}}%%
flowchart LR
  M["MOTTO<br/><sub>one-line core idea</sub>"] --> Pr["PROBLEM<br/><sub>concrete pain</sub>"]
  Pr --> C["CONCEPT<br/><sub>diagrams &amp; intuition</sub>"]
  C --> B["BUILD IT<br/><sub>raw math, no frameworks</sub>"]
  B --> U["USE IT<br/><sub>same thing in PyTorch / sklearn</sub>"]
  U --> S["SHIP IT<br/><sub>prompt · skill · agent · MCP</sub>"]

Getting started

Three ways in. Pick one.

*Option A — learn in your terminal (recommended).* After the Node.js, npx, host, and scope preflight above, install the learning skills into a compatible agent and let the course drive itself:

npx skills add rohitg00/ai-engineering-from-scratch

Use the host-specific invocation table above. The installed skills provide start-learning, learn, course-guide, and the focused learn-mcp and learn-agent-skills routes. Lesson prose can stream from this repository without a clone. A local clone is required for copied repository code commands and executable MCP or Agent Skills labs. Progress lives in LEARNING.md, MCP-LEARNING.md, or AGENT-SKILLS-LEARNING.md in your project, so every session can resume.

Option B — read. Open any completed lesson on aiengineeringfromscratch.com or expand a phase under Contents. No setup, no cloning.

Option C — clone and run.

git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python3 phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py

Cloning also auto-loads the learning skills in Claude Code, and gives every lesson's code to the learn tutor for real execution instead of read-along.

Prerequisites

• You can write code (any language; Python helps).
• You want to understand how AI actually works, not just call APIs.

Prepare for Claude certifications

The Claude Certification Academy is a free, open-source preparation program for all four official Claude certification tracks: Associate Foundations, Developer Foundations, Architect Foundations, and Architect Professional. Each route combines blueprint-mapped lessons, runnable labs, a diagnostic, capstone work, and a full-length original practice exam.

Use the AI-native GitHub onboarding guide with Claude Code, Codex, ChatGPT, Cursor, or another agent. Run claude-certification in Codex, /claude-certification in Claude Code, or ask another host to use claude-certification. It chooses a track, creates a persistent route in CLAUDE-CERTIFICATION.md, teaches one step at a time, runs the real labs, and gives artifact-based feedback. The same curriculum remains available on the certification website.

The academy is independent study material based on public exam objectives. It is not affiliated with Anthropic, does not reproduce live exam questions, and cannot guarantee a passing score.

The learning skills

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Read the core curriculum as a book

The 20-phase core curriculum under phases/ compiles into a six-volume book series. EPUB and PDF are built by CI from the same core lesson sources and attached to every GitHub release; the links below always resolve to the newest release. Volume numbers index the series, not versions: each copy carries a dated edition stamp, and older editions stay downloadable from their release.

Certification curricula are intentionally not converted into the books. Their AI tutor state, runnable labs, interactive figures, diagnostics, and timed mocks remain first-class on GitHub and the website.

The book is the snapshot; this repository is the living edition. Every chapter ends with links back to the lesson's animated figures, quiz, and runnable code. Build locally with python3 scripts/build_book.py (pandoc required); pipeline details in book/README.md.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Every lesson ships something

Other curricula end with "congratulations, you learned X." Each lesson here ends with a reusable tool you can install or paste into your daily workflow.

FIG_002 · A worked sample

Phase 14, lesson 1: the agent loop. ~120 lines of pure Python, no dependencies.

code/agent_loop.py   build it

def run(query, tools):
    history = [user(query)]
    for step in range(MAX_STEPS):
        msg = llm(history)
        if msg.tool_calls:
            for call in msg.tool_calls:
                result = tools[call.name](https://github.com/rohitg00/ai-engineering-from-scratch/blob/main/**call.args)
                history.append(tool_result(call.id, result))
            continue
        return msg.content
    raise StepLimitExceeded

outputs/skill-agent-loop.md   ship it

---
name: agent-loop
description: ReAct-style loop for any tool list
phase: 14
lesson: 01
---

Implement a minimal agent loop that...

outputs/prompt-debug-agent.md

You are an agent debugger. Given the trace
of an agent run, identify the step where
the agent went wrong and explain why...

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Contents

Twenty phases. Click any phase to expand its lesson list.

Phase 0: Setup & Tooling 12 lessons

Lessons 06-18 and 28-31 form the focused Model Context Protocol (MCP) path. Its manifest order is 06, 07, 08, 09, 10, 11, 12, 13, 14, 15, 16, 18, 17, 28, 29, 30, 31. Start it with the host-specific learn-mcp invocation above. Lesson 23 is its only optional capstone and also requires Lessons 19 and 20.

Lessons 22 and 24-27 form the focused Agent Skills learning path, from package contract through real-host release gates. Start it with the host-specific learn-agent-skills invocation shown above; do not follow numeric next navigation from 22 to 23.

Each Phase 14 workbench lesson (31-42) ships a mission.md briefing the agent before it opens the full lesson docs.

Lessons 31-46 form the Agent-Assisted Engineering path. Its manifest order combines the workbench foundation with task framing, planning, delegation, and durable feedback. Lessons 47-54 form the Product Judgment and Delivery path, from outcome framing through evidence, risk, scope, measurement, staged release, and feedback ownership.

Deep-build tracks — multi-lesson series that build a complete subsystem from scratch.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

The toolkit

Every lesson produces a reusable artifact. By the end you have:

outputs/
├── prompts/      prompt templates for every AI task
└── skills/       SKILL.md files for AI coding agents

Plug them into Claude, Cursor, Codex, OpenClaw, Hermes, or any agent that reads a SKILL.md / AGENTS.md directory. Real tools, not homework.

Install course skills into your agent

Two skill sets, two installers:

The learning skills (start-learning, learn, course-guide, learn-mcp, learn-agent-skills, claude-certification, find-your-level, and check-understanding) live under skills/ and install into a supported skill-capable host with one command. Installation needs Node.js and npx, but not a repository clone or Python:

npx skills add rohitg00/ai-engineering-from-scratch

skills writes to the host and scope selected during installation, such as .claude/skills/, .cursor/skills/, .codex/skills/, or another supported skills folder. Verify that the selected host discovers that exact destination.

The lesson artifacts. The repo ships 396 skills and 99 prompts under phases/**/outputs/; install them via scripts/install_skills.py. Requires cloning the repo. Supports tag filters, dry-runs, and per-agent layouts:

python3 scripts/install_skills.py <target>                                 # every skill, default --layout skills (nested)
python3 scripts/install_skills.py <target> --layout skills                 # same as above, explicit
python3 scripts/install_skills.py <target> --type all                      # skills + prompts + agents
python3 scripts/install_skills.py <target> --phase 14                      # one phase only
python3 scripts/install_skills.py <target> --tag rag                       # filter by tag
python3 scripts/install_skills.py <target> --layout flat                   # flat files
python3 scripts/install_skills.py <target> --dry-run                       # preview without writing
python3 scripts/install_skills.py <target> --force                         # overwrite existing files

<target> is the skills directory for your agent (examples: ~/.claude/skills/, ~/.cursor/skills/, ~/.config/openclaw/skills/, .skills/, or any path your agent reads).

By default the script refuses to overwrite an existing destination and exits with code 1 after listing every colliding path. Use --dry-run to preview collisions or --force to overwrite. Every non-dry-run run writes a manifest.json in the target with the full inventory grouped by type and phase. Pick the layout your agent reads:

Drop the agent workbench into your own repo

The Phase 14 capstone ships a reusable Agent Workbench pack (AGENTS.md, schemas, init / verify / handoff scripts). Scaffold it into any repo with:

python3 scripts/scaffold_workbench.py path/to/your-repo            # full pack + seeds
python3 scripts/scaffold_workbench.py path/to/your-repo --minimal  # skip docs/
python3 scripts/scaffold_workbench.py path/to/your-repo --dry-run  # preview only
python3 scripts/scaffold_workbench.py path/to/your-repo --force    # overwrite

You get the seven workbench surfaces wired up, a starter task_board.json, and a fresh agent_state.json at schema_version: 1. From there: edit the task, edit AGENTS.md, run scripts/init_agent.py, hand the contract to your agent. The pack source lives at phases/14-agent-engineering/42-agent-workbench-capstone/outputs/agent-workbench-pack/.

Browse the entire course as JSON

scripts/build_catalog.py walks every phase, every lesson, every artifact on disk and writes catalog.json at the repo root. One file, every course truth.

python3 scripts/build_catalog.py               # writes <repo>/catalog.json
python3 scripts/build_catalog.py --stdout      # to stdout, do not touch repo
python3 scripts/build_catalog.py --out path/to/file.json

The catalog is filesystem-derived, not README-derived, so counts always match what is actually on disk. Use it for site builds, downstream tooling, or to verify the README counts have not drifted. Schema is documented at the top of the script.

A GitHub Action (.github/workflows/curriculum.yml) rebuilds catalog.json on every PR and fails the build if the committed file is stale. After editing any lesson, run python3 scripts/build_catalog.py and commit the result, or CI will reject the PR. The same workflow runs audit_lessons.py in warn-only mode (so existing drift does not block contributors).

Smoke-check every lesson's Python code

scripts/lesson_run.py byte-compiles every .py file under each lesson's code/ directory. Default mode is syntax-check only — no execution, no API keys, no heavy ML deps required. Catches the regressions contributors introduce most often (bad indentation, broken f-strings, stray edits).

python3 scripts/lesson_run.py                  # syntax-check the whole curriculum
python3 scripts/lesson_run.py --phase 14       # one phase only
python3 scripts/lesson_run.py --json           # JSON report on stdout
python3 scripts/lesson_run.py --strict         # exit 1 if any lesson fails
python3 scripts/lesson_run.py --execute        # actually run, 10s timeout per lesson

--execute runs each lesson's code/main.py (or the first .py file) with a 10-second timeout. Lessons whose entry file starts with a # requires: pkg1, pkg2 comment listing non-stdlib deps are skipped with reason needs <deps>. The script is opt-in and not wired into CI.

Stdlib only, Python 3.10+. Set LINK_CHECK_SKIP=domain1,domain2 to override the default skip-list (twitter.com, x.com, linkedin.com, instagram.com, medium.com — domains that aggressively block automated HEAD/GET).

Where to start

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Why this matters now

• Attention Is All You Need — Vaswani et al., 2017 → Phase 7
• Language Models are Few-Shot Learners (GPT-3) → Phase 10
• Denoising Diffusion Probabilistic Models → Phase 8
• InstructGPT / RLHF → Phase 10
• Direct Preference Optimization → Phase 10
• Chain-of-Thought Prompting → Phase 11
• ReAct: Reasoning + Acting in LLMs → Phase 14
• Model Context Protocol — Anthropic → Phase 13

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Contributing

Before submitting a lesson, run the invariant check:

python3 scripts/audit_lessons.py           # full curriculum
python3 scripts/audit_lessons.py --phase 14  # single phase
python3 scripts/audit_lessons.py --json    # CI-friendly output

Exit code is non-zero when any rule fails. Rules (L001–L010) validate directory shape, docs/en.md presence + H1, code/ non-emptiness, quiz.json schema (rejects the legacy q/choices/answer keys that caused issue #102), and relative links inside lesson docs.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

Sponsor the work

Free, MIT-licensed, 523 lessons. Thank you to the sponsors and backers who make the work possible. See all sponsors and backers.

Want to support the work? See sponsorship options, including hardware sponsorships, or sponsor on GitHub.

░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒░░░▒▒▒

If this manual helped you, star the repo. It keeps the project alive.

License

MIT. Use it however you want — fork it, teach it, sell it, ship it. Attribution appreciated, not required.

Maintained by Rohit Ghumare and the community.