Build it on your computer

I build this model with Python and PyTorch, starting from random weights. The source contains the network implementation, data generator and training loop. PyTorch handles the numerical operations and derivatives.

The browser lets you explore my trained models immediately. The commands here train your own model. Setup and training take additional time beyond the 15 to 30 minute walkthrough.

1. Get the files and prepare Python

Download the source and datasets. Extract the archive into a new folder and open a terminal in that folder. Use Python 3.13. These commands are for macOS and Linux:

python3 --version
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r scripts/tiny-llm/requirements.txt

The virtual environment keeps this project's packages separate. The requirements file pins the versions used here. torch is PyTorch. The other packages support export and the later experiments.

py -3.13 -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install -r scripts/tiny-llm/requirements.txt

2. Create the teaching examples

cd course
python scripts/prepare_data.py

This creates addition text for operands from 0 to 99 and writes separate training, validation, calibration and test files. It also creates an outside-range test. The script calculates the teaching answers. The network does not use that calculator when generating.

Browse the actual rows and the generator code. Open data/addition-v1/train.jsonl locally to see the same records.

3. Encode characters and inspect the pairs

python -m tiny_gpt.prepare_tokens
python -m tiny_gpt.inspect_tokens

These commands write numerical token sequences, then print how an input position predicts the following character. See the mapping and shifted targets.

4. Create the network with random weights

Read tiny_gpt/model.py. It defines attention, feed-forward layers, two Transformer blocks and the output scores. Read the explained implementation.

python -m tiny_gpt.inspect_model --save-initial checkpoints/initial.pt

This prints 413,440 parameters and saves their untrained values. A checkpoint is a saved copy of the model's adjustable numbers. The next command learns from those random values.

5. Train and save your model

First run one pass through the data to check the setup:

python -m tiny_gpt.train --run-dir runs/my-first-run --epochs 1 --device cpu

Look for a training loss, validation count and saved best.pt. One epoch is a pipeline check. For the full training run, use a new folder:

python -m tiny_gpt.train --run-dir runs/my-model --epochs 200 --device cpu

The trainer can stop earlier when the validation target is stable. It refuses to reuse a run folder. On an Apple Silicon Mac, replace cpu with mps to use the GPU. Results and time can differ across machines.

See how loss and weight updates work.

6. Generate and test

Copy the generation and test code into course/try_model.py. It loads your runs/my-model/best.pt, predicts tokens and compares them with reserved answers.

python try_model.py

Try both 1+1= and 1+1. Test the full reserved file and the outside-range file. A low training loss does not prove that either test will succeed.

7. Fine-tune and fit probabilities

The basic model is complete. These later experiments use the same source plus the scripts in the archive. Return to the archive root and put your selected weights at the path expected by the experiment:

cd ..
mkdir -p public/tiny-llm
cp course/runs/my-model/best.pt course/runs/pretrain-v2/best.pt
python scripts/tiny-llm/experiments.py --source course --action finetune --device cpu

Fine-tuning teaches the Q and A format and writes scripts/tiny-llm/runs/finetune/best.pt. Its history goes to public/tiny-llm/finetune.json.

The evaluation script compares both original training protocols as well as fine-tuning. Create the first-protocol checkpoint before running it:

cd course
python -m tiny_gpt.train --run-dir runs/my-first-protocol --target-accuracy 0.98 --device cpu
cd ..
cp course/runs/my-first-protocol/best.pt course/runs/pretrain-v1/best.pt
python scripts/tiny-llm/experiments.py --source course --action evaluate --device cpu

This writes test results to public/tiny-llm/evaluation.json and fitted temperatures to public/tiny-llm/calibration.json. Use a fresh extraction for another experiment because these scripts preserve existing output files.

8. Reproduce the recorded feedback experiment

This optional experiment uses my saved fine-tuned weights and Codex's recorded choices. Use a separate fresh extraction and repeat the Python setup in step 1. The choices are tied to those weights, so they cannot be reused with your newly trained model.

python scripts/tiny-llm/restore_checkpoints.py --source course
mkdir -p public/tiny-llm
cp recorded/preferences.json public/tiny-llm/preferences.json
cp recorded/ai-preferences.json public/tiny-llm/ai-preferences.json
python scripts/tiny-llm/rlhf.py --source course --votes public/tiny-llm/ai-preferences.json --feedback-kind ai

Activate the environment first. The script trains a reward head and then updates the language model with PPO. Outputs are saved under scripts/tiny-llm/runs/rlhf/. See the process and its measured regression.

Find your way around the source

The tiny_gpt paths are inside course. The later experiment scripts are under scripts/tiny-llm at the archive root. The included README covers export and verification.

Recorded results and optional file verification. Back to the tutorial.