Recorded results and verification

The playground runs the exported weights locally in a single-thread WASM worker. It uses the original character vocabulary, a 32-token context and greedy prediction over all 98 tokens. The model receives your printable ASCII input verbatim.

Recorded evaluation

Four generated tokens including EOS. Checkpoints were selected with validation, then frozen. Test results never selected a model.

The base model scored 100% on validation, yet missed 9+2= on the final test, continuing with 10. All four evaluated policies scored zero exact answers on the 497 out-of-range examples.

Evaluation counts, slices, errors and dataset hashes. Actual fine-tuning history and selection. Original training history.

Calibration measurements

Temperatures were fitted separately on 496 reserved calibration examples. These metrics cover held-out answer characters and EOS. They measure next-token predictions.

Calibration improved the first attempt's held-out loss. It worsened held-out loss for the base and fine-tuned policies. Fitting on one split does not guarantee improvement on another.

Fitted temperatures, provenance and reliability bins.

Optional file verification

A checkpoint stores the trained weights. SHA-256 is a fingerprint of a file's bytes. If the file changes, its fingerprint changes. I use it to check that the browser loads the intended file and to connect a result to the weights that produced it. It does not measure model quality.

The Python checkpoint and the browser ONNX export are different file formats, so their fingerprints differ. Conversion checks compare their predictions. The manifest is a machine-readable index of model names, file paths, sizes and fingerprints. The browser reads it automatically; you do not need to read it to build or use the model.

Browser model file index. Python conversion checks. Run browser parity verification.

Feedback experiment

The assistant chose 120 preferences over actual generated responses. These are AI labels, not human labels. A reward head learned from 84 training comparisons, with 17 for validation and 19 held out. It correctly ranked 6 of the 19 held-out comparisons. A fixed thirty-update PPO run used this frozen learned reward and a supervised reference policy. Formatted arithmetic regressed from 486 to 447 correct answers out of 502.

Actual AI label decisions and attribution. Reward training, PPO logs and held-out preference results.

Reader and device limits

The reading time is estimated at 15 to 30 minutes. No human usability study has been completed. Narrow-layout emulation is distinct from testing a physical phone. Browser verification reports name the environments actually checked.

Runtime and data provenance

The arithmetic data and model source come from my local Tiny Arithmetic GPT course. The runtime is ONNX Runtime Web 1.24.1, under the MIT license. Model and runtime assets are hosted with the site and use content hashes. No API key or inference server is required.

Official runtime deployment documentation. Runtime asset hashes and license directory.