Efficient Mechanistic Circuit Analysis of Open-Weight LLMs on Apple Silicon

J. Melton · American Code Labs · Preprint, August 2026

Download PDF

Abstract

We replicate and extend the GPT-2 Small indirect object identification circuit analysis on Llama-3.2-3B and Pythia-1.4B using an MLX activation-patching harness on a single Apple Silicon machine. On the standard single-template dataset both models yield sparse, faithful circuits, recovering 66.3% and 85.4% of clean logit difference with all other heads mean-ablated. Re-evaluated on a 15-template dataset with balanced name ordering, faithfulness collapses to 0.228 and -0.028 while both models still perform the task above 90% of their original logit difference. The behaviour transfers; the circuits do not.

Code and data

github.com/american-code/ResearchPapersdata/ioi/

Licensed CC BY 4.0.