← AI Terminology
Mechanistic Interpretability
Mechanistic interpretability reverse-engineers neural networks into human-understandable algorithms — circuits, features, and computational graphs inside the weights.
A research program distinct from post-hoc feature attribution alone.
A research program distinct from post-hoc feature attribution alone.
Why It Matters in AI
If models implement algorithms, we might audit and edit them. Mech interp aims at safety and science of deep nets (induction heads, superposition, SAEs). Growing field for frontier model oversight.
Key Points
| Aspect | Description |
|---|---|
| Goals | Circuits, features, causal interventions |
| Tools | Activation patching, logit lens, SAEs, probes |
| Limits | Scale; polysemanticity; incomplete stories |
| Related | SAE, superposition, explainable AI |
| Findings | Induction heads, copy mechanisms, etc. |
| Community | Academic + lab interp teams |
Simple Analogy
Not only scoring a student’s essay, but reverse-engineering the mental arithmetic steps they actually used — open the head, map the algorithm.
Common Usage Examples
- Activation patching tutorials
- TransformerLens-style tooling
- Circuit papers on induction heads
- SAE feature dashboards
Summary
In short: Mechanistic interpretability reverse-engineers model internals into circuits and features — aiming to explain how networks actually compute.