← AI Terminology
State Space Model (SSM)
State space models (SSMs) map input sequences through latent states with structured linear dynamics, enabling efficient long-range sequence modelling as alternatives or complements to attention.
Modern deep SSMs (S4, Mamba) revived the classic signal-processing idea for AI.
Modern deep SSMs (S4, Mamba) revived the classic signal-processing idea for AI.
Why It Matters in AI
Attention’s cost grows with context; SSMs offer linear-time sequence mixing with strong long-range results. They reshape architecture debates and appear in hybrid LLMs.
Key Points
| Aspect | Description |
|---|---|
| Cons | Tradeoffs on some associative recall tasks vs attention |
| Pros | Long context, efficient inference |
| Family | S4, Mamba, hybrid Transformer-SSM |
| Classic | Control theory: state updates x_{t+1}=Ax_t+Bu_t |
| Related | RWKV, linear attention |
| Deep SSMs | Learned structured A/B/C; HiPPO init lineage |
Simple Analogy
A running summary notebook updated with each new sentence (state), instead of re-reading the entire book every time (full attention).
Common Usage Examples
- S4/Mamba papers for long audio/text
- Hybrid Jamba-style blocks
- Benchmark long-range arena tasks
- Serve long logs with linear-time models
Summary
In short: State space models evolve a latent state through time for efficient sequence modelling — a major alternative lineage to pure attention.