← AI Terminology
Mamba
Mamba is a selective state space model architecture that makes SSM parameters input-dependent, enabling strong performance with linear-time sequence modelling.
It sparked the 2023–2024 wave of attention-free and hybrid LLMs.
It sparked the 2023–2024 wave of attention-free and hybrid LLMs.
Why It Matters in AI
Pure transformers dominate, but Mamba showed SSMs can compete on language with better scaling in sequence length. Selective scanning is the key idea; hybrids now ship in production-scale models.
Key Points
| Aspect | Description |
|---|---|
| Use | Language, genomics, audio; hybrid LLMs |
| Idea | Selective SSM: content-aware state updates |
| Impl | Custom CUDA scan kernels matter |
| Origin | Gu & Dao, Mamba (2023) |
| Related | S4, Mamba-2, Transformer hybrids |
| Complexity | Linear in sequence length |
Simple Analogy
A notepad that chooses what to write down or ignore based on how important each new word seems — selective memory, not a fixed filter.
Common Usage Examples
- Mamba blocks in open LMs
- Long-sequence throughput vs Transformer
- Mamba-2 / SSD variants
- Hybrid attention+Mamba stacks
Summary
In short: Mamba is a selective SSM architecture for linear-time sequence modelling — the flagship modern alternative to pure attention.