← AI Terminology
MQA - Multi-Query Attention
MQA (Multi-Query Attention) uses many query heads but only a single shared key head and value head.
Proposed to accelerate decoder inference by drastically reducing KV-cache bandwidth.
Proposed to accelerate decoder inference by drastically reducing KV-cache bandwidth.
Why It Matters in AI
During generation, reloading huge KV caches dominates latency. MQA minimises that cost and inspired the softer GQA compromise. It explains the design lineage of fast LLM serving (PaLM, some Falcon models).
Key Points
| Aspect | Description |
|---|---|
| Con | Can hurt quality vs MHA; GQA often preferred today |
| Pro | Smallest KV cache; best bandwidth for decode |
| Idea | H_kv = 1 while H_q stays large |
| Origin | Shazeer, Fast Transformer Decoding (2019) |
| Used in | PaLM, some Falcon models, research serving stacks |
| Successor | GQA as quality/speed compromise |
Simple Analogy
One shared answer key for the whole class (K/V), while every student still writes their own questions (Q) — fewer books to carry when recalling past dialogue.
Common Usage Examples
- Single K/V projection in decoder self-attention
- PaLM notes on multi-query
- Larger batch or longer context for same VRAM
- Ablation: MQA vs GQA on chat quality
Summary
In short: MQA shares one K/V across all query heads to slash decode memory traffic — the extreme efficiency end of attention-head design.