← AI Terminology
TTFT - Time to First Token
TTFT (Time to First Token) is the latency from request arrival until the first output token is produced — dominated by prefill and scheduling delays.
It is a primary UX metric for interactive chat and streaming APIs.
It is a primary UX metric for interactive chat and streaming APIs.
Why It Matters in AI
Users feel waiting before anything appears. Optimising TTFT (prefix cache, faster prefill kernels, good scheduling) makes products feel snappy even when total generation is long. SLAs usually split TTFT from per-token latency.
Key Points
| Aspect | Description |
|---|---|
| Report | p50/p95 TTFT under load |
| Vs TPS | TTFT is start delay; TPS is ongoing speed |
| Improve | Prefix caching, smaller models, chunked prefill, better batching |
| Related | Prefill vs decode, streaming |
| Includes | Queue time + prefill compute + first decode |
| Tradeoff | Large batches help throughput, can hurt TTFT |
Simple Analogy
How long until the first word of a reply appears on screen after you hit send — the awkward pause before the conversation feels alive.
Common Usage Examples
- Load-test chat APIs for p95 TTFT
- vLLM metrics: time to first token
- Prefix cache to cut repeated prefill
- Stream tokens so users see TTFT, not full wait
Summary
In short: TTFT is how long until the first output token — the key snappiness metric for streaming LLM apps.