← AI Terminology
TGI - Text Generation Inference
Text Generation Inference (TGI) is Hugging Face’s production-oriented server for serving large language models with batching, streaming, and optimised kernels.
It is a common alternative to vLLM in HF-centric stacks.
It is a common alternative to vLLM in HF-centric stacks.
Why It Matters in AI
Shipping open models needs a robust HTTP server, not notebooks. TGI integrates with the HF ecosystem for scalable text generation, streaming, and deployment on cloud GPUs.
Key Points
| Aspect | Description |
|---|---|
| API | HTTP generate endpoints |
| Ops | Docker images for GPU hosts |
| Use | HF Inference Endpoints and self-host |
| Origin | Hugging Face |
| Related | vLLM, TensorRT-LLM, HF ecosystem |
| Features | Continuous batching-style serving, streaming, tracing |
Simple Analogy
A restaurant branded by the same market where you bought ingredients (HF models) — convenient full-stack dining for that ecosystem.
Common Usage Examples
- Docker run TGI with model id
- Stream tokens to clients
- Deploy on HF Endpoints
- Compare latency/throughput to vLLM
Summary
In short: TGI is Hugging Face’s production text-generation server — a battle-tested way to serve LLMs with streaming and batching.