← AI Terminology
Post-Training Quantization - PTQ
Post-training quantization (PTQ) reduces numerical precision of a trained model (e.g. FP16→INT8/INT4) without full retraining, using calibration data.
It is the fastest path to smaller, faster inference.
It is the fastest path to smaller, faster inference.
Why It Matters in AI
Retraining quantisation-aware is costly; PTQ compresses finished checkpoints for edge and server serving. GPTQ/AWQ-style LLM methods are PTQ family members dominating open LLM deployment.
Key Points
| Aspect | Description |
|---|---|
| Cons | Quality drop if aggressive; needs good calibration |
| Pros | Fast, no full training |
| Vs QAT | QAT trains with quant noise; usually better, heavier |
| Related | Quantization, KV-cache quant |
| Pipeline | Calibrate on samples → quantise weights/activations → validate |
| LLM methods | GPTQ, AWQ, SmoothQuant, GGUF quants |
Simple Analogy
Converting a high-res master recording to a well-tuned compressed file for phones — no re-recording the album, careful encoding instead.
Common Usage Examples
- AWQ/GPTQ quantise HF models
- ONNX Runtime PTQ tools
- Calibrate on domain samples
- Eval perplexity/tasks after quant
Summary
In short: PTQ compresses finished models to lower precision without full retraining — the practical default for fast LLM deployment.