← AI Terminology
ONNX - Open Neural Network Exchange
ONNX (Open Neural Network Exchange) is an open-source format for representing machine learning models as a computation graph — enabling models trained in any ML framework (PyTorch, TensorFlow, scikit-learn) to be exported once and run anywhere: any runtime, hardware, or operating system that supports ONNX.
It is the universal model interchange format for AI deployment.
It is the universal model interchange format for AI deployment.
Why It Matters in AI
Models are typically trained in PyTorch but deployed to diverse targets: ONNX Runtime on Windows, TensorRT on NVIDIA, OpenVINO on Intel, CoreML on Apple. Without ONNX, each framework-to-runtime pair requires bespoke conversion. ONNX provides one standard format: export once, optimise for any hardware. ONNX Runtime from Microsoft is among the fastest cross-platform inference engines and powers AI features in Windows, Office, and Azure.
Key Points
| Aspect | Description |
|---|---|
| Format | Protobuf file (.onnx) containing the model graph: operators, weights, input/output shapes |
| Ecosystem | Hugging Face optimum library: optimum-cli export onnx --model bert-base-uncased ./onnx/ |
| Operators | Standard ONNX opset — conv, matmul, relu, etc. — supported across runtimes; versioned opsets |
| Conversion | torch.onnx.export(), tf2onnx, sklearn-onnx — framework-specific exporters |
| ONNX Runtime | Microsoft's high-performance inference engine — CPU, CUDA, DirectML, TensorRT execution providers |
| Optimisation | onnxruntime.InferenceSession applies graph optimisations (fusion, constant folding) automatically |
Simple Analogy
PDF for ML models: just as a PDF can be opened on any device regardless of what software created it, an ONNX file can be run on any ONNX-compatible runtime regardless of which framework trained it — write once, run anywhere.
Common Usage Examples
torch.onnx.export(model, dummy_input, "model.onnx", opset_version=17)— PyTorch → ONNXsess = onnxruntime.InferenceSession("model.onnx", providers=["CUDAExecutionProvider"])outputs = sess.run(None, {"input": input_array})— run ONNX model inference- HuggingFace Optimum:
optimum-cli export onnx --model meta-llama/Llama-3-8B ./llama3-onnx/ - TensorRT: import ONNX →
trtexec --onnx=model.onnx --saveEngine=model.trt --fp16
Summary
In short: ONNX is the universal model interchange format — export any model from any framework once, then deploy it to any hardware or runtime, eliminating the need for bespoke framework-to-deployment conversions.