← AI Terminology
CoWoS - Chip on Wafer on Substrate
CoWoS (Chip on Wafer on Substrate) is TSMC's 2.5D advanced packaging technology that bonds multiple chiplets — a GPU die and stacks of HBM memory — onto a silicon interposer, enabling ultra-wide memory buses (up to 8,192-bit) that monolithic packaging cannot achieve.
It is the packaging technology inside NVIDIA H100, H200, and B200 accelerators.
It is the packaging technology inside NVIDIA H100, H200, and B200 accelerators.
Why It Matters in AI
AI training is memory-bandwidth-bound: moving weights and activations between compute and memory is the bottleneck, not arithmetic. CoWoS places HBM stacks millimetres from the GPU die on a shared interposer, delivering 3.35 TB/s (H100 SXM) versus ~1 TB/s for GDDR6X — a 3× bandwidth advantage that translates directly into faster matrix operations. TSMC's CoWoS capacity is the physical constraint limiting NVIDIA GPU supply, making it a geopolitical and financial chokepoint for the AI industry.
Key Points
| Aspect | Description |
|---|---|
| Bandwidth | H100 SXM: 3.35 TB/s; H200: 4.8 TB/s — impossible with conventional GDDR packaging |
| Interposer | Passive silicon layer connecting GPU die and HBM stacks via short, dense copper interconnects |
| Competitors | Intel EMIB, AMD Infinity Fabric (via TSMC CoWoS), Samsung H-Cube — same 2.5D concept |
| HBM stacking | 4–8 HBM3e stacks per package — each stack is 4–12 DRAM dies bonded vertically |
| CoWoS-S vs -L | CoWoS-S: silicon interposer; CoWoS-L: larger local silicon interconnect layer — scales to B200 |
| Capacity crunch | TSMC CoWoS capacity was the limiting factor in H100 supply throughout 2023–2024 |
Simple Analogy
A high-speed rail hub connecting airport terminals: instead of building one giant terminal (monolithic die), you build specialised terminals (GPU compute die, HBM memory stacks) and connect them via wide, dedicated rail lines (the interposer's copper interconnects). The result is far more total throughput than any single terminal could handle — and the rail hub is CoWoS.
Common Usage Examples
- NVIDIA H100 SXM: one GH100 GPU die + six HBM3 stacks on a CoWoS-S interposer — 80 GB, 3.35 TB/s
- NVIDIA H200: same die, HBM3e upgrade → 141 GB, 4.8 TB/s — packaging change, not die change
- NVIDIA B200: CoWoS-L, two Blackwell dies + HBM3e → 192 GB, 8 TB/s — interposer spans two dies
- AMD MI300X: CDNA3 GPU dies + HBM3 stacks on CoWoS — 192 GB, 5.3 TB/s
- Supply chain:
TSMC CoWoS capacityis a tracked metric in AI infrastructure investment analysis
Summary
In short: CoWoS is the 2.5D packaging technology that bonds HBM memory stacks to GPU dies via a silicon interposer — delivering the terabyte-per-second memory bandwidth that makes large-scale AI training possible, and whose constrained supply limits global GPU availability.