Why averages lie: oversubscription in AI fabrics

“The back-end averages 45% utilization. Why are we paying for a network that sits half idle?” If you design AI networks, you will hear some version of this. Here is why the answer is still no, and how to size everything else properly.

The airline bet

Oversubscription is the ratio of downlink to uplink capacity on a switch. A leaf with 48 × 100G down and 4 × 400G up is 3:1. It is the same bet an airline makes when it sells more tickets than seats: not everyone will show up at once.

That bet is safe when passengers are independent. It fails badly when they always travel together, like a team that either boards as a group or not at all.

Training traffic always travels together

During training, every GPU computes on its share of the data and then all GPUs exchange results at the same instant, every step. A collective finishes only when its slowest transfer finishes. So the back-end alternates between silence and every port at line rate together.

The 45% average blends the silence with the bursts. It tells you nothing about the moments that set job completion time. Synchronized traffic gets 1:1. Always.

Everything else gets a calculated ratio

For independent traffic (front-end requests, storage, management), a ratio is fine, but it should be an output, never an assumption:

1. Characterize the traffic: synchronized or independent, spreading or converging?
2. Find the peak from measurements, never the average.
3. Apply the failure case and headroom. With N spines, you only own N minus one.
4. Calculate the uplinks you need; the ratio falls out.
5. Check locality before buying bandwidth.

A worked example: peak 1.4 Tb/s per leaf, 20% headroom, survive one spine failure, one 400G uplink per spine.

Spines	Usable after one failure	Need 1.68 Tb/s
4	1.2 Tb/s	fails
5	1.6 Tb/s	fails
6	2.0 Tb/s	passes

Six spines, and a 2:1 ratio that came from the requirement rather than a rule of thumb.

Two traps to remember

• Ratios multiply along a path. 3:1 at the leaf and 2:1 to the border is 6:1 end to end.
• The same ratio behaves differently where traffic converges. A 4:1 leaf is comfortable when demand spreads across many leaves, and a bottleneck in front of storage arrays where it all converges.

Find the savings where traffic is independent or where a network isn’t needed at all. Never where traffic is synchronized.