Writing /

AI infrastructure

GPU fabrics for training and inference: RoCE and InfiniBand, congestion, sizing and reference designs.

Data center & enterprise networkingLow latency & tradingAutomation

Explainer

CPU, GPU and TPU: what each one does, and how they work together in an AI cluster

Why AI runs on GPUs and TPUs rather than CPUs, what is different inside each chip, their pros and cons, and how CPUs, accelerators, memory and the network cooperate in every training step.

November 18, 2026 AI infrastructureFundamentalsGPU

Explainer

AI factories, from the grid to the GPU: size, networks, power, cooling and money

A network architect's tour of the AI factory in 2026: how big the sites really are, the five networks inside one, how training and inference differ, why power and cooling now set the design, what a gigawatt costs, and what is shipping versus what is still a roadmap.

October 28, 2026 AI factoryData centerPower and coolingFabric design

Decision

InfiniBand or RoCEv2 for an AI factory: how each works, and how to choose

Both carry the same RDMA transport; the network underneath is what differs. How InfiniBand and RoCEv2 each deliver lossless, low-latency fabrics, their real strengths and weaknesses for AI training, and the requirement cues that should decide between them.

October 27, 2026 InfiniBandRoCEv2Fabric designCCDE

Reference design

Reference design: a 2,048-GPU training cluster, from brief to acceptance

A complete back-end, storage and front-end design for a 2,048-GPU pre-training cluster that must double without a rebuild. Nine decisions, each traced to a requirement, and the five-stage acceptance plan.

October 22, 2026 Fabric designRoCEv2Reference design

Explainer

Lossless RoCEv2, explained: ECN, DCQCN and PFC, from the packet to the counters

How RDMA runs on Ethernet without losing packets: what a RoCEv2 packet carries, how ECN, DCQCN and PFC work together, where the thresholds go, how much headroom a port needs, the side effects of PFC, and the counters that prove it works.

October 3, 2026 RoCEv2QoSFabric design

Coming soon

EssayComing 25 NovWhy averages lie: oversubscription in AI fabricsThe airline-overbooking bet behind every oversubscribed network, why it fails for GPU training traffic, and a five-step method for sizing the rest.
EssayComing 9 DecThe six places an AI fabric congests, and which fix works whereEvery congestion point is an egress queue. What decides the fix is whether the traffic had a choice of path: spread it better, or slow the senders down. A map, a walk through each point, and a triage flow.
EssayComing 23 DecOne link fails, every job slows: what ECMP hashing really doesPlain modulo hashing reshuffles almost every flow when one link fails. Why that hurts AI fabrics, and what resilient, symmetric and weighted hashing change.
EssayComing 6 JanWhere the GPU's own network ends: scale-up versus scale-outRack-scale systems turned NVLink into a network of its own. Why the size of that domain now shapes the design of the back-end fabric.
ExplainerComing 13 JanUltra Ethernet in plain EnglishWhat the Ultra Ethernet transport changes compared with RoCEv2, layer by layer: delivery modes, sender-driven spraying, selective retransmission and receiver credits, what it asks of your switches, and how to design a fabric today that can carry it tomorrow.
Research noteComing 20 JanResearch note: DCQCN, the paper behind RoCE congestion controlA plain-language reading of 'Congestion Control for Large-Scale RDMA Deployments' (SIGCOMM 2015): the problem it solved, how DCQCN works step by step, and the design rules it still sets for every RoCE fabric.

NVIDIA-Certified Professional: AI Networking

4 parts of the NCP-AIN study series published on this topic; the next part is out on 23 November.

  1. NCP-AIN Domain 1: AI workload traffic patterns, NCCL collectives and fabric design constraints
  2. NCP-AIN Domain 1: rail-optimized topologies and GPU-to-GPU paths
  3. NCP-AIN Spectrum domain: Spectrum-X architecture, and what the switch and the SuperNIC each do
  4. NCP-AIN Spectrum domain: RoCE on Spectrum-X, QoS, ECN and PFC end to end

CCDE Practical: AI Infrastructure

1 part of the CCDE AI study series published on this topic; the next part is out on 27 November.

  1. CCDE AI Infrastructure: how the module works, and a method for design scenarios