Few questions in AI networking start more arguments than this one. Ask a room of engineers whether the GPU back-end should be InfiniBand or Ethernet with RoCEv2, and you’ll hear strong opinions on both sides, usually backed by a benchmark that proves exactly what its author hoped it would.

I’ve sat through a few of those arguments. This article takes the question apart calmly: what is actually the same, what is genuinely different, what each choice costs you, and how to decide from requirements rather than preferences. It’s written for network engineers, and for anyone preparing a design scenario such as the CCDE AI Infrastructure exam, where “InfiniBand or Ethernet?” is decided by the clues in the brief.

In short

  • Both run the same RDMA transport and the same software (verbs, NCCL). The difference is the network underneath.
  • InfiniBand was designed lossless from the first day: credit-based flow control, a central Subnet Manager, adaptive routing and in-network reduction (SHARP). It works very well out of the box, and in practice it comes from one vendor.
  • RoCEv2 runs RDMA over routed Ethernet, made lossless by configuration (PFC, ECN, DCQCN). It brings many vendors, standard tooling and one operating model with the rest of the data center, and it asks more of the design and operations team.
  • Today’s largest operators use both. The deciding factors are usually the team, the vendor policy, multi-tenancy and how the fabric must integrate with everything else, not raw performance.

The picture first: two railways

Think of two ways to run high-speed trains.

InfiniBand is a dedicated railway. It was built only for these trains. A central control room knows every track and sets every signal. A train never leaves a station unless the next station has confirmed it has room, so trains never pile up and nothing is ever lost.

RoCEv2 runs the same trains on the motorway network. The roads already exist everywhere, many companies build them, and every engineer knows how to drive on them. To keep the trains safe, you add dedicated lanes, warning signs and barriers. It works very well when the signs and barriers are set up consistently, and badly when they aren’t.

Same trains, same passengers. Two very different ways to keep them moving.

Same transport, two networks

Protocol stacks side by side: AI framework and NCCL, RDMA verbs and the InfiniBand transport are identical on both; below that line, InfiniBand uses its own network layer, a link layer with LIDs, virtual lanes and credit-based flow control, and InfiniBand physical links at NDR 400G or XDR 800G; RoCEv2 uses UDP/IP on port 4791, Ethernet with PFC and ECN, and Ethernet physical links at 400G or 800G

This is the most important fact in the whole comparison. RoCE stands for RDMA over Converged Ethernet, and RoCEv2 literally carries the InfiniBand transport header (the BTH) inside UDP/IP. Queue pairs, memory registration, RDMA writes and reads, sequence numbers and acknowledgements work the same way on both. NCCL, the library GPUs use to communicate, runs on either with no change to the training code.

So the comparison isn’t between two kinds of RDMA. It’s between two networks that carry the same RDMA: one designed for it, one adapted to it.

How InfiniBand works

A central brain: the Subnet Manager

InfiniBand: one Subnet Manager, running on a server as OpenSM or UFM, discovers the fabric, assigns addresses called LIDs and installs every route into the switches, which just forward. Ethernet: every switch runs BGP with its neighbors, with no single point of control

An InfiniBand fabric (a subnet) is managed by a Subnet Manager, software running on a server or a switch: the open-source OpenSM, or NVIDIA’s UFM. It:

  • discovers every switch, adapter and link;
  • assigns addresses: each adapter port gets a 16-bit Local Identifier (LID), so one subnet can address roughly 48,000 endpoints;
  • computes the routes with a routing engine chosen for the topology (fat-tree, up/down, minimum-hop and others) and installs forwarding tables into every switch;
  • reacts to failures by recomputing and reprogramming.

Switches themselves run no routing protocol. This makes InfiniBand remarkably simple to bring up: connect the cables, start the Subnet Manager, and the fabric configures itself. It also means the Subnet Manager is a critical component that must be redundant (a standby Subnet Manager takes over if the master fails).

Lossless by design: credits

InfiniBand credit-based flow control: the receiver first tells the sender how many buffers it has free, and the sender transmits at most that many packets, so overflow can’t happen. Ethernet with PFC: the sender transmits freely and the receiver sends a pause when its queue passes a threshold; packets already on the wire still arrive, so each port reserves headroom

On every InfiniBand link, the receiver tells the sender how much buffer space it has, in credits. The sender only transmits when it holds credits. A packet is never sent into a buffer that can’t take it, so congestion never causes a drop. Losslessness is a property of the link layer, not a configuration you have to get right.

Each link carries several virtual lanes (VLs) with their own buffers and credits, mapped from a packet’s service level. Traffic classes are separated the same way Ethernet priorities are, but with flow control built in for each.

Features built for collective communication

  • Adaptive routing. Switches can send packets of a flow over different paths based on current load, and the adapters put them back in order. This addresses the ECMP collision problem directly.
  • In-network reduction (SHARP). NVIDIA’s Quantum switches can compute reductions themselves: each GPU sends its data up a tree of switches once and receives the summed result once, instead of passing partial sums around a ring.

Without in-network reduction, GPUs pass partial sums around a ring and data crosses the network about twice. With SHARP, the switch computes the sum in hardware: each GPU sends its data once and receives the result once, with less traffic and lower latency

  • Partitions. A partition key (P_Key) in every packet isolates groups of endpoints, much like VLANs. The adapter enforces it at the receiving side.
  • Congestion control. The specification defines congestion notification, and current adapters implement their own algorithms on top.

Current generations

InfiniBand speeds are named per generation: HDR at 200 Gb/s, NDR at 400 Gb/s (NVIDIA Quantum-2 switches and ConnectX-7 adapters) and XDR at 800 Gb/s (Quantum-X800). In practice, InfiniBand switches today come from one vendor, NVIDIA, which acquired Mellanox in 2020.

How RoCEv2 works

RoCEv2 keeps everything above the transport and replaces the network underneath with standard routed Ethernet. The details are in the lossless RoCEv2 deep dive; in summary:

  • Routing is ordinary IP routing, usually eBGP in a leaf-spine fabric, often unnumbered. Every switch participates; there is no central controller to protect.
  • Lossless behavior is configured, not inherent: a dedicated traffic class with PFC as the safety net, ECN marking in the switches and DCQCN congestion control in the adapters. Every device on the path must agree on the settings.
  • Load balancing starts with ECMP hashing on the UDP source port, which struggles with the few, large flows of training (why). Modern AI Ethernet platforms from several vendors add adaptive routing and packet spraying, usually together with smarter adapters.
  • Isolation uses the standard data center toolbox: VRFs, VXLAN EVPN, ACLs.
  • Vendors: switch silicon and systems from Broadcom-based platforms (Arista, Cisco, Juniper, others), NVIDIA Spectrum, and more; adapters from several vendors.

Side by side

InfiniBand RoCEv2
Transport InfiniBand (verbs, queue pairs) The same, carried in UDP/IP
Lossless Built in: credit-based flow control Configured: PFC, ECN and DCQCN, end to end
Control plane Central Subnet Manager (OpenSM, UFM) Distributed routing (BGP), every switch
Addressing LIDs inside a subnet; routers between subnets IP everywhere, routable across the data center
Load balancing Adaptive routing, mature and integrated ECMP by default; adaptive routing and spraying on AI-focused platforms
In-network compute SHARP reductions in the switch No standard, widely deployed equivalent today
Isolation Partitions (P_Keys) VRFs, VXLAN EVPN, ACLs
Vendors In practice one: NVIDIA Many switch and adapter vendors
Tooling and skills Specialized tools (UFM, ibdiagnet); fewer engineers Standard Ethernet tooling, telemetry and automation; large talent pool
Integration A separate island; gateways to reach Ethernet Same technology as the front-end, storage and WAN
Out-of-the-box performance Excellent with little tuning Excellent when designed and tuned carefully

Strengths and weaknesses

InfiniBand

Strengths

  • Lossless and predictable by design. No thresholds to tune for the basic case; very consistent tail latency.
  • Fast to bring up. The Subnet Manager configures the fabric; validated designs exist for every cluster size.
  • Collective-aware features. Mature adaptive routing and SHARP reduce the work and latency of all-reduce.
  • Proven at the largest scales of AI and HPC, over many years.

Weaknesses

  • One vendor: pricing, lead times and roadmap depend on a single supplier.
  • A separate world: its own tools, skills and operations, and a gateway wherever it meets the Ethernet estate.
  • Smaller talent pool: fewer engineers have run it in production.
  • The Subnet Manager is critical: it must be redundant, monitored and protected.

RoCEv2

Strengths

  • Many vendors: competitive pricing, supply options and negotiating power.
  • One operating model with the rest of the data center: the same routing, automation, telemetry and security tools.
  • Large talent pool: every network team knows Ethernet and BGP.
  • Native multi-tenancy and integration: VRFs and EVPN, routable everywhere, no gateways.
  • The industry is investing here: adaptive routing, packet spraying, SuperNICs and Ultra Ethernet all build on Ethernet.

Weaknesses

  • Losslessness depends on configuration: one inconsistent switch or adapter quietly breaks it.
  • More design and tuning: ECN and PFC thresholds, headroom, classification, load balancing.
  • PFC side effects: head-of-line blocking and congestion spreading must be designed out and monitored.
  • No widely deployed in-network reduction comparable to SHARP today.

What the largest operators actually do

Real deployments show that both work at very large scale:

  • Meta described building two clusters of 24,576 GPUs each in 2024: one with a RoCE fabric built on Arista and Meta’s own OCP rack switches, and one with NVIDIA Quantum-2 InfiniBand. Through “careful co-design of the network, software, and model architectures”, Meta reported using both “for large, GenAI workloads (including our ongoing training of Llama 3 on our RoCE cluster) without any network bottlenecks”.
  • In a SIGCOMM 2024 paper, Meta explained that it chose RoCEv2 for the majority of its AI training capacity, and described the routing and congestion-control work needed to make it perform: low-entropy traffic and plain ECMP were among the main challenges.
  • Many AI clouds and research clusters run InfiniBand, often as part of NVIDIA’s reference architectures, because it delivers predictable performance quickly.

The lesson isn’t that one protocol wins. It’s that both reach excellent performance at scale, by different paths: InfiniBand by design, RoCEv2 by engineering.

Key point. The transport is the same. You’re choosing the network underneath, and the team that will run it.

How to choose

Decision flow: if the network must be shared with the rest of the estate, choose RoCEv2 for one operating model; if the team already runs InfiniBand well, InfiniBand is the lowest-risk choice; for several vendors or a multi-tenant cloud, RoCEv2; for one large single-vendor training cluster, either works, so decide on cost, lead time and the team you can hire

In a design review, and in a design exam, the answer is almost always in the requirements. Cues to look for:

Requirement cue Points toward Why
“Our team has years of InfiniBand experience” InfiniBand Lowest operational risk
“Strong Ethernet, BGP and EVPN skills, no InfiniBand” RoCEv2 The team can operate and troubleshoot it
“Ethernet preferred to align with the estate” or “multi-vendor policy” RoCEv2 Policy is a hard constraint
“Multi-tenant GPU cloud” RoCEv2 VRF and EVPN tenancy, routable everywhere
“Fastest path to a working cluster, one validated vendor stack” InfiniBand Validated design, little tuning
“Collective latency is critical; large all-reduce at scale” InfiniBand, or RoCEv2 with adaptive routing SHARP and mature adaptive routing help directly
“Lead time or budget pressure on one supplier” RoCEv2 Supply options

Two traps to avoid:

  1. Deciding on a benchmark. Most published comparisons measure one configuration, often tuned for one side. At job level, both perform within a small margin when designed well; the difference shows up in how much effort it took.
  2. Ignoring day two. Who will run the fabric at 3 a.m., with which tools, and who can you hire next year? Operations usually costs more than the switches.

What would change my mind

  • If Ethernet gains a widely deployed equivalent to in-network reduction, one of InfiniBand’s clearest technical advantages narrows further.
  • If Ultra Ethernet adapters and switches ship broadly, Ethernet fabrics will need far less PFC tuning, which removes RoCEv2’s main operational weakness.
  • If InfiniBand gained a broad multi-vendor ecosystem, its main business weakness would go away.

Until then: choose the network your team can run excellently, within the constraints the business has set. Both can deliver.

References

Chapter 12 of Building the AI Factory covers InfiniBand in depth, and chapters 9 to 11 cover RDMA, lossless RoCEv2 and load balancing.

This comparison reflects public documentation and published deployments at the time of writing. Product capabilities change quickly; validate against current vendor documentation before making a design decision.