Reference
Glossary
Every technology the articles mention, in plain words, with the standard or vendor document that defines it. In an article, click any dotted-underlined term to see its definition.
8
800 V DC
Distributing power as 800 V direct current from the facility to the rack instead of AC plus 54 V busbars: less current, less copper, fewer conversion steps. NVIDIA plans it for 1 MW racks from 2027.
A
Adaptive routing
Switches choose a packet's next hop by current load rather than a fixed hash. Used in InfiniBand and in AI Ethernet fabrics such as Spectrum-X.
ADD-PATH
A BGP extension that lets a router advertise several paths for the same prefix, each with a path identifier, instead of only its best one.
AI factory
A data centre built to run AI training and inference as one large machine, judged on tokens or trained models produced per watt and per dollar. NVIDIA's name for it.
Aliasing
Remote leaves load-balance to every leaf of an Ethernet segment, even to a leaf that never learned the host's MAC, using the per-EVI type-1 routes.
All-reduce
Every GPU contributes a buffer and every GPU receives the combined result (for example the sum of gradients). The main collective of data-parallel training.
Anycast gateway
Every leaf answers for the subnet's default gateway with the same IP and MAC, so hosts route at their first hop wherever they are, and keep the gateway when they move. Cumulus calls it VRR.
Anycast RP
Several routers share one RP address; each source and receiver uses the nearest, and the RPs share source information with MSDP or PIM so an RP failure goes unnoticed.
ARP suppression
A leaf answers ARP requests itself from the IP-to-MAC bindings it learned through EVPN, instead of flooding them across the fabric. It cannot answer for a host it has never learned.
ASM
Any-Source Multicast: receivers ask for a group without naming its sources, so the network needs an RP to connect them.
ASN
Autonomous System Number: the identifier of a BGP domain. Fabrics use private ranges (64512-65534, or 4200000000-4294967294 for 4-byte ASNs).
B
Back-end network
The scale-out fabric that connects every GPU's own NIC (400G or 800G) between racks and carries the training collectives. Usually RoCEv2 or InfiniBand, built 1:1.
Best-external
A backup border keeps advertising its best eBGP path to iBGP peers even while it prefers a path learned over iBGP, so the rest of the network knows the backup exit before the primary fails.
BFD
Bidirectional Forwarding Detection: a light hello protocol that detects a dead path in milliseconds and tells BGP, instead of waiting for BGP's own hold timer.
BGP PIC
Prefix Independent Convergence: a router installs a backup path in its forwarding table next to the best one, so when the best next hop disappears it switches once for every prefix behind it, without waiting for BGP.
BGP unnumbered
BGP sessions over IPv6 link-local addresses, discovered with router advertisements, carrying IPv4 routes with an IPv6 next hop. No IPv4 address on any fabric link.
BIDIR
Bidirectional PIM: one (*,G) tree per group carries traffic both up towards the RP and down to receivers, with no per-source state. A designated forwarder per link keeps it loop-free.
BMC
Baseboard Management Controller: a small computer on the server board for remote power, console and sensors, reached over the out-of-band management network.
Bus bandwidth
The nccl-tests figure that makes collective results comparable with link speed: algorithm bandwidth (size / time) times an operation factor, 2(n-1)/n for all-reduce.
C
CDU
Coolant Distribution Unit: pumps the rack-side coolant loop and exchanges its heat with the facility water, keeping the two loops separate. Sits in the rack or in the row.
Checkpoint
A saved copy of the model state during training, so a job can restart from it after a failure. Writing one is a large burst of storage traffic.
Clos
The leaf-spine topology: every leaf connects to every spine, so any two servers are the same number of hops apart and capacity grows by adding spines.
Cluster ID
The identifier of a route-reflection cluster, by default the RR's router ID. Each RR adds it to a route's CLUSTER_LIST when reflecting, and drops routes whose CLUSTER_LIST already holds it.
CNP
Congestion Notification Packet: sent by a RoCEv2 receiver back to the sender when it gets ECN-marked packets; the sender's NIC reduces its rate.
Collective
An operation where many GPUs exchange data together, such as all-reduce, all-gather or all-to-all. Training waits for the slowest participant.
CPO
Co-packaged optics: the optical engines sit on the switch package instead of in pluggable modules, cutting power per port (NVIDIA claims about 9 W against 30 W) at the cost of field replaceability. Lasers stay in separate replaceable units.
Credit-based flow control
The receiver tells the sender how much buffer it has (credits); the sender transmits only that much. Lossless by design, as in InfiniBand. Ultra Ethernet defines an optional Ethernet version (CBFC).
D
DCQCN
The congestion control of RoCEv2 NICs: switches mark with ECN, the receiving NIC returns a CNP, the sending NIC cuts its rate and then recovers. Implemented in NIC hardware.
DCTCP
Data Center TCP: TCP that reduces its window in proportion to the fraction of ECN-marked packets, keeping switch queues short.
Designated forwarder
The one leaf of an Ethernet segment elected to deliver flooded traffic to the multihomed device, so it receives one copy, not one per leaf.
Direct-to-chip cooling
Liquid flows through cold plates mounted on the GPUs and CPUs, taking most of the heat away; fans cool what is left. The standard for 100 kW+ GPU racks.
DPU
Data Processing Unit: a NIC with its own processors that offloads networking, storage and security from the host (NVIDIA BlueField). A SuperNIC is a NIC built for the AI back-end network.
DSCP
Differentiated Services Code Point: six bits in the IP header that classify traffic. Lossless RoCE is usually mapped to a priority by DSCP (NVIDIA's default is 26).
E
eBGP
External BGP runs between different autonomous systems; internal BGP (iBGP) inside one. Data center fabrics usually run eBGP between every leaf and spine.
ECMP
Equal-Cost Multi-Path: a switch spreads flows over several equal next hops, usually by hashing packet headers, so each flow stays on one path.
ECN
Explicit Congestion Notification: a switch marks packets (two bits in the IP header) instead of dropping them when a queue grows, so senders slow down before buffers overflow.
Ethernet segment
The set of links connecting one device to several leaves. Its 10-byte identifier, the ESI, is shared by every leaf on the segment; that is how EVPN multihoming replaces MLAG.
All-active multihoming uses type-1 routes (mass withdrawal and aliasing) and type-4 routes (DF election). Split horizon over VXLAN uses local bias.
EVPN
Ethernet VPN: a BGP address family that advertises where MAC addresses, IP addresses and subnets live, so switches learn them from BGP instead of by flooding.
Used as the control plane for VXLAN fabrics. Its route types: type-1 (Ethernet auto-discovery, for multihoming), type-2 (MAC/IP), type-3 (flood list), type-4 (Ethernet segment), type-5 (IP prefix).
EVPN route types
Type-1 Ethernet auto-discovery (multihoming), type-2 MAC/IP advertisement, type-3 inclusive multicast (the flood list), type-4 Ethernet segment (DF election), type-5 IP prefix.
F
Front-end network
The network joining GPU servers to users, APIs, orchestration and the rest of the estate. Looks like a normal data-centre network, but inference and data loading can push it hard.
Full bisection bandwidth
Any half of the hosts can send to the other half at full line rate, as far as capacity goes: a 1:1 fabric. With per-flow ECMP, large flows can still collide on one link, so it is a capacity property, not a guarantee of no blocking.
G
Glean
When a router has a route to a subnet but no ARP entry for the host, it resolves ARP first and typically drops the packet that triggered it. The first packet to a silent host is lost.
GPU
Graphics Processing Unit: thousands of simple cores running the same operation over large data in parallel; the engine of AI training.
GPUDirect RDMA
The NIC reads and writes GPU memory directly over PCIe, without staging data in CPU memory.
GPUDirect Storage
Moves data between storage and GPU memory directly, through the NIC or NVMe, without a copy through CPU memory.
H
HBM
High Bandwidth Memory: stacked memory beside a GPU or TPU, delivering terabytes per second.
Head-of-line blocking
Traffic stuck behind a blocked packet or a paused queue, although its own path is free. PFC causes it because it pauses a whole priority, not one flow.
Headroom
Buffer a switch keeps per lossless port and priority to absorb the data still in flight after it sends a PFC pause. Depends on link speed, cable length and MTU.
Hold timer
How long a BGP speaker waits without hearing from a peer before declaring the session down and removing its routes. 180 seconds by default on IOS-XE and IOS-XR, with keepalives every 60.
Hot-potato routing
Hand traffic to another network at the nearest exit: with otherwise equal routes, BGP prefers the lowest IGP cost to the next hop.
I
iBGP full mesh
Every iBGP speaker in an AS peers with every other, because a router does not pass iBGP-learned routes to other iBGP peers. n routers need n(n-1)/2 sessions.
IGMP
Internet Group Management Protocol: how hosts tell the router on their LAN which multicast groups (and, in IGMPv3, which sources) they want. A querier asks periodically; silence means nobody is left.
IGMP snooping
A switch listens to IGMP and forwards each multicast group only to the ports that asked for it, plus multicast router ports. Memberships expire unless something keeps querying.
Immersion cooling
Whole servers sit in a non-conductive liquid that absorbs the heat. Works, but needs purpose-built servers and handling; direct-to-chip is more common for GPU racks.
Incast
Many senders transmitting to one receiver at the same moment, overflowing the receiver's switch port. Typical of collective operations and storage reads.
InfiniBand
A lossless network designed for RDMA: credit-based flow control at the link layer, and a central Subnet Manager that assigns addresses and computes routes.
Ingress replication
Flooded traffic (broadcast, unknown unicast, multicast) is copied by the sending VTEP, one unicast VXLAN copy per remote VTEP in the segment, instead of using IP multicast in the underlay.
IS-IS
Intermediate System to Intermediate System: a link-state IGP. Every router floods its links and computes shortest paths to every other; common in large enterprise and service-provider cores.
K
KV cache
The keys and values a model has computed for every token of a conversation so far: its working memory during inference. It grows with context length and can be moved between GPUs or offloaded to CPU memory and storage.
L
LACP
Link Aggregation Control Protocol: negotiates which links form one bond (802.3ad). A link is removed after three missed messages: 3 s at the fast rate, 90 s at the slow rate. Defined in IEEE 802.1AX.
LID
Local Identifier: the 16-bit address the Subnet Manager gives each InfiniBand port; switches forward on it.
LLR
Link Level Retry: the link retransmits a corrupted frame itself, hop by hop, so the transport rarely sees the loss. Optional in Ultra Ethernet.
Local bias
Split horizon for EVPN over VXLAN: the leaf where flooded traffic enters delivers it to its own segments; peers drop copies that came from a leaf sharing the same segment.
M
MAC Mobility
An EVPN extended community with a sequence number. When a host moves, its new leaf advertises a higher number, and every switch prefers the newest location.
The same mechanism detects duplicates: by default RFC 7432 flags a MAC that moves 5 times in 180 seconds.
MLAG
Multi-chassis link aggregation: two switches present themselves as one LACP partner, synchronised over a peer link with a vendor-specific protocol. EVPN multihoming is the standards-based alternative.
MSDP
Multicast Source Discovery Protocol: RPs tell each other about active sources over TCP. Used between PIM domains and for anycast RP.
MTU
Maximum Transmission Unit: the largest packet a link carries. VXLAN adds 50 bytes, so a 1500-byte host packet needs 1550 on every underlay link. Path MTU discovery (PMTUD) relies on ICMP "fragmentation needed" messages.
multipath-relax
A BGP option that lets paths with different AS paths of the same length share traffic (ECMP). Without it, BGP only balances over paths with identical AS paths.
N
NCCL
The NVIDIA Collective Communications Library: implements collectives over NVLink, PCIe, InfiniBand and RoCE, choosing algorithms such as ring and tree.
NSCC
Network-Signal-based Congestion Control: Ultra Ethernet's mandatory sender-side congestion control, driven by ECN and delay. RCCC is an optional receiver-credit-based scheme.
NVLink
NVIDIA's GPU-to-GPU interconnect inside a server or rack (the scale-up network), much faster than the network NICs. NVSwitch connects many GPUs at full NVLink speed.
O
OCS
Optical circuit switch: mirrors redirect light between fibres to rewire connections without converting to electrical signals. Google uses them to reconfigure TPU pods.
Optimal Route Reflection
A route reflector computes each client's best path from that client's position in the IGP topology, not its own, so clients keep their nearest exit. Needs the link-state database on the RR.
ORIGINATOR_ID
Set by the first route reflector to the router ID of the route's source in the AS. A router that receives a route carrying its own router ID as ORIGINATOR_ID discards it.
Oversubscription
The ratio between bandwidth that can enter a switch tier and bandwidth that can leave it. 1:1 gives full bisection bandwidth, though per-flow hashing can still put two flows on one link, so it is not strictly non-blocking. GPU back-end fabrics are usually built 1:1.
P
Packet spraying
Sending the packets of one flow over many paths instead of one, for even load. It needs a receiver that accepts packets out of order.
Packet trimming
A congested switch cuts a packet down to its header and forwards that instead of dropping it silently, so the receiver learns at once what to request again.
Parallel file system
A file system that spreads files over many storage servers so thousands of clients can read and write at once at very high total bandwidth. Lustre and GPFS are examples.
Path hunting
After a failure, BGP routers try and withdraw successive alternative paths before converging, multiplying updates. One shared spine ASN avoids it.
PFC
Priority Flow Control (IEEE 802.1Qbb): a switch tells its neighbour to pause one traffic priority when its buffer fills, so that priority is not dropped. It can spread congestion backwards (pause storms, head-of-line blocking).
PFC storm
A broken endpoint, such as a hung host or a NIC whose receive ring stopped draining, sends pause frames without end and freezes a priority on its link and beyond. Different from congestion spreading, which is PFC working as designed. The PFC watchdog exists to break it.
PIM
Protocol Independent Multicast: routers build multicast distribution trees by sending joins towards the source or the RP, using the unicast routing table for direction (RPF).
Prefill and decode
The two phases of serving a model: prefill reads the whole prompt (compute-heavy), decode produces the answer one token at a time (memory-bound). Disaggregated inference runs them on separate GPUs.
PUE
Power Usage Effectiveness: total facility power divided by IT power. 1.0 would mean no overhead at all; large modern sites run at about 1.1 to 1.2.
PXN
PCI x NVLink: NCCL moves data over NVLink to the GPU whose NIC is on the destination's rail, then sends it from there, keeping inter-node traffic rail-local. Introduced in NCCL 2.12.
Q
Queue pair
The RDMA connection endpoint: a send queue and a receive queue on the NIC. Applications post work to them; the NIC does the transfer.
R
Rail-optimized
A GPU cluster design where GPU n of every server connects to leaf (rail) n, so collective traffic between same-numbered GPUs stays one hop away.
RDMA
Remote Direct Memory Access: a NIC reads and writes another machine's memory directly, without the operating system on the data path. The basis of GPU-to-GPU networking.
RoCEv2
RDMA over Converged Ethernet version 2: the InfiniBand transport carried in UDP (port 4791) over routed Ethernet. It performs best on a network that avoids drops.
Route distinguisher
An 8-byte prefix that makes routes from different VRFs or switches unique in BGP. It identifies a route; it does not decide who imports it (the route target does).
Route reflector
An iBGP router that passes routes between its clients so they do not need a full mesh. It advertises only its best path per prefix unless ADD-PATH is used.
Route target
A BGP extended community that decides which switches import a route into which VRF or VNI. Both sides must agree on the value.
Automatic route targets are derived from the local ASN and the VNI. With one ASN per leaf they differ on every leaf: FRR and Cumulus import them as a wildcard (*:VNI), other implementations need an exact match.
Router MAC
An EVPN extended community carrying the MAC address a leaf uses for routed VXLAN traffic. Remote leaves use it as the inner destination MAC with symmetric IRB.
RP
Rendezvous point: in ASM, the router where receivers and sources meet. Receivers join a shared tree towards it; sources register with it.
RPF
Reverse path forwarding: a router accepts a multicast packet only on the interface it would use to reach the packet's source, which prevents loops. Joins are sent the same way, towards the source.
S
Scalable unit
A repeatable building block of a GPU cluster: a fixed number of servers with their leaves, deployed and tested as a unit (NVIDIA reference architectures use 32 servers).
Scale-up
Scale-up: GPUs joined by NVLink inside a server or rack, behaving like one large GPU. Scale-out: many servers joined by the back-end network.
SHARP
Scalable Hierarchical Aggregation and Reduction Protocol: InfiniBand switches perform part of a reduction themselves, halving the data GPUs must send.
SPT
Shortest-path tree: the (S,G) tree from a source to its receivers along the unicast shortest path. In ASM, last-hop routers normally switch to it from the shared tree once they learn the source.
SR-MPLS
Segment Routing over MPLS: the IGP advertises a label (a prefix SID) for each router's loopback, so traffic can be label-switched to any router without LDP. Lets the core forward without BGP routes.
SSM
Source-Specific Multicast: receivers ask for (source, group), so the tree goes straight to the source with no RP. 232.0.0.0/8 is reserved for it; hosts use IGMPv3.
STP edge port
A switch port marked as connecting to a host forwards immediately instead of going through spanning-tree learning. BPDU guard shuts it down if a switch appears on it.
Straggler
The slowest participant in a synchronous step. Every other GPU waits for it, so with N ranks the step time follows roughly the (1 - 1/N) quantile of per-rank latency.
Subnet Manager
The InfiniBand control plane: discovers the fabric, assigns LIDs and programs every switch's forwarding tables (OpenSM, or NVIDIA UFM).
Symmetric IRB
Integrated Routing and Bridging where both the ingress and egress leaf route: the packet crosses the fabric in the tenant L3 VNI, from one router MAC to the other.
T
TCAM
Ternary content-addressable memory: the fast lookup memory in a switch ASIC used for ACLs, QoS and features such as ARP suppression. It is finite and often has to be carved between features.
TPU
Google's Tensor Processing Unit: built around matrix-multiply units (MXUs), systolic arrays of 128x128, or 256x256 from v6e. TPU pods connect chips directly in a torus.
U
UALink
Ultra Accelerator Link: an open scale-up interconnect for joining accelerators in a pod, an alternative to NVLink. AMD's Helios first carries it over Ethernet.
Ultra Ethernet
An Ethernet transport and set of link features for AI and HPC from the Ultra Ethernet Consortium: multipath packet spraying, out-of-order delivery, and new congestion control (NSCC, optional RCCC).
V
VNI
VXLAN Network Identifier: the 24-bit segment number in the VXLAN header. An L2 VNI carries one VLAN; an L3 VNI carries routed traffic for one tenant VRF.
With symmetric IRB, every leaf of a tenant shares one L3 VNI. Switches attach it to a local "transit" VLAN that has no ports; only the VNI travels on the wire.
VRF
Virtual Routing and Forwarding: a separate routing table on a switch, one per tenant, so tenants can reuse addresses without seeing each other.
VTEP
VXLAN Tunnel End Point: the switch (or host) that puts frames into VXLAN and takes them out. Usually identified by a loopback address on each leaf.
VXLAN
Virtual eXtensible LAN: carries an Ethernet frame inside a UDP packet (port 4789) between two switches, adding 50 bytes, so a Layer 2 segment can cross a routed network.
The header carries a 24-bit VNI, so up to about 16 million segments instead of 4094 VLANs. The outer IP header goes from one VTEP to another; the network in between only routes IP.
W
WRED
(Weighted) Random Early Detection: a queue starts marking (with ECN) or dropping packets with rising probability between a minimum and a maximum threshold.