Published Last updated

AI cluster networking glossary

Plain definitions of AI cluster networking terms: scale-up vs scale-out, rail-optimized fabrics, NCP, Spectrum-X, RoCEv2, SONiC and more.

This glossary defines the terms I use on this blog for GPU cluster networks, in plain language and the way I use them in every post. Each term has its own heading, so you can link straight to it (for example, rail-optimized).

A note on naming: NVIDIA program and product names (NCP, Reference Architecture, SuperPOD, Spectrum-X) refer to specific NVIDIA things. When I describe a design that hasn’t been validated by NVIDIA, I call it a “pattern” or a “rail group,” never an RA.

Contents

Scale-up

GPUs sharing memory directly over NVLink and NVSwitch as “one big GPU,” inside a server (HGX = 8 GPUs) or a rack (NVL72 = 72 GPUs). It is not Ethernet.

The set of GPUs on one scale-up fabric. I say “the NVLink domain,” not “the rack network.”

Scale-out / backend / east-west compute fabric

The RDMA network joining GPUs across servers, usually one NIC per GPU into rail leaves. NVIDIA’s NCP documents call it the CIN.

CIN (Cluster Interconnect Network)

The NCP name for the back-end fabric. NCP documents list InfiniBand (Quantum) or Ethernet (Spectrum-X) for it.

Frontend / north-south (converged)

The network on the DPU or front-end NIC ports: storage, in-band management, tenant and internet traffic. It is “converged” when storage rides on it. The NCP name is TAN.

TAN (Tenant Access Network)

NCP’s name for the front-end network. Always Ethernet; NCP’s IaaS table lists “Ethernet (Spectrum).”

Tenant network

A customer’s isolated overlay (a VPC) riding on the front-end, usually EVPN-VXLAN.

OOB / management (SMN)

A separate 1G Ethernet network to BMCs, switch management ports, PDUs and CDUs. NCP calls it the SMN and doesn’t name a vendor.

In-band management

Provisioning, Kubernetes or Slurm control, and monitoring traffic carried over the front-end. Not the same thing as out-of-band management.

Storage fabric

The network to file and object storage. It may be separate or converged onto the front-end. NVIDIA’s storage reference designs default to Spectrum SN5600 switches in the bill of materials.

Border / edge / DCI

The uplink out of the cluster to the internet, the WAN or other sites (sometimes called “scale-across”).

ToR (top-of-rack)

A switch in the rack. On NVL72 the only ToRs are the two management switches; back-end leaves sit in separate network racks.

Leaf / spine / super-spine

The tiers of a Clos fabric. A super-spine joins multiple spine groups.

Rail-optimized

GPU n of every server connects to leaf n, so same-rail traffic is one hop. Rails cut across servers, not racks.

Plane (dual-plane)

Independent, parallel back-end fabrics. Each GPU’s NIC ports are split across the planes for bandwidth and failure isolation.

Oversubscription

Downlink bandwidth divided by uplink bandwidth on a switch. Back ends are usually 1:1 (non-blocking); front ends can be oversubscribed.

SU (Scalable Unit)

A repeatable scale-out building block. NVIDIA defines it differently across its reference architectures; the term is not exclusive to NVIDIA. For non-NVIDIA designs I say “pod” or “rail group” instead.

POD / BasePOD / SuperPOD

“POD” is generic in NCP documents. DGX BasePOD and DGX SuperPOD are NVIDIA products built from DGX systems with NVIDIA networking only; the names don’t apply to other designs.

RA (Reference Architecture)

A documented design for a particular use case and set of assumptions. NVIDIA publishes specific RAs with their own prescribed components. An Edgecore architecture proposal does not establish conformance to one of those NVIDIA designs.

Enterprise RA

NVIDIA’s on-premises designs (for example HGX AI Factory and NVL72 AI Factory) with partner endorsement, built on Spectrum-X.

NCP (NVIDIA Cloud Partner)

An NVIDIA partner program for GPU clouds. It is not a switch certification; there is no such thing as an “NCP-certified switch.”

NCP Hardware Reference Design

The gated (partner login) design that NCP builds are expected to follow closely.

Deployment-specific agreement / exception

The only documented way a baseline NCP requirement is waived. The agreement must identify the affected requirement and approved exception.

Spectrum-X

NVIDIA’s Ethernet AI platform: Spectrum switches and SuperNICs working together for adaptive routing and congestion control. It is more than “NVIDIA Ethernet switches.”

Spectrum (SN5600/SN5610/SN6600/SN2201)

NVIDIA switch families. The SN2201 is the 1G management switch.

InfiniBand (Quantum)

NVIDIA’s non-Ethernet back-end option.

NIC / SuperNIC (ConnectX-7/8/9)

The data mover, one per GPU on the back end, using GPUDirect RDMA straight into GPU memory. ConnectX-8 on HGX B300 runs 2x400GbE.

DPU (BlueField-3/4)

A NIC plus Arm cores running its own OS. Used as the front-end and storage card for overlays, isolation and security. It offloads the host CPU, not the GPU.

BlueField-3 SuperNIC

BlueField silicon sold as a back-end NIC for early Spectrum-X deployments; not the DPU role.

RoCEv2

RDMA over Converged Ethernet, version 2. The classic recipe runs it on a lossless, tuned fabric (PFC plus ECN/DCQCN), Operator-specific alternatives include PFC-only designs with workload coordination and distinct multipath transports such as MRC. Those are not interchangeable configuration recipes.

PFC / ECN / DCQCN

Priority Flow Control, Explicit Congestion Notification and the DCQCN congestion control algorithm: the flow control and congestion signaling that keep RoCE traffic from dropping.

Adaptive routing / DLB / packet spraying

Per-packet or per-flow path balancing. Spectrum-X does it across switch and NIC; Broadcom Tomahawk switches offer dynamic load balancing (DLB).

NCCL

NVIDIA’s collective communications library (all-reduce, all-to-all) that rides the back-end fabric. AMD’s equivalent is RCCL.

Tomahawk (TH5 51.2T, TH6 102.4T, TH Ultra)

Broadcom Ethernet switch ASICs. Tomahawk 5 is 51.2 Tb/s; Tomahawk 6 is 102.4 Tb/s.

AIS800-64O / -64D / -32O / -32D

Edgecore Tomahawk 5 AI switches: 64x800G OSFP or QSFP-DD in 2RU, and 32-port 1RU versions. (Disclosure: I work at Edgecore.)

Jericho3-AI

A Broadcom scheduled-fabric ASIC (cells and credits), an alternative to a Tomahawk Clos fabric.

SONiC / Enterprise SONiC

SONiC is an open source network operating system built on SAI. Enterprise SONiC is Broadcom’s commercially supported distribution.

SAI

The Switch Abstraction Interface: an open API between the NOS and the switch ASIC. It is why SONiC is portable across hardware.

ONIE

The Open Network Install Environment: a whitebox installer that lets you load any compatible NOS onto a switch.

EVPN-VXLAN

The standard multi-tenant overlay on a BGP underlay. NCP inference provider builds require BGP with EVPN.

UALink / UALoE / ESUN / SUE

Open scale-up efforts. UALoE is UALink over Ethernet (used in AMD Helios). None of them can be used as NVIDIA GPU scale-up.

Operator vs tenant

The operator owns the cluster and buys the switches; the tenant (for example an inference provider) rents managed capacity such as Kubernetes. I say “inference provider,” not “ISP.”

AI Cloud Ready / Exemplar Cloud

NVIDIA validation and performance badges for clouds. They are not NCP tiers.

Deployment pattern

What I call a design that hasn’t been tested or validated as an NVIDIA reference architecture. Placing open Ethernet next to NVIDIA systems is a pattern, not an RA.