Disclosure
I work in sales at Edgecore Networks. This is my personal blog, and these are my views. [EDIT: confirm author wording and review status before publication]. No company endorsement, certification or deployment acceptance is claimed.
I use large language models (LLMs) to help draft and edit these posts. The thoughts, ideas and opinions are my own.
Contents
TL;DR
Before choosing switches, name the job each network does. NVIDIA’s terminology is a useful map: NVLink for scale-up, CIN for external compute communication, TAN for tenant access and SMN for infrastructure management. Storage and inference traffic need to be placed deliberately on that map.
That gives us two different conversations about Edgecore: building an independent Ethernet network around NVIDIA GPUs, and proposing specific Edgecore placements in an NCP deployment.
Start with the job, then the hardware
A diagram can make every line look equivalent. They aren’t. A model loading from storage, GPUs exchanging collectives and a management system reaching a BMC all need different things from the network.
NVIDIA’s NCP network table gives those roles names:
| Term | The practical meaning |
|---|---|
| NVLink scale-up | The local GPU communication domain. Its physical boundary depends on the platform. |
| CIN: Cluster Interconnect Network | External compute communication between GPU systems. |
| TAN: Tenant Access Network | Tenant access to storage and external services. Shared services can be logical overlays here. |
| SMN: Secure Management Network | Restricted access to infrastructure management interfaces. |
Those are responsibilities, not a universal instruction to buy four physically separate switch fabrics. A storage network can be converged or dedicated. Tenant isolation involves the switches, endpoints and control plane.
NVL72 is a platform, not an Ethernet topology
This distinction matters when somebody asks whether an open Ethernet switch can replace “the NVL72 network.”
The rack includes an internal NVLink/NVSwitch system. Edgecore Ethernet doesn’t replace that domain. An external scale-out fabric is a different design question. Frontend and infrastructure management connections are different again. NVIDIA’s component description is the place to establish the rack boundary.
An HGX B300 server and a GB300 NVL72 rack also need different attachment plans. We shouldn’t carry a server count, DPU link rate or management-port budget from one into the other without checking the selected profile.
Inference has more than one network conversation
A user request entering the service is one path. Model weights moving from storage are another. Disaggregated serving can move KV cache between prefill and decode workers, while model-parallel GPUs exchange their own communication.
The NVIDIA inference RA separates the serving architecture from the capabilities delivered by the provider. That’s a useful discipline: identify the flow, the endpoints and the required transport before drawing the fabric.
For a customer, I’d put each flow on the diagram and ask what happens when it competes with the others. Then measure the serving result. Time to first token and latency under concurrency tell us something a switch’s port count cannot.
Two Edgecore conversations
An independent open Ethernet deployment. Here, we can propose Edgecore for external compute communication as well as frontend, storage and management. The proposal still needs a selected NOS release, optics, NIC configuration and workload tests. Port speed alone doesn’t establish the implementation.
A deployment following NCP requirements. Here, the selected profile governs the design. The GB300 NVL72 provider requirements prescribe BlueField-3 north-south under NET-1 and InfiniBand or Spectrum-X east-west under NET-2. Proposed Edgecore placements elsewhere require their own design and acceptance review. Calling traffic “inference interconnect” doesn’t remove the requirement that applies to its actual role.
This is why I prefer a placement map to a blanket compatibility statement. It tells the customer which equipment we’re proposing, what stays in place and what needs to be tested.
What I’d want the diagram to tell me
For each connection, show the source and destination, physical interface, logical network and failure boundary. For each number, say whether it is a port-rate sum, an allocation or a measured result.
For each conclusion, distinguish a documented specification, a design assumption, a tested configuration and an accepted deployment. Those are useful answers to different questions.
[EDIT: add one customer-approved example once the exact platform and test profile are available]
FAQ
Does an Ethernet link establish NCP acceptance?
No. The complete applicable requirements and deployment conditions govern acceptance. NVIDIA’s v2.4 baseline extends well beyond the network hardware.
Is the tenant network another physical fabric?
It can be, but it can also be an overlay on TAN. State the isolation model and actual connections rather than assuming a separate tier of switches.
Where should I go next?
Read the open Ethernet fabric discussion for the independent design, or the frontend, storage and OOB discussion for the surrounding networks.