The first time I read the CCDE AI Infrastructure technology list, I did what most CCIEs probably do: I scanned it for protocols. Lossless fabrics, RoCEv2, InfiniBand, PTP, NVMe-oF. Fine, I thought, I know where to read about those. Then I read Cisco’s exam format document properly, and one sentence changed how I study. It tells candidates not to spend time mapping requirements into configuration, and to stay inside the scenario instead.
That’s the whole difference between this exam and the ones most of us have passed. Nobody will ask how PFC is configured. They’ll ask whether this customer needs a lossless fabric at all, given what they told you three pages earlier, and why.
This is part 1 of my CCDE AI study series. I’m preparing the Practical with the AI Infrastructure area of expertise myself, so this series is partly my own notebook, cleaned up. It starts with the exam as Cisco describes it, then gives the method I use on every scenario, and works through a small example. Every later article follows the same pattern: the requirement, the options, the decision, and what would change it.
In short
- The AI module is one scenario in four. The Practical is 8 hours: three core enterprise scenarios, then your area of expertise. Each scenario has its own minimum score, so you can’t sacrifice one.
- The technology list is broad, not deep. It has four domains: AI/ML and governance, network, security, and hardware and environment. Business topics such as sovereignty, sustainability and cost sit right next to RoCEv2.
- Design means judgment. Cisco’s exam topics ask you to recommend, justify, validate, optimize and plan. The right answer is the one that meets the stated requirements, not the most advanced technology.
- Use a method, every time. Sort the requirements, find the one that decides, compare the options, decide and justify, then adapt when new information arrives. There’s no going back to change earlier answers.
The exam, as Cisco publishes it
Everything here comes from Cisco’s public exam format and exam topics documents. Check them again before you book, because Cisco updates them.
| What | What Cisco says |
|---|---|
| Length | 8 hours: four scenarios, at most 2 hours each. Unused time doesn’t carry over |
| Modules | Scenarios 1 to 3 are core enterprise. Scenario 4 is your area of expertise |
| Choosing AI Infrastructure | Cisco’s format document says you select it at the start of the exam day. In practice the registration now shows it: my own booking lists “CCDE Practical Lab (AI Infrastructure)”. Check what your confirmation says |
| Navigation | No going back: once you answer a question and move on, the answer is final |
| Materials | Closed book, but access to relevant documentation on cisco.com is provided |
| Scoring | Each scenario has a minimum score and a pass score. You must clear the aggregated pass score and every scenario’s minimum |
| Prerequisite | The CCDE Written (400-007) |
| Protocols | Designed with dual-stack in mind: expect IPv4 and IPv6 throughout |
The format document describes each scenario as a lot of background information and topology drawings, plus new information as you go: something that affects the existing network, or a new direction from management. It also suggests you take the role of the architect or consultant the scenario puts you in. I take that literally. I read every scenario as if a customer had hired me and would hold me to what I said.
Two consequences of the scoring are worth saying out loud:
- You can’t treat the AI module as a bonus. It has its own minimum score. A brilliant performance in the core scenarios doesn’t rescue a weak one here, and the reverse is true too.
- The core still applies in scenario 4. Cisco’s list says the AI technologies are in addition to the core technology list. An AI scenario can still ask about routing, segmentation or WAN integration.
What the AI technology list covers
Cisco publishes no weights for this list, so I don’t invent any. Here’s its shape, summarized:
| Domain | What it’s about |
|---|---|
| 1. AI/ML, compliance and governance | How training, fine-tuning and inference differ in what they demand; ML, deep learning, LLMs, SLMs and generative AI; service placement; data sovereignty; compliance and governance; moving workloads between sites; sustainability, cost and power; interoperability and lock-in |
| 2. Network | Lossless fabrics and their QoS; redundancy, resiliency and DR; Ethernet and InfiniBand; latency, bandwidth, single- versus multi-site, optics and cables; traditional and SDN fabrics; Layer 2 scale; routing optimizations; RDMA and RoCEv2; isolated islands versus the main data center |
| 3. Security | Hosting implications; air-gapped, public and private infrastructure; protection against malicious use; resource security across on-premises, cloud, clusters and containers |
| 4. Hardware and environment | Compute; GPUs, SmartNICs and DPUs; real-time operations and monitoring (telemetry, log correlation, data scrubbing); storage strategy and protocols; timing such as PTP |
A note on versions. If you search for this list, you may find an older, shorter copy that’s still online. The current list, the one Cisco links today, adds SLMs, workload mobility, interoperability and lock-in, redundancy and DR, direct-attach cables, and resource security. This series follows the current list. Check Cisco’s Learning Network before your exam in case it changes again.
Look at domain 1 again. Half of it isn’t technology at all: it’s sovereignty, governance, cost and power. That’s deliberate, and it’s where I’d expect network engineers like me to lose points. We tend to jump straight to the fabric.
What “design” means in this exam
Cisco’s unified exam topics list what the exam tests you on, and every item is a verb:
- recommend technologies or solutions that align with the stated requirements;
- justify why a decision was made;
- make design choices and fully design solutions;
- validate existing designs against the requirements, and suggest changes when they change;
- optimize existing designs to fix issues or mitigate risks;
- build high-level implementation plans;
- recommend, build or justify strategies.
None of these is “configure”. All of them are anchored to stated requirements. That’s the most useful thing I’ve taken from the documents: the scenario contains the answer’s justification. Your job is to find it, not to bring your favourite design from home.
A method for every scenario
1. Read the brief as a client brief. Who is the customer, what do they want to achieve, and by when? Note what already exists: the network, the team, the contracts. Existing things are constraints even when nobody calls them that.
2. Sort the requirements. Cisco’s exam topics use four kinds of constraint, and I sort everything into them:
| Kind | Examples in an AI scenario |
|---|---|
| Business | Budget, deadline, regulation, data sovereignty, sustainability targets, “no single-vendor lock-in” |
| Technical | Model size, GPU count, job completion time, latency targets, power and cooling limits |
| Operational | Team skills, existing tooling, support contracts, change windows, who runs it day to day |
| Application | Training or inference, batch or real time, how the framework talks (RDMA, NCCL), storage access pattern |
3. Find the requirement that decides. Most questions have one requirement that rules options in or out, and several that only shape the details. “Customer data must stay in-country and under our control” decides placement before you’ve thought about GPUs. Underline it.
4. List the options and their trade-offs. Usually two or three are realistic. For each one, ask what it costs in the four kinds of requirement above. The option that wins on technology often loses on operations.
5. Decide and justify against the deciding requirement. A good justification names the requirement: “Ethernet with RoCEv2, because the team runs Ethernet today and the scale doesn’t need more”. A weak one names the technology’s virtues: “Ethernet, because it’s open and scalable”.
6. Adapt when new information arrives, without rewriting history. Scenarios introduce changes as you go, and you can’t go back. So when the requirements change, the right answer is usually how to evolve the design you already chose, not a fresh design that pretends the earlier answers didn’t happen.
Working it through: a small scenario
Here’s a scenario I made up to practise the method. It isn’t from the exam, but it has the shape Cisco describes: a brief, a set of questions, and a change halfway through.
Northbank is a regional bank. It wants an internal assistant for its 2,000 staff, built by fine-tuning an open-weight language model of about 8 billion parameters on internal documents and past customer correspondence. The regulator requires customer data to stay in the country and under the bank’s control. Northbank has two data centers 40 km apart, both running a VXLAN EVPN spine-leaf fabric that its network team built and operates. There’s no HPC team. Data center 1 has about 80 kW of spare power, and its air-cooled racks are limited to 25 kW each. The budget covers four 8-GPU servers. The CIO adds: “Next year we’ll want to try much larger models.”
Question 1: where should the platform run?
Step 3 settles this quickly. The deciding requirement is the regulator’s: customer data in-country and under the bank’s control. Fine-tuning reads exactly that data, so the training platform runs on the bank’s premises.
What would change my mind? A cloud region in the country, with contractual and technical controls the regulator accepts, such as customer-managed encryption keys. The brief doesn’t offer one, so on-premises it is, in data center 1, where the power is.
Question 2: does it fit the room?
This is the kind of arithmetic the exam expects you to do quickly. An 8-GPU server draws around 10 kW at full load (NVIDIA lists 10.2 kW maximum for a DGX H100). Four servers need about 40 kW, which fits inside the 80 kW of spare capacity. But at 25 kW per rack, each rack holds two servers at most (20.4 kW at their maximum), so the platform needs at least two racks. Notice I planned at the maximum, not the typical draw. With a 20 kW limit, two servers wouldn’t fit, and the answer would have been four racks.
If the requirement had been “one rack”, the answer would have been liquid cooling or fewer, smaller servers, and either one raises cost and schedule questions you’d have to put back to the customer.
Question 3: does it need a dedicated GPU fabric?
This is where step 4 pays off, because the obvious answer isn’t automatically right.
Look at what one server can do first. Training with mixed precision and the Adam optimizer needs about 16 bytes of GPU memory per parameter for the weights, gradients and optimizer state. That figure comes from the ZeRO paper, and it doesn’t include activations. For 8 billion parameters, that’s about 128 GB. One server with eight 80 GB GPUs has 640 GB, so today’s fine-tuning fits inside one server. Its eight GPUs talk over NVLink, not the network.
Now apply the CIO’s remark. A 70-billion-parameter model needs about 1.1 TB by the same rule. That’s more than one server, so next year’s jobs will span servers. The moment training spans servers, gradient traffic crosses the network on every step, and it needs the kind of fabric this series is about.
So the options are:
| Option | For | Against |
|---|---|---|
| A. No GPU fabric now; each server works alone | Cheapest; fits today’s model | Fails the CIO’s stated direction; retrofitting a fabric later means recabling a live platform |
| B. Connect GPU NICs to the existing EVPN fabric | Reuses what the team knows | Mixes bursty training traffic with the bank’s production traffic; that fabric wasn’t built lossless or for this load |
| C. A small dedicated back-end fabric, Ethernet with RoCEv2 | Isolated from production; sized for growth; operated with the team’s existing Ethernet skills | Extra switches and cost now |
My answer is C. The justification names the requirements: the CIO’s growth direction rules out A, protecting the bank’s production network rules out B, and the team’s skills favour Ethernet over InfiniBand. At 32 GPUs, nothing in the brief needs what InfiniBand would add, and it would mean a second operational world for a team with no HPC experience.
In practice that’s two switches, each taking four of every server’s eight GPU NICs. That’s 16 ports per switch today, so a 64-port switch has room for the growth the CIO mentioned. Losing one switch halves the bandwidth but doesn’t cut a server off. The front-end NICs, the users and the storage stay on the fabric the team already runs.
What would change my mind? A requirement for very large training jobs soon, with an HPC team being hired. Then InfiniBand becomes a real contender, and that’s the subject of Part 5 of this series.
The change: new information arrives
Six months in, Northbank’s parent group announces a partnership with a public cloud provider and asks all subsidiaries to use its GPU service for peak demand.
This is step 6. The earlier answers stand; the question is how the design evolves. The deciding requirement hasn’t changed: customer data stays in-country and under the bank’s control. So I’d split the workloads by the data they touch:
- Fine-tuning on customer data stays on-premises. That’s the regulator’s requirement, and the parent group’s policy doesn’t override it.
- Inference peaks can go to the cloud only if the regulator’s conditions hold there: an in-country region, and no customer data leaving the bank’s control. That means checking what reaches the cloud model in prompts and context, not only what’s stored.
- If those conditions can’t be met, peaks stay on-premises, and the answer to the parent group is a capacity plan, not a cloud migration.
Notice what didn’t happen: I didn’t redesign the fabric, move the platform or reopen Question 1. A good answer to a change keeps the design coherent with everything already decided.
Traps I’m watching for
These are from my own preparation, not from the exam:
- Answering the technology question instead of the business one. If sovereignty decides placement, the elegance of the fabric doesn’t matter yet.
- Over-engineering. A 32-GPU platform isn’t a 2,048-GPU reference design. The right size is part of the right answer.
- Ignoring operations. “Who will run this at 3 a.m.?” eliminates more options than any feature comparison.
- Forgetting the core. An AI scenario can still ask about segmentation, routing or WAN, in IPv4 and IPv6.
- Rewriting history. When requirements change, evolve the design you have.
How the series is organized
The next articles follow Cisco’s list domain by domain: use cases and placement, sustainability and cost, then the network, security, and hardware and environment. Each article ends with a mini-scenario like Northbank. Two full practice scenarios and an exam-day checklist close the series. The series page shows the roadmap and what’s published.
Several articles already on this site cover parts of the list, and the series links to them rather than repeating them:
- InfiniBand or RoCEv2
- Lossless RoCEv2, explained
- Why averages lie: oversubscription
- Where the GPU’s own network ends
- AI factories, from the grid to the GPU
Self-check
1. You score well on the three core scenarios and poorly on the AI scenario, but your total is above the aggregated pass score. Do you pass?
Not necessarily. Cisco’s format document says each scenario also has a minimum score, and you must clear it in every scenario as well as the aggregated pass score.
2. Midway through a scenario, you realise an earlier answer was weak. What can you do?
Nothing about the earlier answer: you can’t navigate backwards. What you can do is make later answers coherent and strong, adapting the design from where it stands.
3. A brief says customer data must stay in-country, and the team has never run InfiniBand. Which of these decides where the platform runs, and which shapes the fabric?
Sovereignty decides placement: it’s a business requirement that rules options in or out before technology matters. Team skills are an operational requirement that shapes the fabric choice, favouring the technology the team can run.
4. Why did one 8-GPU server make the Northbank fine-tuning job a non-network problem today, and what changed that?
At about 16 bytes per parameter, an 8-billion-parameter model needs about 128 GB, which fits in one server’s 640 GB of GPU memory, so the GPUs talk over NVLink. The CIO’s plan for larger models (about 1.1 TB for 70 billion parameters) means jobs will span servers, which puts gradient traffic on the network.
5. Rewrite this justification so it would score: "Ethernet, because it's open and scalable."
Something like: “Ethernet with RoCEv2, because the operations team runs Ethernet today, the platform is 32 GPUs, and no requirement needs what InfiniBand adds.” It names the requirements, not the technology’s virtues.
What’s next
Part 2 starts the AI/ML domain: training, fine-tuning and inference, and what ML, deep learning, LLMs and SLMs each ask of compute, network and storage.
References
- Cisco, CCDE v3.1 Written and Practical Exam Format: four 2-hour scenarios, one area of expertise, no backward navigation, cisco.com documentation, minimum and pass scores, the advice to stay in the scenario rather than map to configuration.
- Cisco, CCDE Written (400-007) and Practical Exam v3.1 Unified Exam Topics: the design verbs, the constraint types, dual-stack, the AI items in the core topics.
- Cisco, CCDE Practical v3.1 AI Infrastructure Technology List: the current list this series follows.
- Rajbhandari et al., ZeRO: Memory Optimizations Toward Training Trillion Parameter Models, 2020: 16 bytes per parameter for mixed-precision training with Adam.
- NVIDIA, DGX H100 user guide: 8 GPUs, 10.2 kW maximum.
- Related on this site: a 2,048-GPU reference design and the NCP-AIN series, which covers the operational side of the same fabrics.
This guide follows Cisco’s public exam documents; it contains no exam content, and the scenario is my own invention. Cisco, CCDE and CCIE are trademarks of Cisco Systems, Inc.; NVIDIA and DGX are trademarks of NVIDIA Corporation.