Modular Data Center for AI: NVL72 Density, Speed, and the Retrofit Trap

A modular data center for AI is a factory-built facility engineered for the selected GPU equipment’s power, cooling and structural requirements. An NVIDIA GB200 NVL72 rack draws roughly 120 kW, with deployed systems reported at 130–132 kW under full load (NVIDIA / HPE, 2025); its supported rack design requires direct liquid cooling. An existing hall designed for much lower rack loads may need new distribution, cooling interfaces and structural work before it can accept that system. Purpose-built modular design addresses those interfaces from the start. A dense air-cooled inference deployment is a different brief: for example, ModulEdge’s MDC500 air-cooled reference supports up to 50 kW per rack with rear-door heat exchangers.

This post covers the rack-density explosion, why legacy retrofits fail the math, how modular avoids the problem, the difference between inference and training infrastructure, and what idle GPUs cost while you wait.

Why has AI broken the rack-power assumption?

Because a single rack now draws what a room used to. That one fact invalidates most of the installed base.

The NVIDIA GB200 NVL72 packs 72 Blackwell GPUs and 36 Grace CPUs into one rack-scale system and runs at about 120 kW nominal, with real deployments at 130–132 kW under full load (NVIDIA / HPE, 2025). For context, a single H100 draws around 700 W, and an 8–16 GPU training rack reaches 40–80 kW (Global Data Center Hub, 2026). Legacy colocation from the 2000s was built for 3–5 kW per rack; cloud-era hyperscale for 8–15 kW.

Existing room airflow and floor-distribution systems can limit rack density, while supported air-cooled servers and rear-door heat exchangers can extend the feasible range. The installed server and facility interfaces decide the limit. A 30–40 kW rule does not select every cooling system; NVL72’s direct-liquid requirement follows its supported rack design.

NVIDIA doesn't leave it ambiguous, the NVL72 mandates direct-to-chip liquid cooling. Flow is strongly temperature-dependent: Lenovo publishes 59 L/min per rack at 25 °C rising to 177 L/min at 45 °C, with pressure drop going 2.3 to 18.4 psi across the same range (Lenovo LP2357). The full GB300 NVL72 facility spec has the whole curve. The full thermal breakdown, from rear-door exchangers to immersion, is in liquid cooling for data centers, and what Blackwell specifically demands of a facility is in Nvidia Blackwell explained.

Why can't you just retrofit a legacy data center?

Because three independent variables each disqualify most existing buildings, and together they make the ROI collapse.

Power density is the first. AI-ready racks need 30–100 kW-plus, a 6× to 20× jump over legacy 3–15 kW designs (Introl, 2025). Floor loading is the second, a fully equipped liquid-cooled rack can be 3–5× heavier than its air-cooled predecessor, while older raised floors are often rated at only 1,000–1,500 kg/m² (Introl, 2025). Liquid-cooling plumbing is the third: the pipes, manifolds, leak detection, and structural reinforcement simply don't exist in air-cooled facilities, and adding them "amounts to reconstruction, not incremental retrofit" (Global Data Center Hub, 2026).

Global Data Center Hub documented a four-year-old, OCP-grade facility running an efficient PUE of 1.15, designed for 8–12 kW per rack, that was "structurally disqualified" when a customer asked for 40–50 kW. The retrofit ran to nine figures and the ROI didn't work (Global Data Center Hub, 2026). Their diagnostic is blunt: a credible AI facility delivers 40 kW-plus per rack without new capex, while upgraded cloud-era facilities reach only 20–25 kW. The gap is structural. Plenty of "AI-ready" marketing sits on top of legacy engineering, a stranded-asset risk dressed up as an opportunity.

How does modular avoid the retrofit problem?

By building for the workload from inception instead of apologizing to it afterward.

Purpose-built modular AI infrastructure can integrate the selected power, cooling and structural design in the factory. Air-cooled servers and liquid-cooled rack-scale systems need different facility interfaces. ModulEdge selects among those architectures for the actual rack schedule; a hardware change still requires electrical, thermal and structural review.

Hyperscalers validate the direction. Microsoft, Google, and Amazon are all expanding modular programs, and vendors are shipping prefabricated AI modules claiming up to 60% deployment-time reduction (Introl, 2025; Delta, 2026). If you already have the silicon and are staring at the "now what," that operational gap is exactly what the GPU data center problem nobody budgets for unpacks.

Inference or training, which is this actually for?

Both exist, but they want different infrastructure, and being honest about the difference is where most vendors get sloppy.

Sources: EdgeCore, 2025; Introl, 2025; Dell'Oro, 2026.

Training tolerates milliseconds of external latency, so it chases the cheapest power and land. Think gigawatt-scale campuses. Inference sits in the live user path, so it runs close to users across many regional sites (EdgeCore, 2025). Dell'Oro notes that while much AI capex has funded training, "inferencing will likely become a larger capex driver going forward" as reasoning models spread (Dell'Oro, 2026).

Where ModulEdge fits: distributed, latency-sensitive inference and GPU colocation at 40 kW per rack and above, plus enterprise and private AI. Frontier training at gigawatt scale is industry context, not the pitch, and inference is where the market is heading anyway, so it's the stronger story. The chip-level version of this argument is in GPU vs LPU vs NPU, and the rack-power translation for inference specifically is in edge AI infrastructure for inference.

What do idle GPUs cost while a site gets built?

More than most capex models admit. This is the number that makes time-to-power a financial argument, not a convenience.

Even strong in-house teams need 8–12 months to reach production readiness, during which the GPUs are often already installed and depreciating (Introl, 2025). Deployment delays typically cost 20–30% of cluster capex per half-year; a six-month delay on a 512-H100 cluster represents over $9 million in lost value and unrealized revenue (Rafay, 2026). On a five-year straight-line schedule, six months idle burns roughly 10% of asset value with zero output. Combined depreciation and forgone rental can exceed $100 per H100 per day (Introl, 2025).

Modular answers with speed. Liquid-cooled AI modules have been deployed in 8–10 months, and one operator stood up 36 micro modular data centers across 20 cities in 11 months, three times faster than traditional construction at about 40% lower cost (Introl, 2025). Because idle GPUs bleed money daily, every month a modular deployment saves converts directly into avoided depreciation and captured revenue.

The capex backdrop makes the urgency literal: worldwide data center capex rose 57% in 2025, GPUs and custom accelerators now account for about a third of it, and 2026 spend is forecast past $1 trillion (Dell'Oro, 2026). North American colocation vacancy hit a record-low 1.4% at the end of 2025, and CBRE notes inference AI is "redefining demand toward more regional and distributed data centers" (CBRE, 2026). That's the modular thesis, stated by the market. For the full delivery model, start with the definitive guide to modular data centers.

Map the AI workload to an actual rack and facility schedule before choosing the building. ModulEdge's dense air-cooled and rack-scale liquid-cooled options have different electrical, structural and thermal requirements. Expansion or a change of hardware still needs a review of those interfaces.

Frequently asked questions

How much power does an NVIDIA GB200 NVL72 rack use? NVIDIA's nominal spec is about 120 kW per rack, and deployed systems have been reported at 130–132 kW under full load (NVIDIA / HPE, 2025). That is roughly 15–40× the density of a legacy 3–8 kW rack, which is why it breaks conventional data center design assumptions.

Why can't a GB200 NVL72 be cooled with air? Its supported rack design uses direct liquid cooling for the designated compute components and retains a residual-air load. Select temperatures, flow and pressure from the exact OEM configuration. The NVL72 requirement does not establish a universal rack-power threshold beyond which every air-cooled server is impossible.

Can an existing data center be retrofitted for AI racks? Usually not economically. AI racks require 6–20× more power density, 3–5× heavier floor loading, high-voltage distribution, and liquid-cooling plumbing that "amounts to reconstruction, not incremental retrofit" (Global Data Center Hub, 2026). One documented four-year-old facility faced nine-figure retrofit costs and negative ROI, leaving it structurally disqualified.

How is inference infrastructure different from training infrastructure? Training uses very large, tightly coupled GPU clusters in a few locations and tolerates external latency, so it chases cheap power and land. Inference is latency-sensitive and distributed, running closer to users across many regional sites, often on commodity Ethernet (EdgeCore / Introl, 2025). Inference is expected to become a larger share of AI capex going forward (Dell'Oro, 2026).

What does it cost to leave GPUs idle while a site gets built? A lot. Deployment delays run 20–30% of cluster capex per half-year, and a six-month delay on a 512-H100 cluster is over $9 million in lost value (Rafay, 2026). Combined depreciation and forgone rental revenue can exceed $100 per H100 per day, which is why time-to-power, where modular wins, is the metric that matters (Introl, 2025).

Why is a modular data center better suited to AI than a converted building? A purpose-built modular facility can integrate power, cooling and structural requirements around the selected AI equipment. Dense air-cooled servers and direct-liquid rack-scale systems require different interfaces. Factory manufacture and site work may run in parallel, while the actual programme still depends on power, permits, equipment and commissioning. Compare that complete scope with a surveyed retrofit rather than assuming one universal density, cooling method or delivery time.