Air-Gapped AI: An Enterprise Deployment and Acceptance Guide

Air-gapped AI runs inside an isolated environment without depending on a connection to the public internet for its agreed operating functions. Deploying it requires more than copying model weights to a local GPU: the application, identity services, model artifacts, licensing where applicable, monitoring, backups and recovery procedures must all work inside the defined boundary.

For an enterprise pursuing sovereign AI, isolation is one possible control. Choose it because a documented requirement demands it, then prove that the complete service can operate that way. This guide covers that deployment decision and its acceptance tests. Our sovereign AI infrastructure overview covers the broader strategic question; this article concentrates on implementing a disconnected service.

TL;DR

• Define what must remain local, who may administer it and how long it must operate without external services.
• Separate physical disconnection from network restrictions. A connected private environment with an outbound firewall policy has a different control boundary.
• Inventory every runtime dependency, including authentication, time, DNS, registries, licenses, retrieval, telemetry and backups.
• Prepare and verify approved software/model bundles through a controlled import process. A cached demonstration is insufficient evidence of repeatable deployment.
• Size servers and facility infrastructure from the actual workload and supported configurations. Sovereignty does not impose a universal rack density or facility size.
• Witness disconnected cold starts, application tests, restore and update rollback before accepting the service.

Decide whether the requirement actually calls for an air gap

Start with the failure or exposure the organization needs to prevent. “Keep our AI sovereign” is too broad to specify a system. A useful requirement identifies the information, permitted users and operators, allowed locations, external dependencies and required continuity.

Consider an internal engineering assistant. The organization may allow approved staff to query local technical documents while prohibiting prompts, retrieved passages and outputs from reaching external model APIs. That requirement could be met by several architectures. A separately documented requirement to keep operating indefinitely without a public connection would eliminate options that need online identity, licensing or model services.

Use the following distinctions when comparing proposals. They are a planning framework, rather than legal definitions or certifications.

On-premises equipment can still call a remote inference API or depend on external administrators. Conversely, a locally operated service may sit in a suitable colocation facility. Treat ownership, location and control as separate entries in the evaluation.

Data protection rules also require a specific assessment. The European Data Protection Board describes controller and processor obligations, while its international-transfer guidance explains mechanisms and conditions for transfers. These are not blanket instructions to move every regulated AI workload into an on-premises module.

Compare delivery models before building a new facility

Compare a supported existing-room deployment, an appropriate local operator, a dedicated modular facility and a managed disconnected platform against the same requirement. Include staffing and lifecycle support as well as equipment and space.

Google Distributed Cloud air-gapped is one vendor example of a platform designed for disconnected operation. Google's documentation also distinguishes that product from connected distributed-cloud options. This demonstrates that “cloud” does not always mean a service requiring a public internet path. It does not establish that any particular configuration satisfies your procurement, operating or legal requirements.

Ask each provider for the same evidence: data flows, administrator routes, required external contacts, update process, supported disconnection period and recovery procedure. A customer-operated stack may offer detailed control but requires people to maintain it. A managed platform can transfer operating work, while introducing its own support, hardware and software boundaries.

Keep the broader cloud-versus-owned economics in the on-premise inference guide. For this decision, the additional cost question is narrower: what does the required isolation add to deployment, operations, incident response and replacement?

Draw the complete service boundary

Map a real request from the user's workstation to the result. Include the user interface, identity check, document retrieval, embedding or reranking service, model runtime, tools, output storage and audit records. Draw administrative and update paths on the same diagram.

Then give every external connection an explicit disposition: remove it, replace it with an approved internal service, or retain it in an architecture that is described accurately as connected. A physically isolated design and a connected environment with strict firewall rules should not be accepted under the same test statement.

The boundary also includes the facility's management connections. Inspect server management controllers, remote console access, cooling controls, UPS monitoring and supplier support channels. An isolated application network does not settle how those devices are administered. Specify permitted local access and the process for exceptional intervention.

Isolation does not make every internal actor trustworthy. NIST's Zero Trust Architecture rejects implicit trust based solely on physical or network location. Apply that principle through authenticated users and devices, limited administrative privileges and recorded changes. The exact implementation must fit the organization's threat model and operating capabilities.

Build a dependency register before procurement

The following register is a practical starting point. For every item, name the internal service, accountable owner, version or validity constraints, and the test proving it works offline.

Review the register with application, security, platform and facilities owners together. Procurement should expose unsupported combinations before the equipment arrives. A supplier answering only for the GPU server cannot establish the behavior of an identity service, vector database or backup platform selected by another party.

Make model imports and updates repeatable

Separate the connected acquisition environment from the isolated production environment. Obtain approved artifacts on the connected side, verify their provenance and integrity, and transfer them through the organization's authorized process. Do not create an ad hoc network path merely to simplify installation.

NVIDIA's NIM 2.1.4 air-gap deployment guide documents a connected preparation phase followed by an isolated runtime phase. It requires compatible pre-staged model assets and container images. That distinction matters: possessing a container image does not necessarily mean possessing the model assets it will try to load.

For each approved release bundle, record:

1. The source and license for the model, runtime, drivers and other components.
2. Exact versions and cryptographic digests, together with the verification method.
3. Hardware compatibility and the tested driver/runtime combination.
4. The configuration, model assets and internal services needed to start.
5. The application-quality and security checks that permit promotion.
6. A rollback package and the conditions requiring its use.

Model updates need an application evaluation as well as an installation check. A changed model or quantization may alter document handling, output quality, latency and memory use. Retain an agreed test set and compare results before promoting a bundle. Use representative material under the organization's data-handling rules.

The import path is also a security boundary. Define who approves a transfer, how media or packages are checked, where they are staged and how the result is logged. An air gap still admits new software and data through these procedures. Treating the transfer workstation as an informal convenience leaves a consequential part of the system unmanaged.

Validate licensing for the chosen deployment mode

Avoid both extremes: assuming all commercial AI software must contact a public licensing service, or assuming local inference means licenses can be ignored. The requirement depends on the product, entitlement and deployment mode.

For example, NVIDIA's vGPU for Compute licensing documentation describes enforcement and degraded performance when a required license is unavailable. That behavior concerns the documented vGPU mode; it should not be generalized to every bare-metal CUDA workload. The NVIDIA License System guide includes procedures for a Delegated License Service on an air-gapped network.

Request a supported design that covers the intended operating period, license-server availability, replacement hardware and entitlement renewal. Include the applicable validity conditions in monitoring and runbooks. A short offline demonstration may finish before an unaddressed time-dependent condition appears.

Do not alter system clocks to simulate expiry on a production platform. Use a vendor-approved test method or documentary evidence for conditions that cannot be exercised safely during acceptance. Record which results were witnessed and which depend on contractual or product documentation.

Protect the data throughout the AI workflow

For an enterprise retrieval service, prompts and model weights are only part of the information boundary. Documents, embeddings, retrieved passages, generated answers, audit records and support bundles may all contain sensitive information. Specify where each object is stored, who can read it, how long it is retained and how it is removed.

Test document-level authorization through retrieval. Use accounts with different permissions and confirm that inaccessible material does not appear in search results or model context. An internal model cannot compensate for a retrieval layer that ignores source permissions.

Constrain tool use separately from answer generation. A model that can trigger internal actions needs narrowly defined tool permissions, approval where the business process requires it, and an audit trail. Isolation from the public internet does not establish that an internal action is authorized or safe.

Set limits for logging deliberately. Capturing every prompt and response may help troubleshooting while creating another sensitive dataset. Agree what operators need to diagnose a failure and how they obtain it. Apply the same review to telemetry exports and support bundles before they leave the boundary.

Back up the configuration and access-control state alongside application data. A restored model that no longer respects the original document permissions is not a successful recovery. Include key custody and recovery authority in the operating design without placing secrets in general deployment documentation.

Size compute and facility capacity from a defined service

Begin with the workload: model and precision, context lengths, concurrent users, response targets, retrieval volume and permitted downtime. Benchmark the intended server configuration. A sovereignty requirement does not imply frontier training hardware, a fixed number of GPUs or a minimum rack density.

An RTX PRO 6000 Blackwell Server Edition deployment may be one candidate for enterprise inference. Other supported GPU or CPU configurations may fit a smaller workload. The isolation requirement and the accelerator choice are separate decisions; both have to work in the assembled service.

Translate the equipment list into a rack schedule that includes networking, storage and management. Then establish circuit capacity, cooling conditions, service access, structural limits and the required behavior during maintenance and faults. Keep IT input separate from total facility input, which also includes cooling and other auxiliaries.

An existing room may be sufficient after assessment. A dedicated module becomes relevant when the selected equipment and control requirements exceed what that room or another appropriate location can support. Compare the cost and operating consequences before committing to a new facility; isolation alone does not prove the need for one.

Example: turn an isolation requirement into acceptance tests

Consider an illustrative internal document assistant. Its chosen requirements are: no public internet connectivity during operation; local user authentication; document permissions preserved; and recovery within a buyer-selected service window. This is a planning example, not a reported customer deployment or an industry standard.

The project team can translate those requirements into the following tests. Define pass criteria before the trial so a demonstration cannot quietly substitute for acceptance.

Do not run disruptive tests on an operating service without its approved maintenance plan. The design-stage task is to specify and witness them in an appropriate environment. Include coverage limits: a finite observation period cannot prove that no time-dependent behavior will ever occur.

Maintain a discrepancy register. Assign each failed or untested requirement to an owner, corrective action and retest. Distinguish an accepted exception from an assumption nobody examined. Handover should include the operating baseline and test evidence, rather than just installation credentials and a successful sample response.

Assign responsibility for the operating lifecycle

Name who owns application quality, platform updates, security policy, facility systems and local incident response. Suppliers can support several layers, but the enterprise needs an accountable service owner who can coordinate a failure crossing those boundaries.

Agree how support works when remote access is unavailable: diagnostic collection, permitted export, replacement parts, on-site response and escalation. Include the lead time for preparing and approving an update bundle. An isolated installation can remain exposed to known vulnerabilities if its change process cannot deliver supported updates.

For cost comparisons, include platform staffing, internal repositories, test capacity, spares, backup, update qualification and recovery exercises. Use current scope-matched quotations. There is no universal utilization percentage at which these operating costs disappear or ownership becomes cheaper than a managed alternative.

Where ModulEdge fits

ModulEdge supplies the physical infrastructure around the selected equipment: power, cooling, rack space and associated facility systems. It can define interfaces with the customer's or integrator's compute and software stack. Buying that infrastructure does not by itself establish data sovereignty, software isolation or legal compliance.

For a suitable dense air-cooled configuration, the issued MDC500 air-cooled reference provides 500 kW IT with ten rack positions and rear-door heat exchangers. A smaller workload requires its own appropriately sized assessment. Public product options help frame that discussion; final selection follows the equipment, control requirements, site and operating model.

An air-gapped AI deployment needs operating controls as well as local racks. ModulEdge can define the physical power, cooling, monitoring and service interfaces; the operator must specify isolated administration, software and data transfer, update procedures and acceptance evidence.

Frequently asked questions

Is air-gapped AI the same as sovereign AI?

Air-gapped AI addresses disconnected operation within a defined boundary. Sovereignty can also involve authority over operators, data, keys, software dependencies and procurement. Determine the required controls rather than treating the two labels as equivalent.

Does an on-premises AI server automatically work offline?

No. Authentication, model downloads, licensing, retrieval, monitoring or other services may still depend on an external connection. Test the complete application and its restart and recovery paths.

Can an air-gapped system receive updates?

Yes, through the organization's controlled import process. Acquire and verify the approved release outside the isolated production environment, transfer it by an authorized method, then test and promote it locally.

Do data-residency requirements always require an air gap?

No. Residency concerns location; disconnection addresses a different control. The actual legal, contractual and security requirements must be assessed for the specific processing and operating arrangement.

What should a disconnected acceptance test include?

At minimum, the agreed application workload, clean start, identity and authorization, required licensing arrangements, dependency failures, local restore and update rollback. Add facility and physical-control tests according to the project's scope.

Does sovereign AI require a new modular data center?

No. Assess existing rooms, suitable local facilities and managed options against the same requirements. A dedicated module is appropriate when its supported infrastructure and operating boundary solve an established gap.