Sovereign Cloud vs Private Cloud: Choosing Control for AI Workloads

A private cloud describes how cloud infrastructure is allocated: it is provided for one organization. A sovereign-cloud proposal addresses a set of control objectives, which may include data location, jurisdiction, administration, encryption keys, operating independence and supplier dependencies. These are different dimensions. A private cloud can be part of a sovereignty strategy, and a sovereign service can use private or shared infrastructure.

For AI workloads, the decision needs to follow the complete service: documents, prompts, retrieved material, model execution, outputs, logs, administration and recovery. Owning servers or choosing a local cloud region settles only part of that design.

This guide compares the operating choices and the evidence needed to evaluate them. Our sovereign AI infrastructure overview covers the broader infrastructure question; here the task is to turn control requirements into an arrangement a buyer can specify and test.

TL;DR

• Private, sovereign, on-premises and disconnected are not interchangeable labels. Write the requirement each one is supposed to meet.
• Separate physical location, legal assessment, administrative authority, key custody and continuity. One control cannot stand in for the others.
• A hosted private cloud may keep infrastructure exclusive to one organization while leaving management with a supplier.
• Customer-controlled encryption keys address a defined cryptographic boundary; they do not automatically protect every live application or administrative path.
• An on-premises AI service can still depend on external identity, licenses, registries, support or APIs.
• Compare complete operating arrangements, including recovery and exit, before deciding whether the requirement needs a new facility.

Private cloud is a deployment model

NIST SP 800-145 defines private cloud as cloud infrastructure for exclusive use by one organization. It can be operated by that organization, a third party or a combination, and can exist on or off premises. Public cloud is a separate deployment model. NIST also identifies essential cloud characteristics, including on-demand service, resource pooling and measured service.

That distinction matters when a proposal uses private cloud to mean several different things: dedicated hardware, an isolated tenant environment, a locally managed cloud platform or simply servers inside the buyer's building. Ask what is exclusive, which resources remain shared and who can administer them. A dedicated application tenancy and a private-cloud deployment are not automatically the same scope.

Similarly, putting servers in a local room does not by itself create a cloud. The organization still needs whatever provisioning, resource management, monitoring and operating capabilities its service requires. Some buyers need a small dedicated application system rather than a general-purpose private cloud. Avoid purchasing a larger platform merely to satisfy a label.

Sovereignty objectives need explicit boundaries

There is no useful procurement shortcut in asking whether an entire service is simply sovereign or not. Identify which decisions the customer must retain, which external authorities or dependencies matter, and what evidence will establish the result. Separate a legal conclusion from a technical control and from a commercial assurance.

These objectives can overlap with private infrastructure. Microsoft's Sovereign Private Cloud documentation, for example, describes a portfolio built around Azure Local in customer-managed environments. Its terminology demonstrates that private and sovereign are not opposing product categories. The buyer must still inspect the particular services, deployment mode and operating terms being proposed.

Treat assertions about immunity from foreign law with particular care. Data location, the provider's headquarters or the nationality of an administrator are not sufficient information for a universal legal conclusion. Ask the organization's advisers to evaluate the parties, processing, access, legal mechanisms and relevant sector obligations. Technical evidence should be precise enough for that assessment rather than pretending to replace it.

Map the AI data flow before choosing a platform

An AI application can create several sensitive information objects from one source document. Uploads may be parsed, divided into passages, embedded, indexed, retrieved into prompts and stored again in generated answers or logs. Administrative tools, evaluation datasets and diagnostic bundles create additional paths. Keeping the initial file in a selected region does not establish where all those later objects go.

Trace one representative request from the user to the result. Record the systems involved and the identity under which each action occurs. Include an external model API, if one is used, as a processing destination rather than an invisible implementation detail. Inspect the location and retention policy of prompt logs, traces and support exports separately from the application database.

For retrieval-augmented generation, test source permissions. A user who cannot access a document should not gain its contents through retrieved passages or an answer generated from them. This question applies in shared cloud, hosted private cloud and customer-operated installations alike. An exclusive server does not repair an application that applies the wrong authorization rule.

Define the treatment of model artifacts too. Weights, adapters, tokenizers, configurations and approved evaluation sets need owners, versions and licensing conditions. A local model that silently calls an external embedding, reranking or document-processing service can violate the intended boundary while the main generation process remains local.

This mapping also identifies when a less restrictive arrangement might suffice. Some public or non-sensitive workloads may remain on a managed service, while a separate set of documents requires a controlled local path. The split must be enforced at the application and data level, not left to users remembering which endpoint they should choose.

Inspect keys and the points where plaintext is needed

Encryption requirements should identify the data state and threat being addressed: stored data, network traffic, active computation or backups. Key ownership is one entry in that design. It is not an answer to every question about an AI service.

Google Cloud External Key Manager provides a concrete example. Its documentation describes keys held in a supported external key manager and authorization of cryptographic operations through that system. That is a distinct control boundary from storing all keys with the cloud provider. The organization must also plan access permissions, supported service integrations and key-service availability.

For a live AI application, determine where information is decrypted or otherwise available for processing, which software and administrators operate there, and what evidence supports the claimed isolation. Externally held keys for stored objects do not by themselves establish that the application runtime cannot access the content it has been authorized to process.

Confidential computing addresses a different part of the problem. Google's Confidential VM documentation describes hardware-based memory protection and attestation in supported configurations. These controls can be relevant, but the exact CPU/GPU configuration, software, attestation policy and protected boundary have to match the proposal. They do not automatically establish application authorization, data retention, legal compliance or independence from every management service.

Include key failure and recovery in the operating plan. Who can restore service if a key manager is unavailable? Who can revoke access, and what happens to a running application or an old backup when that authority is exercised? Test these cases with approved non-production data and a controlled plan. A design that gives the customer an effective power to deny access also needs a deliberate decision about the availability consequences.

Administrative control includes changes, support and recovery

Ask for a map of ordinary and exceptional privileged access. Identify who can change the hypervisor, cluster configuration, firmware, identity policies, backups and application deployment. Record whether access is standing, time-limited, customer-approved or permitted through an emergency process, and who can inspect the resulting records.

Physical ownership is only one part of this map. A customer-operated installation may use vendor remote support, an external identity service, internet package repositories or a managed backup destination. A hosted installation may offer strong customer approval and logging controls. Neither should be judged solely from its address.

NIST's Zero Trust Architecture rejects implicit trust based solely on network or physical location and asset ownership. Applied here, the useful question is whether each user, device and administrative action receives the appropriate authorization. Local equipment does not remove that requirement.

Support arrangements deserve a separate review. Establish what a support engineer can see, which diagnostic data can leave the environment and who approves exports. If support must occur without remote access, define on-site response, sanitized diagnostic collection, replacement equipment and escalation. An attractive restriction on access can become an operating problem if nobody has arranged a supported way to diagnose failure.

The change process should preserve the agreed control boundary. New model features, monitoring agents or management integrations can introduce data flows that were absent at initial acceptance. Keep a configuration baseline and require changes affecting location, access or dependencies to undergo the appropriate review.

Connected, restricted and disconnected are separate choices

An environment can have tightly controlled outbound access while remaining connected. It can operate locally during a network outage while still needing periodic external exchanges. A fully disconnected arrangement requires a different dependency and lifecycle design. State which behavior the requirement actually needs and for how long.

Microsoft's disconnected operations for Azure Local illustrates that a private-cloud platform can use a local control plane without a connection to the Azure public cloud. The documented option has eligibility, support and additional capacity requirements. It should not be generalized to every Azure Local or private-cloud configuration.

For an AI service, examine identity, time, certificates, package and model registries, licensing, monitoring, backups and recovery. Restarting an already configured demonstration can miss a dependency needed to replace a failed node or install an approved update. The evidence needs to cover those lifecycle operations as well as a normal request.

Our air-gapped AI deployment guide goes into the implementation and testing process. Disconnection is justified by a documented operating or security requirement; it is not a compulsory interpretation of sovereignty or data residency.

Compare the complete operating arrangements

The following matrix describes questions to resolve, not default assurances attached to each model. A particular provider may distribute responsibilities differently.

Compare these arrangements against the same workload and service level. Customer ownership may move expenditure into equipment, staffing and facility capacity; a managed service may bundle parts of those costs into fees. Use the on-premise inference economics guide as a starting point, but calculate the actual system, support and utilization assumptions. There is no single utilization percentage that settles this choice for all AI services.

Example: specify an internal engineering assistant

Consider an illustrative organization evaluating an assistant over internal technical documents. This is a planning example, not a reported customer deployment. The buyer selects four requirements: document permissions must survive retrieval; specified data objects must stay in approved locations; privileged support access requires a documented process; and the service must recover within a buyer-defined window.

That brief does not yet require ownership of the building, a particular cloud brand or an air gap. It needs an evidence matrix. The team might compare a managed service, a hosted private environment and an existing-room deployment against the same tests.

If the buyer adds a requirement to operate without a public connection, update the comparison. Identify which options have a supported local identity, runtime, licensing and recovery path. Do not quietly convert the original requirement into disconnection after procurement; the added control changes capacity, operations and support.

If existing physical infrastructure is adequate, it may remain the right location. If it lacks power, cooling, service access or a suitable management boundary, compare an upgrade, an appropriate hosting arrangement and a dedicated facility. Our edge AI infrastructure guide explains how workload and site constraints connect. The control requirement and the facility gap should both be demonstrated.

Keep legal assessment specific to the service

For personal-data processing, residency statements are not a complete compliance analysis. The European Data Protection Board's transfer guidance explains that transfers outside the European Economic Area (EEA) have additional conditions and can use mechanisms such as adequacy decisions or appropriate safeguards. General processing obligations still apply. This is a framework for assessment, not a blanket instruction to put every regulated workload on customer premises.

Provide advisers with the real entities, processing purposes, data classes, recipients, administrative paths and contracts. Include sector and national requirements where they apply. Avoid conclusions that a region choice always satisfies sovereignty, that every foreign supplier is prohibited, or that local hardware eliminates transfer and access questions. The applicable facts can change as the service or its subcontractors change.

Our sovereign data infrastructure article provides wider project context. For a purchase decision, the actual operating arrangement and current primary rules take precedence over a broad country table or a vendor's category label.

Make recovery and exit part of procurement

Ask what can be exported: business data, embeddings, configurations, model artifacts, access policies, audit records and infrastructure definitions. Check licensing and technical dependencies before assuming that a virtual machine export or model download recreates the service elsewhere.

Exercise a representative exit or rebuild in an appropriate environment. Measure the time, data movement, required transformations and operating work. Record functions that rely on proprietary APIs or a provider-managed service, together with the approved replacement approach. An export right that has never been exercised is weaker planning evidence than a documented recovery path.

The same discipline applies to backups. State the location, administrator, key dependency, restoration authority and retention behavior. A backup in another permitted region may improve resilience while changing the location and legal assessment; a local backup may still be unusable if it depends on credentials or software unavailable during an incident. Design the required recovery case rather than assuming that one storage location solves everything.

Where physical infrastructure enters the decision

A dedicated facility is relevant when the chosen equipment and operating boundary require capabilities that the available locations cannot provide. Translate the server schedule into electrical and thermal loads, management-network boundaries, physical access and maintenance arrangements. The physical-security guide addresses facility controls; those controls complement application and administrative security.

For local AI, a supported air-cooled server configuration may fit the workload; another system may need direct-liquid interfaces and a separate residual-air path. Neither sovereignty nor private cloud imposes a universal rack-density minimum. A server-specific planning guide can help evaluate one candidate, while the final selection follows application testing and the site.

ModulEdge develops physical infrastructure, with product selection and configuration around the equipment and location. Its product range offers different starting architectures. Providing power, cooling, IT space and facility controls does not by itself supply a sovereign software platform or establish legal compliance. The operator still owns the control requirements and the complete service assessment.

Frequently asked questions

Does a dedicated environment remove every isolation question?

No. Review the boundaries between departments, users, applications, administrators and backup systems within the organization as well. Infrastructure dedicated to one organization can still contain several trust levels. The acceptance plan should include access between those levels, rather than stopping at a statement that no other customer shares the environment.

How should a temporary exception to the location or support policy work?

Specify the approving role, permitted action, data involved, duration and required record before an emergency. State how access expires and who verifies closure. If the requirement allows no exception, the recovery and support plan must work within that constraint instead of assuming someone can waive it during an incident.

What evidence should a supplier provide when data is deleted?

Ask which copies the deletion covers, how replicas and retained backups are treated, what retention exceptions remain and what evidence is available. A deletion certificate should identify its scope and limitations. The buyer should not treat the removal of a production account as proof that every diagnostic export or retained backup was erased simultaneously.

How often should the control assessment be repeated?

Use both a documented review schedule and event-based checks. Relevant events include a new subprocessor, management integration, recovery location, model API or support arrangement. Assign an owner to decide whether the change requires a targeted check or broader reassessment; a calendar review alone can miss a material change introduced between reviews.

Must an existing cloud workload be moved immediately when sovereignty becomes a priority?

Start by identifying the unmet requirement and the workload it affects. Compare remediation in the current arrangement with migration, including the risks and dependencies of the move itself. A controlled transition with explicit interim treatment may be more workable than moving every application before its replacement operating model has been validated.

Should model artifacts receive the same classification as source documents?

Make an explicit classification decision for each artifact and its intended use. Weights, adapters, embeddings, evaluation sets and generated outputs are different objects with different access and licensing questions. Record who approves their movement or reuse; do not automatically classify them as public merely because they are no longer in the source-document format.