How to deploy air-gapped AI in regulated and high-security environments

Air-gapped AI is any AI workload that runs entirely inside a network boundary with no internet connectivity, no outbound traffic, and no runtime dependency on an external provider. For engineers in defense, intelligence, pharma, and financial services, that constraint changes almost every assumption a normal AI infrastructure project makes. Model downloads, package installs, and the certificate checks a cloud-connected team never thinks about all have to be re-engineered to work without a live connection. Gartner notes that sovereignty and operational-isolation requirements can rule out centralized deployment patterns altogether, which limits what an organization can deploy and scale.

This guide covers what the component lists usually skip: the order things have to enter, where the first install fails, how to find the calls that still try to leave, and what a model has to bring with it to pass an audit.

What air-gapped AI means for engineers

Every artifact the AI stack depends on, including container images, model weights, Python packages, GPU drivers, and operating system updates, enters through a controlled and auditable process before it can be used, and nothing inside the boundary reaches back out once it is running. For an engineer, that turns deployment from a pull-based workflow, where the latest weights or dependency is one command away, into a push-based one, where everything is staged in advance and versioned deliberately.

The scale of the change usually shows up at the first deployment attempt. A workspace that resolved packages from a public index and pulled a base image from a public registry now has nowhere to reach, and every one of those calls has to be redirected to an internal mirror someone built in advance. Engineers who have only worked in cloud-connected environments underestimate how many of those calls a typical stack makes until the first install fails against a boundary that permits none of them.

Air-gapped vs isolated vs self-hosted AI deployments

These terms get used loosely, and the differences matter when choosing an architecture.

Gartner draws a similar line for mission-critical workloads, separating physically isolated sovereign cloud regions operated by local personnel from fully disconnected environments where the organization owns the hardware and the software update cycle, and it cautions that incomplete isolation or inconsistent legal-entity separation can weaken a provider's sovereignty claims. The choice is usually made per workload. Many high-security environments run air-gapped for the work that touches classified or export-controlled data and isolated or self-hosted for everything else.

When AI workloads need an air gap

An air gap is expensive to build and to operate, so reserve it for the workloads that require it rather than applying it as a default posture. Gartner puts the sovereign infrastructure premium at 15% to 20%, because sovereign mandates force environment duplication.

Classified and export-controlled work is the clearest case. Defense, intelligence, and other government workloads under classification or export control typically have no alternative to a physically air-gapped installation, and Gartner reports that defense agencies are building private, physically isolated AI environments to run intelligence analysis and logistics optimization with no connection to the public internet. Critical infrastructure operators in energy, water, and telecommunications reach the same conclusion for operational technology networks that cannot absorb the exposure even a well-monitored connection introduces.

Outside those sectors the driver is intellectual property. A pharmaceutical company running proprietary compound research, or a trading firm evaluating a proprietary model, carries less regulatory obligation than a defense contractor but has the same reason to keep the workload, and the vendor operating it, from ever touching an external network.

What has to exist inside the boundary before a model runs

The boundary has to hold a private container registry and a private model registry, offline package mirrors, an internal identity provider and certificate authority, and GPU-backed compute that Kubernetes can schedule. Each replaces a service a connected deployment would reach externally, and each has to be populated and kept current from inside. The components themselves are well documented elsewhere, so this section covers the order they have to arrive in and the places the first install fails.

The bootstrap order, and where the first install fails

Nothing in the AI layer runs until the platform underneath it is in, and the platform has its own dependencies on the outside world. Bring the layers in this order.

1. Operating system and container runtime packages.
2. Kubernetes component images.
3. The GPU stack. The NVIDIA driver on the node has to support the CUDA version baked into every inference image you intend to run, and the GPU Operator that installs the driver, the container toolkit, the device plugin, and the monitoring exporter pulls each of those as its own container image. Every one has to be in the private registry before the Operator reconciles, or it sits in a pending state with nothing to tell you why.
4. A single CUDA sample pod, to validate the GPU path before anything heavier is scheduled.

A driver bump later means a node reboot, and a broken driver is slow to recover from with no internet connection, so pin the driver and toolkit versions in the same manifest as the model and image versions.

Package mirrors are harder than they look

Python, R, and system package managers all assume a live index, so each index has to be mirrored and kept current through the same import process as everything else. A mirror of PyPI alone will not get you there. Transitive dependencies resolve on a connected staging host, and that host has to match the target on Python version and platform tags, or the wheels it downloads will not install inside. GPU builds of PyTorch come from the PyTorch index rather than PyPI, source distributions need a compiler at install time that the target image may not carry, and conda channels have to be mirrored separately. Resolve a lockfile on the staging host, download against that lockfile, and build workspace images that point at the internal mirror in their base configuration, so a dependency change never triggers a call to the outside world that simply times out.

Identity, certificates, and secrets

Authentication, TLS, and secrets management all need an internal source of truth. Most air-gapped environments stand up an internal certificate authority and federate to an existing on-premises identity provider, falling back to local accounts only where federation is not possible. Secrets, including the keys that route traffic between internal services, are stored and rotated inside the boundary with the tooling the enterprise already runs on-premises. Gartner's digital sovereignty framework treats key management as a control of its own: the ability to generate, store, and manage encryption keys independently of any service provider, whether through customer-managed keys, external key management, or customer-operated hardware security modules.

Finding the calls that still try to leave

Before the boundary closes, run the entire stack behind a default-deny egress policy with DNS query logging turned on, and read the log. The list of things that try to phone home is usually longer than expected.

• Model-loading libraries fetch tokenizer and config files from a public hub at startup unless offline mode is set in the environment.
• Some inference servers, vLLM among them, report anonymous usage statistics unless the setting is turned off.
• Some commercial inference containers check a license server at startup.
• RAG code written against a managed vector service calls a remote embedding API from inside a loop, a common way an air-gapped retrieval pipeline ends up with a hole in it.
• Certificate validation tries to reach an OCSP responder.
• NTP tries to reach a public pool.

Each of these is a one-line fix once you know it exists, and a multi-day incident when the first symptom is an inference service that hangs on startup inside the boundary.

Deploying an air-gapped LLM

Serving a large language model inside the boundary follows the same staging discipline as everything else, applied to artifacts that are far larger and more particular about their runtime.

Open-weight models and what has to travel with them

Open-weight models are the practical starting point, since a closed, API-only model has no offline path by definition. Gartner's guidance on open-weight models points the same way: they can be installed locally and run in an air-gapped environment, which supports data sovereignty, with organizations "owning the compute, the network and the data." Gartner also notes that downloaded weights run subject to license and hardware requirements, and the license requirement deserves its own step. Llama, Gemma, Qwen, and Mistral weights ship under different terms, some with acceptable-use restrictions and some with field-of-use clauses that matter to a defense program, so the license review belongs in the same approval as the import, with the reviewed license text stored next to the weights.

Weights move across the boundary through the same documented import process as container images and are staged on shared file storage before an inference service loads them. With the weights go a checksum manifest, verified on both sides of the transfer, since a truncated checkpoint is an expensive way to discover a flaky copy, and a provenance record naming the source, the revision, the date, and the approver, written into the model registry at import. Settle the format before import as well. Prefer safetensors over pickle-based checkpoints, because a pickled checkpoint can execute arbitrary code on load and an air gap does nothing to stop that.

Serving and inference inside the installation

Once staged, the model is loaded by an inference server such as vLLM, running on the GPU nodes provisioned earlier and exposed through an OpenAI-compatible endpoint inside the cluster. Registering that endpoint behind an internal gateway gives the environment a single place to enforce access control and to log every request against the identity of the person or service that made it. That log is frequently the first artifact an auditor asks for.

Retrieval without external services

Retrieval-augmented generation inside an air gap needs a vector store that runs locally, an embedding model that runs locally, and a document ingest path that can accept new material through the same import process as everything else. Plan the embedding model early, because a pipeline designed against a managed vector service rarely ports cleanly until both the store and the embedding model are inside, and the embedding call is easy to miss during the egress audit above.

Architecture patterns for air-gapped AI

Air-gapped deployments differ mainly in how artifacts cross the boundary.

Fully disconnected installation

The strictest pattern has no live connection of any kind. Every import, including models, images, and packages, crosses on physical media carried by an authorized person under a documented chain of custody. This is standard for classified environments, gives the strongest guarantee available, and has the slowest update cycle.

One-way transfer with a data diode

A data diode allows information to flow in one direction only and makes the return path physically impossible. That property is the appeal, and it is also the operational constraint. With no return channel the receiving side cannot request a retransmission, so transfers run over file-based protocol breaks with forward error correction and are verified again on the inside against the manifest. Diodes speed up imports considerably compared with media, and nothing inside the boundary can signal out even if a component inside were compromised. Many programs run them outbound as well, for logs and telemetry leaving an operational network, and whether an inbound diode satisfies the accreditor for software imports is a question to settle with the accreditor before the hardware is ordered.

Central governance across several isolated enclaves

Organizations running several separated environments, whether separate agencies, classification levels, or business units, often want one governance and orchestration layer without merging the installations. The design depends on whether the enclaves are isolated or air-gapped. If each data plane keeps a live management connection to a control plane outside its boundary, the data plane is isolated, not air-gapped, and the design question becomes what the control plane is allowed to hold and which direction management traffic flows. The control plane holds desired state and metadata, never data, prompts, or secrets, and management traffic is pulled from inside rather than pushed in. If the enclaves are air-gapped, the control plane lives inside each one, and central governance means the policy set is an artifact that crosses the boundary through the same import process as a model, so the same rules are enforced in every enclave without any enclave talking to another. Gartner recommends a modular reference architecture along these lines, built on a standardized AI landing zone and runtime environments that replicate across sovereign boundaries, so an organization can adapt when AI or data residency regulations change.

Keeping air-gapped AI systems current

An air-gapped environment does not get easier to maintain once it is running. Every update a connected system receives automatically has to be reintroduced through the import process used for the initial deployment.

Import manifests and approvals

A recurring import cadence keeps images, operating system patches, drivers, and model versions from drifting too far behind upstream. Generating a manifest of exactly what changed since the last import, and requiring a documented approval before the manifest crosses, keeps the process auditable. Gartner's sovereignty framework names change control as a dimension of its own, asking who initiates and approves configuration changes, deployments, patching, and incident interventions, and the manifest plus approval answers that question for each import. Keep the previous model and image versions inside the boundary when the new ones arrive. Rollback after a bad import is a traffic-weight change between two versions that are already there, and it cannot be a re-import, because the next transfer window may be a month away.

Signing, scanning, and validation at the source

Every artifact should be signed where it is built, scanned for known vulnerabilities, and validated against its signature before it is trusted inside. The software supply chain outside the installation still matters even though the installation itself has none, because a compromised upstream image or a tampered checkpoint is exactly the risk an air gap cannot see from inside. Gartner describes software supply chains as extending across organizational boundaries to partners, vendors, open-source software, containers, and AI models, and expects 85% of large enterprise engineering teams to deploy software supply chain security tools by 2028, up from 60% in 2025.

Monitoring and audit logs without egress

No telemetry can leave, so an internal logging and metrics stack stands in for whatever managed observability service a connected deployment would use. Over the long term that stack makes the difference between a quick lookup and a research project when someone asks a routine question, such as who accessed this model and when. Gartner's sovereignty framework raises the same questions for telemetry: where log, metric, and trace data is stored and under which jurisdiction, how far engineer access to it extends, and whether any of it crosses regional boundaries.

Setup tradeoffs in air-gapped AI deployments

The fully disconnected installation gives the strongest guarantee and the heaviest operational burden. A team that chooses it should staff for it, since manual imports, chain-of-custody documentation, and slower incident response are the price of the guarantee.

Hardware constrains model choice before any evaluation begins. The GPUs inside the installation were often ordered and racked months before the software layer, so a model that assumes a cluster of the newest accelerators is not a fair candidate against one that fits the hardware already on the floor. Write the hardware ceiling into the evaluation criteria before the shortlist exists.

Update cadence trades against compliance load. A faster import cadence keeps models and dependencies closer to upstream, but every import is a compliance event that has to be documented, reviewed, and often approved by someone outside engineering. Environments with a lighter burden can import weekly. Those without that flexibility settle for monthly or quarterly, and that cadence should be agreed with the compliance team up front, since they are the ones approving each import.

How to build sovereign AI for high-security environments

Everything above secures the network boundary. It does not, by itself, let anyone trace which weights are running, who approved them, or who has queried them since. Building sovereign AI in a high-security environment means combining the air-gapped pattern with model control, data control, and access control at the platform layer: a private registry with lineage for every model version, a gateway that logs every request by identity, and a governance process that survives a change in personnel. Gartner's 2026 AI playbook frames sovereign AI as a platform decision and advises planning for enterprise AI platforms rather than models alone, listing infrastructure, model lifecycle management, security, GPU cost efficiency, vendor strategy, talent, and governance as the areas to rethink.

Why an enterprise platform simplifies air-gapped AI

Assembling registries, mirrors, identity, GPU scheduling, an inference layer, a gateway, and an update pipeline from separate components is where most air-gapped AI projects lose their schedule. A platform that already ships those pieces as one deployment turns the list into configuration. Gartner projects that by 2030, organizations without architecture standards for data and AI deployment patterns will abandon more than 30% of their AI initiatives, citing escalating sovereignty, latency, governance, and operational complexity.

This is what Domino provides for government and defense customers and for other high-security deployments. Domino is a Kubernetes-native platform that installs from a private registry. The installer enumerates every image a release needs so they can be copied into your registry, by direct transfer or by tar file carried across, before the install begins. Compute environments are built from Dockerfile instructions you control, so pointing pip and conda at the internal mirror happens once in the base image instead of once per user. The model registry carries lineage for every version, hardware tiers decide which GPU node pools a project can schedule onto and who is allowed to use them, and the LLM Gateway fronts self-hosted and external models alike with a log you can filter by user, model, and endpoint. Domino is deployed in DoD IL5 environments, available through Iron Bank, and runs on AWS GovCloud and Azure Government. The U.S. Navy's Project AMMO uses Domino to retrain and deploy automatic target recognition models for unmanned underwater vehicles, shortening model update cycles from months to days. For isolated enclaves that keep a management connection, one Domino control plane governs data planes across regions, clouds, and on-premises sites, so one policy standard holds without merging the environments.

None of this replaces the judgment calls, whether to run a disconnected installation or a diode, how to size hardware against the models that fit it, and what import cadence the compliance process can sustain. What a platform removes is the months spent building the surrounding infrastructure from raw components before the first model can run.

FAQs

What is air-gapped AI?

Air-gapped AI is an AI deployment that runs entirely inside a network boundary with no internet connectivity and no outbound traffic once live. Every dependency, including container images, model weights, packages, drivers, and updates, is imported through a controlled process rather than pulled from an external source. It differs from an isolated deployment, which keeps a controlled path open. The pattern is most common in defense and classified environments, with pharma and financial services adopting it for IP-sensitive workloads.

Can you run an LLM in an air-gapped environment?

Yes, provided every component the model needs is staged before the network closes. Open-weight models are the practical choice, since a closed, API-only model has no offline path. Weights cross the boundary with a checksum manifest, a provenance record, and a reviewed license, and are served by an inference server such as vLLM on local GPU nodes behind an internal gateway that logs every request by identity.

How do you update models in an air-gapped AI deployment?

Updates cross as a documented, one-directional import on a weekly, monthly, or quarterly cadence. Each import carries a manifest of what changed and requires approval before crossing, every artifact is signed and scanned at its source, and the previous version stays inside so rollback is a traffic change rather than another import. Physical media suits the strictest installation; a data diode suits a faster cadence.

What is the difference between air-gapped, isolated, and self-hosted AI deployments?

Self-hosted means the enterprise runs its own models on its own infrastructure and says nothing about network posture. Isolated adds a restricted, monitored boundary with a controlled path still open for updates. Air-gapped removes that path entirely. Many environments mix them, air-gapped for classified workloads and isolated or self-hosted for the rest.

How do you build sovereign AI in a high-security environment?

Air-gapping the network is one layer. A high-security environment also needs model, data, and access control at the platform layer: a model registry with lineage, pipelines that keep sensitive data inside the boundary, and a gateway that logs every request by identity. The air gap secures the boundary. The platform layer proves which model version is running, who approved it, and who has used it.

Sources

Gartner, Digital Sovereignty Is Needed for Future Technological Resilience and Business Outcome, 2 February 2026.

Gartner, Emerging Tech: AI Vendor Race: Sovereign AI Infrastructure Enables AI by the People, for the People, Within Borders, 16 March 2026.

Gartner, Riding the AI Model Innovation Curve: AI Leader's Playbook for 2026, 7 May 2026.

Gartner, Digital Sovereignty Across Data, Operations, and Technology, 11 July 2026.

Gartner, Localize AI or Move Data: 5 Guidelines for AI Deployments, 31 August 2026.

Gartner, Forecasting the $5.3 Billion Opportunity in Software Supply Chain Security, 11 September 2026.

Gartner, Innovation Insight: Open-Weight Models for Software Engineering, 25 September 2026.

Gartner, Sovereign AI: Infrastructure Strategies for Compute Capacity and Operational Control in a Multipolar World, 29 September 2026.