By Chris Du Toit

Copyright thenewstack

For most software teams, integrating AI tools like code assistants is as simple as signing up for a service and adding an extension. You get instant access to powerful models, and with every keystroke, you’re tapping into a vast, cloud-based brain.

But what if your job is to build the software that controls a satellite, manages a power grid or guides a fighter jet? For these teams — and others in defense, aerospace, government and heavily regulated industries — that easy integration isn’t an option. Their work happens in environments that are completely cut off from the public internet, a security measure known as “air gapping.”

The idea of “air-gapped AI” might sound like a paradox. How can you have a tool that learns and evolves with the cloud but lives in a fortress? It’s a question that gets at the heart of what real security and compliance mean in the age of AI. It’s a challenge that goes far beyond just blocking an internet connection.

What ‘Air Gapped’ Really Means for AI

Air gapped means a system that is physically or logically isolated from external networks, and all computation and updates must remain within the controlled perimeter. But the devil is often in the details. For an AI tool, that’s just the beginning. The isolation needs to be absolute. Many tools claiming to be “on-premises” or “private” still send telemetry, fetch updates or rely on external inference.

Imagine a code assistant that promises to run “on-premises.” You might assume that means it’s fully contained within your secure network. Yet, many of these tools still have a “phone home” function — a call to a remote server for updates, a check for new features or just to send a bit of anonymous usage data. Even that seemingly benign outbound connection is a dealbreaker for a truly secure environment.

For a code assistant to be truly air gapped, it has to meet a different set of standards:

Zero external dependencies: It can’t make any external API calls or rely on cloud services for anything — inference, authentication or telemetry. Every operation, from generating a code snippet to providing a contextual suggestion, must happen within your secure network.
Frozen, inspectable models: In the cloud, models are constantly being updated behind the scenes. In an air-gapped world, that’s a non-starter. You need a version-locked model whose behavior is predictable, transparent and reproducible. If an auditor asks why a piece of code was generated, you need to be able to point to a specific, auditable version of the model.
Local context only: The AI’s knowledge base can’t include anything from the outside world. It has to rely exclusively on your local repositories, project files and internal knowledge bases to provide suggestions.
Complete auditability: Every single interaction — the prompt, the generated code, the model version used — must be logged and stored locally. This isn’t just a “nice-to-have” feature; it’s the foundation for compliance and traceability in critical systems.

The Common Traps of ‘Secure’ AI Tools

Many AI tools claim to be “secure” or “on-premises,” but fail the air-gapped test. The truth is, these tools were often built for a world where some degree of external connectivity is assumed.

The most frequent pitfalls include:

Sneaky outbound connections: Some tools send snippets of code or project context out of the secure perimeter — even if they’re encrypted — to perform inference or collect data. In an air-gapped setting, any outbound connection, no matter how small, is a security violation.
Automatic, unseen updates: A vendor pushing a silent model update or changing the underlying training data can introduce unpredictable behavior. This kind of “model drift” is a nightmare for teams that need to certify and reproduce every line of code.
Lack of provenance: When an AI suggests a solution without explaining its origin, it becomes impossible to validate in a safety-critical system. Where did that code snippet come from? Was it from a public repository, a training data set or something else? Without that traceable origin, the suggestion is useless for a certified system.
Ignoring niche languages: Most AI code assistants are trained on web development languages like Python or JavaScript. But what about the languages of mission-critical systems, like Ada, VHDL or SPARK? Many AI tools simply don’t have the knowledge or validation to operate in these specialized domains.

The Realities of Deploying Air-Gapped AI

Moving an AI tool into an isolated environment is a serious undertaking. It’s not just about installing software; it’s about establishing a new architecture and set of operational processes.

Organizations need to build in several core capabilities:

True local execution: The entire AI model must run on approved hardware within your network. There can be no hidden “fallback” to a cloud service if the local inference fails.
Version control for models: You need the ability to manually update models and lock them to a specific version. This gives you complete control over what’s being used and ensures you can reproduce any past results.
Strict logging and auditing: Every interaction must be logged with a timestamp, user identity and model version. This data is the bedrock of your security and compliance efforts.
Integration with secure pipelines: The AI tool needs to fit seamlessly into your existing secure DevOps and CI/CD pipelines. It must work with your access controls, build systems and review processes.

Of course, this level of control comes with trade-offs. Running a large language model on site requires significant hardware, and the absence of a live connection means you’ll need a rigorous process for manual updates. You might also have to accept a different kind of performance — a local model may have higher latency or slightly lower quality than one running on a massive cloud server.

Ultimately, air-gapped AI tools are not about cutting corners. They are a necessity for sectors where security, sovereignty and compliance are non-negotiable. For these teams, the predictability and control offered by a truly isolated tool are well worth the added complexity.