Hosting Open-Weight LLMs on Azure with Zero Egress

American enterprises are looking hard at open-weight models, including Chinese-origin releases like Kimi K2 and GLM, because the capability per dollar is difficult to ignore. The objection arrives on schedule: what about our data going to China?

The correct answer is architectural. A model file cannot send data anywhere. Software running on a network can. So you build a network where nothing can leave, run the model inside it, and let the logs make the argument. This is that build.

The reference architecture

The enclave is an isolated Azure VNet with default outbound access disabled and an explicit deny-all on egress, enforced at the NSG and again at Azure Firewall so no single misconfiguration opens a path. Traffic enters through Private Link endpoints only; there is no public IP anywhere in the enclave, and Private DNS keeps resolution internal.

Everything the workload needs lives inside. Model weights are downloaded on a separate machine, hash-verified, and ingested offline into the enclave's storage; the serving cluster never pulls from Hugging Face or anywhere else at runtime. Containers come from a private registry that mirrors every dependency. Compute authenticates with managed identities, so there are no keys to exfiltrate. RAG indexes and fine-tuning data stay inside the boundary under the same access controls as the source systems they came from.

The proof layer

Containment you cannot demonstrate is a promise, and promises fail security review. The enclave streams VNet flow logs and firewall analytics to your SIEM from day one. Any denied outbound attempt raises an alert, which in a healthy deployment fires only during red-team exercises. Prompt and completion logging rides alongside, so you can answer both questions that matter: did anything leave, and what did the model actually do?

This is the difference between telling a customer no data goes to China and showing them a quarter of flow logs where outbound denials number zero because nothing ever tried. The second conversation is short.

What the model can and cannot do in here

Inside the enclave, the model serves inference against your governed context. It cannot phone home, fetch instructions, update itself, or reach the internet, because the routes do not exist. Guardrails handle the remaining risk surface: input and output filtering, PII redaction, injection defenses, and human checkpoints on any workflow with consequences, the same discipline we apply in agent deployments.

The trade-off is real and worth naming: no live web access means retrieval comes from your own data, refreshed through controlled, one-way ingestion jobs you schedule and audit.

Why this unlocks the open-weight catalog

Once containment is architectural, model selection becomes an engineering decision, which is where it belonged all along. We benchmark candidates, Kimi K2, GLM, Llama, DeepSeek, Mistral, on your actual tasks and score capability per dollar; at sustained volume, a self-hosted open-weight model frequently beats API pricing by a wide margin. Where each model belongs in the estate is the subject of our smart model routing piece, and the platform variant on Databricks Mosaic AI is covered in the Databricks article.

Evidence for the people who ask

The deployment ships with its audit pack: architecture diagrams, egress rule exports, flow log attestations, and control mappings to SOC 2, ISO 27001, ISO 42001, and the NIST AI RMF. Your customers' security questionnaires get answered by artifacts, not adjectives.

BD Emerson designs and builds these environments through our private LLM hosting practice, part of enterprise AI consulting. If the capability per dollar of open-weight models is tempting and the containment question is the blocker, the blocker is buildable.