Hybrid AI for Confidential Data: Local Open-Weight Models for Private Work, Frontier Models for Research

A hybrid AI setup, as a design pattern, splits work between two kinds of models. In this design, open-weight models such as Qwen, LlamaLlama — Llama (Meta open-weight LLM family)Meta's family of open-weight large language models (Llama 3.1, Llama 3.2, and successors). Released under a community license with a 700M monthly active user threshold, widely used for self-hosted and fine-tuned enterprise deployments., Mistral and Gemma run on hardware your organization controls, and hosted frontier models such as Claude, ChatGPT and Gemini handle harder reasoning and research on current, public information, often with web search. Confidential, intellectual-property and financial information stays local; only material cleared for disclosure goes to a hosted model.

Three earlier posts cover neighbouring ground. Private LLM Deployment looks at going fully on-premise, Multi-Model AI Strategy looks at choosing models for quality and cost, and Fine-Tuning vs RAG looks at customization. This one covers privacy routing: what the vendors’ own terms say happens to business data, what open-weight licences allow, and which request goes where.

Treat this as a design pattern, not a survey of what companies do. Vendor terms below are quoted from each vendor’s own page as of November 2026; they change, so read the current version before relying on it.

Anthropic. Anthropic’s privacy centre article on model training (dated August 18, 2026) says: “By default, we will not use your inputs or outputs from our commercial products (e.g. Claude for Work, Anthropic API, Claude Gov, etc.) to train our models.” Anthropic’s Commercial Terms of Service, effective June 17, 2025, state: “Anthropic may not train models on Customer Content from Services.”

OpenAI. OpenAI’s platform data controls documentation states: “As of March 1, 2023, data sent to the OpenAI API is not used to train or improve OpenAI models (unless you explicitly opt in to share data with us).” For ChatGPT workspaces, OpenAI’s ChatGPT Work cloud security documentation says: “Business, Enterprise, and Edu workspace data is encrypted in transit and at rest and is not used to train OpenAI models by default.”

Google. Google’s Gemini API Additional Terms of Service (effective March 23, 2026) draw a line between unpaid and paid use. For Unpaid Services, such as direct use of Google AI Studio and the unpaid Gemini API quota, Google uses submitted content and responses “to provide, improve, and develop Google products and services and machine learning technologies,” human reviewers may read it, and the terms say: “Do not submit sensitive, confidential, or personal information to the Unpaid Services.” For Paid Services, the terms say Google “doesn’t use your prompts … or responses to improve our products.”

Google Cloud’s zero data retention documentation for its Gemini Enterprise Agent Platform says Google “won’t use your data to train or fine-tune any AI/ML models without your prior permission or instruction.” For Gemini in Google Workspace, Google’s Generative AI in Google Workspace Privacy Hub says: “Your content is not human reviewed or otherwise used for Generative AI model training outside your domain without permission.”

In each case the no-training commitment attaches to a business or paid product. A staff member who pastes a client file into a personal or free account can be working under different terms: Anthropic’s training article points consumer plans to a separate article, and Google’s Gemini API terms say not to submit confidential information to its Unpaid Services.

Not training on your data is not the same as not keeping it.

Retention. Anthropic’s commercial data retention article (dated July 1, 2026) says: “For Anthropic API users, we automatically delete inputs and outputs on our backend within 30 days of receipt or generation,” with listed exceptions including a zero data retention agreement, Usage Policy enforcement and legal requirements. Anthropic’s API data retention documentation words its default differently: “Conversation content (your prompts and Claude’s outputs) is not retained by default; the exception is Covered Models, which require 30-day retention.” The two Anthropic pages do not describe the default in the same terms, so confirm the retention that applies to your account and models with Anthropic. The same documentation says that under a zero data retention (ZDR) arrangement, “Anthropic does not store customer prompts or responses at rest after the API response is returned,” and that ZDR is enabled per organization on request. OpenAI’s platform data controls documentation says abuse monitoring logs “are generated for all API feature usage and retained for up to 30 days” by default, and that approved customers “may select between Modified Abuse Monitoring or Zero Data Retention.” Google’s Gemini API terms say that for Paid Services, Google “logs prompts and responses for a limited period of time, solely for detecting and preventing violations of the Prohibited Use Policy to maintain the safety and security of the Services, and any required legal or regulatory disclosures,” and that this data “may be stored transiently or cached in any country in which Google or its agents maintain facilities.”

Residency for Canadian data. As of November 2026:

- OpenAI’s API data controls documentation lists the Canada endpoint (ca.api.openai.com) with regional storage but not regional processing, says that using any region other than the United States requires approval for abuse monitoring controls and a Modified Retention amendment, and says that where a region does not support regional processing, “OpenAI may also process and temporarily store Customer Content outside of the Region to deliver the services.” - Anthropic’s data residency documentation lists two inference settings, “global” (the default) and “us”, and says that “us” is currently the only available workspace geo for data stored at rest. - Google Cloud’s data residency documentation includes Canada (northamerica-northeast1) among the locations in its per-model processing tables, and says requests to a global endpoint “may be processed in any Google Cloud location around the world.”

Web search changes the picture. Google Cloud’s zero data retention documentation says Grounding with Google Search stores queries derived from end-user prompts for up to three days, that “There is no way to disable the storage of this information if you use Grounding with Google Search,” and that customers who require zero data retention should consider Web Grounding for Enterprise instead. Anthropic’s documentation notes for its web fetch tool that “Website publishers might retain request data (such as fetched URLs and request metadata) according to their own policies.” A search query can itself reveal a deal name or a client, so treat outgoing queries as data leaving your control.

Open weights let you run a model where your data lives, but the licence still governs use, and licences differ by model, not only by vendor.

- Qwen. On the Qwen organization’s Hugging Face page, the Qwen3.8-27B model card lists the licence as Apache 2.0. Not every Qwen model uses it: the Qwen3.8-Flash-Next card uses the Qwen Community License 1.0, which says a licensee that “conducts a Model as a Service or AI Work Assistant business” must obtain a separate licence from Qwen before commercial use, while exempting internal use that does not make the software, its outputs or its model capabilities available to any third party. - Llama. Meta’s Llama 4 Community License Agreement (effective April 5, 2025) says that if a licensee’s products or services had more than 700 million monthly active users in the calendar month before the Llama 4 release date, the licensee “must request a license from Meta.” Meta’s licence also requires anyone who distributes or makes available the Llama materials, or a product or service that contains them, to display “Built with Llama”. - Mistral. Mistral AI’s Hugging Face model cards show mixed licensing. The Mistral Small 4 card says the model “is licensed under the Apache 2.0 License.” The Mistral Large 4 card uses the Mistral Ecosystem Open-Weight License, under which “operating a Model-as-a-Service business requires a separate commercial license with Mistral.” The Mistral Medium 3.5 card uses a Modified MIT License “with exceptions for companies with large revenue.” - Gemma. Google’s Gemma 4 12B model card on Hugging Face lists the licence as Apache 2.0.

Runtimes, as their own documentation describes them:

- Ollama’s README says: “Ollama has a REST API for running and managing models.” Ollama also offers hosted cloud models; its cloud documentation says: “To use only local models, disable cloud features.” - vLLM’s README calls it “a fast and easy-to-use library for LLM inference and serving” and lists an “OpenAI-compatible API server” among its features. - llama.cpp’s README says its main goal “is to enable LLM (and VLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.”

Ollama’s OpenAI compatibility documentation says it “supports a subset of the OpenAI API,” so one application can send each request to a local server or a hosted APIAPI — Application Programming InterfaceThe secure software bridge linking your existing data directly into advanced AI models..

The Government of Canada’s Guide on the use of generative artificial intelligence, written for federal institutions, warns that “The suppliers of some generative AI tools may inspect input data or use it to further train their models.” It advises: “Don’t enter sensitive or personal information into any tools not managed by the GC,” and says “Public servants must not input personal information into publicly available online generative AI tools.” It adds: “Public servants may, however, input personal information into generative AI systems that are controlled or configured by the government when the appropriate privacy and security controls are in place.” A business can draw the same line.

Canada’s federal, provincial and territorial privacy regulators, in the principles for generative AI published by the Office of the Privacy Commissioner of Canada on December 7, 2023, tell organizations using generative AI: “Where possible and reasonable, use anonymized or de-identified information within prompts to a generative AI system rather than personal information.” They add: “Unless otherwise required, prompts should not be retained, used for secondary purposes, or disclosed.”

NIST’s Generative AI Profile (NIST AI 600-1), published July 26, 2024, lists Data Privacy as a generative AI risk: “Impacts due to leakage and unauthorized use, disclosure, or de-anonymization of biometric, health, location, or other personally identifiable information or sensitive data.”

As a design, the pattern has five steps:

1. Classify. Use your existing data-classification scheme, or write one with tiers such as restricted (client personal information, unreleased financials, source code, trade secrets), internal and public. 2. Route by tier. Restricted work goes only to the local model. Internal work goes local, or to a frontier business plan whose terms you have checked for training, retention and region. Public or sanitized work can go to a frontier model with web search. 3. Sanitize before anything crosses. Replace names, account numbers and deal names with placeholders and keep the mapping local. A local model can propose redactions; a rule check or a person confirms them. 4. Split the task. Ask the frontier model the public half (for example, what a published standard requires, with sources), then give that answer to the local model with your confidential documents. 5. Lock the plumbing. Allow business accounts only, log what crosses the boundary, and set retention schedules for prompts and outputs on both sides.

Capability. This post makes no claim about how any open-weight model compares with a hosted frontier model. Test the candidates that fit your hardware on your own work before deciding what stays local.

Operations. Running locally moves security, patching, access control and logging onto your team. A local model is only as private as the server, network and accounts around it.

Sanitizing is your job. No vendor term quoted above redacts your data for you. The privacy regulators’ principles tell organizations using generative AI to “Avoid prompting a generative AI system to re-identify any previously de-identified data,” and NIST names de-anonymization among the data privacy risks.

Terms move. Several of the vendor and government pages quoted here are dated 2026. Schedule a recurring review.

List the AI tools and accounts staff use today. Check each business plan’s current terms for default training, retention, zero data retention and inference location. Then classify your data and pilot the split on one workflow that mixes public research with confidential material.

HCC’s Generative AI Strategy & Integration service can help you design how local and hosted AI tools fit your existing systems and data rules. HCC’s AI Governance & Compliance service can help you turn the routing rules into written policy.

Anthropic — Is my data used for model training? Anthropic — Commercial Terms of Service Anthropic — How long do you store my organization’s data? Anthropic — API and data retention Anthropic — Data residency OpenAI — Data controls in the OpenAI platform OpenAI — ChatGPT Work cloud security Google — Gemini API Additional Terms of Service Google Cloud — Gemini Enterprise Agent Platform and zero data retention Google Cloud — Data residency Google — Generative AI in Google Workspace Privacy Hub Qwen — Qwen3.8-27B model card Qwen — Qwen Community License 1.0 (Qwen3.8-Flash-Next) Meta — Llama 4 Community License Agreement Mistral AI — Mistral Small 4 model card Mistral AI — Mistral Large 4 model card Mistral AI — Mistral Medium 3.5 model card Google — Gemma 4 12B model card Ollama — README Ollama — Cloud documentation Ollama — OpenAI compatibility vLLM — README llama.cpp — README Government of Canada — Guide on the use of generative artificial intelligence Office of the Privacy Commissioner of Canada — Principles for responsible, trustworthy and privacy-protective generative AI technologies NIST — AI 600-1, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile NIST — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (publication page)