“Can we use an open model so the data stays private?” sounds like one decision. It contains two. Downloadable weights determine what you can run yourself. The deployment determines who receives the prompts, where logs go, and who can inspect them.
Suppose a support assistant handles public product questions and restricted incident records. Calling a third-party endpoint that serves an open-weight model still sends data to that third party. Running the same checkpoint inside an approved environment changes the data path, even though the model and its license stay the same.
A comparison of open and closed models becomes useful once those choices are separated. Start with the rights you need and the environments you may use. Then test the candidate deployments for quality, reliability, and cost.
What does “open” actually give you?
Open-weight means the model parameters are available to download. The label alone doesn't tell you whether commercial use, modification, redistribution, or a particular application is permitted. Read the terms covering the exact checkpoint and its accompanying code.
Open-source AI is a stronger claim. Here, the term refers to the Open Source Initiative's Open Source AI Definition 1.0, not a synonym for every downloadable model. OSAID requires freedoms to use, study, modify, and share, together with parameters, complete training and inference code, and detailed training-data information under appropriate open terms.[1]
The data requirement is more precise than “publish the entire raw corpus.” It includes descriptions of unshareable data and listings of obtainable data, detailed enough for a skilled person to build a substantially equivalent system. Conversely, a short description such as “trained on web text and code” doesn't satisfy that requirement.[1]
Source-available describes inspectable artifacts whose terms restrict freedoms expected of open source. A release can be both source-available and open-weight. Closed-weight API access gives you a service interface without the parameters; it doesn't imply that every part of the provider's SDK or surrounding software is closed source.
Keep three questions separate in your evaluation notes:
| Question | Evidence to inspect |
|---|---|
| What can we access? | Weights, architecture, inference code, training code, data information |
| What may we do with it? | Exact licenses, use restrictions, attribution, redistribution and derivative terms |
| Who operates our deployment? | Hosting contract, infrastructure account, network and storage boundaries, access controls |
MIT terms on a checkpoint don't establish that its training code and data information are available. Calling it “open-weight under MIT” states the permission and access claims without claiming a reproducible training system.
Read a license before building around it
The following are representative releases checked on September 2, 2026, not a ranking or a list of the newest models. Their differences illustrate why a family name or download button is insufficient.
| Release | Published access and terms | What to verify for your use |
|---|---|---|
| Qwen3.6-35B-A3B | The official model card lists downloadable weights under Apache-2.0.[2] | Pin this checkpoint, its license files, and the code used to serve it; don't assume every Qwen release has identical terms |
| GLM-5.2 | The official model card lists downloadable weights under MIT and describes serving options.[3] | Check the released artifacts and runtime dependencies separately from claims about the openness of the entire training process |
| Llama 4 | Downloadable materials use the custom Llama 4 Community License.[4] | Review use restrictions, attribution, naming, redistribution, and the additional commercial clause |
| GPT-5.6 Sol API | OpenAI documents hosted model access and service pricing.[5] | Review service terms and data controls; API access doesn't provide a checkpoint you can deploy independently |
Llama 4's license, effective April 5, 2025, shows why the distinction matters. It incorporates an acceptable-use policy, requires attribution and “Built with Llama” display in specified distribution situations, and imposes naming duties for certain distributed models built using its materials or outputs. Its additional commercial clause applies when the licensee or affiliates exceeded 700 million monthly active users in the month preceding the release date.[4]
That is not an interchangeable substitute for MIT or Apache-2.0. Have the exact license reviewed against your planned product and distribution. A hosted service, a downloadable desktop app, and a redistributed fine-tuned checkpoint can create different obligations. This overview isn't legal clearance for any of them.
Save the license text and artifact revision with the deployment record. A model card's license badge is a starting point, not a substitute for reading the files or checking terms on adapters, datasets, and dependencies.
Deployment changes the privacy question
The support assistant could run downloadable weights on company-operated infrastructure, call an external provider serving those weights, or call an external provider serving a closed-weight model. Both external options require scrutiny of the provider's data handling; their different weight-access policies don't settle that review.
A provider may offer regional processing, private connectivity, or a dedicated deployment. Those controls have different meanings. A private network connection doesn't, by itself, remove provider access or guarantee deletion. If policy literally prohibits processing outside company-operated infrastructure, a provider contract doesn't silently change that rule.
OpenAI states that API inputs and outputs aren't used to train its models by default.[6] That answers one question. It doesn't establish zero retention, a particular processing region, or whether a given endpoint and feature meet your organization's requirements. Verify those controls separately against the current service and contract.
Self-hosting also needs a data-flow review. Prompts can escape through application traces, remote telemetry, crash reports, backups, or support tools. Record where requests, retrieved documents, outputs, and logs travel, along with who has access and how deletion works.
For the example assistant, assume restricted incident records may use only company-operated inference. Public material may use approved external services. That makes self-hosted candidates eligible for both classes and external candidates eligible only for public material. This is the example's policy, not a general law about incident data or model vendors.
Evaluate a deployment, not a model name
A checkpoint plus its precision, runtime, prompt, tools, and decoding settings is the candidate you actually deploy. A quantized checkpoint may behave differently from the full-precision release. A hosted model's managed tool implementation may differ from your own. Test that complete configuration.
Use a versioned task pack with answerable questions, missing evidence, conflicting policies, malformed inputs, and tool failures. Include rare but consequential cases instead of letting common easy questions dominate the score. Define what counts as an accepted answer before comparing systems: factual support, schema validity, permitted tool use, and whatever else the product requires.
Measure latency and quality together at the expected request mix and concurrency. A large advertised context window doesn't establish reliable retrieval from long inputs, nor does loading weights into memory establish useful serving capacity. Include KV cache, runtime buffers, warm capacity, and headroom for failures. Local LLM Deployment explains memory sizing; GPU Serving & Autoscaling covers load and queue behavior.
Open weights give you a route to checkpoint pinning and weight-level adaptation, subject to license and hardware support. They also make you responsible for validating the adapted or quantized version. Hosted services can supply managed features without transferring weights, but feature support and version guarantees need checking for the selected model. Neither ownership model guarantees better task quality.
Use LLM Benchmarks & Limitations to design comparisons. Public benchmark results can suggest candidates; they don't replace a result on your data and operating conditions.
Compare the same costs on both sides
Suppose the public workload produces 50,000 requests per day, each with 400 billable uncached input tokens and 300 billable output tokens. These counts include all tokens billed in those categories, not just text displayed to the user. Assume one attempt per request and a 30-day month.
On September 2, 2026, GPT-5.6 Sol's documented rates are $4 per million input tokens and $20 per million output tokens. The page lists higher rates for prompts above 272K input tokens, which this example doesn't use.[5] The generation-only calculation is:
- 20 million input tokens per day at $4 per million: $80/day.
- 15 million output tokens per day at $20 per million: $300/day.
- Thirty days at $380/day: $11,400.
This short-input example assumes no cache reads or writes and no batch discounts. A production calculator should use returned usage, the exact rate card, reasoning-token accounting, retries, and tool charges where applicable.
Now give both deployments an application budget and an operations budget. The amounts below, except the cited API rates, are invented assumptions for an accounting exercise. The four-GPU fleet isn't a sizing recommendation for any model named above.
| Monthly category | Hosted API | Self-hosted |
|---|---|---|
| Model execution | $11,400 in API charges | 4 GPUs × $2.50/hour × 720 hours = $7,200 |
| Application infrastructure and retrieval | $2,000 | $2,000 |
| Additional model storage and network | Included in this example's API charge | $600 |
| Allocated operations labor | $1,000 | $4,000 |
| Modeled total | $14,400 | $13,800 |
Both sides use 30 days, or 720 hours. The totals exclude taxes, paid support, human review, migration, and additional resilience capacity. Add those categories when they differ; don't call this a complete production budget.
The self-hosted total is lower, but it may produce fewer acceptable answers. Assume the same grading rules accept 96% of hosted outputs and 90% of self-hosted outputs. Those acceptance rates are hypothetical, not benchmark measurements.
1from math import isfinite
2
3def cost_per_accepted(total: float, requests: int, acceptance: float) -> float:
4 if not isfinite(total) or total < 0:
5 raise ValueError("total must be finite and nonnegative")
6 if type(requests) is not int or requests <= 0:
7 raise ValueError("requests must be a positive integer")
8 if not isfinite(acceptance) or not 0 < acceptance <= 1:
9 raise ValueError("acceptance must be in (0, 1]")
10 return total / (requests * acceptance)
11
12days = 30
13requests = 50_000 * days
14hosted_generation = requests * (400 * 4 + 300 * 20) / 1_000_000
15hosted_total = hosted_generation + 2_000 + 1_000
16self_hosted_total = 4 * 2.50 * (24 * days) + 2_000 + 600 + 4_000
17hosted_acceptance = 0.96
18self_hosted_acceptance = 0.90
19hosted_unit = cost_per_accepted(hosted_total, requests, hosted_acceptance)
20self_hosted_unit = cost_per_accepted(self_hosted_total, requests, self_hosted_acceptance)
21# Equal cost per accepted answer, with workload and monthly budgets fixed.
22break_even_acceptance = self_hosted_total / (requests * hosted_unit)
23
24print(f"Hosted: ${hosted_total:,.0f}/month; ${hosted_unit * 1_000:.2f}/1,000 accepted")
25print(f"Self-hosted: ${self_hosted_total:,.0f}/month; ${self_hosted_unit * 1_000:.2f}/1,000 accepted")
26print(f"Self-hosted break-even acceptance: {break_even_acceptance:.0%}")1Hosted: $14,400/month; $10.00/1,000 accepted
2Self-hosted: $13,800/month; $10.22/1,000 accepted
3Self-hosted break-even acceptance: 92%At 90% acceptance, the smaller monthly bill buys fewer accepted answers and has the higher unit cost. Holding both budgets fixed, the self-hosted configuration reaches equal unit cost at 92% acceptance. This is sensitivity analysis, not a prediction that tuning can achieve that rate for free.
Before using either estimate, establish that the deployment can serve the offered traffic at the required latency and reliability. An assumed fleet price proves nothing about capacity. Also apply minimum quality requirements separately: a low cost per accepted answer doesn't excuse a forbidden action or an unacceptable rejection rate. If rejected requests trigger retries, fallbacks, or human work, include their costs and count successful user tasks without double-counting attempts.
Keep fallback inside the same data policy
The support assistant could use company-operated inference for restricted records and an approved API for public documentation. It could also use the company-operated deployment for both if that proves simpler. Multiple routes are an option, not a requirement.
The subtle failure is classifying only the user's question. “Why did this upload fail?” may look public until retrieval adds a restricted customer incident. The assembled request, including conversation history, retrieved passages, tool results, and attachments, must satisfy the destination's policy.
Use trusted source labels and access-control decisions to constrain routing. A model's guess that content “looks public” shouldn't authorize an external send. Unknown classification should stop or stay within the stricter approved environment, not default to a more capable external model.
Test the outage path explicitly. If the internal model times out on a restricted request, an external fallback remains ineligible. Return a controlled failure, queue the request if the service contract allows it, or use another approved internal deployment. Redaction is a separate transformation that needs validation and authorization, not an automatic exception to the rule.
Record the destination, policy version, source classifications, eligibility result, and fallback reason without copying restricted content into an unrestricted audit system. Model Gateways, Routing, and Fallbacks develops these checks in more detail.
Write down what would change the decision
A short decision record should identify the checkpoint or API version, licenses, approved data classes, measured quality, load test, budget, and operational owner. Include a condition for reconsidering the choice: a new privacy requirement, failed latency target, changed provider terms, or sustained traffic that justifies a different capacity plan.
For the example assistant, restricted requests exclude external processing because the policy says so. Public requests leave more candidates to compare. Open weights offer independent deployment and adaptation options; managed APIs transfer much of the inference work to a provider. The useful answer names which control the product needs, what that control costs, and how the team verified it.