On-Device AI as a Privacy Control: What We Learned Inside a Firewall

Key Takeaways

• Where a vendor runs inference has become a procurement question, and it now gets asked before anyone asks how good the model is.
• Network telemetry is close to the worst payload you can hand to a third-party model. A stream of connection metadata describes what software a person runs, when they work, and how much data they move.
• Apple, Microsoft, and Google all ship a 3-billion-parameter-class model on the endpoint at no inference cost, with a capability ceiling you have to design around.
• The rule that kept our own local-model feature safe: the model explains, the enforcement path decides. A wrong sentence stays a UX problem instead of becoming a policy bypass.
• Any local-AI feature needs a complete non-AI product underneath it, because a large share of your install base will not meet the hardware bar for years.

People could see a process name and a number and still have no idea what to do with either. rapportd means nothing outside Apple’s documentation. A note-taking app moving steady background traffic while nobody is typing looks alarming and is completely ordinary. We watched beta users reach the same two conclusions over and over: block everything unfamiliar, or stop looking at the on-device list entirely. Both outcomes make a security tool useless.

The obvious 2026 fix is to send the process name, the destination, and the volume to a hosted model and stream back a sentence in plain English. We took that design apart in an afternoon and threw it away, because it would have turned a privacy tool into the largest exporter of the user’s behavioral data on their machine. What replaced it turned out to be a broader lesson about where inference belongs in any product that touches sensitive signals.

The explanation gap in security tooling

A verdict without context does not produce a decision. It produces a guess.

This is the same failure that has dogged enterprise security for a decade under the name alert fatigue, the one that pushes organizations toward zero-trust architectures in the first place. Consumer security software inherits it in a harsher form, because there is no analyst on the other side. A dashboard that lists forty processes with network activity is asking an ordinary person to adjudicate forty cases they have no basis to judge. The predictable result is over-blocking, and over-blocking has a cost that shows up immediately: a blocked sync engine, a document that will not save, a video call that freezes. Users learn that the security tool breaks their work, and they turn it off.

We build Mac utilities, and in December 2025 we shipped FireWally, a free macOS firewall that shows which applications open network connections, how much traffic each one moves, and lets you cut any of them off with one toggle. The engineering problem we expected was traffic interception. The problem that actually shaped the product was explanation.

Our operating rule for users has been simple: do not touch what you do not understand and what does not bother you. If your calculator sends large volumes of data to an unfamiliar destination, that deserves attention. If your text editor reaches a cloud provider, that is how the software works. The gap between those two cases is exactly where an explanation earns its place, and it is why we wanted a language model in the product at all.

Why the explanation could not leave the device

Think about what a connection feed actually is once it accumulates. It is an inventory of the software installed on a machine, a timestamped record of when that machine is in use, and a volume series that reveals which tools do heavy work. Correlate a week of it and you have a behavioral profile of a person and a reasonable sketch of their employer’s stack. That is not a byproduct of the security signal, it is the security signal.

Shipping that to a third-party API to generate a friendly sentence would have recreated the exact exposure the product exists to remove. It would also have to be declared in the App Store privacy label, where “we collect your app usage data” sits directly above the download button of a privacy tool.

The market has arrived at the same conclusion from the compliance side. IBM’s 2025 Cost of a Data Breach report put breaches involving shadow AI at one in five organizations, carrying an average of $670,000 in additional cost, and found that 97% of organizations with an AI-related incident lacked AI access controls. Cisco’s 2026 Data and Privacy Benchmark Study, based on responses from 5,200 privacy and security professionals, found 81% of organizations facing heightened demand for data localization and 90% expanding their privacy programs because of AI. Buyers are already asking where the processing happens. Answering “on the device, and here is why that is structural” is a stronger position than any accuracy benchmark.

What “on-device” means in 2026

The choice stopped being aspirational around the middle of last year. Apple’s Foundation Models framework opened the roughly 3-billion-parameter model behind Apple Intelligence to any app on iOS 26, iPadOS 26, and macOS 26, with inference free of cost, available offline, and with guided generation that forces the model into a response shape your code can rely on. Apple’s own framing is blunt: prompts are generated entirely on device, so the user’s entries stay personal.

Microsoft ships a comparable local model to Windows through the Windows App SDK, currently Phi Silica at 3.3 billion parameters running on the NPU of Copilot+ PCs, with a successor model already scheduled to replace it during the autumn. Google exposes Gemini Nano to Android apps through ML Kit’s GenAI APIs, where input, inference, and output data are processed locally and the feature keeps working without a connection.

Between fully local and fully hosted there is now a third tier worth understanding, because it is where the harder workloads will live. Apple’s Private Cloud Compute runs larger models on Apple silicon servers under a guarantee that user data “must not be retained, including via logging or for debugging, after the response is returned,” and devices will send data only to nodes that can “cryptographically attest to running publicly listed software.” Whatever you think of the vendor, that is the shape of the claim a serious buyer should be demanding: verifiable rather than promised.

	On-device model	Attested private cloud	Hosted model API
What leaves the machine:	Nothing	Request payload, under attestation	Request payload
Retention risk:	None	Contractual plus technical controls	Vendor policy and logging
Capability ceiling:	Summarize, classify, explain	Near frontier	Frontier
Marginal cost per call:	Zero	Vendor cost	Vendor cost
Works offline:	Yes	No	No
Coverage of your install base:	Recent hardware only	Any connected device	Any connected device
Audit story:	Strongest	Verifiable if attested	Contractual

What the local model costs you

Free inference is not free engineering. Four constraints shaped our build, and they will shape yours.

The capability ceiling is real. A 3-billion-parameter model with a small context window is good at summarizing, classifying, and explaining a compact input. It will not reason across a large corpus, and it will not replace your detection logic. The job has to be scoped down to something the model can do reliably on its worst day.

Coverage is the constraint nobody plans for. FireWally supports macOS 13 and later. The AI explanations require macOS Tahoe 26 on a Mac with an M1 chip or newer. That gap covers a substantial part of the install base, and it will take years to close. Microsoft’s documentation makes the same point from the other side: the GPU variant of its local model is downloaded on demand at several gigabytes, users can remove it in system settings, and the API returns an explicit “not supported on this system” state that developers are told to handle with a fallback. Design the product so the feature is additive. If your value proposition collapses on a 2019 machine, you have built a demo.

Inference is a power budget. On a laptop, CPU and NPU time is battery life, and a background security process is judged on whether the fan spins. Our discipline on this predates the model: when new traffic arrives, we update a single value for a single application rather than rebuilding the list. The same instinct applies to generation. Explanations are produced on demand, for one application, when a person is actually looking at it.

You lose your telemetry, deliberately. With no server-side prompt logs, you cannot watch how the feature behaves in the field, cannot A/B a prompt across users, and cannot hot-fix a bad output. Improvement happens through local instrumentation, support reports, and the next release. That is a genuine engineering tax, and it is the direct consequence of the guarantee you are selling.

The model explains, the enforcement path decides

The design rule that made all of this safe is worth stating precisely, because it generalizes to every security product now adding a model.

FireWally’s verdicts live in the network extension, which is deterministic code. A connection is allowed or blocked according to rules and user choices, and the same input produces the same output every time, whether or not Apple Intelligence exists on that Mac. The language model sits above that boundary, in the interface, producing human-readable context about an application that has already been observed. Its output informs a person. It never becomes a policy.

The consequence is that the worst-case failure of the AI feature is a sentence that is wrong or unhelpful. That is a user-experience defect. It is not an allowed connection, a bypassed rule, or a non-reproducible security decision that no one can audit after the fact. Any team wiring generated text into a product that enforces something should draw this line explicitly, on a diagram, and defend it in review.

Five questions before you approve a local-AI feature

1. What is in the payload? Read one real request out loud in a meeting. If you would object to seeing it in a vendor’s logs, the inference belongs on the device.
2. How big is the job? Summarize, classify, and explain fit today’s local models. Multi-step reasoning over a large corpus does not.
3. What percentage of the fleet qualifies? Get the number from your own install base, not from a vendor’s device list, and decide what the remainder experiences.
4. Can generated output change an enforcement decision? If yes, redesign until the answer is no, or accept that your control now has a non-deterministic component.
5. How would you prove it? “Processed locally” is worth what your evidence is worth: architecture documentation, store privacy labels, attestation for any cloud tier, and an outbound connection profile anyone can verify.

What to ask vendors now

Security and IT leaders evaluating any AI-enabled endpoint tool this year can get most of the way with six questions:

• Where does inference run for each AI feature, named individually
• What leaves the device, and who retains it, for how long
• What happens on devices that cannot run the local model 
• Does model output influence any enforcement or classification decision, or only presentation
• Can the AI feature be disabled centrally without degrading the core control 
• And for anything in a private cloud tier, is there attestation a third party can check, or a paragraph in a contract

Those answers separate products that treat local processing as an architecture from products that treat it as a marketing line.

Where this is heading

NPUs are becoming standard silicon, the local models on them are improving faster than the hardware refresh cycle, and the regulatory pressure behind data localization is not easing. The result is that “processed on your device” is moving from a differentiator toward a baseline expectation for anything touching behavioral telemetry, health data, or communications. The interesting competition will be over the middle tier, where attested cloud processing has to earn trust that the endpoint gets for free.

For teams building in this space, the practical takeaway is smaller than the trend. Decide where each piece of inference runs before you write the feature, keep generated language on the explanatory side of your enforcement boundary, and make sure the product is worth using on a machine that will never run a model at all.

Read Next

A few related pieces worth your time:

• Scheduling Tickets and Licenses on a Field Service Management Platform
• 8 Best AI Red Teaming Platforms for Application and API Testing
• Best Inbound Call Center Software by Team Size (2026)

FAQ

Is on-device AI inherently more secure than cloud AI?

It removes one specific risk: the payload never leaves the machine, so it cannot be retained, subpoenaed, or breached at a vendor. Other risks remain. A local model can still produce wrong output, and an application that runs a model locally can still transmit data through some other path.

Does local inference actually cost nothing?

There is no per-request API charge on Apple’s, Microsoft’s, and Google’s on-device frameworks. The costs move to storage, memory, power, and engineering time for fallbacks. On laptops, the power budget is the one users notice.

What should happen on devices that cannot run the local model?

The core product should work at full value without it. Apple’s and Microsoft’s APIs both expose an explicit availability state, and both expect developers to branch on it. Treat the AI path as an enhancement to a complete product.

Can a local model leak data anyway?

Not through inference, but the surrounding application can send anything it likes. This is verifiable rather than theoretical: watch the app’s own outbound connections and see whether the claim holds. A network monitor is a reasonable way to audit the tools that promise you privacy, including ours.

Should an AI model make blocking or allow decisions?

Keep it out of the enforcement path. Deterministic rules can be tested, reproduced, and explained to an auditor. Generated text cannot, and a security control that behaves differently on identical inputs is difficult to defend after an incident.

How do we demonstrate local processing to an auditor or customer?

Architecture documentation naming each inference site, platform privacy labels, attestation reports for any cloud tier, and an observed network profile of the application under normal use. The last one is the piece most teams skip, and it is the one that settles the argument.