Privacy by Architecture

Introduction

For most of the last decade, AI and cloud were synonymous. You uploaded your data, a remote model processed it, and the result came back to your device. The architecture was simple: device is terminal, cloud is brain.

That model worked. It scaled. It became the default.

But it quietly introduced a problem that no privacy policy could solve. When you upload medical data, a prescription, a diagnostic report, a list of conditions, that data travels. It leaves your device. It hits a server. It gets logged, cached, and processed by infrastructure you don't control. The best privacy policies in the world can still only describe what happens after that data arrives somewhere else.

This article explains how that premise reshapes architecture, and why it matters more in healthcare than almost anywhere else.

Privacy is Usually Treated as Policy

Most software products approach privacy through policy. They encrypt data in transit and at rest. They request explicit consent before collection. They publish statements describing what they collect, how they use it, how long they retain it, and who they share it with.

These measures matter. Encryption protects data from interception during transmission. Consent mechanisms build user trust and create legal accountability. Privacy statements, when written honestly, help users understand the trade-offs they're accepting.

But every one of these measures shares a structural limitation: they apply after the data has moved.

A privacy policy cannot undo an upload. Encryption protects a transmission channel, but the data still arrives at a server, where it can be decrypted, processed, retained, and potentially breached. A consent form asks you to agree to a transfer, not to prevent one. "We encrypt your medical data" means the data was transmitted. "We delete it after 90 days" means it existed on someone else's infrastructure for at least 90 days.

This is not a criticism of encryption or consent. They are necessary and should exist regardless of where processing happens. It is an observation about their scope.

Privacy Becomes Architecture

What if the question changed?

Instead of: "How do we protect uploaded medical data?"

Ask: "Can we avoid uploading it altogether?"

This shift is not cosmetic. It changes the architecture of the entire system.

When you design a system around the constraint that sensitive data must not leave the device, you eliminate an entire category of risk. You don't need to secure a transmission that doesn't happen. You don't need to manage a server-side data store that doesn't exist. You don't need to worry about a breach of data that was never stored remotely.

It's a stricter constraint. It limits what the system can do without a network connection. Some cloud-only capabilities become impossible. But in healthcare, where data is deeply personal, re-identification risk is real, and the consequences of exposure can be severe, that constraint produces a genuinely better outcome for the user.

Local AI Changes Everything

Until recently, this architectural choice would have required giving up most AI capabilities. Meaningful AI inference requires significant computation. Consumer devices were too slow. Models were too large. Battery consumption was prohibitive at useful accuracy levels.

Modern smartphones have changed this equation.

Today's flagship devices include dedicated Neural Processing Units (NPUs), purpose-built silicon for accelerating machine learning inference. Apple's Neural Engine, Qualcomm's Hexagon NPU, and Samsung's Exynos NPU can perform hundreds of billions of operations per second at a fraction of the energy cost of general-purpose CPUs or GPUs doing the same work.

Language and vision models have also become dramatically more efficient. Quantization reduces model weights from 32-bit floats to 4-bit integers, shrinking model size by 8× with modest accuracy loss. Distillation trains smaller models to replicate the behaviour of larger ones. Architecture innovations, attention variants, mixture-of-experts, state-space models, compress capability into fewer parameters.

The result: capable AI models that run in 1–4 gigabytes of RAM and complete inference in milliseconds on hardware that fits in a pocket.

For FarmakoMed, this makes a specific set of capabilities practical to run entirely on the device:

• Document understanding. Reading a prescription or discharge summary requires parsing structure, recognising medical terminology, and extracting key information. OCR combined with lightweight NLP handles this without leaving the device.
• Semantic search. Finding the right document or medication entry from a natural-language query, "what did my cardiologist prescribe in March?", requires semantic understanding beyond keyword matching. Embedding models small enough to run on-device make this fast and private.
• Classification. Routing a scanned document to the right category, or recognising the type of medication from a photo, is a classification task. On-device vision models handle this in under a second.
• Medication recognition. Scanning a medication box and extracting the active ingredient, dosage form, and strength involves combined vision and text processing. Modern on-device multimodal models are increasingly capable here.
• Health assistant. Answering a question about a medication's common side effects or administration timing requires language understanding. Small instruction-tuned models can provide contextually useful responses without sending the question to a remote server.

Beyond Privacy: The Engineering Case

Privacy is the most visible benefit of on-device AI. It is also the most compelling argument in healthcare. But there are several other engineering reasons to prefer local inference that are worth understanding independently.

No cloud inference costs

Every API call to a hosted AI model has a marginal cost. At modest scale, this is manageable. At millions of users running daily inference, the numbers compound into a significant operating expense. When inference happens on the user's device, the application developer bears no per-inference cost. The user's hardware does the work. The economics shift fundamentally, and in favour of more generous usage.

No upload latency

Sending data to a remote server, waiting for the inference to complete, and receiving the result back introduces latency that is measurable and visible to users. On-device inference is immediate, the processing happens in the same address space as the application, and results are available in milliseconds. For interactive use cases like document scanning or medication identification, the difference between 50ms and 800ms is qualitatively significant.

Works offline

Healthcare is not always convenient. A patient checking a medication at a hospital basement, on a flight, or in a rural area with poor signal still needs their app to function. On-device AI is available wherever the device is, no connectivity required after the initial model download. For a health application, this is not a nice-to-have. It is a reliability requirement.

Improved resilience

Cloud-dependent AI features fail when the server is unavailable, during maintenance windows, traffic spikes, or infrastructure incidents. On-device features continue working regardless of what is happening on the backend. Each device is a fully autonomous compute unit. For a health application where a user may be relying on it at a critical moment, this resilience matters.

Different scaling characteristics

Computation happens on the user's device. As your user base grows, the AI processing load does not accumulate on your servers. Each new user brings their own compute with them. This doesn't eliminate infrastructure costs, you still need servers for sync, backup, account management, and telemetry, but it fundamentally changes the relationship between user growth and operating cost.

Real Engineering Challenges

Privacy by architecture is the right approach. Building it honestly is harder than building a cloud-connected equivalent. Here are the challenges we have encountered, described at a level useful to engineers thinking about similar problems, without exposing proprietary implementation details.

Memory management

On-device models occupy RAM. Modern smartphones have generous amounts of memory, but the OS manages it aggressively on behalf of all running processes. A model loaded for inference may be evicted at any time when another application or the OS needs memory, and on mobile, "any time" includes mid-inference.

Robust on-device AI requires careful lifecycle management: loading models only when needed, releasing them promptly when not in use, handling eviction events gracefully, and never corrupting user-visible application state when a model is unavailable. This is different from server-side AI, where the model is always in memory and the infrastructure team manages capacity.

Model downloads and integrity

AI models are large files, often hundreds of megabytes, sometimes several gigabytes. Downloading them over mobile connections, which may be slow or intermittent, requires robust handling of partial downloads, resumption on reconnection, and integrity verification before use. A partially downloaded model that passes a size check but fails during inference is worse than a clearly missing model. It can produce silent failures or crashes that are difficult to diagnose.

Battery impact

AI inference is computationally intensive. Without care, it creates visible battery drain during active use. Effective on-device AI requires profiling inference energy consumption on representative hardware, preferring NPU execution paths over CPU or GPU fallbacks, limiting inference to foreground use (or brief, bounded background sessions), and batching operations where the user experience allows it.

Background execution constraints

Mobile operating systems restrict what applications can do when they are not in the foreground. Long-running inference tasks started in the background may be suspended, rate-limited, or killed entirely. Applications must design AI tasks to be interruptible, checkpointable, and resumable, which means treating every local inference operation as one that might not complete.

Diagnostics without server-side visibility

One of the less obvious challenges of privacy-first development is debugging. In a cloud architecture, server logs provide rich diagnostic information about failures, latency distributions, and error rates. On-device, the "server" is the user's phone, and you may never see its logs.

Diagnosing a model loading failure, an inference crash, or an unexpected accuracy regression requires a different approach: structured local logs, opt-in crash reporting that transmits no health data, and diagnostic tooling that can reconstruct the state of the inference pipeline from what little information the user can share.

These are solvable problems. They are being solved, across the industry and within FarmakoMed. But any engineer approaching on-device AI should plan for them from day one, not discover them after shipping.