We Tried a Bigger Model. Here's Why We Shelved It.

Introduction

Health AI Developer Foundations, Google's family of medically fine-tuned open models, shipped a 4-billion-parameter vision-and-text variant of MedGemma in a size small enough to consider running entirely on a phone. For an app built around exactly that constraint, on-device extraction from photographed medication packaging, that's not a footnote. It's the kind of release that makes you stop and ask a direct question: is this simply a better version of the model we already ship?

The honest answer took an entire evening to earn, and it wasn't the one anyone expected going in. Getting there required fixing a design gap that had nothing to do with the new model, chasing two leads that looked exactly like real bugs and were not, watching a phone visibly protest under real load, and eventually discovering that our own test of the model had been rigged from the start, by us, weeks earlier, for reasons that had nothing to do with this model at all. This is that evening, in order.

FarmakoMed's default model, a Gemma 4 E2B profile, is deliberately sparse: it activates a fraction of its parameters per token, which is precisely why it runs comfortably on a wide range of Android hardware. MedGemma 1.5 4B is not sparse. It's a dense model, medically fine-tuned, with a genuine multimodal variant built around a SigLIP vision encoder for reading images. On paper, more parameters and domain-specific training sounds like a strict upgrade for a feature whose entire job is reading medication labels. Whether that holds up once it's the phone in your pocket doing the work, rather than a benchmark table, is a different question, and it's the one this evening was actually about.

Chapter 1 — The One-Way Door

The app already had the right shape for this kind of evaluation. A model registry held more than one profile, a settings screen let a user opt into a non-default model, and the download/install pipeline was already model-agnostic. Selecting MedGemma from that settings screen, watching it download, and letting the app warm it up on next use looked like a fully built, low-risk path to finding out.

It wasn't. The first sign was mundane: after tapping the new model in settings and watching the download screen, nothing happened. No progress bar, no error, no retry button. Just a screen that looked like it was thinking, indefinitely.

The actual bug, once found, was a scoping mistake rather than an async one. The settings screen's "select this model" action intentionally avoided persisting the pick as the user's canonical selection until setup finished, a reasonable design goal on its own. But the download screen's "should I start downloading automatically?" check read the user's canonical, already-persisted selection to decide whether the target model was already installed, rather than the model that setup actually had in flight. Every time, that check quietly answered "yes, the currently selected model is already installed", because the currently selected model was still the old one. The auto-start effect ran, correctly, on every render, and correctly concluded there was nothing to start. Nothing was broken loudly enough to log.

The fix was small once found: read the target model that setup is actually tracking, not the user's persisted default selection, in the three places the download screen had conflated them. But the bug that mattered more, structurally, showed up one step later, once the download actually completed and the model was, for the first time, genuinely installed and selectable.

An unrelated effect, written for an entirely different purpose, warmed the AI runtime automatically the moment the app judged a model "ready" on cold start, so the first scan of a session wouldn't have to wait through a cold load. It did this unconditionally, for whichever model happened to be the user's persisted selection, with no distinction between the app's own vetted default and whatever the user had most recently tapped in an experimental picker.

That single unconditional call is the actual subject of this article. Once MedGemma was the selected model, it wasn't warmed once, on request, as an evaluation. It was warmed automatically, unattended, on every single app launch, because the selection persisted and the launch effect had no opinion about which model it was launching. There was no button, no confirmation, and, as it turned out over the next hour, no way back in through the UI if that automatic warmup ever made the app itself unresponsive.

Chapter 2 — What the Phone Told Us

None of that was obvious yet. What was obvious, within minutes of the model actually warming up for the first time, was that the phone itself had opinions about what was happening, and it started expressing them before any log line did.

The screen went dark on its own. When it came back on, the AI feature indicator had turned red, unavailable. The device was warm to the touch, the kind of warm that a phone doing genuinely heavy, sustained compute gets, not the kind from normal use. A moment later, a connected Bluetooth accessory dropped its connection outright. Then the development bridge used to watch the device's logs over the network went offline entirely, the connection simply stopped responding.

None of that is a log line. It's a phone quietly telling you, through everything except the app you're debugging, that something is monopolizing its resources badly enough to starve the radios and the UI thread at the same time. A CPU pegged across every core for a sustained stretch will do exactly that: Bluetooth and Wi-Fi stacks share scheduling with everything else on the device, and a UI thread that can't get a timeslice doesn't just feel slow, it stops responding to touches at all, including the touch meant to cancel whatever's causing the problem.

This is the moment the missing off-ramp from Chapter 1 stopped being a theoretical design gap and became the actual incident. The in-app control meant to switch back to the known-good default model was on the other side of a UI thread that had nothing left to give it. And because the model selection was already persisted, closing the app and reopening it didn't help either; the automatic cold-start warmup simply started the same unattended attempt again, on the exact model that had just made the device unresponsive.

Chapter 3 — Two Red Herrings

Under that kind of pressure, the obvious hypothesis is that something is hanging, forever, and needs a timeout. That hypothesis turned out to be half right, in a way that took two separate wrong turns to sort out.

The first wrong turn looked, from the logs, exactly like a race condition. Two runs of the same startup check produced log lines out of order relative to each other: a fast-path completion appeared to finish before a slower branch's own "still working" warning had even fired, which is only possible if two independent executions were interleaved. The fix that followed, making that startup check single-flight so concurrent callers share one execution instead of racing, was a real, defensible correctness improvement on its own merits. It also, on its own, changed nothing about the actual symptom.

The second wrong turn is the more interesting one, because it's a genuinely easy mistake to make with any timeout built on a common pattern: race a real operation against a timer, and resolve with whichever finishes first.

sequenceDiagram
    participant Code
    participant RealCheck
    participant Timer
    Code->>RealCheck: start
    Code->>Timer: start (5000ms)
    RealCheck-->>Code: resolves (+20ms)
    Note over Code: race settled, code moves on
    Timer-->>Timer: fires anyway (+5000ms)
    Timer-->>Code: logs "timeout" (misleading)

A timeout guard built this way, racing the real call against a timer that resolves to a safe default, is a genuinely good defensive pattern for a native call with no built-in bound. It had already protected this exact codebase from a real, previously-shipped hang. What it doesn't do, unless you explicitly call for it, is cancel the timer once the real call wins the race. Left alone, that timer keeps running on its own schedule and fires its own log line five seconds later, regardless of what actually happened in the meantime. Every trace pulled during the incident showed a call succeeding in milliseconds, followed by what looked like a five-second hang report for that same call. It wasn't a hang. It was punctual, misleading paperwork from a timer nobody told to stand down.

Two real fixes came out of this chapter, and neither of them was the one that mattered most. The actual bug, the one from Chapter 1, had already been found and fixed by the time this was untangled. What this chapter mostly demonstrates is how easy it is to spend real investigative effort chasing a symptom that a piece of your own defensive code was manufacturing, entirely honestly, one abstraction layer away from where the real problem lived.

Chapter 4 — Building the Off-Ramp

With the actual Chapter 1 bug fixed, the download itself worked. Warmup, run explicitly, was next, and it surfaced a gap with nothing to do with any race or any timer: there was no timeout at all on the part of the process that does the real work.

Model warmup on Android has two phases with very different cost profiles, load and generate. Loading reads a multi-gigabyte file from disk and allocates the runtime's internal buffers and cache; generating runs a short prompt through the now-resident model to confirm it actually works. The existing timeout, inherited from the app's original, much smaller default model, bounded only the second phase. The first phase, the one that scales directly with model size, ran inside a plain fire-and-forget task with no bound anywhere in the stack, JavaScript or native. For a model several times the size of the one this code was written against, that gap had simply never been exercised before.

The fix mirrors a pattern the codebase already used elsewhere for exactly this kind of native call: submit the load as a cancelable unit of work, and enforce a generous but real wall-clock limit around the whole thing, load and generate together, not just the fast part. On timeout, the pending native call is cancelled and the surrounding executor is reset so a wedged load can't block whatever tries to use the runtime next. It's worth being precise about what that cancellation does and doesn't guarantee: the underlying native inference call is not always genuinely interruptible mid-flight, so the fix bounds how long the caller waits and guarantees the caller gets a clean, recoverable answer either way. It does not claim to instantly free every resource the abandoned attempt was using. That distinction matters more than it sounds like it should, and it's the same one this codebase had already made, correctly, for its other long-running native calls; this was simply the one call that had never needed it before.

The second half of the off-ramp was the automatic warmup effect from Chapter 1, and the fix there is closer to a policy change than a bug fix. Automatic, unattended warmup on cold start is now gated to the app's own vetted default profile only. An explicit, user-initiated action, like opening the camera to scan something, still warms whatever model is currently selected, on request, exactly as before. What no longer happens is an experimental pick silently becoming a standing, unattended commitment that reasserts itself on every future launch with no confirmation and no way to intervene before it starts.

Both fixes are general. Neither depends on which model is installed, and both would have quietly reduced risk even if MedGemma had never entered the picture, they simply hadn't been needed yet. That's why they were the one thing kept, unconditionally, once the rest of this evaluation was reverted: they're not MedGemma's fixes. They're the app's.

Chapter 5 — Retesting Honestly

With a real timeout in place and no way for a bad warmup to become permanent, the obvious next step was to retest and get an actual answer about whether this model belonged in the app. The retest still ran hot, the phone still warmed noticeably, and the timeout at least meant it now resolved instead of stretching on indefinitely.

That result invited an easy, wrong conclusion: dense health-specialized model, too heavy for a phone, case closed. It would have been the wrong conclusion, because the test itself had a defect nobody had gone looking for. A local development configuration, set weeks earlier for reasons entirely unrelated to this model, hardcoded the AI runtime's inference backend to CPU only. The app's actual, intended default, the one every production build ships with, is auto, meaning try the accelerated path first and fall back only if it genuinely isn't available. A non-default, explicitly configured value always wins over that intended default, by design, no exceptions, which is exactly correct behaviour for a configuration override. It also meant every attempt so far had been running this new, larger model with hardware acceleration switched off entirely, on a device whose GPU had never been asked a single question.

That's a real methodology lesson independent of anything specific to language models: a fair test of a new component requires actually running your own defaults, and a stale local override is exactly the kind of thing that survives silently because it never causes an error, only a misleading result. Fixing the configuration back to the app's intended default was a one-line change. What it unlocked was the actual test.

Chapter 6 — The Real Answer

With acceleration genuinely in play this time, the runtime did exactly what auto is supposed to do: it tried the GPU delegate first. And the GPU delegate failed, immediately, during model compilation, before running a single token of inference.

The failure was an internal engine-creation error from the inference runtime itself, opaque beyond a source file and line number, the kind of message that tells you something failed to compile without telling you which operation in the model's graph it choked on. It reproduced identically, byte for byte, on a second independent attempt, which rules out a transient fluke and points at a structural incompatibility between this specific model artifact and the GPU delegate on this runtime version. The fallback path, exactly as configured, then dropped to CPU, which is the same heavy path every earlier attempt had already been stuck on, just now reached honestly instead of by a misconfigured shortcut.

graph LR
    A[GPU attempt, ~1.8s, fails to compile]
    B[CPU fallback, sustained, hot path]
    A -->|allowCpuFallback=true| B
    style A fill:#F59E0B,color:#1F1300
    style B fill:#EF4444,color:#2A0A0A

The model's own publisher documentation, read only after this finding, says exactly this in different words: hardware acceleration is "realistically needed" for the multimodal variant, specifically because of its convolution-heavy vision encoder, while the text-only variant is explicitly called out as viable on CPU alone. This wasn't a surprise the model's own authors hadn't anticipated. It was a documented constraint the first, misconfigured test had never actually been in a position to violate or honour, because it never gave the accelerated path a chance to try.

What the failure almost certainly is not, worth stating plainly, is a bug in FarmakoMed's own code. The auto path did exactly what it was designed to do, requested the accelerated backend first, and the delegate itself declined to compile this specific artifact on this specific device and runtime version. Hardware NPU support wasn't a fallback option either: the chip in this device is on the runtime's supported NPU list in principle, but no vendor-converted build of this particular model exists yet for it, a gap the team had already run into and documented on a separate project months earlier. "The chip supports it" and "we can use it today" are, once again, different claims.

Chapter 7 — The Decision to Wait

With an honest, reproducible answer in hand, GPU compilation genuinely fails for this artifact, CPU-only is genuinely unsuitable for it per the model's own guidance, and NPU support genuinely doesn't exist for this chip yet, the remaining question wasn't technical. It was what to do with an evaluation that had turned up two shippable, general fixes and one honest "not yet" for the reason it started.

The model was removed from the app's model registry, back to a single, vetted default, rather than left in as a selectable option nobody could responsibly recommend. The two general fixes, the load timeout and the automatic-warmup gate, were kept, because they reduce real risk independent of which models the app ever ships, and reverting working safety nets to make a revert feel cleaner would have been the wrong kind of tidy. A handful of defensive touches that happened to reference the new model by name, redaction patterns that scrub model identifiers out of diagnostic exports, chief among them, were left exactly as they were: harmless, general, and no worse for still recognising a name the app no longer offers.

Nothing here closes the door permanently. An updated build of the model that compiles cleanly on this GPU delegate, or a vendor-converted NPU artifact for this chip landing at some point, would each be a legitimate reason to retest, honestly, with the off-ramp this evening built now already in place. Until then, the app ships the model it can actually stand behind everywhere it runs, and the evaluation that said no is written down clearly enough that nobody has to relearn any of this by triggering the same phone-goes-warm afternoon twice.

Conclusion

Almost none of what made this evening long was the language model itself. A model either compiles for a given backend or it doesn't, and that answer, once honestly obtained, took under two seconds to arrive. Everything else, the missing off-ramp, the misread field that silently blocked a download, an uncancelled timer manufacturing a fake hang, a genuinely missing timeout on the one native call that had never needed one before, a stale local override rigging the very test meant to settle the question, was ordinary application engineering, the kind that would have mattered on any model, including the one already shipping.

That's the pattern worth carrying forward past this one evaluation: a new, more capable component doesn't remove the need for your own system to fail safely, log honestly, and test itself fairly. If anything, it raises the stakes on all three, because the bigger the thing you're evaluating, the more of your own house you'll find out was already a little unfinished, right as you ask it to hold something heavier than before.