LLMs vs SLMs: Why Bigger Isn't Always Better

Introduction

The AI industry seems obsessed with one number. Parameters. Billions of them, then hundreds of billions, and now, by some accounts, trillions. Every few months a new model appears, and the headline is almost always the same: it is larger than the one before it. Size has quietly become the industry's default proxy for progress, a single scalar we point at to say this one is better.

It's worth pausing on that assumption, because it is doing a lot of unexamined work. Ask a plain engineering question and the tidy story starts to fray: does every problem actually require the largest model available? Does reading a short document, sorting an inbox, or pulling the strength off a medication label really need the same machine that can pass a bar exam and write a compiler?

Perhaps not. And that "perhaps not" is the whole subject of this article.

Two clarifications before we start, because this topic attracts tribalism and I want to disarm it up front. This is not an anti-LLM article, large language models are one of the most remarkable engineering achievements of the decade, and the sections below say so plainly. It is also not an argument that Small Language Models are secretly superior; anyone claiming a two-billion-parameter model quietly beats a frontier system is selling something. This is an argument for architectural balance: that the mature version of this field looks less like a leaderboard and more like a well-stocked toolbox.

The Race for Bigger Models

To argue that bigger isn't always better, you first have to be honest about why bigger has, so often, genuinely been better. The scaling of large language models was not a marketing accident. It rested on a real and repeatedly observed empirical finding: across many orders of magnitude, model performance improved smoothly and predictably as researchers increased parameters, training data, and compute together. This is the substance behind the phrase "scaling laws," and it is one of the most important empirical results in modern machine learning.

The consequences of that finding compounded. Bigger models brought larger context windows, the ability to hold an entire codebase, a book, or a long conversation in working memory at once. They brought broader general reasoning, the capacity to chain steps together on problems the model was never explicitly trained on. They brought multimodal intelligence, folding images, audio, and text into a single system. And they surfaced what researchers came to call emergent abilities: capabilities that were essentially absent in smaller models and then appeared, sometimes quite abruptly, past a certain scale.

So the race has a real engine under the hood. If you are a research lab whose mission is to push the frontier of what is possible at all, scaling has been a rational, arguably the single most rational, bet you could make. None of what follows should be read as dismissing that. The advances are genuine, and the largest models can do things that were science fiction a handful of years ago.

But there is a subtlety hiding inside the scaling story that the headline number obscures, and it matters for everything after. Even within the pure-research framing, "bigger" was never the whole recipe. A later and equally influential result found that many large models had actually been trained on too little data for their size, that for a fixed compute budget, a somewhat smaller model trained on more data could outperform a larger one trained on less. In other words, the field's own foundational scaling work already contained the seed of this article's argument: raw parameter count was never a clean stand-in for quality, even at the frontier.

Bigger Models Solve Harder Problems

Here is the part an honest argument cannot skip, and skipping it is exactly what makes most "small models are the future" takes unpersuasive. Very large models remain, as of this writing, genuinely unmatched in a range of demanding domains. Pretending otherwise would undermine everything else in this piece.

Where does frontier scale still clearly earn its cost?

• Complex, multi-step reasoning, problems that require holding many constraints in mind at once and planning several moves ahead, where a small slip early derails the whole chain.
• Serious software engineering, reading a large, unfamiliar codebase, reasoning about it as a system, and producing changes that respect constraints the prompt never spelled out.
• Scientific and analytical research, synthesising across dense technical material and reasoning at the edge of a field.
• Broad world knowledge, the sheer breadth of facts, contexts, and associations that only a very large parameter count seems able to store and retrieve reliably.
• Deep multilingual understanding, nuance, idiom, and register across dozens of languages at once.
• Very long-context tasks, reasoning coherently over a novel-length document or an extended conversation without losing the thread.

These are not small things, and for tasks that genuinely live in this territory, reaching for a small model to save a few cents or milliseconds would be the wrong engineering call, the mirror image of the mistake this article is actually about. Acknowledging that plainly is not a concession that weakens the argument; it is what makes the argument credible. The claim here was never "small is better." It is "size is the wrong first question." Establishing where big genuinely wins is how we earn the right to ask where it doesn't matter.

Most People Don't Need a Trillion Parameters

Now the central argument. Set aside the demanding frontier tasks from the previous section, which are real but, measured by sheer volume of everyday computing, comparatively rare, and look at what people actually ask software to do all day:

• Read a document and pull out the parts that matter.
• Summarise a page of notes into three lines.
• Find a specific piece of information in a pile of text.
• Answer a short, self-contained question.
• Sort or classify a message, is this urgent, is this spam, which folder.
• Assist with a structured record: read a label, extract a field, tidy an entry.

These workloads share a quiet common property: they are bounded. The input is modest, the required reasoning is shallow, and the space of correct answers is narrow. They do not need a model that can also discuss moral philosophy or debug a distributed system. For a large share of these everyday tasks, a well-chosen smaller model can produce output of entirely sufficient quality, and in return offer advantages the frontier model structurally cannot.

I want to be careful with the claim, because the temptation to overreach is strong. The argument is not that a Small Language Model matches a frontier model in raw capability. It plainly does not. The argument is that for a bounded task, "sufficient quality, delivered instantly, privately, and offline" can be a better product than "marginally higher quality, delivered after a network round-trip to a data centre, at a metered cost, only when there's signal." Those are different notions of "better," and the everyday user cares far more about the second one than the leaderboard does.

What smaller models buy you, on a bounded task, is a bundle of properties that matter enormously to real users and almost never appear on a benchmark: latency low enough that the result feels instantaneous rather than "loading"; privacy, because computation that happens on the device never becomes data in someone else's log; cost, because there is no per-token meter running; and offline availability, because a model resident on the phone does not care whether there is signal in the basement, on the plane, or in the rural clinic. None of these are consolation prizes. For a great many tasks, they are the actual point.

Engineering Is About Trade-offs

Step back from language models specifically and this stops being an AI argument at all. It becomes the oldest lesson in engineering: there is no free lunch, only trade-offs you choose deliberately or inherit by accident.

Every model sits somewhere on a set of axes that pull against each other. Larger models generally provide broader capability, deeper reasoning, and more world knowledge, and they pay for it in memory, compute, energy, latency, and, very often, a dependency on someone else's servers. Smaller models generally give up some capability at the top end, and in exchange offer a different bundle entirely.

graph LR
    subgraph Large model wins
      A[Peak capability]
      B[Reasoning depth]
    end
    subgraph Small model wins
      C[Resource / energy cost — lower]
      D[Response latency — lower]
      E[On-device privacy]
      F[Offline availability]
    end

The bundle a smaller model can offer, lower latency, offline execution, lower energy consumption, lower operational cost, greater privacy, and far more deployment flexibility, is not a set of runner-up qualities. Each one is decisive for some product. A feature that must work on a plane needs offline. A feature touching sensitive data may need on-device privacy as a hard requirement, not a nice-to-have. A feature invoked thousands of times a day is governed by per-call cost and latency long before it is governed by the model's ceiling on a reasoning benchmark.

Size Is Not the Metric That Matters

Here is a question no user has ever asked: "How many parameters generated this answer?" What they ask is far simpler and far more demanding, "Did it solve my problem? Was it fast? Can I trust it?" Parameter count is an implementation detail that leaked into the marketing. It is not, and never was, the thing the user is buying.

We already know this instinctively in every other corner of software, because we have lived through the equivalent hype cycles and come out the other side. Consider the pattern:

• Nobody chooses the largest database. They choose the one that fits the data model, the query pattern, and the consistency requirements. A key-value store and a relational database are not ranked on a single ladder.
• Nobody deploys the largest web server. They choose the one whose concurrency model and footprint suit the workload, sometimes that is a heavyweight application server, and sometimes it is a tiny static-file server that does one job perfectly.
• Nobody reaches for the largest compiler, or the one with the most optimisation passes, when a fast incremental build is what the workflow actually needs.

In every one of these cases, the mature engineering culture stopped asking "what's the biggest?" a long time ago and started asking "what fits?" AI is simply younger, and still in the phase where the biggest number wins the press release. It will grow out of it, the same way the rest of the field did, the moment the cost of the mismatch becomes obvious enough that fit becomes the more interesting question.

The Rise of Specialised Intelligence

The most encouraging development in AI right now is not, in my view, the model at the very top of the leaderboard. It is the rapidly maturing ecosystem of capable Small Language Models, families of models designed from the outset to be small enough to run somewhere other than a data centre, while still being genuinely useful.

The names in this space have become familiar quickly. Google's Gemma family targets open, efficient models suited to on-device and edge deployment. Microsoft's Phi family made an influential case that careful, high-quality training data lets a small model punch well above its parameter count. Alibaba's Qwen ships in a wide range of sizes explicitly so developers can pick a point on the curve. Mistral's smaller open models are built for efficient deployment. And Meta's Llama family includes deliberately smaller variants intended to run in constrained environments. This is not a comparison of their benchmark scores, that would miss the point entirely and date this article within a month.

What makes this ecosystem exciting is not that any of these models will dethrone a frontier system on a hard reasoning benchmark. They won't, and they aren't trying to. The innovation is orthogonal to that contest entirely. It lies in making capable AI execute on everyday hardware: a phone, a laptop, a small embedded device, a machine with no reliable internet connection. The centre of gravity shifts from "maximum capability in a data centre" to "sufficient capability, everywhere, in the user's own hands." That is a different axis of progress, and arguably a more democratising one.

FarmakoMed as a Case Study

It would be easy to leave this argument at the level of principle. It is more useful to ground it in something real, so this section describes how FarmakoMed, a local-first medication-management app, applies exactly this thinking. Everything below was checked against the current app codebase rather than assumed; where a capability is planned rather than shipped, that is stated explicitly, because an engineering journal that overstates its own product is not one worth reading.

The starting point is deliberately unglamorous: FarmakoMed does not try to compete with frontier models, and it isn't offered as a replacement for them. Its goal is narrower and, for its users, more useful, provide genuinely helpful intelligence where it actually matters: locally, privately, offline, and with minimal friction.

Concretely, the app's headline AI feature is medication recognition. A user photographs a medication package, and a small on-device Gemma vision-language model, running through Google's LiteRT-LM runtime, reads the image and proposes structured fields, without the photo or the request ever leaving the phone. The active model profile is a small member of the Gemma family (the gemma-4-e2b-it profile, an E2B-class model), chosen deliberately over the larger E4B variant in the same family. The larger sibling is not automatically the better product decision here, the smaller one is the one that fits the device budget the app has to live inside.

The most instructive decision, though, is one where FarmakoMed chose not to use the model it already had loaded, a small case study in matching the tool to the task. When the app needed to extract plain text from scanned documents, the obvious move was to reuse the Gemma multimodal model already resident for medication recognition. The team measured that path and rejected it as the default: for pulling text off a page, a purpose-built, far smaller engine, Google's ML Kit on-device text recognition, did the job in a few hundred milliseconds and a few megabytes, where the general-purpose vision model would have cost multiple seconds and gigabytes of memory for the same result. The large model is kept only as a documented fallback for images the specialised engine can't handle.

Two more details reinforce that the size of the model is not the metric the product optimises for. Because this is healthcare, the app tunes its model for predictability over creativity: the default inference preset for extraction sets temperature to zero and top-k to one, the most deterministic decoding a language model can produce, the opposite of what you'd want from a brainstorming assistant. And the model's output is never shown to the user as fact on its own; it passes through a separate, deterministic safety layer that keeps the original document and extracted text alongside every AI-generated summary, so the smaller model's job is to propose, while a non-AI layer decides what the user is allowed to trust.

The honest summary is that FarmakoMed is not an argument that small models are superior. It is a worked example of the argument this whole article makes: that a smaller, appropriate model, deployed where it fits, can be the right engineering choice, not despite being small, but because the task and the constraints called for exactly that. For a deeper walk through those constraints, the companion article Choosing a Model for On-Device Healthcare AI goes dimension by dimension.

The Future May Be Hybrid

If you follow this argument to its natural end, you do not arrive at "small models win." You arrive somewhere more interesting and less tribal: the future is unlikely to be LLMs replacing SLMs, or SLMs replacing LLMs. It is far more likely to be both, working together, with the system itself deciding which to use.

Picture an application that treats "which model?" as a routing decision made per task, not a one-time architectural commitment. A bounded, private, latency-sensitive request, summarise this note, extract this field, classify this message, is handled by a small model on the device: instant, offline, and private. A genuinely hard request, reason across a large document, tackle a complex multi-step problem, is escalated to a large model in the cloud, with the user's awareness. The specialised task goes to the specialised model; the general task goes to the general one. The size of the model becomes an implementation detail of the router, invisible to the user, exactly as it should be.

graph LR
    T[User task] --> R{Router: which fits?}
    R -->|hard, open-ended| L[Large · cloud]
    R -->|bounded, private| S[Small · on-device]
    R -->|narrow, specialised| P[Specialised model]
    L --> A[Answer]
    S --> A
    P --> A

This is the optimistic conclusion, and it is optimistic precisely because it dissolves the versus in the title. In a hybrid world, "LLMs vs SLMs" is a false binary. The large model and the small model are not competitors for the same slot; they are different instruments in the same orchestra, and the engineering skill is knowing which one the passage calls for.

The Best Model Is the Right One

Engineering has never really been about building the biggest system. It has been about building the right one, the one whose shape matches the problem, whose cost matches the budget, and whose behaviour matches what the people using it actually need. We learned this with databases, with servers, with compilers, with every generation of infrastructure that briefly confused "largest" with "best" before growing up.

Artificial intelligence is now walking through that same adolescence in public, and the parameter count is its growth spurt. It is real, it is impressive, and it is not the point. The point is fit. A frontier model is the right answer to a genuinely frontier problem, and a small on-device model is the right answer to a bounded everyday one, and a mature system will happily use both without treating the choice as a matter of loyalty.

So perhaps it is time to retire the question the whole industry keeps asking first. Not "what's the biggest model?" but "what's the right one?" The second question is harder, because it has no single answer you can put in a headline. It is also the only one worth asking.