LFM2 explained: why we picked Liquid AI's models for the air-gapped tier

When we shipped the air-gapped tier, the question we had to answer was: which model defaults?

The constraints were:

Must run on CPU. Pi 5 has no GPU; mini-PCs at this price point usually don’t either.
Must fit small enough that Q4_K_M quantization runs at usable t/s.
License must allow baking into a redistributed image. (Eliminates anything with a “no-commercial-use” or “no-redistribute” clause.)
Should be at the quality frontier for its size class — local AI that’s noticeably worse than cloud is a non-starter for users.

We benchmarked Llama-3.2 (1B, 3B), Phi-3.5-mini, Gemma-2-2B, and LFM2 (350M, 1.2B, 2.6B). After a few weeks of evaluation, LFM2 won on all three of our tiers.

What LFM2 is

LFM2 is the second-generation foundation model from Liquid AI. The architecture is a “Liquid neural network” — a hybrid that interleaves convolutional, recurrent, and attention-style blocks rather than pure transformer stacks. The result is a model that, at small sizes, has substantially less inference latency than transformer-equivalent params, particularly on CPU.

The Liquid AI team published 350M, 700M, 1.2B, and 2.6B variants in the LFM2 series. We bake 350M (lean), 1.2B (standard), and 2.6B (rich). The 700M is fine too — we just preferred 1.2B for the standard tier because the quality jump justifies the size jump on most hardware that can run standard.

Why we didn’t pick Llama-3.2

Llama-3.2 1B and 3B are great models. We previously baked them in pre-LFM2 builds. The reasons we moved on:

Per-token CPU latency. On a Pi 5, Llama-3.2-1B Q4 runs at ~10 t/s; LFM2-1.2B Q4 runs at ~9 t/s but with noticeably better output quality per token. Net throughput-quality wins go to LFM2.
3B is too big for Pi 5 KV cache + the agent process. We had to refuse TIER=rich on Pi 5 with the old Llama-3.2-3B default. With LFM2-2.6B, Pi 5 16 GB handles it.
License flexibility. Llama-3.2 has a “Meta Llama 3.2 Community License” with use-restriction clauses. LFM2 ships under terms that are simpler for downstream redistribution-as-part-of-an-image.

We still keep local-llamacpp as a provider entry — if you want Llama, you can register it with deputyctl model register <path-to-gguf>. The deputyOS catalog isn’t a one-model-monopoly.

Why we didn’t pick Phi-3.5-mini

Microsoft’s Phi-3.5-mini is also a strong small model. At 3.8B parameters, however, it’s too big for Pi 5’s KV-cache budget alongside the agent process. We could ship it for the mini-PC + cloud targets, but having different default models per target (vs different sizes of the same family) makes the user experience inconsistent. LFM2 lets us scale the same family across lean/standard/rich without changing what kind of model is loaded.

Why we didn’t pick Gemma-2-2B

Gemma is good. We didn’t pick it for similar reasons to Phi — the size doesn’t compose as cleanly with our tier sizes. We may revisit if Google ships smaller variants.

What you actually get

The LFM2 quality profile at our three sizes, from informal testing:

LFM2-350M (lean)

Conversational chat: usable; better than you’d expect.
Summarisation of short text (1–2 paragraphs): good.
Long-form generation: clipped, sometimes loses thread.
Reasoning chains: shallow.
Coding: weak. Don’t expect any.

Use case: an air-gapped Pi 4 that just routes Telegram messages and gives short answers. Lean is mostly about having any local model at all — not about deep reasoning.

LFM2-1.2B (standard)

Conversational chat: surprisingly good.
Summarisation: solid, on par with much bigger models for short inputs.
Long-form generation: coherent up to ~500 words; degrades past that.
Reasoning chains: handles 2-3 step plans well; struggles past that.
Coding: short snippets in mainstream languages; not for complex code.

Use case: the default daily-driver air-gapped tier. Pi 5 8 GB+ and most cloud VPS sizes run it well. Most personal-assistant tasks are within its envelope.

LFM2-2.6B (rich)

Conversational chat: very good.
Summarisation: very good.
Long-form: coherent up to ~1500 words.
Reasoning chains: handles 4-5 step plans.
Coding: passable for short tasks; we ship Qwen2.5-Coder-1.5B alongside it for coding-specific work.

Use case: rich-tier air-gapped on Pi 5 16 GB or x86 mini-PC. Approaches the “I don’t miss the cloud LLM for most things” threshold.

What it doesn’t do

Honestly:

Frontier-model coding (GPT-4o, Claude Sonnet, etc. remain ahead by a meaningful margin).
Long-context multi-doc reasoning (LFM2 context windows are smaller).
Image / multimodal (none of the LFM2 variants are multimodal; deputyOS doesn’t run multimodal locally yet).
High-throughput summarisation (Pi-class CPU speed limits this).

For these, point deputyOS at a cloud provider via the wizard and skip the air-gapped tier.

Why this matters beyond model choice

The LFM2 decision is downstream of a bigger principle: the model family should match the appliance shape. We’re shipping a Pi-and-mini-PC appliance, not a workstation tool. The model has to be CPU-first, small enough to coexist with the agent process, and qualitatively useful enough that air-gapped feels like a real option, not a downgrade.

We expect to revisit the choice every release. If a different small-model family wins the trade-off in 6 months, we’ll switch defaults and document the migration. The deputyctl model register path means users can keep their preferred model regardless of our defaults.

How to swap models on a running device

If you’re on the air-gapped tier and want a different baked model:

deputyctl model list          # see what's baked
deputyctl model set LFM2-1.2B # switch the active llama-server unit

To add a new GGUF you’ve sneakered onto the device:

deputyctl model register /mnt/deputyos/<usb>/your-model.Q4_K_M.gguf
deputyctl model set <id>

The catalog file at /opt/deputyos/airgap/models/catalog.json is updated; the deputyos-llamacpp@<id>.service template handles the runtime swap.

Where to learn more

Liquid AI’s official model cards.
deputyOS air-gapped concept doc.
deputyOS air-gapped build details.
The Pi 5 vs mini-PC benchmark post has real t/s numbers.

If you want to experiment, the picker with the “Air-gapped” toggle gives you the rich tier on Pi 5 16 GB or x86 mini-PC — that’s where LFM2 shines.