Why local-first AI matters now (and what it costs you)

The phrase “local-first” has been overloaded for a while now. In the context of personal AI, it covers a spectrum:

• Strict local-first: model weights, inference, agent code, and memory all on your hardware.
• Hybrid: agent and memory local; inference via a cloud LLM you choose.
• Cloud-first with a local cache: most computation cloud, with a local proxy for latency.

deputyOS is built for the first two. This post is about why we think that matters more in 2026 than it did when the wave of cloud-hosted assistants started.

What “local-first AI” actually means

For a personal assistant, local-first means a specific list of things never leave your device without your explicit consent:

• The conversation history you’ve accumulated.
• The skills, hooks, and prompts you’ve configured.
• Your channel credentials (Telegram bot tokens, Slack signing secrets, Matrix access tokens).
• Your file contents — when the agent reads a mounted folder.
• Your cost ledger — what you spent, with whom, when.

In deputyOS, all of that lives on your hardware’s data partition. With the air-gapped tier, the inference model is also local — a tier-appropriate LFM2 running under llama.cpp. Without the air-gapped tier, the only outbound call is to your chosen model provider (OpenAI / Anthropic / OpenRouter / Bedrock).

Why this is more important now than two years ago

Three things changed:

1. Personal AI now holds enough context to be sensitive

In 2024, “AI assistant” mostly meant a chat window. In 2026, your assistant is reading your calendar, drafting your emails, accessing mounted folders, and learning your tone. Conversation logs are de facto personal records — not because anyone designed them that way, but because the medium accumulates them. A SaaS provider holding your conversation history is qualitatively different from one holding your search history.

2. SaaS terms changed under everyone’s feet

Multiple major providers have, over the past 18 months, quietly updated their data-use clauses. Some now reserve the right to train on user content unless you explicitly opt out (and sometimes only on enterprise tiers). Some changed retention windows. Some added arbitration clauses that limit your ability to challenge a decision. None of this happened with malice — these are just the dynamics of running a public LLM service at scale. It’s not a story about a bad actor. It’s a structural feature of the model. Local-first is the only posture that’s stable across provider whim.

3. Inference got good enough on consumer hardware

LFM2-1.2B on a Pi 5 16 GB does ~5 tokens/sec. That’s fast enough for short-form chat, drafting, and intent routing. It’s not fast enough for code completion or deep reasoning. Two years ago, nothing in this class was usable on a Pi. The hardware/software intersection has crossed a threshold where local inference for everyday tasks is a real option, not a science experiment.

The honest costs

A blog post that pretends local-first is free is wrong. Here are the real costs:

You maintain the device

The Pi runs unattended for months at a time, but eventually you’ll need to power-cycle it, swap an SD card, or apply a security update. deputyOS automates this — A/B image updates with watchdog rollback, signature DB freshness in every image rev, hooks to fire when an update applies — but a cloud assistant has none of these concerns for the user. (The provider still has them.)

Cutting-edge capability lags

Frontier-model class capability — long context windows, multi-modal vision-reasoning, top-tier coding — is genuinely better in the cloud right now. The gap is smaller every quarter, but it exists. If your assistant’s primary job is to write production code, the cloud is still the right call. If it’s to triage Telegram messages, schedule meetings, and remind you of things, local is fine and getting better fast.

Inference is slower on consumer hardware

A Pi 5 doing 5 t/s feels slow next to GPT-class APIs returning at 100+ t/s. For chat, this matters. For background tasks the agent runs unattended, it doesn’t.

Some things only work online anyway

Channels like Telegram or Slack need the internet by definition — that’s where your friends and coworkers are. The deputyOS distinction is that the agent and its memory are local; the transport still uses public networks. The air-gapped tier handles the strict-no-network case separately, accepting that you can’t use Telegram from there.

What you get for those costs

Conversation privacy that survives policy changes

A future term-of-service update can’t reach back and retroactively touch conversations that never left your hardware. This is the rare property that’s genuinely binary: either the data is yours or it’s somebody else’s, with no middle state.

Predictable, capped cost

You spend on power (~5 W idle for a Pi 5; ~25 W for a NUC) and your chosen provider’s tokens (if any). deputyOS surfaces this with cost guardrails — per-day and per-month caps with auto-pause. There is no per-seat subscription to drift upward.

No surprise capability removals

Models get deprecated. Features get pulled. Cloud assistants change shape silently. Your local OpenClaw on the SD card today is the OpenClaw on the SD card next year, until you choose to update.

Latency that doesn’t depend on AWS

Local inference is slower per token but doesn’t go down when an upstream provider has an outage. For automation-style use (“remind me to take out the bins”), latency-stability beats latency-mean.

Build-time, not run-time, dependencies

Because deputyOS bakes everything at image build time, your run-time has no pip install, no npm install, no flaky CDN. Most “AI installer broke” stories are dependency-resolution stories. deputyOS makes those impossible by construction.

The pragmatic version

You probably don’t want to choose between “all cloud” and “no cloud forever.” Most users won’t. The deputyOS design accepts that and gives you both:

• Default tier: agent + memory local, inference via a cloud provider you pick.
• Air-gapped tier: everything local, baked LFM2 GGUF, nftables-locked egress.
• Mixed: the agent’s main provider is local LFM2; you keep a cloud key as a fallback that’s only used when explicitly requested.

You can also start in the default tier and flip to air-gapped later — same SD card, just a new image bake. The configuration the user provided (channels, profile, secrets) transfers via backup and restore.

What we’re betting on

The bet is that local-first AI doesn’t have to feel like a downgrade. The wizard takes ten minutes. The PWA is polished. The channels work. The voice picks up. The drives mount. The updates apply without you noticing.

When the cloud assistant becomes the legacy option — the one where you can’t see what it remembers about you, can’t pin a model version, can’t grep your own conversation history — the people who already have local agents will be the ones who never had to switch.

That’s what we’re building toward. The picker is where you start.