Air-gapped AI on your own hardware with LFM2 and llama.cpp

New to deputyOS? The fastest path is to run the standard image on your laptop first — no Pi, no commitment. This post is about the air-gapped tier specifically (the AIRGAP=1 flag), which is most useful once you have a dedicated host.

The strict version of local-first is air-gapped: the device works with no internet at all, including no cloud LLM. deputyOS supports this as a single build flag: AIRGAP=1. This post walks through what that flag does, what model you get, and how to verify the device is actually offline.

What “air-gapped” means in deputyOS

Concretely, an AIRGAP=1 image differs from a normal image in five places:

Local LLM is baked. A tier-appropriate LFM2 GGUF lives at /var/lib/deputyos/models/. llama.cpp runs as deputyos-llamacpp@<model-id>.service on 127.0.0.1:8090.
apt sources point at a baked mirror. /etc/apt/sources.list.d/deputyos-airgap.list references file:///opt/deputyos/airgap/apt-mirror/. The public Debian mirror is masked.
nftables denies egress. The ruleset in /etc/nftables.conf drops all outbound traffic except: loopback, RFC1918, mDNS (224.0.0.251 / ff02::fb), and the local DNS resolver on 127.0.0.53.
/etc/deputyos/network-policy.json declares mode=airgap. deputyctl limits shows “Airgap: yes”. deputyctl network unlock flips this to open post-boot.
Provider catalog defaults to local. The wizard hides cloud-API providers and pre-selects local-llamacpp-airgap as the active model.

The rest of the image — the agent code, the channels, the wizard, the PWA, the security baseline — is unchanged.

Per-tier LLM defaults

The plan was deliberately to make air-gapped composable with the existing TIER knob, so you can pick the size that fits your hardware.

Tier	Default LLM (GGUF, Q4_K_M)	Model size	Approx image size
lean	`LFM2-350M.Q4_K_M.gguf`	~250 MB	~1.3 GB
standard	`LFM2-1.2B.Q4_K_M.gguf`	~750 MB	~2.8 GB
rich	`LFM2-2.6B.Q4_K_M.gguf` + `Qwen2.5-Coder-1.5B-Instruct.Q4_K_M.gguf`	~2.6 GB	~6.5 GB

Why LFM2? Short answer: per-token quality at this size class beats Llama-3.2-class models we previously baked, and on Pi-class CPUs it stays in the “actually usable” envelope. Full reasoning.

Why Qwen-Coder for rich tier? Because if you can spare 16 GB+, you can probably also use a coding model. deputyctl model set Qwen2.5-Coder-1.5B swaps in mid-flight.

Per-target support

Not every target can run every tier. Authoritative defaults live in roles/deputyos/files/limits.<target>.json under airgap_supported + airgap_max_tier:

Target	airgap_supported	Max tier
rpi5	yes	rich
rpi4	yes	lean only
arm64-generic	yes	standard
x86_64-mini-pc	yes	rich
wsl2	yes	standard
macos-qemu	yes	rich
digitalocean / hetzner / linode / vultr	yes	rich
oracle-arm-free	yes	standard
fly-machines	no	—

rpi4 is lean-only because the Cortex-A72 cannot run LFM2-1.2B at usable token/sec. fly-machines is refused because the slot is too ephemeral and small.

Building an air-gapped image

make build TARGET=rpi5 PROFILE=openclaw TIER=rich AIRGAP=1

This plumbs AIRGAP=1 through scripts/build.sh → Packer (-var airgap=1) → the Ansible role (deputyos_airgap: true). Two role tasks run:

tasks/airgap-baseline.yml — installs nftables, drops the egress-deny ruleset, points apt at the baked mirror, writes /etc/deputyos/network-policy.json.
tasks/llm-airgap.yml — copies the SHA-pinned GGUF(s) into /var/lib/deputyos/models/, drops the catalog at /opt/deputyos/airgap/models/catalog.json, installs the deputyos-llamacpp@.service template, enables the default model unit on boot.

The role refuses to bake if the SHA of any GGUF doesn’t match the pin in roles/deputyos/vars/llm-airgap.yml. That’s load-bearing — it’s the last line of defence between you and a tampered model file.

Verifying the device is actually offline

The most direct test:

qemu-system-x86_64 \
  -nodefaults -nographic \
  -drive file=build/qemu-x86_64-openclaw.qcow2,if=virtio \
  -m 4G -smp 2 \
  -net none

The -net none flag means QEMU exposes no network interface to the guest. The wizard at :8088 should still come up (it binds to all interfaces but obviously there’s no one to reach it). More importantly: ssh in and run curl localhost:8088/chat — the chat endpoint responds, streaming tokens from the baked LFM2.

For a real Pi 5, run it in a closet without an Ethernet cable and don’t configure Wi-Fi. Wizard via HDMI + QR; talk to it via keyboard.

Going back to the regular network

deputyctl network unlock flips the mode flag, regenerates /etc/nftables.conf from the new policy, and reloads. The change is reversible: deputyctl network lock flips back to airgap.

You can also stay air-gapped at the agent level but unlock specific hosts. That’s the egress-whitelist mode — same network-policy.json schema, with mode=whitelist and an allow_hosts: [...] array. The nftables generator for that path is not yet shipped; the schema is forward-compatible today.

Updating an air-gapped device

This is the one place where the “no internet” rule needs an escape hatch — you still want security updates and new ClamAV signatures.

The standard path:

On a trusted machine with internet, fetch the latest signed manifest:

curl -O https://cdn.deputyos.com/stable/manifest.json
curl -O https://cdn.deputyos.com/stable/manifest.json.minisig
minisign -Vm manifest.json -P RWQ...   # verify

Download the image artefacts the manifest points at.
Copy them to a USB stick.
On the air-gapped device: plug the stick (it auto-mounts under /mnt/deputyos/<label>/ per the drive-mount policy).
Run deputyctl update --from /mnt/deputyos/<label>/manifest.json. The on-device path verifies signatures (minisign + cosign) before applying. A/B swap, reboot, watchdog rolls back if the new image fails to come up.

The “trusted machine” can be your laptop. The “USB stick” is the only thing that crosses the air gap. Minimal trusted-computing-base for delivery.

What you can’t do

The honest list:

No Telegram, Slack, Discord, etc. They live on the public internet. The air-gapped tier accepts that you’ll use the device exclusively via the local channels (web chat at /chat, the PWA at /app, voice if you’ve enabled it, IRC/Matrix on a private home network).
No cloud LLM fallback. When the local LFM2 hits a question it can’t answer, it tells you. No silent escalation to GPT.
No public webhook channels. Cloudflare Tunnel, Tailscale Funnel — both presume internet egress. Both are off in airgap mode (you can keep Tailscale enabled but use it for client-to-device, not device-to-public).

What you can do

Have private conversations that physically cannot leave the box.
Run on intermittent power / connectivity (boats, remote cabins, regulated environments).
Pair the device with mounted SMB shares to a NAS for content the agent reads — all internal to your LAN.
Demo “fully offline AI” in places where the audience won’t believe it without seeing -net none in your QEMU command.

What’s next for the air-gapped tier

Egress whitelist mode — for the user who wants “open internet but only to api.openai.com and Telegram.” Schema already designed; nftables generator is the missing piece.
Voice on-device for air-gapped — already works on rpi5 via whisper.cpp + Piper; we’ll polish wake-word reliability.
Profile-specific airgap defaults — Hermes’ RAG works fine with local embeddings; we’ll preload bge-small-en-v1.5 so memory search works offline by default.
SHA pin freshness — the upstream LFM2 GGUFs are still REPLACE_ME placeholders pending a publishable mirror. First public images will replace those with real pins.

If you’ve been waiting for a self-hosted AI assistant that you can prove never phones home, this is the build. Try the picker with the “Air-gapped” toggle on, and let us know how it goes.