Local LLM
Also called: on-device LLM, offline LLM, local language model
A local LLM is a language model whose weights run on your own hardware, so prompts never leave the machine.
A local LLM runs entirely on hardware you own. Prompts, documents and responses never leave the device.
What you can run is set by memory. Roughly, with 4-bit quantisation: 8 GB comfortably handles 7–8B parameter models, 16 GB reaches 13–14B, and 24 GB or more opens up 30B+. Below 4 GB you are looking at small purpose-built models rather than general chat.
Speed depends on the accelerator. A discrete GPU with enough VRAM is fastest; Apple Silicon’s unified memory is slower per token but near-silent and power-efficient; CPU-only inference works but is slow enough to change how you use it.
Local and cloud are not exclusive. A common arrangement is a local model for anything touching private data, with a cloud API for tasks that need frontier capability.