olace

[ local ai ]

llama.cpp

llama.cpp is the inference engine that most local runtimes are built on. Olace runs it directly, which means no second layer between you and the engine. Olace installs a vetted build, starts it on demand, sizes context to your GPU, and makes it reachable from every device you pair. It is the default runtime on computers with a discrete AMD GPU.

Install

Easiest: Settings › Local AI in the app, or olace setup --provider llamacpp.

Unlike Ollama and LM Studio, there is nothing to install yourself. Olace downloads a pinned llama.cpp release from the project's own builds into ~/.olace/llamacpp and picks the right acceleration for your hardware. Nothing is installed system-wide, and nothing installs without your yes.

Before downloading anything, Olace looks for a llama-server you already have, on your PATH or in the usual Homebrew, Scoop, and Chocolatey locations. If that build supports router mode and can use your GPU, or the machine has no GPU to begin with, Olace adopts it and skips the download entirely. A build Olace already manages is never swapped out from under you. To choose a specific binary yourself, set llamacpp_binary_path in daemon.json.

Olace talks to llama.cpp at http://127.0.0.1:8143 (configurable via llamacpp_url).

Models

llama.cpp runs GGUF files from Hugging Face. For anything in Olace's catalog you never have to pick a repo or a quantization: each model already carries a vetted GGUF tag, so naming the model is enough. To fetch something outside the catalog, pass a Hugging Face tag in user/repo-GGUF:QUANT form and Olace routes it to llama.cpp on its own.

olace pull qwen3.5:4b --provider llamacpp
olace list
olace rm llamacpp/qwen3.5:4b
olace pull qwen3.5:14b --device home-pc --provider llamacpp   # on a paired host

Models with a vision projector are paired automatically, so vision models keep vision.

Models you already downloaded

If you have used llama.cpp or Jan before, Olace picks up those GGUFs and lists them alongside everything else. It links them where they sit rather than copying, so nothing is duplicated on disk and the original app keeps working exactly as before. Deleting an imported model in Olace removes only Olace's link.

Models that belong to Ollama or LM Studio are deliberately left alone, so the same weights never show up twice.

Managed for you

  • Starts on demand. The daemon launches the server when a request needs it.
  • Keep-alive and unload work the same as the other runtimes: olace settings keep-alive, olace unload.
  • Context sizing. Every model gets its own context window, sized from what the model costs and what your GPU can spend, then tuned as Olace learns what actually runs on your machine. See Hardware & context.

Running alongside the others

llama.cpp, Ollama, and LM Studio coexist cleanly. Models from all three appear in one library and in every model selector, each runtime's context sizing is tuned independently, and the Olace Bridge serves all of them.