Canonical has released Inference Manager — a tool that brings local AI inference on Ubuntu into one place. It pairs a command-line interface (`inference`) with a background service that acts as a central entry point for local inference: it lists available models and routes inference requests. The proxy is served at http://localhost:8400/v1 by default, so any application that speaks the OpenAI API can use it as a model provider.

The tool builds on an existing push inside Ubuntu's own packaging system. 'Inference Snaps' turn a local model into a single command: `sudo snap install gemma4` pulls the model, its weights and a matching runtime, and serves an OpenAI-compatible API on localhost. Models available this way include Gemma 4, Qwen VL and Nvidia's Nemotron 3 Nano, each shipping as its own snap named after the model. At install time the snap scores the machine's hardware and picks an engine automatically — for example OpenVINO for Intel CPUs or a generic llama.cpp build — and Canonical documents it as the easiest route to optimised local models on Ubuntu 26.04.

Canonical positions Inference Manager for the case where one model is not enough: 'Inference Snaps have already made it easy to run optimised AI models on Ubuntu,' the company says, describing the new tool as 'one place to manage your AI inference on Ubuntu, whether you're running models locally or connecting to cloud providers.' The framing is a control plane: instead of each chat client, editor or coding agent being pointed at a different port and path, they can all talk to a single local endpoint.

Independent testing suggests the approach is promising but still rough. A hands-on review found that the engine picker chose the slower runtime for Gemma 4 on an Intel CPU (OpenVINO at 6.9 tokens per second against llama.cpp at 7.4), that a snap's OpenAI endpoint moves between /v1 and /v3 depending on which engine is active, and that switching models leaves old weights on disk — one snap held 15 GB for a 2B model needing 3.4 GB. On the upside, a 7B reasoning model run through an inference snap decoded about 28% faster than the same model under Ollama on the identical machine.

Inference Manager is published by Ubuntu as a verified snap under the GPL-3.0-only licence, with source at github.com/canonical/inference. It is currently only in the unstable beta channel, which Canonical warns 'could break and change often.'