model-for-opencode

A small model on your own GPU, served as an OpenAI-compatible endpoint, for an opencode that runs outside the capsule: on your laptop, in your project folder. Only the model runs in the capsule.

The model is Qwen3 4B (Qwen's own Q4_K_M GGUF, 2.5 GB) on llama.cpp's server (build b12030). It needs about 6 GB of VRAM for a 32k context, so it fits most recent NVIDIA cards.

What it shows

• A GPU capsule: [hardware] says what it needs (gpu, vram_gb, driver), and m64 checks free memory before it starts. gpu = "NVIDIA" matches any NVIDIA card; a mismatch only warns.
• [[artifacts]]: the weights come from Hugging Face at a fixed commit. m64 downloads them once, checks them and mounts them read-only. The capsule folder stays a few KB.
• An engine image pinned by digest: ghcr.io/ggml-org/llama.cpp@sha256:…, so no Dockerfile and nothing to build.
• Fixed settings: start.sh sets context, offload and KV-cache type. Callers send only messages; temperature and the like get a 400.
• No network and no secret: once the weights are downloaded, nothing leaves the machine.

Run it

m64 check examples/model-for-opencode
m64 examples/model-for-opencode --ui opencode   # serve it, then start opencode on it

Or serve it and connect opencode yourself, from any machine that can reach it:

m64 serve examples/model-for-opencode
opencode auth login <capsule url>               # once
opencode

The first serve downloads the image (2.5 GB) and the weights (2.5 GB). capsule.toml targets linux-x86_64; on a GB10 set platform = "linux-aarch64" and gpu = "NVIDIA GB10" (the image has an arm64 build).

Make it yours

Swap the model by changing [[artifacts]] (repo, revision, include), the file name in start.sh, model_name, and vram_gb. Every change gives the capsule a new id.