# model-for-opencode

A small model on your own GPU, served as an OpenAI-compatible endpoint, for an
[opencode](https://opencode.ai) that runs outside the capsule: on your laptop,
in your project folder. Only the model runs in the capsule.

The model is **Qwen3 4B** (Qwen's own Q4_K_M GGUF, 2.5 GB) on **llama.cpp**'s
server (build b12030). It needs about 6 GB of VRAM for a 32k context, so it fits
most recent NVIDIA cards.

## What it shows

- **A GPU capsule**: `[hardware]` says what it needs (`gpu`, `vram_gb`, `driver`),
  and m64 checks free memory before it starts. `gpu = "NVIDIA"` matches any
  NVIDIA card; a mismatch only warns.
- **`[[artifacts]]`**: the weights come from Hugging Face at a fixed commit. m64
  downloads them once, checks them and mounts them read-only. The capsule folder
  stays a few KB.
- **An engine image pinned by digest**: `ghcr.io/ggml-org/llama.cpp@sha256:…`, so no
  Dockerfile and nothing to build.
- **Fixed settings**: `start.sh` sets context, offload and KV-cache type. Callers
  send only messages; `temperature` and the like get a 400.
- **No network and no secret**: once the weights are downloaded, nothing leaves
  the machine.

## Run it

```sh
m64 check examples/model-for-opencode
m64 examples/model-for-opencode --ui opencode   # serve it, then start opencode on it
```

Or serve it and connect opencode yourself, from any machine that can reach it:

```sh
m64 serve examples/model-for-opencode
opencode auth login <capsule url>               # once
opencode
```

The first serve downloads the image (2.5 GB) and the weights (2.5 GB).
`capsule.toml` targets `linux-x86_64`; on a GB10 set `platform =
"linux-aarch64"` and `gpu = "NVIDIA GB10"` (the image has an arm64 build).

## Make it yours

Swap the model by changing `[[artifacts]]` (`repo`, `revision`, `include`), the
file name in `start.sh`, `model_name`, and `vram_gb`. Every change gives the
capsule a new id.