# model-for-opencode

An OpenAI-compatible chat endpoint for **Qwen3 4B** (Q4_K_M GGUF), served by
llama.cpp's `llama-server` on a local GPU. It only provides the model: run
opencode (or any OpenAI-compatible client) on your own machine and point it here.
Your files and tools stay on your machine; only the model calls come here.
No network, no key.

Base URL: `{{BASE_URL}}`

## Routes

| Method | Path | What |
| --- | --- | --- |
| POST | `/v1/chat/completions` | chat completions, streaming, tool calls |
| GET | `/v1/models` | one model: `qwen3-4b` |
| GET | `/.well-known/opencode` | the provider config `opencode auth login` reads |
| GET | `/status` | phase (`ready` when usable) |
| GET | `/human/` | setup page for opencode |

```sh
curl -s {{BASE_URL}}/v1/chat/completions {{AUTH}} \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hi"}],"max_tokens":50}'
```

Accepted fields: `messages` (required), `tools`, `tool_choice`,
`response_format`, `max_tokens`, `stream`, `seed`. Others, like `temperature`,
are refused with a 400. The model is always `qwen3-4b`.

## opencode

Once, on your machine:

```sh
opencode auth login {{BASE_URL}}
```

opencode then loads this endpoint's settings from `/.well-known/opencode` on every
start. Or write the provider yourself, in `opencode.json`:

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "qwen-local": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Qwen3 4B (capsule)",
      "options": { "baseURL": "{{BASE_URL}}/v1" },
      "models": { "qwen3-4b": { "name": "Qwen3 4B", "tool_call": true } }
    }
  },
  "model": "qwen-local/qwen3-4b"
}
```

When the capsule has a token (served with `--host 0.0.0.0`), add
`"apiKey": "<token>"` to `options`.

## Timings and limits

- Loading: about 20 s once the weights are on disk. The first serve downloads
  2.5 GB.
- Context: 32768 tokens per request, shared by the prompt and the answer.
- One request at a time; others wait their turn.
- Qwen3 thinks before answering (`<think>…</think>`); put `/no_think` in a
  message to skip it.