model-for-opencode

An OpenAI-compatible chat endpoint for Qwen3 4B (Q4_K_M GGUF), served by llama.cpp's llama-server on a local GPU. It only provides the model: run opencode (or any OpenAI-compatible client) on your own machine and point it here. Your files and tools stay on your machine; only the model calls come here. No network, no key.

Base URL: {{BASE_URL}}

Routes

Method	Path	What
POST	/v1/chat/completions	chat completions, streaming, tool calls
GET	/v1/models	one model: qwen3-4b
GET	/.well-known/opencode	the provider config opencode auth login reads
GET	/status	phase (ready when usable)
GET	/human/	setup page for opencode

curl -s {{BASE_URL}}/v1/chat/completions {{AUTH}} \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hi"}],"max_tokens":50}'

Accepted fields: messages (required), tools, tool_choice, response_format, max_tokens, stream, seed. Others, like temperature, are refused with a 400. The model is always qwen3-4b.

opencode

Once, on your machine:

opencode auth login {{BASE_URL}}

opencode then loads this endpoint's settings from /.well-known/opencode on every start. Or write the provider yourself, in opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "qwen-local": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Qwen3 4B (capsule)",
      "options": { "baseURL": "{{BASE_URL}}/v1" },
      "models": { "qwen3-4b": { "name": "Qwen3 4B", "tool_call": true } }
    }
  },
  "model": "qwen-local/qwen3-4b"
}

When the capsule has a token (served with --host 0.0.0.0), add "apiKey": "<token>" to options.

Timings and limits

• Loading: about 20 s once the weights are on disk. The first serve downloads 2.5 GB.
• Context: 32768 tokens per request, shared by the prompt and the answer.
• One request at a time; others wait their turn.
• Qwen3 thinks before answering (<think>…</think>); put /no_think in a message to skip it.