qwen38-flash-next-basalt-5090

An OpenAI-compatible chat endpoint for Qwen3.8-Flash-Next (the jesdga GSQ-RCO IQ3_S pack) on the Basalt engine, running on one RTX 5090 with a 384k-token context (YaRN x1.5 over the trained 262144). It only provides the model: run opencode (or any OpenAI-compatible client) on your own machine and point it here. Your files and tools stay on your machine; only the model calls come to this capsule.

The model runs locally; the container has no network access (unless the user served it with --allow-external-network or --allow-internal-network, which it does not need).

Routes

All paths are relative to the capsule's URL (e.g. http://<tailnet-ip>:<port>), written here as {{BASE_URL}}.

Method	Path	What
POST	/v1/chat/completions	OpenAI chat completions, streaming or not
GET	/v1/models	one model: qwen3.8-flash-next-iq3_s-basalt-384k
GET	/status	phase (ready when usable)
GET	/agent	this document plus live status
GET	/human/	landing page: opencode setup and a quick-test chat
GET	/.well-known/opencode	opencode provider config (opencode auth login {{BASE_URL}})

POST /v1/chat/completions

Body: application/json only. Accepted fields: messages (required), tools, tool_choice, response_format, max_tokens, stream, seed; model, user, stream_options are accepted and ignored or passed on; n must be 1. Any other field (e.g. temperature, top_p) is refused with 400 naming the field: sampling comes from the pack. The model is always this one, whatever model says.

Messages may carry images (image_url parts with data: URLs). Tool calls follow the OpenAI format. The reply may include reasoning_content (the model's thinking) before content.

stream: true returns Server-Sent Events (data: {...} chunks, ending with data: [DONE]).

curl -s {{BASE_URL}}/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hi"}],"max_tokens":200}'

curl -N {{BASE_URL}}/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Count to 5"}],"stream":true}'

Timings and concurrency

• Load: about 30 s when the 87 GB pack is in the page cache, a few minutes cold.
• Generation: about 300 tok/s (speculative decoding, 7-token windows).
• One request at a time. Others wait their turn; send them sequentially.

opencode on your machine

Once: opencode auth login {{BASE_URL}}, then run opencode in any folder. opencode reads this capsule's provider from /.well-known/opencode at every start.

Or on a machine with m64: m64 qwen38-flash-next-basalt-5090 --ui opencode.

Errors

• 503 with a phase body: still loading; retry after Retry-After seconds.
• 400: a field the endpoint does not accept, or a prompt longer than the context.

Access

On the tailnet (the default bind) there is no token: anyone on the tailnet can use the endpoint. With a token (--host 0.0.0.0 etc.), send Authorization: Bearer <token>.