m64_8b0f715a688c / AGENT.md
qwen38-flash-next-strata-gb10
An OpenAI-compatible chat endpoint for Qwen3.8-Flash-Next (Qwen's 125B-A6B mixture-of-experts model; ISTA-DASLab's GSQ-RCO IQ3_S GGUF) on Strata 0.2.13's engine for the DGX Spark (GB10), with a 65536-token context and the MTP draft layer. It only provides the model: run opencode (or any OpenAI-compatible client) on your own machine and point it here.
The model runs locally; the container has no network access (unless the user served it
with --allow-external-network or --allow-internal-network, which it does not need).
On the GB10 the GPU and the CPU share one memory, so Strata reads every expert in place from the model file (one copy, no RAM/VRAM split) and keeps the 29 GB n-gram table locked in memory. Pictures are on (the BF16 vision tower). Measured on gx10: about 63 GB in use under load, so ~50 GB of the 119 GB stayed free through a 5-hour agent benchmark.
Routes
All paths are relative to the capsule's URL (e.g. http://<tailnet-ip>:<port>), written
here as {{BASE_URL}}.
| Method | Path | What |
|---|---|---|
| POST | /v1/chat/completions |
OpenAI chat completions, streaming or not |
| GET | /v1/models |
one model, qwen3.8-flash-next-iq3_s-strata |
| GET | /health |
200 when Strata is serving, else 503 |
| GET | /status |
phase (ready when usable) |
| GET | /agent |
this document plus live status |
| GET | /human/ |
landing page: opencode setup and a quick-test chat |
POST /v1/chat/completions
Body: application/json, the OpenAI chat format. Accepted fields: messages, tools,
tool_choice, response_format, max_tokens, stream, seed (plus model, user, n: 1,
stream_options); anything else, temperature and top_p included, is refused with 400: the
sampling is Strata's own defaults for this model. Only model is replaced. Tool calls follow the OpenAI format. Messages may carry images (image_url
parts with data: URLs). The reply may include reasoning_content (the model's thinking)
before content. stream: true returns Server-Sent Events ending with data: [DONE].
curl -s {{BASE_URL}}/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"Say hi"}],"max_tokens":200}'
curl -N {{BASE_URL}}/v1/chat/completions -H 'Content-Type: application/json' \
-d '{"messages":[{"role":"user","content":"Count to 5"}],"stream":true}'
Timings and concurrency
Measured on a DGX Spark (GB10):
- Ready about 45 s after start with the weights in the page cache (the first start after a reboot reads ~80 GB from disk first).
- Decode ~54 tokens/s for one stream (MTP drafting accepts ~65% of drafts); prompt ~100 tokens/s on a short prompt, faster on long ones.
- One request at a time (
--parallel 1); further requests wait. - airbench.ai with opencode, 64K context, 2026-11-13: Full benchmark 48/49 in 55 min (text 9/9, vision 19/19, mail 6/6, store 3/4, code 11/11); computer use 8/9; frontend 4/4; brand 3/3; hard coding v2 9/18 (the other 9 unanswered at the 3-hour deadline).
opencode on your machine
Once: opencode auth login {{BASE_URL}}, then run opencode in any folder.
Or on a machine with m64: m64 qwen38-flash-next-strata-gb10 --ui opencode.
Errors
- 503: still loading (or Strata stopped); retry.
- 502: Strata did not answer the request.
Access
On the tailnet (the default bind) there is no token: anyone on the tailnet can use the
endpoint. With a token (--host 0.0.0.0 etc.), send Authorization: Bearer <token>.