motif64

Chat · Qwen3.8-Flash-Next IQ3_S · Strata · GB10 · 72 GB VRAM

published by @dh7 on

m64 m64_8b0f715a688c
Download .tar.gz

Qwen3.8-Flash-Next IQ3_S on Strata (GB10). A local OpenAI-compatible chat endpoint for Qwen3.8-Flash-Next (Qwen's 125B-A6B MoE, ISTA-DASLab's GSQ-RCO IQ3_S GGUF) on Strata 0.2.13's DGX Spark engine, 64K context, with the MTP draft layer. Reads images, streams, tool calls. Needs no network.

Lineage

parent
none declared
derived from
no published capsule shares half its files
children
none

Files

6 files

AGENT.md

qwen38-flash-next-strata-gb10

An OpenAI-compatible chat endpoint for Qwen3.8-Flash-Next (Qwen's 125B-A6B mixture-of-experts model; ISTA-DASLab's GSQ-RCO IQ3_S GGUF) on Strata 0.2.13's engine for the DGX Spark (GB10), with a 65536-token context and the MTP draft layer. It only provides the model: run opencode (or any OpenAI-compatible client) on your own machine and point it here.

The model runs locally; the container has no network access (unless the user served it with --allow-external-network or --allow-internal-network, which it does not need).

On the GB10 the GPU and the CPU share one memory, so Strata reads every expert in place from the model file (one copy, no RAM/VRAM split) and keeps the 29 GB n-gram table locked in memory. Pictures are on (the BF16 vision tower). Measured on gx10: about 63 GB in use under load, so ~50 GB of the 119 GB stayed free through a 5-hour agent benchmark.

Routes

All paths are relative to the capsule's URL (e.g. http://<tailnet-ip>:<port>), written here as {{BASE_URL}}.

Method Path What
POST /v1/chat/completions OpenAI chat completions, streaming or not
GET /v1/models one model, qwen3.8-flash-next-iq3_s-strata
GET /health 200 when Strata is serving, else 503
GET /status phase (ready when usable)
GET /agent this document plus live status
GET /human/ landing page: opencode setup and a quick-test chat

POST /v1/chat/completions

Body: application/json, the OpenAI chat format. Accepted fields: messages, tools, tool_choice, response_format, max_tokens, stream, seed (plus model, user, n: 1, stream_options); anything else, temperature and top_p included, is refused with 400: the sampling is Strata's own defaults for this model. Only model is replaced. Tool calls follow the OpenAI format. Messages may carry images (image_url parts with data: URLs). The reply may include reasoning_content (the model's thinking) before content. stream: true returns Server-Sent Events ending with data: [DONE].

curl -s {{BASE_URL}}/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hi"}],"max_tokens":200}'

curl -N {{BASE_URL}}/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Count to 5"}],"stream":true}'

Timings and concurrency

Measured on a DGX Spark (GB10):

  • Ready about 45 s after start with the weights in the page cache (the first start after a reboot reads ~80 GB from disk first).
  • Decode ~54 tokens/s for one stream (MTP drafting accepts ~65% of drafts); prompt ~100 tokens/s on a short prompt, faster on long ones.
  • One request at a time (--parallel 1); further requests wait.
  • airbench.ai with opencode, 64K context, 2026-11-13: Full benchmark 48/49 in 55 min (text 9/9, vision 19/19, mail 6/6, store 3/4, code 11/11); computer use 8/9; frontend 4/4; brand 3/3; hard coding v2 9/18 (the other 9 unanswered at the 3-hour deadline).

opencode on your machine

Once: opencode auth login {{BASE_URL}}, then run opencode in any folder.

Or on a machine with m64: m64 qwen38-flash-next-strata-gb10 --ui opencode.

Errors

  • 503: still loading (or Strata stopped); retry.
  • 502: Strata did not answer the request.

Access

On the tailnet (the default bind) there is no token: anyone on the tailnet can use the endpoint. With a token (--host 0.0.0.0 etc.), send Authorization: Bearer <token>.

capsule.toml

schema = "m64/capsule@1"description = "Qwen3.8-Flash-Next IQ3_S on Strata (GB10). A local OpenAI-compatible chat endpoint for Qwen3.8-Flash-Next (Qwen's 125B-A6B MoE, ISTA-DASLab's GSQ-RCO IQ3_S GGUF) on Strata 0.2.13's DGX Spark engine, 64K context, with the MTP draft layer. Reads images, streams, tool calls. Needs no network."kinds = ["chat"]authors = ["dh7"]license = "mit"[hardware]platform = "linux-aarch64"gpu = "NVIDIA GB10"gpu_count = 1# Unified memory: the experts are read in place from shard 1 (55 GB), the n-gram table (shard 2, 29 GB) is# locked in RAM, plus the pack, the draft layer, the vision tower and a 64K int8 KV cache. Measured on# gx10: ~63 GB in use at peak over a 5-hour agent benchmark (MemAvailable never below 49 GB).vram_gb = 72driver = ">=580.65"disk_gb = 90[runtime]sandbox = "container"# No image: the Dockerfile builds Strata 0.2.13's linux-aarch64 packages on ubuntu:24.04, pinned by digest.gpu = trueshm_gb = 2internal_port = 9000health = "/health"expected_load_seconds = 900model_name = "qwen3.8-flash-next-iq3_s-strata"user = "65534:65534"# ISTA-DASLab's GSQ-RCO shards at the revision Strata 0.2.13 pins (setup.py HF_REVISIONS), and the# picture reader (mounted; pictures are off in this capsule).[[artifacts]]type = "hf"repo = "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF"revision = "ed59f92082b1e93c0e96d60a8b11aab089b52f09"include = ["README.md", "IQ3_S/*.gguf", "mmproj-Qwen3.8-Flash-Next-BF16.gguf"]mount = "/weights/qwen38fn"