Qwen3.8-Flash-Next

The numbers at a glance

llama.cpp server log · 346 of 459 responses from 200 tokens up · run of 3/4 September 2026, 13 h 35 min. The server ran for 844.3 minutes in one go. From the first task to the end of the last one it is 837.0 minutes; this includes one idle pause of 21.9 minutes. The working window is therefore 815 minutes. Pure compute time on the GPU is 798.8 minutes. · Raw log

Results

Prefill over context depth

A peak figure at short context says little about how a model feels after hours. Hence the curve.

Context depth in thousands of tokens · Instantaneous rate within a single continuous prefill: task 185882 of this run processed 69,985 tokens in one go. llama.cpp reports an intermediate status every 2,048 tokens; the rate is the quotient of tokens and time between two consecutive reports. 32 steps remain, values under 1,500 tokens are excluded. From 2,048 to 68,827 tokens the rate drops by a factor of 1.58. Every point can be traced in the linked server log.

llama.cpp launch parameters

Context

131,072

KV-Cache

q8_0 / q8_0

Micro-Batch

512

Speculative

MTP, Shared-Q8_0, n-max 2

Build

—

llama-server ^
  -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf ^
  -md mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf ^
  --spec-type draft-mtp --spec-draft-n-max 2 ^
  --mmproj mmproj-F16.gguf --alias qwen3.8-flash-next ^
  --host 127.0.0.1 --port 8099 --device Vulkan0 --gpu-layers all ^
  --n-cpu-moe 0 --fit off -fa on --load-mode mmap --lazy-mode on ^
  --ctx-size 131072 --parallel 1 --kv-unified ^
  -ctk q8_0 -ctv q8_0 -b 2048 -ub 512 ^
  --ctx-checkpoints 4 --checkpoint-min-step 4096 --jinja ^
  --reasoning on --reasoning-format deepseek ^
  --reasoning-effort xhigh --reasoning-budget 12000 ^
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 ^
  --presence-penalty 0.0 --repeat-penalty 1.0

Assessment

What went wrong

• The loading screen says “Featherstorm wordt geladen”, Dutch instead of German. Exactly one spot in the whole game; the rest is German throughout.
• The game renders to a 1280×720 canvas. On a phone 390 pixels wide it is scaled down to 390×219, playable, but the targets are small.
• The run needed three sessions (20, 376 and 419 minutes). Between the sessions the context was lost and had to be rebuilt.

llama.cpp server log · 346 of 459 responses from 200 tokens up · run of 3/4 September 2026, 13 h 35 min. The server ran for 844.3 minutes in one go. From the first task to the end of the last one it is 837.0 minutes; this includes one idle pause of 21.9 minutes. The working window is therefore 815 minutes. Pure compute time on the GPU is 798.8 minutes.

The numbers at a glance

Halogen 0.6.3 server log, written through a proxy · 652 of 891 responses from 200 tokens up · run of 14/15 September 2026. The log starts on 14 September at 15:32:17 and ends on 15 September at 10:13:53, which is 1,121.6 minutes. Between the end of a response and the next request it contains seven pauses over 60 seconds, 418.3 minutes in total, the longest 292.9 minutes during the night. The working window is therefore 703 minutes. Pure compute time on the server is 634.9 minutes. · Raw log

Results

Decode over context depth

A peak figure at short context says little about how a model feels after hours. Hence the curve.

Context depth in thousands of tokens · The 652 scored responses sorted by prompt size and split into six equally sized groups; each point is the median prompt size and the median speed of its group. Measured between 729 and 143,082 tokens of context. 268 of these responses ran next to a second request, Halogen serves two slots. Every point can be traced in the linked server log.

Prefill by new tokens per request

For every request Halogen logs the prompt tokens, the part of them served from the cache and the prefill time. The rate is the number of new tokens divided by that time. In agent work a request brings only 654 new tokens in the median, and requests with 32 to 511 new tokens already took 1.46 s in the median. That is why the median over all 891 requests is 347.85 t/s, but 1,120.69 t/s from 8,192 new tokens up.

All 891 requests split into six bands by their number of new tokens, time and rate as the median of each band. Traceable in the serve_api lines of the linked server log; parse_logs.py in the evidence repository prints the bands and the figure from 8,192 new tokens up.

Server configuration

Server

halogen-flash-server 0.6.3

Container

ghcr.io/peonist-ai/halogen-flash-server:0.6.3

Weights

W4B with quality overlay, 741 tensors, 2.40 GiB

Context

262,144 per request, one KV pool for two slots

Speculative

MTP at depth 1, only while no second stream runs, plus prompt lookup (pld in the log)

Reasoning

xhigh

Sampling

temp 1.0 · top_p 0.95 · top_k 20 · min_p 0.0 · presence 0.0, only for fields the request does not set

Vision

on

Memory

68.0 GiB of weights locked in RAM, 7.2 GiB KV pool, 21.1 GiB working memory

Lookup table

47.7 GiB, read from disk as needed

iGPU

2.0 GiB reserved for the iGPU in firmware

System

Linux

  Halogen Qwen3.8-Flash-Next W4B + Qualitaets-Overlay + Vision  (ghcr.io/peonist-ai/halogen-flash-server:0.6.3)

  Kontext  : 262144 je Request, KV-Pool 262144 Positionen, 2 Slot
  Budget   : max_tokens-Default 65536 je Anfrage, Cap 262144, Slots 2
  Reasoning: xhigh, getrennt in reasoning_content
  Sampling : temp 1.0  top_p 0.95  top_k 20  min_p 0.0  presence 0.0  (nur fuer fehlende Felder)
  Cache    : Prompt-Cache 2, Queue-Timeout 28800 s
  Vision   : an (qwen38-flash-next-vision.hgn)
  RAM      : 123 GiB sichtbar, 13 GiB belegt, 1179 freie 2-MiB-Bloecke
  Endpoint : http://127.0.0.1:18099/v1 (VS Code ueber Live-Log-Proxy :8099)   Modell: halogen-qwen3.8-flash-next-w4b-quality-overlay
  Watchdog : 0 (0 = aus)

  Laden: gemessen 30 s (13.09., Pool 32768); bei fragmentiertem Speicher ueber 5 min.

All figures come from the header and the startup messages of the raw log. There is no launch line as with llama.cpp, Halogen runs as a container. Above is the header the start script writes into the log, verbatim.

Assessment

The assessment of this run is still pending.

What went wrong

• At startup the server reported 13.7 GiB of memory already in use and warned that prefill could therefore run several times slower than published, with pauses of a minute or more.
• 15 requests ended with HTTP 400 without a single token, 14 of them between 16:11 and 16:13. At 17:08 another one ended with HTTP 504.
• Seven responses ran into their limit of 8,192 tokens between 17:28 and 18:50 and were cut off there.
• Between 23:46 and 00:18 the server aborted three requests: twice because the engine delivered nothing for 300 seconds during decode, once for 1,800 seconds during prefill.
• At 07:31 and at 07:38 VS Code closed the connection in the middle of a response.
• At 08:22 the server refused two requests: three images were announced, none arrived.
• The number of messages per request drops back to seven or fewer 25 times in the log, the last time at 08:33, from 311 to 7.

Halogen 0.6.3 server log, written through a proxy · 652 of 891 responses from 200 tokens up · run of 14/15 September 2026. The log starts on 14 September at 15:32:17 and ends on 15 September at 10:13:53, which is 1,121.6 minutes. Between the end of a response and the next request it contains seven pauses over 60 seconds, 418.3 minutes in total, the longest 292.9 minutes during the night. The working window is therefore 703 minutes. Pure compute time on the server is 634.9 minutes.

The numbers at a glance

llama.cpp server log · 92 of 106 responses from 200 tokens up · run from August 2026. The server ran for 374.4 minutes in one go. From the first task to the end of the last one it is 369.7 minutes, without a single pause of more than 60 seconds in between. Pure compute time on the GPU is 368.1 minutes. · Raw log

Results

Prefill over context depth

A peak figure at short context says little about how a model feels after hours. Hence the curve.

Context depth in thousands of tokens · Instantaneous rate within a single continuous prefill: task 176155 of this run processed 35,009 tokens in one go. llama.cpp reports an intermediate status every 2,048 tokens; the rate is the quotient of tokens and time between two consecutive reports. 15 steps remain; leftover pieces under 1,500 tokens are discarded, because there the call overhead is measured rather than depth. From 2,048 to 33,642 tokens the rate drops by a factor of 1.19. Every point can be recalculated from the linked server log.

llama.cpp launch parameters

Context

262,144

KV-Cache

q8_0 / q8_0

Micro-Batch

512

Speculative

MTP, older version (not the one from Unsloth)

Build

580e88d

llama-server -m Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf ^
  --mmproj mmproj-F16.gguf --alias qwen3.8-flash-next ^
  --host 127.0.0.1 --port 8099 --device Vulkan0 ^
  --gpu-layers all --n-cpu-moe 0 --fit off -fa on ^
  --load-mode mmap --lazy-mode on ^
  --ctx-size 262144 --parallel 1 --kv-unified ^
  -ctk q8_0 -ctv q8_0 -b 2048 -ub 512 ^
  --ctx-checkpoints 4 --checkpoint-min-step 4096 ^
  --jinja --reasoning on --reasoning-format deepseek --reasoning-effort xhigh

Assessment

This assessment is a proposal and has not yet been confirmed by Kai Bennett. Code quality and scope rest on counted files, source and test lines and repair scripts in the evidence repository.

What went wrong

No self-corrections were logged.

llama.cpp server log · 92 of 106 responses from 200 tokens up · run from August 2026. The server ran for 374.4 minutes in one go. From the first task to the end of the last one it is 369.7 minutes, without a single pause of more than 60 seconds in between. Pure compute time on the GPU is 368.1 minutes.