Avalon and Strata on 16 GB: our experience
We ran Avalon and Strata 0.2.10 with Qwen3.8 Flash Next on a 16 GB card: what worked, what it took, real speeds and how far to trust a single measurement.
We used Avalon and Strata 0.2.10 with Qwen3.8 Flash Next on an RX 9070 XT with 16 GB, on November 4 and 5, 2026. Strata gave us less friction and worked with both models we gave it. Avalon ran with its own model, but it needed manual tuning and, with default threads, didn't deliver more sustained speed. This is what we saw on this machine, not a verdict on the engines.
- Similar sustained speed. Over 49 tasks, Swift on Strata generated 66.26 tok/s and IQ3_KS on Avalon 65.46, with default threads.
- Avalon failed with Swift. It aborted on the first task, and we didn't identify the cause. With its own IQ3_KS model it finished the battery with no aborts.
- A single run misleads. The same task, with the same configuration, gave 68.92 and 88.43 tok/s.
- Scores are separate. The full ranking is in Unsloth and Avalon IQ3_KS on the ITLab Bench.
Overall comparison
The versions are Strata 0.2.10 (engine 0.1.56) and Avalon at commit 16de68d28.
| Strata 0.2.10 | Avalon | |
|---|---|---|
| Installation | Precompiled engine, hash verified | Built from source with ROCm for gfx1201 |
| Models that finished the battery | Swift and Unsloth | IQ3_KS, the only one we ran in full |
| Swift | Full battery, 94.7 points | Aborted on the first task |
| Unsloth | Full battery, 95.9 points, with a hand-built pack | 7 of 7 on the only task tried, twice |
| IQ3_KS | Doesn't load | Full battery, 90.5 points |
| Tuning for this card | 262,144 context with a 1,800 MiB reserve | IQ3_KS: batch of 4608 and 1,024 MiB reserve. Swift: `GGML_CUDA_MOE_CACHE_MIN_EXPERT_KB=256` |
| Peak VRAM in the battery | 14.30 and 14.35 GiB | 15.06 GiB |
| Model load | 40 s (Swift) and 70 s (Unsloth) | Between 30 and 55 s |
| Infrastructure errors in the batteries | 0 in two batteries | 0 in one battery |
What it took on Avalon
- It didn't work with Swift. Without the documented setting, Swift aborts on load. With it, Swift loaded, read the prompt and aborted after about 183 generated tokens. We didn't identify the exact cause, so we don't pin it on Swift or on Avalon.
- The published configuration didn't load for us at a context of 262,144. With `-ub 9536`, IQ3_KS ran out of memory on load. The process hung holding the VRAM and had to be killed by hand. We solved it with a 1,024 MiB reserve, as in the project's issue #14, and a batch of 4608. With Unsloth, by contrast, Avalon dropped to 4608 on its own.
- One value means something else. With `reasoning_budget_tokens=0` Avalon doesn't reason; in Strata, 0 means no cap.
- Its model has no vision. The IQ3_KS repository doesn't include a projector, so we used Unsloth's.
- What did work. IQ3_KS finished all 49 tasks and the four later tests with no aborts.
What it took on Strata
- Unsloth needed manual work. The Strata 0.2.10 installer pins an earlier version of the model, so we built the pack with `tools/iq_pack.py --compat-bf16`. According to Strata's notes, that raises perplexity by 6 to 9% on AMD. It may have cost Unsloth quality and speed, but we didn't measure it.
- The new version didn't speed up Swift. On 29 tasks with similar token counts, 0.2.10 gives 67.30 wall-clock tok/s and the earlier campaign, on 0.2.5, 68.31. Reading rose 3.3%. The differences are small, and the batteries aren't identical.
- There's a loose end. GPU vision in 0.2.10 is configured but untested. In these tests, vision ran on the CPU.
Speed
Each figure is total tokens divided by total engine time, not an average per task. "Similar" means tasks without images where both configurations generated a similar number of tokens (±25%), and each comparison uses its own group.
| Group | Swift + Strata | Unsloth + Strata | IQ3_KS + Avalon |
|---|---|---|---|
| All 49 tasks | 66.26 | 49.79 | 65.46 |
| No images (45) | 66.26 | 49.78 | 65.53 |
| No images, no C03 (44) | 66.20 | 49.93 | 65.63 |
| Similar, Swift and IQ3_KS (19) | 63.34 | — | 66.86 |
| Similar, Swift and Unsloth (19) | 64.16 | 50.72 | — |
With reading included, the 19 similar cases for Swift and IQ3_KS come to 61.26 and 62.58 tok/s. Those two packages run at very similar speeds on this machine, and the lead changes sides depending on the group. Swift generates 33% faster than Unsloth on the same engine, although Unsloth uses the manual pack and a different `top_p`.
What Avalon publishes. Its README claims 98 tok/s on an RX 7900 XT with 20 GB, with a short prompt, greedy decoding and 512 tokens. Our conditions are different, so we neither contradict nor confirm it.
What issue #14 reports. In issue #14, a user with this same card reports that `GGML_CUDA_MOE_HOSTED_THREADS=2` nearly doubles their generation, from about 41 to between 73 and 82 tok/s. The project's author says about 40 tok/s isn't expected and cites about 78 tok/s on his card limited to 16 GB. Our battery ran with default threads, and we start from 65 tok/s sustained. With 2 threads we only tried one task, with no gain beyond the variation between runs: 82.27 tok/s with IQ3_KS and 57.87 with Unsloth. We don't know how IQ3_KS would do with 2 threads over the full battery.
Reading
| Prompt size, no images | Strata | Avalon |
|---|---|---|
| Under 1,000 tokens | 438 | 182 |
| 1,000 to 3,999 | 833 | 844 |
| 4,000 and up | 1,091 | 659 |
Figures are tok/s. Strata read short and long prompts faster, and they tied on medium ones. On the four image cases, Avalon shows 61.4 tok/s and Strata 1,599.4 with Swift, adding up to 342 s of reading against 14 s. That doesn't prove Avalon is slow at reading images: the engines don't read the same number of tokens, and the figure may include CPU processing. We also didn't resolve the effect of `-ub 4608` on Avalon's reading.
The only head-to-head with the same model
Unsloth is the only model that ran on both engines, and only on task `n01-a`.
| Run | Generation (tok/s) | Reading (tok/s) | Tokens | Peak VRAM (GiB) |
|---|---|---|---|---|
| Strata 0.2.10 | 65.00 | 1,544.6 | 21,637 | not isolated |
| Avalon, default | 55.90 | 823.0 | 29,258 | 15.65 |
| Avalon, 2 threads | 57.87 | 1,085.8 | 18,723 | 15.64 |
Strata was faster in these runs, and all three passed 7 of 7. It's a single task, so it's a hint and not a measurement of the engine.
How far you can trust a measurement
We repeated `n01-a` with IQ3_KS on Avalon, first unchanged and then with three settings.
| Run | Generation (tok/s) | Tokens | Reading (tok/s) | MTP acceptance | Peak VRAM (GiB) |
|---|---|---|---|---|---|
| Battery, unchanged | 88.43 | 19,926 | 966.3 | 0.904 | not isolated |
| Control, unchanged | 68.92 | 17,639 | 977.3 | 0.905 | 15.03 |
| 2 threads | 82.27 | 21,224 | 1,153.7 | 0.938 | 15.03 |
| Prefetch 6 | 90.13 | 17,627 | 1,182.0 | 0.889 | 15.91 |
| Context 131,072 | 92.97 | 14,203 | 1,213.6 | 0.916 | 15.02 |
The first two rows are the same task and the same configuration, with almost the same acceptance, and they differ by 28%. We don't know why. The tuned runs landed almost within that variation, and each ran once, in a fixed order, without alternating. So they can't show that a setting speeds up Avalon. Prefetch 6 also went over the 15.5 GiB limit we used in the test.
The lesson applies to any engine: an isolated high figure doesn't sum up real use. On this task IQ3_KS reached 88.43 and 92.97 tok/s, and over 49 tasks it sustained 65.46.
What we conclude
On this machine, Strata gave us the path with less friction. Avalon is viable with its own model, but on 16 GB it needs manual tuning. This doesn't let us say one engine is better in general: IQ3_KS only loads in Avalon, each package has different settings, and our 65.46 tok/s is with default threads. Avalon's published reference uses a 20 GB card we didn't have here, and its author cites about 78 tok/s on his card limited to 16 GB.
Limits
- One run per battery. Small speed differences aren't conclusive.
- Different models. Except for task `n01-a` with Unsloth, we compare packages of model plus engine.
- Conditions. RX 9070 XT with 16 GB, Ryzen 7 5700X, 94 GB of RAM, CPU capped at 80% and the desktop in normal use.
- Unresolved. The cause of the Swift abort, the 28% gap between identical runs and the effect of `-ub 4608` on reading.
Frequently asked questions
Is Avalon faster than Strata? On this card and with default threads, not in sustained speed: 65.46 tok/s with IQ3_KS against 66.26 for Swift on Strata. Its published 98 tok/s uses another card and other conditions, which we didn't test, and its author cites about 78 tok/s on his card limited to 16 GB.
Does Avalon work with any Flash Next model? It finished the battery with its IQ3_KS and Unsloth ran on the only task tried, but Swift aborted and we didn't identify the cause.
Can you run Avalon on 16 GB? Yes, with its own model and with the batch lowered to 4608 and the reserve to 1,024 MiB. The published configuration didn't load for us at a context of 262,144.
Which setting do you recommend to speed up Avalon? None. One unchanged run gave 68.92 and another 88.43 tok/s, and the settings landed almost within that variation. As our own suggestion, not a measured result: `GGML_CUDA_MOE_HOSTED_THREADS=2` is the first thing we'd try on this card, given what issue #14 reports, although it gave us no demonstrable gain.
Sources
- Strata, the engine used for the Swift and Unsloth configurations (version 0.2.10).
- Avalon, the engine used with IQ3_KS (commit 16de68d28).
- Qwen3.8 Flash Next Avalon GGUF, the repository for the IQ3_KS model.
- Unsloth, Qwen3.8 Flash Next GGUF, the repository for UD-IQ4_XS and the vision projector.
- Avalon README, with the 98 tok/s figure and its conditions.
- Avalon issue #14, on the 16 GB RX 9070 XT.
Contact
If your organization is considering local AI, ITLab can help you size the hardware and test models and engines on your real cases. Contact us.
The conclusions reflect ITLab's judgment based on its own measurements, on this machine and with these versions.
Do you need someone to take care of your technology?
Tell us what you need to organize, protect or solve. The first conversation has no cost: we assess whether our model fits you and what the next steps would be.
Let's talk about your business