Swift 27B vs. two Flash Next models on the IT Lab Bench

We measured three local AI models on the same real-work battery, on the same machine. None wins everywhere; each stands out at something different.

• Swift Flash Next scored best in programming and document reasoning, and was the only model to pass the most demanding software project.
• Swift 27B finished the battery fastest, generated text more quickly and earned a perfect score on the agentic series.
• All three tied on document reading and vision, and all three shared the same weak spot: structured output that does not always follow the requested format.

What we compared and why it matters

The IT Lab Bench is our own test for evaluating local AI on real office, development and analysis work. In the current edition of the bench we compared Swift 27B, Swift Flash Next and ThinkingCap Flash Next across three campaigns: 51 cases per model, 153 in total, with 195 real generation calls.

All of them ran with the same 16 GiB graphics card. The machine's installed RAM was 32 or 80 GB depending on the test, which is why this report speaks of RAM used rather than capacity. That is hardware within reach of a mid-sized Chilean company, and the underlying reason for this work is simple: a local model processes contracts, spreadsheets or code without that data leaving the organization. Each model was evaluated with its best documented configuration and with medium reasoning.

Results by test family

In programming, Swift 27B failed on tests and ThinkingCap failed to compile. A third project exercise only measured delivery and integration, with no full functional judge, so it is not shown as pass or fail. The writing controls and part of the reasoning series also require human review, which is still pending: those figures come from the automatic judge.

There is no single overall score. Questions, software contracts, cells and weighted points do not carry the same weight and are not added together.

Failures and patterns

The three models shared the same kinds of error, in different proportions:

• Output format. Several deliveries arrived with invalid structures or with data types other than those requested. In an automated process, that alone gets a response discarded.
• Traceability. In tasks with sources, the models attributed facts to references that did not support them, or gave figures when the right move was to abstain.
• Ambiguity. Faced with contradictory instructions, they sometimes chose the less safe reading or did not flag the uncertainty.
• Reasoning without an answer. In one delivery, Swift 27B spent its entire generation on reasoning and left the final answer empty. It was counted as a failure.

Speed, context and memory

Swift 27B's overall reading speed is pulled down by the image tasks; on the programming exercises, which are text only, it read at 919.6 tok/s, against 268.6 and 211.4 for the Flash Next models. Speed says nothing about code quality.

All three used speculative token prediction. Based on the proposed and accepted tokens in the report, acceptance was about 82% for Swift 27B, 70% for Swift Flash Next and 78% for ThinkingCap (our calculation from those figures). It is an efficiency metric, not a measure of correct answers.

On graphics-card memory, all three ran at the limit: VRAM peaks of 15.4 to 15.7 GiB, with less than 500 MiB free at the lowest point. On RAM, with the Flash Next models loaded, total use including the system exceeded 70 GB; with Swift 27B the minimum available RAM was 37.9 GiB. Sampling includes the desktop and other applications, so these are real-condition usage figures, not the model's isolated consumption.

What IT Lab recommends

This is IT Lab's professional judgment based on this measurement; it is not a certification or a purchase recommendation.

1. Choose by workload, not by ranking. For software development and document analysis, Swift Flash Next posted the best result in this pass. For agentic workflows, or where total time matters most, Swift 27B is the quicker option.
2. Always validate the format. Any integration that receives structured data from a local model should check the schema before using it.
3. Keep human review for texts involving sources, regulatory citations or commitments to customers.
4. Size with headroom. With 16 GiB of VRAM, all three models sit at the edge, and the Flash Next models also need generous RAM (over 70 GB in use together with the system). It is best to dedicate the machine to the model rather than share it with other heavy workloads.

Limitations and scope

• A single pass per model. There is no statistical significance and no universal winner.
• Official pass kept apart from diagnostics. The tables cover only the official pass. Earlier parameter sweeps and harness checks add no cases or points.
• Resumptions. The Swift 27B campaign ran in two stretches. The second kept the results of the first and reused an answer already generated, without regenerating it. For the Flash Next models, the agentic series was completed in a later run and the writing controls were re-judged on the same answers.
• Corrections. Cumulative times are computed over each model's comparable total, and reproduced submissions use their original time. Harness contracts were also corrected before this pass.
• Comparison limits. The profiles differ in engine, quantization, context and sampling settings: we compare usage profiles, not the isolated effect of model size. Concurrency and use of the full 262,144-token context were not tested. Time to first token and thermal data were not measured consistently.
• VRAM ceiling. This campaign ran close to the card's maximum, which shapes the observed speeds.
• Reasoning. Everything was measured with medium reasoning. A "reasoning off" mode does not mean the same thing in each model family and is not compared.

Frequently asked questions

Which of the three suits a small or mid-sized business? It depends on the workload. Swift Flash Next performed better in programming and documents; Swift 27B was faster and complete on agentic tasks. The sensible step is to test with your own data.

Is a 16 GiB card enough? For the card, yes, but without margin: VRAM use came close to the limit in every campaign. On RAM, the Flash Next models ask for much more than Swift 27B (over 70 GB in total use with the system).

Are these results final? No. They reflect one pass per model in the current edition of the bench, with human review still pending on some tests.

Contact

If your organization is considering local AI with data that stays in-house, the IT Lab team can help you design tests around your own work. Contact us.

<!-- PENDING: insert here the literal IT Lab Bench footnote ("*The IT Lab Bench is a living instrument…"); source file a01-como-evalua.md was not found. -->

The IT Lab Bench is a living instrument under continuous refinement. Some measurements come from earlier versions of the instrument and from previous generations of models, and some figures may contain inaccuracies that are corrected in each new edition. The conclusions express IT Lab's judgment based on its own measurements.