The same weights, the same prompts, the same GPU
Every engine here loads the very same model file and answers the very same requests, alone on the GPU. We report medians, not best runs, and we report the rows we lose.
A prompt the engine has never seen
The headline measurement: how fast the answer is written, by kind of work. Nothing is remembered from an earlier run.
Writing speed on a prompt the engine has never seen
tokens per second| VEYMEERwith a draft model | VEYMEERno draft model | llama.cppsame draft model | llama.cpp | Ollama | |
|---|---|---|---|---|---|
| Code editing | 241.4 | 241.6 | 55.0 | 33.4 | 24.6 |
| Code explaining | 61.6 | 46.9 | 43.0 | 33.4 | 24.4 |
| Answering from a document | 64.0 | 52.1 | 44.5 | 33.4 | 24.0 |
| Summarising | 46.9 | 36.4 | 38.9 | 33.5 | 24.4 |
| Chat | 49.5 | 34.4 | 34.1 | 33.7 | 25.3 |
| Code generation | 59.3 | 37.3 | 44.7 | 33.8 | 25.3 |
| All prompts | 60.9 | 44.1 | 42.5 | 33.5 | 24.6 |
Median over the prompts of each kind; every prompt's first run. 26 prompts, 256 tokens each, greedy decoding, context 2048.
Where VEYMEER is behind, and what it does about it
Reading a prompt nobody has seen is the one measurement llama.cpp wins on this GPU.
Time to the first word
seconds, less is better| VEYMEER | llama.cpp | Ollama | |
|---|---|---|---|
| A prompt it has never seen | 0.89 | 0.55 | 0.64 |
| The same, with precision = fast | 0.74 | – | – |
| A prompt it has seen before | 0.06 | 0.10 | – |
| A prompt read ahead with /prime | 0.11 | – | – |
Median over all prompts, including the time to read the prompt.
precision = fast gives part of it back for those who prefer the time.
/prime is already read when the question arrives, and the first word comes in about a tenth of a second.
JSON, schemas and tool calls
Three schemas, three tool calls and one request in JSON mode, timed whole, prompt included.
Answers held to a form
seconds for the whole request, less is better| VEYMEERwith a draft model | VEYMEERno draft model | llama.cppsame draft model | llama.cpp | |
|---|---|---|---|---|
| A record (JSON schema) | 1.06 | 1.14 | 3.29 | 3.58 |
| An invoice extracted from text | 1.83 | 2.39 | 6.44 | 6.57 |
| A review analysed into fields | 2.30 | 2.63 | 4.98 | 5.00 |
| Tool call: read a file | 0.19 | 0.32 | 0.76 | 0.75 |
| Tool call: search | 0.33 | 0.45 | 1.08 | 1.10 |
| Tool call: run a command | 0.30 | 0.34 | 1.03 | 1.02 |
| JSON mode | 0.66 | 1.24 | 1.70 | 1.69 |
| All seven | 6.68 | 8.51 | 19.28 | 19.71 |
Seven requests at temperature 0, best of two, wall clock including the prompt. Every answer fits its schema in every engine.
When the model does not fit the GPU
The same 7B model, with the GPU limited to 2.4 GB so that most of it has to live elsewhere.
A model larger than the GPU's memory
tokens per second| VEYMEERwith a draft model | VEYMEERno draft model | llama.cpp | |
|---|---|---|---|
| Writing speed | 27.6 | 12.5 | 9.9 |
The 7B model with 2.4 GB of GPU memory left to it, so 17 of its 28 layers do not fit. Mean of three prompts. llama.cpp with 12 layers on the GPU, the most that fit.
No GPU at all
tokens per second| VEYMEERprecision = balanced | VEYMEERprecision = high | llama.cpp | |
|---|---|---|---|
| Writing speed | 7.87 | 4.58 | 8.44 |
The 7B model on the processor alone (Intel Core i5-13420H, 32 GB DDR4).
The same question a second time
Kept apart from the headline on purpose: what an engine remembers between requests makes a repeated prompt a different measurement.
The same prompt, asked again
tokens per second| VEYMEERmemory on (the default) | VEYMEERmemory off | llama.cppn-gram speculation | llama.cpp | Ollama | |
|---|---|---|---|---|---|
| All prompts | 545.2 | 42.6 | 80.4 | 33.4 | 24.5 |
Median of each prompt's second and third run. This measures answering a question again, not general speed.
Speed is only worth reporting if the answer is the same
answers, with and without a draft model, each compared token by token with the same model decoding one word at a time.
Not close: the same tokens, in the same order, to the end.
prompts gave the same text on every run.
Gemma 4 on the same GPU
Google's Gemma 4 E2B and E4B on the RTX 3050, run on 18 October 2026. No other engine was run on these models here, so the last row compares the engine with what the GPU's memory speed allows instead.
Gemma 4 on the same 6 GB laptop GPU
tokens per second| Gemma 4 E2B | Gemma 4 E4B | |
|---|---|---|
| Writing new text | 76.0 | 42.6 |
| Editing code | 253.0 | 123.0 |
| What the GPU's memory speed allows a model reading every weight once a token | 106.9 | 52.4 |
Gemma 4 E2B and E4B, Q4_K_M, on the RTX 3050 6 GB laptop GPU, 18 October 2026. Greedy decoding. Not compared with another engine: none was run on these models here.
How it was measured
- GPU. NVIDIA GeForce RTX 3050 6GB Laptop GPU.
- Machine. Intel Core i5-13420H, 32 GB, Windows 11, on mains power.
- Model. Qwen2.5 7B, Q4_K_M: 4.68 GB GGUF, the same file loaded by every engine.
- Requests. 26 prompts in six kinds of work, 256 tokens each, context 2048.
- Decoding. Greedy (temperature 0), raw prompts, no repeat penalty in any engine. Ollama applies one by default; it is turned off, because it changes the text.
- One engine at a time. Each alone on the GPU, after two warm-up requests that are not counted.
- Medians. A kind of work's figure is the median over its prompts. No best run is picked.
- First runs. The headline is each prompt's first run. Repeats are reported separately.
- Nothing remembered. VEYMEER's memory of earlier answers is off in the headline rows, although it is on by default.
- Cross-checked. Speeds are measured by the client and agree with each engine's own report; GPU power, clock and temperature are recorded for every run.
Measure it on your own machine
veymeer bench runs three kinds of work on any installed model and prints the speed of each.