Models

Bring the model. It runs.

VEYMEER reads GGUF files, the format open models are shared in. Point it at one, name a model Ollama already downloaded, or fetch one from the Hugging Face Hub with veymeer pull. Every family on this page runs on the processor and on any GPU through WebGPU, and the engine's tests check that both give the same answers. On NVIDIA GPUs we have run and measured Llama, Qwen, Mistral and Gemma 4; the other families are built for them and not yet measured there.

Models and versions

from 27 makers, from GPT-2 to Gemma 4

File formats

from about 1.5 bits a weight to full 32-bit floats

Chat formats

each family spoken to in its own, tools included

Families

What it runs

Every model on this list loads and answers on the processor and through WebGPU. Find yours:

Every common way of packing the weights

A model is published in several sizes of the same weights. Smaller files fit smaller GPUs and answer faster; larger ones are closer to the original. Take the one that fits your machine: the engine reads them all.

Full and half precision

F32F16BF16

Block formats

Q4_0Q4_1Q5_0Q5_1Q8_0

K formats

Q2_KQ3_KQ4_KQ5_KQ6_K

I formats

IQ1_SIQ1_MIQ2_XXSIQ2_XSIQ2_SIQ3_XXSIQ3_SIQ4_NLIQ4_XS

IQ1 to IQ3 on the processor and NVIDIA GPUs; not yet through WebGPU

Newer formats

MXFP4TQ1_0TQ2_0

Conversations

Each model in its own words

A chat model was trained on one way of marking who speaks, and on one way of calling tools. The engine writes each conversation in the model's own format, offers your tools the way that model expects them, and reads its calls back into OpenAI's tool_calls. The arguments always fit the function's schema: the engine only lets the model write what the schema allows.

ChatMLLlama 3MistralGemmaGemma 4Phiharmony (gpt-oss)

Any OpenAI client

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5:7b",
    "messages": [{"role": "user", "content": "Weather in Oslo?"}],
    "tools": [{"type": "function", "function": {
      "name": "get_weather",
      "parameters": {"type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]}}}]
  }'

Measured

Gemma 4 on a 6 GB laptop GPU

Google's newest small models, on the same RTX 3050 laptop GPU as every other number on this site. Run on 18 October 2026.

Writing new text

Gemma 4 E2B76.0

Gemma 4 E4B42.6

Editing code

Gemma 4 E2B253.0

Gemma 4 E4B123.0

Context, Gemma 4 E2B

97,280 tokens

The longest conversation the engine fits beside the whole model on the 6 GB GPU, by itself.

Context, Gemma 4 E4B

17,408 tokens

The same for the larger model.

Without CUDA

24.7 tok/s

Gemma 4 E2B through WebGPU and Direct3D 12: the path AMD, Intel and Apple GPUs take, here on the same NVIDIA GPU. The same text as on CUDA.

How close to the hardware. Writing one token at a time means reading the whole model once per token. At the 156.6 GB/s this GPU's memory delivers, that allows at most 106.9 tokens a second for E2B and 52.4 for E4B. Writing new text, the engine reaches 76 and 42.6. Editing code it passes that limit more than twice over, because most of an edit is text the engine can check many tokens of at once. No other engine was run on these models here, so this section compares the engine with the hardware, not with another engine.

Getting one

Where models come from

Already downloaded

Models Ollama downloaded are found and used where they are, by the same names: qwen2.5:7b. Nothing is copied.

From the Hugging Face Hub

veymeer pull owner/repo:Q4_K_M downloads one file of a repository, resumes if interrupted, and checks it against the Hub's checksum.

Any GGUF file

Name the file: veymeer serve --model D:\models\my-model.gguf. veymeer models lists what it finds and says whether each one runs.

A small model of the same family makes a large one faster as its draft model, and the engine picks one from what is installed by itself. The memory planner shows where a model would live on your machine.