VEYMEER
Download
Hardware

It uses what the machine has

By default the engine looks at the machine and takes the fastest device it finds: an NVIDIA GPU, then any other GPU, then the processor. A model too large for one device is spread over what there is. This page says what runs where, and what we have measured ourselves.

Devices

What it runs on

Each card says how far we have got: measured by us, built but not yet measured on that hardware, or planned.

GTX 10-series through RTX 50-series, and their workstation and datacenter relatives

NVIDIA GPUs

The engine's own kernels. Needs the NVIDIA driver and nothing else.

Measured

Measured on an RTX 3050. Built for the other generations, which we have not yet measured ourselves.

Radeon, Arc, integrated graphics, Apple silicon

AMD, Intel and Apple GPUs

Through Direct3D 12, Vulkan or Metal. The engine asks the system how much memory the GPU has, places as many layers on it as fit, and runs the rest on the processor.

Built, not yet measured

Run on our test machine on an NVIDIA and an Intel GPU through Direct3D 12, with the same answers as the processor gives; Gemma 4 E2B wrote 24.7 tokens a second this way on the RTX 3050. Not yet measured on AMD, Intel Arc or Apple hardware.

x86-64 with AVX2, FMA and AVX-VNNI where the processor has them; ARM with portable kernels

The processor

When there is no GPU, or for the part of a model the GPU cannot hold.

Measured

Measured on an Intel Core i5-13420H. On ARM processors, Apple silicon included, the engine runs but has no tuned kernels yet, and is several times slower there.

Any mix the machine has

Several GPUs in one machine

One model shared across them, each holding as much as its memory allows.

Built, not yet measured

Run on our test machine with one model split across an NVIDIA and an Intel GPU, with the same answers. Not yet measured on a machine with several fast GPUs.

Any mix of the above, over Ethernet, Wi-Fi or Thunderbolt

Several machines

One model across machines on the same network: each lends its GPUs, one coordinates.

Built, not yet measured

Built and tested on one machine. Not yet measured across real machines.

 

NPUs

Neural processors in recent laptops.

Planned

Not supported yet.

An integrated GPU has to earn its place. It is not always faster than the processor beside it. On Intel's UHD graphics, which we have measured, the processor is about six times faster. So the engine times a token on both the first time it serves a model on such a machine, serves from the faster, and remembers. You can still name the GPU yourself.
Whose memory. A discrete GPU plans on its own memory, less what other programs hold. An integrated GPU shares the machine's, and may plan on up to half of it; Apple silicon, on two thirds. veymeer doctor shows the figure for each GPU.
Why so careful. We have one test machine today. Everything marked "built" runs there or in tests; none of it has a number on this site until we have measured it on the hardware it is for. veymeer devices shows what the engine finds on yours, and veymeer bench measures it.
Systems

Operating systems

The engine is one self-contained program on each system. On an NVIDIA GPU it needs the graphics driver and nothing else: no CUDA toolkit, no Python, no runtime to install.

  • Windows 10 and 11, x64MeasuredBuilt and measured.
  • Linux, x64PlannedIn preparation. Not yet verified on a real machine.
  • macOS, Apple siliconPlannedIn preparation. Not yet verified on a real machine.
Memory

Where would a model live on your machine?

The engine fills the GPU first, then system memory, then reads the remainder from disk. Pick a machine and a model to see the split it would choose.

The model's weights Fits the GPU
On the GPU
In system memory
Read from disk
On the GPU
4.4 GB
In system memory
0 GB
Read from disk
0 GB
Conversation cache
0.2 GB

Fits the GPU

The whole model and its conversation cache are on the GPU. This is the case the benchmarks measure.

An estimate, for an NVIDIA GPU, from the model's file size and the context length. On other GPUs the layers that do not fit run on the processor instead of waiting in memory. Other programs using the GPU change it. On your machine, veymeer doctor and the engine's log say what it actually did.

Tier 1GPU memory

As much as fits

The engine first makes room: fewer conversations side by side, then a more compact conversation cache. It says in its log when it does.

Tier 2System memory

What does not fit waits next door

The remaining layers are held in system memory and brought to the GPU as each pass reaches them. A few layers cost a few percent; many cost more.

Tier 3Disk

Larger than memory, too

What system memory cannot hold stays in the model's file and is read ahead of each pass, at the disk's speed, without taking the rest of the machine's memory.

Clusters

Several machines, one model

Machines on the same network can run one model together: each lends its GPUs, and the one that serves the API coordinates. The machines need not match.

  1. Lend a machine

    On each machine that joins, veymeer node --beacon prints a join key and lets a coordinator find it on the local network.

  2. Serve from one

    On the machine that answers, veymeer serve --discover --cluster-key KEY finds the others and spreads the model over them.

  3. Carry on

    A node that lacks the model file is sent it once. If a node stops answering, the others take over its share and running conversations continue.

Security. Every connection is encrypted, and both ends must hold the join key: a machine without it can neither use a node nor pose as one. Nodes accept connections from private networks only, unless told otherwise.
How large. Every tier can cluster: up to 3 other machines on Individual, 4 on Individual-PRO, 10 on Individual-MAX, and any number on Enterprise-MAX. The tiers
Status. Clustering is built and tested on one machine standing in for several. We have not yet measured it across real machines, over Ethernet, Wi-Fi or Thunderbolt, so there are no cluster numbers on this site.

Find out what your machine does

veymeer doctor checks the processor, the GPUs, the models and the port, and says what would stop the engine before it does.

Download Benchmarks →