Written 31 July 2026, updated 24 August. The setup now runs llama.cpp with Qwen3.6-35B-A3B instead of vLLM with Qwen3-14B. The numbers in the table come from that first build.
Anyone looking to run a language model at home runs into the same advice everywhere: buy NVIDIA. I went with an Intel Arc Pro B70 and I would make that choice again. What the card gives you is 30.3 GiB of usable VRAM, and that is the difference between a model that barely fits and a model with room for context on top. There is tuition to pay. This piece is about how much.
The card sits in a privileged LXC container on Proxmox with 16 GiB of system memory, eight CPUs and 250 GB of disk. For inference that is plenty. For compiling things yourself it is not, and that turned out to matter.
What it costs to get started
Three things held me up longer than I expected. I am listing them because you can clear all three in an afternoon if someone tells you about them, and otherwise you lose a week.
First, the card is only recognised by a sufficiently recent compute runtime. If clinfo reports zero platforms, your NEO driver is too old for PCI id e223. There is nothing wrong with your configuration, and the error message points you nowhere useful. Update it and the card appears.
Then there is the container’s cpuset, which on my machine was not a tidy 0 through 7 but 5-6, 8, 11-12, 18, 20, 22. oneCCL wants an affinity CPU that actually exists in that set. Name one that does not and vLLM trips over failed to start worker # 0, which reads like a permissions problem when it is nothing of the sort. Worse, the set shifts when you reboot the host, so a configuration that worked on Monday can be broken on Tuesday. I ended up wrapping the whole thing in a script that recalculates the affinity on every start.
Finally, building vLLM yourself will not work on 16 GiB, because the linker runs out of memory. Use Intel’s own container, intel/llm-scaler-vllm:latest. The documentation advises against :latest, and it is still the road that gets you there.
Pick your model carefully and it simply runs
This is the real constraint of the Intel platform, and one you can work around as soon as you know about it. Intel’s XPU kernels are dependable for architectures Intel has explicitly validated. With newer models things break at the kernel level, and that happens after loading, on the first real inference, which makes it look as though you configured something wrong.
The rule of thumb I came away with: dense architecture, AWQ or GPTQ, and at least six months old. Anything that fits that description runs.
| Configuration | Result |
|---|---|
| Qwen3-14B-AWQ | 55 tok/s at 131k context, main model at the time |
| Qwen3-32B-AWQ | 20.6 tok/s at 72k context |
| gpt-oss-20b MXFP4 | 61 tok/s, weak at agentic tool use |
| Qwen3.5-9B FP8 | broken Gated Delta Network kernel |
| gemma-4-12B w4a16 | fails on every attention backend |
| llama.cpp via Vulkan | 34 tok/s, works without oneAPI |
Two of the six fell over on a broken kernel. That sounds bad until you consider that the four that do work cover every workload I have, and that the top one manages 55 tokens per second with the full 131k context available. This is not a fallback you tolerate. You just get on with your work.
The bottom row deserves its own mention. llama.cpp on Vulkan runs without oneAPI, without Intel’s container, without any of the SYCL scaffolding. Slower, yes. If you want to get going without spending a day on drivers first, it will have you up inside an hour. It has since become my default, and there is now a 35-billion-parameter model running on this card.
Image analysis stays on the processor
That looks like a waste of the GPU until you time the cold start. A vLLM instance needs 130 seconds before it produces its first token. For photos that come along every now and then, the waiting dominates everything about the experience. Running it through llama.cpp on the processor costs 4.1 GB of RAM, no VRAM at all, and leaves the main model at full strength. Per image it takes longer. In practice it is faster.
The 32B turned out to be unnecessary
I put both models on disk with a command to switch between them, since they do not fit in VRAM together. Across forty agentic tests they both scored forty out of forty. The larger model costs you half your context, 72k against 131k, and much of your concurrency, 1.17x against 1.93x.
So for this workload the 32B adds nothing. That is the kind of answer I started the project for: I could have guessed it, and now I know it, and it saved me buying something heavier.
Would I recommend it?
Yes, with one caveat. If the part of the hobby you enjoy most is trying out brand-new models, go and buy NVIDIA, because everything runs there on day one. If what you want is a stable machine with a good model and plenty of context on it, and you do not mind spending a couple of evenings on drivers and containers, Intel gives you more memory for your money than anyone else.
Once those three hurdles were behind me I have barely had to look at it. The card just does its job, which is the highest compliment you can pay a piece of hardware.
Leave a Reply