We had an NVIDIA DGX Spark to evaluate for our self-hosted AI agent, and an RTX 5090 workstation already running it. On paper the Spark looks like the bigger machine: four times the memory. In practice, for the model we actually run, the RTX 5090 answered the same question in roughly a third of the time.
#1. A Fair Test
The DGX Spark ships without models preloaded, so both machines were set up the same way:
- Model: Gemma 4 26B on both.
- Runtime: Ollama 0.31.1 on both.
- Workload: the same agent request (prompt, conversation history and tool definitions), measured end to end.
#2. Results
| Metric | RTX 5090 | DGX Spark |
|---|---|---|
| Response time (same request) | ~15 s | ~41 s |
| Relative response time | 1× | 2.73× slower |
| Extra waiting time | — | ~26 s |
For this workload the DGX Spark brought no speed-up over the workstation.
#3. Why: Bandwidth Beats Capacity When the Model Fits
| Hardware | RTX 5090 | DGX Spark (GB10) |
|---|---|---|
| Memory | 32 GB GDDR7 | 128 GB unified LPDDR5x |
| Memory bus | 512-bit | 256-bit |
| Data rate per pin | 28 Gbps | ~8.533 Gbps |
| Memory bandwidth | 1,792 GB/s | ~273 GB/s |
| Capacity | 1× | 4× |
| Bandwidth | ~6.6× | 1× |
An answer comes in two phases. Prefill reads the prompt, the conversation history and every tool definition. Generation then produces the answer one token at a time, and every token means streaming the model's weights through memory again. That second phase is limited by memory bandwidth far more than by compute.
Gemma 4 26B fits comfortably in the RTX 5090's 32 GB, so the Spark's extra capacity buys nothing here, while the 5090 moves data about 6.6 times faster. That bandwidth gap is the most likely reason for the result.
We timed the whole response, not prefill and generation separately, and this was one request per machine. The direction of the result is clear; the exact ratio will move with prompt length, output length and quantization.
#4. When the DGX Spark Is the Right Box
128 GB of unified memory is a real advantage when the workload does not fit in 32 GB:
- Larger models that would not load on a single consumer GPU at all.
- Much longer context windows, where the KV cache grows with every token of history.
- Several models resident at once (for example an embedding model, a reranker and a chat model).
Choose for the model you will actually run: if it fits in GPU memory, bandwidth decides speed; if it does not, capacity decides whether it runs at all.
#5. What We Test Next
- NVIDIA-native models from NVIDIA's model registry, which may be better optimised for the Spark's hardware.
- Separate timings for prefill and generation, so each machine's strength is visible on its own.
- A change to how the model runs on the server that could make answers roughly twice as fast at the same quality, tested on our own hardware before it goes anywhere else.
#6. Takeaways
- For local LLM inference, check memory bandwidth before memory size.
- More memory only helps once the model, or its context, no longer fits.
- Benchmark with your own model, runtime version and real requests; spec sheets don't tell you the answer for your workload.