Fast GPU, Slow Cable — Running Strata at Home

After getting GLM-5.3-Flash running on my homelab server, the next objective was to make a large local model useful without budgeting an entire coffee break for every response. The hardware was still the same: a Ryzen 7 5800XT, 128 GB of RAM, and an RTX 5070 Ti with 16 GB of VRAM. This time I was installing Strata to run Qwen3.8-Flash-Next. Somewhere between downloading the model and questioning my entire configuration, the experiment became an investigation into a cable.

The short version: my GPU was connected through a PCIe riser inside a DeepCool Quadstellar case. With that connection in place, CUDA transfers between system RAM and the GPU were crawling. Mounting the card directly in the motherboard restored transfer rates to roughly 26–28 GB/s. The subsequent Strata benchmark delivered a median of 58.7 tokens per second in short chat. The server did not need a new GPU. It needed a shorter conversation with the one it already had.

Installing Strata: a different way to use the same hardware

Strata is the inference runtime in this setup. It loads the model and exposes an API that applications such as LibreChat can call. The model I used was Qwen3.8-Flash-Next in IQ3_S quantization, with a 32,768-token context limit. This was a different model and runtime from my earlier GLM experiment, so the two posts are not a controlled speed comparison.

The attraction was how Strata divides work between system RAM, the CPU, and limited GPU memory. Frequently used Mixture-of-Experts weights can stay in a VRAM cache. Other expert work runs on the CPU or uses the PCIe path to the GPU, and prompt processing can stream expert weights across that link. That makes a 16 GB card more useful, but it also makes the connection between the GPU and the rest of the machine worth measuring.

For a fresh Linux installation, the upstream entry point is straightforward:

git clone https://github.com/Niko1221/Strata.git
cd Strata
./setup.sh

The setup flow prepares the chosen model and configuration; running ./setup.sh again starts it. These are the upstream getting-started commands, rather than a reconstruction of every command in my container deployment. My prepared IQ3_S setup produced a run-iq3_s.sh launcher and exposed the OpenAI-compatible API at http://127.0.0.1:8080/v1.

I ran Strata in the llm-strata container and tested it directly before worrying about the chat interface. The run configuration used automatic expert caching, automatic prefill selection, speculative decoding, an INT8 KV cache, and one inference slot. Once it was serving requests, the question became very simple: why was it this slow?

The GPU was busy. That was not the whole story.

The early diagnostic check was around 1.8 tokens per second. Looking at GPU utilization made the situation more confusing: samples showed 100% utilization while the card was drawing roughly 46–48 watts. That is an observation, not a diagnosis. A utilization counter alone does not tell me whether useful computation, memory access, synchronization, or some other part of the workload is setting the pace.

There were legitimate software leads. The CPU was using AVX2 expert kernels, and one startup log warned that only 401 MiB of VRAM remained free. Thread placement, cache sizing, and memory headroom all deserved attention. But another line in the same log was considerably harder to explain away:

PCIe probe: 0.2 GB/s host->device
(best of 0.1 0.1 0.2 0.1)

That was Strata measuring how quickly it could move data to the GPU. Before spending another hour adjusting the inference engine, I wanted to know whether the machine could perform a basic CUDA memory transfer at a reasonable speed.

Testing the transfer path

We moved outside the model and tested CUDA copies in both directions: host to device, meaning system RAM to GPU memory, and device to host, meaning the return trip. Those are bandwidth measurements in GB/s. They are different from model generation in tokens per second and from copies that stay entirely within GPU memory.

The host-side transfer tests were repeatedly around 0.09–0.15 GB/s with the riser connection in place. That made the problem much more concrete. The poor performance was reproducible without asking a language model to generate anything.

We also checked PCIe generation and lane width. A useful inspection command is:

nvidia-smi --query-gpu=name,pci.bus_id,pcie.link.gen.max,pcie.link.gen.current,pcie.link.width.max,pcie.link.width.current --format=csv

There is a trap here: the current PCIe generation can drop while the GPU is idle. Seeing Gen1 in an idle snapshot does not automatically mean a broken link. The useful evidence is the negotiated link under load together with measured transfers. In this case, the transfer benchmark was the part that could not be dismissed as an idle power-saving state.

The Quadstellar side quest

My server lives in a DeepCool Quadstellar, a case with separate compartments and enough visual ambition to look like it should require landing clearance. I had the GPU in its separate graphics-card compartment, connected back to the motherboard with a PCIe riser cable. That layout was why the riser was there in the first place.

The useful discovery was that I could mount this GPU inside the same case, directly in the motherboard's PCIe slot, without using the riser. I moved it into that position and repeated the transfer tests. The improvement was immediate.

The direct-mount rates were back in the tens-of-GB/s range expected from a healthy host–GPU connection on this system. This was not a small optimization hidden inside benchmark noise. The connection had gone from roughly a tenth of a gigabyte per second to the mid-twenties.

That gives me high confidence that the riser connection was a major bottleneck in this setup. It does not identify the exact electrical failure: cable compatibility, signal integrity, connector contact, and link training remain possible explanations. It also does not mean every riser is bad. What the experiment demonstrated was that bypassing mine restored normal transfer performance.

Back to actually running the model

After the move, Strata's own probe reported 27.4 GB/s, and a short diagnostic run reached 72.1 tokens per second. I then ran the fuller benchmark rather than treating the fastest short response as the headline. The upstream suite is launched with ./bench.sh; my run used the server already listening on localhost.

The difference was obvious in use. Short chat had become responsive, and larger prompts no longer looked like an exercise in patience. The GPU, CPU, and system memory were finally connected in a way that let the inference runtime use them sensibly.

I am keeping the short diagnostic checks separate from the fuller suite. The roughly 1.8 and 72.1 tok/s spot checks describe the troubleshooting session; they are not a matched before-and-after benchmark across every prompt and setting. Likewise, I am not turning the old GLM result into a Strata speedup multiplier. Different models, runtimes, and configurations would make that number impressive and mostly useless.

The lesson I am taking from this is to benchmark the layers separately: basic host–GPU transfers, the inference engine, and then the application using it. Otherwise it is very easy to spend an evening tuning model settings around a hardware problem.

I started this project trying to squeeze more intelligence out of a 16 GB GPU. The most effective optimization turned out to involve opening the case and removing a cable. Apparently the first step toward a useful local AI assistant was giving it less infrastructure.

Sources and build record

• Strata repository and setup instructions — upstream Linux entry point and local API.
• How Strata works and technical details — expert caching, CPU work, and PCIe transfers.
• NVIDIA System Management Interface documentation — PCIe link generation, width, and idle-state behaviour.
• DeepCool's original Quadstellar announcement — case layout. The successful direct GPU mount described here is from my own build.
• Big Model, Small GPU — the earlier GLM experiment, included for project context rather than a matched baseline.