Replicas vs tensor-parallel
2 small > 1 big
On the same GPUs, two smaller vLLM replicas served more requests than one large tensor-parallel split. We size client hardware with this in mind.
Best case measured: +136% throughput. Typical gains are lower and depend on the model and load; we quote the low end to clients.