GPU instances went live this month in Frankfurt, Dallas and Singapore. Two configurations to start with: a single RTX 4000 Ada with twenty gigabytes of VRAM, and a single L40S with forty-eight. Both sit on an EPYC 9354P host, attached over PCIe 4.0 at full sixteen lanes, passed straight through to one instance.
Passed through means the card is bound to your instance by the hypervisor at boot and stays there until you cancel. No scheduler, no queue, no other tenant, no partition of VRAM that somebody else is also using.
The decision we actually made
Time-slicing a card between tenants is a supported, well-documented, entirely legitimate way to run this business. We spent about three weeks costing it, because the arithmetic is attractive: a single forty-eight gigabyte card can be presented as four twelve-gigabyte instances, which is four rents from one purchase and four customers from one power budget.
Then we ran the tail latencies.
A shared card is fine when the other tenants are idle and unpleasant the moment they are not. Inference latency on a card with three other active tenants was roughly what you would expect at the median and four to nine times worse at the ninety-ninth percentile, depending on what everyone else happened to be doing. Training jobs fared worse, because a training job is not bursty and therefore never gets the gaps.
There is also the licence problem. The virtualisation software that makes clean partitioning possible carries a per-card annual licence, and that cost does not evaporate; it lands on the invoice of whoever is renting the slice. We would have been charging you for the privilege of sharing.
How the host is built
| Element | What it is |
|---|---|
| Host CPU | EPYC 9354P, thirty-two cores, ECC DDR5-4800 |
| Attachment | PCIe 4.0, sixteen lanes, one IOMMU group per card |
| vCPU pinning | Pinned to the socket and NUMA node the card hangs off |
| Memory | Allocated from the same NUMA node, never interleaved |
| Cards per node | Four maximum, two at most sites, limited by rack power |
| Port | 40 Gbit/s, because a model that will not fit in VRAM has to come from somewhere |
The NUMA pinning matters more than the specification sheet suggests. A vCPU on the wrong socket feeding a card across the interconnect gives up real throughput on anything that streams data, and it is the single most common misconfiguration we see on GPU hosts elsewhere.
Power, and why there are only three sites
A card at three hundred and fifty watts changes the arithmetic of a rack. Sites that comfortably hold twelve general-purpose nodes hold four GPU nodes and then run out of cooling. Frankfurt, Dallas and Singapore were chosen because all three had power headroom and a floor plan that could take the density without us renegotiating anything.
More sites follow as power allows, and the honest constraint is always electricity rather than demand. Where a site cannot take the draw, we would rather not sell the product than sell it thermally throttled.
What you cannot do
- No card pairing across nodes. A two-card instance exists and both cards sit on one NUMA node. Anything larger is a conversation about bare metal.
- No partitioning inside your instance. You have the whole card, and how you divide it is between you and your framework.
- No hot swap. Changing GPU means a rebuild, because the card is bound at boot.
- No consumer gaming cards. Asked about weekly. They lack the cooling profile for a rack and, in most cases, the licence to be rented out at all.
Pricing, plainly
The cards cost what they cost and the margin on them is thinner than on the rest of the catalogue. There is no hourly billing, because hourly billing on dedicated hardware means holding cards idle and charging everyone else for the privilege. Monthly, with the standard seven-day refund, and the same crypto-only payment as everything else.
If your workload genuinely is bursty and hourly, we are the wrong supplier and a large hyperscaler is the right one. That will be true for as long as we refuse to time-slice, which is indefinitely.