Table of Contents

NVIDIA’s GB300 is a beast of a machine that pairs a Grace CPU and Blackwell Ultra GPU with huge shared memory, targeting local office AI at scale. Previously available in servers, it now comes to desktop workstations in the NVIDIA® DGX Station™ design, which we offer through the MSI XpertStation WS300.

These are expensive machines, really built for deploying AI workflows at scale. With access to one, courtesy of MSI, we measured chat and coding use cases across a growing team to see what would strain a single box and found that regular chat will feel instant up to 192 people and fine up to 384. Coding agents stay fine up to 64 running at once. Check out our simulations below to experience the GB300 for yourself.

How it feels as the team grows

Each block is a team size we measured, from a handful of people to over a thousand. Its color is how the wait felt to the people on it.

Instant: first word in under 1 sFine: a short pause, under 3 sStrained: you notice the wait

Chat: Instant from 32 to 192 people, Fine at 256 and 384, Strained from 512 to 1,024. Coding agents: Instant from 1 to 16, Fine at 32 and 64, Strained at 96 and 128. No team size we measured was Broken.

Measuring Experience: Evolving Our AI Benchmarks

We have been looking for better ways to measure AI performance so users know what to expect from different hardware, and this is a new direction we are trying out. To gauge how users are experiencing AI, we created "feeling" bands and grouped teams by size. The bands are set by the wait for the first word, by how many words a second each person then receives, and, for Fine, by the worst-case wait that 1 in 100 requests sees (the p99). Words are tokens × 0.75. The source data for this is massive, but we've included links to download it at the end of the article if you'd like to simulate it yourself.

Band	Rule	What it feels like
■ Instant	Median wait ≤ 1 s, at least 10 words/s	Feels like a local app.
■ Fine	Median wait ≤ 3 s, at least 5 words/s, 1-in-100 wait under 5 s	A short pause, then it outpaces your reading.
■ Strained	Median wait ≤ 10 s, at least 3 words/s	You notice the wait. Still usable.
■ Broken	Slower, over 1% errors, or stalls	People stop using it. No measured size landed here.

See for Yourself: Pick Your Team

To experience this simulation, choose a workload and a team size, then press Send. Your answer arrives after the measured wait for that team size, at the measured speed. The team sizes are the ones we measured, each underlined in its band color, and nothing between them is interpolated. Check the box in the bottom right to see what it is like to be the unlucky 1 in 100.

This demo replays the measured wait and streaming speed for each team size, and needs JavaScript. The measured numbers are in the tables further down.

Be the unlucky 1 in 100

Your Users Feel the Wait

Each chat user sent a 4,096-token prompt and got a 500-token answer (about 375 words). Up to 192 users, chat feels practically instantaneous on this system. With 32 people each person gets 108 words a second. At 128 people that has halved to 52, but your users barely notice because both speeds are far faster than anyone reads. The first things people notice are the blank screen before the first word and the unlucky wait that 1 request in 100 sees.

Wait for the first word, by number of people chatting

InstantFineStrained 5 s p99 limit for Fine

Median wait for the first word: 0.47 s at 32 people, 0.95 s at 128, 1.62 s at 256, 1.80 s at 512 and 4.93 s at 1,024. The 1-in-100 wait passes 5 s at 512 people. All values are in the chat table below.

Dot: Median wait. Whisker: The 1-in-100 (p99) wait. Shading marks the median limits for each band (1, 3 and 10 seconds).

Expand to show the chat numbers

People	Band	Median wait	1-in-100 wait	Words/s each	Tokens/s total
32	Instant	0.47 s	0.56 s	107.8	4,058
64	Instant	0.70 s	0.90 s	79.9	6,000
96	Instant	0.80 s	1.31 s	61.2	6,957
128	Instant	0.95 s	1.64 s	51.6	7,857
192	Instant	0.85 s	2.46 s	36.6	8,503
256	Fine	1.62 s	3.19 s	32.9	9,937
384	Fine	1.71 s	4.87 s	23.4	10,695
512	Strained	1.80 s	5.70 s	18.1	11,243
768	Strained	4.18 s	10.38 s	13.3	11,668
1,024	Strained	4.93 s	13.06 s	9.7	10,692

We found that memory never ended up being the bottleneck as we scaled. Raw compute ends up being what eventually slows us down. Total output peaks at about 11,700 tokens a second at 768 people, then drops at 1,024. Past that peak, more people only means longer waits.

Chat Is So 2025, What About Agents?

Every day, we are learning how to better use agents to support multi-step work and automate technical workflows here at Puget Systems. To reflect that, we wanted to benchmark higher token generation and long outputs like we might get when we use a coding agent. Agent load started to strain our system much more quickly, at 96 concurrent agents.

Each agent step sent a tool-simulated 28,000-token repository context, the same for every agent, plus 4,000 new tokens of its own, and got back a 1,500-token reply. Because the context is shared, the engine reads it once and serves it from the prefix cache. About 86% of every prompt was a cache hit. That is why many agents fit: at 128 agents the KV cache was only 11% full. It's another example of how important caching is to managing load on inference servers.

InstantFineStrained

Seconds per step, each agent

Seconds per step for each agent: 5.6 s alone, 14.2 s at 32 agents, 21.7 s at 64 and 35.6 s at 128.

Steps per hour, whole team

Steps per hour across the team: 644 with one agent, 8,139 at 32, 10,599 at 64 and 12,950 at 128.

One step is the full 1,500-token reply, including the wait for the first token. Hover over a bar for the per-agent rate. The two measures sit in separate panels because their scales differ.

Show the agent numbers

Agents	Band	Median wait	1-in-100 wait	Words/s each	Seconds per step	Steps/h each	Steps/h team
1	Instant	0.24 s	0.56 s	207.9	5.6	644	644
4	Instant	0.24 s	0.92 s	166.4	7	515	2,060
8	Instant	0.22 s	0.44 s	142.5	8.1	447	3,576
16	Instant	0.39 s	0.79 s	116.3	10	361	5,769
32	Fine	1.25 s	2.37 s	86.5	14.2	254	8,139
64	Fine	2.59 s	3.75 s	57.3	21.7	166	10,599
96	Strained	2.18 s	5.91 s	39.1	30.7	117	11,267
128	Strained	2.22 s	7.93 s	33.5	35.6	101	12,950

Ackchyually You Don't Need a GB300 to Run Qwen3.6

Astute readers will notice that everything above ran on Qwen3.6-35B-A3B in NVFP4. Its weights take 22 GB, which fits on a single 32 GB workstation card. If you only need to serve yourself or a few people, a GB300 is far more machine than this model asks for.

We picked it because it is a coding workhorse and something we may want to scale out to our internal dev team. It is a mixture-of-experts model with about 3B parameters active per token, so each token is cheap to produce. In September we ran a correctness check where it passed all three code-tracing items and 17 of 19 overall, and it was the fastest model we put on the system: 258 tokens a second for one person and 4,386 across the server at its busiest. The GB300's power adds room for the team. On vLLM 0.30 the system held about 10 million tokens of live conversation for this model, which is what let 512 people chat at once.

Bigger Isn't Always Better

We also put the big models on the same system, from 1 to 64 people at a time, and ran each through the same 19-question correctness check. Seven of them fit in the GPU's 252 GB. Kimi-K2, at a trillion parameters, is too big to fit on in a single GB300's VRAM, instead overflowing into the shared system memory.

Testing Seven Models and Kimi-K2 on One GB300

Model	Weights	One person, words/s	Busiest, tokens/s	32k chats that fit	Check
Qwen3.6-35B-A3B used above	22 GB	193	4,386	191 (0.28)	17/19
Qwen3.8-27B	22 GB	113	3,256	107 (0.28)	16/19
Qwen3-235B-A22B	125 GB	113	2,303	not kept	18/19
Qwen3.8-Flash-Next	126 GB	145	2,940	136 (0.30)	17/19
DeepSeek V4 Flash FP8	155 GB	115	1,621	28 (0.28)	18/19
GLM-5.3-Flash	184 GB	106	1,667	86 (0.30)	17/19
GLM-4.7 355B · replaced by GLM-5.3	214 GB	not run	not run	5 (0.28)	16/19
Kimi-K2 1.03T · llama.cpp	547 GB	15	n/a	n/a	not run

One person: How fast the answer appears when you're the only one using the system.
Busiest: The most text the system produced for everyone combined, in our internal September tests.
32k chats that fit: How many long conversations (about 25,000 words each) the system can keep in memory at once. The number in brackets is the vLLM version we ran. The newer 0.30 fits more than 0.28, so compare rows with the same version.
Check: How many of our 19 test questions the model got right (with three examples shown below).
Note on Kimi-K2: This model ran on different software (llama.cpp), so it only has a speed figure.

Same Question, Every Model

Pick a question and press Play. Each row replays one model's real answer in the time it took with one person on the system. Every model got all three right. What differs is how long they took and how much they wrote first.

Time to answer the code question, one person on the card: DeepSeek V4 Flash 0.40 s (65 tokens), Qwen3.8-27B 0.55 s (81), Qwen3.8-Flash-Next 0.56 s (109), GLM-5.3-Flash 0.74 s (104), Qwen3-235B-A22B 1.43 s (221), Qwen3.6-35B-A3B 1.95 s (580). All six answered correctly.

These are simulations of the real replies from our September tests, with one user asking and nobody else on the system.
Time: From pressing send to the last word, including the time the model spends reading the question.
Faded text: The model thinking out loud before it gives its answer.
"…": We only saved the start of each reply, so long ones eventually skip to the answer.

As you can see, the replies and speeds vary, and of course your prompt and harness matter too. However, don't just assume you need the largest compute and largest model possible, and drop six figures on a GB300. This machine is all about deploying your AI workflow to your office or a small audience of test clients before you move to servers, or maybe using the largest model if your use case demands it. In all cases, we suggest you test your workflow with different models and choose the best one for your use case.

How We Tested

Here is the hardware and software setup we used for these tests:

Hardware & Software

System	MSI XpertStation WS300 (DGX Station)
GPU	NVIDIA GB300 (Blackwell architecture)
GPU Memory	252 GB HBM3e
CPU	NVIDIA Grace (72 cores)
Total System Memory	748 GB (252GB HBM3e + 496 GB LPDDR5x)
OS	Ubuntu 24.04
Driver	595.84
Inference Software	vLLM 0.30.0
Model	Qwen3.6-35B-A3B, NVFP4 (nvidia/Qwen3.6-35B-A3B-NVFP4)
Load Generator	NVIDIA GenAI-Perf 0.0.16

Test Methodology

We used software to play the part of each person on the team. Every simulated person sent a request, waited for the full answer, then sent the next one. We held each team size for two minutes and recorded how long each person waited for the first word and how fast the rest of the answer arrived.

Chat: Each question was 4,096 tokens (about 3,000 words) and each answer about 375 words (500 tokens). This represents everything the model rereads on each turn in a typical conversational scenario: the system prompt, the conversation so far, and often a pasted document, like the attached contract in the example above.
Coding agents: Each step sent the same 28,000-token project plus 4,000 tokens of its own work, and got back a 1,500-token reply.

How to Read These Results

Chat is a best case. Our simulated people asked from a pool of 20 questions, so the system could reuse much of its earlier work. Real people ask different things, so a real chat team will wait a little longer than our numbers show.
Agents are a worst case. Real coding agents stop to run tools and tests between steps, which gives the system a break. Ours never stopped. A real team of agents will likely do better than our numbers show.
Answers were a fixed length. Real answers vary. Longer answers take longer, and shorter ones finish sooner.
One system, one model, one software version. A different model, a newer vLLM, or a different driver will give different numbers. The September model comparison above used older software and a smaller test.
Not tested yet: Long documents and running the bigger models through the same team sizes. Both are on our list for future tests.

Technical notes: Words are counted as 0.75 per token, waits are time to first token, and speed is each user's own streaming rate after the first token. Chat team sizes of 512 and below allowed 1,024 requests in flight at once; 768 and 1,024 allowed 2,048. Agent replies were forced to full length with ignore_eos. Multi-turn agent sessions report whole-step time only, because GenAI-Perf 0.0.16 can't stream multi-turn runs. At 4, 8, and 16 agents prompts were reused, which raised cache reuse to 95-99% (about 87% elsewhere) without changing the first-word wait. Speculative decoding (MTP) was off. Zero errors and zero preemptions (requests paused for lack of memory) at every team size.

Download the Data

Every figure in this article comes from the raw benchmark results below, so you can check our numbers or run your own simulation.

Chat and coding agent results (team sizes, waits, speeds): Download the benchmark data (ZIP)
September model comparison and correctness checks: Download the benchmark data (ZIP)

Conclusion

So how many people can one GB300 serve? Up to 384 people chatting, or a team of 64 coding agents, all from one machine beside a desk.

If you're looking to deploy local AI for chat or coding agents, the GB300 will let you support a large team from a single machine. In our tests, chat felt instant for up to 192 people at once and stayed fine up to 384. Coding agents stayed fine with up to 64 working at the same time. None of the team sizes we measured broke, even with a thousand simultaneous chats.

Can you link more than one, like the DGX Spark™? Up to a point. The DGX Station / MSI XpertStation has two built-in NVIDIA ConnectX® 400 Gb/s ports, and NVIDIA documents connecting two systems directly; Exxact has a good write-up of the two-system setup. Going past two would need a network switch, and we haven't seen that setup documented yet. The link is much slower than the GPU's own memory, so splitting one model across two boxes is for fitting a model that is too big for one system, not for extra speed. To serve more people, it's simpler to run two systems side by side, each with its own copy of the model, and spread users between them. We haven't tested either setup yet.

The model matters as much as the machine. Qwen3.6-35B-A3B fits on a single workstation card, but the GB300's memory and compute are what let it serve a whole office at once. We always recommend that you test your own workload with a few models before you size the hardware, if possible. There are also calculators available to help estimate hardware needs if you aren't able to run the tests yourself.