InferenceX single-turn (metricscore)¶
BLIS binaries these results describe
(devel)+9e3c509c8788fe0743605a8f06e0bf5c1c39ac1f, binary sha25629fb09054a10ee51dc104f491cb09337e0ba527055b85653d281f23230451dff
cmd/metricscore scores BLIS against InferenceX's single-turn sweeps (random fixed ISL/OSL, closed loop at concurrency CONC, 10*CONC requests) from the InferenceX database dump of 2026-09-28. It replays the exact seed-0 length vectors the client drew; see The published accuracy figures for the method.
Signed error 100 * (BLIS / measured - 1): positive means BLIS predicts higher latency (pessimistic).
Provenance verdict¶
| verdict | sweeps | model families |
|---|---|---|
| EXACT | 0 | -- |
| PARTIAL | 68 | deepseek-v4, gpt-oss, kimi, minimax |
| NOT_REPRODUCIBLE | 45 | llama |
No sweep is EXACT. Every PARTIAL sweep left at least one engine setting unpassed, so the harness used the value vLLM resolves for it (tier MEASURED-OR-RESOLVED). Unpassed settings, by number of PARTIAL sweeps:
| setting | sweeps |
|---|---|
block_size |
48 |
enable_prefix_caching |
1 |
max_num_batched_tokens |
11 |
max_num_seqs |
31 |
NOT_REPRODUCIBLE sweeps have no engine-args log, so the settings they passed are unknown, and their sent prompt lengths are not verified equal to the replayed targets. They enter only the ALL (approximate) tier below, and no conclusion. The workload is single-turn random ISL/OSL, not agentic.
What would make a sweep EXACT (tier STRICT): an engine-args log at every concurrency, every admission setting passed by the run rather than left to vLLM's default, and the sent prompt lengths verified.
Errors by model family (PARTIAL sweeps)¶
TPOT is the output metric here (per-token decode latency); TTFT includes queueing.
TPOT¶
| family | sweeps | points | signed mean | MAPE % | shape error % | signed at lowest c | signed at highest c |
|---|---|---|---|---|---|---|---|
| deepseek-v4 | 5 | 28 | +17.1% | 17.1 | 4.7 | +16.4% | +16.2% |
| gpt-oss | 28 | 143 | -1.0% | 10.7 | 7.9 | -5.0% | -3.7% |
| kimi | 7 | 44 | +22.7% | 22.7 | 5.6 | +19.8% | +21.1% |
| minimax | 28 | 145 | -6.1% | 18.9 | 17.4 | -9.2% | -13.8% |
| all PARTIAL | 68 | 360 | +1.2% | 16.0 | 11.2 | -2.6% | -3.8% |
TTFT¶
| family | points | signed mean | MAPE % | shape error % | signed at lowest c | signed at highest c |
|---|---|---|---|---|---|---|
| deepseek-v4 | 28 | +7.8% | 16.0 | 16.1 | +15.9% | +0.3% |
| gpt-oss | 139 | +2.5% | 24.2 | 22.7 | +24.9% | -19.2% |
| kimi | 44 | +14.0% | 18.5 | 17.4 | +33.0% | +3.7% |
| minimax | 145 | -23.7% | 31.9 | 23.3 | -13.2% | -38.2% |
| all PARTIAL | 356 | -6.3% | 26.0 | 21.7 | +9.4% | -23.2% |
BLIS against the published estimators, by tier¶
metricscore's own tier report: mean absolute error (signed mean in brackets) for BLIS and for the estimators whose predictions the InferenceX snapshot publishes, on the same points. ALL (approximate) includes sweeps that are not the measured runs as they ran.
| tier | estimator | points (TPOT mape) | TPOT shape | TPOT mape | TTFT shape | TTFT mape |
|---|---|---|---|---|---|---|
| STRICT | blis-latency-kernel | 0 | -- | -- | -- | -- |
| STRICT | AISimulate | 0 | -- | -- | -- | -- |
| STRICT | AIC (same project) | 0 | -- | -- | -- | -- |
| MEASURED-OR-RESOLVED | blis-latency-kernel | 360 | 11.16 (+4.81) | 15.97 (+1.24) | 21.71 (-17.43) | 25.98 (-6.31) |
| MEASURED-OR-RESOLVED | AISimulate | 360 | 11.91 (+7.35) | 19.72 (+11.32) | 31.54 (+15.18) | 45.07 (-19.25) |
| MEASURED-OR-RESOLVED | AIC (same project) | 360 | 12.00 (+6.85) | 19.88 (+11.32) | 31.28 (-26.28) | 39.03 (+4.56) |
| ALL (approximate) | blis-latency-kernel | 573 | 9.93 (+0.35) | 16.71 (+6.70) | 17.11 (-13.70) | 24.05 (-4.07) |
| ALL (approximate) | AISimulate | 573 | 10.43 (+4.68) | 15.39 (+7.70) | 23.05 (+9.94) | 37.86 (-21.65) |
| ALL (approximate) | AIC (same project) | 573 | 10.56 (+4.71) | 15.71 (+8.55) | 30.12 (-26.64) | 36.66 (+8.63) |
KV pool: the snapshot rows carry no vLLM start-up KV figure, so no KV-pool ratio is reported for this anchor.