Scenarios and their inputs¶

A kernel is built from a scenario file and the catalog and registry files the scenario names. This page shows a scenario, says where each of its names leads, and describes how this repository fetches the catalog and registry its tests read.

A scenario file¶

A scenario file holds two YAML documents. The first, the scenario, fixes the problem: the model, the hardware, the coefficient sets and the engine version. The second, the deployment, is the configuration chosen for it: one or more pools of GPUs, each with its parallelism and its engine settings. They are kept apart because a search over configurations varies the deployment while the scenario stays fixed.

kind: Scenario
name: minimax-m25-h200-tp8
engine_version: "0.29.0"

model: minimax-m2.5
coefficients: [cost-model-primitives, cost-model-collectives, cost-model-host-overheads, cost-model-attention, cost-model-recurrent, cost-model-memory]

cluster:
  hardware: h200
  nodes: 1
  gpus_per_node: 8
---
kind: Deployment
name: minimax-m25-h200-tp8

pools:
  - role: colocated
    nodes: 1
    parallel:
      tp: 8
      pp: 1
      dp: 1
      enable_expert_parallel: false
    engine:
      quantization: fp8
      cache_dtype: fp8
      block_size: 16
      max_num_batched_tokens: 32768
      max_num_seqs: 256
      max_model_len: 32768
      cudagraph_mode: PIECEWISE
      gpu_memory_utilization: 0.9

blis-schemas defines both documents and documents every field. The kernel reads the two from one file with its own LoadBundle, which rejects an unknown key as strictly as blis-schemas' loaders do, and validates them before building anything.

Where the names lead¶

In the scenario	Read from
`model: minimax-m2.5`	`<catalog>/models/minimax-m2.5/graph.yaml`: the model as a graph of operations
`cluster.hardware: h200`	`<catalog>/hardware/h200.yaml`: the chip's peak rates, memory and SM count
`cluster.fabric`, if present	`<catalog>/networks/<fabric>.yaml`: the network between nodes
(always)	`<catalog>/devices/storage.yaml`: the storage tiers offload can use
`coefficients: [...]`	`<registry>/coefficients/<set>.yaml`, one file per set
`engine_version`	The validation rules blis-schemas publishes for that version of vLLM

A deployment that spans nodes should name its fabric, because collectives that cross nodes are priced from it. Of the six coefficient sets above, three are required: without cost-model-primitives, cost-model-collectives or cost-model-memory, New refuses. A missing attention, recurrent or host-overheads set leaves the terms it covers unpriced, and the kernel records each missing coefficient the deployment needed as an assumption.

Repos gives the three roots: where the scenario files are, the catalog checkout, and the registry checkout. Price a batch from Go shows it in use.

In this repository¶

The scenario files are committed under testdata/: a few written by hand at the top level, and two directories generated by scripts in scripts/, one deployment per file. The catalog and the registry are not committed. scripts/fetch-testdata.sh fetches each at the commit testdata/upstream.lock pins, checks it, and records what it copied; a test fails if the fetched copy does not match the lock.

scripts/fetch-testdata.sh
go test ./...

To test against a different checkout of either, set BLIS_CATALOG or BLIS_REGISTRY. The measurement corpora the scoring commands compare against are other publishers' data and are not distributed here; testdata/README.md says how they are produced and where the scoring suite looks for them.