Write a scenario and a deployment¶

A run is described by two documents: a scenario stating the problem and a deployment stating the choices made against it. This guide writes one of each for a single node, then extends them to a disaggregated deployment across four.

Every example on this page is a file in docs/examples/, and the test suite validates each one.

A scenario¶

scenario.yaml

kind: Scenario
name: llama-3.1-8b-on-h100
model: llama-3.1-8b-instruct
coefficients:
  - cost-model-primitives
  - cost-model-collectives
  - cost-model-host-overheads
  - cost-model-attention
  - cost-model-recurrent
engine_version: "0.29.0"
workload:
  shape: chatbot
cluster:
  hardware: h100
  nodes: 1
  gpus_per_node: 8

model: A directory under models/ in the catalog.
coefficients: Coefficient sets in the registry, named by file. The five above are those named by the kernel-based scenarios in inference-sim's pull request #1851. blis-latency-kernel reads them in the order listed: where two sets both supply a coefficient that applies to the deployment, the later set wins.
engine_version: The engine version the run describes. Version-specific rules run only when it matches a registered rules pack exactly, so write "0.29.0" and not "0.29". Quote it so that YAML reads it as a string.
workload: The traffic, here the catalog's chatbot shape. Omit it for a question about capacity alone.
cluster: The hardware the problem is given. hardware names a file under hardware/ in the catalog; every node carries that chip.

A deployment¶

deployment.yaml

kind: Deployment
name: tp2-dp4
pools:
  - role: colocated
    nodes: 1
    parallel:
      tp: 2
      pp: 1
      dp: 4
    engine:
      block_size: 16
      max_num_seqs: 256
      max_num_batched_tokens: 8192
      gpu_memory_utilization: 0.9

One pool serves both prefill and decode (colocated) on the cluster's single node. Its layout is four data-parallel replicas, each split two ways by tensor parallelism. That uses all eight GPUs: an engine needs pp × tp × pcp × dp GPUs, one per rank, and they must fit in its pool's nodes.

Engine settings are optional. Each one omitted is left to the engine's default, as in a real launch.

Check them¶

There is no command for scenarios yet. Validating them takes a few lines of Go, explained in Validate documents from Go:

s := must(blisschemas.LoadScenario("docs/examples/scenario.yaml"))
d := must(blisschemas.LoadDeployment("docs/examples/deployment.yaml"))
g := must(blisschemas.LoadModelGraph("testdata/models/llama-3.1-8b-instruct/graph.yaml"))
c := must(blisschemas.LoadChip("testdata/hardware/h100.yaml"))

rep := blisschemas.Validate(blisschemas.Bundle{
    Scenario: s, Deployment: d, Model: g, Chip: c,
})
for _, p := range rep.Problems() {
    fmt.Println(p)
}
fmt.Println("ok:", rep.OK(), "rules applied:", rep.RulesApplied)

Output

ok: true rules applied: 0.29.0

Pass the model graph and the chip as well, as above, when you have them. Some rules need the model; whether experts divide evenly across ranks, for instance, depends on how many there are.

Across nodes, disaggregated¶

Prefill and decode can run on separate pools, with KV cache moving from one to the other. This scenario gives DeepSeek-V3 four H200 nodes:

scenario-disaggregated.yaml

kind: Scenario
name: deepseek-v3-on-h200
model: deepseek-v3
coefficients:
  - cost-model-primitives
  - cost-model-collectives
  - cost-model-host-overheads
  - cost-model-attention
  - cost-model-recurrent
engine_version: "0.29.0"
workload:
  shape: summarization
cluster:
  hardware: h200
  fabric: ib-400g
  nodes: 4
  gpus_per_node: 8
  storage:
    - cpu_dram

A cluster of more than one node must name a fabric, because the cost of anything that crosses a node boundary comes from the fabric, not the chip. storage lists the storage classes the cluster provides.

deployment-disaggregated.yaml

kind: Deployment
name: 1p3d-ep8
pools:
  - role: prefill
    nodes: 1
    parallel:
      tp: 1
      pp: 1
      dp: 8
      dp_local: 8
      enable_expert_parallel: true
    engine:
      all2all_backend: deepep_high_throughput
      max_num_batched_tokens: 16384
  - role: decode
    nodes: 3
    parallel:
      tp: 1
      pp: 1
      dp: 8
      dp_local: 8
      enable_expert_parallel: true
    engine:
      all2all_backend: deepep_low_latency
      max_num_seqs: 512
offload:
  tiers:
    - tier: cpu_dram
      bytes: 549755813888
pd_transfer:
  connector: NixlConnector

Four things change.

There are two pools, prefill and decode. Neither may appear without the other, and neither may appear beside a colocated pool.
Their node counts, 1 and 3, sum to the cluster's 4. Pools always account for every node.
pd_transfer is present, as a disaggregated deployment requires, and names the KV connector that moves KV between the pools.
The offload tier uses cpu_dram, one of the storage classes the cluster lists.

Each engine in either pool is dp 8 with tp 1: eight ranks, one node. The decode pool's three nodes therefore run three such engines. Each pool enables expert parallelism. The expert-parallel width is not written down: it is tp × pcp × dp, which is 8, and DeepSeek-V3's 256 experts divide evenly across it. Parallel layouts explains why the width is derived.

Traffic from a trace¶

To use captured traffic instead of a distribution, bind the workload to a trace file:

trace-binding.yaml

kind: Scenario
name: agentic-replay-on-h200
model: deepseek-v3
coefficients:
  - cost-model-primitives
engine_version: "0.29.0"
workload:
  trace:
    data: traces/agentic-run.csv
    rows: 1048576
    header:
      trace_version: 3
      time_unit: microseconds
      mode: real
      server:
        type: vllm
        tensor_parallel: 8
        gpu_memory_utilization: 0.9
      goodput_slo_targets:
        critical: {ttft_ms: 500, itl_ms: 50, e2e_ms: 30000}
cluster:
  hardware: h200
  nodes: 1
  gpus_per_node: 8

The requests stay in the CSV file that data names. A workload takes shape or trace, never both. The keys are in Workload.

Problems you are likely to see¶

Message	What to do
`pool node counts sum to 3 but the cluster declares 4`	Make the pools' `nodes` add up to `cluster.nodes`.
`needs 32 GPUs (pp 1 x tp 8 x pcp 1 x dp 4, one per rank) but the pool's 1 node(s) of 8 GPUs provide 8`	Shrink the layout or give the pool more nodes.
`required for a 4-node cluster: cross-node cost resolves from the fabric, not the chip`	Add `cluster.fabric`.
`a prefill pool with no decode pool has nowhere to send KV`	Add the decode pool, or make the pool `colocated`.
`required for a disaggregated deployment`, at `pd_transfer`	Add a `pd_transfer` block naming a connector.
`"cpu_dram" is not in the cluster storage inventory [nvme_gen4]`	List the class under `cluster.storage`, or use one that is listed.
`a workload is a shape or a trace, not both; ...`	Keep one of `workload.shape` and `workload.trace`.
`required when a deployment is present: ...`, at `scenario`	Validate the deployment together with its scenario.

Every key and every check is listed in Scenario and Deployment.