BLIS Target Design

1How to read this document

Words. MUST, MUST NOT and SHOULD mean what they mean in RFC 2119. An invariant is a statement about the repositories that every merged pull request and every release keeps true.

Registries. Three things in this design are called registries, and the word is never used alone. The coefficient registry is the substrate’s fitted and assumed coefficients, with their provenance (blis-substrate/registry/, the --registry flag); it is what --catalog is paired with. The invariant registry is the project’s rules, each with its statement, checks and status (blis-substrate/contracts/invariants/, §13.1). The metric registry is the definition of every metric a Result may report (contracts/metrics, §8.7). An upstream’s own tables are called registration tables, and engines are said to register a factory, not to be in a registry.

Identifiers. Every invariant has a short identifier, such as X3 or SUB-4. Pull requests cite these identifiers, and CI checks that every cited identifier exists (§13). The prefixes are:

How a rule is enforced. A rule is one of three kinds, and the design says which (§4.2). An automated rule names a deterministic check, and the repository that runs it, which fails the pull request, the nightly run or the release when the rule is broken. A maintainer practice is a rule a person with write access keeps for a repository as a whole, such as its branch protection, who may cut a release, and signing; it is stated once, with a checklist, and no program in another repository audits it. Agent guidance is a rule an agent follows on every change it writes or reviews, and is rendered into each repository’s AGENTS.md. Where a rule depends on judgement, a review question is written for it. A review question is guidance for the review agent, which applies it to every pull request it reviews and answers it in a review comment with a reason and the lines that support it (§4, §13.2). It is never a check: it fails no CI job and waits for no person. The one reviewer with write access reads the agent’s answers as part of reading the change.

This design and the release plans. This document is the standing design: every statement in it holds for every release, and it says what must be true, not what is true at the moment. It contains no dates, milestones or status. Each release has a separate release plan, which records where the repositories stand, schedules the work that brings them in line with this design, and lists the release's open questions. A plan may schedule invariants and may relax one temporarily, naming it and the milestone at which the relaxation ends, but it may not create a rule. A new rule enters this document first, with an identifier. Each plan's status table is generated from the invariant registry (§13.1), so a plan and this design cannot disagree about which invariants exist.

Two short companions. The design is a reference, about ninety printed pages. Two summaries sit beside it for a reader who knows llm-d and vLLM and wants the shape of the simulator first: a three-page summary of the architecture and the seven-page summary of it with the detail a contributor needs before changing code. A page is a standard print page (US Letter, 0.9 inch margins, 10.5 point text). The page limits are part of the design: a summary does not grow when the design does. A design change that belongs in a summary takes the room of something that matters less to its reader, and a change that fits neither is left to the design alone. Each summary states no version, count, date or milestone, links every claim to the section that states it, and defers to the design where they differ. A section a summary cites keeps its number.

Citations. Claims about upstream code cite a file and line range at a pinned release. The releases and commits are the ones stack.json records for each engine and for the llm-d router. When an engine repository moves to a new upstream release, the citations that describe its behaviour move with it. A claim drawn from reading code that a recorded run has not confirmed is marked inferred.

2The shape of the system

2.1 Seven repositories

The target system has seven repositories. Three track an upstream project, one is the shared foundation, one is the simulator, and two are applications that use the simulator.

2.2 Where a fact belongs

Every fact, rule and value has exactly one owner. To decide which, ask what the fact depends on:

• If its value changes when an engine releases a new version, it belongs in that engine's repository, on the branch that models that release (§14.4).
• If it changes when llm-d releases a new version, it belongs in the router fork.
• If it changes with the simulator's own design, it belongs in inference-sim.
• Otherwise it belongs in blis-substrate: it is true of the hardware, the model, the physics or the data, whatever engine runs.

To check a change to the substrate, ask whether a second engine would need the same edit. If not, the change belongs in an engine repository.

The same rule applies to work, not only to facts. Each kind of work has one owning repository, and no other repository holds a second copy of it. Another repository that needs the work calls the owner's command or imports the owner's package.

So inference-sim runs simulations and holds no scorer, and a fitting script that wants an end-to-end figure runs the dashboard's scoring harness rather than writing its own. Two repositories that each compute an accuracy figure would sooner or later disagree about what the figure means.

2.3 How one simulated step flows

Figure 2 follows one engine step through the system. It shows the boundary this design changes most: the engine resolves everything about its own behaviour and hands the cost model a fully resolved description of what runs.

3blis-substrate

3.1 Why one repository

The substrate has four parts: the contracts and data types, the catalog of facts, the coefficient registry, and the cost model. They change together:

• A model whose architecture is already supported needs only a catalog entry. A model with something new in its architecture usually needs a new data field, a catalog entry, new coefficients and a change to the cost model, all at once.
• The data are validated by the contracts' validators, and the cost model is tested against the data. If the parts lived in separate repositories, each change would be released in order, each release would force the next repository to update its pin, and consistency would rest on vendored copies and lock files. The release plan's background describes what that cost when the parts were separate.

In one repository, such a change is one pull request, one review and one release. Each release also embeds its own data, which is small enough to compile into the program. Paths still override the embedded copy, so a data-only recalibration that the released validators accept needs no rebuild (§3.3).

3.2 Layout

blis-substrate/
├── contracts/          the packages other repositories implement or call
│   ├── *.go            package contracts: file loaders and bundle validation
│   ├── engine/         engine.Engine, Batch, Transfer, PoolPlan, StepShape …
│   ├── kernel/         the Kernel interface the cost model implements
│   ├── embed/          the router contract the llm-d fork implements
│   ├── spec/           data types: scenario, deployment, model graph, coefficient, …
│   ├── vocab/          shared names: methods, sources, roles
│   ├── check/          validation helpers shared by spec/ and validate/; fmath/, exact math functions
│   ├── conformance/    test suites implementers run (enginetest, kerneltest, embedtest)
│   ├── invariants/     the invariant registry (§13)
│   ├── toolchain       the one Go toolchain version every BLIS build uses (LIF-2)
│   └── harness/        shared checks engine repositories run in their tests
├── catalog/            published facts (YAML, vendor config.json); Python graph deriver
│   └── ledger/         data sources, units and the role rule (§10)
├── registry/           coefficient sets (YAML); Python fitting scripts
├── kernel/             the cost model (Go); "kernel" here is a historical name
├── validate/           validators for catalog and registry, and their commands (Go)
├── data.go             package substrate: go:embed of catalog/ and registry/
├── load/               the loader: reads data from a path or from the embedded copy
├── tools/              docgen, the release bot (§13.6) and other repository tools
├── docs/               one documentation site, with a section per directory above
├── .github/            workflows, filtered by path (§3.4), and the pull-request template
└── go.mod              the one Go module: github.com/inference-sim/blis-substrate

The substrate is one Go module, tagged vX.Y.Z. One commit gets one tag. Every repository that uses the substrate pins a version of it.

The module has two kinds of package:

• Packages under contracts/ hold interfaces, data types, validators of those types, conformance suites, the engine harness and the invariant registry. They import only the Go standard library, the YAML parser and each other. The engine repositories and the router fork import only these packages, and an import test in each of those repositories enforces it.
• All other packages hold the cost model, the catalog and coefficient registry validators, the data loader, and the embedded data. Only inference-sim imports them.

Why a refit needs no engine or router release. Suppose blis-vllm pins substrate v1.2.0 and inference-sim pins v1.5.0, which carries new coefficients. Go builds every module in a program with the newest version any of them requires, so inference-sim's build uses v1.5.0 for blis-vllm too. The packages under contracts/ change only compatibly, except through a contract-change review set (a group of pull requests across repositories, reviewed and released as one change), which releases every implementer together (SUB-1, §12.8). So blis-vllm compiles against v1.5.0 unchanged, and no new blis-vllm release is needed. The nightly compatibility build (§13.4) confirms that every repository, at the tag the newest inference-sim release pins, still compiles and passes its tests against the newest substrate.

Engines must not move the simulator's data. The same rule works in reverse: if an engine required a newer substrate than inference-sim does, the simulator would be built with that newer substrate, and its embedded coefficients would change because of an engine release. To prevent that:

• An engine or the router fork requires the oldest substrate version whose contracts/ packages it uses. It never requires a version newer than inference-sim's current requirement; the release bot refuses such a release. When an engine needs a newer contract, the release bot moves inference-sim first, in the same review set. On a release branch (§14.4) the bound is the substrate requirement of the inference-sim release branch the fix ships in. A fix that needs a newer contract moves that branch to the newer substrate first, in a PATCH release whose notes carry its price-change report. A fix that would need a substrate past an incompatible contract change is not backported.
• Before every inference-sim release, go test runs every engine and router package in a scratch copy of inference-sim's module, with test dependencies added, so each is tested against the substrate the simulator actually selects (LIF-2).
• inference-sim's own requirement on the substrate must equal the version its build selects. If a dependency raises it, the build fails until the release bot re-pins inference-sim, which produces the usual price-change report (LIF-2).

What one module costs. A repository that pins the substrate downloads all of it, mostly vendor config.json files (the release plan's evidence gives the size at the time of that plan; Go's limit for a module is 500 MB). Go compiles only the packages a program imports, so an engine build never compiles the cost model or the embedded data. A CI check fails if the module exceeds 100 MB uncompressed, well before Go's 500 MB limit (SUB-13).

3.3 Data: a path first, the embedded copy as the default

The catalog and the coefficient registry reach a running program in one of two ways. Both are always available, and every run records which one it used.

• A path. --catalog <dir> and --registry <dir> (or BLIS_CATALOG and BLIS_REGISTRY) name directories with the same layout as blis-substrate/catalog/ and blis-substrate/registry/, such as a checkout, a working tree or an exported revision. When a path is given, the run uses that directory instead of the embedded copy. Recalibration uses this. A fitting script writes candidate coefficients to a directory, and the released simulator scores them with no rebuild. Comparing two coefficient registries means running one binary twice with two paths.
• The embedded copy. When no path is given, the run uses the data compiled into the substrate with go:embed. A release therefore carries the exact data it was tested with, and a user of a released binary needs no checkout. inference-sim cannot be installed with go install …@version, because Go refuses that for a module whose go.mod has a replace directive, and inference-sim carries one (X11). So each release ships signed binaries for the three supported platforms, with the provenance and bill of materials of X21. A source build needs a checkout at the tag.
• What stays fixed. A model runs only if it is in the catalog in use, and it is resolved from its graph.yaml. A missing model is refused with the catalog's source (the embedded release, or a path) and the relative path its entry belongs at. No run creates or changes catalog or coefficient registry files, no run fetches anything from the network, and the deployment always comes from a required scenario.

The two flags are independent. A run may, for example, use the embedded catalog with a candidate coefficient registry.

Private additions. To run a private model against released data, --catalog-overlay <dir> and --registry-overlay <dir> add entries to the data in use. An entry is identified as follows:

• In the catalog, a whole model directory (models/<name>/), or a whole hardware, device, network or workload file.
• In the coefficient registry, one coefficient, identified by its set, its name, its scope and its method.

An overlay entry with the same identity as an existing entry replaces it; any other entry is added. The combined data are then validated in full, including the rules that each lookup has at least one entry (VAL-2) and at most one per rank (VAL-4). An overlay may not contain ledger files.

Overlays have their own flags because a recalibration candidate replaces the whole coefficient registry. If one flag could either replace or add to the data, an A/B comparison would be ambiguous.

Which files count as data

The file patterns are defined once, in the substrate's top-level package (data.go). That package must sit at the top of the repository, because Go's go:embed can only embed files at or below the package's own directory. The patterns are: hardware, devices, networks, workloads, measurement sources, each model's model.yaml and graph.yaml, every coefficient set, and acceptance and routing profiles. Most of the ledger is not runtime data: pricing never reads it, and it can be large, because it lists every unit. Seed-holding jobs (the CI jobs that hold the secret used to assign data roles, §10.2) read it from a checkout of the repository at the substrate tag they use, including those in accuracy-dashboard. The exception is the three published role lists (catalog/ledger/published/), which are embedded so that inference-sim ledger can print them offline.

The go:embed directive must list literal patterns, so a test checks that they equal the loader's. A path is read with the same patterns, and the loader refuses anything that would make the data ambiguous:

• a file under a data directory that matches no pattern, such as coefficients/x.yml where .yaml is expected. Ignoring it would silently drop a candidate's change.
• a symbolic link;
• a data file containing a carriage return;
• two paths that differ only in letter case.

The data directories are exactly those the patterns in data.go name. Files outside them, such as scripts and documents, are ignored. Two things inside them are ignored by name: a model directory's config.json, and everything under catalog/ledger/ except published/. The check that graph.yaml's recorded derived_from hash matches config.json runs in the validators, in CI, not in the loader. So a path load and an embedded load validate the same files in the same way (SUB-9).

Vendor config.json files are not data in this sense. They stay in the repository for provenance and for the graph deriver, but the runtime never reads them. Every model value the runtime needs is a field of the graph, written by the deriver from config.json and the safetensors headers. No runtime code reads config.json, no model value is hard-coded in the cost model or an engine, and no user is asked to copy a value from a checkpoint into a setting (SUB-3).

Recording what was used

Every run computes a content hash for each of the catalog and the coefficient registry. It lists every data file as <sha256 of the file's bytes> <path relative to the catalog or registry directory, with forward slashes>, sorts the lines by path in byte order, ends every line, including the last, with a line feed, and takes the SHA-256 of the result. The loader refuses carriage returns, so the hash is the same on every machine. The Result document records, for each of the catalog and the coefficient registry:

• its source: embedded or path;
• its content hash;
• the hash of each overlay, and the hash of the combined data. With no overlay the combined hash is the base hash. Otherwise it is the SHA-256 of the line <base hash> base followed by one line <overlay hash> overlay for each overlay in the order applied, each line ending in a line feed, so it names the base and every overlay and their order.

It also records the build the simulator came from: a tagged release, or unreleased.

A run uses released data when all of these hold:

• the simulator is a tagged inference-sim release, whose stack.json names a tagged substrate release;
• the catalog and coefficient registry hashes equal that substrate release's embedded hashes;
• no overlay is used.

Every other run is experimental, including every run from stack CI, a local build or go run. Only released-data results are published (X8).

Checking data at load

Whatever its source, the data is validated when it is loaded, by the same validators CI runs. That includes the checks between the catalog and the coefficient registry, for example that every chip a coefficient names exists in the catalog in use. A candidate coefficient registry that does not fit the catalog it runs with is refused.

Each run also reports which coefficient registry entries were selected (a lookup returned them), and which coefficient files were never loaded. The report covers every reader behind the Environment, including acceptance and routing profiles. Without it, a candidate that adds a coefficient family no released code reads would be silently ignored and would look like a no-op. The landing gate uses this report (VAL-6).

The ledger in a candidate

A candidate may add data units, for example after a data drop. The gate and select jobs read units from the candidate's tree and compare them, by unit_id, with the ledger at the pull request's merge base:

• a new unit has its role drawn by the job;
• a unit_id present at the merge base but absent from the candidate is refused, unless an alias table or a new partition version covers it and a person has approved it. A unit whose identifying fields changed shows up exactly this way, because its unit_id is a hash of those fields.

Roles are never read from a file (D4), so a candidate cannot change which units are held out.

3.4 Directories, owners and CI

Each directory keeps its own owners, tools and checks.

CI is filtered by path: a catalog-only change does not run the cost model's slow race suite. Any change also runs the cross-directory checks, because catching mismatches between directories is the reason for the merge: validators run against the data, and the cost model against the data it embeds. Python tooling keeps its own lock files under catalog/scripts/ and registry/scripts/.

4Rules that span repositories

These rules hold across the whole system. A pull request that breaks one must not merge, whatever its tests show. Each rule lists its target checks, the repository that runs each check, and the review guidance that remains.

Review guidance is reserved for questions that need judgement. A question is guidance for the review agent, never a check. When a pull request changes something a question covers, the review agent answers it in a review comment, yes or no, with a reason and the files and lines that support it. No CI job fails or waits on a question, and no person has to answer one. The question’s Expertise line names the knowledge the agent brings, not a group of people. An answer in the unwanted direction is a finding, which the one reviewer with write access reads with the rest of the review (§13.2).

• X1 Users configure BLIS with the upstream's own configuration. What a user would hand llm-d, vLLM or SGLang is what they hand BLIS, in the upstream's own field names, types, defaults and errors. BLIS adds only what the real system learns from its environment: hardware, cluster topology and workload. Engine repositories resolve defaults exactly as the upstream does. The router needs no port of its defaults: the fork runs llm-d's own flag parsing, configuration loader and Kubernetes decoding unchanged, so a value the real router rejects is rejected with the router's own message. An unknown or unsupported field is refused by its upstream name, never ignored. The one exception is the router's flags that only change how a deployed router is served or observed (transport, TLS, leader election, ports, telemetry). Those are accepted, have no effect, and are named in a warning.

  Check	Runs in
  Every configuration field of the pinned release is classified as modelled, proven inert, or not modelled (VLM-3, SGL-3).	blis-vllm, blis-sglang
  An unknown field, or a not-modelled field set away from its default, is refused with its upstream name.	blis-vllm, blis-sglang
  Defaults match: the upstream's own configuration code resolves each oracle scenario, and the port's values are compared field by field.	blis-vllm, blis-sglang
  Every configuration the upstream rejects is rejected. This is tested on a generated set: each field's boundary values, taken from the upstream's own argument and validation metadata, and one case for each branch of the upstream's validation code, with branch coverage required.	blis-vllm, blis-sglang
  The scenario's router flags, configuration file and manifests reach the router unchanged, and the router parses them with its own code.	inference-sim
  A set of malformed router flags, configuration files and manifests is refused, each with llm-d's own error message.	router fork
  Every flag in the router's flag set either is on the fork's list of flags that only matter in a cluster, or can change a decision. Every plugin type llm-d registers either is on the fork's list of audited types, or is refused by name. Both lists are regenerated against each new llm-d release and reviewed as a diff.	router fork
  Every scenario key outside the engine and router blocks is on a fixed list of environment facts.	blis-substrate (contracts)

  Review guidance. Does this new scenario field describe something the real system would observe, rather than a setting the upstream does not have? Expertise: contracts expertise. Applies when: a key is added to the environment list.
  Review guidance. Does this router flag only change how a deployed router is served or observed, with no effect on any routing, admission or flow-control decision? Expertise: router fork expertise. Applies when: a flag is added to the fork's list of flags that only matter in a cluster.
• X2 The substrate knows no engine. Outside its tests, blis-substrate contains no engine name, no engine version and no engine's policy. Engine behaviour reaches it only as a resolved PoolPlan, StepShape or Transfer.

  Check	Runs in
  No engine identifier in non-test Go identifiers, string literals or coefficient registry keys: engine names, engine version strings, upstream source-path citations, and engine-only setting names. The word list is committed in the substrate (invariants/engine-words.txt). A bot regenerates it from each engine release's field list in a pull request, leaving out words the architecture vocabulary also uses, such as dtype or quantization. The count per directory may only go down, measured against the word list of the same version, until it reaches zero.	blis-substrate
  The cost model's public functions accept only plans, step shapes, transfers, and catalog and coefficient registry handles.	blis-substrate (kernel/)
  PoolPlan, StepShape and Transfer contain no field whose values come from an engine's vocabulary, and a zero value means "absent" or the field's documented neutral value (such as stage 0), never an engine default (a reflection test). Separately, a scenario both engines support is priced from each engine's plan, and every price difference is traced to a named difference between the plans.	inference-sim (it imports both engines)
  Coefficient scope keys come from a fixed set with no engine key.	blis-substrate (validate/)
  Model graphs use only architecture vocabulary.	blis-substrate (validate/, catalog/)

  Review guidance. Does this change hide one engine's policy (a default, threshold, backend choice or padding rule) behind a neutral name? Would the other engine need the same edit? Expertise: expertise in the changed directory. Applies when: any non-test change under catalog/, registry/ or kernel/.
• X3 Engine behaviour lives only in its engine repository. Scheduling, KV management, KV connectors and tiering decisions, default resolution, and lowering a model to kernel implementations belong to blis-vllm or blis-sglang. Each engine repository models one upstream release at a time (LIF-5).

  Check	Runs in
  The cost model imports no engine module.	blis-substrate
  No engine module imports another.	blis-vllm, blis-sglang
  The simulator reaches engines only through engine.Engine, at one build site.	inference-sim
  Scheduler and KV types exist only in engine modules.	inference-sim, blis-substrate
  The simulator does no resource arithmetic (KV blocks, bytes, tier capacities); it asks the engine or the cost model. The one exception is dividing a link's bandwidth among transfers and the cross-domain traffic of step collectives that overlap in time (SIM-13) (§6.2).	inference-sim
  Every ported file names the release that the repository's Upstream record names.	blis-vllm, blis-sglang

  Review guidance. Is engine behaviour, such as scheduling, admission or KV accounting, being added to the simulator instead of an engine repository? Expertise: inference-sim expertise. Applies when: a change to the simulator's request-lifecycle or capacity code.
• X4 The router is llm-d's own router code, unchanged. The fork changes only how the router is hosted: how it reads time, starts work, draws random numbers and makes calls. It changes no routing decision. Engine-specific router behaviour comes from llm-d's own code for that engine.

  Check	Runs in
  No new clock, timer, goroutine, randomness, map-order or network call on a decision path, except through the fork's runtime interface; every change to an upstream file is of an allowed kind (RTR-2), and every hosting hazard is converted or reviewed (RTR-6).	router fork
  The fork, run under its default runtime, and the upstream release it is based on make the same decisions on recorded request streams. Because the fork keeps upstream's module path, one program cannot load both, so the upstream release is built separately and each build replays the same recorded requests. A decision that depends on a random draw or on map order is compared as a set: the fork's choice must be one the upstream can make. Upstream's own tests also run on the fork, changed only as RTR-2 allows.	router fork
  Under BLIS's simulated runtime, every decision is one the default runtime can make from the same recorded state. Each decision is replayed from the recorded state just before it, not from the simulation's own history, so one different tie-break cannot make every later decision incomparable. Ties and random draws are compared as sets of the choices the upstream could make.	inference-sim
  BLIS imports only the fork's pkg/embed.New, and only audited plugin types are reachable.	inference-sim
  Every branch the fork takes only when hosted is exercised by a test that compares it with the upstream code it replaces.	router fork

  Review guidance. Does this change to how the router is hosted keep upstream behaviour on paths the recorded streams do not cover, such as errors, timeouts and cancellation? Expertise: router fork expertise. Applies when: a fork pull request converts a call site or updates the lint baseline.
• X5 Fidelity is proven by comparing against the upstream's own code. Each engine repository runs the upstream's own Python code on the same inputs and compares its decisions with the Go port's, step by step. Only the environment is replaced (GPU, clock, transport, environment settings, the hardware platform and outside services), and every replacement is listed in a manifest (§7).

  Check	Runs in
  Every replacement the test harness makes is listed in the manifest; a changed manifest carries a new version.	blis-vllm, blis-sglang
  Decisions match on every recorded scenario.	blis-vllm, blis-sglang
  A recorded trace's decisions change only in a commit marked as reviewed.	blis-vllm, blis-sglang
  Every ported function is exercised by some recorded scenario, and deliberately broken copies of it are caught.	blis-vllm, blis-sglang

  Review guidance. Does this new replacement stand in only for the GPU, clock, transport, environment settings, the hardware platform or an outside service, and not for a decision? If it reproduces a decision, is it declared as a transcription? Expertise: engine repository expertise. Applies when: the manifest changes.
• X6 Same inputs, same output. For fixed scenario bytes, seed, module versions, Go toolchain version and catalog and coefficient registry content hashes, the simulator's standard output is identical byte for byte across runs, thread counts and supported platforms. The data source (embedded or path) is recorded in the Result, but is left out of the output the byte gate compares. Simulated time is passed in as a number, and no code reads the system clock. Randomness comes from named substreams seeded from the scenario (DEV-6).

  Check	Runs in
  Identical output across runs and thread counts; a recorded trace replays identically.	inference-sim
  Identical output on linux/amd64, linux/arm64 and darwin/arm64: nightly, and on every pull request that changes kernel/ or an engine's lowering. Two things differ by platform. First, the Go compiler may fuse x*y+z into one fused multiply-add on some architectures and not others, which changes the last bits; so cost-model and engine code writes float64(x*y) + z where fusion must not happen, or calls math.FMA where it must. Second, some functions in Go's math package, such as Exp and Log, use assembly or CPU-dependent code on some platforms, so their last bits can differ between machines; cost-model and engine code calls the pure-Go copies in contracts/check/fmath instead, which use no fused multiply-add. A lint enforces both rules. stack.json records the toolchain version (SIM-3).	inference-sim
  No clock reads and no goroutines in engine or cost-model code.	blis-vllm, blis-sglang, blis-substrate
  No unsorted map iteration on an output path, and no unnamed randomness. In the router fork this lint covers files the fork wrote; upstream code is covered by the differential test in X4.	every Go repository, using lint settings from contracts/harness/golangci.yml. A generator copies them from the pinned substrate version, and a check confirms the copy matches (X7).

  Review guidance. Could this floating-point reduction give a different result on another platform? Expertise: expertise in the changed repository. Applies when: a parallel or floating-point reduction is added, or the cross-platform check fails.
• X7 One owner for each fact; dependencies point one way. Contracts live in blis-substrate/contracts. A repository never imports one above it in Figure 1. A generated file is written only by its generator. Every copy of a fact is generated from its owner or checked against it. The dashboard's scoring harness is the one thing a lower repository uses from a repository above it: it runs the harness at a pinned release or commit, never imports its code. A lower repository's candidate job may also build inference-sim through stack CI to run that harness (§13.5).

  Check	Runs in
  A lower repository refers to accuracy-dashboard only through the harness pin file.	every repository
  The module graph across all repositories follows Figure 1.	blis-substrate (nightly compatibility build)
  The engine repositories and the router fork import only substrate packages under contracts/.	blis-vllm, blis-sglang, router fork
  Every generated file matches its generator's output.	every repository with a generator
  No hand-copied list of another repository's facts; field lists, knobs and plugin parameters are read from their owner at the pinned version.	blis-config-search, accuracy-dashboard

  Review guidance. Does this change give a second home to a fact another repository owns? Expertise: expertise in the changed repository. Applies when: a pull request adds a literal list or table, or a constant that cites another repository.
• X8 Every number is reproducible, and no data point both fits and judges. Every number records the exact versions and data it came from, and can be regenerated from them. A published number, meaning one on the dashboard or in a committed report, is computed from released data only (§3.3). No data point used to fit or choose a value is used to evaluate it (§10).

  Check	Runs in
  No ledger unit is cited under two roles by any fit, gate or dashboard point.	the nightly run of blis-substrate's gate job, over every consumer's recorded unit reads (D2)
  Every data source is pinned to an exact revision.	blis-substrate
  Every Result records the module versions, the source and content hash of the catalog and the coefficient registry, any overlay hashes, and whether the run used released data.	inference-sim
  The dashboard and committed config-search reports accept only released-data results; an experimental result is refused.	accuracy-dashboard, blis-config-search
  Every published number carries its data units.	accuracy-dashboard, blis-config-search
  Published numbers regenerate: on redistributable fixtures on every pull request, on the full data nightly.	blis-substrate, blis-config-search, accuracy-dashboard
  An accuracy change's success criteria are committed before any commit that reads its held-out data.	any repository landing an accuracy change

  Review guidance. Was this fitting method, gate or threshold chosen after seeing evaluation results? Expertise: expertise in the repository landing the change. Applies when: a pull request changes a fitting method, gate or threshold.
• X9 The upstream feature surface is fully accounted for. For every pinned upstream release, each feature the upstream offers has exactly one status in a generated list. Features include connectors, KV tiers, KV-cache managers, speculative methods, scheduler policies, configuration fields, metrics and event types. The statuses are: modelled, with evidence; refused by name, with an issue; or out of scope, for one of three listed reasons (§7.5). Modelled is the goal; the other two are exceptions, each with evidence.

  Check	Runs in
  The surface list is regenerated from the pinned upstream source and matches the committed list.	blis-vllm, blis-sglang, router fork
  Every item has a status, and every modelled item names its evidence.	blis-vllm, blis-sglang, router fork
  Coverage of the llm-d stack's published configurations is computed and published per release.	inference-sim
  Every out-of-scope item names one of the three allowed reasons and cites upstream. An engine release does not lower the weighted share of modelled items, and does not move an item from modelled to refused without a fidelity issue that explains why.	blis-vllm, blis-sglang, router fork; the release bot

  Review guidance. Is this "out of scope" reason one of the three allowed, with a citation that supports it, and does the llm-d stack never use the feature on the hardware BLIS targets? Expertise: engine repository expertise. Applies when: an item is marked out of scope.

4.1 Process rules

• X10 Repositories are released bottom-up. A change spanning repositories is one review set. Each repository's part is tagged before the repository above it pins that tag.Checked by: release bot (§13.6).
• X11 Pins are exact release tags. The only committed replace directive is the one in inference-sim's go.mod that substitutes the router fork for upstream llm-d. It exists because the fork keeps upstream's module path (github.com/llm-d/llm-d-router), so Go cannot fetch the fork by its own path; a replace is Go's only way to substitute it. It names the fork's exact -blis.N tag, never a branch or a local path, so it pins as exactly as a require does, and go.sum records its hash. It may appear only in inference-sim, because Go ignores replace directives in every module but the one being built. No other module requires the fork; engines and other repositories use only the router contract in contracts/embed. The only committed exclude directives are those in inference-sim that skip a plain tag made before the rule of §14.1, and no repository commits a go.work file.Checked by: selected-version lint (LIF-2), in every repository: no replace outside inference-sim; in inference-sim, exactly one, from github.com/llm-d/llm-d-router to github.com/inference-sim/llm-d-router at a -blis.N tag; no other module requiring github.com/llm-d/llm-d-router; exclude directives only in inference-sim, each naming a plain tag of an engine module; no committed go.work.
• X12 Only derived values are committed, never third-party data, with one exception: vendor config.json files whose licence is on the allow-list. Each such model records the licence identifier in its model.yaml; a model whose licence is not on the allow-list commits only the file's hash and revision, and the deriver fetches it in a nightly job that holds the access token; pull-request jobs check only the committed hash and revision and the graph's derived_from hash, so no pull-request job needs a secret. Each derived value cites its source by URL and commit or hash. No private names appear in files, commits, pull-request text or issue text, and reports and documents name internal data neutrally, as internal experiments.Checked by: name scan in every repository, which also reads the text of every open issue each night; provenance schema rules; a check that every committed config.json has an allow-listed licence in its model.yaml. These are checks on the project’s own repositories, run in CI. No simulation path reads, requires or enforces a licence: the loader checks that data is well formed, and a catalog or coefficient registry a user supplies is not refused for lacking a licence record.
• X13 Every behaviour change has a test that fails without it. A validation change (§8.9) to a published number carries success criteria fixed in advance, plus held-out evidence. A verification change that moves published numbers, such as a fidelity fix, carries the price-change report and the scoring harness's figures on published units, and is never rejected for exposing an error elsewhere (VAL-11). A refactor shows identical output on the byte gate and on the dashboard's corpus: the recorded configuration of every ledger unit the dashboard can simulate, re-run through the scoring harness, with no measured value read. That corpus covers shapes the byte gate does not. Each nightly run of the whole corpus records which functions each point reaches. A pull request then runs every point that reaches a function it changes, in parallel; with no such record for its base, it runs the whole corpus. The whole corpus also runs before every release. A pull request that is both a refactor and a behaviour change is split.Checked by: byte gate in inference-sim; the corpus comparison, run by the scoring harness at the pinned commit from each repository's CI, for pull requests declared as refactors; landing gate (VAL-6); mutation checks: on a pull request, the functions it changes and the functions covered by any test it changes are mutated, and only the tests that cover them run; and the full sweep runs nightly and on release tags.
• X14 A pull request's description is accurate: the commit hashes it lists are the branch heads, it merges cleanly, and every invariant it cites exists.Checked by: the description check, a shared workflow run in every repository.
• X15 Each validation rule exists once, in Go, in blis-substrate. Python tools validate data by calling the substrate's validate command. A Python check that restates a Go rule must pass a test that runs both on the same set of failing examples.Checked by: the parity test in blis-substrate; a lint that flags schema-like rule code in Python scripts.
• X16 A skipped test in a required or gate job is a failure, not a pass, unless it is on the quarantine list of X25. A gate's workflow triggers are among the paths it declares, so removing a trigger needs review. A gate whose last real run is more than seven days old is shown as failing.Checked by: each repository's CI configuration; the nightly status page (§13.4).
• X17 Every repository's protected branch requires its named checks, including formatting and lint. No change merges without a successful run of them, on the tree being merged (X22).Maintainer practice (§4.2): the repository’s own branch protection, set by a person with write access and enforced by the hosting service on every merge; its required checks are the repository’s own jobs. No other repository audits it. The settings are on the maintainer checklist (§13.1).
• X18 Every check in the invariant registry names a known-bad input that must make it fail. A check that cannot fail is not a check.Checked by: a nightly job in blis-substrate that runs each check on its known-bad input and fails if the check passes; the nightly mutation sweep (T7), which covers substrate validators and the simulator's own checks as well as engine code.
• X19 Scripts and generators, in every language, never write an engine default. They leave an unstated setting unstated, or read the default with inference-sim fields. A rendered scenario or report shows each resolved value with its source: stated by the user, recorded by the measured run (including a value it logs as resolved), the engine's default, or, for a workload-dependent value, a cited measurement (D8).Checked by: the engine-name lint (X2), run on *.py and *.sh in every repository that has scripts; in every scenario generator, a test that a generated scenario contains only stated or recorded values, and workload-dependent values with their citation. The name lint cannot see a bare number under a neutral key, so the second test is the one that catches it.
• X20 A commit written with an agent's help says so in one Assisted-by: <tool> line, and carries no Co-authored-by line naming the tool. An agent never signs off in its own name, since only a person can certify the DCO. Signing a commit and adding a DCO Signed-off-by line are encouraged for the person who takes responsibility for it, and an agent that pushes commits for a person, on that person’s instruction and with that person’s key, may sign and sign off in that person’s name; no check requires either. Release-bot commits are signed with the bot's identity and carry no sign-off. In the router fork, this applies to every commit not reachable from the upstream tag being merged; upstream history is kept as upstream wrote it.Checked by: a commit-policy check, required on every pull request in every repository, over the commits the pull request adds. It checks the form an agent can get wrong: one Assisted-by line, no Co-authored-by naming a tool, no Signed-off-by naming a tool. It does not check for a signature or a sign-off. Maintainer practice (§4.2): the person who takes responsibility signs the commits and signs them off, or instructs an agent to push them in that person’s name. It is encouraged and never fails a pull request. Agent guidance (§4.2): how a commit message names agent help.
• X21 Everything a build or a release runs is pinned and traceable. GitHub Actions are pinned by commit, not by tag. Python environments used by oracles and fitting scripts are locked with hashes. Each inference-sim release publishes build provenance and a software bill of materials. Each repository keeps the licence and notice files its code requires, including blis-vllm's NOTICE for code derived from vLLM; by policy blis-sglang keeps one too for code derived from SGLang. Each derived file keeps upstream's licence header (vLLM's SPDX line, SGLang's Apache notice) and states that it was changed, as Apache-2.0 section 4(b) requires.Checked by: a workflow lint that refuses an action not pinned by commit; pip install --require-hashes in every Python CI job; the release job, which fails without provenance and a bill of materials; a check that LICENSE and any required NOTICE exist; the provenance test checks each derived file's licence header.
• X22 Every verdict (a CI result or the review agent's review, including its answers to review questions) names the git tree hash of the exact tree it judged. A CI verdict is on the head tree of the pull request. The review agent’s verdict is on the head tree, and is given again after a push. BLIS repositories do not use a merge queue. Every job that scores, fits or checks data states the minimum number of items it must process, and fails below it. No step in a required job ignores an exit code.Checked by: the shared pull-request check in blis-substrate/.github/workflows, called by every repository, which records each review-agent answer with the head tree; branch protection, which dismisses an approval when a push changes the tree; a workflow lint that refuses ignored exit codes; each job's item count.
• X23 A version is written in one place. stack.json, each repository's go.mod, contracts/toolchain, and each application's or harness pin file are the only files that state a version; every other file that mentions one (Dockerfiles, workflow files, documentation) is generated from them. Documentation does not repeat a version those files state: it says “the pinned release” and links to the file that states it. A page names an upstream release only when the page is about that release, or where a citation needs it (vllm/<path>:<lines>@<commit8>), because a line number means nothing without its release. A workflow reads the Go version from go.mod. A draft specification under specs/ is a record of what was planned and is exempt.Checked by: a lint in every repository, over every file type except records under specs/, for version literals of BLIS modules or Go that are not generated. In the router fork the lint covers the files the fork wrote; upstream’s files are kept as upstream wrote them (X4).
• X24 Every review finding ends in one of three outcomes before merge: fixed, filed as a linked issue, or rejected with a reason. An issue closes only through a commit or pull request that names it, or as not planned, by a reviewer with write access, with a reason.Checked by: branch protection, which refuses a merge while a review conversation is unresolved; a pull-request check that refuses a thread with no outcome; a nightly job in each repository that reopens an issue closed any other way. Agent guidance (§4.2): answer each thread with its commit or issue, and resolve it once the reviewer is satisfied.
• X25 Tests read only their declared inputs, never sibling checkouts, ignored directories or the user's environment. A test failing on the main branch blocks every merge but its fix, unless it is quarantined with an owning issue and an expiry date.Checked by: CI in a clean container with no sibling checkouts; the quarantine list, which fails CI when an entry expires; a required check that fails while main is failing and the pull request does not name the failing test's issue.
• X26 In documentation, every source path, symbol, issue or pull-request reference, and every statement that a feature is supported, is either generated (from surface.json, stack.json or the code) or checked against the pinned tree. A pull request carries the documentation change that its own change needs: a path it removes or renames, a behaviour it changes, a setting it adds or drops. A repository that adopts the check clears the findings it already has in the pull request that first requires the check, so the check is green from then on and every later finding is a pull request’s own. No finding is hidden from the check by a list of known ones. Documentation has two kinds of reader and is written for each. A person needs the concept: the design, the invariants, the process and why they are so, with no version, count or other detail that changes without the idea changing. An agent needs pointers: where a rule, a file or a command lives, so it can change the code correctly. Each page is for one of them, says which, and is no longer than that reader needs. A page for people holds no path, symbol, version or count, so there is nothing in it for a check to fail on; the reference check reads the pages for agents, and skips records such as specs/ and RFCs, which say what was true when written. A check that fails a pull request for something it did not touch, or for a detail that was never needed, is a defect in the check, and the fix is to narrow the check or delete the detail, never to add a list of known findings. Documentation is short, it says where a rule, a file or a command lives and links to it, and it leaves out anything that can drift without a check noticing, such as a version, a count, a line number or a list of what is supported at the moment of writing.Checked by: a documentation reference check in every repository, which fails on any reference that does not resolve in the tree the pull request produces. In the router fork the check covers the documents the fork wrote; upstream’s documents are kept as upstream wrote them (X4). Review guidance (§4.2): the review agent asks whether the change leaves a document describing something it removed or changed, and raises a finding when it does.
• X27 An open bug labelled as breaking an invariant blocks a release of the repository where the bug lives, and of each repository that ships that code, unless the release fixes only that bug or a security issue, or a release owner records a waiver in the release notes.Checked by: the release bot.
• X28 An issue in a BLIS repository that is cited as the reason for a refusal, for a refused or not-modelled item in a field or surface list, or in a code comment that marks a gap or a divergence, stays open. It closes only in the change that removes or updates every citation of it.Checked by: a nightly job in every repository that resolves each cited issue and fails on a closed one; a pull-request check that refuses a change closing an issue still cited elsewhere; the nightly job also reports an issue closed outside a merge.

4.2 Three kinds of rule

Every invariant is kept in one of three ways. The design says which, so that a rule with no program behind it is a decision and not a gap.

The test. A rule is automated when a program can decide it from the repository’s own contents and the repositories below it, the same way release after release, and when running it needs no knowledge of a repository above it. A rule about the repository as a whole, about who holds a permission, or about an act only a person can perform is a maintainer practice. A rule that shapes how a change is written or judged, and that a program cannot decide, is agent guidance. A rule can be both: a program checks the form of a commit message while an agent follows the convention that produces it, and a person signs the commit.

Dependency direction holds for all three. An automated check reads only its own repository and the repositories it depends on (X7). A check in one repository never reads the settings, branches, issues or workflows of another, and no job exists to audit a practice from a different repository. A practice is kept in the repository it concerns, in that repository’s own settings and documentation. Agent guidance names only what the repository it is rendered into may know. A program is added for a rule only when the rule is deterministic and local; complexity, or a link between layers that the dependency direction forbids, is a reason to keep a rule a practice.

5Each repository's design

Each card gives the repository's purpose, what it owns and must never hold, its interfaces, what triggers a change, how correctness is shown, and its own invariants.

6The boundary between engines and the cost model

6.1 The rule

Only an engine repository knows its upstream, so only it works out the engine's behaviour: defaults, backend choices, padding and graph-capture sizes. It hands the cost model three plain objects, described in physical units (bytes, tokens, ranks), never in an engine's vocabulary: a PoolPlan, a StepShape, and, for each KV move, an engine.Transfer whose Move field states the move in physical units: the catalog device, the node and rack that send and receive, and the direction. The cost model never sees an engine's settings and never applies an engine's rules (SUB-6). A release plan may relax this rule for one named input while settings resolution moves into engine lowering: the cost model may read the typed engine-settings carrier (contracts/kernel/enginesettings, Inputs.Engine and the code that reads them) as a transitional input. The relaxation lets nothing new in: no engine default, spelling or backend choice is added to the substrate while it lasts. It ends when engine lowering resolves every setting and the substrate receives only the plan, the step shape, transfers and operation facts. The plan names the relaxation, the date the maintainer approved it and the milestone at which it ends (X2, SUB-6).

6.2 The plan and the step shape

Both are defined in contracts/engine. Every field added later must keep its zero value meaningful, so the contract only grows (SUB-1).

// PoolPlan is fixed for the life of one engine pool.
type PoolPlan struct {
    Hardware string         // catalog chip name
    Fabric   string         // catalog network name; "" on one node
    Layout   Layout         // tensor, pipeline, data, expert and context parallel widths, and rank placement
    Layers   []LoweredLayer // one entry per (layer, primitive)
    KV       []KVGroup      // per cache group: bytes per page, tokens per page, full or sliding window
    KVBudget []int64        // bytes per rank available for KV, one value per pipeline stage
    Tiers    []KVTier       // the KV tiers below GPU memory, as the engine resolved them
    Graphs   GraphPolicy    // CUDA-graph buckets, keyed by token count or by batch size
    Drafts   DraftPolicy    // speculative decoding: chain length, or tree (top-k × steps)
    DraftLayers []LoweredLayer // the draft model's own layers, lowered like the target's
    Host     HostPlan       // per-request and per-token host work (admission, sampling, detokenizing)
    Weights  []ResidentTensor // per rank: shape, stored data type and sharding, after weight post-processing
    Buffers  []Buffer       // engine-chosen buffers and workspaces, sized as the upstream sizes them
    InFlight int            // most steps in flight at once; 1 without asynchronous or overlap scheduling
}

type LoweredLayer struct {
    Layer      int          // index in the model graph
    Stage      int          // pipeline stage that runs it
    Primitive  string       // from a fixed list: "attention.decode", "gemm", "moe.grouped_gemm", "all2all", …
    KernelImpl KernelImpl   // what executes it; the registry looks coefficients up by this
    Shape      PrimitiveShape
    Collective *Collective  // group, bytes, and whether it crosses nodes
}

type KernelImpl struct {
    Library  string         // from a fixed list in contracts, e.g. a named attention library
    Variant  string         // the library's own exported kernel name; an engine's backend names map to it in the engine's lowering
    CodeHash string         // hash of source and target GPU architecture, or of a binary library (below)
}

type KVTier struct {
    Device   string         // catalog device name, e.g. host DRAM, local NVMe
    Bytes    int64          // capacity per rank, as the engine sized it
    Shared   bool           // one copy per node rather than one per rank
}

// StepShape is one engine step on the data-parallel ranks that run in lockstep, already padded by the engine.
type StepShape struct {
    Mode      StepMode      // Mixed, ExtendOnly, DecodeOnly, Verify, Prebuilt or Idle
    Ranks     []RankStep    // per data-parallel rank in the lockstep group: each request's scheduled and cached tokens
    Padded    []PaddedShape // per rank: the shape actually launched (graph bucket, padding)
    Drafts    DraftShape    // tokens drafted and verified this step
    Stage     int           // pipeline stage; 0 without pipeline parallelism
}

Prebuilt is SGLang's P/D decode merge: KV arrived from the prefill pool, so the step runs no forward pass. It still has scheduler and host cost, which the cost model prices.

PaddedShape also records how the step launches: a full CUDA-graph replay, a piecewise replay with some layers run eagerly, or a fully eager run. These have different costs, so the engine states which one applies.

These definitions imply five rules:

• The engine pads the batch, and the cost model prices the padded shape. vLLM pads in its model runner, to token-sized graph buckets and data-parallel padding rules. SGLang captures decode graphs by batch size. Each engine reports the shape it launches in Padded. When data-parallel ranks run in lockstep, padding depends on every rank's tokens in the same step, so a StepShape describes all the ranks that run in lockstep. The engine decides which ranks those are. In vLLM v0.32.0, data-parallel ranks run in lockstep only for mixture-of-experts models; for a dense model each rank is an independent engine (v1/engine/core.py:1473-1481), so its StepShape has one rank.

• The engine names the exact kernel implementation. SGLang, for example, picks an attention backend per chip and model, and forces a page size of 64 for FlashMLA (arg_groups/overrides.py:1146-1154). The engine's lowering records that choice in KernelImpl, and the coefficient registry prices that exact implementation.

  The coefficient registry's lookup key uses a hash of the kernel's source (the kernel hash, CodeHash), not a package version. Engines pin library versions in their lock files (vLLM v0.32.0 pins flashinfer-python==0.7.0.post1), and those versions move often without changing a given kernel. A coefficient lists the kernel hashes it is valid for. A version bump that leaves a kernel unchanged therefore needs no substrate change. A kernel whose source changed gets a new key, and the substrate must supply a value for it (§9). Libraries shipped only as binaries, such as cuBLAS, cuDNN and FlashInfer's precompiled kernels, are keyed by the SHA-256 of the shipped library file together with the target GPU architecture. For a kernel built from source, the hash covers the source and the target GPU architecture. A compiler or CUDA toolkit change can change a kernel's speed without changing its source; the engine records the toolkit version with each kernel implementation, and the coefficient registry treats a toolkit change like a library change that leaves the source alone: the entry stays, and the landing gate (VAL-6) shows whether the old value still holds. Each engine release ships a generated table, kernel_impls.json, of every kernel implementation it can emit, for its pinned library versions and for every library version seen in InferenceX runs. Runtime tuning (autotuned launch configurations, library heuristics, and tuned-configuration files such as vLLM's per-GPU fused-MoE configurations) is part of the implementation's measured behaviour, not a separate key. If a lookup has no entry for an exact implementation, the cost model takes the most trusted entry §9.2 offers for that chip, at worst the primitive family's roofline or assumed entry, and the price records that method (VAL-3). Kernels that ship inside an engine, such as vLLM's custom all-reduce, use the library names vllm-csrc and sglang-kernel. These are library names, and the engine-name check (X2) allows them as such.

• The engine decides which KV blocks move between tiers, and the substrate prices each move. Which blocks move between GPU memory, host memory and storage, when, and under which policy, is engine behaviour: vLLM's offloading connector and tiering factory, SGLang's HiCache. The engine reports each move as an engine.Transfer. The cost model turns each transfer into a demand: bytes per link, from catalog device facts and coefficient registry tier coefficients. The engine returns its transfers to the simulator with each step. The simulator asks env.TransferDemand(engine.Plan(), t.Move) for each one, through env.DemandFor, which returns the bytes each link must carry. env.DemandFor first refuses a transfer whose report and Move disagree, or that its plan cannot make. An engine that starts a transfer implements engine.Planned, whose Plan() returns the PoolPlan the transfer is priced against. The engine states a peer's node and rack from the request (engine.Request.Peer), which a host must set on a request that pulls KV from a peer. The simulator fills in nothing. Two limits are deferred to M3. A plan.Transfer has one sender and one receiver, so for a pool that spans nodes the engine states the node that holds the tier, and the cost model prices that node's link (blis-substrate#48). A plan.Transfer has one token count, so the blocks each KV cache group moves, which differ in a hybrid model, are not stated (blis-substrate#49).
• Sharing a link over time is the simulator's job. A transfer can span many steps, and several transfers can share one link. Dividing a link's bandwidth among concurrent transfers depends on when they happen, so it belongs to the simulator's event loop. This is the one place the simulator does resource arithmetic, and X3 states it as the only exception. Everything else about tiers comes from the engine or the cost model: capacity, bytes per block, and which blocks move.
• The engine combines pipeline stages. With pipeline parallelism the engine emits one StepShape per stage. The cost model prices each stage alone. How stages overlap, and where bubbles occur, is the engine's batch-queue behaviour, so the engine combines the stage times and reports them to the simulator.

6.3 How an engine is built: two phases

An engine cannot size its KV cache without knowing how much memory the model's weights, activations and buffers take. That is the cost model's arithmetic. The cost model, in turn, cannot price a step without the engine's plan. inference-sim breaks the loop by giving the engine an environment when it builds it:

// contracts/engine: what an engine may ask of the substrate while it is built and run.
type Environment interface {
    Graph(model string) (*model.Graph, error)          // the catalog's model graph
    Chip(name string) (*hardware.Chip, error)          // catalog hardware facts
    Memory(partial PoolPlan) (kernel.MemoryBreakdown, error) // weights, activations, buffers; per stage
    Price(plan PoolPlan, step StepShape) (kernel.StepEstimate, error)
    TransferDemand(plan PoolPlan, t plan.Transfer) (kernel.TransferDemand, error) // t is an engine.Transfer's Move; plan is the engine's Plan()
    Acceptance(model, drafter, workload string) (acceptance.Profile, error)
}

// engine.New builds an engine for one pool.
func New(spec Spec, env Environment) (Engine, error)

Here plan.Transfer is the move as the cost model receives it: the tokens moved, the node and rack at each end, the direction, and the catalog device of a tier. It is defined in contracts/engine/plan, with PoolPlan. The simulator takes plan from engine.Plan() and t from the Move of each transfer the engine returns.

Construction then runs in two phases:

1. Plan. The engine resolves its configuration and lowers the model into a partial PoolPlan, one whose KV budget is still empty. The partial plan carries the engine's own profiling step shape (in vLLM, the shape its memory profiling run uses; in SGLang, the shape behind its reserve formula) and the set of graphs it will capture. The engine states the resident weights and the buffers it chooses in the partial plan. env.Memory sums their bytes, and sizes the activations of the profiling shape and the graph pools. Choosing the profiling shape is engine policy; the cost model only sizes the shape it is given. The engine then applies its upstream's budget rule, such as vLLM's gpu_memory_utilization or SGLang's mem_fraction_static, and fills in KVBudget.
2. Run. Each step, the engine forms a StepShape and asks env.Price for its time.

inference-sim implements Environment using the substrate's cost model and data loader. Engines therefore import only packages under contracts/, yet run against the released catalog, coefficient registry and cost model. Engine tests use a small reference implementation in contracts/conformance, which prices from fixed test values.

6.4 Step modes and engine differences the contract must allow

7Fidelity to upstream code

7.1 What an engine port contains

An engine repository ports all of its upstream's code that decides what each step runs and what each request's KV cache costs. That code falls into four parts. Each surface-list item (§7.5) has one primary part, the one that owns most of its decisions. A feature whose code spans parts, such as speculative decoding, LoRA or HiSparse, is still one item, and it names the other parts it touches. The table names each upstream's classes at the release the engine repository pins; the names change between releases, the four parts do not.

Metrics are outputs of the parts that produce them. Each part is ported framework first (§7.6). In vLLM, the KV-cache framework is the layer grouping, the coordinators and the manager base class; in SGLang, it is the tree-cache interface and the allocator and memory-pool base classes. Each further spec type, tree cache, allocator or pool is then a small change on top. The surface list has a category for each part, so a model or configuration that needs an unported manager, tree cache or pool is refused by name and counted, instead of failing inside the engine. The parts are tested by these tiers (§7.4): scheduling by T2 and T3; KV-cache management by T2; transfer and tiering by T4 and T5; resolution by T1; lowering by T0, by the comparison of priced with launched work (FID-11) and by T3's padding comparison. T6 checks the metrics and KV events of every part, and T7 covers all four.

7.2 The method

"Faithful to vLLM" means: give vLLM's own scheduler and BLIS's Go port the same inputs, and they make the same decisions at every step. To run vLLM's scheduler without GPUs, a test harness replaces only its environment. Everything else is vLLM's own code at a pinned commit. Figure 3 shows the arrangement.

7.3 The boundary for each upstream

7.4 Test tiers

Every engine repository uses the same tiers. A trigger's playbook names which tiers must run again.

7.5 The surface list: making gaps visible

The tiers test only ported code. To show what is not ported, each upstream-tracking repository generates a surface list (surface.json) from the pinned upstream's source. The generator reads the upstream's own plugin registration tables, so a new feature in an upstream release appears in the list without anyone remembering to add it.

Each item has one status (X9). The aim is that every item the llm-d stack uses is modelled. The other two statuses are exceptions, and each needs evidence a reviewer can check:

• modelled, naming the oracle scenarios that cover it. This is the default goal for every item.
• refused, meaning the engine refuses it by name until it is modelled. The item names its issue and the backlog counts what it blocks (§11.6). A refused item waits in the backlog, and the item that blocks the most dashboard points or published deployments is the next one modelled.
• out of scope, for an item BLIS will not model. Only three reasons are allowed: the item runs only on hardware BLIS does not simulate (for example AMD-only or TPU-only); it does no work the simulator prices (for example a multimodal encoder when BLIS simulates text serving); or it is deprecated upstream and slated for removal. The reason names which of the three it is, with a citation to the upstream code or announcement. An item may be out of scope only while its weight (below) is zero. Otherwise it is refused, and appears in the backlog.

"Hard to port", "rarely used" and "needed later" are not reasons for either exception; they are priorities, which the weight and the backlog already express.

Each item also has a weight: how many distinct deployments in llm-d's published guides and model recipes use it, and how many accuracy-dashboard points it blocks. surface.json holds only statuses, because an engine repository cannot depend on the guides or the dashboard (X7). The guides commit is pinned once, in inference-sim's guides.lock. Each release records it in stack.json, and inference-sim coverage publishes each item's deployment count. The accuracy dashboard, which sits above every other repository, adds its own blocked-point counts to give the weights, and opens an issue against the engine repository for any out-of-scope item whose weight is not zero. The engine then marks the item refused. The count parses each rendered manifest's engine arguments with that engine's own argument parser. The weighted share of modelled items is published with every release (SIM-5), and an engine release may not lower it (X9). The list is also the reading list for an upgrade: the difference between two releases' lists is the new work.

7.6 Port a shared framework before the features built on it

Connectors, tiers and KV-cache managers are built on shared frameworks. In vLLM these are the connector base class; the offload spec, manager and tiering layers; and the KV-cache coordinator with the manager base class, which every spec type's manager extends. In SGLang they are the HiCache controller and storage interface, and the tree-cache interface with the allocator and memory-pool base classes. Porting one framework completely, with recorded traces of its contract, makes each further connector, tier or manager a smaller change. Without it, each one is a large port of its own. Engine repositories therefore port a framework as a unit before porting the features built on it.

A framework's interface is the abstract class or classes that its features implement or call. For the connector base it is the connector base class, for offload tiering the offloading manager and spec, and for HiCache the storage interface. A framework's traces are taken at that interface and are of two kinds. A recorded trace is the sequence of interface calls that a real feature makes while the oracle runs a scenario. The port receives the same calls, and its answer to each is compared with the upstream's. A direct trace is a sequence of calls made by a driver that the engine repository writes, including orderings that no existing feature makes. Recorded traces keep the call mix realistic. Direct traces test the contract itself, not one feature's use of it. A part of a framework that reads the clock is traced with the clock replacement that FID-1 lists. When only some parts of a framework have traces, the surface list holds the parts as separate items, each with its own status.

7.7 Fidelity invariants

• FID-1 The manifest lists every replacement. Any write the harness makes outside a listed site fails CI, and a changed list carries a new version. The configuration the oracle resolves equals, outside the fields the manifest lists, what the upstream's CUDA platform would resolve for the chip the scenario names. Restoring a CUDA value that depends on choosing a CUDA attention backend is a transcription (FID-8), checked against recorded CUDA start-up logs. Every read of the current platform reachable from ported code is listed in the manifest.Checked by: oracle/check_mocks.py, from contracts/harness, in each engine repository.
• FID-2 The oracle runs in a locked Python environment at the pinned upstream commit, and every recorded trace records that commit, the lock file's hash, the scenario's hash, the seed and the manifest version.Checked by: oracle/check_lock.py and the trace provenance check.
• FID-3 A recorded trace's decisions change only in a commit carrying a golden-reviewed: line that names the trace and gives the reason. Tools refuse to rewrite traces unless an explicit update flag is set.Checked by: oracle/check_goldens.py; the oracle runners refuse to write traces without the update flag.
• FID-4 Every ported function is exercised by at least one recorded scenario.Checked by: coverage gate.
• FID-5 A framework (connector base, offload tiering, HiCache) is marked modelled in the surface list only when it has recorded and direct traces at its own interface (§7.6). A feature built on the framework does not supply them. A part of a framework without those traces is a separate item with its own status. It is not marked modelled, whatever the status of the other parts.Checked by: a surface-list check that a framework marked modelled names recorded and direct traces taken at its interface, and that each part without them is a separate item that is not marked modelled.
• FID-6 A property asserted about an engine port, such as a monotonicity law, is first run against the upstream oracle. It is adopted only if the upstream satisfies it on the oracle scenarios. A law about the cost model names the upstream behaviour it rests on.Checked by: the property tests run in the oracle tier before they run against the port.
• FID-7 For every catalog model and each supported layout, the resident weights an engine reports in its PoolPlan (per-rank shape, stored data type and sharding, after the engine's weight post-processing) equal what the pinned upstream allocates. The graph's tensors equal the checkpoint's safetensors headers.Checked by: oracle tier T0 in each engine repository, run whenever a substrate release changes the catalog or the engine moves to a new upstream release.
• FID-8 The harness may not contain a copy of an upstream decision function. A harness file with a Derived from header, or a function named like an upstream function, is refused unless the manifest declares it as a transcription.Checked by: oracle/check_mocks.py.
• FID-9 The engine is tested as the simulator drives it, not only on its own. Each recorded trace also records the timed inputs the engine received: arrivals, completions, transfer ends and aborts. Replaying those inputs through inference-sim's driver into engine.Engine reproduces the trace's decisions step by step.Checked by: the coupled replay test in inference-sim's CI, over every engine repository's recorded traces.
• FID-10 Every compared field and metric series takes its neutral value and at least one other value somewhere in the scenario set, so a comparison of it can fail. The comparison is two-sided: a field or series that appears on only one side is a mismatch.Checked by: a value-coverage report in each engine repository's CI, which fails on any field or series seen only at one value.
• FID-11 The work a step shape asks the cost model to price equals what the pinned upstream launches for the same step, for each layer: the rows of every matrix multiply, each operation's FLOPs and bytes, the token count of every draft pass, and the context entries each context-dependent operation reads, bounded by its window where it has one. Rows and token counts match exactly; FLOPs and bytes match within a stated tolerance. FID-7 compares what is resident; this compares what runs.Checked by: an oracle comparison, run with T0 and on engine pull requests that change how step shapes are emitted. It executes the upstream forward pass on meta tensors, at the padded batch the upstream launches, under a PyTorch dispatch mode that counts each operation's rows, FLOPs and bytes, with the ranks under a fake process group. Each opaque custom operation, such as attention or a fused mixture of experts, is counted by a rule the manifest lists. Routed-expert rows are compared in total (tokens times experts per token), since routing depends on data. Context reads are counted from the upstream's own attention metadata: sequence lengths, block tables and windows. Known-bad inputs: a matrix multiply priced at fewer rows per token than the upstream runs, and a windowed gather priced over the whole context.

8Simulation method

BLIS is a discrete-event simulator: it moves simulated time from one event, such as a request arriving or a step ending, to the next. Between events, state either stays fixed or changes at a known constant rate. This section states the rules such a simulation must follow, whatever engine it runs. It takes its terms from DEVS, the Discrete Event System Specification introduced by Bernard Zeigler. The rules cover which part of the system may change state and when, what happens when two events fall at the same instant, how random numbers are drawn, and what a run measures. The table gives each term with its meaning in BLIS.

8.1 Components and how they couple

Each component changes its state only through its three transitions, and sends messages only through its output function. The output function runs on the component's state just before an internal or confluent transition, and every message it sends carries that instant's time. An external transition sends nothing. A component that must answer a message at once, as the router answers a request, schedules an internal event with a time advance of zero and sends its answer then.

Between events a component's state is constant, or changes at a constant rate. A link is the one component of the second kind: its state is the set of transfers in progress and the bytes each has left, and its time advance is the earliest time a transfer finishes under the current sharing of the link. Every start or end of a transfer, and every step that starts or ends a cross-domain collective, is an event that recomputes the rate of every other transfer on the link.

Components interact only through messages. No component reads another's state directly. The router learns about an engine only through the metrics and KV events the engine sends, at the times it sends them, after the delay the deployment's scrape and event-delivery settings impose. This rule lets each component be tested alone and then together.

Within a component, nothing depends on goroutine scheduling, the system clock or the order of iteration over a map. Concurrency inside a component, such as the router's background work, runs on the simulated runtime, which serialises it (RTR-7).

8.2 One instant

Many events fall at the same simulated instant, for example a step ending as a metrics scrape is due. BLIS processes an instant in the way Parallel DEVS defines, in three phases:

1. Every component whose internal event falls at this instant runs its output function.
2. Each component receives the bag of every message addressed to it at this instant. BLIS sorts the bag by the key below, so a transition that reads it in order is deterministic.
3. Each component then makes exactly one transition: confluent if it had both an internal event and messages, internal if only the first, external if only the second.

So no component sees another component's state change from the same instant before making its own. The messages a component receives at one instant are ordered by a key: (time, class, sender, the sender's own count of messages sent). The class is a fixed order of message kinds, such as "transfer complete" before "new request". Each class's position is stated with its reason, and a change to the order is a change to the model (§8.9). The key contains no global counter, so it depends only on what each sender has done, not on the order in which the simulator happened to schedule things. Two messages with the same key are a defect.

A component's confluent transition says what it does when its own event and a message coincide. For an engine, this follows from where the upstream's loop reads its input queue, not from a choice BLIS makes, except where the upstream's order is a race (below). A request that arrives while a step runs is an ordinary message: an external transition holds it until the engine next reads its input. When that is depends on the engine and its settings. Under vLLM's asynchronous scheduling, on by default, the next batch is scheduled while the current one runs, so a request arriving during step N usually first joins step N+2, because step N+1 was scheduled when step N started. Without it, the request can join step N+1. Each engine declares, for each kind of external input, the step boundary at which it takes effect, whether it arrives during a step or at the instant one ends. Where the upstream's order is a race between threads, the declaration says so and names the order BLIS uses.

A value one component publishes for others, such as an engine's metrics, carries its time and a publication sequence number, and a reader that caches it keys the cache on both.

8.3 Time and progress

Simulated time is an integer number of microseconds. A step time the cost model prices is converted to microseconds once, by one rounding rule. Two events that rounding makes simultaneous are ordered by the key (§8.2). Arithmetic whose result reaches the order of events or the Result is written so that it gives the same bits on every supported platform (X6).

Simulated time must advance: only finitely many events may happen in any finite stretch of simulated time. Every cycle of components that can pass messages around in zero time contains a component with a positive time advance, or a declared bound on how many times the cycle can repeat at one instant. An engine step that changes no state is an error unless the engine scheduled it as a wake-up, or it is an idle or dummy step the upstream itself runs, such as vLLM's dummy batches on idle data-parallel ranks of a mixture-of-experts model, which the engine declares.

A component changes its own next event only by recomputing its time advance after a transition. The event it no longer wants is marked stale, by a generation number, and dropped when it reaches the front of the queue. It cannot affect the result, wherever it sits in the queue.

8.4 Random numbers

Every random draw comes from a substream named by component, purpose and entity: for example, client 3, prompt length, request 17. The substream's seed is the SHA-256 of the scenario seed and that name, and the generator gives streams for different seeds that can be treated as independent. Its name and version are part of the experimental frame, because a new generator changes every draw. So request 17 of client 3 gets the same prompt length whatever happened to requests 1 to 16, in every configuration.

This is the basis of common random numbers: two runs that differ only in configuration give the same requests the same random inputs. Their outputs are then usually positively correlated, so the difference between them varies less from seed to seed and can be estimated with fewer runs. It does not make the difference free of noise. Some draws depend on how the system responds, for example a retry after a timeout, or a router's tie-break. They use their own substreams, and do not line up between configurations. Comparisons therefore still need replications and intervals (§8.8).

8.5 Input models

Every distribution the workload generator draws from names its family, its parameters, how they were estimated, and the trace they were estimated from, and passes a stated goodness-of-fit test against held trace data. Where a dependence matters, it is modelled, not assumed away: output length against prompt length, the burstiness of arrivals, and the structure of sessions.

A trace records the causes of each arrival, not only its time: which session a request belongs to, and how long the client waited after the previous response. Replaying a trace under a different configuration recomputes the arrivals that depend on earlier completions. A replay of fixed timestamps has no random input, so its figures have no replication interval unless the trace is resampled.

8.6 The experimental frame

A run measures something only relative to its experimental frame. In BLIS the frame is one document, recorded in every Result:

• the generator: the workload and its input models (§8.5), or a trace to replay, with its seeds;
• the initial state: empty caches, or caches warmed by a stated rule;
• the phases: warm-up, measured window and drain, each defined by a rule;
• the stop rule: a request count, a horizon (a simulated time at which the run stops), or both;
• the acceptance conditions under which the run counts for this frame, such as achieved concurrency within a tolerance of the target, and few enough requests unfinished at the horizon;
• the metric definitions (§8.7).

A run that fails its acceptance conditions produces no figure. Running a scenario, exporting its trace and replaying the trace under the same configuration gives an identical Result. Converting another tool's workload into BLIS's preserves, phase by phase, the offered load (requests arriving per second), how variable the gaps between arrivals are, the joint distribution of prompt and output lengths, and the session structure; each converter is checked against its source with two-sample tests.

8.7 What a metric means

Every metric that is reported, compared, ranked or used in a service-level check has one definition, with an identifier, in the metric registry in contracts/ (DEV-8): its formula, its statistic (a mean or a percentile), its population (requests, tokens or steps), its window and its unit. The metric registry includes latency (time to first token, time per output token, inter-token latency, end-to-end latency, scheduling delay and queueing delay), each with its mean and the percentiles a report needs. It also includes throughput (requests, output tokens and total tokens per second; the token rates also per accelerator), interactivity (output tokens per second per request), goodput, and service-level attainment. Time to first token, time per output token and inter-token latency follow the definitions of the benchmark clients BLIS is compared with: time per output token excludes the first token and is averaged per request; inter-token latency is pooled over tokens. Two figures are compared only when their definitions, scopes and aggregates are all the same, and a definition is never inferred from a field's name.

A figure is a metric at a scope: the whole cluster, one pool, or one instance. The scope selects which requests, tokens or steps form the population; the formula, statistic, window and unit are unchanged. A figure that aggregates over several scopes, such as the largest p99 time to first token over instances, names the scope it ranges over and the aggregate. Two figures are compared only when their metric, scope and aggregate are all the same (DEV-8). A report or ranking states the metric, scope and any aggregate of every figure it shows.

A request's fate is how it ended: completed; dropped, with a reason; timed out; in flight at the horizon; queued at the horizon; or not started. It is written once, at one place in the code, and every latency sample is derived from it. A request that emitted no token gives no token-latency sample. The generator keeps its own count of arrivals, and conservation compares that count with the sum of fates, so that a counting error in the fates cannot be hidden by the same error elsewhere.

Requests in flight or queued at the horizon have not finished, so their latency is unknown but at least what they have waited. A latency statistic states how it treats them: excluded, with their count reported beside it, or counted at their waiting time so far as a lower bound. A percentile is not reported when more requests are unfinished than the percentile leaves room for.

8.8 Output analysis

A simulation's outputs are estimates, and are analysed as estimates. A run is either terminating, measuring a fixed amount of work such as a benchmark of a set number of prompts at a set concurrency, or steady-state, measuring a system that has settled.

• A terminating run discards nothing at its start, unless the measured run did. Where the run has random input, its estimate is the mean over several independent replications (runs with different scenario seeds), with a confidence interval from the spread between them. InferenceX benchmarks are terminating runs. Their frame copies the measured run's start state, stop rule and requests (COV-12), so seeds vary only what the requests do not fix.
• A steady-state run discards a warm-up, the stretch at the start where the system is still filling. The warm-up length is found by a stated method, such as MSER-5 (which picks the cut that minimises the variance of the remaining mean), applied to each output series, including the prefix-cache hit rate, which often settles later than queue lengths. Its estimate comes from replications, or from batch means: one long run, after warm-up, split into 10 to 30 equal batches, long enough that neighbouring batch means are not correlated. A percentile gets its interval from replications, each holding enough samples for that percentile, never from batch means.
• A run whose measured window shows a trend has not settled. If the backlog grows, the system is overloaded: its latency figures describe a transient and are not reported, and its departure rate is reported as saturation throughput only if the departure rate itself shows no trend.

Replications start at five and continue until the interval is narrower than a stated fraction of the estimate, or a stated limit is reached, in which case the width reached is reported. A comparison of two configurations is made on the differences between paired replications that share common random numbers (§8.4), and has three outcomes: better, worse, or not distinguished. One configuration is better only when the whole interval of the difference lies beyond a stated margin. Not distinguished includes the case called equivalent, where the whole interval lies within the margin. A search that compares many configurations controls the chance of a false winner across all its comparisons, and re-estimates the winner on fresh seeds, because the best of many noisy estimates is biased upward.

This interval covers the randomness of the workload. The model's own uncertainty is separate: its coefficients have intervals (VAL-10) and its methods have measured errors (VAL-5). A figure states which uncertainty its interval covers. A recommendation is reported with a sensitivity run, in which the coefficients it depends on most are moved across their intervals, and states whether the ranking changes.

A deterministic run is reproducible, not accurate. Where a run has random input, one run gives an estimate but no interval for it. Byte identity shows that a refactor changed nothing; it does not show that a figure is right.

8.9 Verification and validation

BLIS checks two different things. Verification asks whether the code does what the simulation model says (here "model" means BLIS's description of the serving system, not an LLM): the oracle tiers (§7.4), the conservation of requests, determinism and the byte gate. Validation asks whether the model matches the real system: held-out accuracy against measured benchmarks, through the dashboard's scoring harness (§11.7).

A change is a verification change when it must leave the model's behaviour unchanged: a refactor, a speed-up, or a fix that makes the code match an oracle or a written rule. It is a validation change when it changes what is modelled: a pricing form, a coefficient, how links are shared, an input model, a timing rule such as the scrape delay, or the order of classes at one instant. The class follows from what the change does, not from the repository it is in. A change that is both is split. A verification change that moves a published figure, such as a fix that makes the port match its oracle, carries the evidence X13 asks of it.

The model's validity domain is the range of models, chips, layouts, concurrencies, sequence lengths and replica counts its held-out data cover. Every Result states whether its scenario lies inside it. Behaviour that appears only at cluster scale, such as router decisions across many replicas or contention on shared links, is verified against upstream code but not validated until measured cluster runs exist. Where measurements at several sizes exist, the simulated curve across sizes is compared with the measured curve, not only single points. When BLIS models a policy, such as an autoscaler or a saturation detector, its output states whether that policy has been validated, and against what.

8.10 Invariants

• DEV-1 Components interact only through timestamped messages. No component reads another component's state.Checked by: in the router fork and inference-sim, unexported constructors for the router's view types, so only the message-delivery code can build them; an import-graph test that finds no import of engine-internal packages from router-side packages; the coupled replay test (FID-9).
• DEV-2 Each instant is processed in the three phases of §8.2, and messages are ordered by (time, class, sender, sender's send count). No global counter enters the order. Each class's position has a stated reason.Checked by: in inference-sim, a test that permutes the scheduling order of same-instant events from different components and compares the Result; an assertion that no two queued messages share a key; a run of the corpus under a second priority-queue implementation.
• DEV-3 Each engine declares, for each kind of external input, the step boundary at which it takes effect, whether it arrives during a step or at the instant one ends, and whether the upstream's order there is fixed or a race between threads. A declared race is a listed divergence, with the order BLIS uses.Checked by: the oracle, which records where the upstream's loop reads its input queue (each read site is listed in the manifest; where that loop is transcribed, as vLLM's data-parallel busy loop is, the check runs against the transcription, FID-8) and checks that reads happen only at the declared boundaries; a property test in inference-sim that, under the cost model's timing, every external input takes effect at its declared boundary (FID-9).
• DEV-4 No component observes a message before its timestamp. A published value carries its time and a publication sequence number, every cache of it is keyed on both, and the delay between publication and observation follows the deployment's declared scrape and delivery settings.Checked by: in inference-sim, a trace check that every observation is the latest value published at or before its time minus the declared delay; a lint on cache keys of published values.
• DEV-5 Only finitely many events happen in any finite stretch of simulated time. Every zero-time cycle between components has a declared bound. An engine step that changes no state is an error unless the engine scheduled it as a wake-up or declared it as an upstream idle step. Events per request, and wall time per event, stay bounded as runs grow.Checked by: in inference-sim, a watchdog on events per instant across the whole model; a nightly scaling run at 10⁴, 10⁵ and 10⁶ requests that fails on growth in events per request or wall time per event.
• DEV-6 Every random draw comes from a substream named by component, purpose and entity, seeded from the SHA-256 of the scenario seed and the name, by a generator named in the frame. A request gets the same random inputs whatever happened to earlier requests, in every configuration.Checked by: in inference-sim, a test that changes one server setting under a closed-loop session workload and compares every request's prompt and output lengths byte for byte; the lint against unnamed randomness (X6).
• DEV-7 Every run records its experimental frame (§8.6). Run, export and replay under the same configuration give an identical Result. A converted or composed workload keeps, phase by phase, the offered load, the variability of gaps between arrivals, the joint length distribution and the session structure. A run that fails its frame's acceptance conditions produces no figure.Checked by: in inference-sim, a round-trip property test over every workload feature (closed loop, sessions, accumulated context, P/D, deadlines); two-sample tests for each converter against its source.
• DEV-8 Every reported metric names its definition and scope (§8.7), and a comparison is refused unless both sides have the same definition, scope and aggregate.Checked by: the metric registry in contracts/, which the simulator reads directly and the dashboard and config search read through inference-sim metrics; a test that they compare only figures with matching definitions, scopes and aggregates.
• DEV-9 Each request's fate is written once, at one site, and every latency sample is derived from it. The number of requests the generator created equals the number of fates written. A latency statistic states how it treats requests unfinished at the horizon.Checked by: in inference-sim, the fate type, which fails on a second write; the arrival-count check at the end of every byte-gate scenario; a known-bad scenario for each fate (X18).
• DEV-10 A steady-state run is classified as settled, overloaded or incomplete by one detector, and only a settled run gives a latency figure for steady state. The detector states its false-alarm rate and its power for each window length.Checked by: the detector in inference-sim, reached by the applications through inference-sim metrics, and its reference tests against single-server queues whose behaviour is known exactly: open M/M/1 (exponential service times) and M/G/1 (general service times) at loads from 0.5 to 1.5 of capacity, including 0.9 to 0.99 where settling is slow, closed queues for fixed-concurrency benchmarks, and heavy-tailed service, which must be classed incomplete at practical window lengths.
• DEV-11 A figure that compares configurations, or that is published as a property of BLIS, carries an interval from replications or batch means and states which uncertainty that interval covers, or states that the run has no random input and which uncertainty it therefore leaves out. Comparisons are paired under common random numbers, have three outcomes, and control the chance of a false winner across all comparisons made.Checked by: the report schemas of blis-config-search and accuracy-dashboard; a coverage test in which the full procedure, repeated 1,000 times on a model with a known difference, covers that difference at its stated rate.
• DEV-12 Each change is a verification change or a validation change by what it does (§8.9), and is judged as such. A change that is both is split.Checked by: in every repository, the change class in the pull-request template, and a check that the outputs that moved match the declared class. Review guidance. Does this change alter what is modelled? Expertise: expertise in the changed directory. Applies when: a pull request is declared a verification change but moves a Result outside the byte gate.
• DEV-13 A request's sampled output length is written by the workload generator and read only by the trace writer, by replay, and by the engine's stop check, where it acts as an end-of-sequence token at that position. The request's maximum token count stays what the client stated. Every other part of BLIS, including the engine's own scheduler, the router, admission and flow control, sees only what its counterpart in the real system sees: the maximum token count and the tokens emitted so far. The oracle delivers the length the same way, as a forced end-of-sequence in the sampler that the manifest lists.Checked by: a field-access lint in inference-sim and each engine repository that refuses any read of the sampled length outside these declared sites and tests, with a known-bad input (X18).

9Getting a value for every coefficient

9.1 The aim

The cost model needs a coefficient for every primitive on every chip: an attention rate, a GEMM efficiency curve, a collective's latency. Fitted measurements will never exist for every combination. If a missing coefficient caused a refusal, coverage would fall even where a sound estimate exists.

The aim is that every primitive gets a value, and every value records how good it is.

9.2 Sources, in order of preference

For each lookup (primitive, kernel implementation, chip, and where relevant data type and group size) the cost model uses the entry with the smallest rank number:

A kernel whose source did not change needs no transfer: it keeps the same kernel hash, and so the same entry.

Rank 6 needs a FLOP and byte formula for the primitive and at least one measured chip in its family to borrow an efficiency from. Most compute and memory primitives have both. Some do not: host CPU overheads, collective latency floors, and the routing imbalance of expert all-to-all. For those, and for any brand-new primitive family, the substrate MUST carry a rank-7 assumed value for every catalog chip, so that the cost model can always produce a price (VAL-2). Only the trust in that price varies, and the point's grade (§11) reports it.

How to tell measured from calibrated. A value is measured when its data units are microbenchmarks of exactly this kernel implementation on this chip. It is calibrated when its data units are composite, such as end-to-end runs or whole forward passes, from which a factor of a structured formula is inferred.

Every value is stored. Every value of every rank, roofline values included, is a coefficient registry entry written by a writer script. The cost model only looks values up. That keeps every price traceable to a committed entry.

9.3 How the quality of a less trusted value is tracked

• Labelling. Every coefficient records its method, and every price records the methods of the coefficients it used.
• Error measurement. Each method's typical error is measured by leaving one chip out: for chips that have a measured value, compute each less trusted method's estimate as if the measurement did not exist, and compare. That error is published per method and per primitive family.
• Replacement. For each lookup there is at most one entry per rank, and the cost model uses the most trusted one, the entry with the smallest rank number. Removing a more trusted entry (one with a smaller rank number) needs a supersedes: record that cites a defect issue, such as a measurement found to be wrong. That record triggers a review question. Replacing a measured value with a newly measured one is a normal accuracy change, with held-out evidence.
• Backlog. Every value below rank 1 appears in the blocker backlog (§11.6), weighted by how many points it prices.

Extrapolation. A fitted entry records the range of inputs its data covered (VAL-10). A lookup outside that range is extrapolated: it keeps the entry's value, but for VAL-3 it ranks with transferred, so the price and the point's grade show that the value is used where it was never measured.

9.4 Invariants

• VAL-1 Every coefficient records exactly one method from the list in §9.2, or not_charged, with the sources that method needs:

  Method	Required sources
  measured, calibrated	the data units it was fitted to
  transferred	the source coefficient and the transfer rule
  literature	an exact citation
  vendor_spec	a datasheet citation
  roofline	the catalog facts it was computed from
  assumed, not_charged	a written reason

  Checked by: schema rules in blis-substrate/validate.
• VAL-2 For every primitive family and every catalog chip, some rank has a value. A missing measured value never causes a refusal.Checked by: completeness test over every primitive and catalog chip (SUB-7).
• VAL-3 Each price records, among the coefficients it used, the least trusted effective rank, and names its method and any flag. An extrapolated lookup counts as transferred, and an unidentified entry as assumed (VAL-10).Checked by: step-time result field; checked in kernel tests.
• VAL-4 Each lookup resolves to one entry per rank, and the entry with the smallest rank number is used; flags do not change which entry is chosen, only how the price reports it. Where several entries of one rank match a lookup at different scopes, one written order of precedence over every scope dimension (chip, model, layout widths, node) picks one, whatever order the entries were loaded in. Two matching entries that the precedence does not order are refused at load, and a price's provenance names every matching entry it displaced. A more trusted entry is removed only with a supersedes: record citing a defect issue, in a pull request that the reviewer with write access approves.Checked by: validator check against the previous release, plus review guidance for the review agent; a validator rule that refuses unordered matches; a cost-model test that resolves every pair of scope kinds in both load orders and agrees with the validator (X15).
• VAL-5 The typical error of each method is measured by leaving one chip out, and published per primitive family, using data units with role validate (§10).Checked by: nightly job.
• VAL-6 A change to a measured or calibrated value passes the landing gate. The fit and held-out split and the success criteria are committed first. Held-out error and distribution distance must not get worse on any lane, and must improve on the target lane. For each lane the gate computes, on that lane's held units, the paired bootstrap interval of the candidate's error minus the base's error. A lane fails only when that whole interval lies above zero, with the Holm correction for the number of lanes compared. The target lane passes only when its whole interval lies below zero. Every changed measured or calibrated entry is selected by a lookup in at least one operator-level or whole-run gate run. An entry no gate run selects, such as a coefficient for a chip with no held data, is reported as unjudged and lands only with a person's approval. Figures come from a script. A whole-run lane that gets worse does not by itself fail a change whose operator-level held-out check passed: the gate records the regression as a defect in another term (VAL-11), and the change needs a person's approval.Checked by: on fixtures per pull request, full data nightly; operator-level gates in blis-substrate, whole-run gates in accuracy-dashboard's gate job (§13.5).
• VAL-7 A coefficient scoped to a single model marks a cost that the cost model's general forms do not yet explain. It names the missing form. The number of such coefficients only goes down.Checked by: ratchet test.
• VAL-8 A lookup matches a coefficient only for the exact kernel implementation that runs. A value taken from a different implementation is a transferred entry (D5). Every coefficient registry entry is selected by a lookup on the conformance scenarios, or its kernel hash appears in the kernel_impls.json of an engine release that some supported inference-sim release pins.Checked by: the cost model's lookup, which has no fallback to a sibling implementation; a test that lists entries matching neither condition, which are removed in the next substrate release.
• VAL-9 A coefficient states what its measurement includes: device time only or host issue time as well, captured or eager execution, and which harness collected it. The formula that reads it states the same, and the two must agree. A formula that needs both device and host time reads two coefficients, never one that mixes them.Checked by: a validator rule matching each coefficient's measurement boundary with the boundary its reading formula declares.
• VAL-10 A measured or calibrated entry records the interval its fit supports (95%, by bootstrap or profile likelihood) and the range of inputs its fit data covered: shapes, token counts, ranks, chip and kernel. A price that reads an entry outside that range records the lookup as extrapolated, which ranks it with transferred for VAL-3 (§9.3). An entry whose interval is unbounded, or wider than half its value, is flagged unidentified: its data do not determine it, and it ranks with assumed for VAL-3, while keeping its method and its citations (D1).Checked by: validator rules on the interval and range fields; a test that extrapolated lookups report it; the fitting scripts' identifiability check.
• VAL-11 An operator-level check judges a pricing form; an end-to-end check judges how forms combine. A form that passes its operator-level held-out check but makes end-to-end error worse shows that another term is wrong. The disagreement is recorded as one defect, in the other term. The end-to-end figure is not used to reject the form that passed, or to keep a known error. No coefficient or form is kept because it cancels another error.Checked by: review guidance for the review agent; a lint that refuses comments or entries marked as a known over-charge or under-charge without a linked defect. Review guidance. Is this change kept or rejected because it offsets an error elsewhere? Expertise: expertise in the changed directory. Applies when: a pricing change lands or is rejected with end-to-end evidence that contradicts its operator-level evidence.

10The data ledger

10.1 Sources

Three external sources feed BLIS:

• SemiAnalysis InferenceX: end-to-end serving benchmarks for vLLM, SGLang and other engines, plus collective benchmarks (CollectiveX), operator benchmarks (OperatorX), agent traces and speculative acceptance curves.
• NVIDIA AISimulate: operator sweeps per chip, collected under vLLM, SGLang and TRT-LLM.
• NVIDIA's forward-pass dataset on Hugging Face (nvidia/aisimulate-fpm-dataset, Apache-2.0): whole forward-pass timings per model, chip and layout.

Each source is useful for several purposes. The ledger lets each source be used for all of them, while making sure no single data point is used for two purposes that must stay apart.

10.2 Units, identities and roles

A unit is the smallest piece of data the ledger assigns a role to:

• one InferenceX benchmark run of one deployment and recipe fingerprint at one concurrency, with the measurement of each attempt that produced one;
• one point of a CollectiveX or OperatorX run, such as one token count;
• one point of an AISimulate sweep, such as one matrix shape;
• a forward-pass cell.

Its identifier, unit_id, is the SHA-256 of a canonical JSON object holding only the source's own identifying fields. The JSON is UTF-8, with sorted keys and no spaces. Because nothing from BLIS enters the identifier, a BLIS change can never change a unit's identity.

Each unit has exactly one role:

How roles are assigned

A unit's role is computed, not stored. Units that are near-copies of each other must share a role, or one could be fitted while its near-copy judges the fit. So the role is drawn per role key, not per unit. For InferenceX the role key is the source plus model, hardware, precision, framework, benchmark type and configuration hash. It leaves out the recipe fingerprint, concurrency, measured date and benchmark run, so one deployment's whole sweep and its re-measurements share one role, whether or not each record carries a fingerprint. Every field of the role key is an identifying field, so a unit's identity fixes its role key. For CollectiveX, OperatorX and AISimulate it is the run or file, so a whole sweep shares one role. For the forward-pass dataset it is the source, revision, model, chip and layout. Legacy units are the exception: they keep their legacy side (below). The rule is HMAC-SHA-256(seed, role_key) mod 100, compared with cut-points fixed per source in the ledger. For example, one source might send 0–29 to fit, 30–39 to select, 40–49 to validate, 50–79 to held and 80–99 to anchor. For a unit on a legacy side, the same value is compared with two more cut-points fixed per source, one dividing fit from select and one dividing held from anchor. The HMAC's message is the canonical JSON of the role key, and its digest is read as a big-endian unsigned integer. The seed is 32 random bytes from a cryptographic generator. It is a CI secret, and only its SHA-256 is committed; with 256 bits of entropy, the seed cannot be recovered from its hash. Who generated it, when, and who owns its rotation are recorded in the ledger.

What the ledger commits, under catalog/ledger/:

• the sources, each pinned to an exact revision;
• the units, by unit_id, identifying fields and role key;
• the ingest report: for each source revision, the records read, those merged as duplicates, those joined into one unit, and those left out, each with its reason;
• the cut-points and the partition version;
• the legacy table and any alias tables (below);
• the hash of the seed.

Units come from the source, not from the pull request. The unit list is generated by the ingest tool from the pinned source revision, which puts every identifying field in one canonical form. CI regenerates the list and requires it to match byte for byte. Each measurement belongs to exactly one unit. The ingest tool recognises a measurement by the SHA-256 of the canonical JSON of the record's measured values (for InferenceX, its metrics object). A record that reports an error, or whose measured values are empty, is not a measurement; it is left out and counted. The conclusion of the benchmark run, such as a failed or cancelled matrix, is recorded on the unit and does not remove it, because a run that fails in one job still writes real measurements from the others. A source sometimes stores one measurement as several records, as when each attempt of a benchmark run becomes its own row. Records with the same identifying fields and the same measured values are one unit: the ingest tool keeps the record with the smallest source row id (for InferenceX, benchmark_results.id; for a file source, the path and then the line) and lists the others as its duplicates. Because the records agree on every identifying field and every measured value, the choice cannot change the unit's identifier or role. It decides only which record's other fields, such as the image digest and the server log, the unit carries; a field only a duplicate records is taken from the duplicate with the smallest row id that records it. Records with the same identifying fields and different measured values are one unit holding each measurement, and their spread counts as the measurement's own noise (COV-10). The same measured values under two identities with one role key are one unit, under the identity of the record with the smallest row id. Under two identities whose role keys differ, they are left out, because they could let one measurement draw two roles. The ingest report counts every record merged, joined or left out. So a pull request cannot re-add a known measurement under a new unit_id to get a new role draw.

The ledger never commits the role of a select, validate or held unit. At each substrate release, a seed-holding release job writes the three published lists, catalog/ledger/published/fit.yaml, anchor.yaml and report.yaml. These three files are runtime data and are embedded with the rest of the catalog. The rest of the ledger is not.

Who may see which roles

InferenceX data are public. If every role were written down, anyone could work out which units are held out and tune a change against them. So the roles are revealed only as far as each job needs:

For a unit whose role key holds no legacy unit, the published lists do not separate select, validate and held, so a change cannot be tuned against the held units without access to the gate job. Units on a legacy side, legacy or new (below), are different. The old FIT and HELD labels and the rules that assign a side are public, so the fit-side units not on the published fit list are exactly the select units, and the held-side units not on the anchor list are exactly the held units. The gate therefore reports every held unit on a legacy side as a separate figure, marked as not blind, for as long as such units are used.

Pull-request code never sees the seed

A candidate is code the pull request controls. If it ran in a job that holds the seed, it could read the secret, or leak which units are held through its output. So every scoring run has two jobs. A candidate job, with no secret, fetches its modules and the pinned scoring harness, then runs the candidate with no network on every unit not on the published fit, anchor or report lists, and writes one prediction per unit. The seed-holding job runs only scorer code from main, computes roles, and reduces the predictions to aggregates. Each aggregate covers at least 10 units; a smaller lane is merged with its neighbours or not reported. A candidate could still try to learn which units are held, by corrupting its predictions on chosen units and watching the aggregates move. So held-unit aggregates are returned only for a pre-registered change, or by the nightly and pre-release gate runs, which are counted and reported as such; every evaluation is counted (below). Pull-request jobs without a pre-registration may read only the published fit units, through the dashboard's scoring harness (§11.7). Their figures inform review and are never landing evidence. Anchor units are scored only for the published dashboard, so no change can be tuned against the figures it reports.

Each pre-registration commit gets one evaluation on held units. Any further evaluation of the same change is counted, and the count is shown on the pull request and in the release notes, so repeated tuning against held units is visible.

Three properties of the rule

• Append-only. Roles for new units are drawn inside the jobs that hold the seed, the first time a unit appears, and before any job uses it. An existing unit keeps its role unless a new partition version, approved in a pull request by the reviewer with write access, says otherwise.
• Legacy units keep their old side. A partition made before the ledger existed is carried over through ledger/legacy-map.yaml. Each legacy unit keeps its old label, fit side or held side, and the keyed rule chooses only within that side: fit or select, held or anchor. A unit whose legacy records carry both labels takes the fit side, because some of its records may already have been fitted on, and data that was fitted on cannot judge a fit. This happens when the new identity joins records the old partition split, for example by removing the image digest. A new unit whose role key holds legacy units takes their side, and the fit side if they hold both. Legacy units of one deployment can still be on both sides, because the old partition split deployments by concurrency; this is one more reason legacy held units are not blind. A coefficient fitted from legacy records before the ledger existed cites their units. A substrate release refuses such a coefficient while any unit it cites is in select; it is refitted from the fit list first (D1). The ingest tool generates the map from legacy point identifiers to units and commits it with the legacy table, listing each unit that takes a side by these rules. Because the old split is public, legacy held units are not blind (above).
• Stable when identities are recomputed. If InferenceX renames a field, or a change to the ingest tool changes how identifying fields or role keys are computed, the affected unit_ids or role keys change, and units would be redrawn: a held unit could become a fit unit. So such a change must ship an alias table from old to new unit_id and role key, and an aliased unit keeps its role. The alias table is generated, not written: CI runs the merge-base and candidate ingest tools on the same pinned source revision and pairs the identities each gives the same records. A change whose aliases would give one new unit or role key two different roles needs a new partition version, which a person approves. The ingest code detects such a change from the source's contract version (§12.8).

When a partition version is retired, its seed is published. The committed hash then lets anyone check that the split was drawn as claimed.

10.3 Using each source fully

10.4 Invariants

• D1 Every value computed from ledger units cites them, whatever its method. A fitted coefficient cites only units whose role is fit. assumed means no data unit influenced the value.Checked by: a validator rule; the fitting scripts, which write the units they read. Review guidance. Was this value read from, or tuned against, any measurement? Expertise: coefficient registry expertise. Applies when: an assumed entry is added or changed.
• D2 No unit is read under two roles across all consumers.Checked by: a nightly join over every consumer's record of the units it read, run inside the nightly run of blis-substrate's gate job.
• D3 Every source revision is pinned by commit, hash or run id. "latest", "main" and "unknown" are refused.Checked by: a ledger check that refuses latest, main and unknown as revisions.
• D4 Roles are drawn by the keyed rule, per role key, so every unit sharing a role key has the same role, except a legacy unit, which keeps its legacy side even where units of its role key fall on both sides. For a unit in the legacy table, the rule chooses only within its legacy label: FIT gives fit or select, HELD gives held or anchor. A legacy unit whose records carry both labels, and a new unit whose role key holds legacy units of both labels, are on the FIT side; a new unit whose role key holds legacy units of one label takes that label. Roles are append-only, and the ledger stores no role except in the three published lists. A change that alters a source's identifying fields, or how the ingest tool computes them or role keys, ships an alias table, and aliased units keep their roles.Checked by: a check against the ledger at the pull request's merge base; a check that no committed file lists roles, other than the three published lists that the release job writes and the legacy labels in legacy-map.yaml; a check that the committed unit list equals the ingest tool's output for the pinned source; a check that every unit at the merge base keeps its unit_id and role key, or is covered by an alias table or by a new partition version a person approved; a check that each alias table equals the one generated from the merge-base and candidate ingest tools on the same source revision.
• D5 A coefficient's fit data exercised the kernel implementation the coefficient is keyed by, meaning the same library, variant and kernel hash (CodeHash). Otherwise it is a transferred value with a stated rule.Checked by: a validator rule that a coefficient's fit units were collected on the kernel implementation it is keyed by.
• D6 Only redistributable artefacts are committed. CI gates run on redistributable fixtures, and a nightly job checks the fixtures agree with the full data.Checked by: a check that every committed data file is listed in the ledger as redistributable; a nightly job comparing fixtures with the full data.
• D7 The role seed is 256 random bits and a CI secret until its partition version is retired. Only the fit, anchor and report lists are published, by the release job. Results on select, validate and held units come only from the select job, the error job and the gate job, as aggregates over at least 10 units. No job that holds the seed runs code from a pull request. A candidate job runs with no CI environment and a read-only token, and its only output is a predictions file with a fixed schema and size limit, which the seed-holding job reads with a strict loader (SUB-12) as untrusted input. A seed-holding job is started only by a workflow on its own repository's default branch, never by a token a pull request controls.Checked by: the seed is stored in one CI environment per repository (blis-substrate and accuracy-dashboard) whose branch policy allows only main, available only to the release, select, error and gate jobs; a workflow lint that those jobs check out only main; their outputs are checked to contain no per-unit rows and no aggregate over fewer than 10 units. Review guidance. Was the new seed generated as the ledger records? Expertise: ledger expertise. Applies when: a seed is created or rotated.
• D8 A quantity BLIS simulates, such as prefix-cache hits or queueing, is never stored. A quantity that depends on the request stream but that BLIS does not simulate, such as speculative acceptance or expert routing balance, is a property of the workload. It is keyed by workload identity and never stored as a coefficient scoped only to a chip or model. Its value is taken from the command line if stated there, else from the scenario, else derived from the trace, else taken from the substrate's profile for that workload identity, and the Result's inputs record which supplied it (SIM-10). Speculative acceptance is keyed by the target model, the drafter, the speculative method, the number of speculative tokens K, and the workload's category and thinking mode. The drafter is the model's own in-graph speculator, such as its MTP layers, or a separate drafter checkpoint with its own catalog entry. A value stated on the command line, in the scenario or by the workload is used as stated. A profile lookup, in the substrate's profiles or in a profile set the scenario names, may borrow when no measured profile has the run's exact key. It takes the closest measured one, in this order. First, the same target and drafter at another K. Between measured values of K it interpolates linearly. Above the largest it takes the largest's value. Below the smallest it takes that profile's stated per-position rates for the first K positions. A profile that states none gives 1 + K(AL − 1)/K₀, where K₀ is its K. That figure cannot exceed what the profile's own rates would give, so it does not overstate drafting's gain. The lookup never extrapolates. Second, another drafter of the same target, preferring the same method and then the lowest value at the run's K, found by the first step's rules, so that drafting's gain is not overstated. Third, the same method on another target whose graph records the same architecture. Every step keeps the category and thinking mode, apart from a fallback category the user requested. Two profiles that share a key but record different sampling or rejection settings are not interchangeable, and an exact key found in two profile sets is ambiguous; the lookup refuses both. Any remaining tie breaks in an order the substrate's acceptance reference states, so the choice is deterministic. With no profile on any step, the run is refused. The Result records the profile used, the step, and each key field that differed. A scenario or command-line option limits the lookup to exact matches.Checked by: a validator rule refusing such families in the coefficient registry without a workload key; a test per borrowing step, for a tie, for each bound of K with and without stated per-position rates, for an ambiguous key, for the exact-only option and for the refusal.

11Accuracy coverage

When inference-sim releases, the accuracy dashboard scores the new release at its next nightly run, with no change to its own code. The dashboard scores every anchor unit, and grades how close each comparison is. InferenceX often runs older, nightly or custom engine builds, so a perfect comparison is not always possible.

11.1 Why settings must not be classified by the dashboard

A dashboard that keeps its own table of which engine settings are modelled will disagree with the engine. The engine changes, and the table does not. The release plan's background shows what that cost. So the dashboard never classifies a setting. The engine repository does, in its field list (VLM-3), and the dashboard reads that list.

11.2 Four steps for scoring a point

1. State. The dashboard copies the run's recorded engine configuration and environment variables, unchanged, into the scenario's engine block. A run records its configuration in several places: the launch configuration, the arguments the engine logs as passed, the configuration it logs as resolved, and lines it prints about values it chose. The first two record the values passed; the last two record the values resolved. The dashboard reads every one, and a value any of them records is recorded by the run, so it is copied rather than estimated (X19). The engine block holds the values the run passed; a field the run records only as resolved is copied as resolved. This applies to a field the engine models or proves inert; for any other field the resolved value is the engine's own choice, not a setting the run made, so it is recorded on the point and not stated in the scenario, and the field's estimation rule applies only to a value the run passed. A resolved value that differs from a passed one is not a conflict, because the engine resolves its settings: the engine repository must reproduce that resolution. Where the run logs a resolved value that differs from the engine repository's resolution of the passed value, the point is G2, names the field, and the difference goes to the backlog as a fidelity defect. Two sources that record the same stage, such as the launch configuration and the logged passed arguments, with different values are a conflict, and the point is G5 with the reason value_conflict. The dashboard translates nothing, apart from the estimates of the next step. The one exception is engine.version: it is set to the engine release that the scoring inference-sim release runs, and the run's own release is recorded on the point (§11.5). A recorded field that the modelled release does not have would be refused (X1), so the point is G5 with the reason field_not_in_release, naming the field. A workload-dependent value the run records, such as the mean acceptance length its engine logs, is copied into the scenario as stated (D8). One it does not record is taken from the substrate's profiles, and is not an estimate under the next step. With no profile either, the point is G5 with the reason value_not_recorded.
2. Estimate. When the run did not record a value the simulation needs, such as the maximum model length, or used a setting BLIS does not model, such as an unsupported quantisation format, the dashboard applies the engine's estimation rule for that field. Each engine repository writes these rules in its fields.json, beside each field's status: one of: leave the field unset, so the engine resolves the upstream's default (X19); a stated setting that stands in for an unmodelled one, with the reason; or none. An estimation rule may state when the stand-in is exact: preserves, a sentence naming what the upstream resolves identically, and valid_when, a list of clauses; the stand-in is exact for a recorded value when at least one clause holds, and a clause holds when every key in it holds. A value for which no clause holds is not stood in for: the point is G5 with the reason field_not_modelled. Absent valid_when, the rule applies to every value. The dashboard holds no rules of its own. A rule applies in two cases: the run did not record the value, or the run passed a setting the engine does not model. A rule never replaces the recorded value of a field the engine models or proves inert: that value is carried into the scenario as stated. For a setting the engine does not model, the rule's stand-in, or leaving the field unset, takes the place of the recorded value in the scenario, and the point names the field, the recorded value and what stood in for it. A not-modelled field that the engine reads and prices when it is stated is a status to correct, either to modelled with its evidence or to refused when stated (VLM-3); the rule does not stand in for that. Every estimate is named on the point and makes it G2; a field with no estimation rule makes the point G5, with the reason value_not_recorded for a missing value or field_not_modelled for an unmodelled setting.
3. Decide. For each field, the engine's own fields.json says whether it is modelled, proven inert or not modelled. The dashboard keeps no table of its own.
4. Price. The engine lowers the configuration to kernel implementations. For each implementation the cost model uses the most trusted available entry, the one with the smallest rank number (§9.2), and the point records which methods were used.

A gap at Estimate is a missing estimation rule, and a gap at Decide is porting work, both in the engine repository. A gap at Price is a value below rank 1. All three go to the backlog.

11.3 Closeness grades

Every point gets exactly one grade: the highest-numbered (least close) grade whose conditions apply. Grades are computed from data alone, so a condition the data cannot decide is never assumed to hold: a G1 is published only for a point whose every condition was evaluated.

The headline accuracy covers every graded point, G1 to G4, because exact matches are rare: most benchmark runs use an engine release other than the one BLIS models, or a nightly or custom build. Beside the headline, the dashboard publishes the share of points in each grade and the accuracy of each grade on its own, so a reader can see how much of the headline rests on close comparisons. Every published figure shows the number of points it covers, and a figure over fewer points than the minimum stated in the dashboard's summary schema is shown as "too few points", not as a number.

11.4 Reasons a point is not scored (G5)

• code_not_visible: the run loaded code BLIS does not have. That is a class, module or plugin that a configuration field or plugin entry point names and the pinned upstream release does not ship, such as an out-of-tree connector, scheduler, worker or logits processor, or a patched private build. This reason takes precedence over field_not_modelled.
• router_unsupported: the run was measured behind a router BLIS does not host, and the router table has no entry for it or says that no llm-d policy is close enough. BLIS hosts only llm-d's router (X4) and never simulates another, so such a point stays in the coverage counts as not scored. The router table (catalog/routers in blis-substrate) lists each router BLIS does not host, by the mode and session-affinity setting a run used, with the llm-d policy it maps to, or an explicit statement that no llm-d policy is close enough, and the evidence for the match or the mismatch: a citation of both routers' code or documentation at the revisions the runs used. An entry maps a router to an llm-d policy only with that evidence; a router and mode with no such entry stays router_unsupported. A point behind a router the table maps is G2 (§11.3), never G1, and it is never presented as if llm-d had routed the run. Hosting another router would add an upstream to this design.
• value_not_recorded: the result depends on a value the run did not record. It becomes G2 when an estimation rule in the engine's fields.json covers the value (§11.2).
• field_not_modelled: the run sets a field the engine classifies as not modelled, and the field has no estimation rule, or its rule does not cover the recorded value. It clears when the field is modelled or gains a rule.
• field_unclassified: the engine has not classified a field the run sets, or its repository publishes no field list yet. It is a blocker-backlog entry, cleared when an engine release classifies the field or publishes the list.
• engine_unsupported: no BLIS engine repository exists for the run's engine, as with TRT-LLM.
• topology_unsupported: the run was measured on a deployment the scoring harness cannot write as a scenario: several engines, as in a prefill and decode split or a run across nodes, where it builds one colocated pool. It is a blocker-backlog entry, cleared when the harness can write that topology.
• not_in_catalog: the run's model or chip has no catalog entry yet. This is a model-onboarding trigger (§12.1), and the backlog counts it.
• field_not_in_release: the run sets a configuration field that the engine release inference-sim runs does not have. It clears when inference-sim moves to a release that has the field.
• value_conflict: two sources that record the same stage of one field, such as the launch configuration and the logged passed arguments, give different values (§11.2). This is a data issue, reported to the source.
• build_unparseable: the run's recorded engine build cannot be read. This is a data issue, reported to the source.

11.5 One simulator release scores every point

Each inference-sim release runs one upstream release of each engine (LIF-6). The dashboard scores all points with one inference-sim release at a time (COV-9). A point measured on that engine release can be G1 or G2. A point measured on another release is scored on the release inference-sim runs, and graded G3 or G4 by how much of the ported code differs (§11.3). So every point is priced by the same substrate and engine code, and a substrate improvement reaches every point without a backport.

The previous inference-sim release's scores stay published with it, so a reader can compare a point on its own engine release across BLIS releases.

11.6 The blocker backlog

Every night the dashboard publishes a ranked list, per engine, of everything that lowers coverage or grade:

• refused surface items;
• not-modelled fields;
• coefficients below rank 1;
• unpriced execution modes.

Each entry shows how many points it blocks or demotes. Engine and substrate work is chosen from the top of this list, and each entry maps to one playbook (§12).

11.7 The scoring harness, and who may run it

The dashboard's scoring harness is the one place where a simulation is compared with a measured end-to-end benchmark (§2.2). It takes three inputs: a simulator build (a released tag, or a commit of a branch or pull request), the catalog and coefficient registry it should load (embedded, or paths), and a set of ledger units. It turns each unit's recorded run into a scenario as §11.2 describes, runs the simulator, and compares. It writes the same per-point records and aggregates, whoever runs it.

The harness is released from accuracy-dashboard tags and named by its release and commit. It is run as a script on a runner that builds the simulator, not as a container image. A repository whose pull requests run it pins that release or commit in one pin file, as it pins any dependency, and the release bot moves the pin. Every figure records the harness commit that produced it (X8).

Anyone may run it, at any stage of a change: on a feature branch, on a draft pull request, or on a pull request under review. What changes with the stage is which units it may read, because only seed-holding jobs may see select, validate and held units (D7):

A figure computed on a branch is labelled with its build and its data, as every Result is (X8), and is labelled experimental unless both are released. So it can be posted on a pull request, but never published as BLIS's accuracy.

11.8 Invariants

• COV-1 One scorer produces every number, and it lives only in accuracy-dashboard: no other repository compares a simulation with a measured end-to-end benchmark. Every anchor unit is either graded G1 to G4 or listed as G5 with a reason, and the counts sum to the total.Checked by: summary tests in the dashboard; a lint that refuses code outside accuracy-dashboard that reads end-to-end benchmark units to compute an error.
• COV-2 Only ledger anchor units are scored, using the simulator release's embedded data and the anchor list embedded in its substrate release (inference-sim ledger).Checked by: a scorer check that every scored point is an anchor unit from the release's embedded anchor list.
• COV-3 The dashboard copies recorded configuration verbatim, and reads field status and estimation rules only from the engine's fields.json.Checked by: a check that the dashboard holds no settings table; a check that every scenario's engine block equals, field for field, the values the run passed, parsed with the engine's own argument parser, or, for a field the engine models or proves inert, the resolved value where only that is recorded (§11.2). The exceptions are engine.version, which equals the release in stack.json, and each field an estimation rule set, which the point names.
• COV-4 Every point has one grade, computed from data, with every substitution named.Checked by: a scorer check that every point has exactly one grade and every substitution is named.
• COV-5 G5 is used only for the reasons in §11.4. Any other reason fails CI.Checked by: a scorer check that every G5 reason is one of the listed reasons.
• COV-6 Every point in a release's scores is scored by that one inference-sim release. Each point shows its recorded engine build, and is graded G3 or G4 when that build is not the engine release inference-sim runs.Checked by: a scorer check that one simulator release produced every score, and that each point's grade agrees with its recorded build and stack.json.
• COV-7 Coverage (graded anchor units over all anchor units, by engine, engine release, chip and grade, with the share in each grade) is published beside accuracy, with the number of source records the ledger left out (§10.2), and the backlog is published every night.Checked by: the nightly publish job, which fails if coverage or the backlog is missing.
• COV-8 A failed or partial night publishes nothing; published snapshots never change.Checked by: publish guards in api-nightly.yml.
• COV-9 Each simulator release is scored exactly once. The nightly run scores, in the order they were tagged, every release that has no scores. Every provenance field is recomputed at every publish.Checked by: the nightly scoring job; a test that each published pull date is no earlier than the newest measurement it covers.
• COV-10 The dashboard copies the run's whole experimental frame (§8.6), not only its engine block: client concurrency, warm-up, drain, duration and the measured-window rule. The requests are the ones the run sent (COV-12), so BLIS's own substreams (DEV-6) draw only what the run's requests do not fix, such as a router's tie-breaks, and replications (DEV-11) cover those draws. A frame field the run did not record makes the point G5 with the reason value_not_recorded (§11.4); no default is substituted. The achieved concurrency in the simulation matches the measured one within a stated tolerance. Each lane is scored on its percentiles and a distribution distance, not only on its mean. A simulated figure compared with a measurement carries its replication interval, or states that the run has no random input (§8.4); its uncertainty is then the measurement's own noise (below) and the model's (§8.8). Where InferenceX has several units whose identifying fields differ only in measured date or benchmark run, and whose recorded image digests agree, or one such unit holding several measurements (§10.2), their spread is reported as the measurement's own noise, and no point's error is counted for a difference smaller than that noise. Units whose image digests differ or are unrecorded are not pooled for noise, because their builds may differ. A unit holding several measurements is scored against their mean.Checked by: a check that every scored scenario's frame equals the recorded one; a concurrency check per point; the summary schema.
• COV-11 The headline accuracy covers every graded point, G1 to G4. Beside it, the dashboard publishes the share of points in each grade and each grade's own accuracy. Every published figure shows its point count, and a figure over fewer points than the minimum stated in the dashboard's summary schema is shown as "too few points".Checked by: the summary schema, which requires a count with every figure; a publish check that refuses a figure under the minimum count.
• COV-12 The scored simulation receives the requests the measured run sent: the same requests, in the same order, with the same prompt lengths, output lengths and shared prefixes. Their arrival times follow the simulated completions by the client's own rules (§8.5). Where the client draws its requests from a seeded generator, the dashboard recomputes them with the client's own algorithm and seed, at the client release the run records; where the run records none, at the release its image ships, and the point names it. Prompt lengths are counted after the client's tokenisation and verified against the run's tokenizer at a pinned revision. An output length is the length the client forced or, where it forced none, the length the run's response records (DEV-13); where neither is recorded, the length is drawn from the frame's input model, and the point names it. The simulator runs the client's warm-up, so the measured phase starts from the engine state the warm-up left, such as cached prefixes. Every difference between the run's client and the simulated one, including an unrecorded client release or an unverified length, is named on the point and makes it G2.Checked by: a test that recomputes every scored point's requests with an independent implementation of the client's generator and compares them with the replayed trace; a test per client release that its pinned source still draws as the dashboard recomputes; a check that every scored point's trace has the run's request count and order; a scorer check that every point naming a client difference is G2; a test that the replayed warm-up leaves the recorded prefix structure.

12Triggers

A trigger is an outside event that requires work. Each trigger has one playbook, with the same fields every time:

• how the trigger is detected;
• a preflight that computes the scope before any work starts;
• the steps;
• the checks that must pass;
• what done means;
• the expected size of the work.

Figure 5 shows which repositories each trigger touches.

13Automation: how the rules are kept

Rules are kept by three kinds of worker, used in this order of preference:

1. Deterministic checks decide everything that can be decided mechanically. Given the same commit, they give the same answer.
2. Agents do bounded work: porting a changed function, classifying a new field, fitting a coefficient. They work from a preflight, against the checks, inside a budget.
3. People answer the yes/no questions in §4 and §5, approve releases, and set direction.

13.1 The invariant registry

Every invariant in this document is a record in blis-substrate/contracts/invariants/:

- id: SUB-6
  statement: The cost model resolves nothing; every default, backend choice,
             padding and capture decision arrives in the plan or step shape.
  scope: [blis-substrate/kernel]
  checks:
    - repo: blis-substrate
      test: TestKernelResolvesNothing
      runs: pull_request
  review: null                    # or {question, reviewer, trigger}
  status: new                     # ci | partial | new | review
  target: {status: ci, milestone: 4}   # a milestone of a release plan

The status partial means some of an invariant's checks run in CI and others are still new. The status review means the invariant is enforced only by its review question, with no mechanical check; X1 to X9 keep a review question beside their checks, and no invariant in this document is review alone. Each check has its own status, so the invariant registry shows which parts are enforced.

An invariant registry check's runs value is pull_request, nightly or release for an automated check, or practice for a maintainer practice (§4.2). A practice record names the repository it concerns and its checklist item, and has no test: no program in another repository reads it. A practice is confirmed by a person ticking its item on the repository’s maintainer checklist; it has no check run to pass, so the release bot does not wait on it.

A generator turns these records into this document's appendix, into a section of every repository's AGENTS.md that carries the agent guidance, into the checklist of every pull-request template, and into a maintainer checklist for each repository that lists its practices. A rendered copy that does not match its record fails CI.

What a repository’s AGENTS.md holds. It is the first file an agent reads, so it is short and complete. It holds, in this order: what the repository is and owns, and what it must never hold (§5); the rules that apply to a change here, rendered from the invariant registry; the commands to build, test and run every check, exactly as CI runs them; what a change in this repository must also update (the documentation, the generated copy, the invariant registry record, the plan’s issue); and where to look next. It states no version, count or list of supported features, and leaves the concepts to the design, which a person reads. It does not restate a rule that CI enforces and an agent cannot get wrong; it states the rules a check cannot decide. A repository adds to it only through the invariant registry or a pull request that the reviewer with write access approves.

13.2 On every pull request

1. The template lists the invariant identifiers the change claims to keep or introduce. CI fails if any identifier does not exist. The invariant registry lives at contracts/invariants/, and every Go repository pins the substrate, so each reads the invariant registry version it pins; blis-config-search and accuracy-dashboard, which depend only on inference-sim, read it through inference-sim invariants at the release they pin. The shared workflow lives in blis-substrate/.github/workflows, and each repository calls it by commit (X21).
2. For each cited identifier whose status is ci, CI confirms its checks passed. A check in this repository must pass on this pull request. A check in another repository must have passed at the version this pull request pins.
3. The review agent reads the changed paths and the diff against every review question’s trigger. A few triggers depend on what changed, not where, such as “a floating-point reduction is added”; the agent applies those by reading the diff.
4. For each question that applies, the review agent answers it in a review comment with yes or no, a reason, and the files and lines that support the answer. A question is guidance for that review and never a check: no CI job fails because a question is unanswered or answered in the unwanted direction, and no person has to answer or tick one. An answer in the unwanted direction is a finding like any other (§13.7): the reviewer with write access reads it with the rest of the review, and branch protection blocks the merge until that reviewer approves and every conversation is resolved.

13.3 On every release

1. The release bot refuses to tag unless every ci invariant the repository owns passed at the commit being tagged.
2. The release notes are generated, never typed. They list the price-change report (each benchmark scenario's step prices before and after the change), changed coefficient values, changed traces, contract differences, invariants added or changed, review answers, and the exact upstream versions.
3. Once the milestone named in an invariant's target is reached, the release bot refuses to tag until that invariant has its target status.

13.4 Every night

• The nightly compatibility build builds every repository at the tag the newest inference-sim release pins, and at the newest tag on each open release branch, with the substrate raised to its newest tag (skipping a release branch whose contracts the newest substrate no longer fits), on three platforms. It runs each repository's tests and checks the module graph against Figure 1.
• Full-data gates rerun coefficient registry writers, the landing gates and fixture-agreement checks on the full data, which CI cannot hold.
• Detectors poll upstream releases and data sources, and open trigger pull requests.
• The dashboard scores, publishes coverage and grades, and updates the backlog.
• A status page shows every invariant by repository with its latest result.

13.5 Where accuracy gates run

An accuracy gate compares a candidate with held-out data (VAL-6). Where it runs depends on what it needs:

• Operator-level gates need only the cost model and the coefficient registry, for example "does this attention coefficient predict the held-out sweep points?" They run in blis-substrate's gate job, which holds the role seed in that repository's protected environment (D7).
• Whole-run gates need engines and the simulator, for example "does this change improve end-to-end latency on held-out InferenceX deployments?" They run the dashboard's scoring harness (§11.7), the only end-to-end scorer, at the release or commit the repository pins. A candidate job in the pull request's own repository builds the candidate and runs the harness with no secret and a read-only token, on every unit not on the published fit, anchor or report lists, and writes one prediction per unit. accuracy-dashboard's gate job holds the role seed (D7), runs only its own main, refuses predictions made by a harness commit other than one released from its main, reads them as untrusted input, reduces them to aggregates and reports the result on the pull request.

The gate compares two trees: the base, which is the pull request's merge base, and the candidate, which is its head. Comparing against the merge base, rather than against the last release, keeps unrelated changes already on main out of the comparison. Each tree's catalog and coefficient registry are passed with --catalog and --registry.

• If the pull request changes only files under catalog/ and registry/, and both trees load under the released simulator's validators, and the code outside those directories at the merge base is identical to the substrate release that simulator embeds, the gate runs the released simulator twice, once per tree. No rebuild is needed. If main carries unreleased code changes, the next case applies, so the data are judged with the code they will ship with.
• Otherwise, for example when the pull request changes the cost model or an engine, or uses a data field an older simulator cannot read, stack CI (§13.6) builds one simulator from the base and one from the candidate, and each runs on its own tree.

Before a substrate release is tagged, the release bot triggers the whole-run gate job once more, comparing the candidate (its code and its data) with the previous release (its code and its data) on held units. A lane that gets worse by the test of VAL-6 blocks the tag, unless it is recorded as a defect in another term and approved (VAL-11). That final check covers changes that were each fine alone but interact. In every case the substrate's CI imports no repository above it (X7). A substrate pull request's candidate job builds inference-sim through stack CI and runs the pinned harness. Those are the two uses of higher repositories X7 allows, and the candidate job imports neither.

The gate job runs on pull requests that carry a pre-registration, and again nightly on the full data. Any pull request, draft or not, may also run the harness on published units for review (§11.7); that run holds no seed and decides nothing.

13.6 The release bot

A review set spanning repositories is described by one file, stack.lock, which lists each repository's branch head. During review, a "stack CI" job builds and tests the heads together, through a go.work file it generates and never commits (X11). Each repository keeps its released pins until the set is approved. Then the bot:

1. merges and tags each repository bottom-up;
2. writes each pin once;
3. runs the next layer's checks against the new tags.

No person or agent re-pins by hand, and no agent waits on CI.

The bot's code lives in blis-substrate/tools/release, and its tags are signed. If the bot is unavailable, a documented manual procedure, run by a person with write access, produces the same tags in the same order.

13.7 What agents may and may not do

Most of the prohibitions on the right are enforced by CI or by permissions, as listed below. Two depend on the agent's instructions and on review: staying within the preflight's list, and not loading generated files into context.

• Every check declares the paths it consists of: its test files, thresholds, fixtures and workflow file. Those paths, and .github/, are protected. A change to any of them is a trigger for the review agent’s guidance.
• A CI step fails on any new test skip, lint suppression or loosened threshold, unless the pull-request description carries a Waiver: line that names the file and the reason. The line is a fact the step reads; the review agent judges whether the reason holds, and the reviewer with write access approves the pull request that carries it.
• A change to a success criterion, or to the ledger’s cut-points or partition version, is a trigger for the review agent’s guidance. Held-out data is never read before the criterion is committed (§9, §10).
• Trace and generated-file guards are CI checks.
• Agent credentials have no permission to merge or tag.

13.8 Keeping agent work cheap

• The preflight sets the scope. An agent works only on the preflight's list, within its budget. Work outside the list is a new trigger.
• Tools do mechanical work. Re-pinning, re-vendoring, regenerating traces, diffing releases and updating descriptions are commands.
• Agents do not wait. When CI or review is pending, the session ends and resumes on an event.
• Reviews by agents are narrow. An audit agent receives one diff and one trigger's done-when list.

14Versions, releases and blis-sglang

14.1 Version schemes

What an engine tag guarantees

An engine release, like any software, can ship with a bug or a gap that is found later. That is expected, and it is why N exists: the fix is a new -blis.N on the same upstream release, released and pinned the same way as any other.

A tag vX.Y.Z-blis.N of blis-vllm or blis-sglang means:

• It models upstream release vX.Y.Z, at the commit its Upstream record names, and no other release.
• It passed that release's oracle tiers, its field list and surface list checks, and every ci invariant (LIF-3).
• A higher N on the same upstream release is a BLIS-side change only: a bug fix, a newly modelled feature or plugin, a contract or substrate update. It may change prices or decisions. Its release notes say what kind of change it is and whether prices or decisions changed, and include the price-change report (§13.3).
• Every file is either ported from vX.Y.Z or original to BLIS. Nothing is kept for an older release.

The module's version number is the upstream's version number, so the module cannot use it to signal a breaking change to its Go API. That costs nothing here, because only inference-sim imports an engine module, and it pins the exact tag (LIF-2). If an upstream reaches v2, the module path gains /v2, as Go requires.

Upstream versions that are not Go versions

• Release candidates (vLLM's v0.33.0rc0, SGLang's v0.5.3rc0) are not modelled. An engine module models final releases only.
• Post releases, of vLLM or SGLang (for example SGLang's v0.5.6.post1 or vLLM's v0.6.1.post1), are written v0.5.6-post.1.blis.N. The dot after post makes Go compare the number numerically, so post.2 sorts before post.10. Every -post.K.blis.N sorts after every v0.5.6-blis.N and before v0.5.7-blis.1, which is the upstream order. These orderings are Go's own (golang.org/x/mod/semver).
• Four-part versions, such as vLLM's v0.10.1.1, are written v0.10.1-rev.1.blis.N. Go compares these words as text, and both post and rev sort after blis, so a post release or a four-part release sorts after every -blis.N tag of its three-part base and before the next three-part version. If an upstream ever publishes both on one base, the rev form is used for the four-part release.

14.2 Lifecycle invariants

• LIF-1 A tag names one commit forever. Tags are never moved or deleted; a mistake gets a new tag.Checked by: a nightly check, in the repository, that every tag resolves to its recorded commit. Maintainer practice (§4.2): the repository protects its release tags from update and deletion.
• LIF-2 In every build, each BLIS module resolves to the exact version the release records in stack.json, and the router fork resolves to a -blis.N tag through inference-sim's one replace directive (X11). inference-sim's own substrate requirement equals the version its build selects, so no engine or router release can change the simulator's data. One Go toolchain version is named in contracts/toolchain, at the substrate version inference-sim selects. No module BLIS writes declares a go version above it. Every CI job sets GOTOOLCHAIN=local and installs exactly that version, so no build downloads a toolchain. The router fork's go line follows upstream llm-d; when upstream raises it, the release bot raises the toolchain file first, in the same review set. Documented version strings are generated (X23).Checked by: selected-version lint (go list -m all), which for the router reads the replace target that go list -m -json reports, not the placeholder version on the require line; a check that every module in go list -m -json all declares a GoVersion no higher than contracts/toolchain; a CI step that fails if go version differs from it; the pre-release test of every engine and router package against the simulator's selected versions (§3.2).
• LIF-3 Only the release bot cuts releases, and only from a commit where every ci invariant passed. If the bot is unavailable, a person with write access to the repository may follow the documented manual procedure (§13.6); a manual tag is an annotated tag on the checked commit, recorded in the release notes.Checked by: the bot’s own check of every ci invariant; a nightly check, in the repository, that every tag made by a person is annotated and listed in its release notes. Maintainer practice (§4.2): the repository’s settings let only the release bot and people with write access create tags.
• LIF-4 Release notes are generated (§13.3).Checked by: the release bot, which writes the notes from the release's changes.
• LIF-5 An engine module, or the router fork, models one upstream release. Its tags are v<upstream>-blis.N, v<upstream>-post.K.blis.N for a post release, or v<upstream>-rev.W.blis.N for a four-part version, with N from 1. No new plain upstream tag is made; older tags stay (LIF-1). In an engine module, the Upstream record names that release and commit, every ported file names that same release, and no code for another release remains. Release candidates are not modelled. The package that runs the modelled release depends on no package that exists only for another release.Checked by: a test that lists the dependencies of the engine's entry package with go list -deps and fails on any package kept for another release; a tag check in the release bot (format, N one above the previous tag on the same upstream release, no plain tag); in engine modules, the provenance test, which refuses a ported file that names another release.
• LIF-6 Each inference-sim release runs exactly one upstream release of each engine and of llm-d, and lists them in stack.json. A scenario whose engine.version names a different upstream release is refused, with a message naming the release it asked for, the one this build runs, and the newest inference-sim release that runs the one it asked for, or says that none does. The llm-d release is not stated by scenarios; it is the one stack.json lists.Checked by: a refusal test per engine kind; a check that the message's release names match stack.json history.
• LIF-7 A deprecated feature is removed by the first release after its deprecation that may break compatibility: a MAJOR release, or a MINOR release while the repository is at v0. In an engine module, which has no such releases of its own, deprecated code is removed when main moves to a new upstream release. Removal takes with it every code path, flag, field, test and document that existed only for that feature. Each deprecation is recorded where the feature is defined (a command's or flag's deprecation notice, or a // Deprecated: comment) with the release that deprecated it. Nothing in this design depends on a deprecated feature, and no repository keeps code that none of the binaries BLIS ships can reach.Checked by: the release bot, which refuses a MAJOR release, or a MINOR release at v0, that still contains a feature deprecated in an earlier release; a whole-program dead-code check (Go's deadcode), run in stack CI's workspace from the main packages of every binary BLIS ships, with test files not counted as reaching code. Upstream files in the router fork, and the interfaces in contracts/, are exempt.

14.3 blis-sglang

blis-sglang does for SGLang what blis-vllm does for vLLM. It models one SGLang release, pinned by tag and commit. The pin must be exact, because SGLang's main moves fast and often rewrites core components, such as the allocator, between releases. Its first release and stages are in the release plan.

Layout

It does not port SGLang's Rust router (sgl-model-gateway). In the llm-d stack, llm-d is the router, and llm-d already has an SGLang metrics mapping, KV-event adapter, request parser and P/D connector.

Behaviour the port must reproduce

14.4 Release branches

Each upstream-tracking repository has a main branch (llm-d-r in the router fork) that models the newest upstream release BLIS supports. When BLIS moves to a new upstream release, two branches are cut:

• in the upstream-tracking repository, release-<upstream>, for example release-0.32.0, from the last commit that modelled the old release;
• in inference-sim, release-vX.Y, from its last release that ran the old upstream release.

A fix needed on the old release is made first on the main branch if the main branch needs it too, then cherry-picked to the release branch; otherwise it is made on the release branch alone. The release branch then gets the next tag for its upstream release, for example v0.32.0-blis.3 after v0.32.0-blis.2, and the fix is shipped by a PATCH release from inference-sim's release branch. A tag on a release branch alone reaches no user, because every inference-sim release pins exact tags.

Only the inference-sim release before the newest one is supported this way. Both release branches close when inference-sim next moves an engine or llm-d to a new upstream release.

AInvariant index

BGlossary