Skip to content

blis-schemas

BLIS, the Blackbox Inference Simulator, estimates how an LLM serving deployment will perform before anyone runs it. blis-schemas defines the documents BLIS reasons about and checks them. It is a Go module: types for a model, its hardware, the coefficients a cost model uses, a deployment, its traffic and a measured run; a validator for each; and the interface a cost model implements. It holds no data.

The idea in brief

A document's shape is defined here, and its contents live in the repository that owns them, so either side can change without the other. A scenario fixes the problem: the model, the hardware, the traffic, and the engine version. A deployment states the choices made against it: parallelism, engine settings, offload. Validation asks two questions and keeps the answers apart. Is each document well formed? That never depends on an engine release. Would the release the scenario names run it as written? That always does. Every validator reports every problem at once, each naming the field that is wrong.

The BLIS repositories and how they depend on blis-schemas blis-catalog and blis-registry hold data and run blis-schemas validators in CI. blis-latency-kernel imports blis-schemas and reads the catalog and registry files when a kernel is constructed. inference-sim, in pull request 1851, imports the kernel and blis-schemas. data repositories · YAML and JSON blis-catalog declared facts: models, chips, fabrics, storage, workloads blis-registry coefficient sets: measured, cited or assumed values blis-schemas document types, validators, the Kernel interface blis-latency-kernel implements Kernel: prices memory and one forward pass inference-sim discrete-event simulator; uses the kernel in PR #1851 validates validates reads catalog and registry files when constructed Solid: Go import. Dashed: a blis-schemas command the repository runs in CI. Dotted: file reads.
Who depends on whom. blis-schemas imports nothing from the other four. The data repositories contain no Go; they run its validators in CI. The kernel imports its types, and so does the simulator in pull request #1851, which is still open.

Where to start

  • You edit catalog or registry data

    Run the validators the data repositories run in CI, and look up what a key means.

    Validate a catalog or registry · Reference

  • You write Go that reads these documents

    Load documents, validate them together, and read the report.

    Validate documents from Go · Kernel interface

  • You want to know why it is built this way

    The ideas the schemas rest on, then the reasoning behind particular choices.

    Concepts · Design notes

  • You are changing blis-schemas

    Where a change belongs, how an engine release is added, how these pages are built and published.

    Contributing

Install

go get github.com/inference-sim/blis-schemas@latest

The two command-line validators run without a checkout of this repository:

go run github.com/inference-sim/blis-schemas/cmd/validate-catalog@latest  /path/to/blis-catalog
go run github.com/inference-sim/blis-schemas/cmd/validate-registry@latest /path/to/blis-registry

These pages are versioned. The selector in the header switches between release lines; dev follows the main branch.