This summarises the BLIS target design for engineers who already know llm-d and vLLM; the design is the authority, and each claim below links to the section that states it.
BLIS simulates an LLM serving cluster. It runs two kinds of code: the router of llm-d, unchanged, and Go ports of the vLLM and SGLang engines. Each port covers the engine's scheduling, its KV-cache management, its KV transfer and tiering, and how it resolves settings and chooses kernels (§7.1). Each port is tested step by step against the engine's own Python code (§7.2). A cost model, built from published hardware facts and fitted coefficients, prices every step.
One idea organises the design: the engine decides what runs, the cost model says how long it takes, and neither uses the other's vocabulary (§6.1).
blis-substrate holds everything true whatever engine runs: the shared contracts, the catalog of hardware and model facts, the coefficient registry and the cost model (§3.1). Three repositories each track one upstream: blis-vllm, blis-sglang and the llm-d router fork. inference-sim is the simulator. blis-config-search ranks configurations, and accuracy-dashboard measures how accurate BLIS is (§2.1).
Every fact and every kind of work has exactly one owner. To place a fact, ask what it depends on: an engine release, an llm-d release, or the simulator's own design. If none, it is true of the hardware, the model, the physics or the data, and it belongs in the substrate. The test for the substrate is whether a second engine would need the same edit; if not, the change belongs in an engine repository (§2.2). So inference-sim runs simulations and holds no scorer. The dashboard's scoring harness is the only place where a simulation is compared with a measured benchmark (§2.2, §11.7).
Once per pool, the engine resolves its configuration as the upstream does, lowers the model to kernel implementations, sizes its KV budget, and hands the cost model a PoolPlan. At each step the scheduler forms a StepShape, already padded as the upstream pads it, and asks for its price through an Environment that inference-sim supplies. The cost model prices it from the catalog and the coefficient registry and returns a time. The engine reports the step done with its metrics and KV events. inference-sim advances simulated time and delivers the metrics and events to the router (§2.3, §6.3).
Each KV move between tiers is an engine.Transfer. The cost model turns it into the bytes each link must carry, and inference-sim shares each link among the transfers that overlap in time. This is the only resource arithmetic the simulator does (§6.2).
The three objects are described in physical units, never in an engine's terms. The cost model therefore never sees an engine's settings and never applies an engine's rules, and outside its tests the substrate contains no engine name (§6.1, §4). A coefficient is keyed by a hash of the kernel's source, not by a library version, so a version bump that leaves a kernel unchanged needs no substrate change (§6.2). A new engine is a new repository that adds to the contracts and changes nothing else in the substrate (§12.6).
BLIS takes its terms from DEVS. Simulated time moves from one event, such as a request arriving or a step ending, to the next. The parts are components: an engine instance, the router, a link, a client. Components interact only through timestamped messages, and none reads another's state. The router learns of an engine only through the metrics and KV events the engine sends, after the delay the deployment's settings impose (§8.1).
Many events fall at one instant, and BLIS processes an instant in three phases. Every component with an internal event runs its output function. Each component receives the bag of its messages, sorted by the key (time, class, sender, the sender's own message count). Each component then makes exactly one transition: internal, external, or confluent when both coincide. No component sees another's change from the same instant before making its own, and no global counter enters the order (§8.2).
Time is an integer number of microseconds. Every random draw comes from a substream named by component, purpose and entity and seeded from the scenario seed, so request 17 of client 3 gets the same prompt length whatever happened to earlier requests (§8.3, §8.4). For fixed inputs the output is identical byte for byte across runs, thread counts and platforms (§4).
A run measures something only within its experimental frame: the workload and seeds, the initial state, warm-up, measured window and drain, the stop rule, the acceptance conditions and the metric definitions. The frame is recorded in every Result (§8.6). Outputs are estimates. Where a run has random input, the estimate is a mean over replications with a confidence interval, and a comparison of two configurations has three outcomes: better, worse, or not distinguished (§8.8). A deterministic run is reproducible, not accurate. Verification asks whether the code does what the model says; validation asks whether the model matches the real system, and a change that is both is split (§8.9).
The cost model needs a coefficient for every primitive on every chip. For each lookup it uses the entry with the smallest rank number: measured, calibrated, transferred, literature, vendor_spec, roofline, assumed. Every value is labelled with its method, and every price records the least trusted method it used, so coverage can grow without hiding how good an estimate is (§9.1, §9.2, §9.3).
The data ledger lists the external data units (benchmark runs, operator sweeps, forward-pass cells) and gives each exactly one role: fit, select, validate, held, anchor or report. A role is computed from a secret seed, once for each group of near-copies, so a point that fitted a value cannot also judge it (§10.2).
With each simulator release the dashboard scores the anchor units and gives each point one grade: G1 when the run's engine release is the one scoring and every condition for exactness holds, G2 where substitutions are named, G3 and G4 for another engine release, and G5 for a point not scored, for a listed reason. A grade is computed from data, and a condition the data cannot decide is never assumed to hold (§11.3, §11.4). The dashboard copies the run's recorded configuration unchanged and reads each field's status from the engine's own field list; it keeps no table of modelled settings (§11.1, §11.2). Every published figure shows how many points it covers. What lowers coverage or grade is ranked in a backlog, and work is chosen from its top (§11.6).
Every rule has an identifier and is kept in one of three ways. An automated rule is a deterministic check in the repository that owns it. A maintainer practice is kept by a person with write access for a repository as a whole, such as branch protection. Agent guidance is followed by an agent that writes or reviews a change; where a rule needs judgement, a review agent answers a question as guidance, never as a check (§4.2, §13.2). Dependencies point one way, and a repository never imports one above it (§4).
For each upstream release, the engine repositories generate a surface list from the upstream's own registration tables. Every connector, tier, KV-cache manager, speculative method, policy, field, metric and event type has one status: modelled with evidence, refused by name with an issue, or out of scope for one of three listed reasons. CI fails while any item has none, so gaps are counted and ranked, not found by accident (§7.5). The list covers what the upstream registers. A point whose run loaded code the upstream does not ship, such as an out-of-tree connector, is not scored (§11.4). Each simulator release runs exactly one upstream release of each engine and of llm-d, and says which (§14.2).