BLIS design

BLIS simulates an LLM serving cluster. It runs the router of llm-d unchanged, and Go ports of the vLLM and SGLang engines. These pages explain how its repositories are divided, what each one owns, and the rules that keep them correct.

Read in this order

The three documents say the same thing at three depths. Stop when you have what you need.

1. The design in three pages (3 printed pages)
   The shape of the simulator, for a reader who already knows llm-d and vLLM: the repositories, what each owns, and how a simulated step flows between them.
2. The design in seven pages (7 printed pages)
   The same account with the detail a contributor needs before changing code.
3. The full target design (about 90 printed pages)
   The authority. Every rule, how it is checked, and why it exists. Each claim in the summaries links to the section here that states it.

What the design is

The design is a standing document. Every statement in it holds for every release: it says what must be true, not what is true at the moment, and it has no dates or milestones. Where a page says what a release schedules, that belongs to a separate release plan, which is not part of these pages.

The code

The repositories live under github.com/inference-sim. Start with inference-sim, the simulator, and follow the repository table in the design to the others. To ask for a feature or report a problem, open an issue in the repository that owns the behaviour.