User Guide¶
Task-oriented guides for using BLIS effectively. Each guide covers a specific feature with practical CLI examples and expected output.
Guides¶
| Guide | When to Use |
|---|---|
| Routing (llm-d router) | Configuring the embedded llm-d router that distributes requests across instances |
| Admission Control | The router's admission, flow control and request priorities |
| Scheduling & Priority | Controlling request processing order within each instance |
| Latency Model | blis-latency-kernel: what it prices, scenario structure, P/D, DP/EP, speculation, offload, LoRA |
| KV Cache & Memory | Tuning GPU/CPU memory allocation, prefix caching, and chunked prefill |
| Workload Specifications | Defining multi-client traffic patterns with YAML |
| Cluster Simulation | Running multi-instance simulations with the full pipeline |
| Metrics & Results | Understanding JSON output, metrics, anomaly counters, and fitness scores |
| Observe / Replay / Calibrate | Replaying trace workloads (blis replay); observe/calibrate are deprecated (#1901) |
| Skills & Plugins | Claude Code skills, plugin marketplaces, and workflow tooling |
Reading Paths¶
Capacity planning: Quick Start → Tutorial → Cluster Simulation → Metrics & Results
Routing optimization: Routing (llm-d router) → Cluster Simulation → Metrics & Results
Memory tuning: KV Cache & Memory → Metrics & Results
New model evaluation: Latency Model → Workload Specifications → Metrics & Results
Trace replay: Latency Model → Workload Specifications → Observe / Replay / Calibrate → Metrics & Results