Fast LLM serving for agents
Inference engine optimized for agentic workloads: low latency, RadixAttention caching, OpenAI-compatible.
The LLM serving engine from Berkeley/SGD: RadixAttention for prefix caching, structured outputs, and top throughput for open models. The vLLM competitor that benchmarks love
Who it's for
ML teams serving LLMs with high-throughput inference and structured generation.
Official links
Keep exploring
Concepts in the glossary
1 in catalog
SGLang is written in
1 in catalog