Skip to main content

Ctrl+K

Get Started

Install SGLang

Basic Usage

Sending Requests
OpenAI-Compatible APIs
Offline Engine API
SGLang Native APIs
Sampling Parameters
DeepSeek Usage
GPT OSS Usage
Llama4 Usage
Qwen3-Next Usage

Advanced Features

Server Arguments
Hyperparameter Tuning
Speculative Decoding
Structured Outputs
Structured Outputs For Reasoning Models
Tool Parser
Reasoning Parser
Quantization
LoRA Serving
PD Disaggregation
PD Multiplexing
Query Vision Language Model
SGLang Router
Observability
Attention Backend
Hierarchical KV Caching (HiCache)
- SGLang HiCache Best Practices
- HiCache System Design and Optimization

Supported Models

Large Language Models
Multimodal Language Models
Embedding Models
Reward Models
Rerank Models
How to Support New Models
Transformers fallback in SGLang
Use Models From ModelScope

Hardware Platforms

AMD GPUs
CPU Servers
TPU
NVIDIA Jetson Orin
Ascend NPUs

Developer Guide

Contribution Guide
Development Guide Using Docker
Benchmark and Profiling
Bench Serving Guide

References

Troubleshooting and Frequently Asked Questions
Environment Variables
Production Metrics
Multi-Node Deployment
Custom Chat Template
Frontend Language
Learn more

Repository
Show source
Suggest edit
Open issue

.rst

Hierarchical KV Caching (HiCache)

Hierarchical KV Caching (HiCache)#

SGLang HiCache Best Practices
HiCache System Design and Optimization

previous

Attention Backend

next

SGLang HiCache Best Practices

By SGLang Team

© Copyright 2023-2025, SGLang.

Last updated on Oct 12, 2025.