Skip to main content

Ctrl+K

Get Started

Install SGLang

Basic Usage

Sending Requests
OpenAI-Compatible APIs
Offline Engine API
SGLang Native APIs
Sampling Parameters
DeepSeek Usage
GPT OSS Usage
Llama4 Usage

Advanced Features

Server Arguments
Hyperparameter Tuning
Speculative Decoding
Structured Outputs
Structured Outputs For Reasoning Models
Tool and Function Calling
Reasoning Parser
Quantization
LoRA Serving
PD Disaggregation
Query Vision Language Model
SGLang Router
Observability
Attention Backend

Supported Models

Large Language Models
Multimodal Language Models
Embedding Models
Reward Models
Rerank Models
How to Support New Models
Transformers fallback in SGLang
Use Models From ModelScope

Hardware Platforms

AMD GPUs
Blackwell GPUs
CPU Servers
TPU
NVIDIA Jetson Orin
SGLang on Ascend NPUs

Developer Guide

Contribution Guide
Development Guide Using Docker
Benchmark and Profiling

References

Troubleshooting and Frequently Asked Questions
Environment Variables
Production Metrics
Multi-Node Deployment
Custom Chat Template
Frontend Language
Learn more

Repository
Show source
Suggest edit
Open issue

.md

TPU

TPU#

The support for TPU is under active development. Please stay tuned.

previous

CPU Servers

next

NVIDIA Jetson Orin

By SGLang Team

© Copyright 2023-2025, SGLang.

Last updated on Aug 14, 2025.