Skip to main content

Ctrl+K

Installation

Install SGLang

Backend Tutorial

Sending Requests
OpenAI APIs - Completions
OpenAI APIs - Vision
OpenAI APIs - Embedding
SGLang Native APIs
Offline Engine API
Server Arguments
Sampling Parameters
Hyperparameter Tuning

Advanced Features

Speculative Decoding
Structured Outputs
Tool and Function Calling
Custom Chat Template
Quantization

Frontend Tutorial

Structured Generation Language
Choices Methods in SGLang

SGLang Router

Router for Data Parallelism

References

General Guidance
Hardware Supports
Multi-Node Deployment
Performance Tuning

Repository
Show source
Suggest edit
Open issue

.rst

General Guidance

General Guidance#

Supported Models
Contribution Guide
Troubleshooting
Frequently Asked Questions
Learn more
Use Models From ModelScope

previous

Router for Data Parallelism

next

Supported Models

By SGLang Team

© Copyright 2023-2024, SGLang.

Last updated on Feb 23, 2025.