Getting Started

Welcome to GenAI Bench! This section will help you get up and running quickly.

Overview

GenAI Bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems. It supports multiple cloud providers for both model endpoints and storage.

Quick Links

• Installation

  ---

  Get GenAI Bench installed on your system

  Installation Guide

• Task Types

  ---

  Learn about supported benchmark tasks

  Task Definition

• CLI Usage

  ---

  Master the command-line interface

  Command Guidelines

• Metrics

  ---

  Understand benchmark metrics

  Metrics Definition

Prerequisites

Before you begin, ensure you have:

• Python 3.8 or higher
• Access to at least one supported LLM endpoint
• (Optional) Cloud storage credentials for result upload

Next Steps

1. Install GenAI Bench
2. Configure your authentication
3. Run your first benchmark