Rust Bench — Rust Performance Benchmarking & Profiling Framework

中文版

A toolkit for runtime performance benchmarking, data collection, terminal visualization, and profiling of Rust programs. Built on rustc-perf, extended with runtime benchmark capabilities, custom metrics, a TUI comparison interface, and multi-platform (GitCode/GitHub) multi-repository support.

Compile-time benchmark machinery inherited from upstream rustc-perf exists in the codebase but is not the focus of this project. This README and the documentation set cover runtime benchmarks and the distributed benchmarking system.

What it does

• Runtime benchmarks — measure how fast Rust programs execute under a given rustc, using hardware performance counters (perf_event_open), wall-clock timing, and memory high-water mark.
• Local & distributed — benchmark on a single machine (results in SQLite) or run an automated flow where a central Site receives webhooks, Collectors pull jobs from a Postgres queue, and comparison comments are posted back to the PR/MR.
• Profiling — Cachegrind (instruction-level) and perf-record (sampling); cachegrind can diff two rustc builds.
• Multi-platform, multi-repo — benchmark Rust toolchains and non-Rust repos (e.g. the Daft dataframe library) across GitCode and GitHub, across multiple architectures in parallel.

Installation

Pre-built binaries

Pre-built binary packages are available on the Releases page.

Package	Contents
rust-bench-collector	collector binary + runtime benchmark module sources + site-config.toml template
rust-bench-site	site binary + site-config.toml template

Download, extract, and follow the package's README to get started — no compilation required.

Build from source

cargo build --release

See the local benchmarking guide for details.

Quick start

Prerequisites

• Linux (required for perf_event hardware counters).
• Rust stable toolchain.
• Enable userspace perf counters:

  sudo bash -c 'echo -1 > /proc/sys/kernel/perf_event_paranoid'

Run a benchmark (minimal example)

# Benchmark the standard library runtime suite with the nightly toolchain
./target/release/collector bench_runtime_local +nightly

# Compare two runs in the interactive TUI
./target/release/collector bench_cmp --db results.db

<RUSTC> accepts a path to a rustc executable or a +-prefixed toolchain name. Benchmark names are module_path::fn_name, so prefix filters match on the full path (e.g. --include std_bench::hash).

Known limitations

• CI mode (PostgreSQL + webhook) currently supports only the daft module. Other modules (monoio, auron, firecracker, wasmtime, tikv) support local mode only (bench_runtime_local).

Documentation

Full documentation lives in docs/. The single source of truth for commands and details is the relevant guide, not this README.

Guides (procedural — how to do things):

• Run benchmarks locally — bench_runtime_local, filters, output, bench_cmp TUI.
• Write a runtime benchmark — #[bench] macro and the custom framework.
• Profile a benchmark — Cachegrind and perf-record; diffing two rustc builds.
• Deploy the Site — Docker, Postgres, site-config.toml, webhook endpoint.
• Deploy a Collector — registration, collect-job-queue.sh, machine prep.
• Integrate GitCode & GitHub — webhook setup, /rust-bench commands, completion comments.

Reference docs (factual — what things are):

• Architecture · Configuration · Job queue · Deployment model · Comparison analysis · Glossary
• Database schema · Manual modifications

Webhook/commands note: the webhook endpoint is /perf/webhook (not /api/webhook), and the bot command is /rust-bench try|master|tag (not @rust-bench queue). See the platform-integration guide for the exact syntax.

Project structure

rust-bench/
├── collector/                 # benchmark executor + benchlib measurement library
│   ├── benchlib/             # linked into every runtime benchmark binary
│   ├── benchlib-macros/      # #[bench] proc-macro
│   ├── runtime-benchmarks/   # runtime benchmark suite (std, core, alloc, fmt, daft, ...)
│   ├── compile-benchmarks/    # legacy compile-time suite (out of scope)
│   └── src/
│       ├── bin/collector.rs  # CLI entry point
│       ├── runtime/          # runtime benchmark execution + profiling
│       └── compare/screen.rs# bench_cmp TUI
├── database/                 # SQLite/Postgres layer + schema docs
├── site/                     # web server + job queue + platform (GitCode/GitHub) integration
│   └── src/{main,server,load,job_queue,github,request_handlers}.rs
├── docs/                     # reference docs + guides (this documentation set)
├── site-config.toml          # site configuration template
├── Dockerfile                # multi-stage Docker build for the Site
├── docker-compose.yml        # throwaway Postgres 16 for testing
└── Cargo.toml                # workspace configuration

License

The original rustc-perf code is licensed under the MIT license, managed by the Reuse Specification. The compile-time benchmarks have their own separate licenses — see collector/compile-benchmarks/REUSE.toml. Modifications and extensions by the Xuanwu Team are also licensed under the MIT License.

Copyright (c) Xuanwu Team. All rights reserved.