Automatic Flamegraphs for Benchmarks in Rust
This is just a short post about something I recently discovered in the Rust ecosystem:
How to automatically create flamegraphs for benchmarks written with
criterion using the
Criterion is a well-known and often-used benchmarking framework in the Rust ecosystem. If you’ve not yet heard of it, definitely check it out!
v0.3, Criterion supports in-process profiling hooks.
They allow us to use a custom profiler while running benchmarks.
After registering a custom profiler, we can enable them by running the benchmark suite with the
Having hooks for a custom profiler for your benchmarks allows you to do awesome stuff, for example generating flamegraphs for every benchmark with
One question I often ask myself when looking at benchmarks is:
“These numbers are great. But how can we do this faster? Where do we spend the most time?”
This question can often be answered by looking at flamegraphs.
Previously, I got my hands dirty using
cargo-flamegraph, which is a really good tool on its own.
But the problem is that when using
cargo-flamegraph with benchmarks, you will usually get some unnecessary clutter, e.g. the setup routine for your benchmark and the enclosing framework code from Criterion.
And here comes another cool Rust library into play:
pprof with the
pprof is a CPU profiler that can be used to profile specific parts of your program on Linux.
The best of all: It even supports generating flamegraphs with the feature flag
Let’s directly dive in and create a
pprof integration for Criterion!
We will now create a custom profiler for
pprof that will print flamegraphs for each benchmark.
First, add the following dependencies to your
Now create a new file in your
benches/ folder. Let’s call it
Here we will implement our new
The interface for implementing a custom profiler is actually quite straight-forward.
We only need to implement the
criterion::profiler::Profiler trait from Criterion and we are good to go.
To use the new profiler, we need to register it with Criterion. See also Criterion’s advanced configuration.
If you are using the custom test framework feature with
criterion-macro, you can configure it with:
Else, you can do it like:
Now that we have set up our custom profiler we can actually use it for our benchmarks as described in the next section.
Running the Benchmarks and Getting Results
Enable Performance Profiling for Unprivileged Users
To enable performance profiling without running the benchmarks as root, you may need to adjust the value of
perf_event_paranoid in the Linux kernel to an appropriate value for your environment. The most permissive value is
Running the Benchmarks
Now that everything is set up, we can run our benchmark with
This command will run the benchmarks in
benches/my_bench.rs for 5 seconds with the use of our custom
Note: You need to specify the name of your benchmark because otherwise you might get the error
Unrecognized option: 'profile-time'. See also
cargo benchgives “Unrecognized Option” errors for valid command-line options.
Viewing the Flamegraph
We will now find a file called
For the above example, we get the following result:
Cool feature. I really like it 😊