Glossary
For general fuzzing terms, see the glossary from google/fuzzing project.
FuzzBench specific terms
Fuzzer
A tool that tries to find interesting inputs by feeding invalid, unexpected, or random data to a computer program (aka fuzzing). Outside of FuzzBench, it’s often called a fuzzing engine.
Examples: libFuzzer, AFL, honggfuzz, etc.
Benchmark
A fuzz target that is fuzzed to determine the performance of a fuzzer.
It can be an OSS-Fuzz project (example) or a custom one where you explicitly define the steps to checkout the code and build the fuzz target (example integration).
Trial
A single run of a particular fuzzer on a particular benchmark. For example, we might compare AFL and honggfuzz fuzzers by running 20 trials of each on the libxml2-v2.9.2 benchmark.
Experiment
A group of trials that are run together to compare fuzzer performance. This usually includes trials from multiple benchmarks and multiple fuzzers. For example, to compare libFuzzer, AFL, and honggfuzz, we might run an experiment where each of them would fuzz every benchmark. Experiments use the same number of trials for each fuzzer-benchmark pair and a specific amount of time for each trial (typically, 24 hours) so that results are comparable. FuzzBench generates reports for experiments while they are running and after they complete.