ClusterFuzz provides an automated end-to-end infrastructure for finding and triaging crashes, minimizing reproducers, bisecting, and verification of fixes.
ClusterFuzz is written in Python and Go. It runs on Linux, macOS, and Windows.
It runs on the Google Cloud Platform, and depends on a number of services:
- Compute Engine (Not strictly necessary. Bots can run anywhere).
- App Engine
- Cloud Storage
- Cloud Datastore
- Cloud Pub/Sub
- Stackdriver Logging and Monitoring
Note: The only bug tracker supported now is the Chromium hosted Monorail. Support for custom bug trackers will be added in the near future.
It’s possible to run ClusterFuzz locally without these dependencies by using local Google Cloud emulators, but some features which depend on BigQuery and Stackdriver will be disabled due to lack of emulator support.
The two main components of ClusterFuzz are:
- App Engine instance
- A pool of bots
The App Engine instance provides a web interface to access crashes, stats and other information. It’s also responsible for scheduling regular cron jobs.
Bots are machines which run scheduled tasks. They lease tasks from platform specific queues. The main tasks that bots run are:
fuzz: Run a fuzzing session.
progression: Check if a testcase still reproduces or if it’s fixed.
regression: Calculate the revision range in which a crash was introduced.
minimize: Perform testcase minimization.
corpus_pruning: Minimize a corpus to smallest size based on coverage (libFuzzer only).
analyze: Run a manually uploaded testcase against a job to see if it crashes.
There are 2 kinds of bots on ClusterFuzz - preemptible and non-preemptible.
Preemptible means that the machine can shutdown at any time. On these machines
we only run
fuzz task. These machines are often cheaper on cloud providers, so
it’s recommended to scale using these machines.
Non-preemptible machines are not expected to shutdown. They are able to run all
fuzz) and other critical tasks such as
must run uninterrupted.