Setting up a production project
This document walks you through the process of setting up a production project using ClusterFuzz.
- Setting up a production project
Make sure to go through Prerequisites page first.
Create a new Google Cloud project
Follow these instructions to create a new Google Cloud Project.
Verify that your project is successfully created using
gcloud projects describe <your project id>
Export the project id in environment for later use:
export CLOUD_PROJECT_ID=<your project id>
If you’re new to Google Cloud you may be eligible for trial credit.
Create OAuth credentials
Follow these instructions
to create OAuth credentials for our project setup script. Choose
OAuth client ID credential type. When prompted for an application type, choose
Other. You may also need to fill in the application name on “OAuth consent
screen” tab, enter any name of your choice, e.g.
Download these credentials as JSON and place it somewhere safe. Export the path for later use:
Run the project setup script
Now you can run our project setup script to automate the process of setting up a production instance of ClusterFuzz.
This script also creates a config directory for you, which contains some default settings for your deployment and can be later updated.
mkdir /path/to/myconfig # Any EMPTY directory outside the ClusterFuzz source repository. export CONFIG_DIR=/path/to/myconfig python butler.py create_config --oauth-client-secrets-path=$CLIENT_SECRETS_PATH --project-id=$CLOUD_PROJECT_ID $CONFIG_DIR
This can take a few minutes to finish, so please be patient. The script also performs a test deployment to verify that the project has been successfully set up.
Check out the configuration yaml files in
/path/to/myconfig directory and
change the defaults to suit your use cases. Some common configuration items
- Change the default project name using
- Add access for all users of a domain using
- Use a custom domain for hosting (instead of
It’s recommended to check your
/path/to/myconfig directory into your own
version control to track your configuration changes and to prevent loss.
To verify that your project is successfully deployed.
Verify that your application is accessible on
https://<your project id>.appspot.com. If you see an error on missing datastore indexes, this may take some time to be generated after the deployment finished. You can check the status here.
Deploying new changes
Now that the initial setup is complete, you may deploy further changes by running:
python butler.py deploy --config-dir=$CONFIG_DIR --prod --force
Configuring number of bots
See this page for instructions to set up the bots.
Once you make changes to the
clusters.yaml file, you must re-deploy by
following the previous section. An App Engine cron job
will periodically read the contents of this file and create or delete new
instances as necessary.
Other cloud providers
Note that bots do not have to run on Google Compute Engine. It is possible to run your own machines or machines with another cloud provider. To do so, those machines must be running with a service account to access the necessary Google services such as Cloud Datastore and Cloud Storage.
We provide Docker images for running ClusterFuzz bots.