This document walks you through the process of setting up a production project using ClusterFuzz.
- Setting up a production project
Make sure to go through Prerequisites page first.
Follow these instructions to create a new Google Cloud Project.
Verify that your project is successfully created using
gcloud projects describe <your project id>
Export the project id in environment for later use:
export CLOUD_PROJECT_ID=<your project id>
If you’re new to Google Cloud you may be eligible for trial credit.
Follow these instructions to add Firebase to the Google Cloud project you just created. This will be used for authentication and should not incur any additional charges.
In the Firebase console, go to the Auth section and enable “Google” as a Sign-in provider.
In the same section, add the domains you plan on using to the “Authorized domains” list. The default domain for App Engine looks like
<your project id>.appspot.com.
To obtain a web API key,
- Go to the Firebase console and select your project.
- From the project overview page, click Add Firebase to your web app to display the customized code snippet.
- Copy the
apiKeyvalue, and export it like so:
export FIREBASE_API_KEY=<your api key>
Follow these instructions to create OAuth credentials for our project setup script. Choose
OAuth client ID credential type. When prompted for an application type, choose
Other. You may also need to fill in the application name on “OAuth consent screen” tab, enter any name of your choice, e.g.
Download these credentials as JSON and place it somewhere safe. Export the path for later use:
Now you can run our project setup script to automate the process of setting up a production instance of ClusterFuzz.
This script also creates a config directory for you, which contains some default settings for your deployment and can be later updated.
mkdir /path/to/myconfig # Any EMPTY directory outside the ClusterFuzz source repository. export CONFIG_DIR=/path/to/myconfig python butler.py create_config --oauth-client-secrets-path=$CLIENT_SECRETS_PATH \ --firebase-api-key=$FIREBASE_API_KEY --project-id=$CLOUD_PROJECT_ID $CONFIG_DIR
This can take a few minutes to finish, so please be patient. The script also performs a test deployment to verify that the project has been successfully set up.
Check out the configuration yaml files in
/path/to/myconfig directory and change the defaults to suit your use cases. Some common configuration items include:
- Change the default project name using
- Add access for all users of a domain using
- Use a custom domain for hosting (instead of
It’s recommended to check your
/path/to/myconfig directory into your own version control to track your configuration changes and to prevent loss.
To verify that your project is successfully deployed.
Verify that your application is accessible on
https://<your project id>.appspot.com. If you see an error on missing datastore indexes, this may take some time to be generated after the deployment finished. You can check the status here.
Now that the initial setup is complete, you may deploy further changes by running:
python butler.py deploy --config-dir=$CONFIG_DIR --prod --force
See this page for instructions to set up the bots.
Once you make changes to the
clusters.yaml file, you must re-deploy by following the previous section. An App Engine cron job will periodically read the contents of this file and create or delete new instances as necessary.
Note that bots do not have to run on Google Compute Engine. It is possible to run your own machines or machines with another cloud provider. To do so, those machines must be running with a service account to access the necessary Google services such as Cloud Datastore and Cloud Storage.
We provide Docker images for running ClusterFuzz bots.