ads-api-report-fetcher

Gaarf Cloud Workflow

Here you can find additional components for running Gaarf tool in Google Cloud.

There are the following components provided:

So for using Cloud Workflow you need to deploy the workflow and cloud functions. But you can use cloud functions independently if you need.

Table of content

Installation

The easiest way to initialize a Google Cloud infrastructure for automated running of your queries is to use our interactive generator create-gaarf-wf.

npm init gaarf-wf@latest

To better understand what the generator creates for you we recommend to read the following sections.

NOTE: if you’re installing on Google Cloud Shell then it’s worth cleaning npm cache first to make sure you’re using the latest versions: rm -rf ~/.npm/

Cloud Workflow

To deploy workflow for gaarf you can use gcp/workflow/setup.sh script. It accepts parameters:

As soon as you deployed your workflow (you can use the same workflow for many projects by the way) you can call it directly (via gcloud workflows run) or create a Scheduler job for calling it regularly.

For both methods you’ll need to provide parameters for workflow.

Executing workflow directly:

gcloud workflows run gaarf-wf \
  --data='{
  "cloud_function":"gaarf",
  ...
  }'

Create a Scheduler job:

PROJECT_ID=$(gcloud config get-value project 2> /dev/null)
PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="csv(projectNumber)" | tail -n 1)
# be default Cloud Worflows run under the Compute Engine default service account:
SERVICE_ACCOUNT=$PROJECT_NUMBER-compute@developer.gserviceaccount.com

REGION=us-central1
WORKFLOW_NAME=gaarf-wf
JOB_NAME=$WORKFLOW_NAME

data='{
  \"cloud_function\":\"gaarf\",
  ...
  }'

# you might need to delete job first:
#gcloud scheduler jobs delete $JOB_NAME --location $REGION

# daily at midnight
gcloud scheduler jobs create http $JOB_NAME \
  --schedule="0 8 * * *" \
  --uri="https://workflowexecutions.googleapis.com/v1/projects/$PROJECT_ID/locations/$REGION/workflows/$WORKFLOW_NAME/executions" \
  --location=$REGION \
  --message-body="{\"argument\": \"$data\"}" \
  --oauth-service-account-email="$SERVICE_ACCOUNT"

Please notice the escaping of quotes for job’s argument.

Workflow Parameters

Cloud Functions

We provide several cloud functions that correlate to cli tools: gaarf, gaarf-getcids, gaarf-bq and gaarf-bq-view. Default name of main function is ‘gaarf’ and other ones’ names based on it with suffixes. You can change the base name during deployment by supplying a --name/-n parameter for deploy.sh and/or setup.sh.

To deploy cloud functions you need to execute setup.sh which enables required APIs, sets up permissions and does actual deployment of functions using deploy.sh. If later on something changes in functions’ code or Ads API settings you can redeploy by executing deploy.sh.

Additionally you can customize region (-r) and memory (-m).

There are following functions provided:

Functions Parameters

Please note that you need to copy your google-ads.yaml somewhere where functions (gaarf and gaarf-getcids) can find it. Preferably onto Cloud Storage. Then you should provide ads_config_path query argument or ADS_CONFIG environment variable with a path to the Ads config (e.g. gs://myproject/path/to/google-ads.yaml). As a last resort the function will search for google-ads.yaml locally, so we can just copy your config to function’ folder before deployment.
Alternatively you can provide all configuration values for Ads API via environment variables (you can supply env vars via --env-vars-file env.yaml cli option for gcloud tool - you’ll need to adapt the deploy.sh for this):

Normally your functions are called by Cloud Workflows workflow and you don’t need to bother about their parameters’.

Note that actual functions names are chosen on deployment. Here they are mentioned under the default ones.

gaarf

Body:

Query string:

Returns:

gaarf-bq

Body:

Query string:

Returns:

gaarf-getcids

Query string:

Returns: if no flatten specifiedd then the CF returns an object:

If flatten provided then respose is simply an array with CIDs

gaarf-bq-view

Query string:

Deployment

For deployment we recommend using our interactive generator create-gaarf-wf. It greatly simplifies deployment. As a result it creates a bunch of shell scripts that reviewed here.

Common deployment scripts are:

For deployment of all components you can use the setup.sh script with an argument with a name of your project. Providing you supplied a name “myproject” the script will deploy functions with base name “myproject” (“myproject-bq”, “myproject-getcids”, “myproject-bq-view”) and a workflow “myproject-wf”. You can always customize names and other settings for components by adjusting setup.sh scripts in components’ folders.

This site is open source. Improve this page.