The StarThinker project will no longer receive new solution contributions from the Google team.
Please read the full StarThinker Open Source Support Ends At Google article for more details.



All

email

CM360 Report Emailed To BigQuery

Pulls a CM Report from a gMail powered email account into BigQuery.

CM360 Report Emailed To BigQuery Sample Screenshot


lock_openGet Access listGit Hub sourcePython menu_bookColab airAirflow thumb_upTest



Impact Level

Spend Optimization
Spend Growth
Time Savings
Account Health
Csat Improvement

Value Proposition

Acquire client CM data without having an account.
Build new data ingestion service for clients.
Consolidate client data in a central data warehouse.
Automate emailed report downloading.
Build dashboards from emailed reports.

Instructions

The person executing this recipe must be the recipient of the email.
Schedule a CM report to be sent to .
Or set up a redirect rule to forward a report you already receive.
The report must be sent as an attachment.
Ensure this recipe runs after the report is email daily.
Give a regular expression to match the email subject.
Configure the destination in BigQuery to write the data.

Details

Open Source YES
Age March 11, 2020 (2 years, 9 months)
Authors kenjora@google.com
Shedule Days Configured by user.
Shedule Hours Configured by user.
[
    {
        "email": {
            "auth": {
                "field": {
                    "name": "auth_read",
                    "kind": "authentication",
                    "order": 1,
                    "default": "user",
                    "description": "Credentials used for reading data."
                }
            },
            "read": {
                "from": "noreply-cm@google.com",
                "to": {
                    "field": {
                        "name": "email",
                        "kind": "string",
                        "order": 1,
                        "default": "",
                        "description": "Email address report was sent to."
                    }
                },
                "subject": {
                    "field": {
                        "name": "subject",
                        "kind": "string",
                        "order": 2,
                        "default": ".*",
                        "description": "Regular expression to match subject. Double escape backslashes."
                    }
                },
                "attachment": ".*"
            },
            "write": {
                "bigquery": {
                    "dataset": {
                        "field": {
                            "name": "dataset",
                            "kind": "string",
                            "order": 3,
                            "default": "",
                            "description": "Existing dataset in BigQuery."
                        }
                    },
                    "table": {
                        "field": {
                            "name": "table",
                            "kind": "string",
                            "order": 4,
                            "default": "",
                            "description": "Name of table to be written to."
                        }
                    },
                    "header": true,
                    "is_incremental_load": {
                        "field": {
                            "name": "is_incremental_load",
                            "kind": "boolean",
                            "order": 6,
                            "default": false,
                            "description": "Append report data to table based on date column, de-duplicates."
                        }
                    }
                }
            }
        }
    }
]


Run This Workflow In Minutes On Google Cloud

Everything from a quick Google Cloud UI to reference developer code for your team in one GitHub repository.

Deployment Stepslaunch Developer Guidebuild UI How Tolaptop