All

Airflow Composer Example

Demonstration that uses Airflow/Composer native, Airflow/Composer local, and StarThinker tasks in the same generated DAG.

Get Access Git Hub Python Colab Airflow

Impact Level

Spend Optimization
Spend Growth
Time Savings
Account Health
Csat Improvement

Value Proposition

Build Airflow / Composer workflows using StarThinker.

Deploy any existing recipe as a DAG.

Refrence Service and User Credential integration.

Minimize manual DAG coding work.

Instructions

Execute this using Airflow or Composer, the Colab and UI recipe is for refence only.

This is an example DAG that will execute and print dates and text.

Run it once to ensure everything works, then customize it.

Details

Open Source	YES
Age	Feb. 22, 2019 (3 years, 10 months)
Authors	kenjora@google.com
Shedule Days	Configured by user.
Shedule Hours	0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23

[
    {
        "airflow": {
            "__comment__": "Calls a native Airflow operator.",
            "operators": {
                "bash_operator": {
                    "BashOperator": {
                        "bash_command": "date"
                    }
                }
            }
        }
    },
    {
        "starthinker.airflow": {
            "__comment__": "Calls an custom operator, requires import of library.",
            "operators": {
                "hello": {
                    "Hello": {
                        "say": "Hi, there!"
                    }
                }
            }
        }
    },
    {
        "hello": {
            "__comment__": "Calls a StarThinker task.",
            "auth": {
                "field": {
                    "name": "auth_read",
                    "kind": "authentication",
                    "order": 1,
                    "default": "user",
                    "description": "Credentials used for reading data."
                }
            },
            "say": "Hello World"
        }
    }
]

Run This Workflow In Minutes On Google Cloud

Everything from a quick Google Cloud UI to reference developer code for your team in one GitHub repository.