Google Cloud is launching the first public beta of Cloud Composer today, a new workflow automation tool for developers that’s based on the Apache Airflow project.
Typically, IT teams build their own automated workflow as needed, but the result of that is often a mess of different tools and Bash scripts that don’t always work together. Airflow and Cloud Composer allow teams to standardize on a single way of building and orchestrating workflows.
Google notes that its new tool, which uses Python as its default language, will allow teams to build workflows across on-premises tools and across multiple clouds and that the open source nature of the project will ensure that developers can take their workflows and use them across platforms. While the service is deeply integrated with the Google Cloud Platform, the team stressed that there is no lock-in.
“In building Cloud Composer, we wanted to combine the strengths of Google Cloud Platform with Airflow,” the Cloud Composer team writes in today’s announcement. “We set out to build a service that offers the best of Airflow without the overhead of installing and managing Airflow yourself. As a result, you spend less time on low-value work, such as getting Airflow installed, and more time on what matters: your workflows.”
In Airflow — and by extension in Cloud Composer, too — the different tasks and their expected outcomes (as well as what to do when things go wrong), are defined in a so-called Directed Acyclic Graph. These are standard Python files that define the workflow down to its details. You can find the full documentation here.
Google notes that it is actively participating in the Airflow community, too, and that it has contributed a number of pull requests already.
