Skip to content
CLI that makes it easy to create, test and deploy Airflow DAGs to Astronomer
Go Shell Makefile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
airflow astro deploy not showing correct deployments (#276) Aug 21, 2019
auth
cluster
cmd
config Add support for streaming logs (#196) Apr 2, 2019
deployment
docker
docs Add latest v0.3.0 docs Jul 9, 2018
gendocs Refactor cli to new org Nov 28, 2018
houston Update workspace query (#274) Aug 20, 2019
logs Add support for streaming logs (#196) Apr 2, 2019
messages Add helpful message when logging in with multiple workspaces Aug 22, 2019
pkg
serviceaccount
settings
user Change error message to something more meaningful (#257) Jul 22, 2019
vendor Add executor flag to create deployment command (#252) Jul 23, 2019
version Fix create_user command May 29, 2019
workspace
.drone.yml Add docker socket volume May 28, 2019
.env Add docker exec to create default user for local airflow May 28, 2019
.gitignore Enable RBAC for local airflow May 28, 2019
.goreleaser.yml Fix linux/32bit/64bit settings: astro file not found (#216) May 8, 2019
CHANGELOG.md Adds first few auth/airflow commands Feb 8, 2018
CODEOWNERS Add codeowners (#266) Aug 7, 2019
Dockerfile-dev Update base image for unit/integration tests May 27, 2019
Gopkg.lock Add executor flag to create deployment command (#252) Jul 23, 2019
Gopkg.toml Add executor flag to create deployment command (#252) Jul 23, 2019
LICENSE
Makefile
README.md
godownloader.sh Refactor cli to new org Nov 28, 2018
main.go Add more clean error message for start airflow container May 28, 2019

README.md

Astronomer CLI GoDoc Go Report Card

The Astronomer CLI can be used to build Airflow DAGs locally and run them via Docker-Compose, as well as to deploy those DAGs to Astronomer-managed Airflow clusters and interact with the Astronomer API in general.

astro is a command line interface for working with the Astronomer Platform.

Usage:
  astro [command]

Available Commands:
  auth            Manage astronomer identity
  cluster         Manage Astronomer EE clusters
  config          Manage astro project configurations
  deploy          Deploy an airflow project
  deployment      Manage airflow deployments
  dev             Manage airflow projects and deployments
  help            Help about any command
  service-account Manage astronomer service accounts
  upgrade         Check for newer version of Astronomer CLI
  user            Manage astronomer user
  version         Astronomer CLI version
  workspace       Manage Astronomer workspaces

Flags:
  -h, --help   help for astro

Use "astro [command] --help" for more information about a command.

Installing astro

If you're using Astronomer Cloud or Astronomer Enterprise v0.7.x

via curl:

curl -sSL https://install.astronomer.io | sudo bash -s -- v0.7.5

If you're using Astronomer Enterprise v0.8 or later

via curl

curl -sSL https://install.astronomer.io | sudo bash

Previous Versions

via curl

curl -sSL https://install.astronomer.io | sudo bash -s -- [TAGNAME]

ie:

curl -sSL https://install.astronomer.io | sudo bash -s -- v0.3.1

Note: If you get mkdir error during installation please download and run godownloader script locally.

$ cat godownloader.sh | bash -s -- -b /usr/local/bin

Installing on Windows

Note: Make sure you have Windows 10 and Docker installed

  1. Download latest release of astro-cli using this astro_0.8.2_windows_386.zip
  2. Extract astro_0.8.2_windows_386.zip and copy astro.exe somewhere in your %PATH%
  3. Open cmd or PowerShell console and run:
C:\Windows\system32>astro version
Astro CLI Version: 0.8.2
Git Commit: f5cdab8f832da3c6184a7ac167b491e3bac3c022

Troubleshooting

  1. Make sure you go through instruction to install Docker on windows properly https://docs.docker.com/docker-for-windows/install/
  2. Make sure you enabled Hyper-V, it's requre to Docker and Linux Containers, also plesae review this document https://docs.docker.com/docker-for-windows/troubleshoot/

Getting Started

  1. Confirm the install worked:
$ astro
  1. Create a project:
$ mkdir hello-astro && cd hello-astro
$ astro dev init

This will generate a skeleton project directory:

.
├── dags
│   ├── example-dag.py
├── Dockerfile
├── include
├── packages.txt
├── plugins
└── requirements.txt

Dags can go in the dags folder, custom airflow plugins in plugins, python packages needed can go in requirements.txt, and OS level packages can go in packages.txt.

  1. Start airflow

Run astro dev start to start a local version of airflow on your machine. This will spin up a few locally running docker containers - one for the airflow scheduler, one for the webserver, and one for postgres. (Run docker ps to verify)

Help

The CLI includes a help command, descriptions, as well as usage info for subcommands.

To see the help overview:

$ astro help

Or for subcommands:

$ astro dev --help
$ astro deploy --help

Development

How to get started as a developer:

  1. Build:
$ git clone git@github.com:astronomerio/astro-cli.git
$ cd astro-cli
$ make build
  1. (Optional) Install to $GOBIN:
$ make install
  1. Run:
$ astro

Testing Locally

astro-cli is a single component of the much larger Astronomer Enterprise platform. In order to test locally you will need to

  1. setup both houston-api and orbit-ui.
  2. edit your global or project config to enable local development

ex.

local:
  enabled: true
  houston: http://localhost:8871/v1
  orbit: http://localhost:5000

Docs

Docs (/docs) are generated using the github.com/spf13/cobra/doc pkg. Currently this pkg is broken with go vendoring, the following instructions include a workaround

  1. Remove the /vendor/github.com/spf13/cobra pkg, forcing Go to search your go path for the package instead
  2. go run gendocs/gendocs.go
  3. restore /vendor/github.com/spf13/cobra

Versions

The Astronomer platform is under very active development. Because of this we cannot make backwards compatibility guarantees between versions.

The astro-cli is following a semantic versioning scheme

{MAJOR_RELEASE}.{MINOR_RELEASE}.{PATCH_RELEASE}

with all releases up until 1.0.0 considered beta.

astro-cli tightly tracks the platform release versioning, this means that compatibility is only guaranteed between matching minor versions of the platform and the astro-cli. ie. astro-cli v0.9.0 is guaranteed to be compatible with houston-api v0.9.x but not houston-api v0.10.x

You can’t perform that action at this time.