CLI for Astronomer
Latest commit b760eb1 Dec 7, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
airflow Adding link to docs site Dec 6, 2018
auth Refactor cli to new org Nov 28, 2018
cluster Refactor cli to new org Nov 28, 2018
cmd Anchor dockerfile tag to platform version (#151) Nov 28, 2018
config Refactor cli to new org Nov 28, 2018
deployment Refactor cli to new org Nov 28, 2018
docker Add better error wrapping around docker exec Sep 5, 2018
docs Add latest v0.3.0 docs Jul 9, 2018
gendocs Refactor cli to new org Nov 28, 2018
houston Refactor cli to new org Nov 28, 2018
messages Allow airflow webserver port to be configurable locally (#145) Nov 16, 2018
pkg Refactor cli to new org Nov 28, 2018
serviceaccount Refactor cli to new org Nov 28, 2018
user Refactor cli to new org Nov 28, 2018
vendor Add command to output local airflow logs Aug 22, 2018
version Anchor dockerfile tag to platform version (#151) Nov 28, 2018
workspace Refactor cli to new org Nov 28, 2018
.drone.yml Add slack notification to drone Nov 29, 2018
.gitignore Added IDE’s to .gitignore Apr 25, 2018
.goreleaser.yml Change astronomerio to astronomer gopath Nov 28, 2018
CHANGELOG.md Adds first few auth/airflow commands Feb 8, 2018
Gopkg.lock Add command to output local airflow logs Aug 22, 2018
Gopkg.toml Provide base houston integration Apr 18, 2018
LICENSE Adds first few auth/airflow commands Feb 8, 2018
Makefile Anchor dockerfile tag to platform version (#151) Nov 28, 2018
README.md Output to stderr if install script download errs Oct 2, 2018
godownloader.sh Refactor cli to new org Nov 28, 2018
main.go Refactor cli to new org Nov 28, 2018

README.md

Astronomer CLI

The Astronomer CLI is the recommended way to get started developing and deploying Airflow projects to Astronomer Enterprise Edition.

Install

  • via curl
    curl -sSL https://install.astronomer.io | sudo bash
    

Previous Versions

-via curl

curl -sSL https://install.astronomer.io | sudo bash -s -- [TAGNAME]

ie

curl -sSL https://install.astronomer.io | sudo bash -s -- v0.3.1

Note: If you get mkdir error during installation please download and run godownloader script locally.

$ cat godownloader.sh | bash -s -- -b /usr/local/bin

Getting Started

  1. Confirm the install worked:

    $ astro
    
  2. Create a project:

    $ mkdir hello-astro && cd hello-astro
    $ astro airflow init
    

This will generate a skeleton project directory:

.
├── dags
│   ├── example-dag.py
├── Dockerfile
├── include
├── packages.txt
├── plugins
└── requirements.txt

Dags can go in the dags folder, custom airflow plugins in plugins, python packages needed can go in requirements.txt, and OS level packages can go in packages.txt.

  1. Start airflow

Run astro airflow start to start a local version of airflow on your machine. This will spin up a few locally running docker containers - one for the airflow scheduler, one for the webserver, and one for postgres. (Run docker ps to verify)

Help

The CLI includes a help command, descriptions, as well as usage info for subcommands.

To see the help overview:

$ astro help

Or for subcommands:

$ astro airflow --help
$ astro airflow deploy --help

Development

How to get started as a developer.

  1. Build:

    $ git clone git@github.com:astronomerio/astro-cli.git
    $ cd astro-cli
    $ make build
    
  2. (Optional) Install to $GOBIN:

    $ make install
    
  3. Run:

    $ astro
    

Testing Locally

astro-cli is a single component of the much larger Astronomer Enterprise platform. In order to test locally you will need to

  1. setup both houston-api and orbit-ui.
  2. edit your global or project config to enable local development

ex.

local:
  enabled: true
  houston: http://localhost:8870/v1
  orbit: http://localhost:5000

Docs

Docs (/docs) are generated using the github.com/spf13/cobra/doc pkg. Currently this pkg is broken with go vendoring, the following instructions include a workaround

  1. Remove the /vendor/github.com/spf13/cobra pkg, forcing Go to search your go path for the package instead
  2. go run gendocs/gendocs.go
  3. restore /vendor/github.com/spf13/cobra

Versioning

Astronomer Enterprise is under very active development. Because of this we cannot make backwards compatibility guarantees between versions.

THe astro-cli is following a semantic versioning scheme

{MAJOR_RELEASE}.{MINOR_RELEASE}.{PATCH_RELEASE}

with all releases up until 1.0.0 considered beta.

Compatibility

Starting with v0.3.3 the astro-cli began tightly tracking the platform release versioning, this means that compatibility is only guaranteed between matching minor versions of the platform and the astro-cli.

ie. astro-cli v0.4.0 is guaranteed to be compatible with houston-api v0.4.x but with houston-api v0.5.x

Note

These changes were introduced platform wide with v0.4.0