A curated collection of guides to help with building specific ETL pipelines using Airflow.
These are stepwise instructions for using Astronomer and Apache Airflow, and use various repositories from the open-source Airflow Plugins Organization.
The Astronomer website uses the
/guides directory in this repo as a CMS for it's "Airflow Guides" content. To add a new guide to that content, a
.md file will need to be created inside the
/guides repo with appropriate GitHub markdown formatting and some standardized front-matter. Until the Astronomer website is rebuilt, no changes to this repo will be reflected there. Also, only content from the
/guides directory will be parsed and used - no other files or directories will affect the site.
.md files may be added to the
/guides directory - no subdirectories or other file-types may be used. The rest of the repo may include files and directories of any kind.
Building a new guide
- Duplicate an existing guide inside the
- Update the file-name and front-matter, and pay close attention to formatting
- Make sure to use
hyphen-seperated-casewhen naming your file and declaring your slug
- The filename and slug must match: i.e
- Store all images (inline and hero) in the
astronomer-cdnbucket on s3 in the
/website/img/guidesdirectory, and reference them using
- When the guide is finished, commit all changes to
- Rebuild the Astronomer website using the
How to deploy guidessteps below
How to deploy guides
- All changes pushed to airflow-guides will trigger a webhook to rebuild preview.astronomer.io.
- To deploy to www.astronomer.io, publish a new release. We are starting at v1.0.0, using semantic versioning. This means, if you publish a new guide, bumb the 2nd number eg
v1.1.0. If you have to edit a guide, bump the 3rd number