Argoproj - Get stuff done with Kubernetes

Cyrus specific developer notes

To cut a version tag to release to rc run, ./cut_version_tag.sh --environment=rc. This script will switch to the master branch, pull and then cut a version using the current contents of the VERSION file at the HEAD of master.
The new tag will be automatically pushed and will trigger a deploy to rc.

• to trigger a release to staging: ./cut_version_tag.sh --environment=rc
• to trigger a release to production: ./cut_version_tag.sh --environment=production or ./cut_version_tag.sh --environment=hotfix

Jenkins will automatically build images and command line executables for newly pushed tags.

Merges to master in this repository are not automatically deployed. a tag is required to trigger a release

Argoproj - Get stuff done with Kubernetes

Quickstart

kubectl create namespace argo
kubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo/stable/manifests/install.yaml

News

KubeCon 2018 in Seattle was the biggest KubeCon yet with 8000 developers attending. We connected with many existing and new Argoproj users and contributions, and gave away a lot of Argo T-shirts at our booth sponsored by Intuit!

We were also super excited to see KubeCon presentations about Argo by Argo developers, users and partners.

• CI/CD in Light Speed with K8s and Argo CD
  • How Intuit uses Argo CD.
• Automating Research Workflows at BlackRock
  • Why BlackRock created Argo Events and how they use it.
• Machine Learning as Code
  • How Kubeflow uses Argo Workflows as its core workflow engine and Argo CD to declaratively deploy ML pipelines and models.

If you actively use Argo in your organization and your organization would be interested in participating in the Argo Community, please ask a representative to contact saradhi_sreegiriraju@intuit.com for additional information.

What is Argoproj?

Argoproj is a collection of tools for getting work done with Kubernetes.

• Argo Workflows - Container-native Workflow Engine
• Argo CD - Declarative GitOps Continuous Delivery
• Argo Events - Event-based Dependency Manager
• Argo Rollouts - Deployment CR with support for Canary and Blue Green deployment strategies

What is Argo Workflows?

Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).

• Define workflows where each step in the workflow is a container.
• Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a graph (DAG).
• Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes.
• Run CI/CD pipelines natively on Kubernetes without configuring complex software development products.

Why Argo Workflows?

• Designed from the ground up for containers without the overhead and limitations of legacy VM and server-based environments.
• Cloud agnostic and can run on any Kubernetes cluster.
• Easily orchestrate highly parallel jobs on Kubernetes.
• Argo Workflows puts a cloud-scale supercomputer at your fingertips!

Documentation

• Get started here
• How to write Argo Workflow specs
• How to configure your artifact repository

Features

• DAG or Steps based declaration of workflows
• Artifact support (S3, Artifactory, HTTP, Git, raw)
• Step level input & outputs (artifacts/parameters)
• Loops
• Parameterization
• Conditionals
• Timeouts (step & workflow level)
• Retry (step & workflow level)
• Resubmit (memoized)
• Suspend & Resume
• Cancellation
• K8s resource orchestration
• Exit Hooks (notifications, cleanup)
• Garbage collection of completed workflow
• Scheduling (affinity/tolerations/node selectors)
• Volumes (ephemeral/existing)
• Parallelism limits
• Daemoned steps
• DinD (docker-in-docker)
• Script steps

Who uses Argo?

As the Argo Community grows, we'd like to keep track of our users. Please send a PR with your organization name.

Currently officially using Argo:

1. Adevinta
2. Admiralty
3. Adobe
4. Alibaba Cloud
5. BlackRock
6. Canva
7. Codec
8. CoreFiling
9. Cratejoy
10. Cyrus Biotechnology
11. Datadog
12. DataStax
13. Equinor
14. Gardener
15. Gladly
16. GitHub
17. Google
18. IBM
19. Interline Technologies
20. Intuit
21. Karius
22. KintoHub
23. Localytics
24. Max Kelsen
25. Mirantis
26. NVIDIA
27. OVH
28. Preferred Networks
29. Quantibio
30. SAP Fieldglass
31. SAP Hybris
32. Styra
33. Threekit
34. Commodus Tech

Community Blogs and Presentations

• Argo Ansible role: Provisioning Argo Workflows on OpenShift
• Argo Workflows vs Apache Airflow
• CI/CD with Argo on Kubernetes
• Running Argo Workflows Across Multiple Kubernetes Clusters
• Open Source Model Management Roundup: Polyaxon, Argo, and Seldon
• Producing 200 OpenStreetMap extracts in 35 minutes using a scalable data workflow
• Argo integration review
• TGI Kubernetes with Joe Beda: Argo workflow system
• Community meeting minutes and recordings

Project Resources

• Argo GitHub: https://github.com/argoproj
• Argo website: https://argoproj.github.io/
• Argo Slack: click here to join