This repository was archived by the owner on Apr 14, 2023. It is now read-only.
Add custom plugin for running containerized tasks #10
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds a custom Airflow plugin that includes an operator and sensor that can be used together to run completely isolated workflows. There's an example workflow included in the root of this repo to show how it works.
JPrevost
reviewed
Oct 22, 2019
| json.loads(base64.b64decode(os.getenv('ECS_NETWORK_CONFIG'))) | ||
|
|
||
| to retrieve the network configuration. The ``task_definition`` will need to be manually configured after the task has been created in Terraform. There is an example workflow (``example_ecs.py``) in the root of this repo. |
Member
There was a problem hiding this comment.
I don't see any associated work in terraform. Did this not get pushed? It isn't blocking on this work, but I was just trying to better understand the bigger picture.
Author
There was a problem hiding this comment.
Oh, yeah, I haven't pushed it yet. I'll do that shortly.
JPrevost
approved these changes
Oct 22, 2019
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a custom Airflow plugin that includes an operator and sensor
that can be used together to run completely isolated workflows. There's
an example workflow included in the root of this repo to show how it
works.
How can a reviewer see these changes?
There's an example task definition in Fargate called
airflow-stage-example-task. If you want to add a workflow that uses this task, all it does is start up an alpine container, print the date and then exit.Reviewer Checklist