Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add airflow triggerer Deployment #555

Merged

Conversation

thesuperzapper
Copy link
Member

@thesuperzapper thesuperzapper commented Apr 2, 2022

What issues does your PR fix?

What does your PR do?

  • Adds a new Deployment for the airflow triggerer so we fully support Airflow's Deferrable Tasks feature.
    • WARNING: airflow triggerer was added in airflow 2.2.0, so will crash-loop if enabled in previous versions
    • NOTE: The triggerer deployment is automatically (and silently) disabled in the following situations:
      • if airflow.legacyCommands is true
      • if airflow.image.repository is apache/airflow and airflow.image.tag corresponds to an airflow version before 2.2.0
  • Adds the following new values:
    • triggerer.enabled (default: true)
    • triggerer.replicas (default : 1)
    • triggerer.resources
    • triggerer.nodeSelector
    • triggerer.affinity
    • triggerer.tolerations
    • triggerer.securityContext
    • triggerer.labels
    • triggerer.podLabels
    • triggerer.annotations
    • triggerer.podAnnotations
    • triggerer.safeToEvict (default: true)
    • triggerer.podDisruptionBudget:
      • triggerer.podDisruptionBudget.enabled (default: false)
      • triggerer.podDisruptionBudget.maxUnavailable
      • triggerer.podDisruptionBudget.minAvailable
    • triggerer.capacity (default: 1000)
    • triggerer.livenessProbe:
      • triggerer.livenessProbe.enabled (default: true)
      • triggerer.livenessProbe.initialDelaySeconds (default: 10)
      • triggerer.livenessProbe.periodSeconds (default: 30)
      • triggerer.livenessProbe.timeoutSeconds (default: 60)
      • triggerer.livenessProbe.failureThreshold (default: 5)
    • triggerer.extraPipPackages
    • triggerer.extraVolumeMounts
    • triggerer.extraVolumes

Checklist

For all Pull Requests

For releasing ONLY

Michał Mazek and others added 4 commits April 1, 2022 19:56
Signed-off-by: Michał Mazek <michal.mazek@samba.tv>
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com>
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com>
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com>
@thesuperzapper thesuperzapper added this to the airflow-8.6.0 milestone Apr 2, 2022
@thesuperzapper thesuperzapper added this to Unsorted in Issue Triage and PR Tracking via automation Apr 2, 2022
@thesuperzapper thesuperzapper moved this from Unsorted to PR | Ready to Merge in Issue Triage and PR Tracking Apr 2, 2022
Signed-off-by: Mathew Wicks <thesuperzapper@users.noreply.github.com>
@thesuperzapper thesuperzapper added the status/ready-to-merge status - this will be merged into next release label Apr 3, 2022
@thesuperzapper thesuperzapper merged commit eab27f4 into airflow-helm:main Apr 3, 2022
Issue Triage and PR Tracking automation moved this from PR | Ready to Merge to Done Apr 3, 2022
@thesuperzapper thesuperzapper deleted the airflow-triggerer-deployment branch April 4, 2022 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status/ready-to-merge status - this will be merged into next release
Development

Successfully merging this pull request may close these issues.

add airflow triggerer Deployment (support Deferrable Operators)
1 participant