Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[incubator/airflow] Airflow Helm Chart (helm#3959)
* Naive import from gh:gsemet/kube-airflow Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * update chart with recent fixes from https://github.com/gsemet/kube-airflow Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Update the templates from kube-airflow Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Update after review Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Sync with gsemet/kube-airflow Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Move to secrets Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fixed some (find/replace?) errors. Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * fix env vars add missing var name REDIS_USER Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * apply base64 encoding to secrets delete obsolete var/secret REDIS_USER Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Remove trailing whitespace Fix comment (postgres -> Redis) Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Add configurable serviceAccountName to workers Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fix templates API versions Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Update instructions for embedded DAGs Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Rename worker/celery vars Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fix NOTES.txt Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Remove 'v' from chart version Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fix field serviceAccountName Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fix a templating typo There's a small typo in the template which I noticed when I added a custom Secrets volume for the scheduler. The YAML config accidentally works if the typo is on the last line. :) Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * fix celery replica config Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Add quotes around the numbers for celery workers Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Parametrize cpu and mem req/limits for worker statefulset Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fix 'trailing spaces' lint issue Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * added the ability to toggle the scheduler -p option Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * missed the airflow namespace Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * moved command arguments to separate lines Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Headless service name should match statefulset serviceName I had issues where Airflow could not find the log files, so I started looking into it and concluded that everything was working (hostname correct in task instance, could retrieve logs from worker internally) except for the DNS resolution. This commit changes the StatefulSet serviceName suffix to *-worker* instead of *-workers*, so that it matches the Service name. Now all is fine, the logs are shown. Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * compatibility with kube 2.7 Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Configure persistance Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Configure persistance Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Support initcontainer git synchro Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Update configuration description in README Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Adding documentation for persistence.existingClaim Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * airflow.scheduler_do_pickle => dags.donot_pickle Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Injecting ~/.local/bin in PATH so that the commands pip installed can be accessed by a BashOperator Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Force disable xcom pickling will be disabled in airflow 2.0 Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * mkdir ~/.local/bin Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Use real use as base URL Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * more docs examples Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Typo fixes Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * export local PATH before pip install Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Hardcode local path Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * revert BASE_URL Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Reference proper enabled parameter for values.dags.init_container Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * bad order or install in scheduler Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * increase initial delay for webui to 6 min maybe we might want it in the values.yaml? Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fixed some more spots for Values.dags.init_container.enabled Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Doc for donot_pickle Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * notes about num_runs Signed-off-by: Gaetan Semet <gaetan@xeberon.net> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * remove load_examples from example Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * remove load_examples from example Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * airflow: Add support for ingress TLS termination Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * airflow: Allow customisation of worker Pod annotations Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * feature/secrets: add secrets as volume mount Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * feature/secrets: change variable to camel case; add worker secrets; add image pull secret Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * feature/secrets: change variables to camel case Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * feature/secrets: update array spacing Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * feature/secrets: update README Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * feature/secrets: add newline to ingresses Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * feature/secrets: update values Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Update airflow chart with: - RBAC setup - Ability to disable the redis+celery systems and allow for setting an alternative `executor`, such as the kubernetes executor - Wire up the `serviceAccountName` to additional resources - Update to latest airflow release Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Bump versions Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * 'existingClaim' must not be declared for the pvc template to work Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Support setting a custom postgresHost Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fix apps/v1beta1 Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Change minikube values Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fix indendation Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * OWNERS file Signed-off-by: Maciej Bryński <maciej.brynski@xcaliber.com> Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * chore/cleanup: fix spelling Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Move to stable Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * add std labels and component label Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * use template for chart label Signed-off-by: Gaetan Semet <gaetan@xeberon.net> * Fix reference to incubator Signed-off-by: Reinhard Nägele <unguiculus@gmail.com> Signed-off-by: Chaoran Yu <yuchaoran2011@gmail.com>
- Loading branch information