Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trigger gevent monkeypatching via environment variable #28283

Merged
merged 1 commit into from
Dec 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 9 additions & 0 deletions airflow/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@
import sys
from typing import Callable

if os.environ.get("_AIRFLOW_PATCH_GEVENT"):
# If you are using gevents and start airflow webserver, you might want to run gevent monkeypatching
# as one of the first thing when Airflow is started. This allows gevent to patch networking and other
# system libraries to make them gevent-compatible before anything else patches them (for example boto)
from gevent.monkey import patch_all

patch_all()

from airflow import settings

__all__ = ["__version__", "login", "DAG", "PY36", "PY37", "PY38", "PY39", "PY310", "XComArg"]
Expand All @@ -41,6 +49,7 @@
# lib.)
__path__ = __import__("pkgutil").extend_path(__path__, __name__) # type: ignore


# Perform side-effects unless someone has explicitly opted out before import
# WARNING: DO NOT USE THIS UNLESS YOU REALLY KNOW WHAT YOU'RE DOING.
if not os.environ.get("_AIRFLOW__AS_LIBRARY", None):
Expand Down
4 changes: 3 additions & 1 deletion airflow/config_templates/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1233,7 +1233,9 @@
- name: worker_class
description: |
The worker class gunicorn should use. Choices include
sync (default), eventlet, gevent
sync (default), eventlet, gevent. Note when using gevent you might also want to set the
"_AIRFLOW_PATCH_GEVENT" environment variable to "1" to make sure gevent patching is done as
early as possible.
version_added: ~
type: string
example: ~
Expand Down
4 changes: 3 additions & 1 deletion airflow/config_templates/default_airflow.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -640,7 +640,9 @@ secret_key = {SECRET_KEY}
workers = 4

# The worker class gunicorn should use. Choices include
# sync (default), eventlet, gevent
# sync (default), eventlet, gevent. Note when using gevent you might also want to set the
# "_AIRFLOW_PATCH_GEVENT" environment variable to "1" to make sure gevent patching is done as
# early as possible.
worker_class = sync

# Log files for the gunicorn webserver. '-' means log to stderr.
Expand Down
1 change: 1 addition & 0 deletions newsfragments/08212.misc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
If you are using gevent for your webserver deployment and used local settings to monkeypatch gevent, you might want to replace local settings patching with an ``_AIRFLOW_PATCH_GEVENT`` environment variable set to 1 in your webserver. This ensures gevent patching is done as early as possible.