You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened:
I used mutiprocessing Process in PythonOperator callable function, but it reports an error.
File "/home/xxx/airflow/dags/Gen/SecData.py", line 88, in gen
w.start()
File "/usr/lib64/python3.6/multiprocessing/process.py", line 103, in start
'daemonic processes are not allowed to have children'
AssertionError: daemonic processes are not allowed to have children
What you expected to happen:
In airflow1.10, I could use multiprocessing in task function, it works well,
I'm not sure if is't a bug or there are other ways to use multiporcessing in PythonOperator task function.
The text was updated successfully, but these errors were encountered:
Apache Airflow version:
2.0
Environment:
Python3.6
What happened:
I used mutiprocessing Process in PythonOperator callable function, but it reports an error.
File "/home/xxx/airflow/dags/Gen/SecData.py", line 88, in gen
w.start()
File "/usr/lib64/python3.6/multiprocessing/process.py", line 103, in start
'daemonic processes are not allowed to have children'
AssertionError: daemonic processes are not allowed to have children
What you expected to happen:
In airflow1.10, I could use multiprocessing in task function, it works well,
I'm not sure if is't a bug or there are other ways to use multiporcessing in PythonOperator task function.
The text was updated successfully, but these errors were encountered: