Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Set process title for
serve-logs
and LocalExecutor
(#16644)
Follow up of apache/airflow#16623. This PR/commits adds title to serve-logs command and multiprocessing manager for LocalExecutor. The serve-logs process is on celery worker when using CeleryExecutor but for LocalExecutor, it is a separate process in Scheduler. **Before**: ``` root 124 0.0 0.0 6676 4636 pts/1 Ss Jun23 0:00 \_ -bash root 25299 25.3 2.6 988372 326344 pts/1 Sl+ 01:30 0:09 | \_ /usr/local/bin/python /usr/local/bin/airflow webserver root 25510 3.6 0.4 121068 57152 pts/1 S+ 01:31 0:00 | \_ gunicorn: master [airflow-webserver] root 25555 35.7 2.5 983584 316564 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver] root 25556 35.7 2.5 983840 316684 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver] root 25557 35.5 2.5 983840 316548 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver] root 25558 37.2 2.5 984920 317700 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver] root 128 0.0 0.0 6676 4552 pts/2 Ss Jun23 0:00 \_ -bash root 25090 5.8 0.9 467508 118808 pts/2 S+ 01:30 0:03 \_ /usr/local/bin/python /usr/local/bin/airflow scheduler root 25098 0.0 0.7 466080 97800 pts/2 S+ 01:30 0:00 \_ /usr/local/bin/python /usr/local/bin/airflow scheduler root 25099 0.4 0.8 1391812 99788 pts/2 Sl+ 01:30 0:00 \_ /usr/local/bin/python /usr/local/bin/airflow scheduler root 25107 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25109 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25114 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25117 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25120 0.0 0.8 466080 98552 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25125 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor: ['airflow', 'tasks', 'run', 'example_bash_operator', 'run_after_loop', '2021-06-24T01:31:30.507415+00:00', '--lo root 26139 0.0 0.8 468988 102204 pts/2 S+ 01:31 0:00 | \_ airflow task supervisor: ['airflow', 'tasks', 'run', 'example_bash_operator', 'run_after_loop', '2021-06-24T01:31:30.507415+00:00', '--local' root 25128 0.0 0.7 466080 98076 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25132 0.0 0.7 466080 98076 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25137 1.1 0.8 466592 100016 pts/2 S 01:30 0:00 \_ airflow scheduler -- DagFileProcessorManager root@a7c8aa590704:/opt/airflow# ps aux ``` **After**: ``` root 124 0.0 0.0 6676 4636 pts/1 Ss Jun23 0:00 \_ -bash root 25299 25.3 2.6 988372 326344 pts/1 Sl+ 01:30 0:09 | \_ /usr/local/bin/python /usr/local/bin/airflow webserver root 25510 3.6 0.4 121068 57152 pts/1 S+ 01:31 0:00 | \_ gunicorn: master [airflow-webserver] root 25555 35.7 2.5 983584 316564 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver] root 25556 35.7 2.5 983840 316684 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver] root 25557 35.5 2.5 983840 316548 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver] root 25558 37.2 2.5 984920 317700 pts/1 Sl+ 01:31 0:08 | \_ [ready] gunicorn: worker [airflow-webserver] root 128 0.0 0.0 6676 4552 pts/2 Ss Jun23 0:00 \_ -bash root 25090 5.8 0.9 467508 118808 pts/2 S+ 01:30 0:03 \_ /usr/local/bin/python /usr/local/bin/airflow scheduler root 25098 0.0 0.7 466080 97800 pts/2 S+ 01:30 0:00 \_ airflow serve-logs root 25099 0.4 0.8 1391812 99788 pts/2 Sl+ 01:30 0:00 \_ airflow executor -- LocalExecutor root 25107 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25109 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25114 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25117 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25120 0.0 0.8 466080 98552 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25125 0.0 0.8 466080 98548 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor: ['airflow', 'tasks', 'run', 'example_bash_operator', 'run_after_loop', '2021-06-24T01:31:30.507415+00:00', '--lo root 26139 0.0 0.8 468988 102204 pts/2 S+ 01:31 0:00 | \_ airflow task supervisor: ['airflow', 'tasks', 'run', 'example_bash_operator', 'run_after_loop', '2021-06-24T01:31:30.507415+00:00', '--local' root 25128 0.0 0.7 466080 98076 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25132 0.0 0.7 466080 98076 pts/2 S+ 01:30 0:00 \_ airflow worker -- LocalExecutor root 25137 1.1 0.8 466592 100016 pts/2 S 01:30 0:00 \_ airflow scheduler -- DagFileProcessorManager root@a7c8aa590704:/opt/airflow# ps aux ``` GitOrigin-RevId: 1b2535e442599d4607eb6497864af26ad5afa807
- Loading branch information