TriggerDagRunOperator got AirflowException when assign queue name into queue parameter

[Rixkz]

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

I used TriggerDagRunOperator for a while, it's wok fine, but now I need to use parameter "queue" to make my trigger task sending message to queue name "submit_dag_queue", when I used it, I've got some strange error.

[2022-12-14 13:25:00,729: INFO/ForkPoolWorker-15] [***] Executing command in Celery: ['airflow', 'tasks', 'run', '***', '***', 'manual__2022-12-14T13:24:56.989463+00:00', '--local', '--subdir', 'DAGS_FOLDER/dags/contoller_dag/***.py']
[2022-12-14 13:25:02,930: ERROR/ForkPoolWorker-15] Task airflow.executors.celery_executor.execute_command[***] raised unexpected: AirflowException('Celery command failed on host: ip-*** with celery_task_id ***')
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.10/site-packages/celery/app/trace.py", line 451, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/celery/app/trace.py", line 734, in __protected_call__
return self.run(*args, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/celery_executor.py", line 96, in execute_command
_execute_in_fork(command_to_exec, celery_task_id)
File "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/celery_executor.py", line 111, in _execute_in_fork
raise AirflowException(msg)
airflow.exceptions.AirflowException: Celery command failed on host: ip-*** with celery_task_id ***

Here, my code example.

trigger_fact_xxx = TriggerDagRunOperator(
    task_id="trigger_xxx_dag",
    trigger_dag_id="fact_xxx",
    wait_for_completion=True,
    conf=conf,
    pre_execute=check_runnable_status,
    pool=trigger_pool_name,
    queue="submit_dag_queue",
    trigger_rule="none_failed",
)

Let me know if I understand wrong about how to use it.

What you think should happen instead

No response

How to reproduce

No response

Operating System

I used docker image : apache/airflow:2.4.0-python3.10

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

I used redis as a message broker.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!

[potiuk]

Queue is not a parameter of where to TriggerDag but which is the clery queue your "TiggerDagRun" task is running. The stack trace of yours does not show the real error - you shoudl see how your Celery is configured and whether it has the queue defined properly. https://airflow.apache.org/docs/apache-airflow/stable/executor/celery.html#queues