Use the ~airflow.operators.bash_operator.BashOperator
to execute commands in a Bash shell.
../../../airflow/example_dags/example_bash_operator.py
You can use Jinja templates <jinja-templating>
to parameterize the bash_command
argument.
../../../airflow/example_dags/example_bash_operator.py
Warning
Care should be taken with "user" input or when using Jinja templates in the bash_command
, as this bash operator does not perform any escaping or sanitization of the command.
This applies mostly to using "dag_run" conf, as that can be submitted via users in the Web UI. Most of the default template variables are not at risk.
For example, do not do this:
bash_task = BashOperator(
task_id="bash_task",
bash_command='echo "Here is the message: \'{{ dag_run.conf["message"] if dag_run else "" }}\'"',
)
Instead, you should pass this via the env
kwarg and use double-quotes inside the bash_command, as below:
bash_task = BashOperator(
task_id="bash_task",
bash_command='echo "here is the message: \'$message\'"',
env={'message': '{{ dag_run.conf["message"] if dag_run else "" }}'},
)
Add a space after the script name when directly calling a Bash script with the bash_command
argument. This is because Airflow tries to apply a Jinja template to it, which will fail.
t2 = BashOperator(
task_id='bash_example',
# This fails with 'Jinja template not found' error
# bash_command="/home/batcher/test.sh",
# This works (has a space after)
bash_command="/home/batcher/test.sh ",
dag=dag)