Error in my Airflow pipeline when it run Spark job #32618
Unanswered
aalshikhley
asked this question in
Q&A
Replies: 1 comment 3 replies
-
Is this issue resolved? Could you please post the solution if it is resolved. |
Beta Was this translation helpful? Give feedback.
3 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello !
I'm beginner with airflow and spark and I am currently setting up a data pipeline locally using Airflow and Spark.
The DAG I want to do has just one task that runs a pyspark job on Spark.
The dags folder of my application contains two files:
dag-airflow-spark-submitop.py :
sparkjob.py :
However, when I run my DAG I get this error:
Can you help me to resolve it please ?
Beta Was this translation helpful? Give feedback.
All reactions