-
Notifications
You must be signed in to change notification settings - Fork 14k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New dags fail to run #7872
Comments
Thanks for opening your first issue here! Be sure to follow the issue template! |
This is the expected behavior. The Scheduler needs to do various validation before it can execute tasks and hence it needs the DAG to be in the database and the database has the needed tables (and completed migrations). |
So when adding a new dag to airflow the process is: add dag to repo/directory, reinitialize the db, then run? That wasn't the process before as I said in 1.10.5, in that version you just need to add the dag to the repo/directory and it was possible to run it right away... |
Not reinitialize if you have already initialized it. I don't think we had that in 1.10.5, can you try to reproduce that using 1.10.5 in a different VirtualEnv with a different AIRFLOW_HOME and let me know. I think your DB might have been initialized before that is the reason it worked for 1.10.5 but happy to hear what you find. |
I see. So that same behavior does happen in 1.10.5. If the scheduler is running it works fine, the scheduler must add it to the db for me? |
I agreed with cat buddy, the error message is not clear what is going on really. |
Apache Airflow version: 1.10.9
Kubernetes version (if you are using kubernetes) (use
kubectl version
): N/AEnvironment: JupyterLab docker image, Ubuntu 18.04 VM
Cloud provider or hardware configuration: Microsoft Azure
OS (e.g. from /etc/os-release): Ubuntu 18.04
Kernel (e.g.
uname -a
): Linux 5.0.0-1032-azure Carry ignore_dependencies from backfill to run commands #34-Ubuntu SMP Mon Feb 10 19:37:25 UTC 2020 x86_64 x86_64 x86_64 GNU/LinuxInstall tools: apache-airflow
Others:
What happened:
On a fresh install of airflow, I run
airflow initdb
, and then create a dag (bash.py)and then
airflow list_dags
will show that the dag exists, and if you runairflow trigger_dag date
it will give you the following errorAnd it won't run. Now run
airflow initdb
and then trigger the dag again, and it works fine.What you expected to happen: Should just be able to run the dag without initializing the db again...
How to reproduce it: see above
Anything else we need to know: The other version we're running is 1.10.5, and this issue doesn't seem to happen there. Once you run
airflow initdb
after the dag, it'll run always. But add a new dag and trigger it and the same thing occurs.The text was updated successfully, but these errors were encountered: