Conversation
Guide for specific error code:$ airflow error-guide --error-code AERR100
The error you're facing would be with this message: "Invalid DAG structure".
As per our observations, a possible cause could be as follows:
Triggered when a DAG's dependencies or attributes are incorrectly defined.
To resolve this, as first step, you can try the following:
Review the DAG structure for errors. Ensure that all dependencies are correctly defined and that no circular dependencies
exist.
If this doesn't resolve your problem, you can check out the docs for more info:
https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html
You may also ask your questions on the Airflow Slack #user-troubleshooting channel:
https://apache-airflow.slack.com/messages/user-troubleshooting
Happy Debugging! 🐞Listing all exceptions covered in guide:airflow error-guide --list-exceptions
Exception Type
=========================
AirflowApiException
AirflowCliException
AirflowConfigException
AirflowDagCycleException
AirflowDagImportException
AirflowDagNotFound
AirflowDagPausedException
AirflowDatabaseException
AirflowException
AirflowExecutorException
AirflowParseException
AirflowSchedulerException
AirflowTaskException
AirflowTaskTimeout
AirflowTemplateException
AirflowTriggerException
AirflowWebServerException
AirflowWorkerException
AirflowXComException
AttributeError
FileNotFoundError
ImportError
KeyError
ModuleNotFoundError
PermissionError
UnpicklingError
ValueError Dump entire guide as yaml:airflow error-guide --list-guide
AERR001:
description: Happens when dynamically mapped tasks exceed the maximum number of
tasks allowed.
documentation: https://airflow.apache.org/docs/apache-airflow/stable/concepts/dynamic-task-mapping.html
error_message: Dynamic task mapping exceeded limit
exception_type: AirflowException
first_steps: Check the task count limit in the configuration. Consider increasing
the task limit or optimizing task mapping logic.
AERR002:
description: Happens when the scheduler or webserver cannot locate a task instance
in the database.
documentation: https://airflow.apache.org/docs/apache-airflow/stable/administration/task-instances.html
error_message: Task instance not found
exception_type: AirflowException
first_steps: Verify that the database connection is stable. Check if the task instances
exist in the metadata database and consider re-running the DAG.
...truncated |
2ef45ed to
44ff5f8
Compare
44ff5f8 to
1c5a25b
Compare
1c5a25b to
1f5637c
Compare
|
There is some chance we will look at it as one of the first things after all 'critical' airflow 3 things are done. So do not loose your faith :) |
Sure got it, I'll try my best to complete this PR till then. Thanks for the assurance, appreciate it :) |
efc4b9d to
2c37ce3
Compare
2c37ce3 to
6cb35b3
Compare
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions. |
|
@omkar-foss -> should we revive it ? I thought it was a cool idea and we should make it happen eventually |
|
Should this be in airflowctl instead? It does not need anything specific from core. |
The idea was that (eventually) those constants will be used when any exception is raised - so they would eventually have to be imported from somewhere. For me that is likely a better candidate for "shared/errors" library rather than "airfow-ctl" code. |
Sure, let's revive this. We could continue with our initial thought of having airflowctl 'error-guide' command in this PR could be a way for users to access error id info easily (without looking at yaml file), but that'd be after we've the main error id framework in place. @potiuk wdyt? |
6cb35b3 to
de4ef8b
Compare
closes: #43171
related: #44616
This PR adds the
airflow error-guidecommand as per the plan in this comment.