Airflow should be more explicit about unmet dependencies

[kretes]

When running a task from airflow run I got in the logs:

[2016-02-08 16:01:07,745] {models.py:920} WARNING - Dependencies not met yet

I claim that dependencies are met because they were executed before, but maybe airflow is talking about some other dependency. The message should include which dependency was not met.

You may say it is a low-priority issue, but one of the traits of an easy-to-use software is that it expresses the state of the data and helps users fix issues.

[r39132]

Hi!
Dependencies Not Met implies that some pre-conditions for a run to be scheduled were not met.

http://pythonhosted.org/airflow/_modules/airflow/models.html#TaskInstance.are_dependencies_met

For example, if a previous day's run did not complete successfully and if your DAG were not to specify ``depends_on_past': True` as shown in the example below:

default_args = {
    'owner': 'sanand',
    'depends_on_past': True,
    'pool': 'ep_data_pipeline',
    'start_date': START_DATE,
    'email': [import_ep_pipeline_alert_email_dl],
    'email_on_failure': import_airflow_enable_notifications,
    'email_on_retry': import_airflow_enable_notifications,
    'retries': 1,
    'retry_delay': timedelta(seconds=120),
    'priority_weight': import_airflow_priority_weight
}
dag = DAG(DAG_NAME, schedule_interval='@hourly', default_args=default_args, sla_miss_callback=sla_alert_func)

Then, my executing airflow run my_dag my_task 2016-02-07 would show the above error if airflow run my_dag my_task 2016-02-06 did not complete successfully.

It would help if you could provide a detailed example of your issue, so I could better answer it or determine if there were some buggy behavior.

[mistercrunch]

Yeah it's a bit tricky since are_dependencies_met is used in many contexts (the scheduler for instance) and if we were to have more verbose logging in that method it would generate too much logs.

I can add logging but would have to either pass a new verbose arg to the function so that it's only verbose when needed. PR coming up.

[mistercrunch]

This has been bugging me for a while. Finally got to it, thanks for reporting
#976