Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix gantt chart queued duration when queued_dttm is greater than start_date for deferred tasks. #35984

Merged
merged 2 commits into from Dec 5, 2023

Conversation

tirkarthi
Copy link
Contributor

@tirkarthi tirkarthi commented Nov 30, 2023

closes: #35288
related: #35288

When a task is deferred then the same attempt is used to queue and resume causing the queued_dttm to be greater than start_date for an execution. This causes queued duration to be displayed with incorrect value taking queued_dttm - start_date which is not correct since start_date should always be greater than queued time. Also this causes start offset and to have a fixed queued duration bar width of 5. start offset and queued duration min width are shown only when queued_dttm is valid by being greater than start_date.

Before patch : (Incorrect queued duration of 23 hours and start of the bar is incorrect)

gantt_before_patch

After patch : (queued duration is 0 since queued_dttm is greater than start_date. start_date position is also correct in gantt as per start_date)

gantt_after

Sample dag :

from __future__ import annotations

from datetime import datetime

from airflow import DAG
from airflow.models.baseoperator import BaseOperator
from airflow.triggers.file import FileTrigger


class FileCheckOperator(BaseOperator):
    def __init__(self, filepath, **kwargs):
        self.filepath = filepath
        super().__init__(**kwargs)

    def execute(self, context):
        self.defer(
            trigger=FileTrigger(filepath=self.filepath),
            method_name="execute_complete",
        )

    def execute_complete(self, context, event=None):
        pass


with DAG(
    dag_id="file_trigger",
    start_date=datetime(2021, 1, 1),
    catchup=False,
    schedule_interval=None,
) as dag:
    t1 = FileCheckOperator(task_id="t1", filepath="/tmp/a")
    t2 = FileCheckOperator(task_id="t2", filepath="/tmp/b")

    t1
    t2

@boring-cyborg boring-cyborg bot added area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues labels Nov 30, 2023
@bbovenzi bbovenzi added this to the Airflow 2.8.1 milestone Dec 5, 2023
@bbovenzi bbovenzi merged commit 0376e93 into apache:main Dec 5, 2023
47 checks passed
@ephraimbuddy ephraimbuddy added the type:bug-fix Changelog: Bug Fixes label Dec 6, 2023
ephraimbuddy pushed a commit that referenced this pull request Dec 6, 2023
…t_date for deferred tasks. (#35984)

* Fix gantt chart queued duration when queued_dttm is greater than start_date for deferred tasks.

* Merge hasQueuedDttm and validQueuedDttm.

(cherry picked from commit 0376e93)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:UI Related to UI/UX. For Frontend Developers. area:webserver Webserver related Issues type:bug-fix Changelog: Bug Fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Incorrect queued duration for deferred tasks in gantt view
3 participants