Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Improve error handling in workflow compilation when output binding fails #2047

Merged
merged 8 commits into from Jan 31, 2024

Conversation

fg91
Copy link
Member

@fg91 fg91 commented Dec 14, 2023

Why are the changes needed?

@workflow
def wf() -> tuple[str, str, str]:
    return "foo", "bar", None

When registering this workflow, one gets the following obvious error:

~ pyflyte register test_wf.py
...
Python value cannot be None, expected <class 'str'>/<FlyteLiteral simple: STRING>

However, the exception does not hint which workflow/output is causing the error. In a large workflow with numerous sub workflows and dozens of tasks, it took one of our ML engineers quite some time to figure out where exactly the return was missing. They asked whether we could make this simpler.

What changes were proposed in this pull request?

I propose to catch errors raised by binding_from_python_std in the PythonFunctionWorkflow's compile function just as is done e.g. here.
This way, finding the error would have been a no-brainer.

How was this patch tested?

For the example workflow above, the error message now is:

~ pyflyte register test_wf.py
...
Failed to bind output o2 for function test_wf.wf: Python value cannot be None, expected <class 'str'>/<FlyteLiteral simple: STRING>

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com>
Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com>
@fg91 fg91 added the error-message Improve error message label Dec 14, 2023
@fg91 fg91 changed the title Fix: Improve error message in workflow compilation when output binding fails Fix: Improve error handling in workflow compilation when output binding fails Dec 14, 2023
Copy link

codecov bot commented Dec 14, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6cd0a60) 85.98% compared to head (ea81962) 82.53%.
Report is 79 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2047      +/-   ##
==========================================
- Coverage   85.98%   82.53%   -3.46%     
==========================================
  Files         308      313       +5     
  Lines       22946    23602     +656     
  Branches     3468     3535      +67     
==========================================
- Hits        19731    19479     -252     
- Misses       2615     3498     +883     
- Partials      600      625      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com>
Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com>
Signed-off-by: Fabio Graetz <fabiograetz@googlemail.com>
@fg91 fg91 self-assigned this Dec 14, 2023
)
bindings.append(b)
except Exception as e:
raise AssertionError(f"Failed to bind output {output_names[0]} for function {self.name}: {e}") from e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wild-endeavor should these be AssertionErrors? or FlyteExceptions?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used AssertionError in analogy to how it is done here but I'm happy to change to FlyteException if you prefer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do it like this 50ac42f?

@kumare3
Copy link
Contributor

kumare3 commented Jan 2, 2024

one comment else lgtm

Fabio Grätz added 2 commits January 30, 2024 23:00
Signed-off-by: Fabio Grätz <fabiogratz@googlemail.com>
Signed-off-by: Fabio Grätz <fabiogratz@googlemail.com>
@fg91 fg91 requested a review from kumare3 January 30, 2024 22:02
Signed-off-by: Fabio Grätz <fabiogratz@googlemail.com>
@fg91 fg91 merged commit 3b89dc8 into master Jan 31, 2024
83 of 84 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
error-message Improve error message
Projects
None yet
3 participants