Skip to content

fix: report failed task.#315

Merged
k82cn merged 2 commits into
xflops:mainfrom
k82cn:report_failed_task
Jan 20, 2026
Merged

fix: report failed task.#315
k82cn merged 2 commits into
xflops:mainfrom
k82cn:report_failed_task

Conversation

@k82cn
Copy link
Copy Markdown
Contributor

@k82cn k82cn commented Jan 19, 2026

No description provided.

Signed-off-by: Klaus Ma <klausm@nvidia.com>
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @k82cn, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the system's ability to handle and report task failures originating from exceptions within services. By ensuring that detailed error messages are captured and logged when a task fails, it enhances debugging capabilities and provides clearer insights into the root cause of issues. A new end-to-end test validates this improved error handling, alongside minor code quality enhancements for the e2e test suite.

Highlights

  • Enhanced Task Failure Reporting: Exceptions raised within service on_task_invoke methods are now properly captured, logged, and reflected in the task's FAILED state and event messages.
  • New End-to-End Test for Error Handling: A dedicated test service and test case have been added to validate the robust reporting of failed tasks, ensuring error messages are correctly propagated and recorded.
  • Code Quality Tooling for E2E: black and isort have been integrated into the e2e development dependencies to maintain consistent code formatting.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves error handling by ensuring that when a task fails due to an exception in the service, it is correctly reported with a FAILED state and the corresponding error message is captured in the task's events. Previously, this would result in an internal error. The changes involve modifying the gRPC shim to handle non-zero return codes by creating a TaskResult with a Failed state, and updating the storage layer to record the error message from failed tasks into the event log. To validate this fix, a new end-to-end test has been added, which uses a dedicated test service that intentionally raises an exception.

My review includes a couple of suggestions for the new test code to improve its robustness and readability, such as avoiding redundant conditions and replacing unsafe bare except blocks.

Comment thread e2e/tests/test_core.py
Comment on lines +437 to +446
try:
sessions = flamepy.list_sessions()
for sess in sessions:
try:
if sess.application == FLM_ERROR_SVC_APP:
flamepy.close_session(sess.id)
except:
pass
except:
pass
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using bare except: pass is a dangerous pattern as it silences all exceptions, including system-exiting ones like KeyboardInterrupt and SystemExit. This can hide significant issues during test teardown and make debugging very difficult. It's recommended to catch specific exceptions (e.g., flamepy.FlameError) or at least Exception to avoid unintentionally suppressing critical errors. Consider logging the exception instead of just passing.

Suggested change
try:
sessions = flamepy.list_sessions()
for sess in sessions:
try:
if sess.application == FLM_ERROR_SVC_APP:
flamepy.close_session(sess.id)
except:
pass
except:
pass
try:
sessions = flamepy.list_sessions()
for sess in sessions:
try:
if sess.application == FLM_ERROR_SVC_APP:
flamepy.close_session(sess.id)
except Exception:
pass
except Exception:
pass

Comment thread e2e/tests/test_core.py
error_events = [
e
for e in refreshed_task.events
if e.code == flamepy.TaskState.FAILED or e.code == 3
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The condition e.code == flamepy.TaskState.FAILED or e.code == 3 is redundant. flamepy.TaskState.FAILED is an IntEnum member with a value of 3. Using only the enum member is sufficient and improves readability.

Suggested change
if e.code == flamepy.TaskState.FAILED or e.code == 3
if e.code == flamepy.TaskState.FAILED

Signed-off-by: Klaus Ma <klausm@nvidia.com>
@k82cn k82cn merged commit 02c01a1 into xflops:main Jan 20, 2026
4 checks passed
@k82cn k82cn deleted the report_failed_task branch January 20, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant