Fix: Preserve Pydantic details when serialization fails#59401
Merged
edoakes merged 2 commits intoray-project:masterfrom Dec 16, 2025
Merged
Conversation
mgchoi239
force-pushed
the
fix/pydantic-validation-error-serialization
branch
4 times, most recently
from
83d2a7b to
9c1916b
Compare
ray-gardener
bot
added
core
edoakes
reviewed
Dec 12, 2025
Collaborator
edoakes
left a comment
edoakes
left a comment
There was a problem hiding this comment.
looks good, just one minor comment
python/ray/exceptions.py
Outdated
Comment on lines
148
to
151
| logger.warning( | ||
| f"The original cause of the RayTaskError ({err_type}) " | ||
| f"isn't serializable: {e}. Preserving error details." | ||
| ) |
Collaborator
There was a problem hiding this comment.
Suggested change
| logger.warning( | |
| f"The original cause of the RayTaskError ({err_type}) " | |
| f"isn't serializable: {e}. Preserving error details." | |
| ) | |
| logger.exception( | |
| f"The original cause of the RayTaskError ({err_type}) couldn't be serialized." | |
| ) |
Collaborator
There was a problem hiding this comment.
no need for "preserving error details", and logger.exception will include and format stack trace automatically
Contributor
Author
There was a problem hiding this comment.
Thanks for the review. Pushed the new change
Collaborator
|
@eicherseiji PTAL to help validate this solves your original issue |
…ion fails Fixes ray-project#59357 ## Description The Pydantic error is not serializable, making it difficult to root cause the bug. When a Pydantic ValidationError containing ArgsKwargs objects cannot be serialized, Ray now preserves the original error details including the ValidationError message and full traceback in the error message. ## Details Modified RayTaskError.__init__ to include the full traceback_str in the error message when serialization fails, allowing users to see the original ValidationError details even when the exception itself cannot be pickled. Signed-off-by: mgchoi239 <mg.choi.239@example.com>
mgchoi239
force-pushed
the
fix/pydantic-validation-error-serialization
branch
from
58b7869 to
8285b9f
Compare
Contributor
|
Hi @mgchoi239, thanks for the PR! The change works well. However, further investigation showed that #59357 root cause was missing ServeController logs. When the ServeController logs are present, Serve correctly surfaces the Pydantic validation error. In particular, it's not immediately clear why the ServeController logs were missing from the Ray service stdout in the KubeRay setup used for #59357, and launched identically with tl;dr: in the case where ServeController logs are for some reason missing, this change would immediately solve my debugging issue. But in general, users should not need this. @edoakes with that context I will let you make the call on this change. I will update the original issue. |
mgchoi239
force-pushed
the
fix/pydantic-validation-error-serialization
branch
from
3620183 to
508d609
Compare
edoakes
approved these changes
Dec 15, 2025
Collaborator
|
There was a test failure, likely due to the exception message changing: https://buildkite.com/ray-project/premerge/builds/55778#019b1927-7b54-4c26-8738-eb1f9117a6ae/622-1360 |
…dation-error-serialization
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
mgchoi239
force-pushed
the
fix/pydantic-validation-error-serialization
branch
from
9290aca to
eb6dc29
Compare
cszhu
pushed a commit
that referenced
this pull request
Dec 17, 2025
Fixes #59357 ## Description The Pydantic error is not serializable, making it difficult to root cause the bug. When a Pydantic ValidationError containing ArgsKwargs objects cannot be serialized, Ray now preserves the original error details including the ValidationError message and full traceback in the error message. ## Details Modified RayTaskError.__init__ to include the full traceback_str in the error message when serialization fails, allowing users to see the original ValidationError details even when the exception itself cannot be pickled. ## Testing - Created test script that verifies original error details are preserved - Confirmed wrapped error is serializable - Verified original ValidationError information survives round-trip serialization Signed-off-by: mgchoi239 <mg.choi.239@example.com> Co-authored-by: MG <mg@MGs-MacBook-Air.local> Co-authored-by: mgchoi239 <mg.choi.239@example.com>
zzchun
pushed a commit
to zzchun/ray
that referenced
this pull request
Dec 18, 2025
…59401) Fixes ray-project#59357 ## Description The Pydantic error is not serializable, making it difficult to root cause the bug. When a Pydantic ValidationError containing ArgsKwargs objects cannot be serialized, Ray now preserves the original error details including the ValidationError message and full traceback in the error message. ## Details Modified RayTaskError.__init__ to include the full traceback_str in the error message when serialization fails, allowing users to see the original ValidationError details even when the exception itself cannot be pickled. ## Testing - Created test script that verifies original error details are preserved - Confirmed wrapped error is serializable - Verified original ValidationError information survives round-trip serialization Signed-off-by: mgchoi239 <mg.choi.239@example.com> Co-authored-by: MG <mg@MGs-MacBook-Air.local> Co-authored-by: mgchoi239 <mg.choi.239@example.com>
Yicheng-Lu-llll
pushed a commit
to Yicheng-Lu-llll/ray
that referenced
this pull request
Dec 22, 2025
…59401) Fixes ray-project#59357 ## Description The Pydantic error is not serializable, making it difficult to root cause the bug. When a Pydantic ValidationError containing ArgsKwargs objects cannot be serialized, Ray now preserves the original error details including the ValidationError message and full traceback in the error message. ## Details Modified RayTaskError.__init__ to include the full traceback_str in the error message when serialization fails, allowing users to see the original ValidationError details even when the exception itself cannot be pickled. ## Testing - Created test script that verifies original error details are preserved - Confirmed wrapped error is serializable - Verified original ValidationError information survives round-trip serialization Signed-off-by: mgchoi239 <mg.choi.239@example.com> Co-authored-by: MG <mg@MGs-MacBook-Air.local> Co-authored-by: mgchoi239 <mg.choi.239@example.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #59357
Description
The Pydantic error is not serializable, making it difficult to root cause the bug. When a Pydantic ValidationError containing ArgsKwargs objects cannot be serialized, Ray now preserves the original error details including the ValidationError message and full traceback in the error message.
Details
Modified RayTaskError.init to include the full traceback_str in the error message when serialization fails, allowing users to see the original ValidationError details even when the exception itself cannot be pickled.
Testing