gh-154194: Do not fail the whole sample in Tachyon if we can degrade#154195
gh-154194: Do not fail the whole sample in Tachyon if we can degrade#154195maurycy wants to merge 2 commits into
Conversation
|
I believe that the CI failures are caused by the fact that |
pablogsal
left a comment
pablogsal
left a comment
There was a problem hiding this comment.
I still need more time but some nits
|
|
||
| Py_ssize_t len = GET_MEMBER(Py_ssize_t, unicode_obj, unwinder->debug_offsets.unicode_object.length); | ||
| if (len < 0 || len > max_len) { | ||
| if (len < 0 || len > MAX_REMOTE_READ) { |
There was a problem hiding this comment.
This rejects valid task names above 256 KiB instead of truncating them, so async sampling can still fail. We should cap read_len, not len, and test this boundary.
| @@ -0,0 +1,3 @@ | |||
| Fix the Tachyon dropping entire samples when non-fatal reads fail. Frames | |||
There was a problem hiding this comment.
Nit: the wording reads a bit off. Maybe Fix the sampling profiler dropping entire samples when a non-fatal read fails; frames now keep any readable metadata, and long funcnames, filenames and asyncio task names are truncated instead.
2 participants
As always, see #154194 for the context. Shortly: right now remote inspection fails the whole sample when it hits the limits (and we're really not sure what are the best limits) OR always when reading a part fails. This is too strict, wasteful and introduces scary bias, as observed in https://discuss.python.org/t/tachyon-97-error-rate/107619.
The PR introduces graceful degradation with the sentinels like
<unreadable frame>(funcname),<unknown function>(funcname),<unknown file>(filename) and the(len=N)suffix if we hit the cap. Also, it introducesMAX_REMOTE_READas a safety check, and attemps to better differentiate between transient failures in_Py_RemoteDebug_IsFatalReadError().Importantly: degraded frames are never cached, while the truncated strigs are cached.
Fixes #154194
_remote_debugging: Degrade gracefully reads instead of failing the whole read completely #154194