Skip to content

Don't panic if we cannot obtain connection attemp result#2880

Merged
tillrohrmann merged 1 commit intorestatedev:mainfrom
tillrohrmann:issues/2879
Mar 12, 2025
Merged

Don't panic if we cannot obtain connection attemp result#2880
tillrohrmann merged 1 commit intorestatedev:mainfrom
tillrohrmann:issues/2879

Conversation

@tillrohrmann
Copy link
Copy Markdown
Contributor

For logging purposes we try to obtain the result of the connection attempt in a synchronous function if the connection attempt task has finished. Unfortunately, we cannot rely on TaskHandle::is_finished for task_handle.now_or_never() to always return Some(...). The problem is that the underlying Tokio JoinHandle participates in the cooperative task scheduling and if we ran out of budget, then it would return Poll::Pending. That's why we ignore the result in this case and continue.

This fixes #2879.

} else {
match task_handle.now_or_never().expect("should be finished") {
Ok(result) => match result {
match occupied.remove().now_or_never() {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also use unconstrained() but it's always safer to not panic anyway.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I'll remember it for the future :-)

Copy link
Copy Markdown
Contributor

@AhmedSoliman AhmedSoliman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming that the failing test is unrelated

@tillrohrmann
Copy link
Copy Markdown
Contributor Author

LGTM assuming that the failing test is unrelated

I think that the test failure is unrelated but I haven't found an explanation yet. I've created #2885 for investigating it. So far I can only say that somehow the system doesn't see a Logs version update which should happen and therefore the system is stuck at waiting for the reconfiguration.

For logging purposes we try to obtain the result of the connection attempt
in a synchronous function if the connection attempt task has finished.
Unfortunately, we cannot rely on TaskHandle::is_finished for task_handle.now_or_never()
to always return Some(...). The problem is that the underlying Tokio JoinHandle participates
in the cooperative task scheduling and if we ran out of budget, then it would return
Poll::Pending. That's why we ignore the result in this case and continue.

This fixes restatedev#2879.
@tillrohrmann tillrohrmann merged commit 38772b4 into restatedev:main Mar 12, 2025
26 checks passed
@tillrohrmann tillrohrmann deleted the issues/2879 branch March 12, 2025 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TaskHandle::is_finished == true does not mean that TaskHandle is Poll::Ready

2 participants