Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

io: fix possible I/O resource hang #6134

Merged
merged 2 commits into from Nov 8, 2023
Merged

io: fix possible I/O resource hang #6134

merged 2 commits into from Nov 8, 2023

Conversation

carllerche
Copy link
Member

Use a per-resource tick instead of a single global tick counter. This prevents the hang issue described by #6133.

As a follow-up, I would like to increase the number of bits used to track the tick in ScheduledIo. Because this change is not necessary to fix the original issue, I am punting.

Fixes: #6133

Use a per-resource tick instead of a single global tick counter. This
prevents the hang issue described by #6133.

Fixes: #6133
@Darksonn Darksonn added A-tokio Area: The main tokio crate M-io Module: tokio/io labels Nov 8, 2023
@carllerche carllerche merged commit 30b2eb1 into master Nov 8, 2023
76 checks passed
@carllerche carllerche deleted the io-fix-hang branch November 8, 2023 22:05
kodiakhq bot pushed a commit to pdylanross/fatigue that referenced this pull request Nov 10, 2023
Bumps tokio from 1.33.0 to 1.34.0.

Release notes
Sourced from tokio's releases.

Tokio v1.34.0
Fixed

io: allow clear_readiness after io driver shutdown (#6067)
io: fix integer overflow in take (#6080)
io: fix I/O resource hang (#6134)
sync: fix broadcast::channel link (#6100)

Changed

macros: use ::core qualified imports instead of ::std inside tokio::test macro (#5973)

Added

fs: update cfg attr in fs::read_dir to include aix (#6075)
sync: add mpsc::Receiver::recv_many (#6010)
tokio: added vita target support (#6094)

#5973: tokio-rs/tokio#5973
#6067: tokio-rs/tokio#6067
#6080: tokio-rs/tokio#6080
#6134: tokio-rs/tokio#6134
#6100: tokio-rs/tokio#6100
#6075: tokio-rs/tokio#6075
#6010: tokio-rs/tokio#6010
#6094: tokio-rs/tokio#6094



Commits

49eb26f chore: prepare Tokio v1.34.0 release (#6138)
19d96c0 io: increase ScheduledIo tick resolution (#6135)
30b2eb1 io: fix possible I/O resource hang (#6134)
8ec3e0d metrics: update stats when unparking in multi-thread (#6131)
161ecec stream: fix typo in peekable docs (#6130)
61fcc3b time: remove cached elapsed value from driver state (#6097)
944024e chore: update rust-version to 1.63 in all crates (#6126)
65f861f stream: add StreamExt::peekable (#6095)
4c85801 ci: fix docs on latest nightly (#6120)
ed32cd1 task: add tests for tracing instrumentation of tasks (#6112)
Additional commits viewable in compare view




Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
@ThinkRedstone
Copy link

ThinkRedstone commented Nov 15, 2023

Hi, I'm not well-versed with mio, but as far as I can discern from the mio docs, there is no guarantee that an object that has generated a 'ready' event and was not interacted with will not continue to generate 'ready' events on subsequent polls (the documentation warns that you can't rely on this behaviour, which might mean that this behaviour is possible) . If that is true, then this change only makes the race less likely- moving the tick counter to be per ScheduledIo means that if mio generates enough events for a specific ScheduledIo it can cause its tick overflow, and this bug will still occur.

Context: We've ran into this bug in the wild in with our code. When researching it I came across this PR, and I thought of this issue while verifying that this PR matches the bug we ran into.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate M-io Module: tokio/io
Projects
None yet
Development

Successfully merging this pull request may close these issues.

io: I/O resources can hang in rare cases
3 participants