Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tokio::fs::DirEntry::path returns type that does blocking IO #5192

Closed
WhyNotHugo opened this issue Nov 14, 2022 · 6 comments
Closed

tokio::fs::DirEntry::path returns type that does blocking IO #5192

WhyNotHugo opened this issue Nov 14, 2022 · 6 comments
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-fs Module: tokio/fs

Comments

@WhyNotHugo
Copy link

This function:

https://docs.rs/tokio/latest/tokio/fs/struct.DirEntry.html#method.path

Returns this type:

https://doc.rust-lang.org/nightly/std/path/struct.PathBuf.html

But this type's methods do blocking IO (e.g.: .exists()). This completely messes up services which use higher latency network mounts by freezing async threads with blocking IO when doing disk access.

@WhyNotHugo WhyNotHugo added A-tokio Area: The main tokio crate C-bug Category: This is a bug. labels Nov 14, 2022
@WhyNotHugo
Copy link
Author

WhyNotHugo commented Nov 15, 2022

The way async-std works around this is by implementing their own async_std::path::PathBuf type that does async IO. The API has its caveats, but it's still worth mentioning as prior art.

@davidbarsky
Copy link
Member

To be clear, async-std does not perform async IO when doing operations like PathBuf::exists(). Instead, it spawns file IO-related operations onto a dedicated threadpool, which is what Tokio would recommend you do anyways if you know a certain operation might block the executor.

In non-networked file systems, checking whether a file exists wouldn't really block the executor because most file systems are able to return that information pretty quickly, and I'm not entirely convinced that it makes sense to orient Tokio's API around the (relatively niche) networked file system use-case.

@carllerche
Copy link
Member

PathBuf is a pretty foundational buffer type (wrapper around [u8]). Many fns take &Path as an argument w/o doing any FS access. The fact that std includes the exists() method on it is unfortunate.

Regardless, it isn't anything we can change.

@Darksonn Darksonn added the M-fs Module: tokio/fs label Nov 15, 2022
@WhyNotHugo
Copy link
Author

I was thinking of perhaps adding a small note/warning in https://docs.rs/tokio/latest/tokio/fs/index.html, that care should be taken when using methods on PathBufs and other returned types which may have blocking IO.

Thoughts?

@WhyNotHugo
Copy link
Author

I mean, I figured it out and workarounds, but mostly want to make things simpler for the next person stumbling upon this.

@Darksonn
Copy link
Contributor

I would be happy to see more documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-fs Module: tokio/fs
Projects
None yet
Development

No branches or pull requests

4 participants