New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python os.listdir fails with FileNotFoundError when directory exists #80424
Comments
I have the following code:
Occasionally, the call to os.listdir(dir) fails with FileNotFoundError even though the os.path.exists(dir) call says that exists:
I've also seen a similar FileNoteFound on the
line when os.path.exists(dir) says that the directory exists. How can this happen? I'm running Python 3.7.2 64-bit on Windows 10. |
Easily -- this looks like a "Time Of Check To Time Of Use" bug. You check for the existence of a directory, and then a fraction of a second later you attempt to use that directory. But on a multi-processing operating system like Windows, Mac or Linux, a lot can happen in that fraction of a second, including the directory being deleted by another process. https://en.wikipedia.org/wiki/Time_of_check_to_time_of_use I'm not saying that this absolutely cannot be a bug in Python, but the initial indication is that this is simply a TOCTTOU bug in your code. Unfortunately, your code is unrunnable by us, as it if full of mysterious objects like "shouter.shout" which we have no access too. In addition, there's a problem that "handle_empty_directories" calls itself recursively as if it were a method of some "Commiter" object: Commiter.handle_empty_directories but it has no "self" parameter, so I don't know what is going on there. For us to investigate further, you need to replicate the error using much simple code that we can run ourselves. Please read http://www.sscce.org/ for some hints in producing short, self-contained, runnable code so that we can replicate this error. In the meantime, I'm going to change this to Pending. When you have replicated this with simpler code, please feel free to change the status back to Open. |
This problem does not appear to be a race condition ("Time Of Check To Time Of Use" bug) in my case as the directory in question exists both before and after the os.listdir call. I got a workaround saying to wrap the os.listdir call in try/except in my Stack Overflow question at https://stackoverflow.com/questions/55067904/python-os-listdir-fails-with-filenotfounderror-when-directory-exists. This worked as no longer got any spurious FileNotFound exceptions. I realize that the code snippet I gave is not runnable. I'll look to put together a small example illustrating the problem and add to the issue. |
Geoff, it's been over two years. Do you think there is a chance you will want to follow up with a reproducer, or shall we close this issue? |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: