-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
isfile
error on too long file names
#39774
Comments
It's actually system dependent. I believe that silently returning false can cause inexplicable bugs. |
I suspect this issue isn't really about the maximum filename length on a particular file system, but about the question whether/when The POSIX stat system call can fail for various reasons, including EACCES, EIO, ELOOP, ENAMETOOLONG, ENOENT, ENOTDIR. The docstrings of the Julia functions isfile and stat are currently completely silent on what result which of these errors lead to. This may be a problem because the equivalent functions in other languages do not raise an exception, and are in fact frequently used to check if a file exists. For comparison: the Perl/Bash equivalent
In Julia, ENOENT results in |
I tend to agree with @mgkuhn . I recall most of my using cases are about using Also, another important factor is: is there an easy way to check if the filename is too long? |
Looking at the source of stat, this just calls jl_stat which calls uv_fs_stat, none of which say anything about exceptions that they can raise. |
POSIX offers the function pathconf to query path-specific limits on the maximum filename and path length permitted by a file system. Libuv has uv__fs_pathmax_size() to query I don't know a Win32 function to query maximum filename length, and limits there tend to be be measured in UTF-16 words, and not in UTF-8 bytes. So "255" can mean different things! |
I just tried to attack Pluto server (locally) by inputing a too long filename in the path, Pluto is lucky and this error does not break down the server. Not sure every web application can be so lucky under such an attack. 😏 If you input a short filename, looks reasonableIf you input a too long filename, it might confuse Pluto a bitToo many people are assuming |
Not sure if that qualifies as “easy”:
|
This is good. I just want to know if people want to have a |
So the question is: in base/stat.jl should the functions
catch exceptions that If yes: all exceptions that Which exceptions can |
Bump. |
EDIT: Below is trivia of MAX_PATH, an issue I want fixed, and then below on NAME_MAX, that I can confirm is 255, practically speaking. https://insanecoding.blogspot.com/2007/11/pathmax-simply-isnt.html
Can't we assume at least 32768-long paths allowed on all filesystems/operating systems we want Julia to support? Lines 16 to 17 in 3897667
https://www.gnu.org/software/libc/manual/html_node/Limits-for-Files.html
EDIT: I misread the issue here about path length, something I very much want to fix to not be limited to 256/260, 1024 or whatever, which seem possible even on Windows (without the registry hack), but to stay on topic:
Seemingly never longer, but sometimes shorter (all exceptions we can Ignore, i.e. 14 in System V, and 8.3 in FAT, i.e. VFAT ok?): https://www.baeldung.com/linux/bash-filename-limit
And: https://discussions.apple.com/thread/555198
and:
|
The obligatory Wikipedia summary post https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits |
I recently ran into an issue where I got hit with ENAMETOOLONG for a path that is 180 characters. I'm on Linux Mint, btrfs filesystem, which should support <255 filename length just fine. My string did contain special characters, and it wasn't an absolute path. So I'm guessing somewhere or other, my string ballooned to >255 characters? Point is, simple length checks seems brittle. Why not just rely on the error code, and handling it in a way that doesn't surprise users? |
To my thinking the path/segment length issue is moot.
The reason for the failure is not relevant to the purpose of Instead of trying to test if the path is legal which is "complicated", just try to As noted above a number of other similar functions probably should be made consistent. |
I'll have a look. There seem to be additional problems with error handling in
doesn't look like a robust error-handling mechanism either. Shouldn't |
Wow, that's very surprising. Probably worth raising as a separate bug. |
I mostly agree, except for the case of permission issues. Given a folder /foo which I cannot access, and a file therein called bar, I cannot stat /foo/bar even though it is in fact a file. So really the return type should be Boolean|Missing |
The permissions you describe are set so you are not supposed to be able to see the file. It would be a security leak if |
I understand the security implications, but I don't think returning |
True, nothing that uses What about all the other reasons that the file cannot be resolved? On Linux As you say, if the user wants more information they can use if isfile(path)
do_something(path) # ignoring potential race, its "simple" after all
end But returning |
Not what I was saying. I only meant that As for The other one I'm unsure about is
It's a bit unclear what that means, but it sounds like it could be raised even if the path does in fact refer to a file. Eg, if it's on a network mount that's not responding, it might or might not be a file and we just don't know. As for your point about ending up with a |
My question then is why the special pleading for EACCESS? What is the general use-case for it for simplified functions like |
If you return daemon_pid_fname = "..."
isfile(daemon_pid_fname) || spawn_daemon() |> persist_pid
... If we mask the |
But why is And as my comment in the example alluded to, there is a race in your example, another process (or thread) could create the daemon between the test and the spawn. Simple IMHO complicating the simple to incorrectly handle the complex is the wrong thing to do. |
Yeah, ironically my example had misleading characteristics. The concurrency/race condition is really not the point though. The point is misleading the caller that the file does not exist, acting on that false knowledge, but the file did actually exist--we just couldn't access it. If we were discussing The point remains that if something like So to recap the options:
I'm in favor of options 2 (preferred) or 4, but mostly I'm just very against option 1. |
Since all they do is test the return of ($f)(path...) = ($f)(stat(path...)) The OP was that The OP commented "I recall most of my using cases are about using isfile to avoid exceptions.", a reasonable expectation since no exceptions are documented for @mgkuhn to answer your question "If yes: all exceptions that stat/lstat can raise, or only some? Which?" AFAICT UV error codes are converted to exceptions here for any code other than ENOENT, EINVAL, and ENOTDIR. It will also throw if it detects an inconsistency in the Those conditions seem arbitrary IMO, eg why does EINVAL not throw, but ENAMETOOLONG does, both seem to me to be input parameter errors, and both ENOTDIR and ENAMETOOLONG both seem to be path errors but ENOTDIR does not throw. Since these tests are explicit perhaps someone can comment on why it is that way. Perhaps a @nstiurca ok, understand your point, but Also returning |
@elextr Soooo... you support option 3 from my last comment as your preferred resolution? It's better than before, particularly if well documented. I can live with that. As for "feature change"... we're dealing with undefined behavior (the definition doesn't mention exceptions or cases where the call is uncertain. As such, fleshing out the definition isn't technically a breaking change even if it's a change. Point being we can get this change into the next point release if we can get to some consensus on what's the right definition. |
The maximum filename length is 255,
isfile
should return false rather than raising an error when the file name is too long.The same applies for
isdir
relevant code:
julia/base/stat.jl
Line 311 in a163d3e
Julia version: 1.6.0-rc1
The text was updated successfully, but these errors were encountered: