New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Expand the scope of `std::fs` #1044

Merged
merged 12 commits into from Apr 24, 2015

Conversation

Projects
None yet
@alexcrichton
Member

alexcrichton commented Apr 7, 2015

Expand the scope of the std::fs module by enhancing existing functionality,
exposing lower level representations, and adding a few more new functions.

Rendered

alexcrichton and others added some commits Apr 6, 2015

RFC: Expand the scope of `std::fs`
Expand the scope of the `std::fs` module by enhancing existing functionality,
exposing lower level representations, and adding a few more new functions.
Show outdated Hide outdated text/0000-io-fs-2.1.md

@alexcrichton alexcrichton self-assigned this Apr 8, 2015

@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 8, 2015

Just a documentation note, but this can actually be a difference between different filesystems on the same platform; for example, most filesystems on Linux include the type in dirent, but XFS does not, so the dirent.d_type might be DT_UKNOWN and you have to do an extra stat syscall to answer these questions.

Since this can involve an extra syscall, I wonder if it's a good idea to have a series of is_dir, is_file, is_soft_link, rather than something that returns a FileType enum (or integer newtype, to make it non-exhaustive and leave room for expansion and platform differences). Having to make a syscall repeatedly for every directory entry if dispatching on file type can get quite expensive on large directories, while fetching a FileType and then matching on it involves just a single syscall.

lambda commented on text/0000-io-fs-2.1.md in f3153e6 Apr 8, 2015

Just a documentation note, but this can actually be a difference between different filesystems on the same platform; for example, most filesystems on Linux include the type in dirent, but XFS does not, so the dirent.d_type might be DT_UKNOWN and you have to do an extra stat syscall to answer these questions.

Since this can involve an extra syscall, I wonder if it's a good idea to have a series of is_dir, is_file, is_soft_link, rather than something that returns a FileType enum (or integer newtype, to make it non-exhaustive and leave room for expansion and platform differences). Having to make a syscall repeatedly for every directory entry if dispatching on file type can get quite expensive on large directories, while fetching a FileType and then matching on it involves just a single syscall.

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 8, 2015

Owner

Thanks for the info! I like the idea of a FileType newtype of some form, so I'll see if I can update this RFC accordingly.

Owner

alexcrichton replied Apr 8, 2015

Thanks for the info! I like the idea of a FileType newtype of some form, so I'll see if I can update this RFC accordingly.

@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 8, 2015

An important piece that's missing here is a way to get the d_ino out of a dirent entry on Unix-like platforms. If you're iterating over directories that contain a large number of hardlinks, such as rsnapshot or Time Machine backups, or something where you use hardlinks to provide multiple different views of a particular directory hierarchy, it can be important to avoid an extra stat by caching inodes that you have already seen and just reusing the stat information you got the last time. A project I was working on in Python recently got bitten by this using os.walk(), where we had to do expensive lstat and stat system calls for each file, but saw a huge improvement in performance when using ctypes to call the libc API directly and were able to do the caching trick; see this feature request (which appears to have recently been implemented) on the new Python scandir function's DirEntry class for the same reason.

I don't mind whether this is exposed directly in the portable API (and thus would incur an extra syscall on Windows, since there is no inode equivalent in Windows' dirent equivalent as far as I can see; if you want performance on both platforms, you are going to have to follow different code paths), or as a way of lowering the DirEntry to a platform-specific version that contains more information; however, an API that simply returns a full RawMetadata object or requires you to stat it yourself wouldn't be sufficient, as you would then necessarily incur the extra syscall overhead for information that was already returned to you when traversing the directory.

lambda commented on text/0000-io-fs-2.1.md in f3153e6 Apr 8, 2015

An important piece that's missing here is a way to get the d_ino out of a dirent entry on Unix-like platforms. If you're iterating over directories that contain a large number of hardlinks, such as rsnapshot or Time Machine backups, or something where you use hardlinks to provide multiple different views of a particular directory hierarchy, it can be important to avoid an extra stat by caching inodes that you have already seen and just reusing the stat information you got the last time. A project I was working on in Python recently got bitten by this using os.walk(), where we had to do expensive lstat and stat system calls for each file, but saw a huge improvement in performance when using ctypes to call the libc API directly and were able to do the caching trick; see this feature request (which appears to have recently been implemented) on the new Python scandir function's DirEntry class for the same reason.

I don't mind whether this is exposed directly in the portable API (and thus would incur an extra syscall on Windows, since there is no inode equivalent in Windows' dirent equivalent as far as I can see; if you want performance on both platforms, you are going to have to follow different code paths), or as a way of lowering the DirEntry to a platform-specific version that contains more information; however, an API that simply returns a full RawMetadata object or requires you to stat it yourself wouldn't be sufficient, as you would then necessarily incur the extra syscall overhead for information that was already returned to you when traversing the directory.

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 8, 2015

Owner

Oh interesting! This sounds like it's a perfect piece of functionality for a unix extension trait, so I will add that.

Owner

alexcrichton replied Apr 8, 2015

Oh interesting! This sounds like it's a perfect piece of functionality for a unix extension trait, so I will add that.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 8, 2015

Member

I've added two more pieces of information (thanks to @lambda for the suggestions!):

  • There is now a unix extension trait to access the d_ino field of DirEntry on unix.
  • There is now a FileType abstraction in std::fs to represent the type of a file, which is also exported on a DirEntry.
Member

alexcrichton commented Apr 8, 2015

I've added two more pieces of information (thanks to @lambda for the suggestions!):

  • There is now a unix extension trait to access the d_ino field of DirEntry on unix.
  • There is now a FileType abstraction in std::fs to represent the type of a file, which is also exported on a DirEntry.
@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 8, 2015

Looks like a leftover hunk from an earlier edit.

lambda commented on text/0000-io-fs-2.1.md in ebf5514 Apr 8, 2015

Looks like a leftover hunk from an earlier edit.

@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 8, 2015

Contributor

Yep, those two changes address my concerns. 👍

Contributor

lambda commented Apr 8, 2015

Yep, those two changes address my concerns. 👍

Show outdated Hide outdated text/0000-io-fs-2.1.md
@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 8, 2015

This is missing a couple of constants; the SUID, SGID, and sticky bit flags (S_ISUID, S_ISGID, and S_ISVTX in POSIX/Single Unix Specificaiton). I appreciate the more readable names for the other constants, so more readable constants for these would be good too, perhaps something like SETUID, SETGID (with or without an underscore, I don't particularly mind), and STICKY_BIT since that's how people usually refer to them.

lambda commented on text/0000-io-fs-2.1.md in 91c0df5 Apr 8, 2015

This is missing a couple of constants; the SUID, SGID, and sticky bit flags (S_ISUID, S_ISGID, and S_ISVTX in POSIX/Single Unix Specificaiton). I appreciate the more readable names for the other constants, so more readable constants for these would be good too, perhaps something like SETUID, SETGID (with or without an underscore, I don't particularly mind), and STICKY_BIT since that's how people usually refer to them.

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 8, 2015

Owner

Oops thanks for the reminder! I haven't worked with the sticky bit much before, so I'm wondering if the name STICKY would be appropriate? I figure the "bit" is implied as these are all just bit twiddling anyway.

Owner

alexcrichton replied Apr 8, 2015

Oops thanks for the reminder! I haven't worked with the sticky bit much before, so I'm wondering if the name STICKY would be appropriate? I figure the "bit" is implied as these are all just bit twiddling anyway.

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 8, 2015

Just STICKY would probably be unambiguous, but it sounds a little odd. It's almost always referred to as the "sticky bit", even when the others aren't. man 2 chmod on Mac OS X:

 The ISVTX (the sticky bit) indicates to the system which executable files
 are shareable (the default) and the system maintains the program text of
 the files in the swap area. The sticky bit may only be set by the super
 user on shareable executable files.

 If mode ISVTX (the `sticky bit') is set on a directory, an unprivileged
 user may not delete or rename files of other users in that directory. The
 sticky bit may be set by any user on a directory which the user owns or
 has appropriate permissions.  For more details of the properties of the
 sticky bit, see sticky(8).

Though man sticky does use "sticky" alone in at least one case:

 A directory whose `sticky bit' is set becomes an append-only directory,
 or, more accurately, a directory in which the deletion of files is
 restricted.  A file in a sticky directory may only be removed or renamed
 by a user if the user has write permission for the directory and the user
 is the owner of the file, the owner of the directory, or the super-user.
 This feature is usefully applied to directories such as /tmp which must
 be publicly writable but should deny users the license to arbitrarily
 delete or rename each others' files.

 Any user may create a sticky directory.  See chmod(1) for details about
 modifying file modes.

man 2 chmod on Linux:

   S_ISVTX  (01000)  sticky bit (restricted deletion flag, as described
                     in unlink(2))

man unlink on Linux:

   EPERM or EACCES
          The directory containing pathname has the sticky bit (S_ISVTX)
          set and the process's effective UID is neither the UID of the
          file to be deleted nor that of the directory containing it,
          and the process is not privileged (Linux: does not have the
          CAP_FOWNER capability).

So, for whatever reason, it's almost always referred to as "the sticky bit" even when none of the others contain "bit". I don't feel particularly strongly either way, I'd be fine with just STICKY, I just figured I'd point out some usage examples. The standard name, S_ISVTX, is completely opaque, and the traditional name "save swapped text" is entirely obsolete, so STICKY or STICKY_BIT are pretty much the only reasonable options.

As for its use, it's usually used in shared directories, which are writable by multiple users, but in which you don't want one user to accidentally delete another user's files; it prevents users from deleting files belonging to other users, even though they have write access to the directory.

lambda replied Apr 8, 2015

Just STICKY would probably be unambiguous, but it sounds a little odd. It's almost always referred to as the "sticky bit", even when the others aren't. man 2 chmod on Mac OS X:

 The ISVTX (the sticky bit) indicates to the system which executable files
 are shareable (the default) and the system maintains the program text of
 the files in the swap area. The sticky bit may only be set by the super
 user on shareable executable files.

 If mode ISVTX (the `sticky bit') is set on a directory, an unprivileged
 user may not delete or rename files of other users in that directory. The
 sticky bit may be set by any user on a directory which the user owns or
 has appropriate permissions.  For more details of the properties of the
 sticky bit, see sticky(8).

Though man sticky does use "sticky" alone in at least one case:

 A directory whose `sticky bit' is set becomes an append-only directory,
 or, more accurately, a directory in which the deletion of files is
 restricted.  A file in a sticky directory may only be removed or renamed
 by a user if the user has write permission for the directory and the user
 is the owner of the file, the owner of the directory, or the super-user.
 This feature is usefully applied to directories such as /tmp which must
 be publicly writable but should deny users the license to arbitrarily
 delete or rename each others' files.

 Any user may create a sticky directory.  See chmod(1) for details about
 modifying file modes.

man 2 chmod on Linux:

   S_ISVTX  (01000)  sticky bit (restricted deletion flag, as described
                     in unlink(2))

man unlink on Linux:

   EPERM or EACCES
          The directory containing pathname has the sticky bit (S_ISVTX)
          set and the process's effective UID is neither the UID of the
          file to be deleted nor that of the directory containing it,
          and the process is not privileged (Linux: does not have the
          CAP_FOWNER capability).

So, for whatever reason, it's almost always referred to as "the sticky bit" even when none of the others contain "bit". I don't feel particularly strongly either way, I'd be fine with just STICKY, I just figured I'd point out some usage examples. The standard name, S_ISVTX, is completely opaque, and the traditional name "save swapped text" is entirely obsolete, so STICKY or STICKY_BIT are pretty much the only reasonable options.

As for its use, it's usually used in shared directories, which are writable by multiple users, but in which you don't want one user to accidentally delete another user's files; it prevents users from deleting files belonging to other users, even though they have write access to the directory.

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 8, 2015

Owner

Alright it definitely sounds like the conventional way to refer to this is include "bit" in the name, so I'll do that. Thanks for the info!

Owner

alexcrichton replied Apr 8, 2015

Alright it definitely sounds like the conventional way to refer to this is include "bit" in the name, so I'll do that. Thanks for the info!

@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 8, 2015

Contributor

Thanks for the updates!

One more bikeshed after reading it through one more time. This RFC refers to symbolic links as soft links, but on both Unix and Windows, the preferred term seems to be symbolic link, or abbreviated as symlink. I notice that there's already a stabilized std::fs::soft_link function to create one, so that may be sufficient inertia to keep the soft link terminology, but if it's still possible to deprecate that and change to using the term "symlink" or "symbolic link", I think that that would be less surprising to most people.

Many sources say "symbolic link (or soft link)" or "soft link (or symbolic link)" when referring to them, but the actual APIs are symlink on Unix, CreateSymbolicLink on Windows, and the official documentation and standards for most platforms I looked at (Mac OS X, FreeBSD, Linux, Windows, POSIX/Single Unix Specification) uses "symbolic link" exclusively in prose, never "soft link".

Quick poll of other languages as well: Java has createSymbolicLink, C# CreateSymbolicLink, C++ (Boost/draft standard) create_symlink, Ruby symlink, Python symlink, Objective-C/Swift createSymbolicLinkAtPath:withDestinationPath:.

Contributor

lambda commented Apr 8, 2015

Thanks for the updates!

One more bikeshed after reading it through one more time. This RFC refers to symbolic links as soft links, but on both Unix and Windows, the preferred term seems to be symbolic link, or abbreviated as symlink. I notice that there's already a stabilized std::fs::soft_link function to create one, so that may be sufficient inertia to keep the soft link terminology, but if it's still possible to deprecate that and change to using the term "symlink" or "symbolic link", I think that that would be less surprising to most people.

Many sources say "symbolic link (or soft link)" or "soft link (or symbolic link)" when referring to them, but the actual APIs are symlink on Unix, CreateSymbolicLink on Windows, and the official documentation and standards for most platforms I looked at (Mac OS X, FreeBSD, Linux, Windows, POSIX/Single Unix Specification) uses "symbolic link" exclusively in prose, never "soft link".

Quick poll of other languages as well: Java has createSymbolicLink, C# CreateSymbolicLink, C++ (Boost/draft standard) create_symlink, Ruby symlink, Python symlink, Objective-C/Swift createSymbolicLinkAtPath:withDestinationPath:.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 8, 2015

Member

I do agree that I sometimes have to do a double-take whenever I see "soft_link" as opposed to symlink, but it does tend to agree with the current conventions of the I/O modules:

  • The term "symlink" is arguably a unix-ism (not mentioned on Windows), and we don't tend to favor one platform's naming conventions over another.
  • The term "symbolic link" is a little wordy for the standard library, it tends to stick to some smaller terms wherever possible.

With these two the term "soft link" seemed like a good fit in the middle. It is possible that the fs::soft_link function could be deprecated (if this change happened soon), but another possibility would also be to modify accessors to something like is_link instead of is_soft_link.

Member

alexcrichton commented Apr 8, 2015

I do agree that I sometimes have to do a double-take whenever I see "soft_link" as opposed to symlink, but it does tend to agree with the current conventions of the I/O modules:

  • The term "symlink" is arguably a unix-ism (not mentioned on Windows), and we don't tend to favor one platform's naming conventions over another.
  • The term "symbolic link" is a little wordy for the standard library, it tends to stick to some smaller terms wherever possible.

With these two the term "soft link" seemed like a good fit in the middle. It is possible that the fs::soft_link function could be deprecated (if this change happened soon), but another possibility would also be to modify accessors to something like is_link instead of is_soft_link.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 9, 2015

Member

I've pushed a commit that removes the recommendation of Permissions::new

Member

alexcrichton commented Apr 9, 2015

I've pushed a commit that removes the recommendation of Permissions::new

@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 9, 2015

Contributor

Yeah, Windows does use SymbolicLink fairly consistently in their API, but I have found no API that uses "soft link"; every API I have found uses some variant of "symlink" or "symbolic link" in snake or camel case. Even other Windows-centric languages do use symlink; Delphi has FileCreateSymLink, and while it looks like PowerShell has no native way to do it, various people have written PowerShell cmdlets called New-SymLink (one example, another example). And Microsoft also includes the proposed C++ filesystem standard that uses create_symlink in Visual Studio.

I feel like the surprise of being the only API (that I could find) that uses soft_link may outweigh either the length of symbolic_link or the slightly Unix-centric nature of symlink; and one thing to recall is that symbolic links themselves are a concept that originated on Unix and were implemented on Windows mostly for compatibility with Unix:

Symbolic links are designed to aid in migration and application compatibility with UNIX operating systems. Microsoft has implemented its symbolic links to function just like UNIX links.

Anyhow, it is a relatively minor bikeshed, and given the existing soft_link function, I'm wondering whether the discussion is appropriate for this RFC, or whether I should file another RFC to discuss just this one naming point and possible deprecation of the already stabilized soft_link in favor of a more standard naming convention.

Contributor

lambda commented Apr 9, 2015

Yeah, Windows does use SymbolicLink fairly consistently in their API, but I have found no API that uses "soft link"; every API I have found uses some variant of "symlink" or "symbolic link" in snake or camel case. Even other Windows-centric languages do use symlink; Delphi has FileCreateSymLink, and while it looks like PowerShell has no native way to do it, various people have written PowerShell cmdlets called New-SymLink (one example, another example). And Microsoft also includes the proposed C++ filesystem standard that uses create_symlink in Visual Studio.

I feel like the surprise of being the only API (that I could find) that uses soft_link may outweigh either the length of symbolic_link or the slightly Unix-centric nature of symlink; and one thing to recall is that symbolic links themselves are a concept that originated on Unix and were implemented on Windows mostly for compatibility with Unix:

Symbolic links are designed to aid in migration and application compatibility with UNIX operating systems. Microsoft has implemented its symbolic links to function just like UNIX links.

Anyhow, it is a relatively minor bikeshed, and given the existing soft_link function, I'm wondering whether the discussion is appropriate for this RFC, or whether I should file another RFC to discuss just this one naming point and possible deprecation of the already stabilized soft_link in favor of a more standard naming convention.

@jmesmon

This comment has been minimized.

Show comment
Hide comment
@jmesmon

jmesmon Apr 9, 2015

I've also never been a fan of soft_link for the similar reasons. Avoiding symlink as a precieved unix-ism seemed a bit knee-jerky.

jmesmon commented Apr 9, 2015

I've also never been a fan of soft_link for the similar reasons. Avoiding symlink as a precieved unix-ism seemed a bit knee-jerky.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 9, 2015

Member

@lambda I think it's definitely relevant to this RFC as it's expanding the surface area of the "soft link" terminology, but I also think that you may want to open a separate RFC issue and/or discuss post about it as we probably shouldn't centralize discussion about that topic here.

Member

alexcrichton commented Apr 9, 2015

@lambda I think it's definitely relevant to this RFC as it's expanding the surface area of the "soft link" terminology, but I also think that you may want to open a separate RFC issue and/or discuss post about it as we probably shouldn't centralize discussion about that topic here.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 9, 2015

Member

er, it appears you did!

#1048

Member

alexcrichton commented Apr 9, 2015

er, it appears you did!

#1048

@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 9, 2015

Contributor

Indeed, and you beat me to cross-referencing to this issue or cc'ing you on it!

Contributor

lambda commented Apr 9, 2015

Indeed, and you beat me to cross-referencing to this issue or cc'ing you on it!

Show outdated Hide outdated text/0000-io-fs-2.1.md
Show outdated Hide outdated text/0000-io-fs-2.1.md
Show outdated Hide outdated text/0000-io-fs-2.1.md
/// directory as a base, and the current working directory is not managed in a
/// thread-local fashion, so this function may need to be synchronized with
/// other calls to `env::change_dir`.
pub fn canonicalize<P: AsRef<Path>>(p: P) -> io::Result<PathBuf>;

This comment has been minimized.

@nagisa

nagisa Apr 9, 2015

Contributor

I’d like it to be a method on Path.

@nagisa

nagisa Apr 9, 2015

Contributor

I’d like it to be a method on Path.

This comment has been minimized.

@alexcrichton

alexcrichton Apr 9, 2015

Member

I'll continue discussion below, as it's also a method!

@alexcrichton

alexcrichton Apr 9, 2015

Member

I'll continue discussion below, as it's also a method!

Show outdated Hide outdated text/0000-io-fs-2.1.md
pub trait PathExt {
fn exists(&self) -> bool;
fn is_dir(&self) -> bool;
fn is_file(&self) -> bool;

This comment has been minimized.

@nagisa

nagisa Apr 9, 2015

Contributor

This duplication makes me somewhat sad (i.e. .metadata().is_file() would be equivalent I believe)

@nagisa

nagisa Apr 9, 2015

Contributor

This duplication makes me somewhat sad (i.e. .metadata().is_file() would be equivalent I believe)

This comment has been minimized.

@alexcrichton

alexcrichton Apr 9, 2015

Member

Unfortunately you'd need to do .metadata().map(|m| m.is_file()).unwrap_or(false) instead of just .metdata().is_file(). Are you thinking of adding an extension trait for io::Result<Metadata>?

@alexcrichton

alexcrichton Apr 9, 2015

Member

Unfortunately you'd need to do .metadata().map(|m| m.is_file()).unwrap_or(false) instead of just .metdata().is_file(). Are you thinking of adding an extension trait for io::Result<Metadata>?

This comment has been minimized.

@nagisa

nagisa Apr 9, 2015

Contributor

Are you thinking of adding an extension trait for io::Result?

Err, no way.

Unfortunately you'd need to do .metadata().map(|m| m.is_file()).unwrap_or(false)

Ah, yeah. I’m still doubting the usefulness of this abstraction. Especially because not being able to retrieve metadata doesn’t imply it is not a file or a directory (same applies to exists too).

@nagisa

nagisa Apr 9, 2015

Contributor

Are you thinking of adding an extension trait for io::Result?

Err, no way.

Unfortunately you'd need to do .metadata().map(|m| m.is_file()).unwrap_or(false)

Ah, yeah. I’m still doubting the usefulness of this abstraction. Especially because not being able to retrieve metadata doesn’t imply it is not a file or a directory (same applies to exists too).

This comment has been minimized.

@alexcrichton

alexcrichton Apr 10, 2015

Member

I would be pretty worried about never providing conveniences like this in Rust, and I suppose the alternative would be to return a tri-state value (Yes, Maybe, No), but I feel like that's somewhat overkill for these convenience functions...

@alexcrichton

alexcrichton Apr 10, 2015

Member

I would be pretty worried about never providing conveniences like this in Rust, and I suppose the alternative would be to return a tri-state value (Yes, Maybe, No), but I feel like that's somewhat overkill for these convenience functions...

@l0kod

This comment has been minimized.

Show comment
Hide comment
@l0kod

l0kod Apr 9, 2015

👍 for the enum FileType.

Don't forget to include POSIX ACL in Permissions (cf. acl(5)).

The PathExt should include all functions from FileType (i.e. is_soft_link()).

PathExt should inherit from a common trait to file descriptors and paths (e.g. AsIo, cf. rust-lang/rust#21936 (comment)) including this functions:

  • FileType functions
  • metadata()
  • read_link()
  • read_dir()

About all the stat(2) related cache (e.g. is_dir, is_file…), FileType should cache all metadata but provide a refresh() function to explicitly reload the path/fd info when needed and avoid unneeded syscall calls.

I'm not sure the PathBuf is the better return value for read_link(). Some symlinks are virtual like /proc/self/ns/* (but they can be interpreted as path too).

👍 for std::os::MetadataExt: creation_time(), read_time() and write_time() functions for both Windows and UNIX to return an unified time representation + time precision.

l0kod commented Apr 9, 2015

👍 for the enum FileType.

Don't forget to include POSIX ACL in Permissions (cf. acl(5)).

The PathExt should include all functions from FileType (i.e. is_soft_link()).

PathExt should inherit from a common trait to file descriptors and paths (e.g. AsIo, cf. rust-lang/rust#21936 (comment)) including this functions:

  • FileType functions
  • metadata()
  • read_link()
  • read_dir()

About all the stat(2) related cache (e.g. is_dir, is_file…), FileType should cache all metadata but provide a refresh() function to explicitly reload the path/fd info when needed and avoid unneeded syscall calls.

I'm not sure the PathBuf is the better return value for read_link(). Some symlinks are virtual like /proc/self/ns/* (but they can be interpreted as path too).

👍 for std::os::MetadataExt: creation_time(), read_time() and write_time() functions for both Windows and UNIX to return an unified time representation + time precision.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 10, 2015

Member

@l0kod

Don't forget to include POSIX ACL in Permissions (cf. acl(5)).

For now I think I'm going to leave that to a future RFC, for example the ACL system of Windows remains unbound, and Permissions is just the raw mode_t bits on Unix (and a boolean flag on Windows).

PathExt should inherit from a common trait to file descriptors and paths (e.g. AsIo, cf. rust-lang/rust#21936 (comment)) including this functions:

I thought about this, but I think there's quite a bit of design work to figure out how to best expose those functions and I'm could not convince myself that PathExt was the correct trait to abstract over these options. It's also unclear to me why you would want these methods to be based on a trait versus inherent. For now, though, I'm going to leave these flavors of functions to a future RFC.

About all the stat(2) related cache (e.g. is_dir, is_file…), FileType should cache all metadata but provide a refresh() function to explicitly reload the path/fd info when needed and avoid unneeded syscall calls.

This sounds much more like an application-level optimization rather than the behavior of the standard library. The goal of the standard library is to be as clear as possible so you know precisely what's happening, and having a silent stat cache would likely be a little much.

I'm not sure the PathBuf is the better return value for read_link(). Some symlinks are virtual like /proc/self/ns/* (but they can be interpreted as path too).

There's a few things here I think. Primarily this is a #[stable] API where modifications cannot be taken lightly. It's true that PathBuf is basically just the same thing as a OsString (with different semantic meaning), but all PathBuf instances are also OsString instances (and vice versa), so I don't think we're really losing out here (especially because 99% of symlink targets are probably intended to be interpreted as paths)

Member

alexcrichton commented Apr 10, 2015

@l0kod

Don't forget to include POSIX ACL in Permissions (cf. acl(5)).

For now I think I'm going to leave that to a future RFC, for example the ACL system of Windows remains unbound, and Permissions is just the raw mode_t bits on Unix (and a boolean flag on Windows).

PathExt should inherit from a common trait to file descriptors and paths (e.g. AsIo, cf. rust-lang/rust#21936 (comment)) including this functions:

I thought about this, but I think there's quite a bit of design work to figure out how to best expose those functions and I'm could not convince myself that PathExt was the correct trait to abstract over these options. It's also unclear to me why you would want these methods to be based on a trait versus inherent. For now, though, I'm going to leave these flavors of functions to a future RFC.

About all the stat(2) related cache (e.g. is_dir, is_file…), FileType should cache all metadata but provide a refresh() function to explicitly reload the path/fd info when needed and avoid unneeded syscall calls.

This sounds much more like an application-level optimization rather than the behavior of the standard library. The goal of the standard library is to be as clear as possible so you know precisely what's happening, and having a silent stat cache would likely be a little much.

I'm not sure the PathBuf is the better return value for read_link(). Some symlinks are virtual like /proc/self/ns/* (but they can be interpreted as path too).

There's a few things here I think. Primarily this is a #[stable] API where modifications cannot be taken lightly. It's true that PathBuf is basically just the same thing as a OsString (with different semantic meaning), but all PathBuf instances are also OsString instances (and vice versa), so I don't think we're really losing out here (especially because 99% of symlink targets are probably intended to be interpreted as paths)

@eternaleye

This comment has been minimized.

Show comment
Hide comment
@eternaleye

eternaleye Apr 10, 2015

@l0kod

Leaving aside Windows, there are in-progress patches for Linux to use RichACLs, which are based on NFSv4 ACLs.

eternaleye commented Apr 10, 2015

@l0kod

Leaving aside Windows, there are in-progress patches for Linux to use RichACLs, which are based on NFSv4 ACLs.

@rprichard

This comment has been minimized.

Show comment
Hide comment
@rprichard

rprichard Apr 10, 2015

For Windows' sake, should we add a function that creates a directory symlink? It would do the same thing as soft_link on Unix, but on Windows, it would pass a SYMBOLIC_LINK_FLAG_DIRECTORY flag to CreateSymbolicLink. Apparently Boost (and the draft C++ File System Spec) have create_symlink and create_directory_symlink for this reason.

rprichard commented Apr 10, 2015

For Windows' sake, should we add a function that creates a directory symlink? It would do the same thing as soft_link on Unix, but on Windows, it would pass a SYMBOLIC_LINK_FLAG_DIRECTORY flag to CreateSymbolicLink. Apparently Boost (and the draft C++ File System Spec) have create_symlink and create_directory_symlink for this reason.

@l0kod

This comment has been minimized.

Show comment
Hide comment
@l0kod

l0kod commented Apr 10, 2015

@jmesmon

This comment has been minimized.

Show comment
Hide comment
@jmesmon

jmesmon Apr 10, 2015

@rprichard perhaps if we:

  • adjust soft_link()/symlink() to detect the appropriate call options via metadata() on platforms where it is requred.
  • create a symlink_non_dir() which has a debug assert (again using metadata()) which complains about symlinking dirs.
  • Add symlink_dir() with a similar debug assert that complains about symlinking non-dirs.

I suppose we also need to wonder what happens with symlinks created to non existing files. Does windows allow that? Do we expect some other platform to forbid it?

It would allow better exposure of the underlying API, I'm not convinced it's worth it though (I'm also not a windows dev, so I care a bit less)

jmesmon commented Apr 10, 2015

@rprichard perhaps if we:

  • adjust soft_link()/symlink() to detect the appropriate call options via metadata() on platforms where it is requred.
  • create a symlink_non_dir() which has a debug assert (again using metadata()) which complains about symlinking dirs.
  • Add symlink_dir() with a similar debug assert that complains about symlinking non-dirs.

I suppose we also need to wonder what happens with symlinks created to non existing files. Does windows allow that? Do we expect some other platform to forbid it?

It would allow better exposure of the underlying API, I'm not convinced it's worth it though (I'm also not a windows dev, so I care a bit less)

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 13, 2015

Member

If it's ok with everyone as well, I'd like to focus discussion on the term "symlink" or "soft link" in #1048 for now. This RFC only tangentially touches that functionality and it will follow the outcome of #1048 as well. This is some excellent discussion though!

Member

alexcrichton commented Apr 13, 2015

If it's ok with everyone as well, I'd like to focus discussion on the term "symlink" or "soft link" in #1048 for now. This RFC only tangentially touches that functionality and it will follow the outcome of #1048 as well. This is some excellent discussion though!

@nagisa

This comment has been minimized.

Show comment
Hide comment
@nagisa

nagisa Apr 13, 2015

Contributor

@alexcrichton people have been mostly discussing the need for two versions of the function here, rather than naming, though.

The way I see it, symbolically linking a file is a cross platform operation and soft_link should clearly state in its documentation that this will create a symbolic link to a file, but might not work for directories.

Though, I’m fine with not having soft_link in std::fs at all as well.

Contributor

nagisa commented Apr 13, 2015

@alexcrichton people have been mostly discussing the need for two versions of the function here, rather than naming, though.

The way I see it, symbolically linking a file is a cross platform operation and soft_link should clearly state in its documentation that this will create a symbolic link to a file, but might not work for directories.

Though, I’m fine with not having soft_link in std::fs at all as well.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 13, 2015

Member

@nagisa ah yes, right. I'm considering that to fall under the purview of #1048 as well, however, as the only relation it has to this RFC is the naming of is_soft_link or soft_link_metadata, both of which will just use the outcome of #1048.

Specifically #1048 is considering renaming an API, and renaming the APIs in a platform-specific manner also falls in that category (in my mind at least).

Member

alexcrichton commented Apr 13, 2015

@nagisa ah yes, right. I'm considering that to fall under the purview of #1048 as well, however, as the only relation it has to this RFC is the naming of is_soft_link or soft_link_metadata, both of which will just use the outcome of #1048.

Specifically #1048 is considering renaming an API, and renaming the APIs in a platform-specific manner also falls in that category (in my mind at least).

@lambda

This comment has been minimized.

Show comment
Hide comment
@lambda

lambda Apr 14, 2015

Contributor

I have updated #1048 to include splitting up fs::soft_link into os::unix::fs::symlink and os::windows::fs::{symlink_file, symlink_dir}. Further discussion of the changes to fs::soft_link should happen there, since that involves modifying an already stable API (with back-compat wrappers), while this RFC focuses on adding new functionality.

Contributor

lambda commented Apr 14, 2015

I have updated #1048 to include splitting up fs::soft_link into os::unix::fs::symlink and os::windows::fs::{symlink_file, symlink_dir}. Further discussion of the changes to fs::soft_link should happen there, since that involves modifying an already stable API (with back-compat wrappers), while this RFC focuses on adding new functionality.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 14, 2015

Member

I've pushed a few small tweaks to this RFC:

  • The term symlink is favored instead of soft_link in symlink_metadata and is_symlink
  • The DirEntry::file_type function may fail, so it returns io::Result
  • The CreateDirOptions structure was renamed to DirBuilder
Member

alexcrichton commented Apr 14, 2015

I've pushed a few small tweaks to this RFC:

  • The term symlink is favored instead of soft_link in symlink_metadata and is_symlink
  • The DirEntry::file_type function may fail, so it returns io::Result
  • The CreateDirOptions structure was renamed to DirBuilder

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 17, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 17, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 22, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 23, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 23, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 23, 2015

Member

I've posted a PR with a sample implementation of this RFC.

Member

alexcrichton commented Apr 23, 2015

I've posted a PR with a sample implementation of this RFC.

ogham added a commit to ogham/exa that referenced this pull request Apr 23, 2015

Use new io + path + fs libraries (LOTS OF CHANGES)
Exa now uses the new IO, Path, and Filesystem libraries that have been out for a while now.

Unfortunately, the new libraries don't *entirely* cover the range of the old libraries just yet: in particular, to become more cross-platform, the data in `UnstableFileStat` isn't available in the Unix `MetadataExt` yet. Much of this is contained in rust-lang/rfcs#1044 (which is due to be implemented in rust-lang/rust#14711), but it's not *entirely* there yet.

As such, this commits a serious loss of functionality: no symlink viewing, no hard links or blocks, or users or groups. Also, some of the code could now be optimised. I just wanted to commit this to sort out most of the 'teething problems' of having a different path system in advance.

Here's an example problem that took ages to fix for you, just because you read this far: when I first got exa to compile, it worked mostly fine, except calling `exa` by itself didn't list the current directory. I traced where the command-line options were being generated, to where files and directories were sorted, to where the threads were spawned... and the problem turned out to be that it was using the full path as the file name, rather than just the last component, and these paths happened to begin with `.`, so it thought they were dotfiles.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 23, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 23, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 23, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 23, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 23, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

@aturon aturon merged commit d8bc0fe into rust-lang:master Apr 24, 2015

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Apr 24, 2015

Member

Thanks for the fantastic discussion, everyone! I've merged the RFC.

Much of the feedback ended up targeting symlinks, which got moved off into their own RFC. Otherwise, I believe that all of the feedback has been taken into account, and overall we have a strong consensus that these APIs are both reasonable and needed.

Note that, as always, these features will be landing as unstable (for 1.1 beta) and we can continue tweaking details of the design prior to stabilization.

Member

aturon commented Apr 24, 2015

Thanks for the fantastic discussion, everyone! I've merged the RFC.

Much of the feedback ended up targeting symlinks, which got moved off into their own RFC. Otherwise, I believe that all of the feedback has been taken into account, and overall we have a strong consensus that these APIs are both reasonable and needed.

Note that, as always, these features will be landing as unstable (for 1.1 beta) and we can continue tweaking details of the design prior to stabilization.

pub fn file_type(&self) -> io::Result<FileType>;
/// Returns the file name for this directory entry.
pub fn file_name(&self) -> OsString;

This comment has been minimized.

@GuillaumeGomez

GuillaumeGomez Apr 27, 2015

Member

What happen if this is a directory or something else ? Do we call this function ?

@GuillaumeGomez

GuillaumeGomez Apr 27, 2015

Member

What happen if this is a directory or something else ? Do we call this function ?

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 27, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 27, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 27, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 28, 2015

std: Expand the area of std::fs
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.
@haxney

This comment has been minimized.

Show comment
Hide comment
@haxney

haxney Apr 28, 2015

As a general comment, I'd recommend looking into user-defined filesystems like what Java has with its Path and FileSystem classes. Essentially, rather than being able to turn a String into a Path directly, you need to ask some FileSystem object to give you a Path for a String.

It is a bit more complex (but you can do FileSystems.getDefault().getPath("thefile.txt") to use the default file system), but allows you to create things like Jimfs (an in-memory file system for testing) without worrying about things like FUSE.

haxney commented Apr 28, 2015

As a general comment, I'd recommend looking into user-defined filesystems like what Java has with its Path and FileSystem classes. Essentially, rather than being able to turn a String into a Path directly, you need to ask some FileSystem object to give you a Path for a String.

It is a bit more complex (but you can do FileSystems.getDefault().getPath("thefile.txt") to use the default file system), but allows you to create things like Jimfs (an in-memory file system for testing) without worrying about things like FUSE.

bors added a commit to rust-lang/rust that referenced this pull request Apr 28, 2015

Auto merge of #24711 - alexcrichton:fs2.1, r=aturon
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

Closes #24796

bors added a commit to rust-lang/rust that referenced this pull request Apr 29, 2015

Auto merge of #24711 - alexcrichton:fs2.1, r=alexcrichton
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

Closes #24796

bors added a commit to rust-lang/rust that referenced this pull request Apr 29, 2015

Auto merge of #24711 - alexcrichton:fs2.1, r=alexcrichton
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

Closes #24796

alexcrichton added a commit to alexcrichton/rust that referenced this pull request Apr 29, 2015

rollup merge of #24711: alexcrichton/fs2.1
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.

[rfc]: rust-lang/rfcs#1044

The new APIs added are:

* `fs::canonicalize` - bindings to `realpath` on unix and
  `GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
  but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
  syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
  components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
  function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
  function.
* `fs::PathExt::read_link` - convenience method for the top-level
  function.
* `fs::PathExt::read_dir` - convenience method for the top-level
  function.
* `std::os::raw` - type definitions for raw OS/C types available on all
  platforms.
* `std::os::$platform` - new modules have been added for all currently supported
  platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
  are populated with the bare essentials necessary for lowing I/O types into
  their raw representations, and currently largely consist of the `stat`
  definition for unix platforms.

This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.

Closes rust-lang#24796
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment