New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copyFile does not copy metadata #40

Closed
thomie opened this Issue Jan 6, 2016 · 2 comments

Comments

Projects
None yet
2 participants
@thomie
Member

thomie commented Jan 6, 2016

In https://ghc.haskell.org/trac/ghc/ticket/5251, GHC trac user Orphi reports:

The copyFile function fails to copy various metadata to do with a file.

(GHC 7.0.2, directory 1.1.0.0 - not that I expect it to matter.)

The cause is simple: Rather than calling an OS-specific "please copy this file" function, the directory package attempts to implement this logic itself. And does it completely wrong.

The fix should be easy; we just need somebody to figure out what the appropriate native OS function is on each platform.

@thomie

This comment has been minimized.

Show comment
Hide comment
@thomie

thomie Jan 6, 2016

Member

Comment by Orphi:

For reference, here are the Windows-specific things which are not copied by copyFile:

  • The "hidden" attribute.
  • The "system" attribute.
  • The "compressed" attribute.
  • The "indexed" attribute.
  • Security settings.
  • File owner.
  • Auditing settings.
  • Alternative Data Streams.
    The "archive" bit is also not copied - but this is almost certainly the correct thing to do.

Interestingly, the "read only" attribute is copied correctly.

I was unable to test the following, but I believe they will be lost too:

  • Sparse files [will become dense when copied].
  • FS-level encryption.

As you can see, Windows stores a hell of a lot of metadata about files, almost all of which the naive copyFile implementation is blindly stripping off. It wouldn't surprise me if it drops metadata on Unix too, although I haven't had time to test yet.

Member

thomie commented Jan 6, 2016

Comment by Orphi:

For reference, here are the Windows-specific things which are not copied by copyFile:

  • The "hidden" attribute.
  • The "system" attribute.
  • The "compressed" attribute.
  • The "indexed" attribute.
  • Security settings.
  • File owner.
  • Auditing settings.
  • Alternative Data Streams.
    The "archive" bit is also not copied - but this is almost certainly the correct thing to do.

Interestingly, the "read only" attribute is copied correctly.

I was unable to test the following, but I believe they will be lost too:

  • Sparse files [will become dense when copied].
  • FS-level encryption.

As you can see, Windows stores a hell of a lot of metadata about files, almost all of which the naive copyFile implementation is blindly stripping off. It wouldn't surprise me if it drops metadata on Unix too, although I haven't had time to test yet.

@Rufflewind

This comment has been minimized.

Show comment
Hide comment
@Rufflewind

Rufflewind Jan 6, 2016

Member

Windows has CopyFile[Ex], which does copy metadata.

There is no such equivalent on Unix-like systems. The cp utility does have a-p flag that preserves mode, ownership, and timestamps, but all of this is done manually.

I'm not sure it's good idea to change what copyFile does now – it would break backward compatibility in a rather subtle way. It may be better to add a new function that copies the file contents as well as metadata.


TODO:

  • preserve atime and mtime (need copyFileTime?)
  • preserve ownership + group (should ignore failures; need getOwner, setOwner, copyOwner, and similarly for groups)
  • preserve permissions (must occur after chown)
  • what about symlinks? (answer: will operate on the target of the symlinks as a naive implementation would, which is also consistent with the behavior of CopyFile)

Except for Windows where CopyFile[Ex] is available, ACLs will not get copied because every OS seems to implement them differently.

Member

Rufflewind commented Jan 6, 2016

Windows has CopyFile[Ex], which does copy metadata.

There is no such equivalent on Unix-like systems. The cp utility does have a-p flag that preserves mode, ownership, and timestamps, but all of this is done manually.

I'm not sure it's good idea to change what copyFile does now – it would break backward compatibility in a rather subtle way. It may be better to add a new function that copies the file contents as well as metadata.


TODO:

  • preserve atime and mtime (need copyFileTime?)
  • preserve ownership + group (should ignore failures; need getOwner, setOwner, copyOwner, and similarly for groups)
  • preserve permissions (must occur after chown)
  • what about symlinks? (answer: will operate on the target of the symlinks as a naive implementation would, which is also consistent with the behavior of CopyFile)

Except for Windows where CopyFile[Ex] is available, ACLs will not get copied because every OS seems to implement them differently.

@Rufflewind Rufflewind added this to the 1.2.6.0 milestone Feb 3, 2016

@Rufflewind Rufflewind self-assigned this Mar 31, 2016

Rufflewind added a commit to Rufflewind/directory that referenced this issue Apr 14, 2016

Rufflewind added a commit to Rufflewind/directory that referenced this issue Apr 14, 2016

Rufflewind added a commit to Rufflewind/directory that referenced this issue Apr 14, 2016

Rufflewind added a commit to Rufflewind/directory that referenced this issue Apr 14, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment