Copy NTFS files preserving all their attributes ?

[lemonsqueeze]

Hi,

I've been looking for a way to copy NTFS directories preserving all attributes, ACLs, permissions, reparse points etc (surprisingly hard it turns out, short of cloning the whole partition).

I came across the wiki page on Extended Attributes which gives a bit of hope :
Looks like the ntfscp.c and ntfscp.sh additional tools do just that (!)

What's the status on these ?
Why are they not part of ntfs-3g ?
Looks like that would be useful to a lot of people.

(As a side-note it was really confusing intially that it has the same name as ntfs-3g's ntfscp which seem something else entirely)

[jpandre]

ntfscp.c and ntfscp.sh are old code which probably cannot cope with recent situations. They probably cannot copy Windows 10 dlls which contain a compressed stream. At the time they were developed, ntfsprogs was not merged with ntfs-3g, and I did not know there was another ntfscp there, hence the unlucky name.

They are not part of ntfs-3g because they do not make use of ntfs-3g internals, they just use the public file API.

You can adapt them to your own needs under the conditions of GPL2+.

[lemonsqueeze]

Do you think it's possible to make it work with the current state of ntfs-3g provided all the extended attributes are restored ?

Not sure how streams work within ntfs-3g, looks like i should be able to query with ntfs.streams.list at least ?

[jpandre]

What is your use case ?

With current ntfs-3g you probably cannot copy Windows 10 system files with all their attributes. For backing up a Windows 10 system partition, you should clone it.

[lemonsqueeze]

Trying to relocate parts of a Windows 10 installation to a different disk (Program Files, Users). Being able to move Windows directory as well would be nice but i can live without.

Are you saying installing the system compression plugin is not enough to handle all types of compressed files ?

I'm thinking, actually why not just use tar with extended attributes to dump the whole directory and restore it on the other drive ? That's pretty much what ntfscp.c / ntfscp.sh does, right ?

# cd /mnt/c
# tar --xattrs --xattrs-include='*' -cf - "Program Files"  |  (cd /mnt/d 
  tar --xattrs --xattrs-include='*' -xf - )

[jpandre]

With the plugin you will be able to decompress files, but you will not be able to compress them back.

You will not be able to pair short (8+3) names to long ones on the files which have multiple names (hard links).

With tar, you will not be able to collect short names and copy them.

You may also have the adapt the ACLs to the IDs of users and system defined on the target system.

But, maybe you do not need any of these features and more, you may have to try in order to know.

[lemonsqueeze]

Giving up for now trying to make it work on the linux side, although this is probably enough for Program Files.

So far the best (free) solution I found on the windows side is to use ln.exe which surprisingly also has a better-than-robocopy mode for copying files in addition to creating links:

• Boot a Windows 10 LiveCD, Hiren's Boot CD PE here
• Fix drive letters so they match original system (Disk Management)
• cmd.exe, copy files:
  ln.exe --backup --splice --copy "C:\Program Files" "D:\Program Files"
• Rename original folder, symlink to the D: copy:
  mkdir C:\orig
move "C:\Program Files" C:\orig
mklink /D "C:\Program Files" "D:\Program Files"
• Reboot under windows, see what's broken =)

Lacking a good way to compare the 2 directories see what's missing we don't really know what we're doing, but seems ok so far (Xbox gaming overlay is broken but can disable it). At any rate it does much better than robocopy which doesn't handle junctions, so have to be excluded and recreated by hand...

--splice is needed so that junctions are preserved/adapted correctly. ln.exe also has another mode for rescue situations, copying directories to another drive and back (--anchor instead of --splice) but this won't work here.

Really need a good tool to compare directories and all their metadatas...
For ACLs we can use icacls.exe
For the rest if someone knows of a decent tool to list files with all their ntfs attributes please let me know (dir.exe fails miserably). Maybe i'll write something using ntfs-3g metadata, that should cover the majority of cases at least ...

Still find it hard to believe Windows lacks a robust cp and ls, but maybe that's because i'm approaching it from the unix side of things =)