Writing USB key broken after April Update

[angelog0]

After April update I cannot copy files on USB key preserving their timestamps.

Before the update, I had created the mount point:

sudo mkdir -p /mnt/e/

and could copy files preserving timestamps:

sudo mount -t drvfs e: /mnt/e
/bin/cp -p foo /mnt/e/
rsync -av bar /mnt/e/

I did and used this since Fall Creators Update.

Now, I get:

/bin/cp -p foo /mnt/e/
/bin/cp: preservati gli orari di '/mnt/e/foo': Operazione non permessa

/usr/bin/rsync -av foo /mnt/e/
sending incremental file list
foo
rsync: mkstemp "/mnt/e/.foo.lMsStT" failed: Operation not permitted (1)

sent 103 bytes  received 111 bytes  428.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]

If you do not have a quick solution about this, what I described is a big regression of WSL April Update which has brought WSL back before Fall Update was released...

[0xbadfca11]

Maybe #2770

[angelog0]

@0xbadfca11 wrote:

Maybe #2770

but I don't see any clear solution/recipe that fixes that problem. At the moment the possibility of using USB key directly in WSL is of NO usefulness. Things are back to before

sudo mkdir -p /mnt/e/
sudo mount -t drvfs e: /mnt/e

was introduced when WSL knew only fixed drive C:...

I tried also to create /mnt/e whitout root:

cd
mkdir e
sudo mv e /mnt/

Now /mnt/e has owener and group the user, not root, but this gives the same issue.

How is it possible that developers have introduced this big regression/backward incompatibility? It seems that WSL is back to its status of beta...

Please, give with simple and clear examples how we can write an NTFS USB key as we do on /mnt/c, don't speak in abstract, please...

This is another reason for which W10 must leave the user the faculty to do an upgrade or not. My Fall update was working very well and since the system upgraded to April Update I have spent about 80% of my time to fix this or that bug/regression!

ALL this is very annoying! Already in January an update (KB4056892) put KO my PC and I had to reinstall from scratch...

Maybe is time to remove completely Windows from my systems...

[benhillis]

@SvenGroot - would you mind taking a look at this? Seems like this should work.

[angelog0]

Hi @benhillis, please explain what does it mean Seems like this should work. Here also this does not work anymore:

rsync -av foo.tmp/ /mnt/e/foo.tmp/
sending incremental file list
created directory /mnt/e/foo.tmp
rsync: chgrp "/mnt/e/foo.tmp/." failed: Operation not permitted (1)
rsync: chgrp "/mnt/e/foo.tmp/bar" failed: Operation not permitted (1)
./
bar -> foo.txt
foo.txt
rsync: mkstemp "/mnt/e/foo.tmp/.foo.txt.ZUiZwf" failed: Operation not permitted (1)

sent 169 bytes  received 312 bytes  962.00 bytes/sec
total size is 14  speedup is 0.03
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]

and this does not work:

tar -xvof foo.tar.gz -C /mnt/e/
foo.tmp/
foo.tmp/bar
tar: foo.tmp/bar: funzione "utime" non riuscita: Operazione non permessa
foo.tmp/foo.txt
tar: foo.tmp/foo.txt: funzione "utime" non riuscita: Operazione non permessa
tar: foo.tmp: funzione "utime" non riuscita: Operazione non permessa
tar: foo.tmp: impossibile cambiare il modo a rwxr-xr-x: Operazione non permessa
tar: Uscita con stato di fallimento in base agli errori precedenti

That means all my scripts to save my WSL home are broken and I have to resume the old strategy I adopted when WSL knew only C fix drive, i.e. creating a tarball, moving it in Windows and then unpacking it on the USB key...

[CENSORED by @benhillis ]

[benhillis]

I'm saying it sounds like a bug that we should investigate. Please keep your discourse professional and civil.

[angelog0]

@benhillis wrote:

Please keep your discourse professional and civil.

Now you should prove that you are civil restoring what I wrote... if you developers have created a mess, you should be happy that someone stress this...

In any case, for the sake of completeness, it seems that adding the option metadata,

sudo mount -t drvfs e: /mnt/e -o metadata

allows cp - p, tar and rsync to work as expected.

In the example you have put on the WEB, often mount is used without sudo but if I do not use sudo I get an error:

mount -t drvfs e: /mnt/e -o metadata
mount: only root can use "--options" option

mount -t drvfs e: /mnt/e
mount: only root can use "--types" option

Usually I use the same USB key both in Windows and WSL. Now, when I mount it adding metadata option, the Windows files are root root while those copied with WSL are utente utente. Maybe this has its logic but intuitively an user would expect utente utente...

Hi @benhillis, I hope you do not yet censor what I wrote, you would make you a bad figure...

[SvenGroot]

This is indeed a manifestation of #2770. When you mount your USB stick with default options, all the files will be owned by root, and your user doesn't have the permission to change the timestamps in that case. DrvFs used to bypass the owner check, but this is no longer the case since we added support for different owners with the "metadata" option.

Mounting with "metadata" works in this case because it allows the files you create to be owned by you, rather than root, however it's not the only option (and doesn't solve the problem for files already created by Windows, which don't have metadata and would therefore still be owned by root). For this reason, we added mount options to DrvFs that specify the owner uid and gid to use for files that don't have metadata.

The solution is to mount your USB drive using mount options setting the default uid/gid for files without metadata:
sudo mount -t drvfs e: /mnt/e -o uid=1000,gid=1000

You should use your actual uid/gid instead of 1000 (run id to determine your default account's uid/gid). Using the metadata option shouldn't be necessary unless you want to use non-default Linux permissions. If you do want to use this option, I recommend you use it together with the uid/gid options.

DrvFs mount options are described at the bottom of this blog post.

[Mornon]

I'm having a similar issue, but -o metadata doesn't solve it: if I try to run rsync on a USB stick I get errors like this:

rsync: mkstemp "/mnt/f/2017-2018/.5A.xlsx.NbvyUv" failed: Operation not permitted (1)

I tried with --no-t, with --no-owner and with -o metadata, to no avail.

Edit: I tried with uid and gid, and it does work; do you plan to go back to users be able to just use mount, with no additional options?

[Zythyr]

This is indeed a manifestation of #2770. When you mount your USB stick with default options, all the files will be owned by root, and your user doesn't have the permission to change the timestamps in that case. DrvFs used to bypass the owner check, but this is no longer the case since we added support for different owners with the "metadata" option.

Mounting with "metadata" works in this case because it allows the files you create to be owned by you, rather than root, however it's not the only option (and doesn't solve the problem for files already created by Windows, which don't have metadata and would therefore still be owned by root). For this reason, we added mount options to DrvFs that specify the owner uid and gid to use for files that don't have metadata.

The solution is to mount your USB drive using mount options setting the default uid/gid for files without metadata:
sudo mount -t drvfs e: /mnt/e -o uid=1000,gid=1000

You should use your actual uid/gid instead of 1000 (run id to determine your default account's uid/gid). Using the metadata option shouldn't be necessary unless you want to use non-default Linux permissions. If you do want to use this option, I recommend you use it together with the uid/gid options.

DrvFs mount options are described at the bottom of this blog post.

I think some of the issues are coming from how and when WSL will automatically mount a drive. Maybe you can provide some insight on my question below?

Issue: I have noticed that WSL will sometimes automatically mount a drive. If it automatically mounts, the user/owner of the mount is set to the process that launched the bash.exe (generally uid=1000,gid=1000) . If it does NOT automatically mount, then I have to manually mount it using sudo mount -t drvfs e: /mnt/e, which results in the user/owner being set as root.

The general trend I have noticed is as follows:
If a USB flash drive is connected, then WSL will NOT automatically mount the drive
If a HDD is connected using SATA to USB cable, then WSL will automatically mount the drive

Why is this happening? What command can one use to determine if it was automatically mounted?

[benhillis]

@Zythyr - We only automatically mount fixed drives.

When you run sudo, you effectively become root. If you want to mount as a specific user, you will have to specify the uid and gid mount options:

For example:
sudo mount -t drvfs e: /mnt/e -o uid=1000,gid=1000

[Zythyr]

@Zythyr - We only automatically mount fixed drives.

When you run sudo, you effectively become root. If you want to mount as a specific user, you will have to specify the uid and gid mount options:

For example:
sudo mount -t drvfs e: /mnt/e -o uid=1000,gid=1000

Yes, I understand that, however my question was: Why is it that WSL sometimes automatically mounts a drive and sometimes it doesn't? My use case is for using rsync. Prior to me knowing that I have to use sudo mount -t drvfs e: /mnt/e -o uid=1000,gid=1000 instead of sudo mount -t drvfs e: /mnt/e I randomly had permission related issues. Now I realize it was because of how drive was originally mounted (automatically vs. manually). Why is it that WSL automatically mounts external HDD but does NOT automatically mount USB flash drive?

[benhillis]

@Zythyr - USB flash drives are not fixed drives, they are removable.

[Zythyr]

@Zythyr - USB flash drives are not fixed drives, they are removable.

Aren't external drives removable? I have a HDD connected to PC using SATA to USB cable. It act just like a USB flash drive, so why does WSL discriminate between the two?

Independent of a drive is "removable" or not, why does WSL not automatically mount it? Isn't this a logical approach? For example, if I were using a Ubuntu OS, and I connect a USB flash drive, it would automatically mount the drive, I don't have to manually go in the Terminal to mount it... Shouldn't WSL mimic this same behavior?

[therealkenc]

Why is it that WSL sometimes automatically mounts a drive and sometimes it doesn't?

Part of confusion derives from the choice of nomenclature.... (1) Drives aren't auto-mounted in WSL in the sense of the word commonly used in Linux. They're static mounted, at startup. Your SATA drive on a wire is treated as a fixed drive because that's what it looks like at startup. Your USB stick does not. (2) WSL doesn't actually have "drives" in the Linux sense either. There are no block devices in WSL that come and go, like in Real Linux.

For example, if I were using a Ubuntu OS, and I connect a USB flash drive, it would automatically mount the drive, I don't have to manually go in the Terminal to mount it... Shouldn't WSL mimic this same behavior?

The landing zone for that is #560 (message). Rhetoricals usually don't get much of a reply, so you can probably consider that part answered by yourself (affirmative or negative; whichever you prefer).

[angelog0]

With all due respect, from my POV this behavior of WSL respect to USB devices is of no usefulness and I renounced it. I prefer to use a Windows folder as a sort of "cloud" to exchange data between WSL and the USB device. Usually, in the time it would take to mount (and then to unmount) the device as suggested in this discussion I have already transferred the data (with the help of MSYS2 in wich USB device are automounted).

Merry Christmas and Happy New Year.