When a transferring files in nemo, it should, by default, "complete" at 99% and then start immediately syncing files.

[runningvisor]

I made a new discussion since I had a confusing title on my previous one. I had since changed it but still feel like no one was actually getting what I was trying to suggest.

The behavior I'm suggesting is as I stated in the title. Yes, it may be slightly annoying to be "stuck" at 99%, but your average user isn't going to know Linux works differently. They'll just see the file transfer progress disappear at 100%, see the files on the drive with nothing else going on, and assume it's done before pulling the drive and losing data, only learning after that they need to eject drives properly. Even if the drive had a light, Linux seemingly waits for you to eject it to write the data.

It's in this moment many people may learn for the first time what ejecting a drive even is as they never had any problems just unplugging it on Windows and/or have always heard "you don't need to do that anymore". I feel like data loss prevention in this case is better, especially when the files being transferred are irreplaceable and they haven't been properly informed about backups.

[Fantu]

writing many messages about the same thing and opening multiple threads is not good.

I think a reliable end-to-end time estimate (from "copy started" to "physically written to the platter/NAND") is very hard, maybe not realistically achievable, and it's worth laying out why.

There are two caches stacked on top of each other, plus a hardware reliability problem underneath both:

1. Host-side cache. The OS knows how much data is still pending writeback, but that number keeps changing while the transfer is still running, so it's already a moving target before the disk is even involved.

2. Device-side cache. Even if the host-side write cache is disabled, the device controller can still have its own internal delays (wear leveling, mapping table updates, garbage collection) that the host has no visibility into.

3. This is getting worse, not better, on modern hardware. DRAM-less SSDs using dynamic SLC caching add another layer that isn't visible to the host, and some of them even borrow host RAM for this (Host Memory Buffer), meaning part of the "pending" data lives outside the device entirely while still not being durable. Furthermore in such disks the difference in performance from the initial burst with cache (which often becomes shorter) and when there is no more cache is immense.

4. Disabling both caches is technically possible, but not a good default. It kills write performance, in all cases only to improve the estimates and reduce the cases where the disk is still writing when the copy is apparently finished.

5. Even if you wanted to disable it, you often can't rely on it working. Some drives, not just cheap USB/SD bridges but SATA/NVMe SSDs (and some HDDs) too, don't reliably honor cache-flush or write-cache-disable commands. This isn't only a bus/bridge problem, some firmware ignores or only partially implements these commands, sometimes to score better in benchmarks, so even asking a drive to guarantee synchronous writes doesn't always guarantee it.

Given all that, I'd say a precise progress/time estimate for "fully safe to remove" isn't realistic across the range of hardware people actually use.

---

EDIT:

Quick summary on existing mitigations, for reference:

Windows: since 10 1809, "Quick removal" is the default for USB flash drives, disabling write caching so writes go straight to the disk. This is a device-level write-cache policy (ATA/USB), toggleable per device. (in practice this is not always effective, see point 5 above)
Linux (UDisks2): the flush mount option is applied by default, but only for vfat/FAT FWIK. That matches the risk profile: FAT has no journal, so an interrupted write can corrupt the filesystem structure itself, not just lose the latest data. Journaled filesystems (ext4, NTFS) can recover structural consistency via journal replay on next mount, though in-flight data can still be lost either way.

So both OSes already mitigate this at the software level, on the filesystem/device layer most exposed to the risk. Neither is a full guarantee, for the firmware-reliability reasons discussed above.

[billyswong]

Your points are valid. But I think there is still some miscommunication here. What we try to get is not a "reliable end-to-end time estimate", but something that is good enough to convince users that their drive is still not finish writing their files. Something that is good enough to convince users that the computer is not stuck for unrelated nonsense but actually busy working for their files so please be patient. The time estimate being not reliable is totally fine. When we download files from internet, the time estimate is often not reliable too. A lot of Windows software update/installation progress bars don't give reliable time estimates too. People are used to that.

Therefore it will be in users' best interest if the OS can minimize the amount of data not yet written to the drive when the file operations look finished. And thus my proposal in https://github.com/orgs/linuxmint/discussions/429

[runningvisor]

I only created this second thread since it had an unclear title for the majority of the time, and the discussion shifted almost entirely to safe-eject notifications (which we also definitely needed).

I'm not suggesting for even a somewhat accurate progress of the actual writes to the disk. I'm suggesting immediately (by default) to write all the data from the RAM to the drive at "99%" in the nemo transfer dialog with the bouncing indeterminate animation.

I did a test between transferring the same data to a drive in Windows, and doing the same in Linux but also including extra writing time after clicking eject. Both were very similar with Linux leading by up to a minute.

I feel your average user would want to clearly see that it's done over possibly faster transfer speeds (the total time would still be faster than Windows). Advanced users could turn off this "Immediately write after transfer completes" in nemo settings.

[cplittlfield]

It's worth remembering, as well, that this caching was devised as a strategy to mitigate people's impatience with I/O operations that tied up the entire PC while they were happening. Allowing disk I/O to continue in the background while the user proceeded with other tasks was a great benefit, when it was introduced.

Having experienced the old days, I consider the current situation to be a vast improvement, even at the cost of having to wait to unmount/eject safely. I sympathize with people who have no memory of those days, since they have nothing to which to compare the current situation.

I can't help feeling, however, that this is more of an education problem than a technical one. The problem that needs addressing is that, when a USB is unplugged prematurely, it is already too late to issue a warning. Short of finding some way to anticipate the user's intent, Michael Webster's idea of a persistent on-screen warning would seem to mitigate this.

[Fantu]

The mtwebster message I suppose would still appear when performing eject and safe removal operations until it's finished (I assume and hope it doesn't appear during any ongoing operation on removible drives, which I think would be too annoying compared to the benefits). For users who unplug without unmounting, performing eject or safe removal would still be useful, providing a warning and explanation.
So that users who don't know or underestimate it are informed and can improve.
Then for those who, despite everything (out of impatience and stubbornness), continue to physically remove without even unmounting, there's nothing good that can be done (FWIK), and further mitigation measures would be detrimental to all patient and diligent users.

In short, it depends mainly on the user (who has a minimum of knowledge, patience and common sense) and then partly on the quality of the devices.

[billyswong]

A good OS UI design should let the user know, if there are known amount of data pending to be written to a removable drive, better if the user can see how much of them are there. @Fantu is right that there are still unknown extra amount of not written data due to firmware or hardware cheating. But at least for the cache data visible to the OS, they should be visible to the user in the UI design.

About the cache "as a strategy to mitigate people's impatience", it should be used for freeing up the foreground tasks only. For foreground tasks, I mean for example a software application clicking "save" for their huge project file etc. We can take file operations in Nemo (or any other file managers) as yet another form of foreground tasks and add no extra treatment to them. But we are now running a multitasking GUI desktop OS. There is space for a secondary UI sitting in the systray, to tell users to beware there are still cache data writing to their drives in background.

Edit: In the old days, floppy drives got their dedicated LED bulbs for busy writing. Thumb drives and removable drives all got their dedicated LED bulbs for busy writing. the computer cases all got their own LED bulbs for busy writing as a whole too. Now they are all gone. External drives lost their LED indicators for I/O in operation. Computer cases lost their LED indicators for I/O in operations. All the companies save pennies and put us in risk.

[runningvisor]

See my comment here.