windows: should we recommend devdrives for storing caches?

[skshetry]

There is DevDrive support on all Windows 11 SKUs.

So what is DevDrive? The following is an excerpt from Microsoft's blog:

Dev Drive is built on the Resilient File System (ReFS) and combined with the new performance mode capability of Microsoft Defender Antivirus, provides up to 30% better performance for overall build times compared to current, in-market Windows 11 (22H2).

We received feedback from developers that slow file io performance was disruptive to their workflow. In response, we analyzed the file system to understand where the bottlenecks were. Benchmarks were created across multiple end-to-end developer scenarios and across multiple languages. We then made optimizations to the file system and introduced a new performance capability for Microsoft Defender Antivirus. Now available in public preview, this new capability provides asynchronous scanning on a Dev Drive and does not change the security posture of your system drive or other drives Asynchronous scanning provides improved security compared to traditional folder and process exclusions which are often used by developers. As a result, Dev Drive improves overall build performance, most notably in heavy file io based scenarios. For more information, see Protecting Dev Drive using performance mode.

ReFS also has reflink/cloning support, but dvc does not support it unfortunately. Reading through the docs, it seems that it'd be perfect for dvc's cache purpose.

Also, checkout the following video:

[blakeNaccarato]

This seems sensible, and I was going to open an issue over in https://github.com/iterative/dvc about it, but I'll just leave a few details here for now. Output of dvc doctor on a ReFS-formatted "dev drive" with Windows 11 shows that DVC doesn't support reflink style caches on a ReFS partition (at least the one offered by the recent Windows 11 dev drive feature). So that would have to be fixed before this can be recommended in docs. See iterative/dvc#1263 back in October, when implementing this was de-prioritized. That seems reasonable, since Windows 10 Home and Windows 10 Pro cannot read ReFS drives.

Windows 10 Pro for Workstations and Windows 11 Pro for Workstations have "full" ReFS support, which I think means you can have your entire OS on a ReFS partition. However, both Windows 11 Home and Pro now support ReFS via the dev drives feature, generally available late 2023. So now that any Windows 11 system can interface with ReFS partitions, it might be a good idea to start thinking about unblocking iterative/dvc#1263 again. Consider of course that despite a supposed Windows 10 "end of life" in 2025, ~90% of Windows machines are still on Windows 10 today, so it's not absolutely critical to prioritize this now.

Although, "ReFs" is mentioned by name in the DVC documentation, but there is no mention of the compatibility issues that a user utilizing "dev drives" is likely to encounter, that their partition won't support reflink caching despite DVC documentation suggesting it does.

See also:

• Python 3.13 dev drive detection: changelog, PR, os.path.isdevdrive docs
• Dev drive prerequisites, "Windows 11, Build 10.0.22621.2338 or later"

[skshetry]

Even without reflink, I think using devdrive is worth it (due to async AV scan).

---

Regarding reflink support, there is reflink_copy.reflink which a Python wrapper for reflink-copy, which does support Windows. Available via pip install reflink-copy.

Since it's untested, and requires a binary package, I am hesitant to use that. But we could optionally support it for Windows if it works (but keep it undocumented for some period). Contributions are welcome.

Relevant code is here:

https://github.com/iterative/dvc-objects/blob/edc151c366a2653381eca52e34f0d9b8e5a06dcc/src/dvc_objects/fs/system.py#L115

[blakeNaccarato]

Awesome, thanks for the details! I have some medium-heavy DVC projects on both an NTFS and ReFS drive right now, I'll have to see if I can clock any reasonable improvements from the async AV alone. And compare that to an NTFS folder excluded from scanning.

I think with the dual benefits of async AV and reflinks, it would convince me to maintain a dev drive for DVC projects. But I've still got 3/4 machines in lab/home on Windows 10 so it's not quite time for me to jump head first into dev drive life.

I've got a couple prior contribution commitments I need to prioritize before adding on another one, but I'll keep this on the backburner if nobody decides to implement it before I start looking more deeply into it.