Skip to content

Implement reflink copy with automatic fallback for faster tree copying#557

Merged
sourcefrog merged 10 commits intomainfrom
copilot/fix-e6904171-2332-4413-bcef-54a848430d71
Oct 11, 2025
Merged

Implement reflink copy with automatic fallback for faster tree copying#557
sourcefrog merged 10 commits intomainfrom
copilot/fix-e6904171-2332-4413-bcef-54a848430d71

Conversation

Copy link
Contributor

Copilot AI commented Oct 6, 2025

Plan: Implement Reflink Copy Support

  • Explore repository and understand code structure
  • Verify current tests pass
  • Add reflink crate to Cargo.toml dependencies
  • Modify copy_tree.rs to use reflink for file copying with fallback
  • Add debug logging to indicate if reflinks are working or not
  • Test the implementation
  • Run all tests to ensure no regressions
  • Fix clippy warnings
  • Address code review: use short type names with imports
  • Address code review: add warning for non-unsupported reflink errors
  • Document feature in NEWS.md
  • Change reflink error logging from warn to debug
  • Use crate::Result instead of anyhow::Result for consistency
  • Handle Windows reflink failures that don't return ErrorKind::Unsupported

Implementation Summary

Successfully implemented reflink copy support with automatic fallback to regular copying:

Changes Made:

  1. Added reflink crate (v0.1.3) to Cargo.toml dependencies

  2. Created copy_file helper function that:

    • Attempts to use reflink::reflink() first for fast copy-on-write operations
    • Gives up on reflinks after any error (not just ErrorKind::Unsupported) to handle Windows behavior where reflink can fail without returning the expected error kind
    • Logs all reflink errors with debug! before falling back (appropriate since users can't do anything about them)
    • Automatically falls back to regular fs::copy() when reflinks fail
    • Uses an AtomicBool to track support so we don't retry reflink after detecting it's not working
    • Returns the number of bytes copied for progress tracking
    • Uses crate::Result for consistency with the rest of the codebase
  3. Added debug logging:

    • Emits debug!() with error details when reflink fails
    • Final summary log includes reflink_used boolean field
  4. Code quality improvements:

    • Uses short type names (Path, fs) instead of fully qualified names
    • All existing tests pass (11 copy_tree tests + 219 total tests)
    • No clippy warnings
    • Removed unused imports
    • Added explanatory comments about Windows behavior
  5. Documentation:

    • Added entry to NEWS.md describing the reflink optimization

How It Works:

  • On first file copy, attempts reflink
  • If reflink succeeds, uses it for all subsequent files (very fast on supporting filesystems like Btrfs, XFS, APFS)
  • If reflink fails with any error, marks it as unsupported and uses regular copy for all remaining files (handles Windows where reflink failures may not return ErrorKind::Unsupported)
  • Debug logs show whether reflinks were used for the entire tree copy operation

Fixes #473

Original prompt

This section details on the original issue you should resolve

<issue_title>Copy with reflinks</issue_title>
<issue_description>On large trees, where the temp dir is on the same filesystem as the source, copying with reflinks might be a little faster.

https://docs.rs/reflink/</issue_description>

<agent_instructions>In copy_tree, copy using reflinks, using the reflink crate. If reflinks aren't supported on the filesystem, fall back to a regular copy. If possible emit a debug! line saying whether they worked or not.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes #473

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Copy with reflinks Implement reflink copy with automatic fallback for faster tree copying Oct 6, 2025
Copilot AI requested a review from sourcefrog October 6, 2025 14:04
@sourcefrog
Copy link
Owner

@copilot Nice. Also mention this in NEWS.md.

Copy link
Contributor Author

Copilot AI commented Oct 6, 2025

@copilot Nice. Also mention this in NEWS.md.

Added to NEWS.md in commit 7c3b7c7.

Copilot AI and others added 6 commits October 6, 2025 07:39
Co-authored-by: sourcefrog <346355+sourcefrog@users.noreply.github.com>
Co-authored-by: sourcefrog <346355+sourcefrog@users.noreply.github.com>
…r reflink errors

Co-authored-by: sourcefrog <346355+sourcefrog@users.noreply.github.com>
Co-authored-by: sourcefrog <346355+sourcefrog@users.noreply.github.com>
@sourcefrog sourcefrog force-pushed the copilot/fix-e6904171-2332-4413-bcef-54a848430d71 branch from b2e14bf to 362af15 Compare October 6, 2025 14:40
Co-authored-by: sourcefrog <346355+sourcefrog@users.noreply.github.com>
Copilot AI requested a review from sourcefrog October 6, 2025 14:56
@sourcefrog sourcefrog requested a review from Copilot October 6, 2025 15:00
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements reflink (copy-on-write) support for the tree copying operation to improve performance on filesystems that support it, with automatic fallback to regular copying when not supported.

  • Adds reflink crate dependency for copy-on-write file operations
  • Implements smart reflink detection with fallback mechanism using AtomicBool
  • Adds debug logging to indicate reflink support status and usage

Reviewed Changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/copy_tree.rs Implements the core reflink functionality with copy_file helper and reflink support detection
NEWS.md Documents the reflink optimization feature in the changelog
Cargo.toml Adds reflink crate dependency

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

patch = "0.7"
path-slash = "0.2"
quote = "1.0.35"
reflink = "0.1"
Copy link

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using a more specific version constraint like reflink = \"0.1.3\" to match the version mentioned in the PR description, ensuring consistent dependency resolution.

Suggested change
reflink = "0.1"
reflink = "0.1.3"

Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
src/copy_tree.rs Outdated

/// Copy a file, attempting to use reflink if supported.
/// Returns the number of bytes copied.
fn copy_file(src: &Path, dest: &Path, reflink_supported: &AtomicBool) -> anyhow::Result<u64> {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot can you use crate::Result here instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to crate::Result in commit ea4c3f2.

Co-authored-by: sourcefrog <346355+sourcefrog@users.noreply.github.com>
src/copy_tree.rs Outdated
}
Err(e) => {
// Check if this is a "not supported" error
if e.kind() == ErrorKind::Unsupported {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot It looks like on Windows it can fail as unsupported without returning this error. Let's give up on using reflinks the first time we see an error of any kind, still emitting a debug message with the error. Add a comment about this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to give up on reflinks after any error in commit a8f7b29. Added comment explaining Windows behavior and removed the now-unused ErrorKind import.

Co-authored-by: sourcefrog <346355+sourcefrog@users.noreply.github.com>
Copilot AI requested a review from sourcefrog October 11, 2025 14:15
@sourcefrog sourcefrog marked this pull request as ready for review October 11, 2025 23:39
@sourcefrog sourcefrog merged commit 81357d8 into main Oct 11, 2025
81 checks passed
@sourcefrog sourcefrog deleted the copilot/fix-e6904171-2332-4413-bcef-54a848430d71 branch October 11, 2025 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Copy with reflinks

3 participants