Skip to content

Conversation

@janishorsts
Copy link

@janishorsts janishorsts commented Jul 14, 2025

Upgrading to the latest changes from the upstream https://github.com/tonistiigi/fsutil.

Disabled the codecov/codecov-action – not relevant to us. Unless somebody sees the value.

musaprg and others added 30 commits September 26, 2023 03:38
Signed-off-by: Kotaro Inoue <inoue.kotaro@linecorp.com>
Signed-off-by: Kotaro Inoue <inoue.kotaro@linecorp.com>
Before this, copy would always refuse to replace an existing path in the
dest with something from the source of a different type. E.g. if there
was a symlink at the dest, you'd get an error if copying src meant
replacing the symlink w/ a directory.

This is the right default behavior but there are situations in which
callers may want to opt in to instead always replacing what's in dest
with what's in source.

This adds an option to enable that behavior.

Signed-off-by: Erik Sipsma <erik@sipsma.dev>
gogo/protobuf has been deprecated since 2022. This change uses Google's
code generator instead of gogo/protobuf.

Signed-off-by: Kazuyoshi Kato <kato.kazuyoshi@gmail.com>
Add support for always overwriting existing paths
Signed-off-by: Kotaro Inoue <k.musaino@gmail.com>
Downstream issue: https://github.com/macOScontainers/rund/issues/28

Signed-off-by: Marat Radchenko <marat@slonopotamus.org>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
chore: fix typos in NewFilterFS docstring
This reverts commit bea810c.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Revert "Migrate off of gogo/protobuf"
…ce-copy

Fix cross-device copy failing on macOS
…-bits

copy: fix to respect most significant octet of given permission in copyFileInfo
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
test with coverage and send to codecov
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
tonistiigi and others added 16 commits March 18, 2025 10:48
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
…ctions/nick-fields/retry-3.0.2

chore(deps): Bump nick-fields/retry from 3.0.0 to 3.0.2
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
This optional mode allows skipping transferring the file data
if remote side just hopes to analyze how the local files look like.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This is needed because metadata-only parent may have
a data file as a child.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Extended attributes from `com.apple` will bust remote layer caches
and shouldn't be transferred to a Linux environment. Skip loading these
extended attributes as part of loading xattrs.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
Using "/" was causing a silent bug later on at
`fs.go:121` that is expecting platform-specific
separators. See discussion at moby/buildkit#6007

Fix this by using `\\` on Windows and `/` on unix.

Signed-off-by: Anthony Nandaa <profnandaa@gmail.com>
…ecific-separator

fix: send: use platform-specific root path
Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @janishorsts, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly updates the repository by merging upstream changes, focusing on modernizing the build and CI/CD infrastructure, enhancing file system utility features, and improving cross-platform compatibility. It introduces new capabilities for file copying and transfer, along with refinements to internal data handling and static analysis configurations.

Highlights

  • Build & CI/CD Enhancements: Updated Go version from 1.20 to 1.23 and tonistiigi/xx version from 1.1.0 to 1.6.1 in the Dockerfile. Added .github/CODEOWNERS for PR approval requirements and .github/dependabot.yml for GitHub Actions dependency updates. The .gitignore was simplified, and .golangci.yml had its timeout increased and a depguard rule added to deny io/ioutil usage. docker-bake.hcl was significantly refactored to support cross-platform builds, linting, and generated file validation.
  • File Copying Features: Introduced new options in copy/copy.go such as ModeStr for non-octal permission string application, and AlwaysReplaceExistingDestPaths to force overwriting existing destination paths regardless of type. The MkdirAll function now returns a list of created directories, and copyFileInfo logic was updated for Linux and Unix systems to correctly apply special permission bits (setuid, setgid, sticky) using dchapes-mode.
  • Core File Utility Improvements: A new buffer.go and buffer_test.go were added for efficient byte buffer management. diff_containerd.go was updated to use stat.Clone() for safer stat object manipulation and filepath.Separator for platform-agnostic path handling. A WithHardlinkReset function was introduced in hardlinks.go to correctly handle hardlinks in filtered file systems.
  • Cross-Platform Compatibility: New copy_otherbsd.go and copy_windows_test.go files were added to extend platform support. Path normalization was implemented in receive.go and send.go to convert Unix-style paths to platform-specific paths and vice-versa during transfers, improving cross-OS compatibility. Darwin-specific xattr filtering was added in stat_unix.go to skip com.apple. attributes.
  • File Transfer Protocol & Metadata: The receive.go protocol implementation was enhanced with a MetadataOnly option in ReceiveOpt, allowing clients to request only file metadata without content. This feature buffers stat information and writes it to a .fsutil-metadata file upon completion of the transfer.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request is a large upstream merge that introduces a wide range of improvements and new features. Key changes include updating the Go version and dependencies, migrating from gogo/protobuf to the more performant vtprotobuf, and a significant overhaul of the build system using Docker Buildx Bake.

The file copying and synchronization logic has been enhanced with new features like string-based file mode settings, an option to always replace destination paths, and improved handling of directory timestamps. The fsutil file-transfer protocol is now more robust with platform-agnostic path handling. Additionally, there are bug fixes and improvements in filtering, hardlink handling, and platform-specific code for BSD variants.

The overall quality of the changes is high, with new features being accompanied by tests. However, I've identified a critical issue in bench/diffcopy.go where a merge conflict was left unresolved. This must be addressed before the pull request can be merged.

@janishorsts janishorsts force-pushed the 11-merge-tonistiigi-upstream branch from a3f753b to ccf3e4a Compare July 14, 2025 18:37
@janishorsts janishorsts force-pushed the 11-merge-tonistiigi-upstream branch from ccf3e4a to 7850716 Compare July 14, 2025 18:53
@janishorsts janishorsts marked this pull request as ready for review July 14, 2025 19:05
@gilescope gilescope merged commit fd33950 into master Aug 10, 2025
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.