Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace linux_reflink extension module #1294

Merged
merged 2 commits into from Mar 15, 2024

Conversation

floppym
Copy link
Contributor

@floppym floppym commented Mar 2, 2024

Python 3.8 added support for copy_file_range in the os module, so we can just call that directly.

Also, we can use the FICLONE ioctl for fast file clones on supported filesystems (btrfs).

If we think this needs some performance testing, I would appreciate any help with that.

@floppym floppym force-pushed the py-copy-reflink branch 2 times, most recently from f704f47 to 8e5716c Compare March 2, 2024 05:07
@floppym
Copy link
Contributor Author

floppym commented Mar 2, 2024

Closing in favor of #1296.

@floppym
Copy link
Contributor Author

floppym commented Mar 2, 2024

Re-opening this since it has some commentary.

@floppym floppym force-pushed the py-copy-reflink branch 5 times, most recently from 0666ebb to f9d765f Compare March 3, 2024 02:46
Copy link
Member

@thesamesam thesamesam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial comments. Thanks a lot for doing this.

I haven't yet tested it or worked through the calls/compared with docs yet though (I assume Zac is planning on doing the same too).

lib/portage/util/file_copy.py Outdated Show resolved Hide resolved
lib/portage/util/file_copy.py Outdated Show resolved Hide resolved
lib/portage/util/file_copy.py Show resolved Hide resolved
@floppym
Copy link
Contributor Author

floppym commented Mar 3, 2024

I haven't yet tested it or worked through the calls/compared with docs yet though (I assume Zac is planning on doing the same too).

I'm currently dog fooding it with an world update of ~250 packages on my main system. No glaring issues noticed yet.

@floppym floppym force-pushed the py-copy-reflink branch 2 times, most recently from 012fdf8 to 464bad7 Compare March 3, 2024 05:16
@floppym floppym force-pushed the py-copy-reflink branch 2 times, most recently from 9857682 to 3e9a817 Compare March 5, 2024 16:20
Copy link
Contributor

@dwfreed dwfreed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just saw this one issue, otherwise seems correct to me.

@floppym floppym force-pushed the py-copy-reflink branch 4 times, most recently from aeac756 to 1db2b58 Compare March 13, 2024 18:51
@floppym floppym force-pushed the py-copy-reflink branch 2 times, most recently from 3a40acf to c22dd63 Compare March 14, 2024 05:22
Python 3.8 added support for copy_file_range in the os module,
so we can just call that directly.

Also, we can use the FICLONE ioctl for fast file clones on supported
filesystems (btrfs).

Signed-off-by: Mike Gilbert <floppym@gentoo.org>
Actually create sparse blocks at the start and end.
Check file size before/after copying.
Ensure sparse output when _fastcopy succeeds.

Signed-off-by: Mike Gilbert <floppym@gentoo.org>
@gentoo-bot gentoo-bot merged commit 0cc3f9e into gentoo:master Mar 15, 2024
@floppym floppym deleted the py-copy-reflink branch March 15, 2024 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants