Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use rename of directories instead of symbolic links in boot partition. #1967

Open
wants to merge 1 commit into
base: master
from

Conversation

@valentindavid
Copy link

@valentindavid valentindavid commented Nov 3, 2019

This implements @AdrianVovk idea from #1719 (comment) to solve issue #1719.

This uses renameat2 to do atomic swap of the loader directory in the
boot partition. It fallsback to non-atomic rename. This stays atomic
on filesystems supporting links but also provide a non-atomic behavior
when filesystem does not provide any atomic alternative.

This is working with SystemD boot on EFI using boot loader
specifications.

There is still the issue of losing /loader/loader.conf with SystemD
boot. Maybe we should think about copying other files from previous loader directories.

@openshift-ci-robot
Copy link
Collaborator

@openshift-ci-robot openshift-ci-robot commented Nov 3, 2019

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: valentindavid
To complete the pull request process, please assign cgwalters
You can assign the PR to them by writing /assign @cgwalters in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot
Copy link
Collaborator

@openshift-ci-robot openshift-ci-robot commented Nov 3, 2019

Hi @valentindavid. Thanks for your PR.

I'm waiting for a ostreedev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@rh-atomic-bot
Copy link
Collaborator

@rh-atomic-bot rh-atomic-bot commented Nov 3, 2019

Can one of the admins verify this patch?
I understand the following commands:

  • bot, add author to whitelist
  • bot, test pull request
  • bot, test pull request once
@valentindavid valentindavid force-pushed the valentindavid:support-vfat branch from c3d34fb to c36ee32 Nov 3, 2019
@AdrianVovk
Copy link

@AdrianVovk AdrianVovk commented Nov 3, 2019

The challenge with this is that the ESP is vfat, and therefore rename will always be non-atomic. It's still better than what I'm doing now (manually copying files into /efi)

#1951
^ here's a proposed solution that I think could solve this same problem without needing the renameat2 call.

@rh-atomic-bot
Copy link
Collaborator

@rh-atomic-bot rh-atomic-bot commented Dec 11, 2019

The latest upstream changes (presumably #1767) made this pull request unmergeable. Please resolve the merge conflicts.

gnomesysadmins pushed a commit to GNOME/gnome-build-meta that referenced this pull request Jan 3, 2020
Systemd boot requires the boot partition to be the EFI partition which
means FAT is required. OSTree uses symlinking as a way to do atomic
update.

There is no solution yet for atomic update of FAT partitions. THis
patch however changes symlinking by doing directory move which can be
atomic in conditions. In practice filesystems with symbolic link
support usually also support atomic rename of directories. But this
allows to also work when no atomic update is working.

Patch was submitted upstream as
ostreedev/ostree#1967
gnomesysadmins pushed a commit to GNOME/gnome-build-meta that referenced this pull request Jan 6, 2020
Systemd boot requires the boot partition to be the EFI partition which
means FAT is required. OSTree uses symlinking as a way to do atomic
update.

There is no solution yet for atomic update of FAT partitions. THis
patch however changes symlinking by doing directory move which can be
atomic in conditions. In practice filesystems with symbolic link
support usually also support atomic rename of directories. But this
allows to also work when no atomic update is working.

Patch was submitted upstream as
ostreedev/ostree#1967
gnomesysadmins pushed a commit to GNOME/gnome-build-meta that referenced this pull request Jan 13, 2020
Systemd boot requires the boot partition to be the EFI partition which
means FAT is required. OSTree uses symlinking as a way to do atomic
update.

There is no solution yet for atomic update of FAT partitions. THis
patch however changes symlinking by doing directory move which can be
atomic in conditions. In practice filesystems with symbolic link
support usually also support atomic rename of directories. But this
allows to also work when no atomic update is working.

Patch was submitted upstream as
ostreedev/ostree#1967
gnomesysadmins pushed a commit to GNOME/gnome-build-meta that referenced this pull request Jan 14, 2020
Systemd boot requires the boot partition to be the EFI partition which
means FAT is required. OSTree uses symlinking as a way to do atomic
update.

There is no solution yet for atomic update of FAT partitions. THis
patch however changes symlinking by doing directory move which can be
atomic in conditions. In practice filesystems with symbolic link
support usually also support atomic rename of directories. But this
allows to also work when no atomic update is working.

Patch was submitted upstream as
ostreedev/ostree#1967
gnomesysadmins pushed a commit to GNOME/gnome-build-meta that referenced this pull request Jan 14, 2020
Systemd boot requires the boot partition to be the EFI partition which
means FAT is required. OSTree uses symlinking as a way to do atomic
update.

There is no solution yet for atomic update of FAT partitions. THis
patch however changes symlinking by doing directory move which can be
atomic in conditions. In practice filesystems with symbolic link
support usually also support atomic rename of directories. But this
allows to also work when no atomic update is working.

Patch was submitted upstream as
ostreedev/ostree#1967
gnomesysadmins pushed a commit to GNOME/gnome-build-meta that referenced this pull request Jan 23, 2020
Systemd boot requires the boot partition to be the EFI partition which
means FAT is required. OSTree uses symlinking as a way to do atomic
update.

There is no solution yet for atomic update of FAT partitions. THis
patch however changes symlinking by doing directory move which can be
atomic in conditions. In practice filesystems with symbolic link
support usually also support atomic rename of directories. But this
allows to also work when no atomic update is working.

Patch was submitted upstream as
ostreedev/ostree#1967
@bam80
Copy link

@bam80 bam80 commented Feb 21, 2020

Is this PR still proposed or abandoned?

@valentindavid
Copy link
Author

@valentindavid valentindavid commented Feb 21, 2020

Is this PR still proposed or abandoned?

I can rebase and take care of it a bit. But since there was no interaction from maintainers, then I am not sure whether they are considering it.

@bam80
Copy link

@bam80 bam80 commented Mar 23, 2020

@valentindavid have you been able to pay some attention to this issue again? The changes are highly anticipated here, in one form or another. Thanks.

@jjardon
Copy link
Contributor

@jjardon jjardon commented Apr 7, 2020

@mwleeds @rfairley any chance to take a look to this? We currently need this to make GNOME images work, see https://gitlab.gnome.org/GNOME/gnome-build-meta/-/commit/79fb62e0d243a21ab58dc1dda439c23db5d474ab

This uses `renameat2` to do atomic swap of the loader directory in the
boot partition. It fallsback to non-atomic rename. This stays atomic
on filesystems supporting links but also provide a non-atomic behavior
when filesystem does not provide any atomic alternative.

This is working with SystemD boot on EFI using boot loader
specifications.

There is still the issue of losing `/loader/loader.conf` with SystemD
boot.
@jjardon
Copy link
Contributor

@jjardon jjardon commented May 30, 2020

@cgwalters Hey! Any chance to take a look to this?

@bam80
Copy link

@bam80 bam80 commented Aug 6, 2020

Friendly ping.

@AdrianVovk
Copy link

@AdrianVovk AdrianVovk commented Sep 27, 2020

@valentindavid This PR conflicts with upstream again. GNOME OS uses an ancient version of OSTree so they have yet to run into this issue, but on my OS I just updated to the latest version and I'm about to be downgrading again because this patch is broken.

@cgwalters Is there any chance for this patch to be reviewed or considered at all? Without this patch, I have to follow every single ostree admin deploy anywhere in the process of building, generating ISOs, installing the OS, and updating the OS with a bunch of ugly commands that mess with the contents of the ESP. With this patch, it just works. OSTree operates on BLS files, but has yet to support the bootloader that was actually built for the standard 🤷

@damianatorrpm
Copy link

@damianatorrpm damianatorrpm commented Oct 24, 2020

updated.txt
Rebased patch file.

@kowalski7cc
Copy link

@kowalski7cc kowalski7cc commented Nov 25, 2020

Any news on this?

@openshift-ci-robot
Copy link
Collaborator

@openshift-ci-robot openshift-ci-robot commented Apr 12, 2021

@valentindavid: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

8 participants