Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a new ostree admin instutil subcommand to copy EFI binaries to the ESP #1649

Closed
martinezjavier opened this issue Jun 26, 2018 · 10 comments
Closed

Add a new ostree admin instutil subcommand to copy EFI binaries to the ESP #1649

martinezjavier opened this issue Jun 26, 2018 · 10 comments

Comments

@martinezjavier
Copy link
Contributor

Currently the only file in the EFI System Partition (ESP) that is updated is grub.cfg. Every other file in the ESP is the one that was copied by Anaconda on installation. This means that shim, grub, firmware update, etc are never upgraded so users won't get any bugfixes or security updates for the bootloaders files, i.e:

$ rpm -qa | grep grub2-efi-x64
grub2-efi-x64-2.02-26.fc28.x86_64

$ md5sum /boot/efi/EFI/fedora/grubx64.efi 
be1b3601da78bab1483690276c6e47f1  /boot/efi/EFI/fedora/grubx64.efi

$ atomic host upgrade && systemctl reboot

$ rpm -qa | grep grub2-efi-x64
grub2-efi-x64-2.02-34.fc28.x86_64

$ md5sum /boot/efi/EFI/fedora/grubx64.efi 
be1b3601da78bab1483690276c6e47f1  /boot/efi/EFI/fedora/grubx64.efi

ESP is mounted on /boot/efi and the packages copy the EFI binaries to /boot/efi/EFI/fedora/. The rpm-ostree unpacker translates the /boot path to /usr/lib/ostree-boot. So the EFI binaries are in /usr/lib/ostree-boot/efi/EFI/fedora/. They just are never copied to the ESP as a part of a deploy.

This is because the ESP is a vfat partition and so it can't be part of the ostree transactional deployment. But still users should be have the option to update the files in the ESP outside of the normal ostree deployments in case they want to update their bootloaders.

For example there could be a ostree admin instutil esp-upgrade command that copies the EFI binaries from /usr/lib/ostree-boot/efi/EFI/fedora/ to /boot/efi/EFI/fedora/.
@martinezjavier martinezjavier mentioned this issue Jun 26, 2018
Closed
@cmurf
Copy link

cmurf commented Jun 26, 2018

How about a generic ESP updating service that does not depend on a. always keeping the ESP mounted; b. mdraid (because it's explicitly not recommended); c. assuming only one ESP; d. assuming traditional or ostree installation; e. users knowing a multitude of esoteric information, and documenters having to document and translate all of it.

A more reliable design would be to update the binaries in a location common to traditional and ostree based installs own. Maybe this is in /usr, or maybe /boot/efi (no longer the ESP). And a trigger for a sync service that syncs to all ESPs. The service would mount any ESP (by partition type GUID) e.g. /run/mount/ and then use cp -r or rsync -r from our static location to the path we own, EFI$DISTRO, then unmount them. Maybe it'd be done after switchroot, before all other services are running.

@martinezjavier
Copy link
Contributor Author

@cmurf sounds good to me. I don't really have a strong opinion on how this should be implemented.

I just filed this because I think we should provide a way for users to update the ESP, specially since I guess may users are not aware that the bootloaders are not updated as a part of a ostree admin upgrade operation (for example I only realized this from the discussion in #717).

@cmurf
Copy link

cmurf commented Jun 26, 2018

Well if it's simple to do, it's certainly better than not being able to update the binaries. Someone taking on a more wholistic design and implementation may not happen anytime soon.

@martinezjavier
Copy link
Contributor Author

martinezjavier commented Jun 27, 2018 •
edited

@cmurf yes, having it as a ostree admin instutil subcommand should be pretty easy to implement I believe and I can take a look to that once we land all the BLS changes.

The sync service that you propose sounds like a much generic solution and I'll be happy if someone implements it, but I personally don't think that will have the time for that.

@AdrianVovk
Copy link

AdrianVovk commented Jan 4, 2019 •
edited

Let's revive this discussion... @cgwalters @martinezjavier

This is my previous comment on this:

I've been able to mess about with my distro and get some insight about the relationship between /boot, the ESP, and systemd

systemd, by default, really wants the ESP to be mounted to either /boot directly or /efi. If /efi exists, systemd will mount the ESP there. Otherwise, it will try to mount it to /boot. This is the behaviour of the systemd-gpt-auto-generator, which my distro will use to find and mount the rootfs. bootctl checks those two mounts points for an ESP also. It's safe to assume that any systemd-related tool will probably assume the EFI exists in either of these two directories (short of looking at /etc/fstab, which I don't intend to use on my distro)

I've been able to temporarily get OSTree working like so: I have a boot.mount unit which bind-mounts /sysroot/boot to /boot. OSTree can then operate on this directory normally. I created an /efi mount point (and systemd automagically mounts the ESP there) that I can copy files to as necessary. This, of course, is not an atomic layout. If something goes wrong while my distro's scripts are copying from /boot to /efi, I'm screwed. The system would no longer boot.

OSTree depends on symlinks in /boot, so it cannot operate directly on the ESP. If this dependency can be broken, however, it could be possible to work with the ESP in /boot. I think this can be done with the renameat2 syscall to atomically swap two folders, instead of using a symlink and rename. More info and initial patch. The new flow could go something like this: empty out /boot/loader.inactive/, generate the ostree config files, copy in non-ostree config files from /boot/loader, renameat2 to swap the two folders. This would give an atomic swap without relying on symlinks. If I'm not missing something, this would allow OSTree to work directly on the ESP. Steps like setting permissions on the files can be skipped or can error out silently, since vfat doesn't support permissions anyway AFAIK

If, for compatability, you need to keep the old behavior, you can translate all of this new behavior to the /efi directory. This new atomic swap would happen in /efi, and the old behavior would happen in /boot.

I've already borked the system by accidentally typing in ostree admin upgrade instead of running my script that also copies over from /boot to /efi. I'm sure that someone at some point will run that command and destroy their system's ability to boot. Making the user run a separate command to copy things to the EFI will lead to this very problem. OSTree should, on EFI and BLS bootloaders (like sd-boot), work with the EFI directly. I don't see a downside

What are your thoughts?

@AdrianVovk AdrianVovk mentioned this issue Jan 4, 2019
Open
@wjt
Copy link
Contributor

wjt commented Jan 4, 2019

I think this can be done with the renameat2 syscall to atomically swap two folders, instead of using a symlink and rename.

I can't find my test program right now, but I'm pretty sure I tested and renameat2(..., RENAME_EXCHANGE) is not supported on vfat.

@AdrianVovk
Copy link

@wjt I tested it and you are correct, trying to do the exchange on the ESP returns EINVAL

OSTree can rely on creating/deleting files in the correct order instead

For example:

  1. /efi/loader contains a.conf and b.conf
  2. Create c+3.conf.staged
  3. Write the contents into the file
  4. Rename the file to c+3.conf
  5. Delete a.conf

martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
7fb879b 
The files in the EFI System Partition (ESP) are never updated by OSTree so
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem, which doesn't support
symbolic links so the update can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones needed for firmware updates, it
should be a way to update the ESP with the latest version of the binaries.

These components are stored in the /usr/lib/ostree-boot/efi directory of a
deployment due rpm-ostree replacing the /boot path to /usr/lib/ostree-boot.

Add a ostree-admin-esp-upgrade subcommand, that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the booted deployment.

Closes: ostreedev#1649
martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
7236536 
The files in the EFI System Partition (ESP) are never updated by OSTree so
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem, which doesn't support
symbolic links so the update can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones needed for firmware updates, it
should be a way to update the ESP with the latest version of the binaries.

These components are stored in the /usr/lib/ostree-boot/efi directory of a
deployment due rpm-ostree replacing the /boot path to /usr/lib/ostree-boot.

Add a ostree-admin-esp-upgrade subcommand, that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the booted deployment.

Closes: ostreedev#1649
martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
33832de 
The files in the EFI System Partition (ESP) are never updated by OSTree so
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem, which doesn't support
symbolic links so the update can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones needed for firmware updates, it
should be a way to update the ESP with the latest version of the binaries.

These components are stored in the /usr/lib/ostree-boot/efi directory of a
deployment due rpm-ostree replacing the /boot path to /usr/lib/ostree-boot.

Add a ostree-admin-esp-upgrade subcommand, that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the booted deployment.

Closes: ostreedev#1649
martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
5a4da4c 
The files in the EFI System Partition (ESP) are never updated by OSTree so
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem which doesn't support
symbolic links so the upgrade can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones used for firmware update, there
should be a way to update the ESP with the latest version of these files.

These binaries are stored in the /usr/lib/ostree-boot/efi directory of a
deployment, because rpm-ostree places there everything that is in /boot.

Add a ostree-admin-esp-upgrade subcommand, that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the booted deployment.

This sub-command only works on a system that was booted with EFI and uses
/boot/efi as the mountpoint for the ESP.

Closes: ostreedev#1649
martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
94d9e10 
The files in the EFI System Partition (ESP) are never updated by OSTree so
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem which doesn't support
symbolic links so the upgrade can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones used for firmware update, there
should be a way to update the ESP with the latest version of these files.

These binaries are stored in the /usr/lib/ostree-boot/efi directory of a
deployment, because rpm-ostree places there everything that is in /boot.

Add a ostree-admin-esp-upgrade subcommand, that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the booted deployment.

This sub-command only works on a system that was booted with EFI and uses
/boot/efi as the mountpoint for the ESP.

Closes: ostreedev#1649
martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
c60b8f3 
The files in the EFI System Partition (ESP) are never updated by OSTree so
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem which doesn't support
symbolic links so the upgrade can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones used for firmware update, there
should be a way to update the ESP with the latest version of these files.

These binaries are stored in the /usr/lib/ostree-boot/efi directory of a
deployment, because rpm-ostree places there everything that is in /boot.

Add a ostree-admin-esp-upgrade subcommand, that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the booted deployment.

This sub-command only works on a system that was booted with EFI and uses
/boot/efi as the mountpoint for the ESP.

Closes: ostreedev#1649
martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
46930b2 
The files in the EFI System Partition (ESP) are never updated by OSTree,
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem which doesn't support
symbolic links so the upgrade can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones used for firmware update, there
should be a way to upgrade the ESP with the latest version of these files.

These binaries are stored in the /usr/lib/ostree-boot/efi directory of a
deployment, because rpm-ostree places there everything that is in /boot.

Add a ostree-admin-esp-upgrade subcommand that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the booted deployment.

This sub-command only works on a system that was booted with EFI and has
the ESP mounted in /boot/efi.

Closes: ostreedev#1649
@martinezjavier martinezjavier mentioned this issue Jun 14, 2019
Closed
martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
092cd8f 
The files in the EFI System Partition (ESP) are never updated by OSTree,
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem which doesn't support
symbolic links so the upgrade can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones used for firmware update, there
should be a way to upgrade the ESP with the latest version of these files.

These binaries are stored in the /usr/lib/ostree-boot/efi directory of a
deployment, because rpm-ostree places there everything that is in /boot.

Add a ostree-admin-esp-upgrade subcommand that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the booted deployment.

This sub-command only works on a system that was booted with EFI and has
the ESP mounted in /boot/efi.

Closes: ostreedev#1649
martinezjavier added a commit to martinezjavier/ostree that referenced this issue Jun 14, 2019
@martinezjavier
admin: Add an esp-upgrade subcommand
b9f2b65 
The files in the EFI System Partition (ESP) are never updated by OSTree,
users will always have the same ESP that was created during installation.

This happens because the ESP uses a vfat filesystem which doesn't support
symbolic links so the upgrade can't be atomic and part of the transaction.

But since the ESP contains the bootloader components (shim, grub, etc) and
other important EFI binaries like the ones used for firmware update, there
should be a way to upgrade the ESP with the latest version of these files.

These binaries are stored in the /usr/lib/ostree-boot/efi directory of a
deployment, because rpm-ostree places there everything that is in /boot.

Add a ostree-admin-esp-upgrade subcommand that just copies the content of
/usr/lib/ostree-boot/efi to /boot/efi, so users can update the ESP and get
the latest version of the files that are present in the current deployment.

This sub-command only works on a system that was booted with EFI and has
the ESP mounted in /boot/efi.

Closes: ostreedev#1649
@bam80
Copy link

bam80 commented Feb 21, 2020

Any update? (didn't read all the conversation)

@martinezjavier
Copy link
Contributor Author

@cgwalters I think this issue could be closed now that https://github.com/coreos/bootupd exists?

@cgwalters
Copy link
Member

Yep, thanks!

To clarify, ostree doesn't depend on bootupd, and while bootupd currently only works with (rpm-)ostree based systems the idea is to support other systems too.

So using bootupd is one solution; others may want to just take a similar approach with other software.

martinezjavier added a commit to martinezjavier/linux that referenced this issue May 19, 2022
@martinezjavier
selftests/filesystems: add a vfat RENAME_EXCHANGE test
b2e38a4 
Add a test for the renameat2 RENAME_EXCHANGE support in vfat, but split it
in a tool that just does the rename exchange and a script that is run by
the kselftests framework on `make TARGETS="filesystems/fat" kselftest`.

That way the script can be easily extended to test other file operations.

The script creates a 1 MiB disk image, that is then formated with a vfat
filesystem and mounted using a loop device. That way all file operations
are done on an ephemeral filesystem.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Series-cc: Alexander Larsson <alexl@redhat.com>
Series-cc: Peter Jones <pjones@redhat.com>
Series-cc: Lennart Poettering <lennart@poettering.net>
Series-cc: Colin Walters <walters@verbum.org>
Series-cc: Alberto Ruiz <aruiz@redhat.com>
Series-cc: Christian Kellner <ckellner@redhat.com>
Series-cc: Chung-Chiang Cheng <cccheng@synology.com>
Series-version: 2
Cover-letter:
fat: add support for the renameat2 RENAME_EXCHANGE flag
Hello,

This series add support for the renameat2 system call RENAME_EXCHANGE flag
(which allows to atomically replace two paths) to the vfat filesystem code.

There are many use cases for this, but we are particularly interested in
making possible for vfat filesystems to be part of OSTree [0] deployments.

Currently OSTree relies on symbolic links to make the deployment updates
an atomic transactional operation. But RENAME_EXCHANGE could be used [1]
to achieve a similar level of robustness when using a vfat filesystem.

Patch #1 is just a preparatory patch to introduce the RENAME_EXCHANGE
support in patch #2 and finally patch #3 adds some kselftests to test it.

This is my first contribution to the fs/* subsystem, so I'm marking this
set as RFC, in case I got anything wrong with the patches. But they work
correctly on my local testing.

[0]: https://github.com/ostreedev/ostree
[1]: ostreedev/ostree#1649
END
martinezjavier added a commit to martinezjavier/linux that referenced this issue May 19, 2022
@martinezjavier
selftests/filesystems: add a vfat RENAME_EXCHANGE test
6be2d6c 
Add a test for the renameat2 RENAME_EXCHANGE support in vfat, but split it
in a tool that just does the rename exchange and a script that is run by
the kselftests framework on `make TARGETS="filesystems/fat" kselftest`.

That way the script can be easily extended to test other file operations.

The script creates a 1 MiB disk image, that is then formated with a vfat
filesystem and mounted using a loop device. That way all file operations
are done on an ephemeral filesystem.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Series-changes: 2
- Call fsync to flush the page cache before checking the file contents
  (Alex Larsson).

Series-cc: Alexander Larsson <alexl@redhat.com>
Series-cc: Peter Jones <pjones@redhat.com>
Series-cc: Lennart Poettering <lennart@poettering.net>
Series-cc: Colin Walters <walters@verbum.org>
Series-cc: Alberto Ruiz <aruiz@redhat.com>
Series-cc: Christian Kellner <ckellner@redhat.com>
Series-cc: Chung-Chiang Cheng <cccheng@synology.com>
Series-version: 2
Cover-letter:
fat: add support for the renameat2 RENAME_EXCHANGE flag
Hello,

This series add support for the renameat2 system call RENAME_EXCHANGE flag
(which allows to atomically replace two paths) to the vfat filesystem code.

There are many use cases for this, but we are particularly interested in
making possible for vfat filesystems to be part of OSTree [0] deployments.

Currently OSTree relies on symbolic links to make the deployment updates
an atomic transactional operation. But RENAME_EXCHANGE could be used [1]
to achieve a similar level of robustness when using a vfat filesystem.

Patch #1 is just a preparatory patch to introduce the RENAME_EXCHANGE
support in patch #2 and finally patch #3 adds some kselftests to test it.

This is my first contribution to the fs/* subsystem, so I'm marking this
set as RFC, in case I got anything wrong with the patches. But they work
correctly on my local testing.

[0]: https://github.com/ostreedev/ostree
[1]: ostreedev/ostree#1649
END
martinezjavier added a commit to martinezjavier/linux that referenced this issue May 19, 2022
@martinezjavier
selftests/filesystems: add a vfat RENAME_EXCHANGE test
fdbcb6c 
Add a test for the renameat2 RENAME_EXCHANGE support in vfat, but split it
in a tool that just does the rename exchange and a script that is run by
the kselftests framework on `make TARGETS="filesystems/fat" kselftest`.

That way the script can be easily extended to test other file operations.

The script creates a 1 MiB disk image, that is then formated with a vfat
filesystem and mounted using a loop device. That way all file operations
are done on an ephemeral filesystem.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Series-changes: 2
- Call sync to flush the page cache before checking the file contents
  (Alex Larsson).

Series-cc: Alexander Larsson <alexl@redhat.com>
Series-cc: Peter Jones <pjones@redhat.com>
Series-cc: Lennart Poettering <lennart@poettering.net>
Series-cc: Colin Walters <walters@verbum.org>
Series-cc: Alberto Ruiz <aruiz@redhat.com>
Series-cc: Christian Kellner <ckellner@redhat.com>
Series-cc: Chung-Chiang Cheng <cccheng@synology.com>
Series-version: 2
Cover-letter:
fat: add support for the renameat2 RENAME_EXCHANGE flag
Hello,

This series add support for the renameat2 system call RENAME_EXCHANGE flag
(which allows to atomically replace two paths) to the vfat filesystem code.

There are many use cases for this, but we are particularly interested in
making possible for vfat filesystems to be part of OSTree [0] deployments.

Currently OSTree relies on symbolic links to make the deployment updates
an atomic transactional operation. But RENAME_EXCHANGE could be used [1]
to achieve a similar level of robustness when using a vfat filesystem.

Patch #1 is just a preparatory patch to introduce the RENAME_EXCHANGE
support in patch #2 and finally patch #3 adds some kselftests to test it.

This is my first contribution to the fs/* subsystem, so I'm marking this
set as RFC, in case I got anything wrong with the patches. But they work
correctly on my local testing.

[0]: https://github.com/ostreedev/ostree
[1]: ostreedev/ostree#1649
END
martinezjavier added a commit to martinezjavier/linux that referenced this issue Jun 1, 2022
@martinezjavier
selftests/filesystems: add a vfat RENAME_EXCHANGE test
13c528b 
Add a test for the renameat2 RENAME_EXCHANGE support in vfat, but split it
in a tool that just does the rename exchange and a script that is run by
the kselftests framework on `make TARGETS="filesystems/fat" kselftest`.

That way the script can be easily extended to test other file operations.

The script creates a 1 MiB disk image, that is then formated with a vfat
filesystem and mounted using a loop device. That way all file operations
are done on an ephemeral filesystem.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Acked-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Series-changes: 4
- Add Muhammad Usama Anjum Acked-by tag.

Series-changes: 3
- Add a .gitignore for the rename_exchange binary (Muhammad Usama Anjum).
- Include $(KHDR_INCLUDES) instead of hardcoding a relative path in Makefile
  (Muhammad Usama Anjum).

Series-changes: 2
- Call sync to flush the page cache before checking the file contents
  (Alex Larsson).

Series-cc: Alexander Larsson <alexl@redhat.com>
Series-cc: Peter Jones <pjones@redhat.com>
Series-cc: Lennart Poettering <lennart@poettering.net>
Series-cc: Colin Walters <walters@verbum.org>
Series-cc: Alberto Ruiz <aruiz@redhat.com>
Series-cc: Christian Kellner <ckellner@redhat.com>
Series-cc: Chung-Chiang Cheng <cccheng@synology.com>
Series-cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
Series-version: 4
Cover-changes: 2
- Drop RFC prefix since the patches already got some review.

Cover-letter:
fat: add support for the renameat2 RENAME_EXCHANGE flag
Hello,

This series add support for the renameat2 system call RENAME_EXCHANGE flag
(which allows to atomically replace two paths) to the vfat filesystem code.

There are many use cases for this, but we are particularly interested in
making possible for vfat filesystems to be part of OSTree [0] deployments.

Currently OSTree relies on symbolic links to make the deployment updates
an atomic transactional operation. But RENAME_EXCHANGE could be used [1]
to achieve a similar level of robustness when using a vfat filesystem.

Patch #1 is just a preparatory patch to introduce the RENAME_EXCHANGE
support, patch #2 moves some code blocks in vfat_rename() to a set of
helper functions, that can be reused by tvfat_rename_exchange() that's
added by patch #3 and finally patch #4 adds some kselftests to test it.

This is a v4 that addresses issues pointed out in the third version posted:

https://lwn.net/Articles/896359/

[0]: https://github.com/ostreedev/ostree
[1]: ostreedev/ostree#1649
END
martinezjavier added a commit to martinezjavier/linux that referenced this issue Jun 1, 2022
@martinezjavier
selftests/filesystems: add a vfat RENAME_EXCHANGE test
e00add0 
Add a test for the renameat2 RENAME_EXCHANGE support in vfat, but split it
in a tool that just does the rename exchange and a script that is run by
the kselftests framework on `make TARGETS="filesystems/fat" kselftest`.

That way the script can be easily extended to test other file operations.

The script creates a 1 MiB disk image, that is then formated with a vfat
filesystem and mounted using a loop device. That way all file operations
are done on an ephemeral filesystem.

Signed-off-by: Javier Martinez Canillas <javierm@redhat.com>
Acked-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
Series-changes: 4
- Add Muhammad Usama Anjum Acked-by tag.

Series-changes: 3
- Add a .gitignore for the rename_exchange binary (Muhammad Usama Anjum).
- Include $(KHDR_INCLUDES) instead of hardcoding a relative path in Makefile
  (Muhammad Usama Anjum).

Series-changes: 2
- Call sync to flush the page cache before checking the file contents
  (Alex Larsson).

Series-cc: Alexander Larsson <alexl@redhat.com>
Series-cc: Peter Jones <pjones@redhat.com>
Series-cc: Lennart Poettering <lennart@poettering.net>
Series-cc: Colin Walters <walters@verbum.org>
Series-cc: Alberto Ruiz <aruiz@redhat.com>
Series-cc: Christian Kellner <ckellner@redhat.com>
Series-cc: Chung-Chiang Cheng <cccheng@synology.com>
Series-cc: Muhammad Usama Anjum <usama.anjum@collabora.com>
Series-version: 4
Cover-changes: 2
- Drop RFC prefix since the patches already got some review.

Cover-letter:
fat: add support for the renameat2 RENAME_EXCHANGE flag
Hello,

This series add support for the renameat2 system call RENAME_EXCHANGE flag
(which allows to atomically replace two paths) to the vfat filesystem code.

There are many use cases for this, but we are particularly interested in
making possible for vfat filesystems to be part of OSTree [0] deployments.

Currently OSTree relies on symbolic links to make the deployment updates
an atomic transactional operation. But RENAME_EXCHANGE could be used [1]
to achieve a similar level of robustness when using a vfat filesystem.

Patch #1 is just a preparatory patch to introduce the RENAME_EXCHANGE
support, patch #2 moves some code blocks in vfat_rename() to a set of
helper functions, that can be reused by tvfat_rename_exchange() that's
added by patch #3 and finally patch #4 adds some kselftests to test it.

This is a v4 that addresses issues pointed out in the third version posted:

https://lwn.net/Articles/896359/

[0]: https://github.com/ostreedev/ostree
[1]: ostreedev/ostree#1649
END
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@wjt @bam80 @cgwalters @martinezjavier @cmurf @AdrianVovk