Support local volumes with a lifetime longer than that of the podman host

[samuel-f]

Feature request description

Having the ability to retain volumes on a mounted filesystem is desirable as it allows for the replacement of the host system without losing the content of persistent volumes.

This is particularly useful when podman is used on a virtual machine with attached storage, where system upgrades involve replacing the virtual machine. While bind mounts or hostPath volumes could be used, managed volumes are more convenient, especially when podman-kube is already used to manage all other supported resources. Managed volumes offer the advantage of automated cleanup of volumes.

Suggest potential solution

As far as I can tell, the current behavior, although not guaranteed or documented, works as is with volume_path set to a directory on a mounted filesystem. Since podman does not modify the volume directory during volume creation, podman can be made aware of pre-existing volumes, simply by (re)creating volumes with the same name. However, to make it a reliable and supported feature, there may be additional changes required.

Have you considered any alternatives?

No response

Additional context

The behaviour was also mentioned in #16245, but with the opposite intention of using transient volumes.

[rhatdan]

You can set the volume_path in the containers.conf file to point to a shared network volume, that multiple nodes could use. Isn't that what you want.

We also support volume plugins, which you could support remote volumes.

man containers.conf
...
       volume_path="/var/lib/containers/storage/volumes"

       Directory where named volumes will be created in using the default volume  driver.   By  default  this
       will  be configured relative to where containers/storage store containers. This convention is followed
       by the default volume driver, but may not be by other drivers.

[samuel-f]

Setting the volume_path to a shared (over time, not concurrently) location is what I'm currently doing and it is working fine.
As far as I understand, the behaviour, that podman volume create name does not overwrite any content in the existing directory volume_path/name when volume name does not exist in podman's database yet, is not guaranteed, which makes me hesitant to use that setup in production.

[rhatdan]

@mheon WDYT?

[mheon]

I would suggest that this is what podman volume import and podman volume export are for? Alternatively, using the local volume driver to mount an NFS share as a volume would do similar things. I don't know if we want to hard-code volume semantics like handling of pre-existing directories, given alternatives like these exist.

[samuel-f]

Thanks for sharing your thoughts. In my opinion, creating a tarball solely for "adding an entry to a database" seems unnecessary, especially when dealing with large volumes. Additionally, I prefer not to use nfs in this situation.
What if there was an alternative command, kind of a hybrid of create and import, that would make this behaviour explicit?

[mheon]

It doesn't look like we can do bind mounts in podman volume create right now (our syntax for calling mount doesn't allow it), but in theory that would do that you want - use a pre-existing directory at an arbitrary location as the backing store for a volume. You'd have to disable copy-up with --opt nocopy as well to guarantee that the contents were not overwritten on mount to be fully safe, but that's easy enough.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@samuel-f interested in opening a PR?

[rhatdan]

@mheon do you think this is worth pursuing, or should we just add tests to make sure the current behavior survives.

[samuel-f]

@samuel-f interested in opening a PR?

Sorry, I missed that. I don’t want to open a PR since I‘m not sure what exactly would have to be done and if it is actually supposed to become an official feature I just noticed the good first issue badge. Maybe I'll take a look at it.

[samuel-f]

I now have time to work on this issue. Can @mheon or @rhatdan please advise on what, if anything, should be done about this?

[mheon]

The mount code (https://github.com/containers/podman/blob/main/libpod/volume_internal_common.go#L80-L119) for volumes needs to be adjusted to allow bind mounts in the mount command (instead of passing options and a type, pass --bind). There may also be some work required higher in the stack to make bind a valid type for podman volume create -o type= but I am not sure.