set podman image storage location #1916

abitrolly · 2018-12-02T12:40:06Z

[//]: # kind feature

or, rather, support request.

My root is mounted at SSD for speed, and so does my home. However, I am not rich, so SSD is small, and now I want more space. First I want to move images to another disk, and if that won't help, move containers as well.

I found podman info, which printed some useful info at the end.

...
store:
  ContainerStore:
    number: 3
  GraphDriverName: vfs
  GraphOptions: []
  GraphRoot: /home/anatoli/.local/share/containers/storage
  GraphStatus: {}
  ImageStore:
    number: 1
  RunRoot: /run/user/1000/run

The path /home/anatoli/.local/share/containers/storage is 6.3G, and there is only one image 745Mb in size there.

✗ podman images -a
REPOSITORY                 TAG     IMAGE ID       CREATED        SIZE
docker.io/library/golang   1.9.4   a6c306bd0b2f   8 months ago   754MB
✗ du -hs  ~/.local/share/containers/storage
6.3G	/home/anatoli/.local/share/containers/storage

I don't see how can I move images to the other place, because vfs-images contains only meta information and vfs hosts only hash dirs.

6.3G	/home/anatoli/.local/share/containers/storage/vfs
132K	/home/anatoli/.local/share/containers/storage/vfs-containers
28K	/home/anatoli/.local/share/containers/storage/vfs-images
1.8M	/home/anatoli/.local/share/containers/storage/vfs-layers

Is there a setting that I can change to store images in a separate dir?

Output of podman version:

Version:       0.10.1.3
Go Version:    go1.10.4
OS/Arch:       linux/amd64

The text was updated successfully, but these errors were encountered:

mheon · 2018-12-02T14:24:28Z

You can make a copy of /etc/containers/storage.conf in ~/.config/containers (this directory may need to be created first) and configure where rootless Podman stores things in that file. Be aware that, if you change these directories, it's recommended that you delete the old ones (you may end up with an unusable Podman otherwise). It sounds like you'll want to change the graph root directory, where permanent files and container images are stored. Start by changing graph driver to the rootless default (vfs) and run root to a directory in /run/user/$YOUR_UID/ (this directory does not need to exist, we'll create it if it doesn't). Then set the graph root to a location where your user has write access, and all container and image permanent files will be stored at that location.

…

On Sun, Dec 2, 2018, 07:40 Anatoli Babenia ***@***.*** wrote: [//]: # *kind feature* or, rather, *support request*. My root is mounted at SSD for speed, and so does my home. However, I am not rich, so SSD is small, and now I want more space. First I want to move images to another disk, and if that won't help, move containers as well. I found podman info, which printed some useful info at the end. ... store: ContainerStore: number: 3 GraphDriverName: vfs GraphOptions: [] GraphRoot: /home/anatoli/.local/share/containers/storage GraphStatus: {} ImageStore: number: 1 RunRoot: /run/user/1000/run The path /home/anatoli/.local/share/containers/storage is 6.3G, and there is only one image 745Mb in size there. ✗ podman images -a REPOSITORY TAG IMAGE ID CREATED SIZE docker.io/library/golang 1.9.4 a6c306bd0b2f 8 months ago 754MB ✗ du -hs ~/.local/share/containers/storage 6.3G /home/anatoli/.local/share/containers/storage I don't see how can I move images to the other place, because vfs-images contains only meta information and vfs hosts only hash dirs. 6.3G /home/anatoli/.local/share/containers/storage/vfs 132K /home/anatoli/.local/share/containers/storage/vfs-containers 28K /home/anatoli/.local/share/containers/storage/vfs-images 1.8M /home/anatoli/.local/share/containers/storage/vfs-layers *Is there a setting that I can change to store images in a separate dir?* *Output of podman version:* Version: 0.10.1.3 Go Version: go1.10.4 OS/Arch: linux/amd64 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1916>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHYHCKfbXbszilQURUZo7R-wUPFUP6a9ks5u08opgaJpZM4Y9Yi-> .

abitrolly · 2018-12-02T16:15:28Z

Sounds like podman doesn't support changing image storage location with a simple user setting. Does that qualify for a future request? Having an option in ~/.config/containers/storage.conf- ImageStore.location = /data/podman-image.

mheon · 2018-12-02T16:54:42Z

We are moving over to creating the file by default, with the correct settings (so you'll only need to swap the one setting). That should be out in our next release (hopefully tomorrow). If you're looking for a way to just swap image storage, though, I don't think that's possible as of now - we storage images and containers in the same place as part of the architecture of containers/storage. We might be able to split that, but for now both must be in the same place.

abitrolly · 2018-12-03T05:01:54Z

Let me know what should I do when it is released. I guess removing podman, cleaning up config files and then reinstalling it again to create the config in my home dir? Then I will be able to edit it to set storate location.

rhatdan · 2018-12-03T19:32:57Z

SImpler fix would be to setup a bind mount
cp -R ~/.local/share/containers /data
rm -rf ~/.local/share/containers
mkdir ~/.local/share/containers
mount -o bind /data/containers ~/.local/share/containers

clebergnu · 2018-12-10T17:36:47Z

+1 for the file based configuration of the storage location.

rhatdan · 2018-12-12T15:58:38Z

With podman 0.12.1.1 It should generate ~/.lcontig/containers/storage.conf
This file can be edited to point at /data/containers.

mheon · 2018-12-12T17:17:47Z

~/.config/containers/storage.conf to be clear

abitrolly · 2018-12-13T09:20:58Z

➜  ~ rm -rf ~/.local/share/containers
➜  ~ rm -rf ~/.config/containers
➜  ~ sudo dnf --enablerepo=updates-testing install podman
...
podman                            x86_64       0.12.1.1-1.git66d3499.fc28        updates-testing       7.0 M

➜  ~ cat .config/containers/storage.conf 
[storage]
  driver = "vfs"
  runroot = "/run/user/1000"
  graphroot = "/home/anatoli/.local/share/containers/storage"
  [storage.options]
    mount_program = ""

mheon · 2018-12-13T14:49:01Z

It's not really clear from the generated file, but graphroot there is container and image storage

clebergnu · 2018-12-13T14:56:37Z

@mheon you're right that it's not clear... not sure if the generated ~/.config/containers/storage.conf can include comment/documentation. That alone would help a lot.

mheon · 2018-12-13T15:30:07Z

It could, but it would be a fair bit of effort on our end (straight TOML generation to templated output). Probably worth it, though.

abitrolly · 2018-12-13T15:56:54Z

Looks like Tom doesn't count comments as a feature - toml-lang/toml#284

rhatdan · 2019-03-08T15:07:08Z

@giuseppe I think your code is doing this, could you take a look.

abitrolly · 2019-03-11T06:24:39Z

How often does podman touch storage folder while sitting in a background and not starting other containers? Is there a way to measure that?

giuseppe · 2019-03-11T07:49:52Z

You'd need to change both storage.conf and libpod.conf to update the path to static dir.

Migration is currently blocked by us validating the config file settings with the db.

It should work if you start with a fresh storage

abitrolly · 2019-03-12T09:40:57Z

If migration is impossible, then the only way is remove podman, then edit files

vim $HOME/.config/containers/libpod.conf
# volume_path = "$HOME/.local/share/containers/storage/volumes"
# static_dir = "$HOME/.local/share/containers/storage/libpod"
vim $HOME/.config/containers/storage.conf
# [storage]
#   graphroot = "$HOME/.local/share/containers/storage"

, kill $HOME/.local/share/containers, and install podman again.

giuseppe · 2019-03-12T11:03:01Z

why do you need to remove podman and install it again?

rhatdan · 2019-04-13T09:42:25Z

I believe this is no longer an issue.

sunshine69 · 2021-02-27T01:35:16Z

Not sure if anything wrong with my setup as of now I seems to hit this bug. The only way to make it work is using mount bind.

as long as I have .config/containers/storage.conf with all correct settings podman always says:
Error: mkdir /var/run/containers: permission denied

podman is in fedora 33,

podman --version
podman version 2.2.1

the only thing it works by remove the custom config and then copy the storage over the other directory and empty the original one, mount bind two dir together.

Should we re-open this issue?

giuseppe · 2021-03-01T08:54:00Z

how does the .config/containers/storage.conf file look like?

locaIhost · 2021-03-04T04:32:49Z

The same situation, but tune in /etc/containers/storage.conf
and no one bothers to do symlink.

sunshine69 · 2021-03-07T08:48:49Z

how does the .config/containers/storage.conf file look like?

I have to remove it. But when I have issues described the content is as basic

[storage]
driver = "overlay"
runroot = "/mnt/portdata/containers/stevek/storage-runroot"
graphroot = "/mnt/portdata/containers/stevek/storage"
rootless_storage_path = "/mnt/portdata/containers/stevek/storage"

locaIhost · 2021-03-07T21:07:21Z

how does the .config/containers/storage.conf file look like?

I have to remove it. But when I have issues described the content is as basic
[storage]
driver = "overlay"
runroot = "/mnt/portdata/containers/stevek/storage-runroot"
graphroot = "/mnt/portdata/containers/stevek/storage"
rootless_storage_path = "/mnt/portdata/containers/stevek/storage"

Well, you have many solutions

Request a list of package file from the repository.
rpm -vlq podman |grep ".conf"
Make a choice of a package dependency, go over them ;)
yum deplist podman |grep containers-common
Look it.
rpm -qdf /usr/bin/podman
4.Significant!
https://github.com/containers/podman/blob/master/vendor/github.com/containers/storage/storage.conf
My default? ok
https://gist.github.com/locaIhost/7404c69e22e18c0a4a58b9cd41136934
......... and
man containers-storage.conf
.........
If that write - happy to answer:: Give a Certification RHCSA Discount on Women's Day!

Bill0412 · 2022-08-18T13:02:40Z

Thanks, this saves my life!

I don't have root permission on the machine. Solved the problem by using the following steps:

mkdir -p ~/.config/containers/
cp /etc/containers/storage.conf ~/.config/containers
Change runroot, graphroot to directories inside /run/user/$UID, you don't have to create these directories manually, podman will create them for you.
export TMPDIR="/run/user/$UID/podman-tmp", maybe you can add this to your ~/.bashrc.
mkdir -p $TMPDIR.

referenced this: #5412

YevheniiPokhvalii · 2022-09-29T00:24:07Z

My /etc/containers/storage.conf in Ubuntu WSL2 is empty.

rhatdan · 2022-09-29T10:35:13Z

Check /usr/share/containers/storage.conf

YevheniiPokhvalii · 2022-09-29T20:06:56Z

Check /usr/share/containers/storage.conf

No such file

rhatdan · 2022-09-29T20:56:39Z

Here is mine on Fedora.
storage.conf

RichardFevrier · 2022-11-22T19:29:21Z

@giuseppe can you summarize the full process with docker support please?
Let's take for example the path /var/mnt/data/containers/storage as a target.
What folders/files to create/update? (storage.conf, libpod.conf, etc...)
What should we update in those files? (driver, runroot, graphroot, rootless_storage_path, etc...)
Thank you a lot 🙏

rhatdan · 2022-11-22T20:01:34Z

It is only storage.conf, you should never do anything with libpod.conf Remove it if you have on on your system.

graphroot is used to change the system storage directories for images and containers.
rootless_storage_path will override the default location for rootless users ~/.local/share/containers/

No need to update driver or runroot.

RichardFevrier · 2022-11-23T11:53:24Z

Thanks @rhatdan
Can you update your answer (to keep it simple for anyone who's looking for a concise solution) for the case of someone already having containers. (from toolbox create, docker-compose up, etc...)
Can we simply move them? (the whole graphroot directory)
Should we recreate them?
Should we stop the systemd service(s) first? (podman-auto-update.service, podman-restart.service, podman.service, etc...)
How to check that our configuration is well applied? (sudo podman info)

YevheniiPokhvalii · 2022-11-23T19:47:10Z

If you want to copy your images from docker to podman, I was only advised this solution: #16020

docker save IMAGE | podman load

I'm not sure for containers, but I guess, you could use something like

docker export <CONTAINER ID> | podman import - some-name:latest

rhatdan · 2022-11-29T21:21:10Z

I would prefer this hack.

# docker images --format docker-daemon:{{.Repository}}:{{.Tag}} | grep -v none  | xargs podman pull

See

# podman images
REPOSITORY  TAG         IMAGE ID    CREATED     SIZE
# docker images
REPOSITORY                              TAG       IMAGE ID       CREATED         SIZE
<none>                                  <none>    cf88db92256c   4 weeks ago     5.59MB
<none>                                  <none>    0a793d27c5b7   4 weeks ago     5.59MB
<none>                                  <none>    aa3749da95de   4 weeks ago     5.59MB
<none>                                  <none>    63d102f81adb   2 months ago    5.59MB
nginx                                   latest    670dcc86b69d   4 months ago    142MB
<none>                                  <none>    a0d5654bf697   5 months ago    5.59MB
<none>                                  <none>    fa94027fbe29   5 months ago    5.59MB
<none>                                  <none>    810085503773   6 months ago    5.59MB
<none>                                  <none>    2a9b6aeb963c   8 months ago    153MB
<none>                                  <none>    0022b42cdf0d   8 months ago    1.38GB
i386/alpine                             edge      c9930b809cba   8 months ago    5.46MB
registry.access.redhat.com/ubi8-micro   latest    a4b6df2ac2df   10 months ago   36.5MB
fedora                                  latest    b78af7a83692   12 months ago   153MB
alpine                                  latest    c059bfaa849c   12 months ago   5.59MB
# docker images --format docker-daemon:{{.Repository}}:{{.Tag}} | grep -v none  | xargs podman pull
Getting image source signatures
Copying blob abc66ad258e9 done  
Copying blob f931b78377da done  
Copying blob d7783033d823 done  
Copying blob 43b3c4e3001c done  
Copying blob 4553dc754574 done  
Copying blob 243243243ee2 done  
Copying config 670dcc86b6 done  
Writing manifest to image destination
Storing signatures
670dcc86b69df89a9d5a9e1a7ae5b8f67619c1c74e19de8a35f57d6c06505fd4
Getting image source signatures
Copying blob 8329b0e13e56 done  
Copying config c9930b809c done  
Writing manifest to image destination
Storing signatures
WARNING: image platform ({386 linux  [] }) does not match the expected platform ({amd64 linux  [] })
c9930b809cbaef2f8784f3b8ed929caa8e95a944815bf91721cb050fe9779613
Getting image source signatures
Copying blob fe7a061dd3ad done  
Copying blob 5f70bf18a086 done  
Copying config a4b6df2ac2 done  
Writing manifest to image destination
Storing signatures
a4b6df2ac2dfb315a4edf1c62f4e44c466ca3ad81d8acc4beba31b07daa9fe77
Getting image source signatures
Copying blob 389adea752cd done  
Copying config b78af7a836 done  
Writing manifest to image destination
Storing signatures
b78af7a836928efd44317e82c8f2f9c86bb83ae915deef1b58dc6465dfa5436e
Getting image source signatures
Copying blob 8d3ac3489996 done  
Copying config c059bfaa84 done  
Writing manifest to image destination
Storing signatures
c059bfaa849c4d8e4aecaeb3a10c2d9b3d85f5165c66ad3a4d937758128c4d18
# podman images
REPOSITORY                             TAG         IMAGE ID      CREATED        SIZE
docker.io/library/nginx                latest      670dcc86b69d  4 months ago   146 MB
docker.io/i386/alpine                  edge        c9930b809cba  8 months ago   5.75 MB
registry.access.redhat.com/ubi8-micro  latest      a4b6df2ac2df  10 months ago  38.7 MB
docker.io/library/fedora               latest      b78af7a83692  12 months ago  159 MB
docker.io/library/alpine               latest      c059bfaa849c  12 months ago  5.87 MB

auwsom · 2023-04-24T05:03:55Z

I had to create the config file: /home/user/.config/containers/storage.conf

[storage]
driver = "overlay"
#runroot = "//containers/storage-runroot"
graphroot = "//containers/storage"
#rootless_storage_path = "//containers/storage"

The two commented lines were not needed to run rootless so far.
Move the container/* files to target and then sudo systemctl restart podman and podman info to check the config was changed.

rhatdan assigned giuseppe Mar 8, 2019

rhatdan closed this as completed Apr 13, 2019

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Aug 26, 2023

github-actions bot locked as resolved and limited conversation to collaborators Aug 26, 2023

set podman image storage location #1916

set podman image storage location #1916

Comments

abitrolly commented Dec 2, 2018

mheon commented Dec 2, 2018 via email

abitrolly commented Dec 2, 2018

mheon commented Dec 2, 2018

abitrolly commented Dec 3, 2018

rhatdan commented Dec 3, 2018

clebergnu commented Dec 10, 2018

rhatdan commented Dec 12, 2018

mheon commented Dec 12, 2018

abitrolly commented Dec 13, 2018

mheon commented Dec 13, 2018

clebergnu commented Dec 13, 2018

mheon commented Dec 13, 2018

abitrolly commented Dec 13, 2018

rhatdan commented Mar 8, 2019

abitrolly commented Mar 11, 2019

giuseppe commented Mar 11, 2019

abitrolly commented Mar 12, 2019

giuseppe commented Mar 12, 2019

rhatdan commented Apr 13, 2019

sunshine69 commented Feb 27, 2021

giuseppe commented Mar 1, 2021

locaIhost commented Mar 4, 2021

sunshine69 commented Mar 7, 2021

locaIhost commented Mar 7, 2021 • edited

Bill0412 commented Aug 18, 2022 • edited

YevheniiPokhvalii commented Sep 29, 2022

rhatdan commented Sep 29, 2022

YevheniiPokhvalii commented Sep 29, 2022

rhatdan commented Sep 29, 2022

RichardFevrier commented Nov 22, 2022 • edited

rhatdan commented Nov 22, 2022

RichardFevrier commented Nov 23, 2022 • edited

YevheniiPokhvalii commented Nov 23, 2022 • edited

rhatdan commented Nov 29, 2022 • edited

auwsom commented Apr 24, 2023

locaIhost commented Mar 7, 2021 •

edited

Bill0412 commented Aug 18, 2022 •

edited

RichardFevrier commented Nov 22, 2022 •

edited

RichardFevrier commented Nov 23, 2022 •

edited

YevheniiPokhvalii commented Nov 23, 2022 •

edited

rhatdan commented Nov 29, 2022 •

edited