Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rootless support for General Parallel File Systems (GPFS) #3478

Closed
jwflory opened this issue Jul 2, 2019 · 6 comments

Comments

@jwflory
Copy link
Contributor

commented Jul 2, 2019

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

Podman is unable to pull images as a rootless user. When I try to pull images, while storing signatures, Podman attempts to open /root/.bash_logout, which the rootless user expectedly does not have access to.

In my case, I am using the VFS storage driver as well, although this is intended as short-term.

Steps to reproduce the issue:

  1. podman pull docker.io/fedora:latest

Describe the results you received:

[jwflory@compute0011 ~]$ podman pull docker.io/fedora:latest
Trying to pull docker.io/fedora:latest...Getting image source signatures
Copying blob 8f6ac7ed4a91 done
Copying config 289289d1a1 done
Writing manifest to image destination
Storing signatures
ERRO[0006] Error while applying layer: ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied 
ERRO[0007] Error pulling image ref //fedora:latest: Error committing the finished image: error adding layer with blob "sha256:8f6ac7ed4a91c9630083524efcef2f59f27404320bfee44397f544c252ad4bd4": ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied 
Failed
Error: error pulling image "docker.io/fedora:latest": unable to pull docker.io/fedora:latest: unable to pull image: Error committing the finished image: error adding layer with blob "sha256:8f6ac7ed4a91c9630083524efcef2f59f27404320bfee44397f544c252ad4bd4": ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied

Describe the results you expected:

[jwflory@compute0011 ~]$ podman pull docker.io/fedora:latest
Trying to pull docker.io/fedora:latest...Getting image source signatures
Copying blob 8f6ac7ed4a91 done
Copying config 289289d1a1 done
Writing manifest to image destination
Storing signatures
# sha256sum hash

Additional information you deem important (e.g. issue happens only occasionally):

This is a self-compiled and built version of Podman 1.4.2 for RHEL 7.5. It does use the newer shadow-utils-newxidmap and slirp4netfs packages however. This could be a local issue, but it would be helpful to know in advance while preparing for a future RHEL 7.7.

Output of podman version:

Version:            1.4.2
RemoteAPI Version:  1
Go Version:         go1.12.6
Git Commit:         9b6a98cfd7813513e5697888baa93318395a2055
Built:              Fri Jun 28 13:46:14 2019
OS/Arch:            linux/amd64

Output of podman info --debug:

debug:
  compiler: gc
  git commit: 9b6a98cfd7813513e5697888baa93318395a2055
  go version: go1.12.6
  podman version: 1.4.2
host:
  BuildahVersion: 1.9.0
  Conmon:
    package: cri-o-1.14.4-8521.x86_64
    path: /usr/libexec/crio/conmon
    version: ""
  Distribution:
    distribution: '"rhel"'
    version: "7.5"
  MemFree: 414194458624
  MemTotal: 540603346944
  OCIRuntime:
    package: runc-1.0.0-59.dev.git2abd837.el7.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.0'
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 27
  hostname: compute0011
  kernel: 3.10.0-862.3.2.el7.jump1.x86_64
  os: linux
  rootless: true
  uptime: 172h 29m 19.02s (Approximately 7.17 days)
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
  - quay.io
  - registry.access.redhat.com
store:
  ConfigFile: /home/jfloryintern/.config/containers/storage.conf
  ContainerStore:
    number: 0
  GraphDriverName: vfs
  GraphOptions: null
  GraphRoot: /physical/gpfs/home/home01/data_files/jfloryintern/.local/share/containers/storage
  GraphStatus: {}
  ImageStore:
    number: 0
  RunRoot: /run/user/43228
  VolumePath: /physical/gpfs/home/home01/data_files/jfloryintern/.local/share/containers/storage/volumes

Additional environment details (AWS, VirtualBox, physical, etc.):

Physical node.

@jwflory

This comment has been minimized.

Copy link
Contributor Author

commented Jul 2, 2019

Per IRC, here is the output with debug logs:

$ podman --log-level=debug pull docker.io/fedora:latest
INFO[0000] running as rootless                          
DEBU[0000] Initializing boltdb state at /physical/gpfs/home/home01/data_files/jfloryintern/.local/share/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver vfs                       
DEBU[0000] Using graph root /physical/gpfs/home/home01/data_files/jfloryintern/.local/share/containers/storage 
DEBU[0000] Using run root /run/user/43228               
DEBU[0000] Using static dir /physical/gpfs/home/home01/data_files/jfloryintern/.local/share/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/user/43228/libpod/tmp     
DEBU[0000] Using volume path /physical/gpfs/home/home01/data_files/jfloryintern/.local/share/containers/storage/volumes 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] [graphdriver] trying provided driver "vfs"   
DEBU[0000] Initializing event backend journald          
DEBU[0000] parsed reference into "[vfs@/physical/gpfs/home/home01/data_files/jfloryintern/.local/share/containers/storage+/run/user/43228]docker.io/library/fedora:latest" 
Trying to pull docker.io/fedora:latest...DEBU[0000] reference rewritten from 'docker.io/library/fedora:latest' to 'docker.io/library/fedora:latest' 
DEBU[0000] Trying to pull "docker.io/library/fedora:latest" 
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration 
DEBU[0000]  Using "default-docker" configuration        
DEBU[0000]  No signature storage configuration found for docker.io/library/fedora:latest 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/docker.io 
DEBU[0000] GET https://registry-1.docker.io/v2/         
DEBU[0000] Ping https://registry-1.docker.io/v2/ status 401 
DEBU[0000] GET https://auth.docker.io/token?scope=repository%3Alibrary%2Ffedora%3Apull&service=registry.docker.io 
DEBU[0000] GET https://registry-1.docker.io/v2/library/fedora/manifests/latest 
DEBU[0000] Using blob info cache at /home/jfloryintern/.local/share/containers/cache/blob-info-cache-v1.boltdb 
DEBU[0000] Source is a manifest list; copying (only) instance sha256:46bba3ca0e509a3c46d60907f7ae3f6835f1f33861aa6ef855dac805574fd258 
DEBU[0000] GET https://registry-1.docker.io/v2/library/fedora/manifests/sha256:46bba3ca0e509a3c46d60907f7ae3f6835f1f33861aa6ef855dac805574fd258 
DEBU[0000] IsRunningImageAllowed for image docker:docker.io/library/fedora:latest 
DEBU[0000]  Using default policy section                
DEBU[0000]  Requirement 0: allowed                      
DEBU[0000] Overall: allowed                             
DEBU[0000] Downloading /v2/library/fedora/blobs/sha256:289289d1a15b92ace1a822f9920f5c1477691aa01ca567a8502eb7eb67eb4a80 
DEBU[0000] GET https://registry-1.docker.io/v2/library/fedora/blobs/sha256:289289d1a15b92ace1a822f9920f5c1477691aa01ca567a8502eb7eb67eb4a80 
Getting image source signatures
DEBU[0001] Manifest has MIME type application/vnd.docker.distribution.manifest.v2+json, ordered candidate list [application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.v1+prettyjws, application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v1+json] 
DEBU[0001] ... will first try using the original manifest unmodified 
DEBU[0001] Downloading /v2/library/fedora/blobs/sha256:8f6ac7ed4a91c9630083524efcef2f59f27404320bfee44397f544c252ad4bd4 
DEBU[0001] GET https://registry-1.docker.io/v2/library/fedora/blobs/sha256:8f6ac7ed4a91c9630083524efcef2f59f27404320bfee44397f544c252ad4bd4 
DEBU[0001] Detected compression format gzip             
DEBU[0001] Using original blob without modification     
Copying blob 8f6ac7ed4a91 done
DEBU[0006] No compression detected                      
DEBU[0006] Using original blob without modification     
Copying config 289289d1a1 done
Writing manifest to image destination
Storing signatures
DEBU[0006] Start untar layer                            
ERRO[0007] Error while applying layer: ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied 
ERRO[0007] Error pulling image ref //fedora:latest: Error committing the finished image: error adding layer with blob "sha256:8f6ac7ed4a91c9630083524efcef2f59f27404320bfee44397f544c252ad4bd4": ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied 
Failed
ERRO[0007] error pulling image "docker.io/fedora:latest": unable to pull docker.io/fedora:latest: unable to pull image: Error committing the finished image: error adding layer with blob "sha256:8f6ac7ed4a91c9630083524efcef2f59f27404320bfee44397f544c252ad4bd4": ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied
@jwflory

This comment has been minimized.

Copy link
Contributor Author

commented Jul 2, 2019

@giuseppe debugged with me on IRC today and we determined that this is an issue with the General Parallel File System (GPFS) from IBM:

[jfloryintern@compute0011 ~]$ unshare -r bash -c "mkdir foo; chmod 000 foo; touch foo/bar"
touch: cannot touch ‘foo/bar’: Permission denied

This can probably be re-tagged from kind/bug to kind/feature. 🙂

@jwflory jwflory changed the title `podman pull` with VFS driver as rootless user requires opening /root/.bash_logout Add rootless support for General Parallel File Systems (GPFS) Jul 2, 2019

@rhatdan

This comment has been minimized.

Copy link
Member

commented Jul 3, 2019

Is this like NFS in that GPFS does not understand User Namespace?

@jwflory

This comment has been minimized.

Copy link
Contributor Author

commented Jul 3, 2019

@rhatdan As I understand, yes.

@rhatdan

This comment has been minimized.

Copy link
Member

commented Jul 4, 2019

Could you add a PR to modify rootless.md to add this to the NFS section?

jwflory added a commit to jwflory/libpod that referenced this issue Jul 8, 2019
rootless.md: Include GPFS as a parallel filesystem
Per @rhatdan's request in containers#3478, this commit makes a note of supporting
General Parallel File System by IBM since it shares the same root issue
as NFS for rootless containers.

Signed-off-by: Justin W. Flory <git@jwf.io>
jwflory added a commit to jwflory/libpod that referenced this issue Jul 8, 2019
rootless.md: Include GPFS as a parallel filesystem
Per @rhatdan's request in containers#3478, this commit makes a note of supporting
General Parallel File System by IBM since it shares the same root issue
as NFS for rootless containers.

Signed-off-by: Justin W. Flory <git@jwf.io>
EmilienM added a commit to EmilienM/libpod that referenced this issue Aug 5, 2019
rootless.md: Include GPFS as a parallel filesystem
Per @rhatdan's request in containers#3478, this commit makes a note of supporting
General Parallel File System by IBM since it shares the same root issue
as NFS for rootless containers.

Signed-off-by: Justin W. Flory <git@jwf.io>
@rhatdan

This comment has been minimized.

Copy link
Member

commented Aug 5, 2019

Ok we have updated the docs.

@rhatdan rhatdan closed this Aug 5, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.