Skip to content

Overlayfs and containers

Amir Goldstein edited this page Aug 23, 2020 · 31 revisions

Table of Contents

Overlayfs nr. 1 user

Containers wish list

  • shiftfs/fsid mapping
  • Userns root mount (FS_USERNS_MOUNT) (coming up)
  • "volatile" overlay (coming up)
  • Containerized syncfs(2) (RFC)

Why use overlayfs?

  • Because it is the default
  • Share page cache
  • Fast startup time
  • Native diff - faster image build

Historical issues

Recent progress

  • Rename directories (redirect_dir=on) (v4.10)
  • Preserve hardlinks (index=on) (v4.13)
  • NFS export (nfs_export=on) (v4.16)
  • Unique object identifier (xino=auto) (v4.12..v4.17)
  • Consistent file data (excluding mmap) (v4.19)
  • Metadata only copy up (metacopy=on) (v4.19)

Why are the features disabled?

  • Backward compatibility with old kernels
  • Enabling xino only impacts 32bit compat st_ino
  • Impact some container use cases

Container use cases

Local instance

  • Stays on one container host and storage driver (overlay2)
  • Can safely enable: redirect_dir and metacopy
  • Enabling index and nfs_export exposes mount leak bugs
  • You've got nothing to lose!

Migratable instance

  • Migrate instance between container hosts with same native storage driver (overlay2)
  • Include trusted.overlay xattrs in image layers
  • No need to convert opaque xattr and whiteout chardev to portable .wh. files
  • Can safely enable: redirect_dir and metacopy
  • The index and nfs_export features are NOT "natively migratable"

Portable image

  • Enabling redirect_dir and metacopy requires more work
  • vnd.oci.image.layer.v1 conforms to decade old overlay/aufs whiteouts and opaque directories
  • Is it time for vnd.oci.image.layer.v2 with metacopy/redirect support?
  • Downgrade to vnd.oci.image.layer.v1 format with overlayfs tools
    • IF is_metacopy_file_or_redirect_dir(upperdir/$path):
      • rm -rf imagedir/$path
      • cp -a mountdir/$path imagedir/$path

Recommendations

  • Enable redirect_dir,metacopy for new instances
  • Allow user to configure NFS exportable instances (index,nfs_export)
  • Disable index before instance migration
  • Downgrade redirect_dir,metacopy during image creation