compression can't be done when specify option --compression-format in push command.

[fhwsuper]

Issue Description

compression can't be done when specify option --compression-format in push command.

Steps to reproduce the issue

podman push --compression-format=gzip --compress --log-level debug httpd dir:/tmp1

DEBU[0005] Checking if we can reuse blob
sha256:01597d537f57d33370c31559a880654ee80571bb5e7c22584629576b4d72b834: general substitution = true, compression for MIME type "application/vnd.docker.image.rootfs.diff.tar" = true
DEBU[0005] exporting filesystem layer "7eee2c97dd0632f3796ea2900ce78ea3f753f4452920b90c583b2
Copying blob 6485bed63627 done
Copying blob 34e2a1369fc5 done
Copying blob c672c6903747 done
Copying blob c7c38e3c39ad done
Copying blob 01597d537f57 done
DEBU[0005] exporting opaque data as blob "sha256:a981c8992512d65c9b450a9ecabb1cb9d35bb6b03f3640f86471032d5800d825"
DEBU[0005] No compression detected
DEBU[0005] Compression change for blob sha256:a981c8992512d65c9b450a9ecabb1cb9d35bb6b03f3640f86471032d5800d825 ("application/vnd.docker.container.image.v1+json") not supported
DEBU[0005] Using original blob without modification
Copying config a981c89925 done
Writing manifest to image destination
Storing signatures
DEBU[0005] Called push.PersistentPostRunE(./podman push --compression-format=gzip --compress --log-level debug httpd dir:/tmp1)

Describe the results you received

no compression action,

Describe the results you expected

compression should be done according to the format gzip or zest

podman info output

host:
  arch: amd64
  buildahVersion: 1.28.0
  cgroupControllers:
  - cpuset
  - cpu
  - cpuacct
  - blkio
  - memory
  - devices
  - freezer
  - net_cls
  - perf_event
  - net_prio
  - hugetlb
  - pids
  - rdma
  - misc
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: conmon_2.0.9-1_amd64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.1.2, commit: b1ca51f59a8865565425a3c85bfe90e0275d465a'
  cpuUtilization:
    idlePercent: 99.67
    systemPercent: 0.16
    userPercent: 0.17
  cpus: 2
  distribution:
    codename: focal
    distribution: ubuntu
    version: "20.04"
  eventLogger: journald
  hostname: fanhuawei-Standard-PC-i440FX-PIIX-1996
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.15.0-48-generic
  linkmode: dynamic
  logDriver: journald
  memFree: 318783488
  memTotal: 3125534720
  networkBackend: cni
  ociRuntime:
    name: runc
    package: runc_1.1.4-0ubuntu1~20.04.3_amd64
    path: /usr/sbin/runc
    version: |-
      runc version 1.1.4-0ubuntu1~20.04.3
      spec: 1.0.2-dev
      go: go1.18.1
      libseccomp: 2.5.1
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: ""
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 1942048768
  swapTotal: 2147479552
  uptime: 5735h 41m 43.00s (Approximately 238.96 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
  - quay.io
  - registry.fedoraproject.org
  - sys-zosdev-containers-team-docker-local.artifactory.swg-devops.com
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 33
    paused: 0
    running: 5
    stopped: 28
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 104557666304
  graphRootUsed: 27983331328
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /tmp1
  imageStore:
    number: 6
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.3.1
  Built: 1682589247
  BuiltTime: Thu Apr 27 17:54:07 2023
  GitCommit: ""
  GoVersion: go1.19
  Os: linux
  OsArch: linux/amd64
  Version: 4.3.1

root@fanhuawei-Standard-PC-i440FX-PIIX-1996:/home/fanhuawei/podman/podman-4.3.1/bin#

Podman in a container

Yes

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[Luap99]

@vrothberg @mtrmac PTAL

[mtrmac]

I’d expect quite a few more debug logs. Can you run this maybe with |& cat to avoid the interactive progress bars that are probably hiding data, please?

---

(As a guess, layer reuse is prioritized over using the requested compression format. That might need to change eventually, for correctly building Zstd and Gzip-only images.)

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@giuseppe isn't this what you have been working on?

[giuseppe]

I think this is a different issue

[mtrmac]

We are missing debug logs.

If my guess is correct, @flouthoc ’s work on zstd clones will will ~require fixing that, e.g. containers/image#1987 (comment) .

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[mtrmac]

@flouthoc has implemented requireCompressionFormatMatch in c/image, it’s not yet a public API there.

But there are no debug logs in this issue, so we are just guessing that that’s what’s going on.

[flouthoc]

Yes I will wire it up with CLI in this coming week, we have a similar issue in buildah.

[flouthoc]

PR #19640 should close this.