REST API: "\u003cmissing\u003e" instead of "<missing>" in returned Id attribute ( /v1.24/images/${img}/history ) #17769

eriksjolund · 2023-03-14T07:06:21Z

Issue Description

In the JSON ouput from the REST API (/v1.24/images/${img}/history)

docker returns

<missing>

podman returns

\u003cmissing\u003e

I guess both variants are just two ways to represent the same JSON content so one could argue if this issue is really a bug, but I think there will be less incompatibility issues with docker if podman would use the same output.

(There might be buggy clients that are using the REST API although I haven't seen any examples of problems related to this issue)

Steps to reproduce the issue

On a Fedora 37 computer (arch: amd64)

sudo -i
dnf install -y podman
dnf install -y moby-engine
systemctl start docker
useradd test1
usermod -aG docker test1
machinectl shell test1@
podman pull alpine:3.17.2
run podman images and detect the image id. Record the result in a shell variable
img=b2aa39c304c2
docker pull alpine:3.17.2
systemctl --user start podman.socket
curl -s --unix-socket /var/run/docker.sock http://localhost/v1.24/images/${img}/history | grep -o -P '.{0,7}missing.{0,7}'
curl -s --unix-socket $XDG_RUNTIME_DIR/podman/podman.sock http://localhost/v1.24/images/${img}/history | grep -o -P '.{0,7}missing.{0,7}'

Describe the results you received

$ curl -s --unix-socket /var/run/docker.sock http://localhost/v1.24/images/${img}/history | grep -o -P '.{0,7}missing.{0,7}'
"Id":"<missing>","Siz
$ curl -s --unix-socket $XDG_RUNTIME_DIR/podman/podman.sock  http://localhost/v1.24/images/${img}/history  | grep -o -P '.{0,7}missing.{0,7}'
"\u003cmissing\u003e"
$

Describe the results you expected

$ curl -s --unix-socket /var/run/docker.sock http://localhost/v1.24/images/${img}/history | grep -o -P '.{0,7}missing.{0,7}'
"Id":"<missing>","Siz
$ curl -s --unix-socket $XDG_RUNTIME_DIR/podman/podman.sock  http://localhost/v1.24/images/${img}/history  | grep -o -P '.{0,7}missing.{0,7}'
"Id":"<missing>","Siz
$

podman info output

host:
  arch: amd64
  buildahVersion: 1.29.0
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.6-3.fc37.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.6, commit: '
  cpuUtilization:
    idlePercent: 99.92
    systemPercent: 0.02
    userPercent: 0.06
  cpus: 16
  distribution:
    distribution: fedora
    variant: workstation
    version: "37"
  eventLogger: journald
  hostname: asus
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 50147
      size: 1
    - container_id: 1
      host_id: 37020512
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 50147
      size: 1
    - container_id: 1
      host_id: 37020512
      size: 65536
  kernel: 6.1.15-200.fc37.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 838037504
  memTotal: 7691788288
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.8.1-1.fc37.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.8.1
      commit: f8a096be060b22ccd3d5f3ebe44108517fbf6c30
      rundir: /run/user/50147/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  remoteSocket:
    path: /run/user/50147/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-8.fc37.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 7691300864
  swapTotal: 7691300864
  uptime: 11h 10m 1.00s (Approximately 0.46 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/test612/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/test612/.local/share/containers/storage
  graphRootAllocated: 407822663680
  graphRootUsed: 378301325312
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/user/50147/containers
  transientStore: false
  volumePath: /home/test612/.local/share/containers/storage/volumes
version:
  APIVersion: 4.4.2
  Built: 1677669779
  BuiltTime: Wed Mar  1 12:22:59 2023
  GitCommit: ""
  GoVersion: go1.19.6
  Os: linux
  OsArch: linux/amd64
  Version: 4.4.2

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

No response

Additional information

No response

The text was updated successfully, but these errors were encountered:

github-actions · 2023-04-18T00:06:29Z

A friendly reminder that this issue had no activity for 30 days.

nwoythal · 2023-08-02T15:09:59Z

It seems that it's pretty easily fixable with the following:

diff --git a/pkg/api/handlers/utils/handler.go b/pkg/api/handlers/utils/handler.go
index f2f8ab1dc..2e457c39e 100644
--- a/pkg/api/handlers/utils/handler.go
+++ b/pkg/api/handlers/utils/handler.go
@@ -160,7 +160,7 @@ func WriteJSON(w http.ResponseWriter, code int, value interface{}) {
 	w.WriteHeader(code)
 
 	coder := json.NewEncoder(w)
-	coder.SetEscapeHTML(true)
+	coder.SetEscapeHTML(false)
 	if err := coder.Encode(value); err != nil {
 		logrus.Errorf("Unable to write json: %q", err)
 	}

It strikes me as being possibly dangerous, though, to disable escaping HTML chars for an encoding issue, when tools like jq already understand these OOTB.

jackgris · 2023-10-12T03:23:02Z

Maybe it's correct that you are saying, @nwoythal. If we examine the Moby project, you can check the WriteJSON function at this link: https://github.com/moby/moby/blob/b33ad6fff6123f16f042d15623194f5c4bcd3ea6/api/server/httputils/httputils.go#L94, where the escape HTML is set to false.

This is related to the issue containers#17769 Signed-off-by: Gabriel Pozo <jackgris2@gmail.com>

jackgris · 2023-10-16T16:47:31Z

I see now that the ID is displayed in this way: Id : sha256:<missing>. The only difference compared to Docker is that Podman shouldn't display the sha256 in the missing cases. I can't find the commit where that change was made, but the HTML escape is still set to coder.SetEscapeHTML(true).

eriksjolund · 2023-10-17T04:06:09Z

I noticed that Podman and Docker had different values for the Size field so I opened a new issue for that

REST API: /v1.24/images/${img}/history contains "Size":0 where docker has "Size":7458929 #20375

eriksjolund · 2023-12-07T21:17:34Z

I tried out the reproducer on Fedora 39 with podman 4.8.1

$ curl -s --unix-socket /var/run/docker.sock http://localhost/v1.24/images/${img}/history | grep -o -P '.{0,7}missing.{0,7}'
"Id":"<missing>","Siz
$ curl -s --unix-socket $XDG_RUNTIME_DIR/podman/podman.sock  http://localhost/v1.24/images/${img}/history  | grep -o -P '.{0,7}missing.{0,7}'
:\u003cmissing\u003e"
$

I got the same result as before but with a small difference. There is a colon : instead of a quote (") before \u003cmissing\u003e"

Here is the whole output (without using grep)

$ curl -s --unix-socket $XDG_RUNTIME_DIR/podman/podman.sock  http://localhost/v1.24/images/${img}/history 
[{"Id":"sha256:b2aa39c304c27b96c1fef0c06bee651ac9241d49c4fe34381cab8453f9a89c7d","Created":1676090802,"CreatedBy":"/bin/sh -c #(nop)  CMD [\"/bin/sh\"]","Tags":["docker.io/library/alpine:3.17.2"],"Size":0,"Comment":""},{"Id":"sha256:\u003cmissing\u003e","Created":1676090802,"CreatedBy":"/bin/sh -c #(nop) ADD file:40887ab7c06977737e63c215c9bd297c0c74de8d12d16ebdf1c3d40ac392f62d in / ","Tags":["docker.io/library/alpine:3.17.2"],"Size":7337984,"Comment":""}]

$ curl -s --unix-socket /var/run/docker.sock http://localhost/v1.24/images/${img}/history 
[{"Comment":"","Created":1676090802,"CreatedBy":"/bin/sh -c #(nop)  CMD [\"/bin/sh\"]","Id":"sha256:b2aa39c304c27b96c1fef0c06bee651ac9241d49c4fe34381cab8453f9a89c7d","Size":0,"Tags":["alpine:3.17.2"]},{"Comment":"","Created":1676090802,"CreatedBy":"/bin/sh -c #(nop) ADD file:40887ab7c06977737e63c215c9bd297c0c74de8d12d16ebdf1c3d40ac392f62d in / ","Id":"<missing>","Size":7049701,"Tags":null}]

I formatted the outputs with jq -S and then ran diff -u

--- /tmp/jq-output.podman	2023-12-07 21:48:35.263207666 +0100
+++ /tmp/jq-output.docker	2023-12-07 21:48:27.174388226 +0100
@@ -6,17 +6,15 @@
     "Id": "sha256:b2aa39c304c27b96c1fef0c06bee651ac9241d49c4fe34381cab8453f9a89c7d",
     "Size": 0,
     "Tags": [
-      "docker.io/library/alpine:3.17.2"
+      "alpine:3.17.2"
     ]
   },
   {
     "Comment": "",
     "Created": 1676090802,
     "CreatedBy": "/bin/sh -c #(nop) ADD file:40887ab7c06977737e63c215c9bd297c0c74de8d12d16ebdf1c3d40ac392f62d in / ",
-    "Id": "sha256:<missing>",
-    "Size": 7337984,
-    "Tags": [
-      "docker.io/library/alpine:3.17.2"
-    ]
+    "Id": "<missing>",
+    "Size": 7049701,
+    "Tags": null
   }
 ]

Note the extra sha256: Podman prints. (It was already mentioned by @jackgris
in #17769 (comment))
I don't know if this should be split out into a new GitHub issue.

eriksjolund added the kind/bug Categorizes issue or PR as related to a bug. label Mar 14, 2023

github-actions bot added the stale-issue label Apr 18, 2023

rhatdan added Good First Issue This issue would be a good issue for a first time contributor to undertake. bugweek and removed stale-issue labels Apr 21, 2023

openshift-ci bot assigned jakecorrenti May 30, 2023

jakecorrenti removed their assignment Jun 16, 2023

jackgris added a commit to jackgris/podman that referenced this issue Oct 16, 2023

fix: escaped character when the image history returns a missing ID

79b69a9

This is related to the issue containers#17769 Signed-off-by: Gabriel Pozo <jackgris2@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

REST API: "\u003cmissing\u003e" instead of "<missing>" in returned Id attribute ( /v1.24/images/${img}/history ) #17769

REST API: "\u003cmissing\u003e" instead of "<missing>" in returned Id attribute ( /v1.24/images/${img}/history ) #17769

eriksjolund commented Mar 14, 2023 •

edited

github-actions bot commented Apr 18, 2023

nwoythal commented Aug 2, 2023

jackgris commented Oct 12, 2023

jackgris commented Oct 16, 2023

eriksjolund commented Oct 17, 2023

eriksjolund commented Dec 7, 2023

REST API: "\u003cmissing\u003e" instead of "<missing>" in returned Id attribute ( /v1.24/images/${img}/history ) #17769

REST API: "\u003cmissing\u003e" instead of "<missing>" in returned Id attribute ( /v1.24/images/${img}/history ) #17769

Comments

eriksjolund commented Mar 14, 2023 • edited

Issue Description

Steps to reproduce the issue

Describe the results you received

Describe the results you expected

podman info output

Podman in a container

Privileged Or Rootless

Upstream Latest Release

Additional environment details

Additional information

github-actions bot commented Apr 18, 2023

nwoythal commented Aug 2, 2023

jackgris commented Oct 12, 2023

jackgris commented Oct 16, 2023

eriksjolund commented Oct 17, 2023

eriksjolund commented Dec 7, 2023

eriksjolund commented Mar 14, 2023 •

edited