\t in --format does not preserve tab character

[asottile]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When using \t in a --format (such as with podman history) I expect to receive tab characters in the output -- but it appears that podman is expanding them to spaces. docker preserves these characters in the output.

Steps to reproduce the issue:

1. podman history $'--format={{.ID}}\t{{.CreatedBy}} someimage

(it may be useful to pipe this to hd -c)

Describe the results you received:

I do not see any tab characters in the output

Describe the results you expected:

I expect tab characters to separate the ID and the command

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

$ podman version
Version:      3.2.2
API Version:  3.2.2
Go Version:   go1.15.2
Built:        Wed Dec 31 19:00:00 1969
OS/Arch:      linux/amd64

Output of podman info --debug:

$ podman info --debug
host:
  arch: amd64
  buildahVersion: 1.21.0
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: 'conmon: /usr/libexec/podman/conmon'
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.0.27, commit: '
  cpus: 5
  distribution:
    distribution: ubuntu
    version: "20.04"
  eventLogger: journald
  hostname: babibox
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.8.0-59-generic
  linkmode: dynamic
  memFree: 4439875584
  memTotal: 8347717632
  ociRuntime:
    name: crun
    package: 'crun: /usr/bin/crun'
    path: /usr/bin/crun
    version: |-
      crun version 0.20.1.5-925d-dirty
      commit: 0d42f1109fd73548f44b01b3e84d04a279e99d2e
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 1.1.8
      commit: unknown
      libslirp: 4.3.1-git
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.4.3
  swapFree: 1960964096
  swapTotal: 1964396544
  uptime: 5h 18m 57.43s (Approximately 0.21 days)
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /home/asottile/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: 'fuse-overlayfs: /usr/bin/fuse-overlayfs'
      Version: |-
        fusermount3 version: 3.9.0
        fuse-overlayfs: version 1.5
        FUSE library version 3.9.0
        using FUSE kernel interface version 7.31
  graphRoot: /home/asottile/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 33
  runRoot: /run/user/1000/containers
  volumePath: /home/asottile/.local/share/containers/storage/volumes
version:
  APIVersion: 3.2.2
  Built: 0
  BuiltTime: Wed Dec 31 19:00:00 1969
  GitCommit: ""
  GoVersion: go1.15.2
  OsArch: linux/amd64
  Version: 3.2.2

Package info (e.g. output of rpm -q podman or apt list podman):

$ apt list podman
Listing... Done
podman/unknown 100:3.2.2-1 amd64
podman/unknown 100:3.2.2-1 arm64
podman/unknown 100:3.2.2-1 armhf
podman/unknown 100:3.2.2-1 s390x

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

I'm running virtualbox, but I don't think it affects this

[mheon]

@jwhonce PTAL

[jwhonce]

@asottile We utilize the go text/tabwriter which is converting your given tab to a column separator and padding the cells with spaces. The go given solution is:

podman history $'--format={{.ID}}\xff\t\xff{{.Created}}' someimage

od -c will show you the 377 characters output bracketing the tab. I hope that does not break your solution.

[asottile]

@jwhonce my problem is the output is different than docker so I can't currently get a working solution that works in both

[mheon]

@jwhonce If we were to change to match Docker, would that be a breaking change - is anyone relying on tab as a separator?

[jwhonce]

@mheon Correct, we are using text/tabwriter's for the formatted output in all of our commands.

[jwhonce]

@asottile Is something like podman history $'--format={{.ID}}: {{.Created}}' someimage a solution for you to use with docker and podman?

[asottile]

that produces the same output between podman and docker but I don't see how it's relevant as it does not include a tab character

[vrothberg]

@jwhonce, any chance we could turn the single strings with tabs into string slices? That would allow different packages for rendering (e.g, https://github.com/olekukonko/tablewriter).

I wrote a similar package a couple of years ago when porting vgrep to Go: https://github.com/olekukonko/tablewriter

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@vrothberg @jwhonce Any ideas on this one?

[vrothberg]

Let's wait for a head nod from @jwhonce since he has a deep understanding of the formatting code.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[jwhonce]

@vrothberg @rhatdan Dug through the text/tabwriter library, docker and podman formatting code to determine the root reason for the difference.

Summary: Both leverage Go's text/template library. Podman always uses a text/tabwriter when rendering the output for commands using the common/report pkg. Docker only uses text/tabwriter when the "table" keyword is given in the --format value.

I have an idea I'm going to research. Another potential issue I have found in this research is Docker uses different values for their tabwriter instances per command. We currently use the same values for all commands. This could cause padding to be different when the output is rendered. The common/report library does support unique values.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

I believe this is now fixed. Reopen if I am mistaken.