podman play kube does not pull images like it should

[ghost]

/kind bug

podman play kube does not seem to be pulling updated images as directed by the :latest tag or imagePullPolicy: always.

Steps to reproduce the issue:

1. podman stop/remove an outdated container
2. podman play kube a .yml containing the :latest tag or imagePullPolicy: always
3. The container is recreated from the outdated image

Output of podman version:

Version:            1.7.0
RemoteAPI Version:  1
Go Version:         go1.13.5
OS/Arch:            linux/amd64

Output of podman info --debug:

debug:
  compiler: gc
  git commit: ""
  go version: go1.13.5
  podman version: 1.7.0
host:
  BuildahVersion: 1.12.0
  CgroupVersion: v2
  Conmon:
    package: conmon-2.0.2-1.fc31.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.2, commit: 186a550ba0866ce799d74006dab97969a2107979'
  Distribution:
    distribution: fedora
    version: "31"
  OCIRuntime:
    name: crun
    package: crun-0.10.6-1.fc31.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.10.6
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  arch: amd64
  cpus: 12
  eventlogger: journald
  kernel: 5.3.16-300.fc31.x86_64
  os: linux
  rootless: false
registries:
  search:
  - docker.io
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - quay.io
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 26
  GraphDriverName: overlay
  GraphOptions:
    overlay.mountopt: nodev,metacopy=on
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  ImageStore:
    number: 12
  RunRoot: /var/run/containers/storage
  VolumePath: /var/lib/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

podman-1.7.0-2.fc31.x86_64

[rhatdan]

@komic Do you have a simple kubernernetes.yaml file we could test with?

[ghost]

A basic example:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2020-01-15T22:26:19Z"
  labels:
    app: plex
  name: pod-plex
spec:
  containers:
    image: docker.io/linuxserver/plex:latest
    imagePullPolicy: always
    name: plex
    resources: {}
    securityContext:
      allowPrivilegeEscalation: true
      capabilities: {}
      privileged: false
      readOnlyRootFilesystem: false
    workingDir: /
status: {}

[baude]

@komic I can understand that imagePullPolicy: always would trigger a pull of the latest but at least in podman, there is no precedent for triggering a pull based on the latest tag. Would honoring the pull policy be sufficient?

[ghost]

Of course, that does not match the behavior described by https://kubernetes.io/docs/concepts/containers/images/ though.

[baude]

@komic both pieces were added to play kube!

[ghost]

Good stuff, thank you all!