Issues with container for postgres:latest

[atodorov]

We're using postgres:latest for testing purposes and a recent CI job from 3 days ago was passing:

https://github.com/kiwitcms/Kiwi/actions/runs/18640739377/job/53138995941

REPOSITORY   TAG       IMAGE ID       CREATED       SIZE
postgres     latest    194f5f2a900a   3 weeks ago   456MB

Ones from today are all failing and I can see that IMAGE ID has changed meanwhile:

https://github.com/kiwitcms/Kiwi/actions/runs/18742159756/job/53460818742?pr=4152

REPOSITORY   TAG       IMAGE ID       CREATED      SIZE
postgres     latest    a38f9f77ff88   7 days ago   456MB

Both should be Postgres 18 so not quite sure why the error. What I see in one of our logs is

2025-10-23T08:48:23.4521845Z STDERR: Error response from daemon: Container 6fe6b6f0e3a524cde0119b63498251d705b2e8335baf7089dc554a8e09bf1736 is restarting, wait until the container is running

So it looks like the postgres container is restarting itself and not being able to run properly. Will investigate more on my side to figure out what's wrong.

[Vincentp90]

Idk if it's the same issue. My docker-compose was suddenly failing today to start postgres (still worked yesterday), it also got in a restart loop. My issue was that i was mounting the volume to /var/lib/postgresql/data but it seems that with 18 you now have to mount your volume to /var/lib/postgresql.

I fixed my issue by deleting the /data at the db_data volume definition in my docker-compose.yml:

db:
    image: postgres
    restart: always
    shm_size: 128mb
    ports:
      - 5432:5432
    environment:
      POSTGRES_PASSWORD: example
    volumes:
      - db_data:/var/lib/postgresql/data

changed to

db:
    image: postgres
    restart: always
    shm_size: 128mb
    ports:
      - 5432:5432
    environment:
      POSTGRES_PASSWORD: example
    volumes:
      - db_data:/var/lib/postgresql

This is the error I had:
Error: in 18+, these Docker images are configured to store database data in a format which is compatible with "pg_ctlcluster" (specifically, using major-version-specific directory names). This better reflects how PostgreSQL itself works, and how upgrades are to be performed.

See also https://github.com/docker-library/postgres/pull/1259⁠

Counter to that, there appears to be PostgreSQL data in: /var/lib/postgresql/data (unused mount/volume)

This is usually the result of upgrading the Docker image without upgrading the underlying database using "pg_upgrade" (which requires both versions).

The suggested container configuration for 18+ is to place a single mount at /var/lib/postgresql which will then place postgreSQL data in a subdirectory, allowing usage of "pg_upgrade --link" without mount point boundary issues.

See https://github.com/docker-library/postgres/issues/37⁠ for a (long) discussion around this process, and suggestions for how to do so.

[redbeardymcgee]

I have also tried to remove /data from my mount point in a podman quadlet, but it results in the same error. The only difference is Counter to that, there appears to be PostgreSQL data in: /var/lib/postgresql rather than /var/lib/postgresql/data.

I'm guessing this means we will need to export the data and reimport it to a new volume, or maybe we can just move the data around on the filesystem. I'm not sure, but this is going to take down dozens of my containers for an indefinite period of time.

EDIT: After re-reading the error a few more times, there is an instruction to use the above suggested mount point, then run pg_upgrade --link. However, this is not possible because the container will error and die before the user is able to enter it and run the upgrade command. Potential workaround may be to change the entrypoint and/or exec command to run the pg_upgrade --link operation, then remove the changed entrypoint/exec to allow the container to run normally again. Not tested.

EDIT#2: You must identify the directory where the old cluster binaries reside. Please use the -b command-line option or the PGBINOLD environment variable. Failure, exiting is the result of a naive Exec=sh -c 'sleep 3 && su postgres -c "pg_upgrade --link"' in my quadlet. I don't know what old cluster binaries even are, yet, let alone where they are stored in my container.

EDIT#3: More fumbling around resulted in no progress, but I did come to understand that there may be no upgrade process in place for this scenario. At least, if there is, I couldn't figure it out. Noted in the above edit, you need to point to the old cluster binaries with the -b option or PGBINOLD env var; this would simply be the old versions of the postgres binaries, which simply will not exist on an upgraded container, so this requirement is impossible to fulfill. As you work out more cli options that must be supplied, you will continue to run into errors that are mostly unresolvable without unusual manual workarounds. Remember that the error states you should place a single mount at /var/lib/postgresql which will then place postgreSQL data in a subdirectory. What subdirectory, and is it created automatically?

Is this upgrade process documented anywhere?

[atodorov]

Just to clarify:

I understand that a new data structure is in place for 18+ in order to make upgrades easier. That's fine and on my side at least I would take care to adjust container mount points to abide by the new expectations.

It's a bit unfortunate that this appears to have been enforced just recently inside the version 18 container, while it wasn't the case with earlier versions. My beef is with not knowing what changed internally to cause this issue!

I guess closing this for now as I know how to work around it in my particular case!

[redbeardymcgee]

@atodorov Please don't close the issue while others are still working through it. Your workaround is not universal.

[mikefrancis]

@Vincentp90 I had the same issue and your fix worked, thanks so much for posting.