Latest update causes Postgres errors when using mounted directory

[Tawmy]

Describe the bug

I have mounted a local directory into /var/lib/postgresql/data. I am then streaming data into this Postgres database (60+ million entries). While doing so, I get the following error:

  CREATE INDEX IF NOT EXISTS <redacted> ON <redacted> USING gist (cli_pfx) TABLESPACE pg_default;

Exception Npgsql.PostgresException (0x80004005): XX000: unexpected data beyond EOF in block 15313 of relation base/16384/16797

[...]

Exception data:
Severity: ERROR
SqlState: XX000
MessageText: unexpected data beyond EOF in block 15313 of relation base/16384/16797
Hint: This has been seen to occur with buggy kernels; consider updating your system.
File: bufmgr.c
Line: 1997
Routine: ExtendBufferedRelShared

This issue does not happen on version 1.6.1. I cannot verify whether it does or doesn't happen on 1.6.2 as that version is not available for download. I believe I was using 1.6.2 when it still worked, but cannot say for certain.

To Reproduce

1. mount a local directory into the Postgres data directory
2. stream a large amount of data (gigabytes) into this database
3. Create an index for the table after data is streamed

Expected behavior

Data is written as normal. No error happened in version 1.6.1.

Diagnostic report (REQUIRED)

OrbStack info:
Version: 1.6.3
Commit: 10d8fa4cb507142066b51b2febbd367720a5732a (v1.6.3)

System info:
macOS: 14.5 (23F79)
CPU: arm64, 11 cores
CPU model: Apple M3 Pro
Model: Mac15,6
Memory: 36 GiB

Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2024-07-03T15-23-54.368356Z.zip

Screenshots and additional context (optional)

No response

[Tawmy]

It seems the build linked in #1300 (comment) fixes this issue. Leaving it open for you to review.

[Tawmy]

Nevermind, it does not. Happened again.

[siemendev]

Also having Problems with Volumes in an PHP App after the recent Update. Project worked fine but after updating Orbstack somehow the Framework (symfony) cannot build up a service container - no userland classes are available on service container (symfony) build.
Looks to me like symfony cannot load/access the files and therefore does not recognize them. Only happens on compose with volumes, I changed the system to docker compose watch to sync the project manually and everything works fine again.

@kdrag0n this might have a big impact on many users?

[siemendev]

After a downgrade to 1.6.1 I can confirm that everything works fine again, so I can pinpoint it to the 1.6.3 as well.

[tmontes]

Chiming in with a very similar experience when writing to PostgreSQL -- doesn't need to be huge, but a good chunk... In my case I can easily reproduce by loading a 1 GB table or VACUUMing it.

[siemendev]

Just talked to the guys on discord, they are aware of the issue and can finally reproduce it. I don't think they will rollback the release though, the answer seems to be downgrading to 1.6.1 until there's a fix in >=1.6.4

The downgrade can be done stopping & removing the app and then installing 1.6.1 from here: https://docs.orbstack.dev/downgrade

[kdrag0n]

Thanks for the details, especially the size. We were able to put together a reproducer using a large dataset.

Fixed for the next version.

[kdrag0n]

Released in v1.6.4.

[perrornet]

In Version 1.6.4 (17192), this error still exists:

2024-08-05 03:10:24.095 UTC [27] LOG:  checkpoint complete: wrote 38 buffers (0.2%); 0 WAL file(s) added, 0 removed, 33 recycled; write=269.267 s, sync=0.008 s, total=269.368 s; sync files=68, longest=0.002 s, average=0.001 s; distance=529040 kB, estimate=529040 kB; lsn=1F/5C81A788, redo lsn=1F/57004220
2024-08-05 03:11:24.149 UTC [916] ERROR:  unexpected data beyond EOF in block 3544 of relation base/127918/136090