Skip to content

Conversation

bigpresh
Copy link
Contributor

@bigpresh bigpresh commented Sep 4, 2025

Built new Postgres 17.5 binaries following the instructions in the README (had to add a couple more prereqs before it would build; separate commit adding those prereqs to the dnf install command included).

Postgres 17.5 appears to be the most recent Aurora for Postgres version available at the current time, support for it was announced in July 2025

Closes #78

Built as per instructions in README on an Amazon Linux 2023 EC2
instance.
On building new Postgres 17.5 binaries, the documented `dnf install`
command din't provide all prereqs - these additional ones were needed.
Remove the 16.3 binaries, and update config.js to use the new 17.5
binaries.
@bigpresh bigpresh changed the title bigpresh/postgres 17.5 Postgres 17.5 binaries Sep 4, 2025
@bigpresh bigpresh mentioned this pull request Sep 8, 2025
@bigpresh
Copy link
Contributor Author

Would you like this to also include the pg_restore binary? Another user was looking for pg_restore for 16.x in #77, and, whilst my opinion on that ticket was that "the lambda is only for making backups and streaming them to S3, so I'd imagine that having the tools to restore them in whatever way you see fit is on you", I now realise that the system my team are working on uses the pg_restore included with pgdump-aws-lambda 1.5.1, making me look a bit like a hypocritical tool there :)

If I build the pg_restore binary and add it into this PR too, could it make its way out into a release?

In the instructions for building new binaries, include copying
`pg_restore`.

Also, avoid repeating the filenames, just naming the source dir is
enough.
@bigpresh
Copy link
Contributor Author

I have included the 17.5 pg_restore binary here too. (I rebuilt Postgres 17.5 on a fresh ec2 instance to get it, following the README steps to the letter (including therefore testing the new deps added to the README here); I confirmed that the pg_dump and libpq.so.5 files generated were identical by md5sum then grabbed the pg_restore binary to add here.)

I'd greatly appreciate if this could be released, as I'm waiting on upgrading our Postgres cluster and really don't want to maintain a fork if I can help it :) If there's anything I can do to help / any changes you want to the PR, let me know and I'll jump on them ASAP.

@bigpresh
Copy link
Contributor Author

bigpresh commented Sep 22, 2025

Hmm. It looks like SSL support isn't included in these binaries - having tested them manually bundled into one of our lambdas, I got this:

2025-09-22T21:42:29.314Z	c8a93a9f-9d27-4c3b-a869-f8b3c669084b	ERROR	Child process stderr: /var/task/bin/postgres-17.5/pg_restore: error while loading shared libraries: libssl.so.3: cannot open shared object file: No such file or directory

They were built as per the instructions with ./configure --with-ssl=openssl. Not sure if libssl.so.3 needs to be included in the bin/ dir - it hasn't been there before, so presumably previous versions had the SSL support built in?

EDIT: also needed libcrypto.so.3

I have a libssl.so.3 and libcrypto.so.3 from the Amazon Linux EC2 instance the binaries were built on, adding that to the bundled binaries used in our lambda made the "error while loading shared libaries" error go away, but it was replaced by glibc version incompatibilities:

2025-09-22T22:53:47.225Z	b2ccef6a-8a86-4178-91ca-d8d30c69794a	ERROR	Child process stderr: /var/task/bin/postgres-17.5/pg_restore: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by /var/task/bin/postgres-17.5/pg_restore)
/var/task/bin/postgres-17.5/pg_restore: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/task/bin/postgres-17.5/pg_restore)
/var/task/bin/postgres-17.5/pg_restore: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /var/task/bin/postgres-17.5/libpq.so.5)
/var/task/bin/postgres-17.5/pg_restore: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by /var/task/bin/postgres-17.5/libpq.so.5)
/var/task/bin/postgres-17.5/pg_restore: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /var/task/bin/postgres-17.5/libpq.so.5)
/var/task/bin/postgres-17.5/pg_restore: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by /var/task/bin/postgres-17.5/libssl.so.3)
/var/task/bin/postgres-17.5/pg_restore: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/task/bin/postgres-17.5/libssl.so.3)
/var/task/bin/postgres-17.5/pg_restore: /lib64/libc.so.6: version `GLIBC_2.33' not found (required by /var/task/bin/postgres-17.5/libcrypto.so.3)
/var/task/bin/postgres-17.5/pg_restore: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/task/bin/postgres-17.5/libcrypto.so.3)

Claude suggests (I haven't yet verified) that:

AWS Lambda currently runs on Amazon Linux 2 (which has glibc 2.26), but you likely compiled on Amazon Linux 2023 (which has glibc 2.34+).

https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html has a list of runtimes and versions to expect.

https://aws.amazon.com/blogs/compute/introducing-the-amazon-linux-2023-runtime-for-aws-lambda/ - the announcement of the Amazon Linux 2023 runtime for AWS Lambda it suggests it will be the runtime for Node 20+:

With the launch of provided.al2023 you can migrate your AL2 custom runtime-based Lambda functions right away. It also sets the foundation of future Lambda managed runtimes. The future releases of managed language runtimes such as Node.js 20, Python 3.12, Java 21, and .NET 8 are based on Amazon Linux 2023 and will use provided.al2023 as the base image.

So - the bundled binaries need to be considered carefully - building on an Amazon Linux 2023 AMI probably only makes sense if you know your lambda runner will be running it (because it's an OS-only runtime, or Node 20+ or other runtime that uses the provided.al2023 base image not the provided.al2 base image.

Now, the thought occurs: if Postgres (and dep libs) are built on the al2 AMI, will they work on both? Launching an EC2 instance, though,the only al2 options are "Amazon Linux 2 LTS with SQL Server 2019 Standard" and ...2017 - hmm. I had to find it via the marketplace and "subscribe" to the AMI.

Anyway - I built again on al2, and tried those binaries with our lambda, but still seeing the same issues with glib versions, unless I'm doing something stupid.

What should happen here? Bundle both Amazon Linux 2023 and Amazon Linux 2 versions of the binaries? Or will the al2 versions work on al2023 too?

As an aside: are there any licencing considerations for bundling the binaries here? Does there need to be a clear documentation that the Postgres and openssl binaries are provided for convenience and a link to the project repos to obtain source code (e.g. https://www.gnu.org/licenses/gpl-faq.html#SourceInCVS )?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PG 17.5
2 participants