Build packages for postgres extensions #90
Conversation
sjmiller609
changed the title
| @@ -0,0 +1,54 @@ | |||
| ARG UBUNTU_VERSION=22.04 | |||
There was a problem hiding this comment.
This is imitating this dockerfile, less the not applicable parts
https://github.com/postgresml/postgresml/blob/master/pgml-extension/Dockerfile
There was a problem hiding this comment.
The purpose of this file is to build an .deb (package) file for ubuntu with a PGX extension in it
There was a problem hiding this comment.
will all of our CoreDB postgres images be ubuntu based then, or just the images that contain extensions which we author?
There was a problem hiding this comment.
I think we have to pick one distro to do first. For example, postgres ML is only doing ubuntu or inside a container (or build from source): https://postgresml.org/user_guides/setup/v2/installation/#install-the-extension
| @@ -0,0 +1,29 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
similar to this script
https://github.com/postgresml/postgresml/blob/master/pgml-extension/build_extension.sh
There was a problem hiding this comment.
It uses a container to build the extension, and outputs to the local filesystem the .deb package
| @@ -1,7 +1,51 @@ | |||
| FROM postgres:15.1 | |||
| FROM ubuntu:22.04 | |||
There was a problem hiding this comment.
Since we built on ubuntu, the shared libraries don't work unless we also run on Ubuntu. For example, I had an issue before I switched this to Ubuntu with an error
postgres=# CREATE EXTENSION pgx_pgmq;
ERROR: could not load library "/usr/lib/postgresql/15/lib/pgx_pgmq.so": /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /usr/lib/postgresql/15/lib/pgx_pgmq.so)
And that's because the distro has different versions of system software available.
We kind of have to pick one distro to support first.
| @@ -0,0 +1,340 @@ | |||
| #!/usr/bin/env bash | |||
There was a problem hiding this comment.
This is copy/pasted from here https://github.com/docker-library/postgres/blob/master/docker-entrypoint.sh
| - name: Run pre-build hooks | ||
| shell: bash | ||
| run: | | ||
| cd ${{ inputs.docker_directory }} | ||
| if [[ -f pre-build-hook.sh ]]; then | ||
| echo "detected pre-build hook, running" | ||
| /bin/bash pre-build-hook.sh | ||
| else | ||
| echo "no pre build hook detected" | ||
| fi |
There was a problem hiding this comment.
To get the extension installed into the postgres container, we have a script that builds the extension, and puts the package into a directory "extensions" in the postgres directory before doing the build of the postgres image.
I think we ought to instead publish our packages and just pull them in in the docker build over the internet from our repository instead of doing this
There was a problem hiding this comment.
publish the pgx extension as a crate, then pull from crates.io and install?
as opposed to cloning the repo, then pushing to crates.io and quay.io at same time (or in sequence)?
There was a problem hiding this comment.
I mean publish to an APT repository the .deb package. Seems to me like that's the best way to package and distribute extensions.
There was a problem hiding this comment.
Like I mean i'm imagining our workflow will build and publish to our APT repository all our extensions (and I think we don't need to publish extensions to crates.io). So that's another artifact type.
| @@ -0,0 +1,53 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
| @@ -180,7 +180,7 @@ TODO: | |||
| Delete a message with id `1` from queue named `my_queue`. | |||
| ```sql | |||
| pgmq=# select pgmq_delete('my_queue', 1); | |||
| pgmq_delete | |||
| pgmq_delete | |||
There was a problem hiding this comment.
my editor is automatically removing trailing whitespaces
* remove sleep on success Can we remove this? I dont think we need to sleep after successfully processing messaging, or does this serve some other purpose? * remove unused code
I'm looking at how postgres ML is doing packaging:
They are building with pgx inside docker and then copying a
.debpackage out from the container, which can be installed on ubuntu.At the moment, we do not have an APT repository. I think we ought to build and upload our extensions to an APT repository, then install from there into our postgres image, that way our extensions and other extensions are being installed in the same way. For this PR, we are just building the extension before doing the docker build of postgres, then using COPY in the dockerfile to copy the .deb into our postgres image and install it.