does Neon support Postgres plugins?

[timharsch]

Specifically, I'm hoping it might support ZomboDB. But also interested in others, even if it is through some trusted plugin approach similar to what RDS does.

[hlinnaka]

We plan to allow running custom docker images in the compute node, where you can install any extension or even modified version of PostgreSQL itself. We're not there yet; we need to figure out the user interface and all the plumbing to support that. But the architecture definitely allows it. The only exception would be extensions that store data directly on disk, without using the PostgreSQL WAL-logging facilities, but those are rare.

Looking quickly at ZomboDB, it relies on an external ElasticSearch cluster, but on the Postgres side I don't see anything special. So yeah, I think we can make that work.

[mingfang]

I am also trying to get ZomboDB to work in Neon.
I added the code below to https://github.com/neondatabase/neon/blob/main/Dockerfile.compute-node

#########################################################################################
#
# Layer "zombodb-build"
# compile zombodb extension
#
#########################################################################################
FROM build-deps AS zombodb-build
COPY --from=pg-build /usr/local/pgsql/ /usr/local/pgsql/

RUN apt update && \
    apt install -y bison flex zlib1g zlib1g-dev pkg-config make libssl-dev libreadline-dev \
    gcc build-essential libz-dev strace clang git

# hack permissions for build
RUN chmod -R 777  /usr/local/pgsql

ARG USER=docker
ARG UID=1000
ARG GID=1000
RUN useradd -m ${USER} --uid=${UID}
USER ${UID}:${GID}
WORKDIR /home/${USER}

# install rust and cargo

RUN wget -O - https://sh.rustup.rs | bash -s -- -y
ENV PATH="/home/${USER}/.cargo/bin:/usr/local/pgsql/bin:${PATH}"

RUN cargo install cargo-pgx --version 0.6.0
RUN cargo pgx init --pg15=/usr/local/pgsql/bin/pg_config

# build zombodb

RUN git clone --depth 1 https://github.com/zombodb/zombodb.git
RUN cd zombodb* && \
    cargo pgx install --release

The build was successful but when I tried to enable the extension, I'm getting this error.

postgres=# CREATE EXTENSION zombodb;
ERROR:  incompatible library "/usr/local/lib/zombodb.so": ABI mismatch
DETAIL:  Server has ABI "Neon Postgres", library has "PostgreSQL".

Any help would be appreciated.

[mingfang]

I was able to hack around the problem by changing
https://github.com/neondatabase/postgres/blob/33f976345490351f951d72f81621c2263c186c9a/src/include/pg_config_manual.h#L60
from Neon Postgres to PostgreSQL. This is definitely not the right way, but it works.

I can confirm that the ZomboDB extension does work with Neon.

@hlinnaka What is the correct way to fix the ABI mismatch error?

[hlinnaka]

We've changed the ABI string to enforce that extensions are built against the neon-patched version of Postgres. Out of caution, really. I don't actually think we've made any changes to the ABI that extensions use, but we haven't really done the analysis or committed to keeping the ABI unchanged.

I recommend compiling zombodb against the headers from neon postgres installation, in pg_install/v15. So replace the 'cargo pgx init' step with something like:

RUN cargo pgx init --pg15=/.../neon/pg_install/v15/bin/pg_config

[mingfang]

@hlinnaka Thanks for the explanation. I will try your recommendation.

[eeeebbbbrrrr]

Hi! ZomboDB and pgx developer here.

We've put up a PR at pgcentralfoundation/pgrx#1033 to address this, which was prompted by pgcentralfoundation/pgrx#1032.

There's quite a bit of discussion there around this ABI string being different. Our next pgx release (and then ZDB release based on that) will no longer hardcode "PostgreSQL", but that won't mean pgx will be properly compatible with Neon, especially if Neon changes anything that breaks pgx' assumptions.

[tomdavidson]

@hlinnaka can you comment on the bundled or blessed plug-ins that are included?

[vadim2404]

Here is the list of supported extensions.