v0.5.6

This is pgx v0.5.6. It is a small bugfix release that includes the fix from PR #807.

Please cargo install --locked cargo-pgx and update your Cargo.toml:

pgx = "0.5.6"

Fixed Bugs

• Functions using the name!() macro in their return type (returning a TableIterator or (rust, tuple)) can now use Rust reserved keywords as the first "column name" argument. This was a regression introduced in 0.5.3.

Full Changelog: v0.5.4...v0.5.6

v0.5.4

pgx v0.5.4 is out, please cargo install --locked cargo-pgx and update your Cargo.toml:

pgx = "0.5.4"

This is a fairly small release, containing a bugfix and some new API surface:

• An incorrect handling of some Datum arguments fixed thanks to @eeeebbbbrrrr in #795
• Some missed SpinLock macro bindings were added thanks to @thomcc in #793
• A new base API for dynamic background worker registration was introduced thanks to @yrashk in #710!

We're excited to see further improvements to the Dynamic Background Worker API and are looking forward to any feedback you have!

Full Changelog: v0.5.3...v0.5.4

v0.5.3

What was presumed to be a no-op change after testing on multiple operating systems nonetheless broke the build on Red Hat distros. We're looking into adding another distro (presumably one that is fond of fedoras?) in our CI just to prevent this from happening again and allow us to better anticipate repackages of Postgres with a nonzero number of quirks. This release primarily is to fix that by removing allowlist_* from our bindgen invocation for now, thanks to @BradyBonnette in #760.

It also fixes a use-after-free that PGX was inducing in Postgres: Datum arguments were being unboxed (into... new boxes, usually) during set-returning functions, but not using the multi-call memory context. This was fixed thanks to @eeeebbbbrrrr in #784

However, it does have a number of other changes.

Less unnecessary dynamic linkage

Some of the dynamic linkage PGX was asking for was largely unnecessary. In new PGX projects, you should now be able to use exotic linkers which don't accept various command line arguments other linkers do... on Linux, as the dynamic lookup argument remains necessary on MacOS (for now), thanks to "Only specify -undefined dynamic_lookup on macOS" by @thomcc in #754
To get this benefit in an existing project, replace the existing [build] key in your ./.cargo/config{,.toml} with

[target.'cfg(target_os="macos")']
# Postgres symbols won't be available until runtime
rustflags = ["-Clink-arg=-Wl,-undefined,dynamic_lookup"]

cargo-pgx improvements

cargo pgx init now accepts the flags --base-port and --base-testing-port, thanks to "Problem: working with independent extensions simultaneously" by @yrashk in #765. These values can be later changed in $PGX_HOME/config.toml. This is intended to make it easier to work with multiple extensions at once.

We now support --profile <profile name> in cargo-pgx thanks to @thomcc in #758, to allow you to better tune the testing and packaging profiles you may want to use.

New pg_sys bindings

• Enable low-level access to the database catalog by @yrashk in #709
• Added includes for "storage/spin.h" by @osawyerr in #775

Other minor fixups

• We've stopped distorting the signatures of variadic functions thanks to @thomcc in #757
• The --bgworker template was found to have fallen out of sync and was patched by @workingjubilee in #768
• Remove lifetimes from FunctionMetadata impl by @thomcc in #772
• derive Debug for PgLogLevel and PgSqlErrorCode by @yrashk in #786
• Make syntect an optional dependency, only used by cargo-pgx by @thomcc in #755
• Avoid some dependencies in pgx-pg-sys by @thomcc in #756

New Contributors

• @osawyerr made their first contribution in #775

Full Changelog: v0.5.2...v0.5.3

v0.5.2

This is pgx v0.5.2. It is a very minor release that fixes issue #762.

Full Changelog: v0.5.0...v0.5.2

v0.5.0

This is the release of pgx 0.5.0! Special thanks to @BradyBonnette, @JamesGuthrie, @Smittyvb, and @willmurnane for contributing fixes for bugs that we noticed in the last round of testing for 0.5.0-beta.1. Most of these release notes will be a reprise of those if you have seen them already. For everyone else, welcome aboard!

With a rustup or cargo toolchain installed, you can download pgx from crates.io, just cargo install --locked cargo-pgx --version 0.5.0 and add this to your Cargo.toml:

pgx = "0.5.0"

API documentation is available on docs.rs.

For newer users

Code written using pgx can now use pgx::prelude::*; instead of use pgx::*; and in many cases this will make the extension compile with only a few additional explicit imports. Existing extensions may require extra effort and may want to approach this more incrementally.

Upgrading

Most of the transition should be smooth, except for certain extensions. You will need to reinstall cargo pgx first: everything regarding SQL generation changed internally, and cargo pgx is the binary that handles most of the actual SQL compilation. We now recommend using cargo install --locked cargo-pgx --version 0.5.0 for this.

There are four major pain points you may immediately encounter: code that directly handles datums, using FromDatum, date/time types, and set-returning or table-returning functions. But after that compiles, you will also want to address any soundness or behavior questions that might arise regarding Arrays in your code.

Handling Datums

Since 0.5.0-beta.0, pg_sys::Datum is no longer type Datum = usize;, but a "newtype": a struct wrapping a pointer that indicates it is an opaque type. This is for correctness and to better represent the unique nature of the type with respect to Postgres. A fair amount of your code may have relied on T as pg_sys::Datum or datum as T working. Certainly some in pgx did, resulting in patterns like:

// generating fresh arrays for data
let mut datums: Vec<pg_sys::Datum> = vec![0; N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum as i32;

// treating a datum as a pointer type
let varlena = datum as *mut pg_sys::varlena;

Now it doesn't! To make the transition easy, From<T> for pg_sys::Datum is implemented for everything it is reasonable to convert directly into it in a "by-value" fashion. Use IntoDatum for anything more complex. To get the usize hiding inside Datum out, use Datum::value(self) -> usize, to cast it as a pointer, Datum::cast_mut_ptr::<T>(self) -> *mut T is available.

// (getting|putting) data (from|into) Postgres as a datum array
let mut datums = vec![pg_sys::Datum::from(0); N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum.value() as i32;

// treating a datum as a pointer type
let varlena = datum.cast_mut_ptr::<pg_sys::varlena>();

Because a pg_sys::Datum is a union of types, only certain traits that are implemented on usize are also implemented on Datum, as it was deemed safest to limit implementations to those that are also valid if Datum is a pointer or float. This can induce code to reveal any assumptions it was making.

If you were an early upgrader to 0.5.0-beta.0: First, thanks for trying things out! Second, you may have used Datum::ptr_cast: it has the new name of cast_mut_ptr. Datum::to_void was simply dropped as it is unlikely to be what you want, and if it is, cast_mut_ptr::<c_void> works.

FromDatum::from_polymorphic_datum

FromDatum::from_polymorphic_datum now holds the behavior of FromDatum::from_datum and a new FromDatum::from_datum which only accepts 2 arguments (omitting the pg_sys::Oid argument) has been added. By default, the new from_datum calls from_polymorphic_datum with pg_sys::InvalidOid as the third argument. Most people use the provided implementations, but you should override this if you have a custom impl of FromDatum for a polymorphic-like type like AnyElement, or ignore the third parameter in your implementation of FromDatum::from_polymorphic_datum if you don't.

Otherwise, you should be fine with a find-replace for the most part.

Date/Time types

Previously, pgx leaned on the excellent support for dates and times provided by the time crate. Unfortunately, that also meant that we ran into problems like "not representing all time values that Postgres understands". It also meant using heavy conversions when touching these types, even to just copy them out of Postgres, resulting in large amounts of switching between Rust and Postgres during control flow for e.g. constructing a Vec<Timestamp> from a pg_sys::ArrayType that was a handle to a varlena of pg_sys::Timestamp. To allow fixing these performance and correctness problems, pgx now implements these types in terms of the Postgres representations in Rust code.

This also means that all the {Date,Time,Timestamp}{,WithTimeZone}::new functions are now deprecated.

For easy transition, {,Try}{Into,From} for types in the time crate are available with pgx { version = "0.5.0.beta-1", features = ["time-crate"] }.

With thanks to @mhov, @workingjubilee, and @jsaied99's contributions:

• Opaque Date type with +/- infinity support by @mhov in #622
• Opaque Timestamp/Tz types by @mhov in #643
• Match the Postgres repr in Time and TimeWithTimeZone by @workingjubilee in #677
• Adding Clone to Timestamp and Date types by @jsaied99 in #700

Set/Table-returning functions

Functions that previously would be RETURNS SET OF or RETURNS TABLE used to return impl Iterator. Unfortunately, not being able to name the type of the iterator greatly complicated pgx 0.5.0's redesign for SQL generation lead by @Hoverbear and @workingjubilee. pgx::iter::SetOfIterator and pgx::iter::TableIterator now need to be used instead to return these values. In all existing cases you should be able to simply wrap the previous expression in {SetOf,Table}Iterator::new, but for cases where you are returning a single row, TableIterator::once makes for a sweeter alternative.

For example, this:

#[pg_extern]
fn fluffy() -> impl Iterator<Item = i32> {
   vec![1,2,3].into_iter()
}

Becomes this:

#[pg_extern]
fn fluffy() -> SetOfIterator<'static, i32> {
   SetOfIterator::new(vec![1,2,3].into_iter())
}

Array soundness issues

There were several soundness issues regarding interactions between the several ways of constructing pgx::datum::Array<'a, T>, Array::as_slice, and impl<'a, T> Drop for Array<'a, T>. While the critical soundness issue for Drop has been fixed and the resulting leak plugged, these have resulted in the deprecation ofArray::over and Array::as_slice.

Array::as_slice allowed wildly incorrect slices, or viewing data Postgres had marked as "null" in the SQL sense. Instead, this function now panics if either of these conditions would be met, or returns a newly-correct slice for simple data types. Because the usually-trivial fn as_slice panicking is rarely expected, this function is deprecated.

Array::over allows constructing improper Arrays from pointers to random data that is probably controlled by Rust and not Postgres, and thus is data in a form Rust prefers, when Array is supposed to be a zero-copy type that handles ArrayType data in the form Postgres prefers. This makes it almost impossible for the underlying type to be useful for both cases without severely impacting correctness and performance.

Both expose implications about the way that Postgres represents data that are not always true. If you still want to interact with the underlying representation of a Postgres Array and you know the Array was correctly constructed, consider the new experimental pgx::array::RawArray type, created with support from bitvec. It is possible pgx::datum::Array<'a, T> may be replaced entirely in the future by a new type that exposes less assumptions about internal repr.

New features and fixes

In general, PGX added significant improvements for working with uncommon use-cases or existing configurations, with thanks to @steve-chavez, @anth0nyleung, and @TimeToogo.

We also moved some code into a new crate: pgx-pg-config.

Fixes in cargo pgx and pgx-pg-sys

• Hang onto libraries by @Hoverbear in #573
  A cargo pgx bug that could cause segfaults on old versions of glibc was fixed by no longer using dlclose on the libraries we dynamically load until the process ends.

We also found and fixed some unsound behavior in the way PGX handled symbols on MacOS thanks to @Smittyvb. T...

v0.5.0-beta.1

This is a pilot release for 0.5.0 and no further functional changes are intended from now until the 0.5.0 release. Bikeshedding, renaming, and purely architectural/internal rearrangements may still occur, but the only things anyone should have to change would be within the scope of simple find-replace rules. Bugfixes and other non-functional changes will be merged. We do not intend to merge any additional new features.

For newer users

Code written using pgx can now use pgx::prelude::*; instead of use pgx::*; and in many cases this will make the extension compile with only a few additional explicit imports. Existing extensions may require extra effort and may want to approach this more incrementally.

Upgrading

Most of the transition should be smooth, except for certain extensions. You will need to reinstall cargo pgx first: everything regarding SQL generation changed internally, and cargo pgx is the binary that handles most of the actual SQL compilation. We now recommend using cargo install --locked cargo-pgx --version 0.5.0-beta.1 for this.

There are three major pain points you may immediately encounter: code that directly handles datums, date/time types, and set-returning or table-returning functions. But after that compiles, you will also want to address any soundness or behavior questions that might arise regarding Arrays in your code.

Handling Datums

Since 0.5.0-beta.0, pg_sys::Datum is no longer type Datum = usize;, but a "newtype": a struct wrapping a pointer that indicates it is an opaque type. This is for correctness and to better represent the unique nature of the type with respect to Postgres. A fair amount of your code may have relied on T as pg_sys::Datum or datum as T working. Certainly some in pgx did, resulting in patterns like:

// generating fresh arrays for data
let mut datums: Vec<pg_sys::Datum> = vec![0; N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum as i32;

// treating a datum as a pointer type
let varlena = datum as *mut pg_sys::varlena;

Now it doesn't! To make the transition easy, From<T> for pg_sys::Datum is implemented for everything it is reasonable to convert directly into it in a "by-value" fashion. Use IntoDatum for anything more complex. To get the usize hiding inside Datum out, use Datum::value(self) -> usize, to cast it as a pointer, Datum::cast_mut_ptr::<T>(self) -> *mut T is available.

// (getting|putting) data (from|into) Postgres as a datum array
let mut datums = vec![pg_sys::Datum::from(0); N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum.value() as i32;

// treating a datum as a pointer type
let varlena = datum.cast_mut_ptr::<pg_sys::varlena>();

Because a pg_sys::Datum is a union of types, only certain traits that are implemented on usize are also implemented on Datum, as it was deemed safest to limit implementations to those that are also valid if Datum is a pointer or float. This can induce code to reveal any assumptions it was making.

If you were an early upgrader to 0.5.0-beta.0: First, thanks for trying things out! Second, you may have used Datum::ptr_cast: it has the new name of cast_mut_ptr. Datum::to_void was simply dropped as it is unlikely to be what you want, and if it is, cast_mut_ptr::<c_void> works.

Date/Time types

Previously, pgx leaned on the excellent support for dates and times provided by the time crate. Unfortunately, that also meant that we ran into problems like "not representing all time values that Postgres understands". It also meant using heavy conversions when touching these types, even to just copy them out of Postgres, resulting in large amounts of switching between Rust and Postgres during control flow for e.g. constructing a Vec<Timestamp> from a pg_sys::ArrayType that was a handle to a varlena of pg_sys::Timestamp. To allow fixing these performance and correctness problems, pgx now implements these types in terms of the Postgres representations in Rust code.

This also means that all the {Date,Time,Timestamp}{,WithTimeZone}::new functions are now deprecated.

For easy transition, {,Try}{Into,From} for types in the time crate are available with pgx { version = "0.5.0.beta-1", features = ["time-crate"] }.

With thanks to @mhov, @workingjubilee, and @jsaied99's contributions:

• Opaque Date type with +/- infinity support by @mhov in #622
• Opaque Timestamp/Tz types by @mhov in #643
• Match the Postgres repr in Time and TimeWithTimeZone by @workingjubilee in #677
• Adding Clone to Timestamp and Date types by @jsaied99 in #700

Set/Table-returning functions

Functions that previously would be RETURNS SET OF or RETURNS TABLE used to return impl Iterator. Unfortunately, not being able to name the type of the iterator greatly complicated pgx 0.5.0's redesign for SQL generation lead by @Hoverbear and @workingjubilee. pgx::iter::SetOfIterator and pgx::iter::TableIterator now need to be used instead to return these values. In all existing cases you should be able to simply wrap the previous expression in {SetOf,Table}Iterator::new, but for cases where you are returning a single row, TableIterator::once makes for a sweeter alternative.

For example, this:

#[pg_extern]
fn fluffy() -> impl Iterator<Item = i32> {
   vec![1,2,3].into_iter()
}

Becomes this:

#[pg_extern]
fn fluffy() -> SetOfIterator<'static, i32> {
   SetOfIterator::new(vec![1,2,3].into_iter())
}

Array soundness issues

There were several soundness issues regarding interactions between the several ways of constructing pgx::datum::Array<'a, T>, Array::as_slice, and impl<'a, T> Drop for Array<'a, T>. While the critical soundness issue for Drop has been fixed and the resulting leak plugged, these have resulted in the deprecation ofArray::over and Array::as_slice.

Array::as_slice allowed wildly incorrect slices, or viewing data Postgres had marked as "null" in the SQL sense. Instead, this function now panics if either of these conditions would be met, or returns a newly-correct slice for simple data types. Because the usually-trivial fn as_slice panicking is rarely expected, this function is deprecated.

Array::over allows constructing improper Arrays from pointers to random data that is probably controlled by Rust and not Postgres, and thus is data in a form Rust prefers, when Array is supposed to be a zero-copy type that handles ArrayType data in the form Postgres prefers. This makes it almost impossible for the underlying type to be useful for both cases without severely impacting correctness and performance.

Both expose implications about the way that Postgres represents data that are not always true. If you still want to interact with the underlying representation of a Postgres Array and you know the Array was correctly constructed, consider the new experimental pgx::array::RawArray type, created with support from bitvec. It is possible pgx::datum::Array<'a, T> may be replaced entirely in the future by a new type that exposes less assumptions about internal repr.

New features and fixes

In general, PGX added significant improvements for working with uncommon use-cases or existing configurations, with thanks to @steve-chavez, @anth0nyleung, and @TimeToogo.

This includes a soundness fix for the way PGX handled symbols on MacOS thanks to @Smittyvb. This should fix a number of random crashes users were experiencing, but we can't tell for sure (by definition: UB!). Please give it a try and let us know if there are any issues remaining.

We also moved some code into a new crate: pgx-pg-config.

Better soundness

@willmurnane contributed several soundness and correctness improvements, allowing input functions to validate or reject input (now accepting Options).

BackgroundWorker functions may now panic if used from unregistered workers. Thanks to @EdMcBane for catching the previously unsound behavior.

Optional CREATE OR REPLACE FUNCTION

In 0.4.5, PGX switched to using CREATE FUNCTION by default, as this is nondestructive and more secure. However, that prevented migration beyond 0.4.4 for some users, [as they relied on CREATE OR ...

v0.5.0-beta.0

This is an experimental version for the upcoming 0.5.0 release.
Make sure to run cargo install --locked cargo-pgx --version 0.5.0-beta.0.

Handling Datums

pg_sys::Datum is no longer type Datum = usize;, but a "newtype": a struct wrapping a pointer that indicates it is an opaque type. This is for correctness and to better represent the unique nature of the type with respect to Postgres. A fair amount of your code may have relied on T as pg_sys::Datum or datum as T working. Certainly some in pgx did, resulting in patterns like:

// generating fresh arrays for data
let mut datums: Vec<pg_sys::Datum> = vec![0; N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum as i32;

// treating a datum as a pointer type
let varlena = datum as *mut pg_sys::varlena;

Now it doesn't! To make the transition easy, From<T> for pg_sys::Datum is implemented for everything it is reasonable to convert directly into it in a "by-value" fashion. Use IntoDatum for anything more complex. To get the usize hiding inside Datum out, use Datum::value(self) -> usize, to cast it as a pointer, Datum::ptr_cast::<T>(self) -> *mut T is available.

// (getting|putting) data (from|into) Postgres as a datum array
let mut datums = vec![pg_sys::Datum::from(0); N];
let mut nulls = vec![false; N];

// treating a datum as a pass-by-value type
let index = datum.value() as i32;

// treating a datum as a pointer type
let varlena = datum.ptr_cast::<pg_sys::varlena>();

Because a pg_sys::Datum is a union of types, only certain traits that are implemented on usize are also implemented on Datum, as it was deemed safest to limit implementations to those that are also valid if Datum is a pointer or float. This can induce code to reveal any assumptions it was making.

Enhancements

Considerable enhancements have been made to heap tuple and trigger support, introducing pg_trigger and a new PgHeapTuple type! Try them out!

• Improved "HeapTuple" support by @eeeebbbbrrrr in #532
• Trigger support improvements by @Hoverbear in #558

Fixes

• Hang onto libraries by @Hoverbear in #573
  A cargo pgx bug that could cause segfaults on old versions of glibc was fixed by no longer using dlclose on the libraries we dynamically load until the process ends.

Experimental features

You may have noticed the "postgrestd" feature. This is a highly experimental feature for using PGX to back a very special configuration of Rust as a trusted procedural language: PL/Rust. It's best to assume this feature can induce unsound changes if they aren't combined with the rest of the configuration for the trusted language handler's building and running of functions.

Full Changelog: v0.4.5...v0.5.0-beta.0

v0.4.5

This is pgx v0.4.5 It's generally a minor release without backwards incompatible changes.

As usual, make sure to run cargo install cargo-pgx.

What's Changed

• Lockdown semver on pgx deps by @Hoverbear in #559
• Annotate and refactor pgx-tests by @workingjubilee in #563
• Clone less while beginning to unwind by @workingjubilee in #564
• Do not use CREATE OR REPLACE FUNCTION by @Hoverbear in #554
• Remove dubious thread assertion by @workingjubilee in #562
• Pre-patch fixes by @workingjubilee in #568
• Bump to v0.4.5 by @workingjubilee in #569

Full Changelog: v0.4.4...v0.4.5

v0.4.4

This is pgx v0.4.4. A minor release which adds support for "versioned .so" files.

Notable Changes

• Optionally emit versioned .so by @JamesGuthrie in #546
• Add impl Debug for Numeric by @Hoverbear in #551
• Fix namespace.rs safety issues by @eeeebbbbrrrr in #553
• Add "parser/parse_oper.h" to includes by @JamesGuthrie in #549

Minor Changes

• Fix spelling mistakes by @kiwicopple in #548
• Fix url typo on Detailed cargo pgx usage section by @msAlcantara in #550
• Add aggregate article by @Hoverbear in #555

New Contributors

• @kiwicopple made their first contribution in #548
• @msAlcantara made their first contribution in #550

Full Changelog: v0.4.3...v0.4.4

v0.4.3

This is pgx v0.4.3. It brings a number of new code-level features, faster compilation, more PostgreSQL header bindings, and other fun stuff!

When upgrading, please ensure to run cargo install cargo-pgx and to also update your crate dependencies to reference 0.4.3.

What's Changed

General Compilation and Bindings

• #[pg_guard] cleanup by @eeeebbbbrrrr in #526
  • This significantly improves compilation time of the pgx-pg-sys crate
• Add pg_operator.h by @einarmo in #512
• Add support for pg_authid by @nikkhils in #527

cargo-pgx changes

• Remove dead --workspace flag on cargo pgx test by @Hoverbear in #520
• Support --pgcli by @Hoverbear in #538 (see cargo-pgx docs
• Pass RUSTFLAGS to rustc on schema generate by @Hoverbear in #513

Source-code Additions

• Add Copy,Clone for pgx::Timestamp* types by @jamessewell in #509
• Add From<Timestamp> impls by @jamessewell in #510
• Get TimestampWithTimeZone from OffsetDateTime by @einarmo in #539
• Allow compilation on OpenBSD by @ledeuns in #359

Misc

• Updates Github Action runners for PGX by @BradyBonnette in #541
• update copyright headers by @eeeebbbbrrrr in #511

New Contributors

• @BradyBonnette made their first contribution in #541
• @ledeuns made their first contribution in #359

Full Changelog: v0.4.2...v0.4.3