Skip to content

Performance improvements for beam-postgres#797

Merged
LaurentRDC merged 2 commits intomasterfrom
from-backend-row-optimization
Apr 16, 2026
Merged

Performance improvements for beam-postgres#797
LaurentRDC merged 2 commits intomasterfrom
from-backend-row-optimization

Conversation

@LaurentRDC
Copy link
Copy Markdown
Member

This PR is about improving performance of beam-postgres to match postgresql-simple.

Fixes #769

@LaurentRDC
Copy link
Copy Markdown
Member Author

LaurentRDC commented Apr 15, 2026

The baseline is:

$ BEAM_BENCH_ROWS=100000 cabal -v0 bench beam-backend-bench:beam-postgres
Starting benchmark with 100000 rows
(set BEAM_BENCH_ROWS to override)
All
  SELECT * FROM imdb_names (100000 rows)
    postgresql-simple: OK
      249  ms ±  12 ms
    beam-postgres:     OK
      321  ms ± 8.8 ms

This is showing a 50% slowdown in beam-postgres vs postgresql-simple

@LaurentRDC
Copy link
Copy Markdown
Member Author

LaurentRDC commented Apr 16, 2026

After removing some redundant work from parsing every row:

$ BEAM_BENCH_ROWS=100000 cabal -v0 bench beam-backend-bench:beam-postgres
Starting benchmark with 100000 rows
(set BEAM_BENCH_ROWS to override)
All
  SELECT * FROM imdb_names (100000 rows)
    postgresql-simple: OK
      243  ms ±  13 ms
    beam-postgres:     OK
      263  ms ± 7.0 ms

beam-postgres is now showing a ~10% slowdown over postgresql-simple

@LaurentRDC
Copy link
Copy Markdown
Member Author

LaurentRDC commented Apr 16, 2026

I believe that the remaining slowdown is due to the use of free monads. This means that speeding up the beam-postgres benchmark would require a breaking change in beam-core, such that backends would need to adjust.

Thankfully, this shouldn't be too bad, as there is only one "third-party" backend released on Hackage (beam-mysql) which would need quite a bit of work to bring back to life anyways.

In any case, this will be left for further work

@LaurentRDC LaurentRDC added enhancement breaking change Change will require a major version bump labels Apr 16, 2026
@LaurentRDC LaurentRDC force-pushed the from-backend-row-optimization branch from a12c2f5 to 4f05df1 Compare April 16, 2026 20:18
@LaurentRDC LaurentRDC changed the base branch from master to benchmark-suites April 16, 2026 20:19
@LaurentRDC LaurentRDC force-pushed the from-backend-row-optimization branch from 4f05df1 to 3509a6c Compare April 16, 2026 20:34
@LaurentRDC LaurentRDC marked this pull request as ready for review April 16, 2026 20:34
Base automatically changed from benchmark-suites to master April 16, 2026 20:42
@LaurentRDC LaurentRDC removed the breaking change Change will require a major version bump label Apr 16, 2026
@LaurentRDC LaurentRDC force-pushed the from-backend-row-optimization branch from 3509a6c to d82f007 Compare April 16, 2026 20:44
@LaurentRDC
Copy link
Copy Markdown
Member Author

I am perplexed. The benchmarks on my local machine are showing that this PR closes the gap between beam-postgres and postgresql-simple. However, Github Actions are NOT showing these gains. I can't explain this

@LaurentRDC LaurentRDC merged commit 80bd37e into master Apr 16, 2026
13 checks passed
@LaurentRDC LaurentRDC deleted the from-backend-row-optimization branch April 16, 2026 21:03
LaurentRDC pushed a commit that referenced this pull request Apr 23, 2026
renderExecReturningList called cachedGetFields but then freed the result
via unsafeFreeResult, allowing libpq to reuse the pointer address on the
next Pg.exec. A subsequent query with different column types would get a
false cache hit and inherit stale field OID metadata, causing a
BeamRowReadError on column 0.

Fix: renderExecReturningList calls getFields directly. The cache remains
for stepProcess (CursorBatching), where it is safe and beneficial.

Regression test added to beam-postgres-tests.

Fixes #802, regression introduced in #797.

Co-authored-by: Christian Berg <christian.berg@kraftwerk.io>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Postgres de/serialization appears to be 2x slower than postgresql-simple

1 participant