Skip to content

fix: compute payload size correctly for pg_notify#2873

Merged
abelanger5 merged 1 commit intomainfrom
belanger/fix-pgnotify-payloads
Jan 31, 2026
Merged

fix: compute payload size correctly for pg_notify#2873
abelanger5 merged 1 commit intomainfrom
belanger/fix-pgnotify-payloads

Conversation

@abelanger5
Copy link
Copy Markdown
Contributor

Description

Users running in Postgres-only mode will occasionally see errors like the following:

ERROR: payload string too long (SQLSTATE 22023)

We have a check for 8000 bytes, but this check was failing to account for the extra marshaling we do when sending the message through pg_notify. We wrap the message in PubSubMessage which has some overhead with additional fields and queue names, and we're also base64-encoding the raw payload a second time, which adds overhead. As a result, even ~6kb payloads could occasionally see this behavior.

This fix does a few things:

  1. Removes the double base64-encoding by using json.RawMessage instead of []byte
  2. Only marshals/unmarshals this message once (instead of twice, we previously did the length check with a separate call to Marshal)
  3. Properly checks the size of the payload we're actually sending over the wire.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

@vercel
Copy link
Copy Markdown

vercel Bot commented Jan 28, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
hatchet-docs Ready Ready Preview, Comment Jan 28, 2026 4:16pm

Request Review

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes incorrect payload size calculations for PostgreSQL pg_notify by removing double base64-encoding, centralizing message wrapping, and enforcing the 8KB limit on the actual on-wire payload, with test coverage around the new behavior.

Changes:

  • Introduces multiplexedListener.wrapMessage to wrap a queue payload into PubSubMessage using json.RawMessage, eliminating the extra base64 layer and making size calculations reflect the true JSON sent over pg_notify.
  • Updates messageQueueRepository.Notify to use wrapMessage, enforce the 8000-byte limit on the wrapped payload, and fall back to database storage when exceeded; pubNonDurableMessages now delegates all size handling to Notify.
  • Adds tests around wrapped message sizing and empty payload handling in multiplexer_test.go, and changes PubSubMessage.Payload to json.RawMessage to align with the new encoding approach.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
pkg/repository/multiplexer_test.go Adds tests that exercise wrapMessage for various payload sizes and empty payload behavior, plus a helper to synthesize JSON payloads of a target size.
pkg/repository/multiplexer.go Extracts wrapMessage to build the PubSubMessage JSON using json.RawMessage, and simplifies notify to send pre-wrapped payloads.
pkg/repository/mq.go Changes PubSubMessage.Payload to json.RawMessage and updates Notify to use wrapMessage, enforce the 8KB pg_notify limit, and fall back to AddMessage when exceeded.
internal/msgqueue/postgres/exchange.go Simplifies non-durable publish path to always call repo.Notify, relying on the repository’s new size-aware logic instead of performing its own 8KB check.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/repository/mq.go
Comment on lines +19 to +20
QueueName string `json:"queue_name"`
Payload json.RawMessage `json:"payload"`
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing PubSubMessage.Payload from a []byte to json.RawMessage means existing code that constructs PubSubMessage values with a []byte payload (for example in multiplexer_test.go where Payload: []byte("test-payload") is used) will no longer compile. Those call sites need to be updated to construct a json.RawMessage (or otherwise pass a value of the correct type) to keep tests and any other callers building cleanly.

Suggested change
QueueName string `json:"queue_name"`
Payload json.RawMessage `json:"payload"`
QueueName string `json:"queue_name"`
Payload []byte `json:"payload"`

Copilot uses AI. Check for mistakes.
Comment on lines +360 to +365
// createJSONPayload creates a JSON string of exactly the specified size
func createJSONPayload(size int) string {
// Create a simple JSON object with a large string field
// The JSON structure will be: {"data":"<padding>"}
// Structure overhead: { (1) + "data" (6) + : (1) + " (1) + " (1) + } (1) = 11 bytes
const jsonStructureOverhead = 11
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment on createJSONPayload says it "creates a JSON string of exactly the specified size", but the implementation only approximates the size (e.g. it hardcodes jsonStructureOverhead and clamps paddingSize to >= 0), so for many inputs the resulting JSON length will differ from size. Either adjust the implementation to guarantee the exact length or relax the comment to describe that it produces a payload of approximately the requested size to avoid misleading future readers and test authors.

Copilot uses AI. Check for mistakes.
@abelanger5 abelanger5 merged commit 0495312 into main Jan 31, 2026
47 of 49 checks passed
@abelanger5 abelanger5 deleted the belanger/fix-pgnotify-payloads branch January 31, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants