[Bug]

[vyrodovalexey]

Apache Cloudberry and cloudberry-backup version

2.1.0

What happened

gpbackup crashes Cloudberry segment backend (exit 255) on segment-dispatched commands

What you think should happen instead

maybe to write dedicated lib to work with cloudberry/greenplum

How to reproduce

Reproduction

Against the running cb-prod cluster (HA coordinator + 2 primary segments + 2 mirrors), with mydb containing distributed tables:

gpbackup --dbname mydb --backup-dir /tmp/gpb --jobs 1 --no-history

Result (reproducible 100% of the time, even on a freshly created 1000-row database, independent of --jobs, --no-history, --single-data-file, or the S3 plugin):

gpbackup ... [INFO]:-Starting backup of database mydb
gpbackup ... [CRITICAL]:-ERROR: Error on receive from seg0 10.1.10.236:5432 pid=3646:
  server closed the connection unexpectedly
  This probably means the server terminated abnormally
  before or while processing the request. (SQLSTATE 58M01)
EXIT=2

Evidence from the segment log (gpseg0/log)

... "LOG","00000","server process (PID 3655) exited with exit code 255", ...,"postmaster.c",4264
... "LOG","00000","terminating any other active server processes", ...,"postmaster.c",4000
... "LOG","00000","all server processes terminated; reinitializing", ...,"postmaster.c",4576
... "LOG","00000","database system was not properly shut down; automatic recovery in progress"
... "LOG","00000","database system is ready"

Exit code 255 with "terminating any other active server processes" + "reinitializing" is the postmaster's response to a backend that died abnormally (not a clean ERROR return) — a crash of the segment backend, not a SQL-level error.

The crash trigger sequence (what gpbackup does)

The coordinator log pinpoints the failing statement and the session state around it:

seg0   ... "SET transaction_isolation TO 'serializable'"   (worker sessions; CB falls back to repeatable read)
seg0   ... server process (PID 3655) exited with exit code 255
seg-1 (coordinator) ... "ERROR","58M01","Error on receive from seg0 ... server closed the connection unexpectedly"
seg-1 (coordinator) ... "An exception was encountered during the execution of statement:
                         SELECT pg_size_pretty(pg_database_size('mydb')) as dbsize"

gpbackup opens its connection, sets the session to serializable (Cloudberry logs "serializable isolation requested, falling back to repeatable read until serializable is supported"), exports a distributed snapshot, and then dispatches a series of commands to segments. The first to crash a segment was the informational pg_database_size query. After patching that out (it's cosmetic — only used for the report's "database size:" line), the crash simply moved to the next segment-dispatched command, and so on through:

• pg_database_size('mydb') — report DB-size query
• LOCK TABLE ... ON SEGMENT
• SAVEPOINT
• COPY <table> TO PROGRAM ... ON SEGMENT — the core MPP data-backup operation

The discriminating fact (root cause isolation)

The same statements that crash the segment from gpbackup succeed from psql:

Statement	From psql (libpq / C)	From gpbackup (Go database/sql, pgx & lib/pq)
SELECT pg_database_size('mydb')	✅ returns 309 MB	❌ crashes segment (exit 255)
SELECT pg_database_size(oid) FROM pg_database	✅ returns values for all DBs	—
BEGIN ISOLATION LEVEL SERIALIZABLE; SELECT pg_database_size('mydb')	✅ works	—
COPY ... TO PROGRAM ON SEGMENT	✅ (psql path)	❌ crashes segment

So the crash correlates with the client driver / wire-protocol path, not the SQL text or the isolation level alone. The Go database/sql drivers send the extended query protocol / specific bind-parameter and snapshot-synchronization patterns that this Cloudberry build's segment backend mishandles, leading to the backend dying instead of returning an error.

Operating System

rockylinux:9.6

Anything else

No response

Are you willing to submit PR?

• Yes, I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct.