Proposal: Conflict log history table for Logical Replication

[pg-hub-mirror[bot]]

Read-only mirror. Join the conversation on pgsql-hackers; activity here is not sent upstream.

• Original author: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
• Mailing list: pgsql-hackers
• Message-ID: CAFiTN-sJmqGSD92b8fng0uCW_9gp=tHj+uBatnkm4xyBpujauA@mail.gmail.com
• Original email

---

On Tue, Sep 1, 2026 at 4:03 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:

Here is detailed analysis and summary of the problem and all the
alternatives we tried. The problem is that converting a
TupleTableSlot to JSON can result in rendered JSON text exceeding
PostgreSQL's MaxAllocSize limit (1 GB). When that happens, memory
allocation (palloc() or enlargeStringInfo()) throws a hard ERROR.
This issue is not unique to conflict logging:

1. pgoutput does not handle this: if a row's formatted textual
   representation exceeds 1 GB in logicalrep_write_tuple(),
   OidOutputFunctionCall() or palloc() fails with an unrecoverable hard
   error.
2. For conflict logging, converting to JSON exacerbates the problem
   because a) every byte below 0x20 expands into a 6-byte \uXXXX
   sequence. b) JSONB values (e.g., repeated numeric expansions like
   1e131071), arrays, and composites can be compact on disk but expand to
   hundreds of megabytes or gigabytes when serialized to text. c)
   user-Defined Types (UDTs): A compact binary UDT (e.g., a run-length
   encoded vector) can expand into gigabytes inside its typoutput
   function.
3. Other conflict logging extensions (e.g., pgactive and pgEdge) also
   have this issue, and none of them handle it.
   Because storing multi-gigabyte or hundreds-of-megabytes values in a
   conflict log table is neither practical nor desirable for post-mortem
   debugging, we have been exploring several options to gracefully handle
   oversized attributes without erroring out. Below is a summary of the
   options explored, along with their pros and cons.
   Option 1: Attribute-Level Size Capping via ErrorSaveContext (escontext)
   Enforce a reasonable threshold (e.g., 16 KB) per attribute during JSON
   conversion. Pass an ErrorSaveContext to an extended serialization
   function (datum_to_json_extended()). As serialization recurses through
   arrays, composites, or JSONB containers, cumulative size is monitored.
   If an attribute exceeds the limit, errsave() records the soft error
   and returns (Datum) 0. The caller detects
   SOFT_ERROR_OCCURRED(&escontext) and cleanly replaces that attribute in
   the outer JSON tuple with an omission object: {"omitted": true,
   "length": ...}.
   Pros:

• Retains the full tuple structure and all other normal-sized
  columns for analysis.
• Cleanly handles unbounded recursion in nested structures (arrays,
  composites, and JSONB) by aborting early.
  Cons:
• Does not protect against typoutput failures: standard PostgreSQL
  output functions invoked via OidOutputFunctionCall() do not accept an
  ErrorSaveContext parameter.
• If a UDT (or built-in bytea with a large payload in hex format)
  exceeds 1 GB inside its typoutput function, palloc() throws a hard
  ERROR before size checks run; escontext cannot intercept it.
  Option 2: Record Only Replica Identity (RI) Columns Instead of Full Tuples
  Instead of serializing entire remote and local tuples to JSON, only
  serialize the Replica Identity key columns (typically Primary Key or
  Unique Index attributes) identifying the conflicting row.
  Pros:
• Drastically reduces conflict log storage overhead and serialization cost.
• For standard integer, UUID, or short text keys, the payload is
  tiny and never approaches memory limits.
  Cons:
• Does not fully eliminate the problem: Replica Identity can be
  defined on a UDT or large composite key. If that key attribute expands
  to > 1 GB in typoutput, conflict logging will still error out. In
  short, this leaves us with the same problem as Option 1.
  Option 3: Whitelist Only Fixed-Length / Safe Built-in Data Types in v1
  In the initial version of conflict logging, only serialize columns
  with guaranteed small, bounded types (e.g., fixed-length types like
  int2, int4, int8, float4, float8, bool, date, timestamp, uuid). Any
  varlena type, container, or UDT is automatically omitted without
  invoking its output function.
  Pros:
• Completely immune to palloc() 1 GB overflow by construction.
• Simple to implement with zero chance of erroring out.
• Safe baseline that can be incrementally expanded in future releases.
  Cons:
• Could be restrictive: common types like text, varchar, jsonb, and
  numeric are omitted even when their values
  are just a few bytes (e.g., a 10-character varchar column).
  Option 4: Wrap Attribute Serialization in PG_TRY() / PG_CATCH()
  While serializing each attribute of the tuple to JSON, wrap the
  conversion (specifically OidOutputFunctionCall()) inside an internal
  subtransaction with a PG_TRY() / PG_CATCH() block:
  Pros:
• Catches all hard errors, including 1 GB palloc() exhaustion inside
  uncooperative typoutput functions, memory allocation failures, or
  corrupted data.
• Enables full support for all data types (built-in, JSONB, varlena,
  and UDTs) without risking apply worker retry loops.
• Does not require changing PostgreSQL's global typoutput function
  signature to support escontext.
  Cons:
• Using PG_TRY() and internal subtransactions adds management
  overhead (though conflict logging is an exceptional path, not the main
  transaction fast-path).
• Consumes Transaction IDs (subXIDs), though read-only in-memory
  subtransactions are relatively lightweight.
  In summary, only Options 3 and 4 are viable; while the first two
  options reduce the error surface, they do not eliminate it entirely.
  --
  Regards,
  Dilip Kumar
  Google

[pg-hub-mirror[bot]]

shveta malik <shveta(dot)malik(at)gmail(dot)com> via pgsql-hackers · original email

On Wed, Sep 9, 2026 at 9:11 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:

On Tue, Sep 1, 2026 at 4:03 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:

Here is detailed analysis and summary of the problem and all the
alternatives we tried. The problem is that converting a
TupleTableSlot to JSON can result in rendered JSON text exceeding
PostgreSQL's MaxAllocSize limit (1 GB). When that happens, memory
allocation (palloc() or enlargeStringInfo()) throws a hard ERROR.

This issue is not unique to conflict logging:

1. pgoutput does not handle this: if a row's formatted textual
   representation exceeds 1 GB in logicalrep_write_tuple(),
   OidOutputFunctionCall() or palloc() fails with an unrecoverable hard
   error.
2. For conflict logging, converting to JSON exacerbates the problem
   because a) every byte below 0x20 expands into a 6-byte \uXXXX
   sequence. b) JSONB values (e.g., repeated numeric expansions like
   1e131071), arrays, and composites can be compact on disk but expand to
   hundreds of megabytes or gigabytes when serialized to text. c)
   user-Defined Types (UDTs): A compact binary UDT (e.g., a run-length
   encoded vector) can expand into gigabytes inside its typoutput
   function.
3. Other conflict logging extensions (e.g., pgactive and pgEdge) also
   have this issue, and none of them handle it.

Because storing multi-gigabyte or hundreds-of-megabytes values in a
conflict log table is neither practical nor desirable for post-mortem
debugging, we have been exploring several options to gracefully handle
oversized attributes without erroring out. Below is a summary of the
options explored, along with their pros and cons.

Option 1: Attribute-Level Size Capping via ErrorSaveContext (escontext)
Enforce a reasonable threshold (e.g., 16 KB) per attribute during JSON
conversion. Pass an ErrorSaveContext to an extended serialization
function (datum_to_json_extended()). As serialization recurses through
arrays, composites, or JSONB containers, cumulative size is monitored.
If an attribute exceeds the limit, errsave() records the soft error
and returns (Datum) 0. The caller detects
SOFT_ERROR_OCCURRED(&escontext) and cleanly replaces that attribute in
the outer JSON tuple with an omission object: {"omitted": true,
"length": ...}.
Pros:

• Retains the full tuple structure and all other normal-sized
  columns for analysis.
• Cleanly handles unbounded recursion in nested structures (arrays,
  composites, and JSONB) by aborting early.
  Cons:
• Does not protect against typoutput failures: standard PostgreSQL
  output functions invoked via OidOutputFunctionCall() do not accept an
  ErrorSaveContext parameter.
• If a UDT (or built-in bytea with a large payload in hex format)
  exceeds 1 GB inside its typoutput function, palloc() throws a hard
  ERROR before size checks run; escontext cannot intercept it.

Option 2: Record Only Replica Identity (RI) Columns Instead of Full Tuples
Instead of serializing entire remote and local tuples to JSON, only
serialize the Replica Identity key columns (typically Primary Key or
Unique Index attributes) identifying the conflicting row.
Pros:

• Drastically reduces conflict log storage overhead and serialization cost.
• For standard integer, UUID, or short text keys, the payload is
  tiny and never approaches memory limits.
  Cons:
• Does not fully eliminate the problem: Replica Identity can be
  defined on a UDT or large composite key. If that key attribute expands
  to > 1 GB in typoutput, conflict logging will still error out. In
  short, this leaves us with the same problem as Option 1.

Option 3: Whitelist Only Fixed-Length / Safe Built-in Data Types in v1
In the initial version of conflict logging, only serialize columns
with guaranteed small, bounded types (e.g., fixed-length types like
int2, int4, int8, float4, float8, bool, date, timestamp, uuid). Any
varlena type, container, or UDT is automatically omitted without
invoking its output function.
Pros:

• Completely immune to palloc() 1 GB overflow by construction.
• Simple to implement with zero chance of erroring out.
• Safe baseline that can be incrementally expanded in future releases.
  Cons:
• Could be restrictive: common types like text, varchar, jsonb, and
  numeric are omitted even when their values
  are just a few bytes (e.g., a 10-character varchar column).

Option 4: Wrap Attribute Serialization in PG_TRY() / PG_CATCH()
While serializing each attribute of the tuple to JSON, wrap the
conversion (specifically OidOutputFunctionCall()) inside an internal
subtransaction with a PG_TRY() / PG_CATCH() block:
Pros:

• Catches all hard errors, including 1 GB palloc() exhaustion inside
  uncooperative typoutput functions, memory allocation failures, or
  corrupted data.
• Enables full support for all data types (built-in, JSONB, varlena,
  and UDTs) without risking apply worker retry loops.
• Does not require changing PostgreSQL's global typoutput function
  signature to support escontext.
  Cons:
• Using PG_TRY() and internal subtransactions adds management
  overhead (though conflict logging is an exceptional path, not the main
  transaction fast-path).
• Consumes Transaction IDs (subXIDs), though read-only in-memory
  subtransactions are relatively lightweight.

In summary, only Options 3 and 4 are viable; while the first two
options reduce the error surface, they do not eliminate it entirely.
Thanks for the detailed analysis, Dilip. I gave it some thought. If we
plan to go with Option 4 (assuming the subtransaction overhead is
acceptable), then another approach worth considering would be to
combine Options 1 and 4.
Even though Option 4 acts as a complete safety net and protects
against failures in typoutput, UDTs, etc., it may still bloat the
table. If a few high-volume conflicts log 500 MB to 1 GB of JSON text
each, it will rapidly fill up disk space and bloat the CLT.
We can combine Option 1 and Option 4: limit attributes to 16KB via
Option 1, and use Option 4 for any UDT-related or other such issues.
Option 4 can catch errors and convert them directly to an omitted
marker. That way, the worker never crashes and the CLT doesn't bloat.
thanks
Shveta

[pg-hub-mirror[bot]]

Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> via pgsql-hackers · original email

On Wed, Sep 9, 2026 at 8:41 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:

On Tue, Sep 1, 2026 at 4:03 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:

Here is detailed analysis and summary of the problem and all the
alternatives we tried. The problem is that converting a
TupleTableSlot to JSON can result in rendered JSON text exceeding
PostgreSQL's MaxAllocSize limit (1 GB). When that happens, memory
allocation (palloc() or enlargeStringInfo()) throws a hard ERROR.

This issue is not unique to conflict logging:

1. pgoutput does not handle this: if a row's formatted textual
   representation exceeds 1 GB in logicalrep_write_tuple(),
   OidOutputFunctionCall() or palloc() fails with an unrecoverable hard
   error.
2. For conflict logging, converting to JSON exacerbates the problem
   because a) every byte below 0x20 expands into a 6-byte \uXXXX
   sequence. b) JSONB values (e.g., repeated numeric expansions like
   1e131071), arrays, and composites can be compact on disk but expand to
   hundreds of megabytes or gigabytes when serialized to text. c)
   user-Defined Types (UDTs): A compact binary UDT (e.g., a run-length
   encoded vector) can expand into gigabytes inside its typoutput
   function.
3. Other conflict logging extensions (e.g., pgactive and pgEdge) also
   have this issue, and none of them handle it.

Because storing multi-gigabyte or hundreds-of-megabytes values in a
conflict log table is neither practical nor desirable for post-mortem
debugging, we have been exploring several options to gracefully handle
oversized attributes without erroring out. Below is a summary of the
options explored, along with their pros and cons.
Thank you for summarizing the potential solutions. It made the problem
much clearer. IIUC it can technically happen that the constructed
JSONB data exceed 1GB limitation, but it's a relatively rare case in
practice.
Option 1: Attribute-Level Size Capping via ErrorSaveContext (escontext)
Enforce a reasonable threshold (e.g., 16 KB) per attribute during JSON
conversion. Pass an ErrorSaveContext to an extended serialization
function (datum_to_json_extended()). As serialization recurses through
arrays, composites, or JSONB containers, cumulative size is monitored.
If an attribute exceeds the limit, errsave() records the soft error
and returns (Datum) 0. The caller detects
SOFT_ERROR_OCCURRED(&escontext) and cleanly replaces that attribute in
the outer JSON tuple with an omission object: {"omitted": true,
"length": ...}.
Pros:

• Retains the full tuple structure and all other normal-sized
  columns for analysis.
• Cleanly handles unbounded recursion in nested structures (arrays,
  composites, and JSONB) by aborting early.
  Cons:
• Does not protect against typoutput failures: standard PostgreSQL
  output functions invoked via OidOutputFunctionCall() do not accept an
  ErrorSaveContext parameter.
• If a UDT (or built-in bytea with a large payload in hex format)
  exceeds 1 GB inside its typoutput function, palloc() throws a hard
  ERROR before size checks run; escontext cannot intercept it.
  This is the most complete answer if we do want full-tuple logging. But
  there is a large amount of new code in the JSON code required for v1,
  and as you note it still doesn't close the typoutput hole. I don't
  think the complexity is justified by the extra diagnostic value.

Option 2: Record Only Replica Identity (RI) Columns Instead of Full Tuples
Instead of serializing entire remote and local tuples to JSON, only
serialize the Replica Identity key columns (typically Primary Key or
Unique Index attributes) identifying the conflicting row.
Pros:

• Drastically reduces conflict log storage overhead and serialization cost.
• For standard integer, UUID, or short text keys, the payload is
  tiny and never approaches memory limits.
  Cons:
• Does not fully eliminate the problem: Replica Identity can be
  defined on a UDT or large composite key. If that key attribute expands
  to > 1 GB in typoutput, conflict logging will still error out. In
  short, this leaves us with the same problem as Option 1.
  I think recording only the RI columns would work for conflict
  detection/resolution purposes. Showing the RI columns of a RI FULL
  table would still have the problem, so we might want to either show
  nothing or ask users to disable conflict history logging in that case.
  But recording only the RI columns makes an already rare case
  even rarer, and would cover most use cases. We can improve such cases
  later in a separate patch.

Option 3: Whitelist Only Fixed-Length / Safe Built-in Data Types in v1
In the initial version of conflict logging, only serialize columns
with guaranteed small, bounded types (e.g., fixed-length types like
int2, int4, int8, float4, float8, bool, date, timestamp, uuid). Any
varlena type, container, or UDT is automatically omitted without
invoking its output function.
Pros:

• Completely immune to palloc() 1 GB overflow by construction.
• Simple to implement with zero chance of erroring out.
• Safe baseline that can be incrementally expanded in future releases.
  Cons:
• Could be restrictive: common types like text, varchar, jsonb, and
  numeric are omitted even when their values
  are just a few bytes (e.g., a 10-character varchar column).
  Text and numeric keys are the common case, and omitting them would be
  worse to me than the problem we are solving.
  Option 4: Wrap Attribute Serialization in PG_TRY() / PG_CATCH()
  While serializing each attribute of the tuple to JSON, wrap the
  conversion (specifically OidOutputFunctionCall()) inside an internal
  subtransaction with a PG_TRY() / PG_CATCH() block:
  Pros:
• Catches all hard errors, including 1 GB palloc() exhaustion inside
  uncooperative typoutput functions, memory allocation failures, or
  corrupted data.
• Enables full support for all data types (built-in, JSONB, varlena,
  and UDTs) without risking apply worker retry loops.
• Does not require changing PostgreSQL's global typoutput function
  signature to support escontext.
  Cons:
• Using PG_TRY() and internal subtransactions adds management
  overhead (though conflict logging is an exceptional path, not the main
  transaction fast-path).
• Consumes Transaction IDs (subXIDs), though read-only in-memory
  subtransactions are relatively lightweight.
  I don't think we can classify the error reliably. enlargeStringInfo()
  reports ERRCODE_PROGRAM_LIMIT_EXCEEDED, an oversized palloc() request
  reports ERRCODE_INTERNAL_ERROR, and a genuine allocation failure
  reports ERRCODE_OUT_OF_MEMORY. We cannot tell "this value was too
  large to render" from "this backend is really out of memory" or from a
  bug in some type's output function, and silently swallowing the latter
  two in an apply worker seems worse than the disease.
  Overall, I prefer option 2 and deal with the very rare cases in a
  separate patch if necessary.
  Regards,
  --
  Masahiko Sawada
  Amazon Web Services: https://aws.amazon.com

[pg-hub-mirror[bot]]

A patch is now available as PR #407.