Skip to content

MINOR: [java] bound zero-byte allocation in standalone fast reader - #3946

Open
arib06 wants to merge 1 commit into
apache:mainfrom
arib06:java-fastreader-datum-scope
Open

MINOR: [java] bound zero-byte allocation in standalone fast reader#3946
arib06 wants to merge 1 commit into
apache:mainfrom
arib06:java-fastreader-datum-scope

Conversation

@arib06

@arib06 arib06 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What is the purpose of the change

AVRO-4241 made the zero-byte collection-element allocation cap (SystemLimitException) cumulative across a decoded datum, so a record with many array<null>-style fields cannot over-allocate in aggregate even when each field stays under the per-collection limit. That cumulative accounting only holds while an allocation scope is open. GenericDatumReader.read opens one for the whole datum, and the fast array reader opens one around its block loop, but the datum-level fast readers do not. When the fast reader is used standalone via FastReaderBuilder.createDatumReader(...) (i.e. not reached through GenericDatumReader.read), decoding a record resets the running total per array field, because each field's array reader opens and closes its own outermost scope. Each field then passes the per-collection check while the aggregate exceeds the cap, so a small payload can drive a large allocation. The fix wraps the reader returned by createDatumReader in a scope, matching GenericDatumReader.read. Scopes nest, so the delegated path (where GenericDatumReader.read already opened the outer scope) is unchanged, and valid input decodes exactly as before.

Verifying this change

This change added tests and can be verified as follows:

  • Added standaloneFastReaderRecordRejectedCumulativelyAcrossDatum: a record with two array<null> fields of 600 nulls each (cap set to 1000) decoded through the standalone fast reader is rejected on the second field with SystemLimitException. Before the change both fields decoded, bypassing the cap.
  • Added standaloneFastReaderRecordWithinCumulativeLimitStillDecodes: two fields totalling 800 elements (under the cap) still decode, and the per-datum budget resets between reads.

Documentation

  • Does this pull request introduce a new feature? no

@github-actions github-actions Bot added the Java Pull Requests for Java binding label Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Java Pull Requests for Java binding

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant