Parse a JSON column once per record for repeated jsonPath* transforms

[xiangfu0]

Motivation

A common ingestion pattern extracts many fields from a single JSON-string column — e.g. several JSONPATHSTRING(message, '$.x') transforms on one record. Each jsonPath* call re-parses the whole document, so extracting N fields parses it N times. On a JSON-heavy realtime log table this re-parsing dominated transform CPU.

#18711 added an explicit, opt-in way to avoid this (jsonExtractObject(col) materializes a parsed intermediate column). This PR makes it automatic, with the parsed document scoped to the record.

Change

• InbuiltFunctionEvaluator recognizes the JsonFunctions.jsonPath* family (whose first argument is a JSON document) and, on the ingestion execute(GenericRow) path, parses a String first argument once per record, stashing the parsed document in a new transient per-record scratch on GenericRow. The other jsonPath* extractions on the same column reuse it.
• GenericRow gains a transient, identity-keyed scratch (getScratchValue/putScratchValue, reset in clear()). It is not record data: not serialized, not indexed, not copied by init(), lazily created, and freed with the record.
• JsonFunctions.parseDocOrSelf parses a JSON object/array string to a Map/List, or returns the input unchanged for anything else (non-container/invalid string, scalar, already-parsed value, null) — so a downstream jsonPath* call behaves exactly as if it received the raw value.

Why on the record, not a static cache

The cache lives on the GenericRow: per-record scoped, freed with the row, carrying no cross-record or cross-thread state. That scales for high-throughput, many-partition ingestion — unlike a static/thread-local cache that would retain stale parsed documents per thread. It's the automatic equivalent of the explicit jsonExtractObject pattern; like that pattern, a path landing on a container returns a live node of the shared parsed model, so jsonPath/jsonPathArray results are read-only (the jsonPathString/Long/Double overloads copy out a scalar/string and are unaffected).

Performance

JMH transform-pipeline benchmark on a realistic log record (message is a JSON string read 8×): the unmodified config goes from ~69K to ~141K rows/s (~2×), matching an explicitly parse-once-restructured config (which then adds nothing).

Testing

• InbuiltFunctionEvaluatorTest#testJsonPathParsesDocumentOncePerRecord: the document is parsed once and stashed on the record (asserts the scratch holds the parsed Map, reused across extractions), non-JSON columns pass through unchanged.
• JsonFunctionsTest#testParseDocOrSelf: container→Map/List; non-container/scalar/invalid/null/already-parsed→unchanged.
• Touched test classes pass (InbuiltFunctionEvaluatorTest, JsonFunctionsTest).

Backward compatibility

None required. Results are identical to the un-cached path; only the number of parses changes. New GenericRow methods are additive (no signature changes); the transient scratch is excluded from serialization and equals/hashCode. No config / wire-format surface touched.

cc SPI maintainers — this touches GenericRow in pinot-spi (additive only).

[xiangfu0]

Closing — this isn't the right tradeoff. The explicit jsonExtractObject parse-once intermediate (#18711) already solves the repeated-extraction case cleanly and opt-in.

An automatic per-record parse cache adds a per-call check plus a per-record scratch allocation to the hot ingestion path, and new SPI surface on GenericRow, but only helps the narrow pattern of many jsonPath* extractions on the same column:

• at high ingestion throughput, that broad per-record/per-call overhead (and the extra allocation/GC across all partition-consumer threads) isn't a clean win;
• at low ingestion throughput, there's no re-parse perf problem to solve.

Users who extract many fields from one JSON column should use jsonExtractObject(col) with an intermediate column (now also valid at config time via #18721).

[codecov-commenter]

Codecov Report

❌ Patch coverage is 85.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.57%. Comparing base (a60a8fc) to head (1e23114).

Files with missing lines	Patch %	Lines
.../org/apache/pinot/spi/data/readers/GenericRow.java	71.42%	1 Missing and 1 partial ⚠️
...not/common/evaluator/InbuiltFunctionEvaluator.java	88.88%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #18722      +/-   ##
============================================
+ Coverage     56.98%   64.57%   +7.58%     
- Complexity        7     1291    +1284     
============================================
  Files          2587     3372     +785     
  Lines        150212   208722   +58510     
  Branches      24292    32621    +8329     
============================================
+ Hits          85602   134783   +49181     
- Misses        57341    63103    +5762     
- Partials       7269    10836    +3567     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (?)
java-21	64.57% <85.00%> (+7.58%)	⬆️
temurin	64.57% <85.00%> (+7.58%)	⬆️
unittests	64.57% <85.00%> (+7.58%)	⬆️
unittests1	57.01% <85.00%> (+0.02%)	⬆️
unittests2	37.13% <50.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.