Fix remaining reported issues for arrays of map

[comphead]

@comphead, smaller follow-ups are reasonable, provided this PR preserves the previous safe behavior for the affected cases. The remaining [P2] concerns come from the actual BASE-to-HEAD changes:

• The TRY/bracket and collation examples contain folded map literals. BASE declines those non-null complex literals; HEAD newly admits and rebuilds them, exposing the consumer limitations. That admission is not restricted to an outer array-of-map literal.
• The stateful example exercises the new ANSI guard, which serializes the left operand twice. BASE has no second serialization. This change affects nullable array lookups beyond literal expansion.
• The Slice example does construct an array of maps. The new deepNullable casts change its nested map type, while Slice still declares the original element type. BASE passes that single map child through without the new cast.

So a narrow solution here is fine: restrict the newly enabled paths or retain safe fallback until the corresponding consumer fix lands; broader collation/collection improvements can remain separate PRs. The scope distinction above is from pinned source comparison, not an additional executed reproduction.

Originally posted by @sunchao in #5452 (review)

[sunchao]

Two additional P2 follow-up cases from the re-review of #5452 at ec9832366, recorded here under the agreed follow-up scope.

Both were reproduced on Spark 4.1.3 / JDK 17 with normal constant folding and a Parquet table t(id INT) containing 1, 2, 3 in one file.

1. Duplicate struct field names inside folded map values

SELECT id, array(map(1, named_struct('x', 10, 'x', 20))) AS a
FROM t;

Spark preserves both struct values, 10 and 20. Comet selects CometProject and fails with NullPointerException when the generated writer writes the second field. The literal reconstruction sends the struct value through whole-map JVM codegen, bypassing the duplicate-name guard on CreateNamedStruct. Arrow initializes struct children by name, collapsing the two x fields into one child; the writer's ordinal-1 vector is null.

The direct map(...) variant also fails. Distinct x/y field-name controls pass. A narrow follow-up can retain Spark fallback for these nested shapes until the output path preserves duplicate field positions.

2. Calendar interval map values outside Arrow's nanosecond range

SELECT id, array(map(1, make_interval(0, 0, 0, 0, 3000000, 0, 0))) AS a
FROM t;

Spark accepts the 3000000 hours value, while Comet throws ArithmeticException: long overflow. The new admission check allows this map value type; the calendar-interval restriction applies to map keys. The generated output writer multiplies the interval's microseconds by 1000L using Math.multiplyExact, exceeding the nanosecond field's range.

The direct map(...) variant also fails; a 24-hour control passes natively. A narrow follow-up can retain Spark fallback for out-of-range calendar interval values.

Validation: all four failing variants (direct map and array of maps for each case) were reproduced with a freshly built native library, current JVM classes, and CometProject asserted. Ordinary Spark succeeds. All eight reproduction/control tests pass after substituting the exact-base 215ab706a literal serializer, restoring Spark projection for these folded literals. That control replaces only the literal serializer on the current native/JVM runtime; it is not a full base build. Local execution was limited to Spark 4.1.3.

[sunchao]

Two P1 updates from the final pass of #5452 at ec9832366, against base 215ab706a. The UTF-8 case is new; the stateful element_at case below strengthens and raises the priority of the previously reported P2.

1. [P1] Newly admitted malformed UTF-8 map literals can abort the JVM

With normal constant folding, ANSI disabled, and t(id INT) read from one Parquet file containing 1, 2, 3:

SET spark.sql.ansi.enabled = false;

SELECT id,
       levenshtein(element_at(map(1, CAST(X'FF' AS STRING)), id), 'a') AS v
FROM t;

Spark returns (1, 1), (2, NULL), (3, NULL). The current head selects CometProject and aborts the entire JVM with SIGABRT, rather than returning a query exception:

unsafe precondition(s) violated: hint::unreachable_unchecked must never be reached
thread caused non-unwinding panic. aborting.

The new literal admission rebuilds the folded map without validating its string values. The generated writer copies Spark's raw string bytes into Arrow, and the native bridge imports them through unchecked Arrow FFI. Native character iteration then receives bytes that violate its valid-UTF-8 requirement.

This also has a silent-result manifestation: length(element_at(map(1, CAST(X'C080' AS STRING)), id)) returns 1 for id = 1, where Spark returns 2; a binary round-trip control preserves C080 on both paths.

The underlying bridge hazard predates this PR: disabling ConstantFolding also exposes it with the base serializers. The regression here is admitting these folded queries under normal optimizer settings, where the base retains safe Spark evaluation. A narrow guard can decline expansion of literals containing malformed UTF-8 until the broader bridge/string handling is fixed. Replacing the bytes with normalized text would not preserve Spark's raw-byte semantics.

2. [P1] The ANSI element_at guard silently changes values and NULL placement for stateful operands

This is the existing duplicated-left-operand family, now reproduced with a simpler primitive-array query. For one Parquet file written with IDs 1 through 16 in order:

SET spark.sql.ansi.enabled = true;

SELECT id,
       element_at(
         IF(monotonically_increasing_id() % 2 = 0,
            array(1), CAST(NULL AS ARRAY<INT>)),
         1) AS v
FROM t;

Spark returns non-NULL values for all eight odd IDs. Comet returns non-NULL values only for IDs 1, 5, 9, 13: four expected values silently become NULL. The simpler random variant, element_at(IF(rand(7L) < 0.5, array(1), CAST(NULL AS ARRAY<INT>)), 1), also changes NULL placement, including producing a value on a row where Spark returns NULL.

The left operand is serialized for the lookup and again for the new null guard. Reevaluating a stateful expression over different row selections changes its state and results. Both examples pass with ANSI disabled, and both pass with the base serializers while still executing a native projection.

I would raise this from P2 to P1 because it regresses an already-supported primitive-array path, requires no malformed data or incompatible-expression flag, and silently changes downstream counts, filters, or persisted results. ANSI is enabled by default in the tested Spark 4.1.3 session. The demonstrated trigger is an inline nullable, stateful left operand; these tests do not show corruption of ordinary deterministic array columns. Retaining Spark fallback for affected nondeterministic operands is a narrow safeguard while single-evaluation handling is implemented.

Validation and limits: both findings were independently reproduced on Spark 4.1.3 / JDK 17 with native and JVM artifacts built from the pinned head and CometProject verified. Exact-base controls substitute only arrays.scala, maps.scala, and literals.scala serializers on the current native/JVM runtime; they are not a full base build. The UTF-8 process abort was reproduced with the debug native build and active unsafe-precondition checks. Release-build failure behavior, heap corruption, and exploitability were not tested or established.

I recommend retaining safe fallback for these cases before merging #5452; the broader fixes can remain follow-ups here.