Describe the enhancement requested
Follow-up of #50326. GH-50327 makes Array.to_pylist() convert without per-element Scalars, but the maps_as_pydicts option still routes to the Scalar-based path: every map row allocates a MapScalar, converts its keys via per-element as_py, and builds the dict in Python.
Converting maps to dicts is the natural consumption pattern for engines whose map values are Python dicts (e.g. Spark's Arrow-serialized Python UDFs currently receive association lists from to_pylist() and rebuild a dict per row in pure Python — one of the dominant remaining costs in that path).
Proposal: thread maps_as_pydicts through the scalar-free _getitem_py mechanism introduced in GH-50327 so MapArray builds the dict directly from the flattened keys/items children:
- default (
None) semantics unchanged (association lists);
'lossy'/'strict' build the dict in one pass; when the dict size shows duplicate keys, the row is redone with the careful per-key loop so the warning ('lossy') and KeyError ('strict') semantics stay exactly identical to MapScalar.as_py;
- invalid values still raise the same
ValueError when a map value is converted;
- the option propagates through nested types (list/struct/map values) as before, and unspecialized types keep the exact Scalar fallback.
Component(s)
Python
Describe the enhancement requested
Follow-up of #50326. GH-50327 makes
Array.to_pylist()convert without per-element Scalars, but themaps_as_pydictsoption still routes to the Scalar-based path: every map row allocates aMapScalar, converts its keys via per-elementas_py, and builds the dict in Python.Converting maps to dicts is the natural consumption pattern for engines whose map values are Python dicts (e.g. Spark's Arrow-serialized Python UDFs currently receive association lists from
to_pylist()and rebuild a dict per row in pure Python — one of the dominant remaining costs in that path).Proposal: thread
maps_as_pydictsthrough the scalar-free_getitem_pymechanism introduced in GH-50327 soMapArraybuilds the dict directly from the flattened keys/items children:None) semantics unchanged (association lists);'lossy'/'strict'build the dict in one pass; when the dict size shows duplicate keys, the row is redone with the careful per-key loop so the warning ('lossy') andKeyError('strict') semantics stay exactly identical toMapScalar.as_py;ValueErrorwhen a map value is converted;Component(s)
Python