feat: Add items() or unnest() method to MapValue type

[vspinu]

Would it be possible to enrich current MapValue with items() and/or unnest() methods?

What is the motivation behind your request?

My current motivation is to unnest a map in spark. Looks like spark does not support multiple simultaneous unnests so when I attempt the following (working in duckdb btw):

df = spark.createDataFrame(pd.DataFrame({"keys": [["a", "b"], ["b"]], "values": [[1, 2], [3]]}))
df.createOrReplaceTempView("tmp")
con = ibis.pyspark.connect(spark)
tbl = con.table("tmp").mutate(mapcol=ibis.map(_.keys, _.values))

tbl.select(key=_.mapcol.keys().unnest(), value=_.mapcol.values().unnest())

I get the error:

AnalysisException: [UNSUPPORTED_GENERATOR.MULTI_GENERATOR] The generator is not supported: only one generator allowed per SELECT clause but found 2: "explode(map_keys(mapcol))", "explode(map_values(mapcol))".

Describe the solution you'd like

So either items() or unnest() or a method to zip keys and values of the map would be really useful here.

What version of ibis are you running?

10.5.0

What backend(s) are you using, if any?

pyspark

Code of Conduct

• I agree to follow this project's Code of Conduct

[vspinu]

My current solution is to rewrite it as a double mutate

tbl.mutate(key=_.keys.unnest()).mutate(value=_.mapcol.get(_.key)).show()