[Java] Configuration does not provide a mapping for array column

[asfimport]

I tried to leverage org.apache.arrow.adapter.jdbc.JdbcToArrow.sqlToArrow to query a Hive table but got the following error message on array columns. 

Configuration does not provide a mapping for array column 234

The error message makes wondering whether it is possible to provided a customized configuration for array columns?

 

Reporter: Benjamin Du

Related issues:

• [Java][JDBC] Allow users to config the mapping between SQL types and Arrow types (is blocked by)

_{Note: This issue was originally created as ARROW-10762. Please see the migration documentation for further details.}

[asfimport]

Benjamin Du:
And similarly, I got an error message on a column with the map type. 

| java.lang.UnsupportedOperationException: java.lang.UnsupportedOperationException|

 |
|-|-|

[asfimport]

Liya Fan / @liyafan82:
This issue may be fixed by the patch for ARROW-10492.

[asfimport]

Benjamin Du:
Is there any example code doing customized type mapping based on PR branch? 

[asfimport]

Liya Fan / @liyafan82:
Yes. An example can be found in JdbcToArrowVectorIteratorTest of the PR (https://github.com/apache/arrow/pull/8597/files#diff-7ba75c47a9fad67647ed7ce4ee6da10464ee8d0acc72de3dd2222092a33323d0).

In particular, to support customized mapping, we need to

1. Define the mapping from JDBC types to Arrow types.
2. Create a JdbcToArrowConfig object with the mapping.
3. Create a ArrowVectorIterator object with the config object above.
4. Use the iterator above to consume data.

[asfimport]

Benjamin Du:
Thank you, [~fan_li_ya]!

Is a default mapping of JDBC array to arrow array not provided? I think it is better to provide a default mapping for array and map types. 

[asfimport]

Liya Fan / @liyafan82:
[~dclong] Sure we have a default mapping, which can be found here (https://github.com/apache/arrow/pull/8597/files#diff-5f6ecfada9677653fcba498dced5daec420625828ee6e726fc1fcb10f9bd0b99). If we do not specify a customized mapping, the default one will be used.

However, it seems the default mapping failed to process your data. If you could provide more details, maybe we can help you do some more investigations.