Query with map field got wrong result.

[JackyWoo]

We use hive 1.x and presto 315 in my company. And we have a table which is defined by hive like below :

CREATE TABLE T.A(
ip string,
event_info map<string,string>)
PARTITIONED BY (
dayno string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe'
WITH SERDEPROPERTIES (
'colelction.delim'=',',
'field.delim'='\t',
'mapkey.delim'=':',
'serialization.format'='\t')
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.RCFileInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.RCFileOutputFormat'
LOCATION
'hdfs://xxx'
`

When I send a query to presto :

select cardinality(event_info), event_info from T.A limit 1

I get the result which is wrong :

1 {fk_1=1，fk_2:2，fk_3:3}

Finally I find that for map field collection deliminator config key hive 1.x use "colelction.delim" but hive 3.x use "collection.delim".

Prestosql 315 relies on hive 3.x, so it use “collection.delim” config.

Resolution : when getting deliminator first use “collection.delim” config, if get null then use “colelction.delim” config.

So prestosql 315 can query both hive 1.x and 3.x defined table correctly.

[cla-bot]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. For more information, see https://github.com/prestosql/cla.

[cla-bot]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. For more information, see https://github.com/prestosql/cla.

[findepi]

Thanks for reporting this & fixing!

Finally I find that for map field collection deliminator config key hive 1.x use "collection.delim" but hive 3.x use "colelction.delim".

@JackyWoo is it reversed?

[findepi]

( https://issues.apache.org/jira/browse/HIVE-16922 )

[JackyWoo]

@electrum @findepi Thanks for advising, I correct it.

[cla-bot]

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please submit the signed CLA to cla@prestosql.io. For more information, see https://github.com/prestosql/cla.

[findepi]

@JackyWoo would you be able to sign the CLA? See https://github.com/prestosql/cla for more info.

[JackyWoo]

@findepi I have submit CLA to cla@prestosql.io.

[martint]

@cla-bot check

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!

[findepi]

LGTM. @JackyWoo could you please squash the commits?

[JackyWoo]

@findepi Thanks for help, it is done.

[findepi]

Merged as aaf43e7, thanks!