java.io.IOException when selecting nested JSON array of mixed type objects via DataGrip

[maxjustus]

Clickhouse version: 23.1
clickhouse-jdbc version: 0.3.2.11 (also tried 0.4.0 and got the same behavior)
Datagrip version: 2022.3

steps to reproduce:

set allow_experimental_object_type = 1;
CREATE OR REPLACE TABLE json
(
    o JSON
) ENGINE = Memory;
insert into json (o) values ('{"a": [{ "b": [{"c": 1, "d": "value"}] }]}');
select * from json;

select * from json raises java.io.IOException: Reached end of input stream after reading 4 of 118 bytes

if I change the json insert to
insert into json (o) values ('{"a": [{ "b": [{"d": "value"}] }]}');
the select succeeds and shows ([[[['value']]]])

if I change the json insert to
insert into json (o) values ('{"a": [{ "b": [{"c": 1}] }]}');
the select succeeds and shows ([[[[1]]]])

so it appears that the error arises when you have mixed types in a nested array in a json type column.

I also created an issue in the datagrip tracker since it's not clear whether it's a problem with Datagrip or the clickhouse-jdbc driver

[zhicwu]

In the past ClickHouse use nested tuple with names to represent json, but now it seems changed to something else?

select toTypeName(o), toString(o) from json

┌─toTypeName(o)───────────────────────────────┬─toString(o)─────────┐
│ Tuple(a Nested(b Nested(c Int8, d String))) │ ([([(1,'value')])]) │
└─────────────────────────────────────────────┴─────────────────────┘

[zhicwu]

I think it's a server side issue as the query result is not same as what we provided in insert statement.

[maxjustus]

interesting - if I run my example in clickhouse-client against the same server the select result I get is ([([(1,'value')])]) which shows the nested tuple structure, so the server seems to be behaving correctly? At least for clickhouse-client.

[zhicwu]

Looks like nested object array in json is mapped to Nested and object is named Tuple. Will try to fix this.

insert into json(o) values('{"a":{"x":1,"y":2}, "b":[{"c":3}]}')

select toTypeName(o), toString(o) from json
┌─toTypeName(o)────────────────────────────────────┬─toString(o)───┐
│ Tuple(a Tuple(x Int8, y Int8), b Nested(c Int8)) │ ((1,2),[(3)]) │
└──────────────────────────────────────────────────┴───────────────┘

[maxjustus]

Thank you!

[zhicwu]

@maxjustus, forgot to mention that you can download the nightly build at here to unblock your work.

[C0mbatwombat]

First of all, thank you @zhicwu for your hard work on this new driver, it's really quite nice.
I think there's a bug in this commit, the size check done in ClickhouseNestedvalue.java on line 73 checks the number of nested items against the number of columns, eg. if you have 6 columns in a nested field, you can only insert 6 rows.
I suppose taking this size check out would fix the issue?

[C0mbatwombat]

Created a bug report #1259