Druid connector join mysql come to nothing

[beepupa]

2020-03-26T19:12:39.002+0800 ERROR remote-task-callback-160 com.facebook.airlift.concurrent.BoundedExecutor Task failed
java.lang.IllegalArgumentException: com.facebook.presto.server.TaskUpdateRequest could not be converted to JSON
at com.facebook.airlift.json.JsonCodec.toJsonBytes(JsonCodec.java:214)
at com.facebook.presto.server.smile.JsonCodecWrapper.toBytes(JsonCodecWrapper.java:45)
at com.facebook.presto.server.remotetask.HttpRemoteTask.sendUpdate(HttpRemoteTask.java:646)
at com.facebook.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.joda.time.Interval and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: com.facebook.presto.server.TaskUpdateRequest["sources"]->com.google.common.collect.SingletonImmutableList[0]->com.facebook.presto.execution.TaskSource["splits"]->com.google.common.collect.SingletonImmutableSet[0]->com.facebook.presto.execution.ScheduledSplit["split"]->com.facebook.presto.metadata.Split["connectorSplit"]->com.facebook.presto.druid.DruidSplit["segmentInfo"]->com.facebook.presto.druid.metadata.DruidSegmentInfo["interval"])
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:313)
at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71)
at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.std.ReferenceTypeSerializer.serialize(ReferenceTypeSerializer.java:381)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:604)
at com.facebook.presto.metadata.AbstractTypedJacksonModule$InternalTypeSerializer.serialize(AbstractTypedJacksonModule.java:115)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400)
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1392)
at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120)
at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1017)
at com.facebook.airlift.json.JsonCodec.toJsonBytes(JsonCodec.java:211)
... 6 more

2020-03-26T19:12:39.127+0800 ERROR 20200326_111238_00078_j58fx.3.0.0-0-739 com.facebook.presto.operator.Driver Error closing operator 0 for task 20200326_111238_00078_j58fx.3.0.0
java.lang.NullPointerException
at com.mysql.jdbc.MysqlIO.clearInputStream(MysqlIO.java:892)
at com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:171)
at com.mysql.jdbc.ResultSetImpl.realClose(ResultSetImpl.java:6644)
at com.mysql.jdbc.ResultSetImpl.close(ResultSetImpl.java:841)
at com.facebook.presto.plugin.jdbc.JdbcRecordCursor.close(JdbcRecordCursor.java:219)
at com.facebook.presto.operator.ScanFilterAndProjectOperator.finish(ScanFilterAndProjectOperator.java:182)
at com.facebook.presto.operator.ScanFilterAndProjectOperator.close(ScanFilterAndProjectOperator.java:166)
at com.facebook.presto.operator.Driver.closeAndDestroyOperators(Driver.java:546)
at com.facebook.presto.operator.Driver.processInternal(Driver.java:406)
at com.facebook.presto.operator.Driver.lambda$processFor$8(Driver.java:283)
at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:675)
at com.facebook.presto.operator.Driver.processFor(Driver.java:276)
at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1077)
at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:162)
at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:545)
at com.facebook.presto.$gen.Presto_0_233_1_2d75403____20200326_084339_1.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

[beepupa]

What should I do

[beepupa]

[tooptoop4]

share more details so others can replicate the issue. ie your query, presto version, druid setup. does plain druid sql without joining mysql work?

[beepupa]

share more details so others can replicate the issue. ie your query, presto version, druid setup. does plain druid sql without joining mysql work?

Version: 0.233.1 ，Use alone is normal work，
my druid seting:
druid.broker-url=http://myip:8082
druid.coordinator-url=http://myip:8081
druid.schema-name=druid
my request :
presto:druid> select user from test a join mysql.test.ab b on a.sum_added=b.sum_added;

Query 20200327_055710_00006_f9pwa, RUNNING, 1 node, 65 splits
0:12 [ 3 rows, 0B] [ 0 rows/s, 0B/s] [===========>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 26%

 STAGES   ROWS  ROWS/s  BYTES  BYTES/s  QUEUED    RUN   DONE

0.........R 0 0 0B 0B 0 16 0
1.......R 3 0 90B 7B 0 32 16
2.....R 0 0 0B 0B 0 0 0
3.....F 3 0 0B 0B 0 0 1

wrong:
2020-03-27T13:57:11.199+0800 ERROR remote-task-callback-25 com.facebook.airlift.concurrent.BoundedExecutor Task failed
java.lang.IllegalArgumentException: com.facebook.presto.server.TaskUpdateRequest could not be converted to JSON
at com.facebook.airlift.json.JsonCodec.toJsonBytes(JsonCodec.java:214)
at com.facebook.presto.server.smile.JsonCodecWrapper.toBytes(JsonCodecWrapper.java:45)
at com.facebook.presto.server.remotetask.HttpRemoteTask.sendUpdate(HttpRemoteTask.java:646)
at com.facebook.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.joda.time.Interval and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS) (through reference chain: com.facebook.presto.server.TaskUpdateRequest["sources"]->com.google.common.collect.SingletonImmutableList[0]->com.facebook.presto.execution.TaskSource["splits"]->com.google.common.collect.SingletonImmutableSet[0]->com.facebook.presto.execution.ScheduledSplit["split"]->com.facebook.presto.metadata.Split["connectorSplit"]->com.facebook.presto.druid.DruidSplit["segmentInfo"]->com.facebook.presto.druid.metadata.DruidSegmentInfo["interval"])
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:313)
at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71)
at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.std.ReferenceTypeSerializer.serialize(ReferenceTypeSerializer.java:381)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:604)
at com.facebook.presto.metadata.AbstractTypedJacksonModule$InternalTypeSerializer.serialize(AbstractTypedJacksonModule.java:115)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serializeContents(CollectionSerializer.java:145)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:107)
at com.fasterxml.jackson.databind.ser.std.CollectionSerializer.serialize(CollectionSerializer.java:25)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:727)
at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:719)
at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:155)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:400)
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1392)
at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120)
at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1017)
at com.facebook.airlift.json.JsonCodec.toJsonBytes(JsonCodec.java:211)
... 6 more

[beepupa]

I'm trying to use
select user from test a join mysql.test.ab b on a.sum_added=b.sum_added limit 2;
Is the same

[pigxuyu]

your sql will cause scan the whole druid table, it maybe have a bug about serializer.
you may try add some filter, such as __time >= xx. it will not scan the whole druid table.

[skyahead]

Scanning a whole druid table takes the SEGMENT path, which has the same bug as reported here: #14156 (comment).

@zhenxiao After your latest aggregation pushdown, I still face the same Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class org.joda.time.Interval issue and could not resolve it.

[zhenxiao]

hi @skyahead @beepupa @tooptoop4
I fixed some druid bugs in recent commits. Could you please take a try of latest master branch? Or, we might have to wait till 235 release

[beepupa]

嗨@skyahead @beepupa @ tooptoop4
我修复了最近提交中的一些德鲁伊错误。您能尝试一下最新的master分支吗？或者，我们可能要等到235发布

Thanks,I've solved it

[CCColby]

I alse meet this problem, how to fix it? thanks @beepupa
com.facebook.presto.operator.Driver Error closing operator 0 for task