broker query error when druid with kerberos

[sydt2014]

Affected Version

0.12.3

Description

when druid configed with kerberos and broker query, but it occur error:

2020-03-20T10:53:39,869 INFO [DruidSchema-Cache-0] io.druid.security.kerberos.DruidKerberosUtil - Re-Login from key tab [/root/imply-2.7.8/druid.keytab] with principal [druid@PROD.YIRAN.COM]
2020-03-20T10:53:39,886 INFO [DruidSchema-Cache-0] io.druid.sql.calcite.schema.DruidSchema - Refreshed metadata for dataSource[kdc_metrics_test] in 21 ms (1 segments queried, 0 segments left).
2020-03-20T10:53:39,891 WARN [DruidSchema-Cache-0] io.druid.sql.calcite.schema.DruidSchema - Metadata refresh failed, trying again soon.
io.druid.java.util.common.RE: Failure getting results for query[4431c4df-b988-44c3-a1a7-e8641a3aabf4] url[http://tst1-tst-tdrd-1.prod.yiran.com:8101/druid/v2/] because of [Invalid type marker byte 0x3c for expected value token
at [Source: java.io.SequenceInputStream@4f7bd162; line: -1, column: 1]]
at io.druid.client.DirectDruidClient$JsonParserIterator.init(DirectDruidClient.java:623) ~[druid-server-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.client.DirectDruidClient$JsonParserIterator.hasNext(DirectDruidClient.java:555) ~[druid-server-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.BaseSequence.makeYielder(BaseSequence.java:88) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.BaseSequence.toYielder(BaseSequence.java:68) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.MappedSequence.toYielder(MappedSequence.java:49) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.MergeSequence.lambda$toYielder$0(MergeSequence.java:66) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.MappingAccumulator.accumulate(MappingAccumulator.java:40) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.BaseSequence.accumulate(BaseSequence.java:46) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.MappedSequence.accumulate(MappedSequence.java:43) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.MergeSequence.toYielder(MergeSequence.java:63) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.LazySequence.toYielder(LazySequence.java:46) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.query.RetryQueryRunner$1.toYielder(RetryQueryRunner.java:102) ~[druid-processing-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.common.guava.CombiningSequence.toYielder(CombiningSequence.java:80) ~[druid-common-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.MappedSequence.toYielder(MappedSequence.java:49) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.MappedSequence.toYielder(MappedSequence.java:49) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.WrappingSequence$2.get(WrappingSequence.java:87) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.WrappingSequence$2.get(WrappingSequence.java:83) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.query.CPUTimeMetricQueryRunner$1.wrap(CPUTimeMetricQueryRunner.java:74) ~[druid-processing-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.WrappingSequence.toYielder(WrappingSequence.java:82) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.WrappingSequence$2.get(WrappingSequence.java:87) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.WrappingSequence$2.get(WrappingSequence.java:83) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.SequenceWrapper.wrap(SequenceWrapper.java:55) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.WrappingSequence.toYielder(WrappingSequence.java:82) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.java.util.common.guava.Yielders.each(Yielders.java:32) ~[java-util-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.sql.calcite.schema.DruidSchema.refreshSegmentsForDataSource(DruidSchema.java:424) ~[druid-sql-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.sql.calcite.schema.DruidSchema.refreshSegments(DruidSchema.java:390) ~[druid-sql-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.sql.calcite.schema.DruidSchema.access$1100(DruidSchema.java:81) ~[druid-sql-0.12.3-iap7.jar:0.12.3-iap7]
at io.druid.sql.calcite.schema.DruidSchema$2.run(DruidSchema.java:227) [druid-sql-0.12.3-iap7.jar:0.12.3-iap7]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_171]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_171]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_171]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: com.fasterxml.jackson.core.JsonParseException: Invalid type marker byte 0x3c for expected value token
at [Source: java.io.SequenceInputStream@4f7bd162; line: -1, column: 1]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419) ~[jackson-core-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508) ~[jackson-core-2.4.6.jar:2.4.6]
at com.fasterxml.jackson.dataformat.smile.SmileParser.nextToken(SmileParser.java:688) ~[jackson-dataformat-smile-2.4.6.jar:2.4.6]
at io.druid.client.DirectDruidClient$JsonParserIterator.init(DirectDruidClient.java:606) ~[druid-server-0.12.3-iap7.jar:0.12.3-iap7]
... 34 more
2020-03-20T10:54:45,866 INFO [DruidSchema-Cache-0] io.druid.sql.calcite.schema.DruidSchema - Refreshed metadata for dataSource[auto_test_t2] in 12 ms (1 segments queried, 0 segments left).
2020-03-20T10:54:45,871 INFO [DruidSchema-Cache-0] io.druid.security.kerberos.DruidKerberosUtil - Re-Login from key tab [/root/imply-2.7.8/druid.keytab] with principal [druid@PROD.YIRAN.COM]

[sydt2014]

broker query realtime task,it query error

[sydt2014]

This may a bug for druid-kerbeos!!! KerberosHttpClient retain CookieManager for url request. However it use uncorrectly.For example , KerberosHttpClient send request to query broker:
http://tst1-tst-tdrd-1.com:8082/druid/v2 and return correctly. but next ,
KerberosHttpClient send request to query historical node :
http://tst1-tst-tdrd-1.com:8083/druid/v2 which will use the cookie for http://tst1-tst-tdrd-1.com:8082/druid/v2

and autrhorize failure:

<title>Error 403 org.apache.hadoop.security.authentication.util.SignerException: Invalid signature</title>

HTTP ERROR 403

Problem accessing /druid/v2. Reason:

    org.apache.hadoop.security.authentication.util.SignerException: Invalid signature

---

Powered by Jetty:// 9.4.10.v20180503

---

[github-actions]

This issue has been marked as stale due to 280 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If this issue is still
relevant, please simply write any comment. Even if closed, you can still revive the
issue at any time or discuss it on the dev@druid.apache.org list.
Thank you for your contributions.

[github-actions]

This issue has been closed due to lack of activity. If you think that
is incorrect, or the issue requires additional review, you can revive the issue at
any time.