-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CombiningFirehoseFactory cannot be cast to FiniteFirehoseFactory #9389
Comments
It looks like a bug since |
Hey, Sorry i couldn't respond on this earlier. It's rejected as a bad request with the following error
|
FYI - I encountered the exact same issue following the tutorial, not what you want to see while evaluating a new technology, will keep an 👁 on this. |
@gunjdesai That exception is generally thrown when the indexing spec is incorrect. Could you double check your ingestion json with the format for |
FYI - I encountered the exact same issue following the tutorial, when submit task which |
I also facing the same issue when trying on their official tutorial when submitting the task: bin/post-index-task --file quickstart/tutorial/updates-append-index.json --url http://localhost:8081 |
Encountered this as well when running said command straight out from the tutorials bin/post-index-task --file quickstart/tutorial/updates-append-index.json --url http://localhost:8081 |
Hi, |
I am attempting to combing two datasources using the CombiningFirehoseFactory
For reference, I am following the update data tutorial
My ingestion spec looks like this
{ "type" : "index", "spec" : { "dataSchema" : { "dataSource" : "updates-tutorial", "parser" : { "type" : "string", "parseSpec" : { "format" : "json", "dimensionsSpec" : { "dimensions" : [ "animal" ] }, "timestampSpec": { "column": "timestamp", "format": "iso" } } }, "metricsSpec" : [ { "type" : "count", "name" : "count" }, { "type" : "longSum", "name" : "number", "fieldName" : "number" } ], "granularitySpec" : { "type" : "uniform", "segmentGranularity" : "hour", "queryGranularity" : "minute", "intervals" : ["2018-01-01/2018-01-03"], "rollup" : false } }, "ioConfig" : { "type" : "index", "firehose" : { "type": "combining", "delegates": [ { "type" : "ingestSegment", "dataSource" : "s1", "interval" : "2018-01-01/2018-01-03" }, { "type" : "ingestSegment", "dataSource" : "s2", "interval" : "2018-01-01/2018-01-03" } ] }, "appendToExisting" : false }, "tuningConfig" : { "type" : "index", "maxRowsPerSegment" : 5000000, "maxRowsInMemory" : 25000 } } }
After reading into the code a bit, I believe that CombiningFirehoseFactory allows you to combine multiple datasources
But when i submit the task, i get the following error
2020-02-21T11:35:39,222 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.common.task.IndexTask - Encountered exception in NOT_STARTED. java.lang.ClassCastException: org.apache.druid.segment.realtime.firehose.CombiningFirehoseFactory cannot be cast to org.apache.druid.data.input.FiniteFirehoseFactory at org.apache.druid.indexing.common.task.IndexTask$IndexIOConfig.getNonNullInputSource(IndexTask.java:1148) ~[druid-indexing-service-0.17.0.jar:0.17.0] at org.apache.druid.indexing.common.task.IndexTask.runTask(IndexTask.java:477) [druid-indexing-service-0.17.0.jar:0.17.0] at org.apache.druid.indexing.common.task.AbstractBatchIndexTask.run(AbstractBatchIndexTask.java:138) [druid-indexing-service-0.17.0.jar:0.17.0] at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:419) [druid-indexing-service-0.17.0.jar:0.17.0] at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:391) [druid-indexing-service-0.17.0.jar:0.17.0] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_232] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_232] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_232] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
I am not sure, if this is a issue with the datasource of my selection or config issue.
In the interim, i have tried to alternate the datasource from
ingestSegment
tolocal
as well, but i continue to get the same error.Would really appreciate if i could get any help on this or if someone can help rectify my understanding on the matter.
The text was updated successfully, but these errors were encountered: