Integer overflow in CompactionTask

[gianm]

Got this when running a CompactionTask, it looks like it was trying to compute maxRowsPerSegment. Probably it should have some reasonable maximum.

2019-02-02T02:56:28,778 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner - Exception while running task[AbstractTask{id='compact_ds-1_2019-02-02T02:56:01.131Z', groupId='compact_ds-1_2019-02-02T02:56:01.131Z', taskResource=TaskResource{availabilityGroup='compact_ds-1_2019-02-02T02:56:01.131Z', requiredCapacity=1}, dataSource='ds-1', context={priority=25}}]
java.lang.ArithmeticException: integer overflow
	at java.lang.Math.toIntExact(Math.java:1011) ~[?:1.8.0_131]
	at org.apache.druid.indexing.common.task.CompactionTask$PartitionConfigurationManager.computeTuningConfig(CompactionTask.java:712) ~[druid-indexing-service-0.13.0-incubating-iap5.jar:0.13.0-incubating-iap5]
	at org.apache.druid.indexing.common.task.CompactionTask.createIngestionSchema(CompactionTask.java:325) ~[druid-indexing-service-0.13.0-incubating-iap5.jar:0.13.0-incubating-iap5]
	at org.apache.druid.indexing.common.task.CompactionTask.run(CompactionTask.java:240) ~[druid-indexing-service-0.13.0-incubating-iap5.jar:0.13.0-incubating-iap5]
	at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:422) [druid-indexing-service-0.13.0-incubating-iap5.jar:0.13.0-incubating-iap5]
	at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:394) [druid-indexing-service-0.13.0-incubating-iap5.jar:0.13.0-incubating-iap5]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131]

[jihoonson]

This is because maxRowsPerSegment of indexTuningConfig is an integer. Do you know what were targetCompactionSizeBytes, # of rows per segment, and segment size?

[gianm]

I don't know, sorry. It was reported to me by someone else.

[gianm]

Perhaps the right thing to do is still throw an error, but with a better message.