Druid Batch Load is Very slow

[averma111]

Druid Version:0.15

Druid cluster setup with two historical nodes. We are trying to insert 100 millions into druid in tsv format and it took 7 hrs for Druid cluster setup to insert the rows.

Any parameters we can setup to increase the load performance.

Appreciate any help!!

[gianm]

If you are doing batch loads, try making sure that there are enough parallel tasks running, and that your files are split up enough that we can launch enough parallel tasks.

[averma111]

@gianm could you please tell me what exact setting I have to change in cluster setup. I want test it first in lower environment before moving to prod

[jihoonson]

@averma111 it's maxNumConcurrentSubTasks in tuningConfig. Please check out the documentation here: https://druid.apache.org/docs/latest/ingestion/native-batch.html.

[liran-funaro]

I tried @gianm suggestions and they worked great to speed up the performance in a different project I worked on (see our ingestion schema).
We managed to increase the performance by 800% (see our performance evaluation).
But note that when increasing the number of workers, the RAM usage is increased dramatically.
If this is an issue for you, please consider checking out our improved implementation that reduces the RAM and CPU usage by over 50% during the ingestion process (#10001).

[github-actions]

This issue has been marked as stale due to 280 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If this issue is still
relevant, please simply write any comment. Even if closed, you can still revive the
issue at any time or discuss it on the dev@druid.apache.org list.
Thank you for your contributions.

[github-actions]

This issue has been closed due to lack of activity. If you think that
is incorrect, or the issue requires additional review, you can revive the issue at
any time.