[VL] Make velox writer queue size configurable

[yikf]

What changes were proposed in this pull request?

This PR aims to make velox writer queue size is configurable to avoid oom from being caused by too many batches.

How was this patch tested?

GA

[yikf]

cc @PHILO-HE

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[github-actions]

Run Gluten Clickhouse CI

[PHILO-HE]

@yikf, it looks good to me. Did you observe that this config can help relieve the OOM issue on your side?

BTW, please fix code style issue, see reference:
https://github.com/apache/incubator-gluten/blob/main/docs/developers/NewToGluten.md#javascala-code-style

[github-actions]

Run Gluten Clickhouse CI

[yikf]

BTW, please fix code style issue, see reference:
https://github.com/apache/incubator-gluten/blob/main/docs/developers/NewToGluten.md#javascala-code-style

thanks for the reminding of codestyle.

it looks good to me. Did you observe that this config can help relieve the OOM issue on your side?

yes, we have a scenario which trigger an oom is due to a large number of batches being held in a queue. making queue configurable may reduce oom occurrence, but it will also block the query thread, which is not optimal, but it is the fastest way to prevent oom. then we can implement push-based writing to refactor the logic of this piece