export in chunks?

[sashk8]

Hi!

First of all - thank you for this great tool. :)

Is there a way to make the export in chunks?

I would like to define maximum amount of rows per file - so that the exported data is written into multiple files.

[pacman82]

Hello @sashk8 ,

First of all - thank you for this great tool. :)

Thank you

Is there a way to make the export in chunks?

There are two ways even, to limit the size of each individual file:

--row-groups-per-file and --file-size-threshold. odbc2parquet query --help will tell you more.

I would be happy to learn if these can solve your problem.

Cheers, Markus

[sashk8]

Hi @pacman82,

Thank you. This solves my problem completely. :) I didn't know that all these parameters exist.

Great tool. I was testing several ways how to export data from MSSQL to Parquet and this tool is by far the fastest. 👍

Best regards,
Sašo

[pacman82]

Happy to hear that