Support query response dump in different formats and location

[xiangfu0]

From the query endpoint, Pinot currently only supports customized JSON format.

It would be great to support:

1. user can specify the response format, e.g. CSV
2. user can specify a file to dump the query response to, e.g. local or remote(s3) file or directory.
3. user can specify the compression algorithm for output file.

This can be implemented in broker query API or we can do all the reformat and response dump using org.apache.pinot.tools.admin.command.PostQueryCommand

[khushbu14]

@xiangfu0 It would be great to extend the support to include compression as well for the output.

[xiangfu0]

@xiangfu0 It would be great to extend the support to include compression as well for the output.

cool! Added to bullet 3

[walterddr]

this sounds like a cool feature. I would like to try implementing it. To clarify a few details.

• this feature only applies on the admin query layer?
  • is there a particular place we have in mind if we add this to broker query API layer? (passing in as QueryOptions?)
• the plan here is to only support several out of the box format/algorithms at this stage, yes?
  • do we plan to support SPI for users to override output format and/or compress algorithms in the future?

thx :-)