An option to export CSV file with 10,000 rows limitation

[FromPAUS]

Is it possible to have a feature to allow user to download CSV without maximum of 10,000 rows limitation? Can someone give me some hints how to implement such feature? Where will be the best place (code file) to add a checkbox to enable such feature? Where will be the best place I can utilize "scroll" feature to retrieve all the data? I am not very familiar with ES and Kibana source code layout. But eager to have such feature implemented. Thanks!

[zhongnansu]

@fabioued Hi Fabrice, could you help with this?

[anirudha]

We thought about this a lot during the design phase and finally decided there is a distinction between reporting and a data export feature.

Currently, the system allows for a maximum of 10000 records returned with reporting to align with ES query defaults.
we can take 2 options

1. make it a config. param. ( this will enable some users to shoot themself in the foot if they use it incorrectly )
2. users can use paging ( manually or with an API )
3. we provide a deep paging reporting dump i.e large no limit export feature.

let me know your thoughts...

[nickmy9729]

I would go with 1 short term. This would align with the xpack reporting feature currently available. They use a param xpack.reporting.csv.maxSizeBytes to the kibana.yml file. I really love the idea of number 3, but realize it would take more work.

[zhongnansu]

This issue is moved to https://github.com/opensearch-project/dashboards-reports/issues/12