Is your feature request related to a problem? Please describe.
When benchmarking BigQuery and SedonaDB's geography implementation, I noticed that some queries return large numbers of rows, or numbers of rows that grow with the scale factor.
- q5 returns over 3 million rows for at SF10 (300,000 ish for SF1), almost 30 million for SF100.
- q7 returns 6 million rows (SF1), 60 million rows (SF10), 600 million rows (SF100)
- q8 returns 10,000 rows (SF10), 70,000 rows (SF100)
Describe the solution you'd like
I'd like the number of rows to be bounded (less than 10,000 / 100 MB would be ideal). Issuing queries that return large numbers of results often measures bandwidth between the driver and the engine, and, while these types of queries are common, the usual pathway is to write the result directly to storage rather than materialize the result.
Describe alternatives you've considered
Right now harnesses have to be careful not to materialize an entire result (e.g., by fertching batch or rowwise and discarding the result).
Additional context
Add any other context or screenshots about the feature request here.
Is your feature request related to a problem? Please describe.
When benchmarking BigQuery and SedonaDB's geography implementation, I noticed that some queries return large numbers of rows, or numbers of rows that grow with the scale factor.
Describe the solution you'd like
I'd like the number of rows to be bounded (less than 10,000 / 100 MB would be ideal). Issuing queries that return large numbers of results often measures bandwidth between the driver and the engine, and, while these types of queries are common, the usual pathway is to write the result directly to storage rather than materialize the result.
Describe alternatives you've considered
Right now harnesses have to be careful not to materialize an entire result (e.g., by fertching batch or rowwise and discarding the result).
Additional context
Add any other context or screenshots about the feature request here.