Verbose query statistics

[ozhyrenkov]

Hello!
Thanks for your awesome package I managed to set dozen of automated scripts which use Athena quite intensive. So, thank you so much!

It's a feature request or\and idea for future work rather then issue.
dding the show_statistics extra param in dbGetQuery() or additional function dbGetQueryStatistics() that uses the QueryExecution['Statistics'] part of boto3 get_query_execution function's response.

Why it might be helpful? The pricing of Athena for now is $5 per 1TB of scanned data, and Athena itself is designed to query huge amounts of data. It might be helpful for managing the operations' costs in complex environments.

[DyfanJones]

I am really glad you are enjoying the package. I am happy to put these new features in the package

[DyfanJones]

Currently dbGetQuery returns some of the Statistics from boto3 get_query_execution with the amount of Data Athena scanned. This should be fairly easy to implement.

[DyfanJones]

Created initial implementation of returning statistics from queries to AWS Athena (https://github.com/DyfanJones/RAthena/tree/query-stats). New function dbStatistics and parameter statistics in dbGetQuery address this in the following way:

library(DBI)
library(RAthena)

con = dbConnect(athena())

# method 1:
res = dbSendQuery(con, "show databases")
dbStatistics(res)
dbClearResult(res)

# method 2:
dbGetQuery(con, "show databases", statistics = TRUE)

Will need to check unit tests to see if this has any implementations. Plus need to double check dplyr integration.

[DyfanJones]

PR #68 passed all current unit tests

[ozhyrenkov]

Thank You so much, it's working perfectly. Will share with you our use-cases in some time :)

[DyfanJones]

These features will be pushed to cran shortly