Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Support use of `fields`, and use it to minimize data transfer for some paging/polling operations. #153
(This is an offshoot of #58.)
List operations in the BQ API generally return all information about each object by default -- for trying to examine lists of objects, this is great. However, if we're (say) polling for job completion, or looking for a specific table/dataset, it's often wasteful to pull all the data every time. (In particular, it's annoying for manual investigation/debugging.)
IWBN to use
I am also vexed by how to handle fields. In googledrive, we ultimately decided to not handle them, i.e. to use the same fields everywhere (I think the default set?), until that proved to be problematic. So far, so good.
However in googlesheets4, I must handle fields, so am thinking about it once again.
@hadley You closed this with a really huge commit. Did you do anything clever with fields that I should study?
I don't think there will be any take aways for google sheets here because the way in which I need to use fields is quite different. For example, most bigquery operations take place by inserting a job into a queue, and then regularly polling until it's done. I'm now using fields so that when I poll I just get whether or not it's complete, not a bunch of other information that I don't use.