Random Auth Errors with BigQuery

[MarkHerhold]

Insert calls to BigQuery randomly result in auth errors.

I have a job that is responsible for sending data in bulk to BigQuery by using .insert([ 'lots', 'of', 'items' ]) repeatedly in synchronous fashion. This process goes on for hours at a time, calling insert after the prior operation finishes.

After a few hours, my calls to BigQuery abruptly fail. I'm not sure if this issue is time-related or the call itself randomly fails. It does seem to occur after a few hours. This happens repeatedly.

Error:

Unhandled promise rejection (rejection id: 1): ApiError: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

Environment details

• OS: OSX El Capitan
• Node.js version: 7.7.3
• npm version: 4.1.2
• @google-cloud/bigquery version: 0.8.0

Steps to reproduce

The above description tells it all, but I'm working on a script. I'll update this if I can find a way to reproduce in a more predictable manner.

[lukesneeringer]

@stephenplusplus This smells like a rate limit to me, other than the error message. Might this be a case where we are clobbering an error?

[stephenplusplus]

I don't think we're clobbering it, just judging by the error message not being one that we've written. Before every request, we do a token grab, refreshing if necessary (that work is done by google-auth-library).

With this large amount of requests, maybe we need a larger safety window around the expiration timestamp for when the auth client makes the grab. @jmdobry does that sound possible? (please re-tag if someone else is closer to the auth library code)

[stephenplusplus]

@ofrobots wdyt?

[stephenplusplus]

Friendly re-ping.

[stephenplusplus]

This issue was moved to googleapis/google-auth-library-nodejs#133

[stephenplusplus]

We haven't been able to attract the right attention over here, so I've moved the issue to the source of the error (I think). googleapis/google-auth-library-nodejs#133

[MarkHerhold]

Can please we keep this open so I can easily track it?

[stephenplusplus]

For tracking, just subscribe to the new issue with the button in the right column: googleapis/google-auth-library-nodejs#133 -- I assumed you'd be automatically subscribed because of the @ mention of your name over there, but it wouldn't hurt to confirm.

[ghost]

I have similar problem with the auth-library.
Set up a google cloud function to monitor a bucket with frequent file changes every hour.
The error: Could not load the default credentials. Broswer to https://developers..... randomly showed up once a while.

Not sure what the problem is.