Importing file to BigQuery fails when apiResponse is undefined

[sonicoder86]

Environment details

• OS: Linux Ubuntu 14.04
• Node.js version: 6.9.1
• npm version: 3.10.3
• google-cloud-node version: 0.37.0

Steps to reproduce

Uploaded a file to Google Cloud Storage
Started an import job to a BigQuery table with truncate (importAsync)
Got the following error:

 TypeError: Cannot read property 'status' of undefined at /app/node_modules/gcloud/lib/bigquery/job.js:337:20 at /app/node_modules/gcloud/lib/common/service-object.js:249:7 at Object.handleResp (/app/node_modules/gcloud/lib/common/util.js:133:3) at /app/node_modules/gcloud/lib/common/util.js:453:12 at Request.onResponse [as _callback] (/app/node_modules/retry-request/index.js:99:9) at self.callback (/app/node_modules/request/request.js:187:22) at emitOne (events.js:96:13) at Request.emit (events.js:188:7) at Timeout._onTimeout (/app/node_modules/request/request.js:762:12) at ontimeout (timers.js:365:14) at tryOnTimeout (timers.js:237:5) at Timer.listOnTimeout (timers.js:207:5)

Most of the time it works, we have been using it for weeks, now it is the first time it happened.

[stephenplusplus]

We recently changed some internal code and must have let this slip. I'll take a look at it now and try to release a fix asap.

[stephenplusplus]

Oh, just realized you're running google-cloud-node 0.37.0. Can you try upgrading to the latest, 0.44.0? Note, the module is now called google-cloud-node as opposed to gcloud.

Here are each release's release notes so you can watch for any breaking changes your app might need to account for. I've included the changes which affect the core module & BigQuery:

0.38.3 (renamed to google-cloud): https://github.com/GoogleCloudPlatform/google-cloud-node/releases/tag/v0.38.3

- require('gcloud')
+ require('google-cloud')

0.39.0: https://github.com/GoogleCloudPlatform/google-cloud-node/releases/tag/v0.39.0
0.40.0: https://github.com/GoogleCloudPlatform/google-cloud-node/releases/tag/v0.40.0
0.41.2: https://github.com/GoogleCloudPlatform/google-cloud-node/releases/tag/v0.41.2
0.42.0 (promise support): https://github.com/GoogleCloudPlatform/google-cloud-node/releases/tag/v0.42.2

- bigquery.getDatasets()
+ bigquery.getDatasetsStream()
    .on('data', function(dataset) {})

- bigquery.getJobs()
+ bigquery.getJobsStream()
    .on('data', function(job) {})

- bigquery.query('SELECT ...')
+ bigquery.createQueryStream('SELECT ...')
    .on('data', function(row) {})

var dataset = bigquery.dataset('my-dataset-name');

- dataset.getTables()
+ dataset.getTablesStream()
    .on('data', function(table) {})

- dataset.query('SELECT ...')
+ dataset.createQueryStream('SELECT ...')
    .on('data', function(row) {})

var table = dataset.table('my-table-name');

- table.getRows()
+ table.createReadStream()
    .on('data', function(row) {})

- table.query('SELECT ...')
+ table.createQueryStream('SELECT ...')
    .on('data', function(row) {})

0.43.0: https://github.com/GoogleCloudPlatform/google-cloud-node/releases/tag/v0.43.0
0.44.0 (introduces the PartialFailureError): https://github.com/GoogleCloudPlatform/google-cloud-node/releases/tag/v0.44.0

- table.insert([row1, row2, row3], function(err, insertErrors, apiResponse) {})
+ table.insert([row1, row2, row3], function(err, apiResponse) {}) // `err` contains the insertErrors

See the example section on the docs for table.insert() for the new way to handle these errors.

[sonicoder86]

Thanks, we upgraded, will reopen if anything happens.

Note: gcloud package should be deprecated in npm.

[stephenplusplus]

deprecating in npm means npm install leaves a warning. The gcloud package is "deprecated" in this sense.

[gergaczd]

Hi @stephenplusplus ,

We got the same error message using google-cloud version 0.44.0 (all the other settings are the same, also the same project).

The stack trace points to this location (@google-cloud/bigquery/src/job.js:320):
if (apiResponse.status && apiResponse.status.errors) {

I think, it should check also the apiResponse existence too.

Stack trace:

TypeError: Cannot read property 'status' of undefined
    at /app/node_modules/google-cloud/node_modules/@google-cloud/bigquery/src/job.js:320:20
    at /app/node_modules/google-cloud/node_modules/@google-cloud/common/src/service-object.js:250:7
    at Object.handleResp (/app/node_modules/google-cloud/node_modules/@google-cloud/common/src/util.js:134:3)
    at /app/node_modules/google-cloud/node_modules/@google-cloud/common/src/util.js:454:12
    at Request.onResponse [as _callback] (/app/node_modules/google-cloud/node_modules/retry-request/index.js:99:9)
    at self.callback (/app/node_modules/google-cloud/node_modules/request/request.js:186:22)
    at emitOne (events.js:96:13)
    at Request.emit (events.js:188:7)
    at ClientRequest.<anonymous> (/app/node_modules/google-cloud/node_modules/request/request.js:781:16)
    at ClientRequest.g (events.js:291:16)
    at emitNone (events.js:86:13)
    at ClientRequest.emit (events.js:185:7)
    at TLSSocket.emitTimeout (_http_client.js:620:10)
    at TLSSocket.g (events.js:291:16)
    at emitNone (events.js:86:13)
    at TLSSocket.emit (events.js:185:7)
    at TLSSocket.Socket._onTimeout (net.js:339:8)
    at ontimeout (timers.js:365:14)
    at tryOnTimeout (timers.js:237:5)
    at Timer.listOnTimeout (timers.js:207:5)`

Thanks!

[stephenplusplus]

Would you mind placing another console.log here:

--- node_modules/@google-cloud/common/src/service-object.js
+++ node_modules/@google-cloud/common/src/service-object.js
@@ -246,6 +246,7 @@ ServiceObject.prototype.getMetadata = function(callback) {
   // Ensure we call the original `request` method.
   ServiceObject.prototype.request.call(this, reqOpts, function(err, resp) {
+    console.log(resp)
     if (err) {
       callback(err, null, resp);
       return;
     }

It must be undefined, but I would like to make sure. Thank you!

[gergaczd]

The error happened on deployed code. I can not reproduced it on local, and it happens only sometimes (2 times in the last 8 days).

[stephenplusplus]

Sorry about that, PR sent in #1853.

[gergaczd]

Thanks!