Incomplete error object for streaming inserts

[piotr-sobczyk]

I'm using nodejs-bigquery version 1.0.

I'm using the following code to stream records into the database:

module.exports.sendToBigQuery = (rows) => {
    bigquery
        .dataset(DATASET_NAME)
        .table(TABLE_NAME)
        .insert(rows)
        .catch(err => {
            if (err && err.name === 'PartialFailureError') {
                if (err.errors && err.errors.length > 0) {
                    console.log('Insert errors:');
                    err.errors.forEach(err => console.error(err));
                }
            } else {
                console.error('ERROR:', err);
            }
        });
};

Unfortunately, whenever my data doesn't match the schema, all I'm getting is this cryptic error object: { errors: [ { message: 'no such field.', reason: 'invalid' } ],

There is no location field that would tell me which field is missing which makes debugging this code a nightmare for more complex schemas.

Is there any way to enable debug level of errors somehow? Or it's just a bug in client implementation? Any idea how I could access this information?

[stephenplusplus]

Unfortunately, this error is coming directly from the API. We are simply returning it. All we can do is return the invalid row itself, which is alongside the errors array, e.g.

var rowWithInvalidField = {
  property: 'a',
  extraProperty: 'b'
}

table.insert(rowWithExtraField, partialFailureError => { 
  partialFailureError.errors.forEach(console.log)
  // {
  //   row: {
  //     property: 'a',
  //     extraProperty: 'b'
  //   },
  //   errors: [
  //     {
  //       message: 'no such field.',
  //       reason: 'invalid'
  //     }
  //   ]
  // }
})

You can find the BigQuery team members from the links on the Support page, who may be able to assist further.

[stephenplusplus]

Linking the report on the official issue tracker: https://issuetracker.google.com/issues/75704775