asset-inventory : job failed

[sdenef-adeo]

Hello,

Where trying to add new CAI supported asset types to our inventory. But Dataflow jobs failed for the following ones:

• datamigration.googleapis.com/MigrationJob
• networkconnectivity.googleapis.com/Spoke

See errors below.

datamigration.googleapis.com/MigrationJob

Error from Dataflow job logs:

google.api_core.exceptions.BadRequest: 
400 Error while reading data, error message: 
JSON table encountered too many errors, giving up. Rows: 1; errors: 1. 
Please look into the errors[] collection for more details. 
File: gs://...../networkconnectivity.googleapis.com/Spoke.0.json 
[while running 'load_to_bigquery/load_to_bigquery']

Error from BigQuery logs:

Error while reading data, error message: 
JSON parsing error in row starting at position 0: No such field: resource.data.error.details.detail. 
File: gs://...../datamigration.googleapis.com/MigrationJob.0.json

networkconnectivity.googleapis.com/Spoke

Error from Dataflow job logs:

google.api_core.exceptions.BadRequest: 
400 Error while reading data, error message: 
JSON table encountered too many errors, giving up. Rows: 1; errors: 1. 
Please look into the errors[] collection for more details. 
File: gs://...../networkconnectivity.googleapis.com/Spoke.0.json 
[while running 'load_to_bigquery/load_to_bigquery']

Error from BigQuery logs:

Error while reading data, error message: 
JSON parsing error in row starting at position 0: No such field: resource.data.linkedRouterApplianceInstances.name. 
File: gs://...../networkconnectivity.googleapis.com/Spoke.0.json

[bmenasha]

Is this repeatable? Is it possible for you to grant me access to the export file for me to recreate? (bmenasha@google.com) thanks

[sdenef-adeo]

Hello,

Yes, it is repeatable.

I sent you the files via email as they are small (a few KB).

Regards,
Sébastien

[bmenasha]

Thanks Sebastian, I have a fix for which I tested and the "latest" pipeline is now running but it has yet to be submitted as a pull request. Will submit the code fix in another pull request after PR #905 is reviewed and merged to avoid conflicts.
thanks

[agold-rh]

@bmenasha PR #905 has been merged so feel free to submit your code fix.