Skip to content

Log RPC requests when they fails to improve error messages #18037

@kennknowles

Description

@kennknowles

For example, BigQuery sometime doesn't provide enough information in the respond. Users can only know what is invalid in the request, if we log the it.

Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid value for: String is not a valid value",
"reason" : "invalid"
} ],
"message" : "Invalid value for: String is not a valid value"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.dataflow.sdk.util.BigQueryTableInserter.tryCreateTable(BigQueryTableInserter.java:427)
at com.google.cloud.dataflow.sdk.util.BigQueryTableInserter.tryCreateTable(BigQueryTableInserter.java:416)
at com.google.cloud.dataflow.sdk.util.BigQueryTableInserter.getOrCreateTable(BigQueryTableInserter.java:373)
at com.google.cloud.dataflow.sdk.io.BigQueryIO$StreamingWriteFn.getOrCreateTable(BigQueryIO.java:2127)
at com.google.cloud.dataflow.sdk.io.BigQueryIO$StreamingWriteFn.finishBundle(BigQueryIO.java:2099)
at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.finishBundle(DoFnRunnerBase.java:158)

Imported from Jira BEAM-482. Original Jira may contain additional context.
Reported by: peihe0@gmail.com.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions