Skip to content

Provide a better error message for 410 Gone errors from GCS #19659

@damccorm

Description

@damccorm

For example 410 errors might get raised here.

com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:431)
        at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:289)
        at com.google.cloud.dataflow.sdk.io.FileBasedSink$FileBasedWriter.close(FileBasedSink.java:571)
        at com.google.cloud.dataflow.sdk.io.FileBasedSink$FileBasedWriter.close(FileBasedSink.java:474)
        at com.google.cloud.dataflow.sdk.io.Write$Bound$WriteBundles.finishBundle(Write.java:202)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone

 

Example client side error message that gives a better description to the end-user: "GCS upload of processed data transiently failed and will need to be retried. Some produced data might need to be recomputed"

Imported from Jira BEAM-7549. Original Jira may contain additional context.
Reported by: chamikara.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions