Skip to content

LargeObjectFile should support cleanup in case of upload error #99

@majewsky

Description

@majewsky

The internal struct implementing the LargeObjectFile interface keeps track of which segments it has uploaded since {Dynamic,Static}LargeObjectCreate. When the upload fails (either during PUT of another segment or PUT of the manifest), it is useful to know which segments were already uploaded in order to perform cleanup.

Since LargeObjectFile does not have a method to expose its segments, one can only perform this cleanup by listing the uploaded segments, i.e.

names, err := conn.ObjectNameAll(segmentContainer,
  &swift.ObjectsOpts{Prefix: segmentPrefix},
)
... handle err ...
result, err := conn.BulkDelete(segmentContainer, names)

However, this is prone to eventual consistency issues: The segments might not yet be visible in the container listing. I see two possible options:

  1. Add a method like SegmentNames() []string to the LargeObjectFile interface.
  2. Add a method Cleanup() error or Reset() error to the LargeObjectFile interface that deletes all segments which were uploaded since {Dynamic,Static}LargeObjectCreate.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions