Formalize Object State Machine and Recovery Logic

[irq0]

What needs to be done

To define recovery logic and error handling we need to formalize the object state machine, especially the transitions and tasks that must happen when a data path encounters an object in a state.

We have the following object states:
OPEN,
WRITING,
COMMITTED,
LOCKED,
DELETED,

Example Error Conditions:

Crash during atomic writer leaves an object in OPEN/WRITING. When a client retries, what happens to the old object. Do we resume? Do we delete it? Do we create a new one? Does the GC clean up OPEN/WRITING after a while?

Client starts a PUT, but disappears mid transfer timing out the connection.

Why it needs to be done

Acceptance Criteria

Additional Information

[irq0]

https://github.com/aquarist-labs/s3gw/pull/497

[jhmarina]

This has already been addressed