Add "history" mode to versioned_writes middleware

This change introduces the concept of a "versioning mode" for
versioned_writes. The following modes are supported:

 * stack

    When deleting, check whether any previous versions exist in the
    versions container. If none is found, the object is deleted. If the
    most-recent version in the versions container is not a delete
    marker, it is copied into the versioned container (overwriting the
    current version if one exists) and then deleted from the versions
    container. This preserves the previous behavior.

    If the most-recent version in the versions container is a delete
    marker and a current version exists in the versioned container, the
    current version is deleted. If the most-recent version in the
    versions container is a delete marker and no current version exists
    in the versioned container, we copy the next-most-recent version
    from the versions container into the versioned container (assuming
    it exists and is not a delete marker) and delete both the
    most-recent version (i.e., the delete marker) and the just-copied
    next-most-recent version from the versions container.

    With this mode, DELETEs to versioned containers "undo" operations
    on containers. Previously this was limited to undoing PUTs, but now
    it will also undo DELETEs performed while in "history" mode.

 * history

    When deleting, check whether a current version exists in the
    versioned container. If one is found, it is copied to the versions
    container. Then an empty "delete marker" object is also put into the
    versions container; this records when the object was deleted.
    Finally, the original current version is deleted from the versioned
    container. As a result, subsequent GETs or HEADs will return a 404,
    and container listings for the versioned container do not include
    the object.

    With this mode, DELETEs to versioned containers behave like DELETEs
    to other containers, but with a history of what has happened.

Clients may specify (via a new X-Versions-Mode header) which mode a
container should use. By default, the existing "stack" mode is used.

Upgrade consideration:
======================

Clients should not use the "history" mode until all proxies in the
cluster have been upgraded. Attempting to use the "history" mode during
a rolling upgrade may result in some requests being served by proxies
running old code (which necessarily uses the "stack" mode), leading to
data loss.

Change-Id: I555dc17fefd0aa9ade681aa156da24e018ebe74b

Author: tipabu

api-ref/source/parameters.yaml

@@ -720,21 +720,24 @@ X-Trans-Id-Extra:
   type: string
 X-Versions-Location:
   description: |
-    Enables versioning on this container. The value
-    is the name of another container. You must UTF-8-encode and then
-    URL-encode the name before you include it in the header. To
-    disable versioning, set the header to an empty string.
+    The URL-encoded UTF-8 representation of the container that stores
+    previous versions of objects. If not set, versioning is disabled
+    for this container. For more information about object versioning,
+    see `Object versioning <http://docs.openstack.org/developer/
+    swift/api/object_versioning.html>`_.
   in: header
   required: false
   type: string
-X-Versions-Location_1:
+X-Versions-Mode:
   description: |
-    Enables versioning on this container. The value
-    is the name of another container. You must UTF-8-encode and then
-    URL-encode the name before you include it in the header. To
-    disable versioning, set the header to an empty string.
+    The versioning mode for this container. The value must be either
+    ``stack`` or ``history``. If not set, ``stack`` mode will be used.
+    This setting has no impact unless ``X-Versions-Location`` is set
+    for the container. For more information about object versioning,
+    see `Object versioning <http://docs.openstack.org/developer/
+    swift/api/object_versioning.html>`_.
   in: header
-  required: true
+  required: false
   type: string
 
 # variables in path

api-ref/source/storage-container-services.inc

@@ -172,6 +172,7 @@ Request
    - X-Container-Sync-To: X-Container-Sync-To
    - X-Container-Sync-Key: X-Container-Sync-Key
    - X-Versions-Location: X-Versions-Location
+   - X-Versions-Mode: X-Versions-Mode
    - X-Container-Meta-name: X-Container-Meta-name
    - X-Container-Meta-Access-Control-Allow-Origin: X-Container-Meta-Access-Control-Allow-Origin
    - X-Container-Meta-Access-Control-Max-Age: X-Container-Meta-Access-Control-Max-Age
@@ -302,6 +303,7 @@ Request
    - X-Container-Sync-To: X-Container-Sync-To
    - X-Container-Sync-Key: X-Container-Sync-Key
    - X-Versions-Location: X-Versions-Location
+   - X-Versions-Mode: X-Versions-Mode
    - X-Remove-Versions-Location: X-Remove-Versions-Location
    - X-Container-Meta-name: X-Container-Meta-name
    - X-Container-Meta-Access-Control-Allow-Origin: X-Container-Meta-Access-Control-Allow-Origin
@@ -409,6 +411,7 @@ Response Parameters
    - Content-Type: Content-Type
    - X-Container-Meta-Quota-Bytes: X-Container-Meta-Quota-Bytes
    - X-Versions-Location: X-Versions-Location
+   - X-Versions-Mode: X-Versions-Mode
 
 
 

doc/source/api/object_versioning.rst

@@ -6,19 +6,19 @@ You can store multiple versions of your content so that you can recover
 from unintended overwrites. Object versioning is an easy way to
 implement version control, which you can use with any type of content.
 
-Note
-~~~~
+.. note::
+    You cannot version a large-object manifest file, but the large-object
+    manifest file can point to versioned segments.
 
-You cannot version a large-object manifest file, but the large-object
-manifest file can point to versioned segments.
+.. note::
+    It is strongly recommended that you put non-current objects in a
+    different container than the container where current object versions
+    reside.
 
-It is strongly recommended that you put non-current objects in a
-different container than the container where current object versions
-reside.
-
-To enable object versioning, the cloud provider sets the
-``allow_versions`` option to ``TRUE`` in the container configuration
-file.
+To allow object versioning within a cluster, the cloud provider should add the
+``versioned_writes`` filter to the pipeline and set the
+``allow_versioned_writes`` option to ``true`` in the
+``[filter:versioned_writes]`` section of the proxy-server configuration file.
 
 The ``X-Versions-Location`` header defines the
 container that holds the non-current versions of your objects. You
@@ -29,13 +29,21 @@ object versioning for all objects in the container. With a comparable
 container automatically create non-current versions in the ``archive``
 container.
 
-Here's an example:
+The ``X-Versions-Mode`` header defines the behavior of ``DELETE`` requests to
+objects in the versioned container. In the default ``stack`` mode, deleting an
+object will restore the most-recent version from the ``archive`` container,
+overwriting the curent version. Alternatively you may specify ``history``
+mode, where deleting an object will copy the current version to the
+``archive`` then remove it from the ``current`` container.
+
+Example Using ``stack`` Mode
+----------------------------
 
 #.   Create the ``current`` container:
 
    .. code::
 
-       # curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: archive"
+       # curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: archive" -H "X-Versions-Mode: stack"
 
    .. code::
 
@@ -70,7 +78,7 @@ Here's an example:
 
    .. code::
 
-       <length><object_name><timestamp>
+       <length><object_name>/<timestamp>
 
    Where ``length`` is the 3-character, zero-padded hexadecimal
    character length of the object, ``<object_name>`` is the object name,
@@ -117,12 +125,10 @@ Here's an example:
 
        009my_object/1390512682.92052
 
-Note
-~~~~
-
-   A **POST** request to a versioned object updates only the metadata
-   for the object and does not create a new version of the object. New
-   versions are created only when the content of the object changes.
+   .. note::
+      A **POST** request to a versioned object updates only the metadata
+      for the object and does not create a new version of the object. New
+      versions are created only when the content of the object changes.
 
 #. Issue a **DELETE** request to a versioned object to remove the
    current version of the object and replace it with the next-most
@@ -163,21 +169,163 @@ Note
    on it. If want to completely remove an object and you have five
    versions of it, you must **DELETE** it five times.
 
-#. To disable object versioning for the ``current`` container, remove
-   its ``X-Versions-Location`` metadata header by sending an empty key
-   value.
+Example Using ``history`` Mode
+----------------------------
+
+#.   Create the ``current`` container:
+
+   .. code::
+
+       # curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: archive" -H "X-Versions-Mode: history"
+
+   .. code::
+
+       HTTP/1.1 201 Created
+       Content-Length: 0
+       Content-Type: text/html; charset=UTF-8
+       X-Trans-Id: txb91810fb717347d09eec8-0052e18997
+       Date: Thu, 23 Jan 2014 21:28:55 GMT
+
+#. Create the first version of an object in the ``current`` container:
+
+   .. code::
+
+       # curl -i $publicURL/current/my_object --data-binary 1 -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token"
+
+   .. code::
+
+       HTTP/1.1 201 Created
+       Last-Modified: Thu, 23 Jan 2014 21:31:22 GMT
+       Content-Length: 0
+       Etag: d41d8cd98f00b204e9800998ecf8427e
+       Content-Type: text/html; charset=UTF-8
+       X-Trans-Id: tx5992d536a4bd4fec973aa-0052e18a2a
+       Date: Thu, 23 Jan 2014 21:31:22 GMT
+
+   Nothing is written to the non-current version container when you
+   initially **PUT** an object in the ``current`` container. However,
+   subsequent **PUT** requests that edit an object trigger the creation
+   of a version of that object in the ``archive`` container.
+
+   These non-current versions are named as follows:
+
+   .. code::
+
+       <length><object_name>/<timestamp>
+
+   Where ``length`` is the 3-character, zero-padded hexadecimal
+   character length of the object, ``<object_name>`` is the object name,
+   and ``<timestamp>`` is the time when the object was initially created
+   as a current version.
+
+#. Create a second version of the object in the ``current`` container:
 
    .. code::
 
-       # curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: "
+       # curl -i $publicURL/current/my_object --data-binary 2 -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token"
 
    .. code::
 
-       HTTP/1.1 202 Accepted
-       Content-Length: 76
+       HTTP/1.1 201 Created
+       Last-Modified: Thu, 23 Jan 2014 21:41:32 GMT
+       Content-Length: 0
+       Etag: d41d8cd98f00b204e9800998ecf8427e
        Content-Type: text/html; charset=UTF-8
-       X-Trans-Id: txe2476de217134549996d0-0052e19038
-       Date: Thu, 23 Jan 2014 21:57:12 GMT
+       X-Trans-Id: tx468287ce4fc94eada96ec-0052e18c8c
+       Date: Thu, 23 Jan 2014 21:41:32 GMT
+
+#. Issue a **GET** request to a versioned object to get the current
+   version of the object. You do not have to do any request redirects or
+   metadata lookups.
+
+   List older versions of the object in the ``archive`` container:
+
+   .. code::
+
+       # curl -i $publicURL/archive?prefix=009my_object -X GET -H "X-Auth-Token: $token"
+
+   .. code::
+
+       HTTP/1.1 200 OK
+       Content-Length: 30
+       X-Container-Object-Count: 1
+       Accept-Ranges: bytes
+       X-Timestamp: 1390513280.79684
+       X-Container-Bytes-Used: 0
+       Content-Type: text/plain; charset=utf-8
+       X-Trans-Id: tx9a441884997542d3a5868-0052e18d8e
+       Date: Thu, 23 Jan 2014 21:45:50 GMT
+
+       009my_object/1390512682.92052
+
+   .. note::
+      A **POST** request to a versioned object updates only the metadata
+      for the object and does not create a new version of the object. New
+      versions are created only when the content of the object changes.
+
+#. Issue a **DELETE** request to a versioned object to copy the
+   current version of the object to the archive container then delete it from
+   the current container. Subsequent **GET** requests to the object in the
+   current container will return 404 Not Found.
+
+   .. code::
+
+       # curl -i $publicURL/current/my_object -X DELETE -H "X-Auth-Token: $token"
+
+   .. code::
+
+       HTTP/1.1 204 No Content
+       Content-Length: 0
+       Content-Type: text/html; charset=UTF-8
+       X-Trans-Id: tx006d944e02494e229b8ee-0052e18edd
+       Date: Thu, 23 Jan 2014 21:51:25 GMT
+
+   List older versions of the object in the ``archive`` container::
+
+   .. code::
+
+       # curl -i $publicURL/archive?prefix=009my_object -X GET -H "X-Auth-Token: $token"
+
+   .. code::
+
+       HTTP/1.1 200 OK
+       Content-Length: 90
+       X-Container-Object-Count: 3
+       Accept-Ranges: bytes
+       X-Timestamp: 1390513280.79684
+       X-Container-Bytes-Used: 0
+       Content-Type: text/html; charset=UTF-8
+       X-Trans-Id: tx044f2a05f56f4997af737-0052e18eed
+       Date: Thu, 23 Jan 2014 21:51:41 GMT
+
+       009my_object/1390512682.92052
+       009my_object/1390512692.23062
+       009my_object/1390513885.67732
+
+   In addition to the two previous versions of the object, the archive
+   container has a "delete marker" to record when the object was deleted.
+
+   To permanently delete a previous version, issue a **DELETE** to the version
+   in the archive container.
+
+Disabling Object Versioning
+---------------------------
+
+To disable object versioning for the ``current`` container, remove
+its ``X-Versions-Location`` metadata header by sending an empty key
+value.
+
+.. code::
+
+    # curl -i $publicURL/current -X PUT -H "Content-Length: 0" -H "X-Auth-Token: $token" -H "X-Versions-Location: "
+
+.. code::
+
+    HTTP/1.1 202 Accepted
+    Content-Length: 76
+    Content-Type: text/html; charset=UTF-8
+    X-Trans-Id: txe2476de217134549996d0-0052e19038
+    Date: Thu, 23 Jan 2014 21:57:12 GMT
 
-       <html><h1>Accepted</h1><p>The request is accepted for processing.</p></html>
+    <html><h1>Accepted</h1><p>The request is accepted for processing.</p></html>