Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document Versioning #291

Merged
merged 33 commits into from
Mar 17, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
a778a33
Add support for versioned documents
joshvillbrandt Mar 7, 2014
a2b1421
Add support for ?version=diffs
joshvillbrandt Mar 7, 2014
c6533ed
Clean up existing versioning functionality
joshvillbrandt Mar 7, 2014
5bcd67b
Add version number to response in write requests
joshvillbrandt Mar 12, 2014
b301f9c
Fix failing unit tests
joshvillbrandt Mar 12, 2014
220df85
Merge branch 'develop' into versioning
joshvillbrandt Mar 12, 2014
e7b9f45
Add test unit plan for versioning
joshvillbrandt Mar 12, 2014
3991633
Implement TestLateVersioning unit tests
joshvillbrandt Mar 13, 2014
94d68f7
Enable version control before test data is entered
joshvillbrandt Mar 13, 2014
10d1e9a
Add version controlled contacts after setUp instead
joshvillbrandt Mar 13, 2014
41e46a6
Add TestPartialVersioning write method tests and fix patch
joshvillbrandt Mar 14, 2014
9fce172
Complete TestPartialVersioning tests
joshvillbrandt Mar 14, 2014
6e4bf9d
Add primary tests for TestCompleteVersioning
joshvillbrandt Mar 14, 2014
53dbc1e
Add a nice error for a bad projection with a version query
joshvillbrandt Mar 14, 2014
9d37001
Implement ?version= test cases
joshvillbrandt Mar 14, 2014
c3dd1d9
Add stubs for projection test cases
joshvillbrandt Mar 14, 2014
fd9a732
Clean up document versioning description
joshvillbrandt Mar 14, 2014
fa4a248
Merge branch 'develop' into versioning
joshvillbrandt Mar 14, 2014
3ea276d
Add projection tests
joshvillbrandt Mar 14, 2014
2de0e18
Add stubs for data relation version
joshvillbrandt Mar 14, 2014
51a07a3
Add support for data_relation with version
joshvillbrandt Mar 16, 2014
202bf5c
Complete implementation of version data relations to non-versioned fi…
joshvillbrandt Mar 16, 2014
2e1539b
Add resource versioning defaults check
joshvillbrandt Mar 16, 2014
269a50b
Complete implementation of version data relations to versioned fields
joshvillbrandt Mar 16, 2014
878f45a
Add embedded test for version data relation
joshvillbrandt Mar 16, 2014
2d1bad9
Add support for embedded resources from versioned data relations
joshvillbrandt Mar 16, 2014
54066c0
Merge branch 'develop' into versioning
joshvillbrandt Mar 16, 2014
8572253
Fix TravisCI runtime error
joshvillbrandt Mar 16, 2014
e473169
Format as pep8
joshvillbrandt Mar 16, 2014
21ea852
Format as pep8
joshvillbrandt Mar 16, 2014
ee11a49
Python3 support for dict_keys to list
joshvillbrandt Mar 16, 2014
3a1e1a3
Add doc ver delete test cases
joshvillbrandt Mar 16, 2014
7d6d6bd
Add support for data_relation with late document versioning
joshvillbrandt Mar 17, 2014
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
36 changes: 35 additions & 1 deletion docs/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,35 @@ uppercase.
otherwise. See :ref:`jsonxml`. Defaults to
``True``.

``VERSIONING`` Enabled documents version control when``True``. Can be erridden by resource settings. Defaults to ``False``.

``VERSIONS`` Suffix added to the name of the primary
collection to create the name of the shadow
collection to store document versions. Defaults
to ``_versions``. When ``VERSIONING`` is enabled
, a collection such as ``myresource_versions``
would be created for a resource with a
datasource of ``myresource``.

``VERSION_PARAM`` The URL query parameter used to access the
specific version of a document. Defaults to
``version``. Omit this parameter to get the
latest version of a document or use
`?version=all`` to get a list of all version of
the document. Only valid for individual item
endpoints.

``VERSION`` Field used to store the version number of a
document. Defaults to ``_version``.

``LASTEST_VERSION`` Field used to store the latest version number
of a document. Defaults to ``_latest_version``.

``VERSION_ID_SUFFIX`` Used in the shadow collection to store the
document id. Defaults to ``_document``. If
``ID_FIELD`` is set to ``_id``, the document id
will be stored in field ``_id_document``.

``MONGO_HOST`` MongoDB server address. Defaults to ``localhost``.

``MONGO_PORT`` MongoDB port. Defaults to ``27017``.
Expand Down Expand Up @@ -774,13 +803,18 @@ defining the field validation rules. Allowed validation rules are:
- ``resource``: the name of the resource being referenced;
- ``field``: the field name in the foreign resource;
- ``embeddable``: set to ``True`` if clients can request the referenced document to be embedded with the serialization. See :ref:`embedded_docs`. Defaults to ``False``.
- ``version``: set to ``True`` to require a ``_version`` with the data relation. See :ref:`document_versioning`. Defaults to ``False``.

``nullable`` If ``True`` the field value can be set to
``nullable`` If ``True``, the field value can be set to
``None``.

``default`` The default value for the field. When serving
POST (create) requests, missing fields will be
assigned the configured default values.

``versioned`` If ``True``, this field will be included in the
versioned history of each document when
``versioning`` is enabled. Defaults to ``True``.
=============================== ==============================================

Schema syntax is based on Cerberus_ and yes, it can be extended. In fact, Eve
Expand Down
71 changes: 68 additions & 3 deletions docs/features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -630,7 +630,7 @@ validation class to validate that. Or say you want to make sure that a VAT
field actually matches your own country VAT algorithm; you can do that too. As
a matter of fact, Eve's MongoDB data-layer itself extends Cerberus
validation by implementing the ``unique`` schema field constraint. For more
information see :ref:`validation`
information see :ref:`validation`.

.. _cache_control:

Expand All @@ -653,15 +653,44 @@ The response above includes both ``Cache-Control`` and ``Expires`` headers.
These will minimize load on the server since cache-enabled consumers will
perform resource-intensive request only when really needed.

Versioning
----------
API Versioning
--------------
I'm not too fond of API versioning. I believe that clients should be
intelligent enough to deal with API updates transparently, especially since
Eve-powered API support HATEOAS_. When versioning is a necessity, different API
versions should be isolated instances since so many things could be different
between versions: caching, URIs, schemas, validation, and so on. URI versioning
(http://api.example.com/v1/...) is supported.

.. _document_versioning:

Document Versioning
-------------------
Eve supports automatic version control of documents. By default, this setting is
turned off, but it can be turned globally or configured individually for each
resource. When enabled, Eve provides version control by storing versions of
documents in a shadow collection. All HTTP methods act on the latest version of
the document except when retrieving an indivual item. In this case, adding a
query parameter of ``?version=VERSION`` can be used to point to a specific
version. Special values of ``?version=all`` and ``?version=diffs`` are also
valid. (Bonus tip - try a projection on top of ``?version=all``!) Additional
fields called ``_version`` and ``_latest_version`` get automatically added to
documents when versioning is turned on.

It is important to note that there are a few non-standard scenarios which could
produce unexpected results when versioning is turned on. In particular, document
history will not be saved when modifying collections outside of the Eve
generated API. Also, if at anytime the ``VERSION`` field gets removed from the
primary document (which cannot happen through the API when versioning is turned
on), a subsequent write will re-initialize the ``VERSION`` number with
``VERSION`` = 1. At this time there will be multiple versions of the document
with the same version number. In normal practice, ``VERSIONING`` can be enable
without worry for any new collection or even an existing collection which has
not previously had versioning enabled.

For more information see and `Global Configuration`_ and `Domain Configuration`_.


Authentication
--------------
Customizable Basic Authentication (RFC-2617), Token-based authentication and
Expand Down Expand Up @@ -775,6 +804,42 @@ toggling the ``embedding`` value). Furthermore, only fields with the
``embeddable`` value explicitly set to ``True`` will allow the embedding of
referenced documents.

Embedding also works with a data_relation to a specific version of a document,
but the schema looks a little bit different. To enable the data_relation to a
specific version, add ``'version': True`` to the data_relation block. You'll
also want to change the ``type`` to ``dict`` and add the ``schema`` definition
shown below.

.. code-block:: python
:emphasize-lines: 5, 6, 11

DOMAIN = {
'emails': {
'schema': {
'author': {
'type': 'dict',
'schema': {
'_id': {'type': 'objectid'},
'_version': {'type': 'integer'}
},
'data_relation': {
'resource': 'users',
'field': '_id',
'embeddable': True,
'version': True,
},
},
'subject': {'type': 'string'},
'body': {'type': 'string'},
}
}

As you can see, ``'version': True`` changes the expected value of a
data_relation field to a dictionary with fields names ``data_relation['field']``
and ``VERSION``. With ``'field': '_id'`` in the data_relation definition above
and ``VERSION = '_version'`` in the Eve config, the value of the data_relation
in this scenario would be a dictionary with fields ``_id`` and ``_version``.

Predefined Resource Serialization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is also possible to elect some fields for predefined resource
Expand Down
1 change: 1 addition & 0 deletions eve/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
ITEMS = '_items'
LINKS = '_links'
ETAG = '_etag'
VERSION = '_version'

# must be the last line (will raise W402 on pyflakes)
from eve.flaskapp import Eve # noqa
19 changes: 18 additions & 1 deletion eve/default_settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@

.. versionchanged:: 0.4
URL_PROTOCOL added and set to ''.
'VERSION' added and set to '_version'.
'VERSIONS' added and set to '_versions'.
'VERSIONING' added and set to False.
'VERSION_PARAM' added and set to 'version'.
'LATEST_VERSION' added and set to '_latest_version'.
'VERSION_ID_SUFFIX' added and set to '_document'.
'VERSION_DIFF_INCLUDE' added and set to [].

.. versionchanged:: 0.3
X_MAX_AGE added and set to 21600.
Expand Down Expand Up @@ -64,6 +71,13 @@
ITEMS = '_items'
LINKS = '_links'
ETAG = '_etag'
VERSION = '_version' # field that stores the version number
LATEST_VERSION = '_latest_version' # field returned on GET requests so we know
# if we have the latest copy even if we access
# a specific version
VERSION_ID_SUFFIX = '_document' # appended to ID_FIELD, holds the original
# document id in parallel collection
VERSION_DIFF_INCLUDE = [] # always include these fields when diffing

API_VERSION = ''
URL_PREFIX = ''
Expand All @@ -87,6 +101,9 @@
PAGINATION = True # pagination enabled by default.
PAGINATION_LIMIT = 50
PAGINATION_DEFAULT = 25
VERSIONING = False # turn document versioning on or off
VERSIONS = '_versions' # suffix for parallel collection w/old versions
VERSION_PARAM = 'version' # URL param for specific version of a document

RESOURCE_METHODS = ['GET']
ITEM_METHODS = ['GET']
Expand All @@ -100,7 +117,7 @@

# list of extra fields to be included with every POST response. This list
# should not include the 'standard' fields (ID_FIELD, LAST_UPDATED,
# DATE_CREATED,ETAG).
# DATE_CREATED, and ETAG).
EXTRA_RESPONSE_FIELDS = []


Expand Down
10 changes: 7 additions & 3 deletions eve/endpoints.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,9 @@ def item_endpoint(**lookup):
def home_endpoint():
""" Home/API entry point. Will provide links to each available resource

.. versionchanged:: 0.4
Prevent versioning collections from being added in links.

.. versionchanged:: 0.2
Use new 'resource_title' setting for link titles.

Expand All @@ -117,9 +120,10 @@ def home_endpoint():
response = {}
links = []
for resource in config.DOMAIN.keys():
links.append({'href': '%s' % resource_uri(resource),
'title': '%s' %
config.DOMAIN[resource]['resource_title']})
if not resource.endswith(config.VERSIONS):
links.append({'href': '%s' % resource_uri(resource),
'title': '%s' %
config.DOMAIN[resource]['resource_title']})
response[config.LINKS] = {'child': links}
return send_response(None, (response,))
else:
Expand Down