Merge 7c78898 into f3753e6

docs/source/types.rst

@@ -25,6 +25,15 @@ To get the schema of a content type, access the ``/@types`` endpoint  with the n
 
 The content type schema uses the `JSON Schema <http://json-schema.org/>`_ format.
 The tagged values for the widgets are also exposed in the the "properties" attribute of the schema.
-If a 'vocabulary' is defined, it will be the name of the vocabulary which should be used via the `@vocabularies` endpoint on the actual resource.
+
+For ``Choice`` fields, their vocabulary or source will be linked to in a ``vocabulary`` or ``querysource`` property (one or the other, never both):
+
+
+- If a ``querysource`` property is included, that field is backed by an ``IQuerysource``.
+  In that case, the source's terms can't be enumerated, and the terms need to be **queried** by issuing a request to the linked endpoint and including the user's search terms in the ``?query=`` parameter.
+- If a ``vocabulary`` property is included, the field is backed by a vocabulary or another kind of iterable source.
+  The terms can then be **enumerated** by issuing a request to the linked endpoint.
+
+See :ref:`vocabularies` for details on these endpoints.
 
 See :ref:`types-schema` for a detailed documentation about the available field types.

docs/source/vocabularies.rst

@@ -1,49 +1,94 @@
-Vocabularies
-============
+.. _vocabularies:
 
-Vocabularies are utilities containing a list of values grouped by interest or different Plone features.
-For example, ``plone.app.vocabularies.ReallyUserFriendlyTypes`` will return all the content types registered in Plone.
-The vocabularies return a list of objects with the items ``title`` and ``token``.
+Vocabularies and Sources
+========================
+
+Vocabularies are a set of allowed choices that back a particular field.
+They contain so called *terms* which represent those allowed choices.
+Sources are a similar, but are a more generic and dynamic concept.
+
+Concepts
+--------
+
+**Vocabularies** contain a list of terms.
+These terms are usually tokenized, meaning that in addition to a term's value, it also has a ``token`` which is a machine-friendly identifier for the term (7bit ASCII).
 
 .. note::
-	These docs are generated by code tests, therefore you will see some 'test' contenttypes appear here.
+    Since the underlying value of a term might not necessarily be serializable (it could be an arbitrary Python object), ``plone.restapi`` only exposes and accepts tokens, and will transparently convert between tokens and values during serialization / deseralization.
+    For this reason, the following endpoints only support *tokenized* vocabularies / sources, and they do not expose the terms' values.
+
+Terms can also have a ``title``, which is intended to be the user-facing label for the term.
+For vocabularies or sources whose terms are only tokenized, but not titled, ``plone.restapi`` will fall back to using the token as the term title.
+
+**Sources** are similar to vocabularies, but they tend to be more dynamic in nature, and are often used for larger sets of terms.
+They are also not registered with a global name like vocabularies, but are instead addressed via the field they are assigned to.
+
+**Query Sources** are sources that are capable of being queried / searched.
+The source will then return only the subset of terms that match the query.
+
+The use of such a source is usually a strong indication that no attempt should be made to enumerate the full set of terms, but instead the source should only be queried, by presenting the user with an autocomplete widget for example.
+
+Both vocabularies and sources can be context-sensitive, meaning that they take the context into account and their contents may therefore change depending on the context they're invoked on.
+
+This section can only provide a basic overview of vocabularies and related concepts.
+For a more in-depth explanation please refer to the `Plone documentation <https://docs.plone.org/develop/plone/forms/vocabularies.html>`_.
+
+Endpoints overview
+------------------
+
+In ``plone.restapi`` these three concepts are exposed through three separate endpoints (described in more detail below):
+
+- **@vocabularies**/(vocab_name)
+- **@sources**/(field_name)
+- **@querysources**/(field_name) **?query=** (search_query)
 
-Get all vocabularies
---------------------
+While the ``@vocabularies`` and ``@sources`` endpoints allow to *enumerate* terms (and optionally have terms filtered server-side), the ``@querysources`` endpoint **only** allows for searching the respective source.
 
-To retrieve a list of all the available vocabularies, send a GET request to the @vocabularies endpoint:
+
+List all vocabularies
+---------------------
+
+.. http:get:: (context)/@vocabularies
+
+To retrieve a list of all the available vocabularies, send a ``GET`` request to the ``@vocabularies`` endpoint:
 
 ..  http:example:: curl httpie python-requests
     :request: ../../src/plone/restapi/tests/http-examples/vocabularies.req
 
-The response will include a list with the URL (``@id``) the dotted names (``title``) of all the available vocabularies in Plone:
+The response will include a list with the URL (``@id``) and the names (``title``) of all the available vocabularies in Plone:
 
 .. literalinclude:: ../../src/plone/restapi/tests/http-examples/vocabularies.resp
    :language: http
 
+
 Get a vocabulary
 ----------------
 
-To get a particular vocabulary, use the ``@vocabularies`` endpoint with the name of the vocabulary, e.g. ``/plone/@vocabularies/plone.app.vocabularies.ReallyUserFriendlyTypes``.
+.. http:get:: (context)/@vocabularies/(vocab_name)
+
+To enumerate the terms of a particular vocabulary, use the ``@vocabularies`` endpoint with the name of the vocabulary, e.g. ``/plone/@vocabularies/plone.app.vocabularies.ReallyUserFriendlyTypes``.
 The endpoint can be used with the site root and content objects.
 
 ..  http:example:: curl httpie python-requests
     :request: ../../src/plone/restapi/tests/http-examples/vocabularies_get.req
 
-The server will respond with a list of terms. The title is purely for display purposes.
-The token is what should be sent to the server to retrieve the value of the term.
+The server will respond with a list of terms.
+The title is purely for display purposes.
+The token is what should be sent to the server to address that term.
 
 .. note::
-    Vocabulary terms will be **batched** if the size of the
-    resultset exceeds the batch size. See :doc:`/batching` for more
-    details on how to work with batched results.
+    Vocabulary terms will be **batched** if the size of the resultset exceeds the batch size.
+    See :doc:`/batching` for more details on how to work with batched results.
 
 .. literalinclude:: ../../src/plone/restapi/tests/http-examples/vocabularies_get.resp
    :language: http
 
 Filter Vocabularies
 ^^^^^^^^^^^^^^^^^^^
 
+.. http:get:: (context)/@vocabularies/(vocab_name)?title=(filter_query)
+.. http:get:: (context)/@vocabularies/(vocab_name)?token=(filter_query)
+
 Vocabulary terms can be filtered using the ``title`` or ``token`` parameter.
 
 Use the ``title`` paramenter to filter vocabulary terms by title.
@@ -68,3 +113,59 @@ E.g. search the term ``doc`` in the token:
 .. note::
     You must not filter by title and token at the same time.
     The API returns a 400 response code if you do so.
+
+
+Get a source
+------------
+
+.. http:get:: (context)/@sources/(field_name)
+
+To enumerate the terms of a field's source, use the ``@sources`` endpoint on a specific context, and pass the field name as a path parameter, e.g. ``/plone/doc/@sources/some_field``.
+
+Because sources are inherently tied to a specific field, this endpoint can only be invoked on content objects, and the source is addressed via the field name its used for, instead of a global name (which sources don't have).
+
+Otherwise the endpoint behaves the same as the ``@vocabularies`` endpoint.
+
+Example:
+
+..  http:example:: curl httpie python-requests
+    :request: ../../src/plone/restapi/tests/http-examples/sources_get.req
+
+The server will respond with a list of terms.
+The title is purely for display purposes.
+The token is what should be sent to the server to address that term.
+
+.. literalinclude:: ../../src/plone/restapi/tests/http-examples/sources_get.resp
+   :language: http
+
+.. note::
+    Technically there can be sources that are not iterable (ones that only implement ``ISource``, but not ``IIterableSource``).
+    These cannot be enumerated using the ``@sources`` endpoint, and it will respond with a corresponding error.
+
+
+Querying a query source
+-----------------------
+
+.. http:get:: (context)/@querysources/(field_name)?query=(search_query)
+
+Query sources (sources implementing `IQuerySource`) can be queried using this endpoint, by passing the search term in the ``query`` parameter.
+This search term will be passed to the query source's ``search()`` method, and the source's results are returned.
+
+Example:
+
+..  http:example:: curl httpie python-requests
+    :request: ../../src/plone/restapi/tests/http-examples/querysources_get.req
+
+The server will respond with a list of terms.
+The title is purely for display purposes.
+The token is what should be sent to the server to address that term.
+
+.. literalinclude:: ../../src/plone/restapi/tests/http-examples/querysources_get.resp
+   :language: http
+
+.. note::
+    Even though technically sources that implement ``IQuerySource`` are required to implement ``__iter__`` as well (when strictly following the interface interitance hierarchy), they usually are used in Plone in situations where their full contents shouldn't or can't be enumerated (imagine a source of all users, backed by a large LDAP, for example).
+
+    For this reason, ``plone.restapi`` takes the stance that the ``IQuerySource`` interface is a strong indication that this source should **only** be queried, and therefore doesn't support enumeration of terms via the ``@querysources`` endpoint.
+
+    *(If the source does actually implement IIterableSource in addition to IQuerySource, it can still be enumerated via the @sources endpoint)*

news/790.feature

@@ -0,0 +1,2 @@
+Add @sources and @querysources endpoints, and link to them from JSON schema in @types response.
+[lgraf]

src/plone/restapi/serializer/configure.zcml

@@ -71,6 +71,7 @@
              />
 
     <adapter factory=".vocabularies.SerializeVocabularyToJson" />
+    <adapter factory=".vocabularies.SerializeSourceToJson" />
     <adapter factory=".vocabularies.SerializeTermToJson" />
 
     <adapter factory=".registry.SerializeRegistryToJson" />

src/plone/restapi/serializer/vocabularies.py

@@ -6,6 +6,7 @@
 from zope.i18n import translate
 from zope.interface import implementer
 from zope.interface import Interface
+from zope.schema.interfaces import IIterableSource
 from zope.schema.interfaces import ITitledTokenizedTerm
 from zope.schema.interfaces import ITokenizedTerm
 from zope.schema.interfaces import IVocabulary
@@ -14,8 +15,11 @@
 
 
 @implementer(ISerializeToJson)
-@adapter(IVocabulary, Interface)
-class SerializeVocabularyToJson(object):
+class SerializeVocabLikeToJson(object):
+    """Base implementation to serialize vocabularies and sources to JSON.
+
+    Implements server-side filtering as well as batching.
+    """
     def __init__(self, context, request):
         self.context = context
         self.request = request
@@ -66,6 +70,18 @@ def __call__(self, vocabulary_id):
         return result
 
 
+@adapter(IVocabulary, Interface)
+class SerializeVocabularyToJson(SerializeVocabLikeToJson):
+    """Serializes IVocabulary to JSON.
+    """
+
+
+@adapter(IIterableSource, Interface)
+class SerializeSourceToJson(SerializeVocabLikeToJson):
+    """Serializes IIterableSource to JSON.
+    """
+
+
 @implementer(ISerializeToJson)
 @adapter(ITokenizedTerm, Interface)
 class SerializeTermToJson(object):

src/plone/restapi/services/configure.zcml

@@ -19,11 +19,13 @@
   <include package=".history"/>
   <include package=".locking" />
   <include package=".principals"/>
+  <include package=".querysources"/>
   <include package=".querystring"/>
   <include package=".querystringsearch"/>
   <include package=".registry"/>
   <include package=".roles"/>
   <include package=".search"/>
+  <include package=".sources"/>
   <include package=".types"/>
   <include package=".users"/>
   <include package=".vocabularies"/>

src/plone/restapi/services/querysources/configure.zcml

@@ -0,0 +1,23 @@
+<configure
+    xmlns="http://namespaces.zope.org/zope"
+    xmlns:plone="http://namespaces.plone.org/plone">
+
+  <plone:service
+    method="GET"
+    accept="application/json"
+    for="Products.CMFPlone.interfaces.IPloneSiteRoot"
+    factory=".get.QuerySourcesGet"
+    name="@querysources"
+    permission="plone.restapi.vocabularies"
+    />
+
+  <plone:service
+    method="GET"
+    accept="application/json"
+    for="Products.CMFCore.interfaces.IContentish"
+    factory=".get.QuerySourcesGet"
+    name="@querysources"
+    permission="plone.restapi.vocabularies"
+    />
+
+</configure>

src/plone/restapi/services/querysources/get.py

@@ -0,0 +1,70 @@
+# -*- coding: utf-8 -*-
+from plone.restapi.batching import HypermediaBatch
+from plone.restapi.interfaces import ISerializeToJson
+from plone.restapi.services.sources.get import get_field_by_name
+from plone.restapi.services.sources.get import SourcesGet
+from z3c.formwidget.query.interfaces import IQuerySource
+from zope.component import getMultiAdapter
+from zope.interface import implementer
+from zope.publisher.interfaces import IPublishTraverse
+
+
+@implementer(IPublishTraverse)
+class QuerySourcesGet(SourcesGet):
+
+    def reply(self):
+        if len(self.params) != 1:
+            return self._error(
+                400, "Bad Request",
+                "Must supply exactly one path parameter (fieldname)"
+            )
+        fieldname = self.params[0]
+
+        field = get_field_by_name(fieldname, self.context)
+        if field is None:
+            return self._error(
+                404, "Not Found",
+                "No such field: %r" % fieldname
+            )
+        bound_field = field.bind(self.context)
+
+        source = bound_field.source
+        if not IQuerySource.providedBy(source):
+            return self._error(
+                404, "Not Found",
+                "Field %r does not have an IQuerySource" % fieldname
+            )
+
+        if 'query' not in self.request.form:
+            return self._error(
+                400, "Bad Request",
+                u'Enumerating querysources is not supported. Please search '
+                u'the source using the ?query= QS parameter'
+            )
+
+        query = self.request.form['query']
+
+        result = source.search(query)
+
+        terms = []
+        for term in result:
+            terms.append(term)
+
+        batch = HypermediaBatch(self.request, terms)
+
+        serialized_terms = []
+        for term in batch:
+            serializer = getMultiAdapter(
+                (term, self.request), interface=ISerializeToJson
+            )
+            serialized_terms.append(serializer())
+
+        result = {
+            "@id": batch.canonical_url,
+            "items": serialized_terms,
+            "items_total": batch.items_total,
+        }
+        links = batch.links
+        if links:
+            result["batching"] = links
+        return result

src/plone/restapi/services/sources/configure.zcml

@@ -0,0 +1,14 @@
+<configure
+    xmlns="http://namespaces.zope.org/zope"
+    xmlns:plone="http://namespaces.plone.org/plone">
+
+  <plone:service
+    method="GET"
+    accept="application/json"
+    for="Products.CMFCore.interfaces.IContentish"
+    factory=".get.SourcesGet"
+    name="@sources"
+    permission="plone.restapi.vocabularies"
+    />
+
+</configure>