-
-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Search endpoint - request format #45
Comments
@lukasgraf thanks for the writeup! I tried to wrap my head around this issue as well and haven't reached a final conclusion yet. You can ignore searching.rst. This was a very early draft before we even looked at existing solutions. I will try to find some time to do some more research. Though, this seems to be a topic that most API architects seem to avoid talking about. ;) Just a remark regarding POST vs. GET. It seems that some people argue that you actually create a search with a query and therefore a POST request can also be correct in REST terms. Though, you are of course correct with the other disadvantages (caching, no URL, etc.). |
Great job @lukasgraf ! I love your ideas, in some projects we are using JWT (jwt.io) to have the search query on a parameter as they can be complex. If we wanna support complex queries , the parameters get complex and jwt allows to define a JSON format. I would support only GET verb. It may be an option to have to endpoints ? one for text search and one for complex search ? |
Yes, I've also seen
Yes, for any non-trivial query I don't really see anything besides JSON as an option any more. We really don't want to write our own query DSL.
We (4tw) also have a strong tendency towards that, if we can make it work. I'd argue that then means that 3) (
Yes, we also considered that. For now I'd like to focus on an implementation that can gracefully handle complex queries, because that's the hard case, but I could very well see a limited |
I'd love to have a simple JSON-payload rather than having to build a compliacated GET-querystring. |
For 3) it would be about as complicated as this 😜 query_json = quote_plus(json.dumps(query))
requests.get(url, params={'q': json_query}) Or with curl -G -H 'Accept: application/json' $URL --data-urlencode 'q={"SearchableText": "foobar"}' Or with
Maybe offering both is the way to go? As long as the serialization format is JSON for both, implementation and documentation overhead should be minimal. |
That's for me ok as well. I think we should just implement a first version; it can be extended later.. |
Updated: This significantly diminishes the advantage of the use of JSON as the query serialization format, because we still need to have a mechanism in place to reconstruct typing information based on knowledge about the index that's being queried. Under these circumstances I really don't see a point in also offering For this reason, I think |
A first version implementing |
I'd like to start off the discussion about implementing a service that handles searching / querying the Plone site, and hope that we can come to a decision on the fundamental points so I can start with an implementation.
I already took a stab at implementing the way search is currently outlined in docs/source/collections/searching.rst. However, I've encountered a couple issues with this approach. So I evaluated several different possible implementations, each with their own set of issues.
Common pros/cons:
GET
:POST
:Note that in all examples that use a query string the parameters would need to be URL escaped.
1)
GET
with query params in query string (no type hints)Example:
search?SearchableText=lorem&path.query=/Plone/my-folder&path.depth=2
This approach is using
GET
requests with parameters in query strings, without using Zope style type hints (:record
,:list
, etc.). Parameters that contain a dot in their key will be merged into corresponding dictionaries.Advantages:
Disadvantages:
Arguments to index-specific query options can't be typed. This is the big one. Imagine a query like
?path.query=/Plone&path.depth=0
. The0
value for thedepth
option needs to be an integer when passed tocatalog.searchResults()
, otherwise theExtendedPathIndex
will fail with aTypeError
. Given the query string syntax without the Zope type hints, there is no way for the API consumer to declare a type for these arguments, they'll all end up as a string inrequest.form
.This means that, even ignoring validation for now, with this approach the server side needs to do some type conversions. This is tricky, because there's no programmatic way to determine the required types for these type of index-specific options. All you get is a listing of
query_options
.So we'd need to maintain a list of information objects that describe the required types of query options, not unlike to what @vangheem has done in
collective/elasticsearch/indexes.py
. These would need to cover at least the index types present in a standard Plone, and support some kind of extension mechanism (adapter lookup) to deal with other index types.I've gotten an approach like this to work, but it's not pretty, and I wouldn't be looking forward to maintaining that.
2)
GET
with query params in query string (plus Zope type hints)This would be the exact same style of query string that Plone currently uses for its
@@search
view.Example:
search?SearchableText=lorem&path.query:record=/Plone/my-folder&path.depth:record:int=2
Advantages:
Disadvantages
TypeError
with the value (but no key) is usually all you get3)
GET
with an URL encoded JSON doc as single query string paramExample:
search?q={"path": {"query": "/plone/folder", "depth": 0}}
(Obviously would need to be URL encoded)
For the query, there's a single query string parameter
q
that contains an URL encoded JSON document that, when deserialized to a Python dictionary, contains a query that can be handed off tocatalog.searchResults()
.Advantages
Disadvantages
4)
POST
with query as JSON document in bodyExample Request:
Advantages:
datetime
)searchResults()
and send it on its way withrequests.post(url, json=query)
.Disadvantages:
POST
is the wrong HTTP method. This mightPOST
requests5)
GET
with query as JSON document in bodyI briefly considered this as an approach (with a
POST
alternative), but the fact is, HTTP clients can't deal withGET
+ body very well, andZPublisher
can't either. So I think this is already out of the question, just mentioning it here for completeness.Advantages:
datetime
)Disadvantages:
ZPublisher
without some trickery@tisto @bloodbare
So, moving forward, do you guys see any other options that I haven't covered? Should we swallow the red pill and continue implementing search as outlined in docs/source/collections/searching.rst, accepting that we need to maintain a list of index descriptions? Or do you see one of the mentioned approaches as a viable alternative?
The text was updated successfully, but these errors were encountered: