Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Add offset alias property to be used in query_method. #76

Open
wants to merge 3 commits into from

2 participants

Sander Striker Danny Hermes
Sander Striker

Some clients require more traditional offset, limit functionality. For example an implementation of a dojo Store on top of an endpoints API (dojo-endpoints). As this store needs to stick to a set API, it can't use the more efficient nextPageToken approach.

This patch adds an offset to _EndpointsQueryInfo and uses it when calling fetch_page.

Sander Striker sstriker Add offset alias property to be used in query_method.
* endpoints_proto_datastore/ndb/model.py
  (_EndpointsQueryInfo.__init__): Add _offset member.
  (_EndpointsQueryInfo._GetOffset, _SetOffset, offset): Add offset property.
  (EndpointsModel.OffsetSet, offset): Add offset property.
  (EndpointsModel.query_method.QueryFromRequestMethod): Add offset to
     query_options which is passed to fetch_page.
492a3b6
endpoints_proto_datastore/ndb/model.py
((10 lines not shown))
+
+ Args:
+ value: A potential value for a offset.
+
+ Raises:
+ AttributeError: if query on the object is already final.
+ AttributeError: if the offset has already been set.
+ TypeError: if the value to be set is not a positive integer.
+ """
+ if self._query_final is not None:
+ raise AttributeError('Can\'t set offset. Query info is final.')
+
+ if self._offset is not None:
+ raise AttributeError('Offset can\'t be set twice.')
+ if not isinstance(value, (int, long)) or value < 0:
+ raise TypeError('Offset must be a positive integer.')
Danny Hermes Collaborator
dhermes added a note

0 is acceptable given the if statement, but 0 is not a positive integer. Do you want positive or non-negative?

non-negative. I'll fix the comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Danny Hermes dhermes commented on the diff
endpoints_proto_datastore/ndb/model.py
@@ -1494,7 +1546,10 @@ def QueryFromRequestMethod(service_instance, request):
raise endpoints.ForbiddenException(
QUERY_MAX_EXCEEDED_TEMPLATE % (request_limit, limit_max))
- query_options = {'start_cursor': query_info.cursor}
+ query_options = {
+ 'start_cursor': query_info.cursor,
Danny Hermes Collaborator
dhermes added a note

Is there any sort of issue if cursor and offset are used simultaneously?

Fair question. I'll add some protection so you can't set both.

Danny Hermes Collaborator
dhermes added a note

@sstriker Did this ever get added?

Sander Striker
sstriker added a note

@dhermes Assuming this refers to the protection against using cursor and offset simultaneously, that should be in the updated pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
sstriker added some commits
Sander Striker sstriker Fix comments regarding accepted values for offset. Protect against se…
…tting both cursor and offset.

* endpoints_proto_datastore/ndb/model.py
  (_EndpointsQueryInfo.__init__): Fix comment.
  (_EndpointsQueryInfo._SetCursor, _SetOffset): Prevent setting of both cursor
     and offset.  Fix comment.
c7b4e76
Sander Striker sstriker Fix comments regarding raised exceptions in relation to accepted values
for offset and cursor.

* endpoints_proto_datastore/ndb/model.py
  (_EndpointsQueryInfo._SetCursor, _SetOffset): Add comment regarding
   AttributeError raised when both both cursor and offset are attempted
   to be set.
207816c
Sander Striker

Updated pull request; I think this addresses all outstanding questions/comments.

Danny Hermes
Collaborator

@sstriker I just realized this is orphaned.

Still worth doing? Been addressed by a different fix?

Sander Striker
Danny Hermes
Collaborator

Good to hear.

Can you rebase to HEAD in master and then I'll review and we can get it in?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jul 21, 2013
  1. Sander Striker

    Add offset alias property to be used in query_method.

    sstriker authored
    * endpoints_proto_datastore/ndb/model.py
      (_EndpointsQueryInfo.__init__): Add _offset member.
      (_EndpointsQueryInfo._GetOffset, _SetOffset, offset): Add offset property.
      (EndpointsModel.OffsetSet, offset): Add offset property.
      (EndpointsModel.query_method.QueryFromRequestMethod): Add offset to
         query_options which is passed to fetch_page.
Commits on Jul 23, 2013
  1. Sander Striker

    Fix comments regarding accepted values for offset. Protect against se…

    sstriker authored
    …tting both cursor and offset.
    
    * endpoints_proto_datastore/ndb/model.py
      (_EndpointsQueryInfo.__init__): Fix comment.
      (_EndpointsQueryInfo._SetCursor, _SetOffset): Prevent setting of both cursor
         and offset.  Fix comment.
Commits on Jul 24, 2013
  1. Sander Striker

    Fix comments regarding raised exceptions in relation to accepted values

    sstriker authored
    for offset and cursor.
    
    * endpoints_proto_datastore/ndb/model.py
      (_EndpointsQueryInfo._SetCursor, _SetOffset): Add comment regarding
       AttributeError raised when both both cursor and offset are attempted
       to be set.
This page is out of date. Refresh to see the latest.
Showing with 64 additions and 1 deletion.
  1. +64 −1 endpoints_proto_datastore/ndb/model.py
65 endpoints_proto_datastore/ndb/model.py
View
@@ -181,6 +181,7 @@ class _EndpointsQueryInfo(object):
_ancestor: An ndb Key to be used as an ancestor for a query.
_cursor: A datastore_query.Cursor, to be used for resuming a query.
_limit: A positive integer, to be used in a fetch.
+ _offset: A non-negative integer, to be used in a fetch.
_order: String; comma separated list of property names or property names
preceded by a minus sign. Used to define an order of query results.
_order_attrs: The attributes (or negation of attributes) parsed from
@@ -210,6 +211,7 @@ def __init__(self, entity):
self._ancestor = None
self._cursor = None
self._limit = None
+ self._offset = None
self._order = None
self._order_attrs = ()
@@ -325,6 +327,8 @@ def _SetCursor(self, value):
Raises:
AttributeError: if query on the object is already final.
AttributeError: if the cursor has already been set.
+ AttributeError: if the offset has already been set; cursor and offset
+ are mutually exclusive.
TypeError: if the value to be set is not an instance of
datastore_query.Cursor.
"""
@@ -333,6 +337,8 @@ def _SetCursor(self, value):
if self._cursor is not None:
raise AttributeError('Cursor can\'t be set twice.')
+ if self._offset is not None:
+ raise AttributeError('Cursor can\'t be set. Cursor and offset are mutually exclusive.')
if not isinstance(value, datastore_query.Cursor):
raise TypeError('Cursor must be an instance of datastore_query.Cursor.')
self._cursor = value
@@ -365,6 +371,36 @@ def _SetLimit(self, value):
limit = property(fget=_GetLimit, fset=_SetLimit)
+ def _GetOffset(self):
+ """Getter to be used for public offset property on query info."""
+ return self._offset
+
+ def _SetOffset(self, value):
+ """Setter to be used for public offset property on query info.
+
+ Args:
+ value: A potential value for a offset.
+
+ Raises:
+ AttributeError: if query on the object is already final.
+ AttributeError: if the offset has already been set.
+ AttributeError: if the cursor has already been set; offset and cursor
+ are mutually exclusive.
+ TypeError: if the value to be set is not a positive integer.
+ """
+ if self._query_final is not None:
+ raise AttributeError('Can\'t set offset. Query info is final.')
+
+ if self._offset is not None:
+ raise AttributeError('Offset can\'t be set twice.')
+ if self._cursor is not None:
+ raise AttributeError('Offset can\'t be set. Offset and cursor are mutually exclusive.')
+ if not isinstance(value, (int, long)) or value < 0:
+ raise TypeError('Offset must be a non-negative integer.')
+ self._offset = value
+
+ offset = property(fget=_GetOffset, fset=_SetOffset)
+
def _GetOrder(self):
"""Getter to be used for public order property on query info."""
return self._order
@@ -830,6 +866,28 @@ def limit(self):
"""
return self._endpoints_query_info.limit
+ def OffsetSet(self, value):
+ """Setter to be used for default offset EndpointsAliasProperty.
+
+ Simply sets the offset on the entity's query info object, and the query
+ info object handles validation.
+
+ Args:
+ value: The offset value to be set.
+ """
+ self._endpoints_query_info.offset = value
+
+ @EndpointsAliasProperty(setter=OffsetSet, property_type=messages.IntegerField)
+ def offset(self):
+ """Getter to be used for default offset EndpointsAliasProperty.
+
+ Uses the ProtoRPC property_type IntegerField since an offset.
+
+ Returns:
+ The integer (or null) offset from the query info on the entity.
+ """
+ return self._endpoints_query_info.offset
+
def OrderSet(self, value):
"""Setter to be used for default order EndpointsAliasProperty.
@@ -1376,6 +1434,8 @@ def query_method(cls,
customizing queries:
limit: allows a limit to be passed in and augment the query info on the
deserialized entity.
+ offset: allows an offset to be passed in and augment the query info on the
+ deserialized entity.
order: allows an order to be passed in and augment the query info on the
deserialized entity.
pageToken: allows a websafe string value to be converted to a cursor and
@@ -1494,7 +1554,10 @@ def QueryFromRequestMethod(service_instance, request):
raise endpoints.ForbiddenException(
QUERY_MAX_EXCEEDED_TEMPLATE % (request_limit, limit_max))
- query_options = {'start_cursor': query_info.cursor}
+ query_options = {
+ 'start_cursor': query_info.cursor,
Danny Hermes Collaborator
dhermes added a note

Is there any sort of issue if cursor and offset are used simultaneously?

Fair question. I'll add some protection so you can't set both.

Danny Hermes Collaborator
dhermes added a note

@sstriker Did this ever get added?

Sander Striker
sstriker added a note

@dhermes Assuming this refers to the protection against using cursor and offset simultaneously, that should be in the updated pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ 'offset': query_info.offset
+ }
if use_projection:
projection = [value for value in collection_fields
if value in cls._properties]
Something went wrong with that request. Please try again.