Skip to content

Commit

Permalink
Deprecate some methods.
Browse files Browse the repository at this point in the history
 1. Deprecate QueryRequest.set_continuation_key
 2. Deprecate QueryRequest.get_continuation_key
 3. Change the examples to use a loop to get query results.
 4. Change the tests to use a loop to get query results.
 5. Change the documents to use a loop get query results.
  • Loading branch information
pengfei0107 committed Apr 17, 2020
1 parent 0e46d48 commit b6efaa8
Show file tree
Hide file tree
Showing 10 changed files with 105 additions and 78 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ _______
programmatically with user profile.
* Change the method SignatureProvider.create_with_instance_principal to allow
setting Region with instance principal.
* Deprecate QueryRequest.set_continuation_key and
QueryRequest.get_continuation_key, use QueryRequest.is_done instead.

Fixed
_____
Expand Down
16 changes: 13 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -304,10 +304,20 @@ that the *borneo* package has been installed.
statement = (
'select * from ' + table_name + ' where id > 2 and id < 8')
request = QueryRequest().set_statement(statement)
result = handle.query(request)
print('Query results for: ' + statement)
for r in result.get_results():
print('\t' + str(r))
#
# If the :py:meth:`borneo.QueryRequest.is_done` returns False, there
# may be more results, so queries should generally be run in a loop.
# It is possible for single request to return no results but the
# query still not done, indicating that the query loop should
# continue.
#
while True:
result = handle.query(request)
for r in result.get_results():
print('\t' + str(r))
if request.is_done():
break
#
# Delete the row
Expand Down
4 changes: 0 additions & 4 deletions docs/api/borneo.QueryRequest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ QueryRequest
~QueryRequest.close
~QueryRequest.get_compartment
~QueryRequest.get_consistency
~QueryRequest.get_continuation_key
~QueryRequest.get_limit
~QueryRequest.get_math_context
~QueryRequest.get_max_memory_consumption
Expand All @@ -25,7 +24,6 @@ QueryRequest
~QueryRequest.is_done
~QueryRequest.set_compartment
~QueryRequest.set_consistency
~QueryRequest.set_continuation_key
~QueryRequest.set_limit
~QueryRequest.set_math_context
~QueryRequest.set_max_memory_consumption
Expand All @@ -40,7 +38,6 @@ QueryRequest
.. automethod:: close
.. automethod:: get_compartment
.. automethod:: get_consistency
.. automethod:: get_continuation_key
.. automethod:: get_limit
.. automethod:: get_math_context
.. automethod:: get_max_memory_consumption
Expand All @@ -52,7 +49,6 @@ QueryRequest
.. automethod:: is_done
.. automethod:: set_compartment
.. automethod:: set_consistency
.. automethod:: set_continuation_key
.. automethod:: set_limit
.. automethod:: set_math_context
.. automethod:: set_max_memory_consumption
Expand Down
56 changes: 27 additions & 29 deletions docs/tables.rst
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,12 @@ ctx=en/cloud/paas/nosql-cloud&id=sql_nosql>`_ for a full description of the
query language.

To execute a query use the :func:`borneo.NoSQLHandle.query` method. For example,
to execute a *SELECT* query to read data from your table:
to execute a *SELECT* query to read data from your table, a
:class:`borneo.QueryResult` contains a list of results. And if the
:func:`borneo.QueryRequest.is_done` returns False, there may be more results, so
queries should generally be run in a loop. It is possible for single request to
return no results but the query still not done, indicating that the query loop
should continue. For example:

.. code-block:: pycon
Expand All @@ -317,33 +322,13 @@ to execute a *SELECT* query to read data from your table:
# statement
statement = 'select * from users where name = "Taylor"'
request = QueryRequest().set_statement(statement)
result = handle.query(request)
# look at results for this single request
for res in result.get_results():
print(str(res))
A :class:`borneo.QueryResult` contains a list of results as well as an optional
*continuation key*. If the continuation key is not empty there may be more
results, so queries should generally be run in a loop. It is possible for single
request to return no results but still have a continuation key, indicating that
the query loop should continue. For example:

.. code-block:: pycon
from borneo import QueryRequest
statement = 'select * from users where name = "Taylor"'
request = QueryRequest().set_statement(statement)
result = handle.query(request)
# handle results so far
handle_results(result) # do something with results
# loop until request is done, handling results as they arrive
while not request.is_done():
while True:
result = handle.query(request)
# handle results
handle_results(result) # do something with results
if request.is_done():
break
When using queries it is important to be aware of the following considerations:

Expand Down Expand Up @@ -374,13 +359,26 @@ Here is an example of using a prepared query with a single variable:
pstatement = presult.get_prepared_statement()
pstatement.set_variable('$name', 'Taylor')
qrequest = QueryRequest().set_prepared_statement(pstatement)
# use the prepared query in the query request
qresult = handle.query(qrequest)
# loop until qrequest is done, handling results as they arrive
while True:
# use the prepared query in the query request
qresult = handle.query(qrequest)
# handle results
handle_results(qresult) # do something with results
if qrequest.is_done():
break
# use a different variable value with the same prepared query
pstatement.set_variable('$name', 'another_name')
qresult = handle.query(qrequest)
qrequest = QueryRequest().set_prepared_statement(pstatement)
# loop until qrequest is done, handling results as they arrive
while True:
# use the prepared query in the query request
qresult = handle.query(qrequest)
# handle results
handle_results(qresult) # do something with results
if qrequest.is_done():
break
-----------
Delete Data
Expand Down
18 changes: 12 additions & 6 deletions examples/multi_data_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,10 +108,13 @@ def main():
# Query, using the prepared statement
#
request = QueryRequest().set_prepared_statement(prepared_result)
result = handle.query(request)
print('Query results for the prepared statement: ')
for r in result.get_results():
print('\t' + str(r))
while True:
result = handle.query(request)
for r in result.get_results():
print('\t' + str(r))
if request.is_done():
break

#
# Multiple delete the rows
Expand All @@ -126,10 +129,13 @@ def main():
#
request = QueryRequest().set_prepared_statement(
prepared_result)
result = handle.query(request)
print('Query results for the prepared statement (should be no rows): ')
for r in result.get_results():
print('\t' + str(r))
while True:
result = handle.query(request)
for r in result.get_results():
print('\t' + str(r))
if request.is_done():
break

#
# Drop the table
Expand Down
9 changes: 6 additions & 3 deletions examples/single_data_ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,13 @@ def main():
#
statement = 'select * from ' + table_name + ' where id > 2 and id < 8'
request = QueryRequest().set_statement(statement)
result = handle.query(request)
print('Query results for: ' + statement)
for r in result.get_results():
print('\t' + str(r))
while True:
result = handle.query(request)
for r in result.get_results():
print('\t' + str(r))
if request.is_done():
break

#
# Delete the row
Expand Down
11 changes: 11 additions & 0 deletions src/borneo/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from sys import version_info
from threading import Lock
from time import ctime, time
from warnings import simplefilter, warn

from .exception import IllegalArgumentException

Expand All @@ -24,6 +25,16 @@ def enum(**enums):
return type('Enum', (object,), enums)


def deprecated(func):
@wraps(func)
def wrapper(*args, **kwargs):
simplefilter('default', DeprecationWarning)
warn("Call to deprecated function {}.".format(func.__name__),
DeprecationWarning)
return func(*args, **kwargs)
return wrapper


def synchronized(func):
@wraps(func)
def wrapper(self, *args, **kwargs):
Expand Down
12 changes: 5 additions & 7 deletions src/borneo/driver.py
Original file line number Diff line number Diff line change
Expand Up @@ -371,13 +371,11 @@ def query(self, request):
:py:meth:`QueryRequest.set_max_read_kb`. This limits the amount of data
*read* and not the amount of data *returned*, which means that a query
can return zero results but still have more data to read. This situation
is detected by checking if the :py:class:`QueryRequest` has a
continuation key, using :py:meth:`QueryRequest.get_continuation_key`.
For this reason queries should always operate in a loop, acquiring more
results, until the continuation key is None, indicating that the query
is done. Inside the loop the continuation key is applied to the
:py:class:`QueryRequest` using
:py:meth:`QueryRequest.set_continuation_key`.
is detected by checking if the :py:class:`QueryRequest` is done using
:py:meth:`QueryRequest.is_done`. For this reason queries should always
operate in a loop, acquiring more results, until
:py:meth:`QueryRequest.is_done` returns True, indicating that the query
is done.
:param request: the input parameters for the operation.
:type request: QueryRequest
Expand Down
35 changes: 25 additions & 10 deletions src/borneo/operations.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

from .common import (
CheckValue, Consistency, FieldRange, PreparedStatement, PutOption, State,
SystemState, TableLimits, TimeToLive, Version)
SystemState, TableLimits, TimeToLive, Version, deprecated)
from .exception import (
IllegalArgumentException, RequestTimeoutException, TableNotFoundException)
from .serde import (
Expand Down Expand Up @@ -1825,7 +1825,7 @@ def close(self):
query at the driver. An application should use this method if it wishes
to terminate query execution before retrieving all of the query results.
"""
self.set_continuation_key(None)
self.set_cont_key(None)

def is_done(self):
"""
Expand Down Expand Up @@ -2046,6 +2046,7 @@ def get_consistency(self):
"""
return self._consistency

@deprecated
def set_continuation_key(self, continuation_key):
"""
Sets the continuation key. This is used to continue an operation that
Expand All @@ -2057,19 +2058,24 @@ def set_continuation_key(self, continuation_key):
:returns: self.
:raises IllegalArgumentException: raises the exception if
continuation_key is not a bytearray.
:deprecated: There is no reason to use this method anymore, because
setting the continuation key is now done internally.
"""
if (continuation_key is not None and
not isinstance(continuation_key, bytearray)):
raise IllegalArgumentException(
'set_continuation_key requires bytearray as parameter.')
return self.set_cont_key(continuation_key)

@deprecated
def get_continuation_key(self):
"""
Returns the continuation key if set.
:returns: the key.
:rtype: bytearray
:deprecated: There is no reason to use this method anymore, because
getting the continuation key is now done internally.
"""
return self._continuation_key

Expand Down Expand Up @@ -3696,14 +3702,23 @@ class QueryResult(Result):
queries either return values based on a RETURNING clause or, by default, the
number of rows affected by the statement.
If the value returned by :py:meth:`get_continuation_key` is not None there
are additional results available. That value can be supplied to the query
request using :py:meth:`QueryRequest.set_continuation_key` to continue the
query. It is possible for a query to return no results in an empty list but
still have a non-none continuation key. This happens if the query reads the
maximum amount of data allowed in a single request without matching a query
predicate. In this case, the continuation key must be used to get results,
if any exist.
A single QueryResult does not imply that all results for the query have been
returned. If the value returned by :py:meth:`QueryRequest.is_done` is False
there are additional results available. This can happen even if there are no
values in the returned QueryResult. The best way to use
:py:class:`QueryRequest` and :py:class:`QueryResult` is to perform
operations in a loop, for example:
.. code-block:: pycon
handle = ...
request = QueryRequest().set_statement('SELECT * FROM foo')
while True:
result = handle.query(request)
results = result.get_results()
# do something with the results
if request.is_done():
break
"""

def __init__(self, request, computed=True):
Expand Down
20 changes: 4 additions & 16 deletions test/query.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,11 +158,6 @@ def testQuerySetIllegalConsistency(self):
self.query_request.set_consistency,
'IllegalConsistency')

def testQuerySetIllegalContinuationKey(self):
self.assertRaises(IllegalArgumentException,
self.query_request.set_continuation_key,
'IllegalContinuationKey')

def testQuerySetIllegalStatement(self):
self.assertRaises(IllegalArgumentException,
self.query_request.set_statement, {})
Expand Down Expand Up @@ -207,24 +202,21 @@ def testQueryNoStatementAndBothStatement(self):
self.query_request)

def testQueryGets(self):
continuation_key = bytearray(5)
context = Context(prec=10, rounding=ROUND_HALF_EVEN)
self.query_request.set_consistency(Consistency.EVENTUAL).set_statement(
query_statement).set_prepared_statement(
self.prepare_result_select).set_limit(3).set_max_read_kb(
2).set_max_write_kb(3).set_max_memory_consumption(
5).set_math_context(context).set_continuation_key(continuation_key)
5).set_math_context(context)
self.assertIsNone(self.query_request.get_compartment())
self.assertFalse(self.query_request.is_done())
self.assertTrue(self.query_request.is_done())
self.assertEqual(self.query_request.get_limit(), 3)
self.assertEqual(self.query_request.get_max_read_kb(), 2)
self.assertEqual(self.query_request.get_max_write_kb(), 3)
self.assertEqual(self.query_request.get_max_memory_consumption(), 5)
self.assertEqual(self.query_request.get_math_context(), context)
self.assertEqual(self.query_request.get_consistency(),
Consistency.EVENTUAL)
self.assertEqual(self.query_request.get_continuation_key(),
continuation_key)
self.assertEqual(self.query_request.get_statement(), query_statement)
self.assertEqual(self.query_request.get_prepared_statement(),
self.prepare_result_select.get_prepared_statement())
Expand Down Expand Up @@ -305,10 +297,8 @@ def testQueryStatementSelectWithContinuationKey(self):
result, read_kb + (prepare_cost if count == 0 else 0),
read_kb * 2 + (prepare_cost if count == 0 else 0), 0, 0)
count += 1
if result.get_continuation_key() is None:
if self.query_request.is_done():
break
self.query_request.set_continuation_key(
result.get_continuation_key())
self.assertEqual(count, num_records // limit + 1)

def testQueryStatementSelectWithDefault(self):
Expand Down Expand Up @@ -455,10 +445,8 @@ def testQueryPreparedStatementUpdateWithContinuationKey(self):
write_kb = (0 if num_update == 0 else num_update * 4)
self.check_cost(result, read_kb, read_kb * 2, write_kb, write_kb)
count += 1
if result.get_continuation_key() is None:
if self.query_request.is_done():
break
self.query_request.set_continuation_key(
result.get_continuation_key())
self.assertEqual(count, 1)
# check the updated row
prepared_statement = self.prepare_result_select.get_prepared_statement()
Expand Down

0 comments on commit b6efaa8

Please sign in to comment.