This repository has been archived by the owner on Nov 23, 2020. It is now read-only.
[Fixes #70584000 Fixes #104] Manual pagination for page/pageSize in QueryRunner #140
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Disable automatic pagination when the `page` and/or `pageSize` options are passed to `QueryRunner#run`. The previous use of `get_all_results` meant that: - `page` was stripped out of the options hash and we always began at page 1 - `pageSize` was honoured, but all pages would be fetched, resulting in many smaller and inefficient queries Allowing the use of these options will allow consumers to make much more efficient queries. The first use case for this is [#70584000] and gds-operations/vcloud-edge_gateway#104 where I want to fetch the single most recent task by using `page: 1, pageSize: 1, sortDesc: startDate`. It's not feasible to integration test the `page` option without `pageSize` because we'd need a result set bigger than the default of 25.
Release new minor version with QueryRunner changes that are needed by gds-operations/vcloud-edge_gateway integration tests.
|
Added changelog and bumped version for release, because I'll need this for edge-gateway. |
dcarley
added a commit
to gds-operations/vcloud-edge_gateway
that referenced
this pull request
Nov 10, 2014
Integration tests which confirm atomicity and idempotency of updates by
checking the number of tasks generated during an operation were sensitive to
time differences between the clocks on the local machine and vCloud
Director.
This could be reproduced by putting your local clock forward one hour. You'd
get a test failure like the following:
1) Vcloud::EdgeGatewayServices Test EdgeGatewayServices with multiple services Check update is functional should only create one edgeGateway update task when updating the configuration
Failure/Error: expect(task_list_after_update.size - task_list_before_update.size).to be(1)
expected #<Fixnum:3> => 1
got #<Fixnum:1> => 0
Compared using equal?, which compares object identity,
but expected and actual are not the same object. Use
`expect(actual).to eq(expected)` if you don't care about
object identity in this example.
# ./spec/integration/edge_gateway/edge_gateway_services_spec.rb:44:in `block (4 levels) in <module:Vcloud>'
This is because it was using the local time to find all tasks that had
occurred since the beginning of that test and vCloud Director didn't know of
any tasks that had occurred in (what it considers to be) the future.
Fix this by using a previous Edge Gateway task as a marker and find all
tasks that have occurred since that. The timestamp of that last task is
generated by the clock available to vCloud Director, so any difference
between local and remote time is no longer an issue.
In most cases the previous task will be another one of our tests or the
`reset_edge_gateway` before block. The helper will raise an exception if
it's unable to find anything at all. We make sure *that* task is removed
from the elapsed results by matching it's unique `href` field.
The option `pageSize 1` is used in the initial query to optimise the amount
of data we get from vCloud, because we only care about the single most
recent. This depends on a new release of vcloud-core for
gds-operations/vcloud-core#140 - without which it will fetch all results
one-by-one, very slowly.
|
👍 |
annashipman
added a commit
that referenced
this pull request
Nov 10, 2014
…ation [Fixes #70584000 Fixes #104] Manual pagination for page/pageSize in QueryRunner
dcarley
added a commit
to gds-operations/vcloud-edge_gateway
that referenced
this pull request
Nov 10, 2014
Integration tests which confirm atomicity and idempotency of updates by
checking the number of tasks generated during an operation were sensitive to
time differences between the clocks on the local machine and vCloud
Director.
This could be reproduced by putting your local clock forward one hour. You'd
get a test failure like the following:
1) Vcloud::EdgeGatewayServices Test EdgeGatewayServices with multiple services Check update is functional should only create one edgeGateway update task when updating the configuration
Failure/Error: expect(task_list_after_update.size - task_list_before_update.size).to be(1)
expected #<Fixnum:3> => 1
got #<Fixnum:1> => 0
Compared using equal?, which compares object identity,
but expected and actual are not the same object. Use
`expect(actual).to eq(expected)` if you don't care about
object identity in this example.
# ./spec/integration/edge_gateway/edge_gateway_services_spec.rb:44:in `block (4 levels) in <module:Vcloud>'
This is because it was using the local time to find all tasks that had
occurred since the beginning of that test and vCloud Director didn't know of
any tasks that had occurred in (what it considers to be) the future.
Fix this by using a previous Edge Gateway task as a marker and find all
tasks that have occurred since that. The timestamp of that last task is
generated by the clock available to vCloud Director, so any difference
between local and remote time is no longer an issue.
In most cases the previous task will be another one of our tests or the
`reset_edge_gateway` before block. The helper will raise an exception if
it's unable to find anything at all. We make sure *that* task is removed
from the elapsed results by matching it's unique `href` field.
The option `pageSize 1` is used in the initial query to optimise the amount
of data we get from vCloud, because we only care about the single most
recent. This depends on a new release of vcloud-core for
gds-operations/vcloud-core#140 - without which it will fetch all results
one-by-one, very slowly.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Disable automatic pagination when the
pageand/orpageSizeoptions arepassed to
QueryRunner#run. The previous use ofget_all_resultsmeantthat:
pagewas stripped out of the options hash and we always began at page 1pageSizewas honoured, but all pages would be fetched, resulting in manysmaller and inefficient queries
Allowing the use of these options will allow consumers to make much more
efficient queries. The first use case for this is [#70584000] and
gds-operations/vcloud-edge_gateway#104 where I want to fetch the single most
recent task by using
page: 1, pageSize: 1, sortDesc: startDate.It's not feasible to integration test the
pageoption withoutpageSizebecause we'd need a result set bigger than the default of 25.