Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datastore: unable to set query batch size #88

Closed
delphinus opened this issue Sep 21, 2017 · 0 comments
Closed

datastore: unable to set query batch size #88

delphinus opened this issue Sep 21, 2017 · 0 comments

Comments

@delphinus
Copy link
Contributor

I did query such below and trace of the request says there are too many datastore.next() calls.

// I used goon in real. here is equivalent code, maybe
q := datastore.NewQuery("SomeKind").KeysOnly()
for t := q.Run(ctx);; {
  k, err := t.Next(nil)

  ...

}
Issue Description Recommendation
Many datastore.next() calls. Your app made 193 remote procedure calls to datastore.next() while processing this request. This was likely due the use of 20 as query batch size. Increase the value of query batch size to reduce the number of datastore.next() calls

I cannot find the method to choose "query batch size". In NDB, batch_size option exists in query class, but GAE/Go does not seem to have.

I tried to patch query.go and set the Count (= batch_size?) to 200. I found it was finely fetching 200 entities in each Next request with no warnings.

I want to enable to set the Count with the NewQuery method. But is this a correct way?

delphinus pushed a commit to delphinus/appengine that referenced this issue Sep 24, 2017
This adds q.BatchSize() method to avoid the error that claims too many
datastore.next() calls due to the lack of specifying the `count`
property (= batch size).

Fixes golang#88
@sbuss sbuss closed this as completed in #90 May 16, 2018
sbuss pushed a commit that referenced this issue May 16, 2018
* datastore: enable to set query batch size

This adds q.BatchSize() method to avoid the error that claims too many
datastore.next() calls due to the lack of specifying the `count`
property (= batch size).

Fixes #88

* Disallow negative values for BatchSize

* Deal with negative value for t.limit

and clean up to be more readable

* Add comment about possible values for t.count

* Use the same `if` block as in `Run()` for `count`

* Make BatchSize restrict to be greater than zero
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant