Skip to content
Permalink
Browse files
fix(datastore): increase deferred key iter limit (#2878)
A customer was hitting the upper bound for iteration through
deferred keys, and the logic underlying the 100 iteration limit
was flawed. Based on internal discussions and what's happening
in other clients, it's fine to increase this value.
  • Loading branch information
tritone committed Sep 17, 2020
1 parent 5110dde commit 7f1057a30d3b8691a22c85255bb41d31d42c6f9c
Showing with 4 additions and 8 deletions.
  1. +4 −8 datastore/datastore.go
@@ -421,15 +421,11 @@ func (c *Client) get(ctx context.Context, keys []*Key, dst interface{}, opts *pb
}
found := resp.Found
missing := resp.Missing
// Upper bound 100 iterations to prevent infinite loop.
// We choose 100 iterations somewhat logically:
// Max number of Entities you can request from Datastore is 1,000.
// Max size for a Datastore Entity is 1 MiB.
// Max request size is 10 MiB, so we assume max response size is also 10 MiB.
// 1,000 / 10 = 100.
// Upper bound 1000 iterations to prevent infinite loop. This matches the max
// number of Entities you can request from Datastore.
// Note that if ctx has a deadline, the deadline will probably
// be hit before we reach 100 iterations.
for i := 0; len(resp.Deferred) > 0 && i < 100; i++ {
// be hit before we reach 1000 iterations.
for i := 0; len(resp.Deferred) > 0 && i < 1000; i++ {
req.Keys = resp.Deferred
resp, err = c.client.Lookup(ctx, req)
if err != nil {

0 comments on commit 7f1057a

Please sign in to comment.