New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No indices set up, causes 500s on views that involve read queries #8

Closed
brenns10 opened this Issue Feb 15, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@brenns10
Contributor

brenns10 commented Feb 15, 2017

I just got some data into my instance, and when I checked the "me" and "leaderboard" views (for example), I got 500s. It appears that the queries required indexes, and the indexes weren't created yet. The documentation was helpful reading. I checked and there were no indexes in the Datastore section of my console, even though we have an index.yaml file. I ran appcfg.py update_indexes . and now I can see them in my console (for some reason they didn't seem to be included in the initial deployment?). Not sure if there is anything that can be done about this error except maybe include some documentation about it. Regardless here are some sample tracebacks in the hopes that Google will bring others with the same problem here.

For "explore" endpoint:

Traceback (most recent call last):
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/util/decorators.py", line 19, in decorated_view
    return func(*args, **kwargs)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/views/web.py", line 117, in explore
    sent_loves=sent_love.get_result(),
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 995, in _run_to_list
    batch = yield rpc
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
    result = rpc.get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2921, in __query_result_hook
    yaml_index=yaml, xml_index=xml)
NeedIndexError: no matching index found. recommended index is:
- kind: Love
  properties:
  - name: secret
  - name: sender_key
  - name: timestamp
    direction: desc

The suggested index for this query is:
- kind: Love
  properties:
  - name: secret
  - name: sender_key
  - name: timestamp
    direction: desc

For "me" endpoint:

Traceback (most recent call last):
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/util/decorators.py", line 19, in decorated_view
    return func(*args, **kwargs)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/views/web.py", line 66, in me
    sent_loves=sent_love.get_result(),
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 995, in _run_to_list
    batch = yield rpc
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
    result = rpc.get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2921, in __query_result_hook
    yaml_index=yaml, xml_index=xml)
NeedIndexError: no matching index found. recommended index is:
- kind: Love
  properties:
  - name: sender_key
  - name: timestamp
    direction: desc

The suggested index for this query is:
- kind: Love
  properties:
  - name: sender_key
  - name: timestamp
    direction: desc

For top_lovers_and_lovees endpoint:

Traceback (most recent call last):
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/lib/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/util/decorators.py", line 19, in decorated_view
    return func(*args, **kwargs)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/views/web.py", line 136, in leaderboard
    top_lovers, top_lovees = logic.love_count.top_lovers_and_lovees(utc_week_start, dept=department)
  File "/base/data/home/apps/p~cwrulove/1-0-2.399201916859640285/logic/love_count.py", line 17, in top_lovers_and_lovees
    sent = LoveCount.query(LoveCount.week_start == utc_week_start).order(-LoveCount.sent_count).fetch()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/utils.py", line 160, in positional_wrapper
    return wrapped(*args, **kwds)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 1218, in fetch
    return self.fetch_async(limit, **q_options).get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 995, in _run_to_list
    batch = yield rpc
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
    result = rpc.get_result()
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2921, in __query_result_hook
    yaml_index=yaml, xml_index=xml)
NeedIndexError: no matching index found. recommended index is:
- kind: LoveCount
  properties:
  - name: week_start
  - name: sent_count
    direction: desc

The suggested index for this query is:
- kind: LoveCount
  properties:
  - name: week_start
  - name: sent_count
    direction: desc
@brenns10

This comment has been minimized.

Contributor

brenns10 commented Feb 15, 2017

In addition to running appcfg.py update_indexes ., I also had to run appcfg.py update_queues . to get the events queue working. This seems to be another useful thing to add to documentation for initial deploys.

@rockdog

This comment has been minimized.

Member

rockdog commented Feb 16, 2017

Closing this as #14 is addressing these issues and has just been merged.

@rockdog rockdog closed this Feb 16, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment