New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pymongo throwing an exception when changing index #1180

Closed
Scipi opened this Issue Aug 17, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@Scipi

Scipi commented Aug 17, 2018

When updating an index through Eve with mongo_indexes, if the keys in the index changes after the index was already created, pymongo will throw an exception instead of Eve dropping the existing index and recreating it as claimed.


Expected Behavior

From http://python-eve.org/config.html

Also remember that, by default, any already existent index for which the definition has been changed, will be dropped and re-created

In my settings.py

DOMAIN = {
  'users': {
    'schema': {'username': {'type': 'string'}, 'token': {'type': 'string'}},
    'mongo_indexes': {'my_index': [('username', 1)]}
  }
}
# Run the app, the collection with the index gets created fine
# Later, we change the index here to
DOMAIN = {
  'users': {
    'schema': {'username': {'type': 'string'}, 'token': {'type': 'string'}},
    'mongo_indexes': {'my_index': [('username', 1), ('token', 1)]}
  }
}

# And run the app again

Actual Behavior

The existing index does not get dropped and recreated. Instead, pymongo throws an exception since it attempts to create another index with the same name but different keys

Traceback (most recent call last):
  File "run.py", line 2, in <module>
    app = Eve()
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/eve/flaskapp.py", line 178, in __init__
    self.register_resource(resource, settings)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/eve/flaskapp.py", line 932, in register_resource
    ensure_mongo_indexes(self, resource)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/eve/io/mongo/mongo.py", line 969, in ensure_mongo_indexes
    _create_index(app, resource, name, list_of_keys, index_options)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/eve/io/mongo/mongo.py", line 1020, in _create_index
    coll.create_index(list_of_keys, **kw)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/pymongo/collection.py", line 1754, in create_index
    self.__create_index(keys, kwargs, session, **cmd_options)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/pymongo/collection.py", line 1656, in __create_index
    session=session)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/pymongo/collection.py", line 245, in _command
    retryable_write=retryable_write)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/pymongo/pool.py", line 517, in command
    collation=collation)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/pymongo/network.py", line 125, in command
    parse_write_concern_error=parse_write_concern_error)
  File "/home/dev/envs/medscan3.6/lib/python3.5/site-packages/pymongo/helpers.py", line 145, in _check_command_response
    raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Index must have unique name.The existing index: { v: 2, key: { username: 1 }, name: "my_index", ns: "eve.users" } has the same name as the requested index: { v: 2, key: { username: 1, token: 1 }, name: "my_index", ns: "eve.users" }

Additional Info

I believe this is because the code Eve uses to recreate the index expects an error code from pymongo to be 85
https://github.com/pyeve/eve/blob/master/eve/io/mongo/mongo.py#L1059

A quick test shows the exception I receive has an error code of 86, which according to mongo's source represents IndexKeySpecsConflict (85 is IndexOptionsConflict)

https://github.com/mongodb/mongo/blob/master/src/mongo/base/error_codes.err#L87

Environment

  • Python version: 3.5
  • Eve version: 0.8
  • Pymongo version: 3.6.1
  • Mongo version: 3.6.5

@nicolaiarocci nicolaiarocci added this to the 0.8.1 milestone Aug 27, 2018

@nicolaiarocci

This comment has been minimized.

Show comment
Hide comment
@nicolaiarocci

nicolaiarocci Aug 27, 2018

Member

Thanks for reporting this.

Member

nicolaiarocci commented Aug 27, 2018

Thanks for reporting this.

nicolaiarocci added a commit that referenced this issue Aug 27, 2018

@nicolaiarocci

This comment has been minimized.

Show comment
Hide comment
@nicolaiarocci

nicolaiarocci Aug 27, 2018

Member

Can you please test the branch I just pushed: https://github.com/pyeve/eve/tree/pymongo_throwing_exception_when_changing_index_%231180

And let me know if it solves the issue for you?

Member

nicolaiarocci commented Aug 27, 2018

Can you please test the branch I just pushed: https://github.com/pyeve/eve/tree/pymongo_throwing_exception_when_changing_index_%231180

And let me know if it solves the issue for you?

@Scipi

This comment has been minimized.

Show comment
Hide comment
@Scipi

Scipi Aug 29, 2018

I can confirm that fixes the issue. Thanks c:

Scipi commented Aug 29, 2018

I can confirm that fixes the issue. Thanks c:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment