Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eve document count broken with concurrent requests #1271

Closed
ermescs opened this issue May 17, 2019 · 4 comments

Comments

Projects
None yet
2 participants
@ermescs
Copy link

commented May 17, 2019

This issue affects Eve when it is executed either on the Flask's development server with the default multi-threaded configuration, or when deployed in a production server such as Gunicorn using an async worker type (e.g. gevent).

Expected Behavior

For a given GET request, Eve should always return the proper document count in the _meta field, based on the client requested filter.

Actual Behavior

When performing multiple concurrent GET requests, in the scenario described above, there is a high chance of getting wrong results for the document count.

For instance, consider the following resource:

{
    "_id": 1,
    "value": "a"
},
{
    "_id": 2,
    "value": "a"
},
{
    "_id": 3,
    "value": "b"
}

If we perform the following two concurrent requests:

1. GET http://server/resource?where={"value": "a"}
2. GET http://server/resource?where={"value": "b"}

There is a high change of getting the following wrong results:

1.
{
    "_items": [
        {
            "_id": 1,
            "value": "a"
        },
        {
            "_id": 2,
            "value": "a"
        }
    ],
    "_meta": {
        "total": 2
    }
}

2.
{
    "_items": [
        {
            "_id": 3,
            "value": "b"
        }
    ],
    "_meta": {
        "total": 2    <---------------- WRONG!!!!
    }
}

or:

1.
{
    "_items": [
        {
            "_id": 1,
            "value": "a"
        },
        {
            "_id": 2,
            "value": "a"
        }
    ],
    "_meta": {
        "total": 1    <---------------- WRONG!!!!
    }
}

2.
{
    "_items": [
        {
            "_id": 3,
            "value": "b"
        }
    ],
    "_meta": {
        "total": 1
    }
}

This seems to be caused by the changes introduced in 61de6d8 and ac3bd78. With these changes, Eve stores the last used cursor and filter specification in a global object and later uses them to retrieve the document count. This can lead to the above issues when document count is accessed in a concurrent/async way for multiple requests/

Environment

  • Python version: 3.7.0
  • Eve version: 0.9
  • Flask version: 1.0.2
  • Gunicorn version: 19.9.0
  • Gevent version: 1.4.0
@nicolaiarocci

This comment has been minimized.

Copy link
Member

commented May 20, 2019

hi, which mongodb version are you on?

@ermescs

This comment has been minimized.

Copy link
Author

commented May 20, 2019

Hi, I'm using mongodb v4.0.1

@nicolaiarocci nicolaiarocci added this to the 0.9.1 milestone May 20, 2019

@nicolaiarocci

This comment has been minimized.

Copy link
Member

commented May 20, 2019

Could you please test if the this branch solves the issue for you?

@ermescs

This comment has been minimized.

Copy link
Author

commented May 20, 2019

Yes, with the changes in that branch the issue is solved ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.