Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eve document count broken with concurrent requests #1271

Closed
ermescs opened this issue May 17, 2019 · 4 comments
Closed

Eve document count broken with concurrent requests #1271

ermescs opened this issue May 17, 2019 · 4 comments
Milestone

Comments

@ermescs
Copy link

@ermescs ermescs commented May 17, 2019

This issue affects Eve when it is executed either on the Flask's development server with the default multi-threaded configuration, or when deployed in a production server such as Gunicorn using an async worker type (e.g. gevent).

Expected Behavior

For a given GET request, Eve should always return the proper document count in the _meta field, based on the client requested filter.

Actual Behavior

When performing multiple concurrent GET requests, in the scenario described above, there is a high chance of getting wrong results for the document count.

For instance, consider the following resource:

{
    "_id": 1,
    "value": "a"
},
{
    "_id": 2,
    "value": "a"
},
{
    "_id": 3,
    "value": "b"
}

If we perform the following two concurrent requests:

1. GET http://server/resource?where={"value": "a"}
2. GET http://server/resource?where={"value": "b"}

There is a high change of getting the following wrong results:

1.
{
    "_items": [
        {
            "_id": 1,
            "value": "a"
        },
        {
            "_id": 2,
            "value": "a"
        }
    ],
    "_meta": {
        "total": 2
    }
}

2.
{
    "_items": [
        {
            "_id": 3,
            "value": "b"
        }
    ],
    "_meta": {
        "total": 2    <---------------- WRONG!!!!
    }
}

or:

1.
{
    "_items": [
        {
            "_id": 1,
            "value": "a"
        },
        {
            "_id": 2,
            "value": "a"
        }
    ],
    "_meta": {
        "total": 1    <---------------- WRONG!!!!
    }
}

2.
{
    "_items": [
        {
            "_id": 3,
            "value": "b"
        }
    ],
    "_meta": {
        "total": 1
    }
}

This seems to be caused by the changes introduced in 61de6d8 and ac3bd78. With these changes, Eve stores the last used cursor and filter specification in a global object and later uses them to retrieve the document count. This can lead to the above issues when document count is accessed in a concurrent/async way for multiple requests/

Environment

  • Python version: 3.7.0
  • Eve version: 0.9
  • Flask version: 1.0.2
  • Gunicorn version: 19.9.0
  • Gevent version: 1.4.0
@nicolaiarocci
Copy link
Member

@nicolaiarocci nicolaiarocci commented May 20, 2019

hi, which mongodb version are you on?

@ermescs
Copy link
Author

@ermescs ermescs commented May 20, 2019

Hi, I'm using mongodb v4.0.1

@nicolaiarocci nicolaiarocci added this to the 0.9.1 milestone May 20, 2019
@nicolaiarocci
Copy link
Member

@nicolaiarocci nicolaiarocci commented May 20, 2019

Could you please test if the this branch solves the issue for you?

@ermescs
Copy link
Author

@ermescs ermescs commented May 20, 2019

Yes, with the changes in that branch the issue is solved ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants