New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API occasionally sends a malformed query and Mongo closes connection #246

Closed
fra967 opened this Issue Apr 26, 2017 · 5 comments

Comments

Projects
3 participants
@fra967
Copy link
Member

fra967 commented Apr 26, 2017

This issue appears occasionally in an API production environment used by Publish

MongoDB log reports an error and the database closes the connection, causing a sequence of API errors and data loss

The error reported by MongoDB is:

Assertion: 10307:Client Error: bad object in message: invalid bson type in object with unknown _id
AssertionException handling request, closing client connection: 10307 Client Error: bad object in message: invalid bson type in object with unknown _id

Being a live environment, it is difficult to trace which query creates this issue, but it has been seen after PUT or DELETE requests.

Probably the first step would be to introduce verbose logging (with the full content of the request sent to the database) in order to gather more info

API 1.15.5
Node 6.10.2

@jimlambie

This comment has been minimized.

Copy link
Member

jimlambie commented Apr 26, 2017

@fra967 what is th mongodb version?

@fra967

This comment has been minimized.

Copy link
Member Author

fra967 commented Apr 26, 2017

MongoDB 3.2.10 (it's a replicaset)

@fra967

This comment has been minimized.

Copy link
Member Author

fra967 commented May 2, 2017

I managed to trace the offending query, which is probably sent by a badly written hook, yet to be identified

It's a simple empty _id: { _id: }

If I try to hit API with a similar query, it does not generate the MongoDB error

Maybe there is some validation, which doesn't happen when the query is sent directly from the model?

@jimlambie

This comment has been minimized.

Copy link
Member

jimlambie commented May 3, 2017

The error originates from a hook that passes API a query containing an ObjectID. API attempts to make this case insensitive and it looks as if a case insensitive ObjectID = "".... The hook can be changed to pass a string in the query instead, as a temporary fix.

Two pieces of work:

  1. make the case-insensitive method ignore internal fields such as _id
  2. include a check inside the relevant model operation that determines if the call came from a hook and if it does, skip the hook step.

@jimlambie jimlambie added this to Backlog in API Version 2 May 31, 2017

@jimlambie jimlambie modified the milestone: 2.0.0 May 31, 2017

@jimlambie jimlambie referenced this issue Jun 3, 2017

Merged

Version 2.0.0 #297

@jimlambie jimlambie moved this from Backlog to Review in API Version 2 Jun 3, 2017

@jimlambie

This comment has been minimized.

Copy link
Member

jimlambie commented Jun 3, 2017

Closed by d103f62

@jimlambie jimlambie closed this Jun 3, 2017

@jimlambie jimlambie removed the in progress label Jun 3, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment