Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregation Freamwork - cursor id <NUMBER> didn't exist on server #457

Closed
Roycohen opened this issue Nov 14, 2016 · 7 comments
Closed

Aggregation Freamwork - cursor id <NUMBER> didn't exist on server #457

Roycohen opened this issue Nov 14, 2016 · 7 comments
Labels

Comments

@Roycohen
Copy link

@Roycohen Roycohen commented Nov 14, 2016

Hi There,

I'm using the aggregation framework and most of the features are working very good.
There is one issue if there are long queries and there is no way to set the cursor timeout or something like that.

The Error I'm getting is: "cursor id xxxxxxx didn't exist on server".
Please help me resolve it.
maybe configuration on the Mongo server themselves.
Maybe you have some idea or insights whether Mongo going to address this issue.

I'm using the following version:
mongodb
mongodb support => enabled
mongodb version => 1.2.0alpha3
mongodb stability => alpha
libmongoc version => 1.5.0-rc0
libbson version => 1.5.0-rc0
mongodb.debug => no value => no value

Thanks Guys.

@jmikola
Copy link
Member

@jmikola jmikola commented Nov 15, 2016

There is one issue if there are long queries and there is no way to set the cursor timeout or something like that.

Can you elaborate on what a "long query" is in your case? According to the MongoDB manual, idle cursors will time out after an inactivity period of 10 minutes. I believe that inactivity period is the time between getmore requests for additional batches of results.

While the find command does have a noCursorTimeout option to prevent inactive cursors from timing out, there is no such option for the aggregate command. If you'd like to make a case for this, you could open a SERVER ticket in JIRA or inquire on the mongodb-user mailing list.

A possible work-around may be to lower your batchSize on the aggregate command in an attempt to cause the driver to issue getmore requests more frequently during cursor iteration.

@Roycohen
Copy link
Author

@Roycohen Roycohen commented Nov 16, 2016

OK.
Thanks for the response.
I understand that they are planing to make an option to support timeout or something like this. DO you know something about it?

In addition, assuming my query performs FULL TABLE SCAN in several shards, do you think reducing the batch size may help?

@lindelius
Copy link

@lindelius lindelius commented Nov 17, 2016

Depending on your query you might want to change the filter and/or the sorting in order to return the results quicker, and then instead do any additional checks in your application or script.

@jmikola
Copy link
Member

@jmikola jmikola commented Nov 17, 2016

I understand that they are planing to make an option to support timeout or something like this. DO you know something about it?

The only related issue I found in JIRA was SERVER-6036, which looks relevant based on the linked Python driver issue. The feature is still not scheduled for implementation, and I don't know anything more than what is in the ticket itself.

In addition, assuming my query performs FULL TABLE SCAN in several shards, do you think reducing the batch size may help?

Batch size does not influence whether a full table scan is performed. That is more of a question of the index (if any) that is used to start your aggregation pipeline. If you're not starting the pipeline with some combination of a $match, $limit, $sort , or $geoNear, it is going to need to consider every document in the collection and perform a full scan.

@Roycohen
Copy link
Author

@Roycohen Roycohen commented Nov 24, 2016

OK, Thanks.
So there is nothing we can do on our end...

Another question about the batch size.
If I set it to lower number, the result will be with all the data that fits to the aggregation pipeline, meaning I don't need to do some loops and and and iterations.
The batch size is just how many results the driver will get from the server and will issue get more commands until it gets the entire data, right?

@jmikola
Copy link
Member

@jmikola jmikola commented Nov 25, 2016

The batch size is just how many results the driver will get from the server and will issue get more commands until it gets the entire data, right?

That is correct. We're only modifying the payload size between the driver and the server. This won't affect how you iterate on the results via a cursor object, since those getMore commands happen behind the scenes.

@jmikola
Copy link
Member

@jmikola jmikola commented Dec 9, 2016

Closing this, as the last question was answered. Feel free to re-open as needed if the issue isn't fully resolved.

@jmikola jmikola closed this as completed Dec 9, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants