Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aggregate query causes skip/batch size controls to disappear and search results to be limited in 50 docs #1058

Closed
LanceBeasley opened this issue Mar 17, 2016 · 24 comments

Comments

@LanceBeasley
Copy link

commented Mar 17, 2016

Here is the query I was using:

db.getCollection('col_buckets').aggregate([
    {$match: {vDevId: "HMM1_5_F_17"}},
    {$limit: 500},
    {$sort: {startTime: -1}},
    {$unwind: "$values"},
    {$project: {'value': '$values.value', 'dischargeMs': '$values.dischargeMs', updateTime: "$values.updateTime",_id:0}},
    {$sort: {"updateTime": -1}}
    ])

Basically pulling values from an array in several documents.

The skip/batch controls disappear after running the query. The following screen shots show what happens.

Before running query...
robomongoo before

After running query...
robomongo after

@kfitzgerald

This comment has been minimized.

Copy link

commented Mar 18, 2016

So it seems that 0.9 changed the way aggregates are run, using a cursor instead of just returning all documents. I'm reverting to the 0.8 releases until this can issue can be resolved. :(

@schetnikovich

This comment has been minimized.

Copy link
Contributor

commented Mar 25, 2016

Confirmed. We will fix this problem in newest release. Thank you for reporting this issue!

@schetnikovich

This comment has been minimized.

Copy link
Contributor

commented Mar 30, 2016

@LanceBeasley @kfitzgerald As a workaround, until we will make it more convenient for users, you can view all documents of the aggregate command using the following script:

var cursor = db.collection.aggregate([ ... ]);
while (cursor.hasNext()) {
   print(cursor.next());
}

If you have less than 10,000 documents this will probably work for you. If you have more, or you still need to view documents in a smaller portions, you can do the following:

var cursor = db.collection.aggregate([ ... ]);
cursor.shellPrint()

Now run this script. If you want to view the next page, select cursor.shellPrint() text and run only this selected script (Ctrl/Cmd + Enter):

cursor-shell-print

Function shellPrint() is a part of MongoDB shell and it simply iterates over the cursor by DBQuery.shellBatchSize documents (which is 50 by default).

If you need to start from the beginning, remove any selection, and execute full script. You'll be on the first page.

Advanced

Method shellPrint is called automatically by MongoDB shell, when you "execute" cursor. It means that you can have a shorter version:

cursor

@LanceBeasley

This comment has been minimized.

Copy link
Author

commented Mar 30, 2016

Thanks for the tip!

Lance Beasley
ENCO Electronic Systems http://www.encoelectronics.com/

On Wed, Mar 30, 2016 at 2:56 AM, Dmitry Schetnikovich <
notifications@github.com> wrote:

@LanceBeasley https://github.com/LanceBeasley @kfitzgerald
https://github.com/kfitzgerald As a workaround, until we will fix this
issue, you can view all documents of the aggregate command using the
following script:

var cursor = db.collection.aggregate([ ... ]);
while (cursor.hasNext()) {
print(cursor.next());
}

If you have less than 10,000 documents this will probably work for you. If
you have more, or you still need to view documents in a smaller portions,
you can implement simple paging yourself:

function showNextPage(cursor) {
var n = 0;
while (cursor.hasNext() && n < 50) {
print(cursor.next());
n++;
}
}

And use it in the following way:

var cursor = db.collection.aggregate([ ... ]);
showNextPage(cursor);

If you want to view next page, select nextPage(cursor) text and run this
selected script (Ctrl/Cmd + Enter):

[image: next-page]
https://cloud.githubusercontent.com/assets/678660/14135355/afe8b866-f665-11e5-8478-ec0bf69b577d.png


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1058 (comment)

@anorsich anorsich added the bug label Mar 31, 2016

@juliashibalko juliashibalko changed the title Aggregate query causes skip/batch size controls to disappear Aggregate query causes skip/batch size controls to disappear and search results to be limited in 50 docs Jun 16, 2016

@stasm

This comment has been minimized.

Copy link

commented Aug 23, 2016

As another workaround, you can specify { $out: "temp_result" } as the last step of the aggregation pipeline which will put the results in the temp_result collection. You can then view and query them as normal and drop the collection once you're done with it.

@WannerZoer

This comment has been minimized.

Copy link

commented Oct 20, 2016

any new report?

@juliashibalko

This comment has been minimized.

Copy link
Collaborator

commented Oct 25, 2016

Hi @WannerZoer,
this issue is in our to-do list. Now we are working on Replica Set implementation. Please check our feature list for future release: https://github.com/paralect/robomongo#whats-planned-for-the-next-releases
At moment we include this ticket into release, we will get back to you with all updates!

@Nowaker

This comment has been minimized.

Copy link

commented May 17, 2017

Is there really no way to display more than 50 aggregated records?

@simsekgokhan

This comment has been minimized.

Copy link
Collaborator

commented May 18, 2017

Hi @Nowaker , currently the workaround below, from #1058 (comment), seems to do the trick.

var cursor = db.collection.aggregate([ ... ]);
while (cursor.hasNext()) {
   print(cursor.next());
}
@abulhawa

This comment has been minimized.

Copy link

commented Jun 11, 2017

This does not work for me:

var cursor = db.collection.aggregate([ ... ]);
while (cursor.hasNext()) {
   print(cursor.next());
}

When I ran it, robomongo appeared busy for a few seconds then suddenly crashed. all unsaved scripts were lost.
env: windows 10 pro, Robomongo 1.0, MongoDB server 3.4.4

@simsekgokhan

This comment has been minimized.

Copy link
Collaborator

commented Jun 12, 2017

Hi @abulhawa , Robomongo 1.0 does not support MongoDB 3.4.
Please use Robomongo 1.1 - Beta version with MongoDB 3.4 Support.

@abulhawa

This comment has been minimized.

Copy link

commented Jun 12, 2017

Hi @simsekgokhan, thanks for this. I tried it, but Robomongo still crashes.

@simsekgokhan

This comment has been minimized.

Copy link
Collaborator

commented Jun 12, 2017

@abulhawa , I just tried with Windows 10, was not able to reproduce the crash.
Can you please create a new ticket here https://github.com/Studio3T/robomongo/issues/new.
Please also include crash report/log file and exact/very similar script that you are running for further investigation.

@thechile

This comment has been minimized.

Copy link

commented Jun 16, 2017

still only shows 50 using latest 1.1 release.

@shishir99111

This comment has been minimized.

Copy link

commented Jun 21, 2017

FINALLY GOT PERFECT SOLUTION AFTER TRYING FOR DAYS!!..
try using toArray with the query, it will return all the records in Array.
eg:
db.getCollection('debits').aggregate([{"$group" : { _id: "$timestamp"}}]).toArray()

screen shot 2017-06-21 at 1 42 34 pm

@Nyadesune

This comment has been minimized.

Copy link

commented Jul 3, 2017

i'm using 1.1.1 version and still get this issue.

".toArray()" trick works, is a good workaround.

@leandrosa

This comment has been minimized.

Copy link

commented Jul 4, 2017

You must be careful for use .toArray(), because it return all the documents to the memory. If many documents were returned, it can be fill the memory so much

@crertel-packlane

This comment has been minimized.

Copy link

commented Aug 17, 2017

It'd be great to have this fixed! What's the rough cause, if I can ask?

@donnyv

This comment has been minimized.

Copy link

commented Aug 22, 2017

@crertel-packlane Studio3T bought RoboMongo, thats why its never getting fixed.

@benevbright

This comment has been minimized.

Copy link

commented Oct 26, 2017

Thanks for the trick but why it has not fixed so long time.

@tomasznaumowicz

This comment has been minimized.

Copy link
Collaborator

commented Oct 27, 2017

As of the 0.9 release, results of a query are being returned as a cursor instead of just returning all documents. Otherwise one wouldn't be able to run larger queries. Not being able to iterate over the results makes this improvement insufficient. We will review this decision and -if needed - revert the change.

In the meantime, please use the $out stage to dump the results to a temp collection.

@RachelScodes

This comment has been minimized.

Copy link

commented Oct 17, 2018

As of 1.2.1 this still is not fixed. Is anyone maintaining this project?

@abulhawa

This comment has been minimized.

Copy link

commented Oct 18, 2018

@RachelScodes it is working for me, robo 3T 1.2.1, on windows 10, MongoDB 3.6.3

@simsekgokhan

This comment has been minimized.

Copy link
Collaborator

commented Nov 16, 2018

Hi all, sorry for the long delay, this issue should be fixed in Robo 3T 1.2.
Please see: https://blog.robomongo.org/robo-3t-1-2/#a2

@RachelScodes
Your problem might be different than this ticket (which is for "Aggregate query results are not pagaeble" problem). Please also see the blog link above.
Can you give more details about your problem?
Error screenshot, your script, versions of your OS, Robo, MongoDB. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.