Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scripts end prematurely and randomly when using loop #1106

Closed
Galan2X opened this issue Jun 2, 2016 · 28 comments
Closed

Scripts end prematurely and randomly when using loop #1106

Galan2X opened this issue Jun 2, 2016 · 28 comments
Assignees
Labels

Comments

@Galan2X
Copy link

@Galan2X Galan2X commented Jun 2, 2016

When running a script within Robomongo that includes document enumeration, such as:
db.MyCollection.find().forEach(function(doc) {
// do some work, or print some data
});

If the amount of work is not trivial (such as a 'printjson(doc)' only) and the collection has hundreds of documents, the enumeration ends at random times before the end is reached, as if the cursor is sometimes advanced more than one step, or it believes it has reached the end. There are no errors thrown. The same script run locally on the server with the MongoDB shell does not exhibit this behavior.

This is for Robomongo 0.9.0-RC8 on Mac. It did not occur on earlier versions of Robomongo, like probably pre-0.9.0.

@simsekgokhan
Copy link
Collaborator

@simsekgokhan simsekgokhan commented Jun 6, 2016

Hi @Galan2X, we have reproduced the issue, we are now investigating the problem.
Thanks a lot for reporting.

Problem Summary:

Long running scripts (inserting) from shell ends before the end of script.

Investigation Details:

Steps to reproduce:
var document = {"link": {"status":65},"clients": {"enddt" : "test", "type" : "testtype", "clentid" : 1}};
for (var j=0; j<500; j++) // for j=100 working
{
db.aa6.insert(document)
}

Next Actions:

  • Investigate logs.
  • Check roboshell timeouts.
@juliashibalko juliashibalko changed the title Scripts end prematurely and randomly 0.9-RC8+ Scripts end prematurely and randomly when using loop Jun 29, 2016
@juliashibalko juliashibalko changed the title 0.9-RC8+ Scripts end prematurely and randomly when using loop Scripts end prematurely and randomly when using loop Jun 29, 2016
@juliashibalko juliashibalko added the bug label Jun 29, 2016
@sthammadi-spsc
Copy link

@sthammadi-spsc sthammadi-spsc commented Jul 1, 2016

Just to add, this issue also happen on Windows OS.

@simsekgokhan
Copy link
Collaborator

@simsekgokhan simsekgokhan commented Jul 1, 2016

Thanks for the input Sandeep. We will investigate the issue as soon as possible.

@juliashibalko
Copy link
Collaborator

@juliashibalko juliashibalko commented Jul 4, 2016

@Galan2X, @sthammadi-spsc, @oschrenk For Robomongo 0.9.x version as workaround let me suggest you next solution:

  1. Make sure Robomongo is closed

  2. Open robomongo.json configuration file .

[Update]: Adding Robomongo-Config-File-Guide

Please mind that for Robomongo 0.9.x version configuration file location differs from app 0.8.x version:

Windows
  0.9.x
  C:\Users\<user>\.config\robomongo\0.9\robomongo.json
  0.8.x
  C:\Users\<user>\.config\robomongo\robomongo.json   
MAC
    0.9.x
    /Users/<user>/.config/robomongo/0.9/robomongo.json
    0.8.x
   /Users/<user>/.config/robomongo/robomongo.json     
Linux
    0.9.x
    /home/<user>/.config/robomongo/0.9/robomongo.json
    0.8.x
    /home/<user>/.config/robomongo/robomongo.json
  1. Change "shellTimeoutSec" attribute value to higher number in seconds. (i.e. "shellTimeoutSec" : 60)
  2. Save config file and re-launch Robomongo app.
  3. Run necessary script.
    If script isn't executed entirely, try to increase shellTimeoutSec value.
    Please let me know in case of any questions
@sthammadi-spsc
Copy link

@sthammadi-spsc sthammadi-spsc commented Jul 4, 2016

@Galan2X, thanks for the workaround. Unfortunately, I couldn't find 'shellTimeoutSec' attribute in my robomongo.json file. I'm assuming I could add it in.

Below is my config:
{
"autoExec" : true,
"autoExpand" : false,
"autocompletionMode" : 1,
"batchSize" : 50,
"connections" : [
// Removed other connections here
{
"connectionName" : "Localhost",
"credentials" : [
{
"databaseName" : "admin",
"enabled" : false,
"mechanism" : "SCRAM-SHA-1",
"userName" : "",
"userPassword" : ""
}
],
"defaultDatabase" : "",
"serverHost" : "localhost",
"serverPort" : 27017
}
],
"disableConnectionShortcuts" : false,
"lineNumbers" : false,
"loadMongoRcJs" : false,
"style" : "Native",
"textFontFamily" : "",
"textFontPointSize" : -1,
"timeZone" : 0,
"toolbars" : {
"connect" : true,
"exec" : true,
"explorer" : true,
"logs" : false,
"open_save" : true
},
"uuidEncoding" : 0,
"version" : "2.0",
"viewMode" : 1
}

@juliashibalko
Copy link
Collaborator

@juliashibalko juliashibalko commented Jul 5, 2016

@sthammadi-spsc For version 0.9.0-RC8 or 0.9.0-RC9 this parameter usually set by default.
Let me offer you just to double-check version of Robomongo using and configuration file location on C:\Users\<user>\.config\robomongo\0.9\robomongo.json.
If there is still no "shellTimeoutSec" parameter, you could add both missed attributes manually after "loadMongoRcJs" : false, line:

 "mongoTimeoutSec" : 10,
 "shellTimeoutSec" : 60,

Then save robomongo.json file and start Robomongo.
Please let me know if you have any questions.

@teliatko
Copy link

@teliatko teliatko commented Jul 12, 2016

@juliashibalko hi, what does the mongoTimeoutSec mean? How is this timeout different from shellTimeoutSec? Where is which one used?

Btw. can you check my comment in #1098 (comment)

Thanks.

@juliashibalko
Copy link
Collaborator

@juliashibalko juliashibalko commented Jul 12, 2016

Hi @teliatko,
Thanks for interest in Robomongo! As you know Robomongo embeds mongo shell. So we've set two different timeout value attributes for MongoDB shell and for Robomongo application itself. Therefore shellTimeoutSec is used for scripts running in the shell only (e.g. db.getCollection().find() executing) and mongoTimeoutSec is used for actions executed outside the shell (e.g. expanding collections list).
Hope I've answered your question!

@simsekgokhan
Copy link
Collaborator

@simsekgokhan simsekgokhan commented Dec 8, 2016

Hi @jfamme, thanks a lot for suggestions. All make perfect sense.
We have created two issues for this problem. We will work on these when we will have chance.
#1254
#1253

@snarlynarwhal
Copy link

@snarlynarwhal snarlynarwhal commented Dec 9, 2016

This has been an issue for about 8 months.. and it's a pretty significant issue. :/ Can we get a status update? Has any progress been made?

@simsekgokhan
Copy link
Collaborator

@simsekgokhan simsekgokhan commented Dec 12, 2016

Hi @snarlynarwhal , currently priority is delivering critical features like Replica Sets and Export/Import. And the number of contributors of Robomongo is not too many. This issue is tagged as high vote and major, for sure it will be one of the first to work after delivering features mentioned above.

The workaround here #1106 (comment) seems to be working for lots people. Can you let us know if this helps you for now? If you have another problem, we will be happy to help.

@snarlynarwhal
Copy link

@snarlynarwhal snarlynarwhal commented Dec 12, 2016

Okay sounds good, thank you very much! :)

@jeremycooper1
Copy link

@jeremycooper1 jeremycooper1 commented Dec 14, 2016

Hi I tried up shellTimeoutSec" : 120 - the query I'm trying to run in version 9 only takes 6 seconds, yet it still won't appear running:
db.currentOp().inprog.forEach( function(op) { if(op.secs_running > 5) printjson(op); } )
but does in version 8 - not sure this is the same issue of this thread

@juliashibalko
Copy link
Collaborator

@juliashibalko juliashibalko commented Dec 16, 2016

@jeremycooper1 Yes, you are correct - it's another issue. So let's continue investigation in scope of #1259 card.

@jsheely
Copy link

@jsheely jsheely commented Feb 5, 2017

@simsekgokhan I'd consider that this is a higher priority issue than that of new features.

Given this example:
In its current form you run a script that loops over a collection of say 10,000 items and you insert them into another collection. The script says it ran successfully, no warnings, no errors.. But you only get 300 items in the new collection.

Now you're in a position where you could lose a significant amount of data. This is something I would urge you to re-consider the importance of if time permits.

The current work around still has some significant issues. Unless you set these timeouts to be in terms of hours it is still possible to get false positive of completion if your script would happen to run longer than the timeout.

@simsekgokhan
Copy link
Collaborator

@simsekgokhan simsekgokhan commented Feb 7, 2017

Hi @jsheely, I totally agree with you. Thanks for the example and reminding us.

@roberterdin
Copy link

@roberterdin roberterdin commented Mar 8, 2017

I agree with @jsheely. The fact that there is no warning is very dangerous. I was just seconds away from a data loss that would have been very expensive to us. Data that was purchased from a third party was almost lost because two collections were supposedly merged and the obsolete collection deleted. Luckily I double-checked with a document at the end of the target collection so this was avoided.

@snarlynarwhal
Copy link

@snarlynarwhal snarlynarwhal commented Mar 8, 2017

I think this needs to be resolved asap also.

@gauravkalal
Copy link

@gauravkalal gauravkalal commented Mar 16, 2017

@juliashibalko This solution works. Thank you.

@RokoMijic
Copy link

@RokoMijic RokoMijic commented Apr 18, 2017

It might be a good idea to edit the error message that is shown to make the user aware of what happened and what the resolution is. I can't imagine that it would take long to edit the text of the error message to say something more sensible.

@simsekgokhan
Copy link
Collaborator

@simsekgokhan simsekgokhan commented Apr 21, 2017

@RokoMijic , thanks a lot for your interest and suggestion. We will consider.

simsekgokhan added a commit that referenced this issue May 19, 2017
…the long lasting scripts were ending prematurely and silently due to shell timeout reached ("shellTimeoutSec" parameter in config file)
@simsekgokhan simsekgokhan self-assigned this May 19, 2017
@simsekgokhan simsekgokhan added this to Finished in Robomongo 1.1 May 19, 2017
simsekgokhan added a commit that referenced this issue May 20, 2017
…o change shell timeout ("shellTimeoutSec") at program run time (and save into config file), effective immediately on all existing/new tabs and persistent after program restart.
@juliashibalko juliashibalko moved this from Fixed / Ready for testing to Done in Robomongo 1.1 Jun 12, 2017
@simsekgokhan
Copy link
Collaborator

@simsekgokhan simsekgokhan commented Jun 22, 2017

Hoping the related code changes for this problem and UI option to configure shell timeout which are included in the latest version Robo 3t 1.1 will solve this problem.

More:
Blog section: Fix for Robo 3T Shell Timeout Issue
All sections: http://blog.robomongo.org/robomongo-is-robo-3t

@RokoMijic
Copy link

@RokoMijic RokoMijic commented Jun 22, 2017

@simsekgokhan Looks good!

@afirdousi
Copy link

@afirdousi afirdousi commented Jan 3, 2020

In 2019 at Robo 3T 1.2.1, still seeing the same issue while running aggregation queries. My shell timeout is set to 360.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.