-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Sorting issues / DataTables warning: table id=runs - Ajax error. #70
Comments
Hi Johannes,
Thanks a lot. |
Hi, I can reproduce the error every time I change the sorting by clicking on "Experiment name", "Command" or "Hostname". I remember having the error when deactivating some of the statuses on Friday but I could not reproduce that today. I tested with Firefox 54.0 and Chromium 59.0.3071.109 on Ubuntu 17.04. There are only Errors for missing files and a Server error on the JavaScript console. Here are some screenshots: |
Thanks to your observation, I discovered another minor issue but that was probably not causing your problem. But I was unable to reproduce it. Sorry for the inconvenience. |
Might this be related to stdout/stderr logging? for me, this happened when I had only a handful of runs stored, but with long outputs. I get sth like this:
Which looks to be related to this |
Sorry for my late reply. I get this error on the console:
My outputs are also pretty long because my code displays progress bars. |
Thanks for posting the output! It really seems to be related to the issue that @black-puppydog posted. |
@pinae I'm pretty sure it's a CORS error. Check out the datatables.net link they provide and read about it some more for different solution. A quick test to see if this is the case is to check out the web app from a different computer on the same network (swapping 127.0.0.1 for xxx.x.x.xx for whatever your local IP is). |
Hey there, I'm frequently stumbling upon this issue as well. I assume that it happens when the stored log output of the experiment is very long (e.g. training a model with Tensorflow for a couple of days). Any updates on a potential fix? |
@schroederen Try to add some indices. I added some and it fixed the problem. I missed to write down what exactly I did and realized after I reported the issue that it would have been beneficial. |
@pinae Thanks for the reply. For what key did you create the indices? i.e. which parameters did you use for db.collections.createIndex()? |
In the meantime, one can increase the limit of the search buffer, as described here. I Increased it to 50MB (from 30) and this fixes the issues I'm having. However, I expect to run out of buffer again eventually, so the thing with the indices might be a more elegant solution. |
I created an index for each column in the board and it fixed the problem for me:
To create an index see https://docs.mongodb.com/manual/indexes/ |
Did this and it fixed the issue for a while, but it has returned. Additionally, I'm getting other wierd issues now: Some experiments stop to show up and also sorting by ID does not work correctly anymore. Maybe it wasn't such a good idea to create indices for all columns or I didn't do it correctly? :D @chovanecm Is there any "official" fix incoming? Would be greatly appreciated! |
For those that don't know how to create an index (like I didn't), you can use createIndex in the Mongo CLI. So to add a heartbeat index I did the following:
|
I am considering letting Sacredboard automatically create indices for the displayed columns. I am just afraid of what happens if I implement #24 (adding custom columns). |
I added indices for all possible entries, but I'm still getting this error on some columns. I've managed to get a trace from the console:
This looks like a different error, but it happens when trying to sort entries by some of the columns. |
@anibali I had the same problem. Adding the index you described immediately resolved the problem. So probably letting sacredboard do this automatically is a good idea. |
Same here, adding the indices worked like magic!
|
Hi I am getting this issue when using the FileObserver... me:~/Projects/sacred_test$ sacredboard -F experiments/tests/
SacredBoard shows:
|
Sacredboard shows this error if I try to edit the filters:
This seems to be a problem with missisng indexes in the mongodb (as far as I know). I originally got this error when starting sacredboard but after I created an index for start and end dates it only shows up when I change the filters.
I'm using the default settings for a mongodb installation on Ubuntu 17.04. Memory usage for sorting without an index seems to be limited to 32MB in this configuration.
If this is no bug in Sacredboard please add some documentation for the correct settings.
The text was updated successfully, but these errors were encountered: