OctoPrint web interface becomes very sluggish (but print continues) #1065
Comments
|
I had custom controls set in the config that are now missing, not sure if that is related. |
|
That sounds like something in the websocket was acting up, blocking stuff on the network layer in the process. Did the starting of the observed behaviour happen along the same time the Also please provide the full |
foosel
added
the
needs information
foosel
changed the title
|
@foosel, sorry, I can't say for sure if the behavior happened along the same time as the error, this error report came from one of our users. Regarding the whole log, unfortunately it is gone now, but when I copied and pasted the above there was nothing else in there except the startup lines. Regarding reproducibility, I might have captured it again (And this time I have the full logs and terminal tab). Similar behavior where the terminal tab stopped showing lines go by, but the print was still running. The time remaining stopped going up as well. Reloading the page did not fix the issue. I looked at the javascript console and no errors were reported. I then loaded OctoPrint on another machine and everything seemed fine there. I reloaded OctoPrint on the original machine and everything was working as expected. However after about 2 more minutes the printer stopped moving and the terminal tab repeatedly said: "Communication timeout during printing, forcing a line". I had also SSH'd into the pi running octoprint to check system status. octoprint.log: http://pastebin.com/jkWiFnqN The Please let me know if there is any other data I can collect that could be useful. |
|
I'm experiencing very similar symptoms, after upgrading to the latest 1.2.7 update available for the raspberry pi Octoprint distribution. The interface is perfect - until I hit the print button. Then, during the printing (which also appears fine), network connectivity to the pi fails. The web interface becomes unresponsive & I cannot ssh to the pi. As soon as printing finishes, everything returns to normal. This did not happen on previous versions of the distribution. Happy to help diagnose - what can I capture to help here? Thanks Tim |
|
@normanclanger that sounds like a different issue than what @jminardi is reporting, considering that apparently for him it was the interface not getting data pushed from the server but the Pi still being responsive, whereas your symptoms more sound like the whole Pi is under such a heavy load that it no longer can respond in time to anything else (e.g. SSH login, serving web requests etc). The question is what the load is on your server when that happens - considering that you cannot login that might be a bit tricky to figure out though. Just a guess but: make sure you do not have the "Log communication to serial.log" checkbox checked (in Settings > Serial Connection). If you happen to have a webcam connected, try if the same issue occurs if you don't. Also try without any non-bundled plugins being enabled. |
foosel
added
unreproduced
|
Oh, and of course: logs, logs, logs ;) |
|
Thanks. I've checked the "Log Communication to serial log" - that is not checked. I have the pi camera attached, but it wasn't enabled. I haven't any bundled plugins installed. octoprint.log below - of significance is the statement 'error: [Errno 113] No route to host' - could this be causing the loss of connectivity? 2015-10-26 09:33:28,207 - octoprint.server.util.sockjs - INFO - New connection from client: 192.168.2.21 |
|
That indeed sounds like the pi simply losing its network connection (hence no route to the client) but happily churning along otherwise. Are you sure the network connection is solid? When you start a print, the traffic increases due to the amount of data pushed to the client (terminal tab contents, progress). If the network can't handle that increase in traffic, I could imagine connectivity of the client to the server breaking down altogether being a symptom of that. |
|
I'm having a similar issue since the upgrade to 1.2.7 on my OctoPi image. Used the standard upgrade option to install the update and nothing seemed out of the ordinary. What I experience is after it prints for a while the web interface just decides to not update for a while and in some cases causes Chrome in Windows 10 to give the Aw Snap error, something went wrong. It never seems to cause the print to mess up and a page refresh always brings it back to being responsive again. Not sure I have my settings set correctly for good debugging on this issue, as there were only errors related to urllib3 and ssl connections. Only custom thing I have set-up on my OctoPi install is ssl configuration in haproxy, client certificate verification in haproxy config and user authentication in haproxy. OctoPrint settings set to disable users completely since I handle it all through haproxy. I use events in config to email snapshots, upload video to youtube, as described in the wiki. The only non-foosel plugins I have installed is FlashArduino (0.1). |
|
I really wish I'd see this myself, I have absolutely no idea to tackle it right now :( What you could try is downgrading to an earlier version by checking out the tag, e.g. |
|
Can you suggest a script to run to capture some diagnostics while we print? For example, I left a ping going writing to a file and this continued to work. Are there any http server metrics we could gather? Sent from my HTC ----- Reply message ----- I really wish I'd see this myself, I have absolutely no idea to tackle it right now :( What you could try is downgrading to an earlier version by checking out the tag, e.g. Reply to this email directly or view it on GitHub: |
|
The issue is most likely client side from how it is described, so there's
nothing server side to track by a script.
If there are no errors in the javascript error console (that's something
that could be checked) or constant disconnects of the websocket visible in
the network tab, it boils down to trying to find a way to reliably
reproduce it and then poke with a stick from all sides, see if it manifests
in earlier versions, etc.
You could check if a continuous ping from the client to the server during
printing behaves properly, that at least would rule out any general
connectivity issues causing this.
It might also be interesting to know the exact browser versions this is
happening with and whether the tab is in the foreground when it happens or
is not visible (maybe some overly eager tab suspending?).
|
|
I've experienced it from 3 different browsers on 3 different os. Plus ssh fails. Will try ping. Sent from my HTC ----- Reply message ----- The issue is most likely client side from how it is described, so there's If there are no errors in the javascript error console (that's something You could check if a continuous ping from the client to the server during It might also be interesting to know the exact browser versions this is Reply to this email directly or view it on GitHub: |
|
Ok, so I was able to reproduce the aw snap error in chrome with developer tools open. There's nothing that seems out of the ordinary in network tab, the server is still responsive to pings on the network and as stated before a simple refresh of the browser reloads OctoPrint without any issues. The only error in console is as follows: Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check http://xhr.spec.whatwg.org/. less.min.js:8 |
|
Can you give more information about how you reproduced the bug. Any other plugin installed then FlashArduino? What were you doing? What Terminal settings were set? What filters were activated? |
|
It happens extremely randomly, so not really sure if it's specifically reproducible. But this time it was on the third print of the day that it happened. I've disabled timelapse completely to insure the pi wasn't getting over taxed. The following plugins are the only ones that aren't listed as "bundled'. Yamlpatcher (0.1.0), Flash Arduino (0.1) Other customization are as follows:
Connecting to RAMBo board at 250000 baud. I just looked and octoprint.log is non-existant as I deleted it the last time to start from scratch. I was going to try Foosels downgrade procedure but have been printing pretty regular the whole time. As stated before, it doesn't effect the print at all it just seems the web interface chokes up or causes some memory issue with Chrome running in Windows 10 (for me). I haven't personally tried in another browser. |
|
So with the developer tools opened this time and network tab active i noticed the Aw snap error on the page and the developer tools has an error "inspected target disconnected". I had a ping running from the same client and once the problem happened I started getting much higher latency (300-600+ms) on the ping results, along with a couple of time outs. If I try to refresh while the developer tools is open and print ongoing I get a line in the status bar of chrome stating waiting for available socket. Closed developer tools and refreshed the page and ping latency dropped below 10ms and no timeouts. |
|
I'm seeing similar responses. Ping goes from 3ms to 1000-3000ms - or just reports host unresponsive. Nmap detects ssh and port 5000 prior to printing, but nothing when printing. The network becomes unresponsive almost as soon as the printer head starts moving; while heating it is fine. My router reports no traffic from the pi, so the network card isn't saturated. I'm pinging from a machine not running a web browser, so there is no client side interaction. Thanks Tim
|
|
From my view, a model sliced with Cura causes sluggishness in OctoPrint
during the initial stages of printing, vs Slic3r where there is no
noticeable sluggishness.
|
|
I know this is off topic, so apologies. However, when I try to roll back to a previous version, I can only get OctoPrint to upgrade (to 1.3 devel). The screen output is as follows: pi@pi2 ~/OctoPrint $ git checkout 1.2.6 pi@pi2 ~/OctoPrint $ ./venv/bin/python setup.py install Using /home/pi/OctoPrint/venv/lib/python2.7/site-packages/pytz-2015.4-py2.7.egg Why can I not roll back? Thanks, Tim |
|
You are rolling back, but apparently there is still a bug in the part that generates the version number during install off the checkout. It doesn't recognize that a tag is checked out and so it falls back on the default for unrecognized "branches", which is 1.3.0 Notice how the part after the g is the exact commit hash the tag represents. So you have 1.2.6 checked out and installed just fine, it's just reporting wrong. I'll need to fix that, but can't do that retroactively, so just ignore it for now. |
…visible This might help with #1065 if indeed is related to background tab suspending behaviours in browsers, but is a completely blind guess at this point since I still have not been able to reproduce that issue myself.
|
Ok, after @Salandora could reproduce the problem, we took a guess at what might be causing it and I created a fix for that. At least for @Salandora the problem didn't reproduce any more, but I'd feel more confident marking this issue as closed if other people seeing this behaviour also got better results with the current maintenance branch (future 1.2.8 release). So - could you guys please update to the (For custom installations, change the path to your OctoPrint checkout folder (here: Anything above Background: It might be caused by a combination of background tab suspending and how the GCODE viewer reacts to progress updates. There might be a backup in updates when the tab is backgrounded, causing the viewer to try to render hundreds of updates once the tab regains focus. The patch changes the behaviour such that OctoPrint now tracks if the browser tab containing OctoPrint and the tab within OctoPrint with the GCODE viewer are in focus, only trying to update the viewer if both is the case. While at it I also made the webcam view unload the webcam stream if the browser tab loses focus, no need to transfer that much data if nobody is watching. That change should improve performance in general, even if it doesn't fix that specific issue you are observing (I still haven't been able to reproduce it myself, making it somewhat tricky to debug it...) |
foosel
added
needs testing
|
Hi - I'm still seeing the same I'm afraid - here is the capture in from the Recv: ok T:79.9 /80.0 B:62.7 /64.0 @:20 B@:128 On Fri, Nov 20, 2015 at 11:03 AM, Gina Häußge notifications@github.com
|
|
Meant also to say however that the interface is more responsive & I could flick between the different tabs - although they weren't being updated. As soon as I reloaded though, got page not available. The clock stops ticking even on the lh print panel. |
|
Ok, so more debugging output needed for the socket connection I guess. No chance to figure this out otherwise. @Salandora does this match what you were observing at all, or is it another kind of behaviour? I'm getting the sneaking suspicion that we have more than one type of problem here and are mixing up between them. |
|
That's another kind of problem. EDIT: well after a fast read through the history, I guess the problem normanclanger has is a complete different to this one here. As @foosel already stated it more looks like there is some overload done on the pi while printing, while the original problem here sound like a broken websocket connection after some time. |
|
Yes, I'll try and alternative route to my router via a wired connection
|
|
For me ping would still respond and ssh would work, and so far so good. After applying the maintenance release chrome hasn't yet given me the aw snap error after several long prints. I have seen where the browser will be in kind of a frozen state, but shortly after clicking in the white space on the page it "catches up" and starts responding again. I think long term this is a good change no matter what, because it doesn't make sense to consume processing when the tab isn't active anyway. Specifically video streams as that seems like it would consume a lot of memory. |
|
@jneilliii that's good new I guess, thanks. Also, I was going to leave this change in no matter what, even if it doesn't solve this specific problem here - it indeed makes too much sense. |
|
Hi - after testing the ethernet over power connection I was using, I was shocked by the low bandwidth I was getting. The printer is in the garage - perhaps the wiring is too much of a challenge. Switching to wlan ironically, significantly improved the situation. Ping & ssh back & the web experience similar to jneiliii - periods of inactivity, but it catches up eventually, or on a reload. Something changed after upgrading to 1.2.7 to cause my ethernet over power link to become unsuitable, as it was fine before that (and I haven't had the garage rewired). Now I can ssh to the pi, I don't see excessive bandwidth utilisation. Is these something which is latency sensitive in the scripts, which might cause it to freeze if a packet is dropped? Thanks Tim |
|
From personal experience I found power lan to be very finicky. Performance
could be outright killed by placing the wrong charger in the wrong outlet
somewhere else in the house. So I'm not surprised if it suddenly stopped
working reliably even without rewiring anything. Switching a lamp in the
wrong moment might already cause hiccups.
In general though, OctoPrint is a purely user space side program. There's
nothing I can imagine I could do that would allow me to nuke formerly
reliable network connections, especially not to the point of pinging the pi
would become impossible.
|
|
Ok thanks - that makes sense. Happy to write my particular experience down to poor network connectivity, |
|
Anyone still seeing this? |
|
I have not seen it recently. |
|
I still have been running smoothly since upgrading to 1.2.8. |
|
Yes same for me - all fine. Thanks Sent from my HTC ----- Reply message ----- I still have been running smoothly since upgrading to 1.2.8. Reply to this email directly or view it on GitHub: |
|
Ok, then I dare to close this now. Thanks everyone for the input! |
|
Says she'll close it now, forgets to hit the close button... \o/ |
Running a print through the web interface
Octoprint to stay responsive
The interface was responsive but any request was very sluggish. (I have the request spinner plugin installed so I could see the requests hanging for minutes) The terminal tab did not show the GCode as it was streaming (the print was running though so GCode was streaming.) Reloading the page happened at the usual speed, and after reload the terminal tab would show gcode streaming for a few seconds before freezing again. Manually sending gcode did not display on the UI, but a few minutes after sending the command would actually execute on the printer. Cancel button worked after some time. A restart of the raspberry pi brought OctoPrint into normal working order.
Version: 1.2.6 (master branch)
(if applicable - always include if unsure):
Marlin Integration branch
System running Browser (if applicable - always
include if unsure):
Chrome on OS X
(ALWAYS INCLUDE AND DO NOT TRUNCATE):
gist.github.com or pastebin.com (if applicable - always
include if unsure or reporting communication issues AND
DO NOT TRUNCATE):
Not logged - Lost after reboot
on gist.github.com or pastebin.com or alternatively a
screenshot (if applicable - always include if unsure
or reporting UI issues):
Not recorded
include if unsure or reporting UI issues):
I have read the FAQ.
The text was updated successfully, but these errors were encountered: