Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Octoprint disconnects from Wanhao D6 printer with "too many consecutive timeouts" #1506

Closed
gzcwnk opened this issue Sep 23, 2016 · 22 comments

Comments

Projects
None yet
5 participants
@gzcwnk
Copy link

commented Sep 23, 2016

Please read the "guidelines for contributing" that are linked ^-- just
up there. Also read the FAQ: https://github.com/foosel/OctoPrint/wiki/FAQ.

This is a bug and feature tracker, please only use it to report bugs
or request features within OctoPrint (not OctoPi, not any OctoPrint
plugins and not unofficial OctoPrint versions). Mark requests with
a [Request] prefix in the title please. Fully fill out the bug reporting
template for bug reports.

Do not seek support here ("I need help with ..."), that belongs on
the mailing list or the G+ community (both linked in the "guidelines
for contributing" linked above, read it!), NOT here.

Thank you!


What were you doing?

Trying to PID tune but prints always fail as well.

What did you expect to happen?

No error messages/disconnects

What happened instead?

I get an unhandled communication error every time.

Branch & Commit or Version of OctoPrint

Version: 1.2.15 (master branch)

Printer model & used firmware incl. version

Wanhao D6, 3.01 firmware (latest)

Browser and Version of Browser, Operating System running Browser

Firefox 48.0.2 on win7 64bit business.

Link to octoprint.log

[On gist.github.com or pastebin.com. Always include and never truncate.]

Well it would be nice if you said where to go looking for the log..........no joy on google....

Link to contents of terminal tab or serial.log

========8><-----
Communication timeout while idle, trying to trigger response from printer. Configure long running commands or increase communication timeout if that happens regularly on specific commands or long moves.
Send: M105
No response from printer after 3 consecutive communication timeouts, considering it dead. Configure long running commands or increase communication timeout if that happens regularly on specific commands or long moves.
Changing monitoring state from 'Operational' to 'Offline: Too many consecutive timeouts, printer still connected and alive?'

Connection closed, closing down monitor

Link to contents of Javascript console in the browser

[On gist.github.com or pastebin.com or alternatively a screenshot. If applicable -
always include if unsure or reporting UI issues.]

Screenshot(s) showing the problem:

[If applicable. Always include if unsure or reporting UI issues.]

I have read the FAQ.

@gzcwnk

This comment has been minimized.

Copy link
Author

commented Sep 23, 2016

root@warlocke:/var/log# find / -name octoprint.log

root@warlocke:/var/log#

seems there is no log on my file system

@foosel

This comment has been minimized.

Copy link
Owner

commented Sep 23, 2016

At the very top of the ticket form there was a big yellow link telling you to read something. At the very top of the text field to fill in it said

Please read the "guidelines for contributing" that are linked ^-- just up there

Those guidelines also contain where to find the requested log ;)

On top of octoprint.log missing you've sadly also only provided a truncated version of the contents of the terminal tab with the crucial bits are missing. Please provide the full contents.

Finally, just from your description it sounds like you might want to add the pid tune gcode to Settings > Serial > Advanced Options > long running commands, and of that doesn't help potentially also increase the value for max consecutive timeouts during long running commands there.

@foosel

This comment has been minimized.

Copy link
Owner

commented Oct 5, 2016

@gzcwnk Still waiting for some information from you, see above.

@anelson425

This comment has been minimized.

Copy link

commented Dec 11, 2016

I am seeing almost the exact same issue as he has described, here is a gist to the issue I am seeing: https://gist.github.com/anelson425/5e86a9b3e572a637a215eabe52319a87 and the corresponding terminal logs: https://gist.github.com/anelson425/d589c47f2998be23886c343c994be1c0

Mine was not related to a pid tune, but different gcode altogether. I tried to start up a print and saw a failure immediately.

Let me know if you need more and I can help out.

@foosel

This comment has been minimized.

Copy link
Owner

commented Dec 11, 2016

Your terminal log is missing the important bits thanks to your client being too slow to keep up. Could you enable the serial.log under Settings > Serial, reconnect, reproduce and then provide that plus the full octoprint.log file as well? In general, these timeout issues can be caused by communication issues or as said above specific commands that take longer to be responded to that you need to tell OctoPrint about. If it reliably fails pretty much at the start of your print, chances are high it's some specific command in your start gcode being the culprit here that makes the printer appear non responsive for too long. But to determine that we'll need a serial.log

@anelson425

This comment has been minimized.

Copy link

commented Dec 11, 2016

Sadly, I wasn't able to reproduce it now. I will try to get the code to reproduce it again tomorrow but as of now, no luck.

@anelson425

This comment has been minimized.

Copy link

commented Dec 11, 2016

My print that just finished disconnected my printer again, here are the last few thousand lines from the serial logs. I think what you need is in there but let me know if you need more.

https://gist.github.com/anelson425/70f13d3a298a125c505460921d62534e

@foosel

This comment has been minimized.

Copy link
Owner

commented Dec 12, 2016

Indeed:

2016-12-11 03:43:09,281 - Send: N404501 M106 S0*83
2016-12-11 03:43:09,409 - Changing monitoring state from 'Printing' to 'Operational'
2016-12-11 03:43:10,820 - Send: N404502 M105*16
2016-12-11 03:43:14,183 - Recv: ok 404501
2016-12-11 03:43:15,847 - Send: N404503 M105*17
2016-12-11 03:43:20,864 - Send: N404504 M105*22
2016-12-11 03:43:44,229 - Communication timeout while idle, trying to trigger response from printer. Configure long running commands or increase communication timeout if that happens regularly on specific commands or long moves.
2016-12-11 03:43:44,242 - Send: N404505 M105*23
2016-12-11 03:44:14,277 - Communication timeout while idle, trying to trigger response from printer. Configure long running commands or increase communication timeout if that happens regularly on specific commands or long moves.
2016-12-11 03:44:14,294 - Send: N404506 M105*20
2016-12-11 03:44:44,318 - No response from printer after 3 consecutive communication timeouts, considering it dead. Configure long running commands or increase communication timeout if that happens regularly on specific commands or long moves.
2016-12-11 03:44:44,335 - Changing monitoring state from 'Operational' to 'Offline: Too many consecutive timeouts, printer still connected and alive?'
2016-12-11 03:44:44,380 - Connection closed, closing down monitor

OctoPrint sends a "Fans off" command to the printer (probably from the streamed GCODE). The printer responds ok. OctoPrint sends a "get temperatures" to printer, which it does regularly to keep the graph updated. Printer doesn't respond. OctoPrint tries again. No response. OctoPrint tries again. No response. OctoPrint tries again. No response. OctoPrint tries a final time. No response. Printer has gone AWOL completely and stopped responding altogether over the serial line. There's sadly nothing I or OctoPrint can do in such a situation besides raising an error and disconnecting. The printer is completely dead in the water (from OctoPrint's perspective) because it simply doesn't reply to anything sent over serial anymore, not even with an error message.

Scrambled messages can usually be fixed with resending. Non responsive printers can't.

I've seen this kind of behaviour with overheated controllers and also of course with crashed firmware. Power issues with the Pi (when running OctoPrint from a Pi) also appear to play a role based on reports from users. In any case, it's nothing that can be solved from OctoPrint's end of the communication.

@nophead

This comment has been minimized.

Copy link
Contributor

commented Dec 12, 2016

I suspect 2016-12-11 03:43:03,677 - Send: N404500 G4 P120000*107 is the problem. It will pause for two minutes. OctoPrint should know that and not send for two minutes. The confusing thing is that it and the next command is acknowledged. Perhaps OctoPrint is prepared to wait for the G4 but when it sees the ok it thinks it has executed.

@foosel

This comment has been minimized.

Copy link
Owner

commented Dec 12, 2016

Ah, good point, I overlooked that since I was looking for a command that went unacknowledged. OctoPrint will set its timeout to the dwell value, but of course that dwell should not be acknowledged until it is done executing. G0 to G3 are buffered, but G4 isn't (right?), so it doesn't really make sense that it gets acknowledged pretty much immediately

@nophead

This comment has been minimized.

Copy link
Contributor

commented Dec 12, 2016

No it doesn't make sense but I think this is Repetier firmware, so could differ from Marlin in this respect.

@anelson425

This comment has been minimized.

Copy link

commented Dec 15, 2016

Yes, I am using Repetier and yes I have code at the end of my prints that allow the fan to help cool the extruder for 2 minutes. I can remove the code for that but it is interesting that the connection is affected by it.

@foosel

This comment has been minimized.

Copy link
Owner

commented Dec 15, 2016

but it is interesting that the connection is affected by it.

Actually, if the 2min dwell there is the reason but the firmware does that dwelling asynchronously and simply stops to receive or reply to commands after acknowledging that dwell and even the follow up command, it's perfectly reasonable that the host (OctoPrint) runs into a timeout here. How is it supposed to know when that dwell will even be executed and hence the printer won't be able to respond?

Maybe @repetier can shed some light on why it looks like the G4 is being handled asynchronously here (or verify that it isn't the case and we are barking up the wrong tree). I so far understood it to be a blocking command that waits the specified amount and then sends an acknowledgment (ok) back. So nothing that enters the movement ring buffer and should get acknowledged immediately after being put there but executed later (like G0 through G3 and G28 through G32 do).

@Erutan409

This comment has been minimized.

Copy link

commented Feb 13, 2017

I want to try to add some supporting information for what I believe to be a similar issue to what I was having:

I, too, was having issues with my printer disconnecting a lot with my Robo 3D R1+ and I chalked it up to some personal modifications I had done to my Arduino's USB PCB extender:

20170121_195147

Regardless for why I did it, I was sure that I was going to have some issues with USB printing now that I've exposed those connections to electronic interference. And sure enough, right after I made this modification and used the same USB cable I had been (came with the printer), I started to have disconnect issues most of the time during prints, but even sometimes before I'd start one.

So, I figured that I had basically made my setup faulty. Then, it occurred to me to try USB printing from my laptop and print right from Simplify3D - it worked, flawlessly, on a print that kept failing on me relying on OctoPrint.

I figure it's an issue of power and signal being generated from the laptop versus what the raspberry pi is outputting with the addition of powering a webcam. I shortened another USB cable I had laying around in anticipation of mounting the RP underneath my printer and based on two prints I've completed in the last 24 hours, my observations would have me believe that I've corrected my issue. Not to mention that the USB cable I modified still has the two chokes that were already installed on it (one on each end).

So, I would recommend (if you haven't done so, already) try using a shorter or even different USB cable if you're having random disconnect issues. Because, I wasn't having any issues printing with OctoPrint until I made the aforementioned modifications.

I wasn't sure where the appropriate ticket would be to write this up, but based on the audit trail of suggestions of where to report this reoccurring issue and the age of the ticket, I figured this would be the best place, for now.

I hope this is beneficial in any way for anyone else having random disconnect issues. I'll update if I start having disconnect issues, again.

@anelson425

This comment has been minimized.

Copy link

commented Feb 13, 2017

Very interesting, I will definitely try a different cable. I had just been using the cable that i got with my i3.

@Erutan409

This comment has been minimized.

Copy link

commented Feb 13, 2017

@anelson425 I hope it solves your issue. I'd encourage you to find/use a cable that also has two chokes instead of one; for added benefit.

@foosel

This comment has been minimized.

Copy link
Owner

commented Feb 13, 2017

While an as short as possible properly shielded usb cable with ferrite beads is in fact strongly recommended in general, it won't help with the very specific issue described in this ticket, which is caused by a combination of the end gcode apparently recommended for (by?) Wanhao printers and how Repetier firmware (which these printers ship with) handle the G4 dwell command or rather command output in general. The 2min "stop and do nothing while we wait for the hotend to cool" that sends its acknowledgement before starting is what is causing issues here since OctoPrint expects acknowledgements for a G4 to only arrive after it was executed. Repetier firmware behaves differently here than others, and in a way which makes this difficult to work around without breaking things for the rest. I just came back from a health related forced timeout, but before that I had started on a work around for this specific scenario, and I hope I can get that ready for 1.3.2

That being said: As pointed out in various other tickets about sudden disconnects over the years, what @Erutan409 recommends here with regards to proper cabling mirrors my own personal recommendation 100%. That and making sure that - when running from a Pi - said Pi actually has a solid power supply attached that indeed delivers 5V 2A (2.5A for the Pi3) continuously because otherwise the Pi will run into various exciting brown out issues. Phone chargers or cheap stuff from eBay are prime candidates for not being suited.

And since that keeps coming up I put adding an entry on that to the FAQ to my to-do list. I hope it's ok to link to your great comment for that @Erutan409, because for some reason people tend to not believe me when I say that but might accept it from a fellow user ;)

Edit typos/autocorrect errors

@Erutan409

This comment has been minimized.

Copy link

commented Feb 13, 2017

Absolutely.

@anelson425

This comment has been minimized.

Copy link

commented Feb 13, 2017

That and making sure that - when running from a Pi - said Pi actually has a solid power supply attached that indeed delivers 5V 2A (2.5A for the Pi3) continuously because otherwise the Pi will run into various exciting brown out issues.

While I haven't tested the actual output of my power supply, I am hopeful that power supply I purchased from Microcenter is actually producing 2.5a for my pi3. Perhaps, I will check that out when I get home this evening.

@Erutan409

This comment has been minimized.

Copy link

commented Feb 13, 2017

@anelson425 Looks correct and comparable to what I picked up along with my raspberry pi that I purchased from Fry's Electronics.

Per what @foosel mentioned: some people try to go the cheap route and use a typical phone charger that supplies the same amount of volts. It's the amps that need to be available depending on what you're running off of the RP.

@foosel

This comment has been minimized.

Copy link
Owner

commented Feb 24, 2017

The problem with the G4 (which is the reason for the originally reported problem in this ticket) is now "solved" in a way on maintenance and will be part of the 1.3.2 release.

If OctoPrint detects Repetier firmware (or the corresponding flags are properly configured) it will now actively suspend any communication with the printer after it sends a dwell command to it for the duration of said dwell command. This sadly was the only way to assure an actual pausing on OctoPrint's part considering that Repetier firmware will immediately send an ok on receiving the G4 before it stops responding altogether. This is in contrast to other firmwares which send the ok only after the G4 has fully processed (which OctoPrint could already handle with regards to resetting timeouts).

The solution is not ideal, but the only way to handle the issue at hand here considering that the version of Repetier firmware that is shipped with the affected printers does not yet send a busy message during a blocking dwell (if it did you wouldn't run into problems).

As a side note, I think fully blocking the printer's communication for two full minutes at the end of every print just to prevent heat creep in the hotend from a fan that's shut off too early is a very cheap approach to solving a problem that should either be solved in hardware (by a better cooler solution, e.g. a dedicated cooling fan controlled by the printer itself) or firmware (by delaying fan off commands until the hotend temperature is under an acceptable threshold).

@foosel foosel added this to the 1.3.2 milestone Feb 24, 2017

@foosel

This comment has been minimized.

Copy link
Owner

commented Mar 16, 2017

1.3.2 was just released.

@foosel foosel closed this Mar 16, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.