Serial Freezing after a long time - then unfreezing

[hcl337]

We have been using this package successfully - except that after the create 2 has been in the charger for a long time, it seems to become either partially or fully unresponsive. By taking it off the charger by hand, or by sending it random combinations of of FULL/PASSIVE commands and /UNDOCK commands, it seems to sometimes become fully responsive after a minute or two but not always.

Expected behavior: We should just send it an /undock command which puts it into full mode and it should start responding to movement commands.

Actual behavior: When it is in this mode, we are sending in an /undock command and it is not showing a toggled /mode or /battery/charging_state in the ROS topic outputs and does not drive around or go beep, etc.

Background:

We have seen this across 3 Create 2's, multiple USB-> Serial cables and multiple computers so it doesn't seem to be a state of the hardware but instead something core to the system.

I have been running this for 4 days (over the weekend) and could see our code internally toggling passive / full every few minutes but then tried to undock it and back it up and it would not come out of the passive mode when I sent a FULL command.

• Using the USB --> Serial cable that comes with the create 2
• ROS Kinetic
• Toggling into FULL mode and back to PASSIVE every 4 minutes to try to make sure it doesn't go asleep (A hack based on forum posts)
• We have seen the Create start in sleep mode (Booted computer 20 minutes before, then started it - and it was frozen from the beginning. We have also had it start in regular mode where it seems to get stuck, even when we are triggering a change in states to full every 4 minutes (Well within the 5 minute limit)

Below I can see that there 9M packets which have been sent. When it was not responding, this number was staying constant for a while then jerking up then flat.

Excerpt from /diagnostics

        {
            "message": "Serial connection is good",
            "hardware_id": "CREATE_2",
            "values": [
                {
                    "key": "Corrupt packets",
                    "value": "0"
                },
                {
                    "key": "Total packets",
                    "value": "9731905"
                }
            ],
            "name": "ca_driver: Serial Status",
            "level": 0
        }

After toggling things for a while, I started to see it incrementing again by 60 packets / s and it was fully responsive (No ROS reset, unplugging, moving off the charger, etc).

Questions:

• Is Create Autonomy sending in the wake up command and other elements to toggle it back awake? Is there something I am missing?
• Have other people encountered it freezing after a while.
• Is there a command we should be sending in separately to keep the robot from going to sleep "hard" when in passive mode rather than toggling to FULL then PASSIVE again?

Thanks!

[jacobperron]

Thanks for reporting this. I can't recall myself experiencing this issue, but I have not had the driver running continuously for days. To clarify, does this issue only present itself when the robot is charging (or just coming off of an extended charge)?

[hcl337]

Only when it is charging. We have actually isolated it to 2 things. 1) hardware flow control has to be on to trigger the keep alive on the create USB cable or it goes to sleep after 5 min. This is not enabled in libcreate. 2) we have found that when it toggles from full to passive and then charges, we are the packets dying right at that point which says it could be a power thing on our setup. When we toggle it a few times more, it comes back with packets so it is the serial connection resetting.

But a key change would be to enable hardware flow control.

Verified on multiple roombas.

[hcl337]

Small Code Change Recommendation

To clarify here, there is one thing that should change on create_autonomy/libcreate. flow control needs to be changed from none to hardware for the BTC wire to trigger a wake up if the create has gone to sleep.

libcreate: src/serial.cpp

@@ -22,7 +22,7 @@ namespace create {
     using namespace boost::asio;
     port.open(portName);
     port.set_option(serial_port::baud_rate(baud));
-    port.set_option(serial_port::flow_control(serial_port::flow_control::none));
+    port.set_option(serial_port::flow_control(serial_port::flow_control::hardware));
 
     usleep(1000000);

Separate possible hardware comm weakness in Create 2

Separately, it seems there could be a hardware communications weakness on the create 2 which I have documented here and also contacted irobot (create@irobot.com) for help on as we have narrowed in the information to an exact point in time where communication fails independent of which create 2 (we tried 3), computer (tried 2 NUC's, 1 Mac laptop) and communications protocol (Tried Create Autonomy and also echoing packets in on command line). Link