Unresponsive IB Gateway in Digital Ocean

[AlexCatarino]

This GitHub Issue is created in Lean-CLI and IBAutomater because it has not been observed in QuantConnect Cloud or locally, only within Digital Ocean environment.

1. The algorithm crashes with a "no connected" message. the crash is raised, in general, at 5.00 AM UTC.
2. The algorithm is unable to connect with another paper-trading account. IB Gateway gets frozen for 15 mins and crashes because it is not able to connect.

Based on the syslog, it looks like the IBGateway process is still running but not responding, because, after the countdown popups, no other events are received (especially the usual Shutdown/OnExited message, which triggers the restart).

We can see the countdown starting after 2021-09-27T23:41:01.1807775Z:

2021-09-27T23:41:01.1924755Z TRACE:: InteractiveBrokersBrokerage.OnIbAutomaterOutputDataReceived(): Content: Set Auto Restart Time (HH:MM) The application will automatically restart in 4 minutes 30 seconds EXIT SESSION SETTING

After connected to the docker instance about 23.47 UTC, we confirmed that the java process which contains the IBGateway was running. In fact, the memory consumption of that process on top command was about 35% of the 1GB.

IB Gateway's launcher.log, we can verify that IBGateway did actually restart:

2021-09-29 23:45:22.662 [WR] INFO  [JTS-Main] - ------------------------------- IB GATEWAY RESTART --------------------------------
2021-09-29 23:45:22.702 [WR] INFO  [JTS-Main] - installer version: 2.82, installer type: standalone, install dir: '/root/ibgateway/', Jts dir: '/root/Jts', VM options file: '/root/ibgateway/ibgateway.vmoptions'

Ref.: QuantConnect/IBAutomater#62

[Martin-Molinero]

Much has changed since this bug was reported (gateway & automater), suggest retesting with the newest versions, feel free to reopen if the issues is still reproducing.