Properly handle situations when a ws agent is killed by OOM

[ghost]

When a workspace agent stop responding for whatever reason, projects disappear from project tree, and page refresh results in Cannot get project types error.

Reproduction Steps:

1. Start a workspace with 1GB
2. Import a fairly large Java project
3. Start Maven build

Expected behavior:

Build is a success and the IDE remains functional

Observed behavior:

When OOM happens in a workspace container, kernel may kill WS agent or the WS agent becomes unreachable for some reason. The IDE keeps trying to reach ws agent and fails to do so. As a result, projects disappear from project tree. Page refresh results in Cannot load project type error which is caused by the client trying to reach API deployed with a workspace agent.

Proposed solution

A user should be at least notified that the workspace is malfunctioning probably because of OOM. It may also be a good idea to try to restart a workspace agent, however warn a user that this situation is likely to happen again with this kind of workspace and more RAM may be required.

• Problem started happening recently, didn't happen in an older version of Che: [Yes]
• Problem can be reliably reproduced, doesn't happen randomly: [Yes]

[riuvshin]

it is same issue as #1817

[bmicklea]

#1817

[JamesDrummond]

@vparfonov Please assign this to yourself. Thanks.

[vparfonov]

We can't detect properly OOM but we can in some check that ws-agent process still alive and notify if it killed by OS

[vparfonov]

We found other solution without additional process on dev machine side.
If client (IDE) lost websocket connection to the ws-agent it will ask some other service in our infrastructure to check ws-agent state if it not available for this service to IDE will propose stop workspace.

[bmicklea]

This sounds reasonable - thanks Vitalii

[vparfonov]

Workaround #2369