Redundant start of workspace which had been removed in time of starting on OCP

[dmytro-ndp]

Description

Some selenium tests could eventually send successful requests to stop and to remove test workspace when they it starting. It causes unreasonable start of removed workspace in a couple of minutes. For example, for workspace with id='workspace9na18kbhbyiwjnvp' we had fail of start at the 2018-01-29 06:23:47:

2018-01-29 06:19:33,514[io-8080-exec-10]  [INFO ] [o.e.c.a.w.s.WorkspaceManager 461]    - Workspace 'user1517188450/workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' created by user 'user1517188450'
2018-01-29 06:19:33,945[nio-8080-exec-7]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 228]   - Starting workspace 'user1517188450/workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' by user 'user1517188450'
...
2018-01-29 06:20:00,884[aceSharedPool-7]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 268]   - Workspace 'user1517188450:workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' started by user 'user1517188450'
2018-01-29 06:20:18,027[nio-8080-exec-2]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 338]   - Workspace 'user1517188450/workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' is being stopped by user 'user1517188450'
2018-01-29 06:20:30,379[aceSharedPool-1]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 375]   - Workspace 'user1517188450/workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' stopped by user 'user1517188450'
2018-01-29 06:20:31,085[nio-8080-exec-6]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 228]   - Starting workspace 'user1517188450/workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' by user 'user1517188450'
2018-01-29 06:20:48,794[nio-8080-exec-1]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 338]   - Workspace 'user1517188450/workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' is being stopped by user 'user1517188450'
2018-01-29 06:21:00,347[aceSharedPool-2]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 375]   - Workspace 'user1517188450/workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' stopped by user 'user1517188450'
2018-01-29 06:21:03,856[nio-8080-exec-8]  [INFO ] [o.e.c.a.w.s.WorkspaceManager 272]    - Workspace 'workspace9na18kbhbyiwjnvp' removed by user 'user1517188450'
...
2018-01-29 06:23:47,974[aceSharedPool-5]  [WARN ] [w.i.o.OpenShiftInternalRuntime 150]  - Failed to start OpenShift runtime of workspace workspace9na18kbhbyiwjnvp. Cause: Server 'wsagent/http' in machine 'dev-machine' not available.
2018-01-29 06:23:48,756[aceSharedPool-5]  [INFO ] [o.e.c.a.w.s.WorkspaceRuntimes 284]   - Workspace 'user1517188450:workspaceomp7pn' with id 'workspace9na18kbhbyiwjnvp' start failed
2018-01-29 06:23:48,772[aceSharedPool-5]  [WARN ] [o.e.c.a.w.s.WorkspaceManager 425]    - Cannot set error status of the workspace workspace9na18kbhbyiwjnvp. Error is: Workspace with id 'workspace9na18kbhbyiwjnvp' doesn't exist

Reproduction Steps

There is selenium test CheckStopStartWsTest which is starting and stopping the same workspace twice and it could eventually send second stop request and sequential remove workspace request in time when workspace is secondly starting.

OS and version:
Multiuser Eclipse Che 6.0.0-M5-SNAPSHOT which is run on OCP infrastructure.

Diagnostics:

[sleshchenko]

I think it should be fixed by the following issue #9354

[sleshchenko]

I think it should not be possible to reproduce it since K8s/OS support start interruption. @dmytro-ndp @SkorikSergey Could you confirm that? Does this error message still appear in Che Servers logs?

If error doesn't appear anymore, then I think we can close this issue and cause of it should be fixed by #9354

[garagatyi]

Should be fixed already. Closing as fixed. Feel free to reopen