Fail to create a new workspace when one already exist on OSIO

[l0rd]

When che-starter is requested to create a new workspace but one already running it should:

1. stop the old workspace
2. create the new workspace.
   This currently fails. Probably because che-starter fails stopping the old workspace due to Update "removeContainer" PR so that it would be mergeable to che master branch #175. When Update "removeContainer" PR so that it would be mergeable to che master branch #175 will be fixed this one should fixed this one to (need to verify).

This is rh-che version of issue fabric8-ui/fabric8-ui#1214 (comment) and should be fixed when

[ibuziuk]

@l0rd @sunix are you able to create workspace on osio against the latest template (1.0.245) ?
I keep getting:

Workspace agent is no longer responding. To fix the problem, verify you have a good network connection and restart the workspace.

reproducible form "codebases' view & dashboard

Could this be related to #157 ?
Currently I can not verify this issue because of this problem

[ibuziuk]

not sure why this happens but when I update tenant via profile on osio workspace creation works fine and fails when I update che manually via cli

[ibuziuk]

on minishift with quotas set I got the following error when trying to deploy che-server:
Error syncing pod, skipping: timeout expired waiting for volumes to attach/mount for pod "eclipse-che"/"che-1-nfn1g". list of unattached/unmounted volumes=[che-data-volume]

[ibuziuk]

Image that was used for quota limit problem verification ibuziuk/che-server:quota-fix

[ibuziuk]

@l0rd @amisevsk I was able to verify PRs [1] [2] on minishift and can confirm that those fix quota limit issue. Steps for verification that I have done:

1. build openshift-connector-wip with PRs applied
2. start minishift
3. build rh-che via ./minishift_build_fabric8_and_deploy.sh -Dfast clean
4. when che-server is running on minishift apply quota limits to the eclipse-che project. Details can be found in Need to provide instructions for setting quota limits on minishift #222
5. run che-starter locally and create workspace against che-server on minishift
6. SUCCESS: workspace has been created an running
7. create new workspace via che-strarter
8. SUCCESS: first workspace has been stopped, new workspace is running - no quota limit error meesage
9. create workspace via dashboard
10. SUCCESS: expected error while creating workspace:

Error creating: pods "che-ws-um71apcqqxjqjwpf-2133038293-" is forbidden: exceeded quota: compute-resources, requested: limits.memory=1300Mi, used: limits.memory=2000Mi, limited: limits.memory=3Gi

So, I believe PRs [1], [2] can be now safely applied (However, this does not guarantee that quota issue will be fixed on osio, because there might be some related problems on core side).

[1] eclipse-che/che#5766
[2] eclipse-che/che#5773

[l0rd]

I've been able to successfully test that eclipse-che/che#5766 eclipse-che/che#5773 fix this issue on OSIO. Well done @ibuziuk! 🎉 🎉

Something noteworthy is that the second workspace startup takes a while because the old PV is not released immediatly. I saw this log 7 times in 7 minutes when performing the startup of the new worksapce (and finally the workspace started successfully):

Failed to attach volume "pvc-afe722e4-370e-11e7-b6db-0233cba325d9" on node "ip-172-31-76-202.us-east-2.compute.internal" with: Error attaching EBS volume "vol-0ad5fd7d83f234295" to instance "i-062bed3befe567051": VolumeInUse: vol-0ad5fd7d83f234295 is already attached to an instance status code: 400, request id:

[ibuziuk]

@l0rd verified on prod - seems to work just fine. I believe this one can be closed