Create a list of OpenShift objects that we can/can't support in devfile or workspace recipe

[skabashnyuk]

Description

It would be nice to have an up to date list of OpenSnift objects that we can/can't support as a tool in devfile or workspace recipe.

[sleshchenko]

Small tip: list of currently supported objects can be found in the following components
https://github.com/eclipse/che/blob/master/infrastructures/kubernetes/src/main/java/org/eclipse/che/workspace/infrastructure/kubernetes/environment/KubernetesEnvironmentFactory.java#L107
https://github.com/eclipse/che/blob/master/infrastructures/openshift/src/main/java/org/eclipse/che/workspace/infrastructure/openshift/environment/OpenShiftEnvironmentFactory.java#L106

List of unsupported objects should be investigated

[sleshchenko]

Table of supported(partially supported) objects:

Object	API	Kubernetes Infra	OpenShift Infra	Notes
Pod	Kubernetes	✔️	✔️
Deployment	Kubernetes	✔️	✔️
ConfigMap	Kubernetes	✔️	✔️
PVC	Kubernetes	✔️	✔️
Secret	Kubernetes	✔️	✔️
Service	Kubernetes	✔️	✔️
Ingress	Kubernetes	✔️	❎	I've checked and minishift allows to create Ingress and it works fine when host is specified(OpenShift creates Route for it), but loadBalancer IP is not provisioned. As alternative way, to support Ingresses is generate Route based on provided Ingress. So, it is possible to add Ingresses support for OpenShift infra as well.
Route	OpenShift	❎	✔️	I guess we can make OpenShift recipe more compatible with Kubernetes Infra and generate Ingress instead of provided Route
Template	OpenShift	✔️	✔️	AFAIK Kubernetes API does not support templates, but I checked and workspace with Template in recipe started successfully, default parameters are resolved. I guess Fabric8 client does this magic

Table of unsupported objects which should be considered to support:

Object	API	Notes
Role RoleBinding ServiceAccount	Kubernetes	A user may need special permissions for his application and then he may need define this objects in his application definition; It makes sense to consider implementing it
DeploymentConfig	OpenShift	It's common object for deploying a Pod on OpenShift. We support only alternative from Kubernetes API, it's Deployment. There is an issue to implement support for it #5967
DaemonSets	Kubernetes	Is similar to Deployment in that they both create Pods, and those Pods have processes which are not expected to terminate (e.g. web servers, storage servers). Use a Deployment for stateless services, like frontends, where scaling up and down the number of replicas and rolling out updates are more important than controlling exactly which host the Pod runs on. Use a DaemonSet when it is important that a copy of a Pod always run on all or certain hosts, and when it needs to start before other Pods. More here So, I think it may be implemented but is not critical at all.
ReplicaSet	Kubernetes	It allows to control replicas of specified pods template. But it is recommended to use Deployment for the same purpose. More https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/
ReplicationController	Kubernetes	ReplicaSets are the successors to ReplicationControllers. The two serve the same purpose, and behave similarly, except that a ReplicationController does not support set-based selector requirements as described in the labels user guide. As such, ReplicaSets are preferred over ReplicationControllers
StatefulSet	Kubernetes	It's similar to Deployment. Difference description from this article. The StatefulSet acts as a controller in Kubernetes to deploy applications according to a specified rule set and is tailored towards the use of persistent and stateful applications. However, a Deployment can accomplish the same thing. So what’s stopping you other than the Kubernetes documentation explicitly calling out “If an application doesn’t require any stable identifiers or ordered deployment, deletion, or scaling, you should deploy your application with a controller that provides a set of stateless replicas. Controllers such as Deployment or Replica Set may be better suited to your stateless needs.
Job, CronJob	Kubernetes	Some of applications may define jobs for init purpose or others, cron job - like for cleaning up some data. It makes sense to add support for them but it is needed to be investigated on details on real use-case to answer some questions, like should we create Job on first workspace start, or each?
BuildConfig ImageStream	OpenShift	They may be defined in application definition but now it is not clear how it can be used in Che Workspaces.

Table of unsupported objects which should not be considered to support:

Object	API	Notes
ClusterRole ClusterRoleBinding	Kubernetes	A user may need to specify it for his application, but it is not right thing to do. It is not really to implement support for such objects in workspace
Endpoints	Kubernetes	usually it is not used in application definition, K8s API automatically provides them for created Services;
PersistentVolume	Kubernetes	it is cluster-admin obligation to create PersistentVolumes, should not be in Application definition;
ResourceQuota	Kubernetes	it is cluster-admin obligation to create ResourceQuota per system or for a particular user/namespace, should not be in Application definition;
Scale	Kubernetes	AFAIU it's K8s managed objects that is created after chaning value of Deployment#spec#replicas
SecurityContextConstraints	Kubernetes	It's more cluster-admin's duties but not an application;
Namespace	Kubernetes	an application definition should not contain project definition inside of its definition; A user should precreate project and just specify where application should be deployed. In Che Case, Che Server creates namespace/project for workspace;
Project ProjectRequest	OpenShift	The same as for Namespace

Objects listed here are collected from different places. But some objects still may be missing
If someone knows about objects which may be used for an application definition and which are not listed here - let me know.
Corrections and comments about investigation result are welcome
cc @skabashnyuk @l0rd @slemeur @amisevsk @eivantsov

[amisevsk]

Regarding templates -- the template never actually reaches the API afaik. The kubernetes client resolves parameters and converts it to a list, along the lines of oc process -f template.yaml | oc apply -f -

[amisevsk]

Other comments

• DeploymentConfig: One thing to note, apart from the implementation difficulties is from the docs:

  Because deployment configurations existed in OpenShift Container Platform prior to deployments being added in Kubernetes 1.2, the latter object type naturally diverges slightly from the former. The long-term goal in OpenShift Container Platform is to reach full feature parity in Kubernetes deployments and switch to using them as a single object type that provides fine-grained management over applications.

• Daemonset: Agree that it's not critical; I view daemonsets are more of a cluster administration tool
• ReplicaSet: Supporting would potentially get complicated (a third way to specify a pod, basically) and using replica sets directly is not advised by API docs.
• ReplicationController: Similar to the ReplicaSets; these are generally not supposed to be used directly and instead managed via DeploymentConfigs. We would also likely run into similar issues as with DeploymentConfigs in implementing

On the ingress vs route issue, I'm kind of split between automatically converting or pushing users to make an appropriate configuration. It depends on how much detail is needed in the conversion; I'd rather a workspace break because we don't support the conversion rather than it break in a weird way because of some assumption we made regarding how conversion is to be done. Another factor here is in use cases: how often do we expect a user to be developing and OpenShift specific app on a Kubernetes cluster?

[l0rd]

@sleshchenko this is really good work. If not yet I think we should put it in che-docs. And I have added the label target/che7GA to #5967.

[sleshchenko]

#12899 (comment)
cc @rkratky I think you're person who knows the best place for such article, if it is Che6 Doc or Che 7. Could you please create an issue for documenting it with a place specified.

[rkratky]

@sleshchenko, apologies, I totally missed this:

cc @rkratky I think you're person who knows the best place for such article

Is the above content ready?

[skabashnyuk]

#12899 (comment) @rkratky yes it is.