PUBDEV-6852 - Kubernetes support #4370
Conversation
Pscheidl
force-pushed
the
pavel/pubdev-6852
branch
2 times, most recently
from
17556df
to
d07eb3d
Compare
Pscheidl
force-pushed
the
pavel/pubdev-6852
branch
from
d07eb3d
to
05ade3c
Compare
…l for H2O cloud forming are logged.
Pscheidl
force-pushed
the
pavel/pubdev-6852
branch
from
d4602fe
to
e6623b2
Compare
|
Memory limits should be handled on JVM level and are the container's responsibility ... |
h2o-k8s/src/main/java/water/k8s/KubernetesEmbeddedConfigProvider.java
Outdated
Show resolved
Hide resolved
h2o-k8s/README.md
Outdated
|
|
||
| Exposing the H2O cluster is a responsibility of the Kubernetes administrator. By default, an | ||
| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) can be created. Different platforms offer | ||
| different capabilities, e.g. OpenShift offers [Routess](https://docs.openshift.com/container-platform/4.3/networking/routes/route-configuration.html). |
There was a problem hiding this comment.
I couldn't find in the documentation a bit where actually h2o is started, am I missing something?
There was a problem hiding this comment.
Thanks, I'll add more explanation. H2O is started as soon as all pods cluster. Which means the headless service is created and the H2O pods are started. The ingress only makes H2O accessible from the outside. It may even change without affecting the underlying layers.
You may spawn H2O in one go by doing kubectl apply -f h2o.yaml. The YAML looks like this and it will work on your local minikube as well:
apiVersion: v1
kind: Pod
metadata:
name: h2o-k8s
labels:
app: h2o-k8s
namespace: default
spec:
containers:
- name: h2o-k8s
image: pscheidl/h2o-k8s
ports:
- containerPort: 54321
env:
- name: H2O_KUBERNETES_SERVICE_DNS
value: h2o-service.default.svc.cluster.local
---
apiVersion: v1
kind: Service
metadata:
name: h2o-service
spec:
type: ClusterIP
clusterIP: None
selector:
app: h2o-k8s
ports:
- protocol: TCP
port: 54321
There was a problem hiding this comment.
@Pscheidl Routess looks like a typo
|
Nice work!, mentioned few nits and proposals |
|
One more question: are all nodes accessible on the default port from the outside word? In Sparkling Water we need all nodes to be exposed (just the API port). When running GET request Question 2: Could you please try with HTTPS? I guess it should work out of box, but would be good to also mention this in the documentation |
|
Next step is to do health checks in order to allow automatic cluster restarts when required (those are turned on and off by K8S cluster administrator or the orchestration software.). Created a separate JIRA, as this functionality is definitely not required, it's a convenience feature. https://0xdata.atlassian.net/browse/PUBDEV-7359 |
|
Also, the REST API clustering could also be a convenient option for some. It has no huge advantages and it requires role bindings set up. I've created a JIRA for that as well, to keep track of the task. We can always postpone/decline the JIRA. |
Pscheidl
requested review from
michalkurka,
pkozelka,
honzasterba,
abal5,
bilcus and
pastx
|
Note to self: Detect K8S presence in any other way - sometimes we do NOT want the lookup to be triggered, even if running on K8S. |
…RVICE_DNS' env var presence.
|
The commit 9517354 makes it easy to disable this behavior. As the clustering will only work with a headless service set-up and the environment variable This means freedom for the users. For example, it'd not serve us well in our test pipeline once it starts running inside K8S. |
|
@michalkurka @mn-mikke This is it. Here, on my side, this is the final version, as I can't think of any improvements related to DNS-based clustering. Did a small change today and tested it manually. Please review. |
|
Thank you @pkozelka . |
Should only go to master This reverts commit e435d42.
Brings back support for Kubernetes from Pavel
https://0xdata.atlassian.net/browse/PUBDEV-6852
It is recommended to read the README.md introduced in this PR.
Works even locally on minikube:
Example output of a Pod