installation: add `minikube` installation #15

diegodelemos · 2017-03-22T08:47:07Z

Adds local installation documentation. (addresses docs: amend installation #4)
Adds minikube specific installation option to REANA configuration
script. (addresses tests: prepare local development test cluster #13)

Signed-off-by: Diego Rodriguez diego.rodriguez@cern.ch

jirikuncar · 2017-03-22T09:35:10Z

docs/installation.rst

+
+.. code-block:: console
+
+   $ MINIKUBE_IP=$(minikube ip)


Did you mean $ export MINIKUBE_ID=$(minikube ip)?

yes... Thanks!

hjhsalo

I hope summary is OK even if I didn't use line comments.

I tested on Ubuntu 16.04 x64.
I didn't test the Kubernetes cluster creation on Openstack.

Fixes needed in docs/installation.rst:

There are hardcoded strings of "Installing minikube for OSX ..." and "Installing latest kubectl for OSX ...". There should be Linux alternatives for them.
There are two instances of typo CONTOLLER_PORT which should be CONTROLLER_PORT since instructions guide you to open "http://$MINIKUBE_IP:$WORKFLOW_CONTROLLER_PORT" which won't work.
open "http://$WORKFLOW_CONTROLLER_IP:$WORKFLOW_CONTROLLER_PORT" gave a 404 even with corrected port (port is not written at all because of the CONTOLLER_PORT typo). Despite this, the cluster might be running fine since both of the other test links worked.

Fixes needed in scripts/reana

Could't install because my Linux-system didn't pass the if ! "darwin linux" | grep -F -q -w -i "$(uname -s)" test and the script prompted that "Your OS is not supported." In order to proceed with the installation I had to comment out the test.

General notes:

Maybe it should be added that for now you should install with pip install -e .[all] before running reana install-minikube. It might not be clear for all who check out the repo on their own.
Also same thing that is said in Kubernetes cluster part about Python virtual environments, should probably be said in Minikube part as well: $ mkvirtualenv reana-resources-k8s
I had a non-blocking (cluster seemed to work fine) problem with ZeroMQ where $ kubectl get pods reported that zeromq-pod is not running: zeromq-msg-proxy-1617754619-qjvc7 0/1 CrashLoopBackOff 5 4m
Maybe it should be added that what you should expect when opening the links. Either text or small screenshots. E.g. for link "http://$JOB_CONTROLLER_IP:$JOB_CONTROLLER_PORT/jobs" theinstructions would say something like "Displays JSON with a key 'jobs' which is empty since no jobs are running on the minikube" and for link "http://$WORKFLOW_MONITOR_IP:$WORKFLOW_MONITOR_PORT/helloworld" that "Displays yadage-monitor".

diegodelemos · 2017-03-24T10:07:16Z

There are hardcoded strings of "Installing minikube for OSX ..." and "Installing latest kubectl for OSX ...". There should be Linux alternatives for them.

Done

There are two instances of typo CONTOLLER_PORT which should be CONTROLLER_PORT since instructions guide you to open "http://$MINIKUBE_IP:$WORKFLOW_CONTROLLER_PORT" which won't work.

👍

open "http://$WORKFLOW_CONTROLLER_IP:$WORKFLOW_CONTROLLER_PORT" gave a 404 even with corrected port (port is not written at all because of the CONTOLLER_PORT typo). Despite this, the cluster might be running fine since both of the other test links worked.

This is foreseen because we just have a yadage endpoint which only accepts POST requests. I have opened a ticket and assigned you reanahub/reana-workflow-controller#20 it is quick change, just add an empty workflows endpoint!

Could't install because my Linux-system didn't pass the if ! "darwin linux" | grep -F -q -w -i "$(uname -s)" test and the script prompted that "Your OS is not supported." In order to proceed with the installation I had to comment out the test.

I might be wrong, but I think that this is not currently happening, there was an echo missing in the tests statement but not anymore.

Maybe it should be added that for now you should install with pip install -e .[all] before running reana install-minikube. It might not be clear for all who check out the repo on their own.

I am not sure about this lets chat IRL

Also same thing that is said in Kubernetes cluster part about Python virtual environments, should probably be said in Minikube part as well: $ mkvirtualenv reana-resources-k8s

Not sure about this neither - IRL

I had a non-blocking (cluster seemed to work fine) problem with ZeroMQ where $ kubectl get pods reported that zeromq-pod is not running: zeromq-msg-proxy-1617754619-qjvc7 0/1 CrashLoopBackOff 5 4m

Work in progress. I am fixing reanahub/reana-workflow-monitor#13

Maybe it should be added that what you should expect when opening the links. Either text or small screenshots. E.g. for link "http://$JOB_CONTROLLER_IP:$JOB_CONTROLLER_PORT/jobs" theinstructions would say something like "Displays JSON with a key 'jobs' which is empty since no jobs are running on the minikube" and for link "http://$WORKFLOW_MONITOR_IP:$WORKFLOW_MONITOR_PORT/helloworld" that "Displays yadage-monitor".

Totally agree, thanks for the idea 👍

p.s. next time could you please comment in line? it is a bit messy to go from the comment to the code back and forth! thanks 🤙

hjhsalo · 2017-03-24T10:33:34Z

Sorry, these comments were unnecessary. I didn't notice that they are already in the instructions. 🤦‍♂️

Maybe it should be added that for now you should install with pip install -e .[all] before running reana install-minikube. It might not be clear for all who check out the repo on their own.

Also same thing that is said in Kubernetes cluster part about Python virtual environments, should probably be said in Minikube part as well: $ mkvirtualenv reana-resources-k8s

hjhsalo · 2017-03-24T12:23:48Z

Note that I just added a PR21 to reana-workflow-controller that addresses comments about /workflows -endpoint.

tiborsimko · 2017-03-28T09:07:12Z

docs/installation.rst

+   {
+     "workflows": {}
+   }
+   $ open "http://$MINIKUBE_IP:$WORKFLOW_MONITOR_PORT/helloworld"


open ... is macOS specific. Something like xdg-open ... works on GNU/Linux. If it works on macOS too, then please switch. Otherwise let us simply use firefox ... as people will know what that means...

tiborsimko · 2017-03-28T09:10:52Z

docs/installation.rst

+
+.. code-block:: console
+
+   $ export WORKFLOW_CONTROLLER_PORT=$(kubectl describe service workflow-controller | grep 'NodePort:' | cut -f 4 | sed -e "s@/TCP@@")


For these and other environment variables, it would be good to use the common prefix REANA_... so that we don't create a mess in people's environments...

tiborsimko · 2017-03-28T09:11:14Z

docs/installation.rst

+.. code-block:: console
+
+   $ reana-resources-k8s build-manifests
+   $ kubectl create -Rf configuration-manifests


-rf should be enough, let's keep it lowercase.

For some reason kubectl uses only the capital r for the recursive option

tiborsimko · 2017-03-28T09:11:24Z

docs/installation.rst

+
+.. code-block:: console
+
+   $ rm -Rf configuration-manifests


tiborsimko · 2017-03-28T09:11:41Z

scripts/reana

+
+    # Deploy system
+    echo 'Deploying REANA on minikube ...'
+    kubectl create -Rf configuration-manifests


tiborsimko · 2017-03-28T09:23:40Z

docs/installation.rst

+
+   $ reana install-minikube
+
+which will do the following:


It would be good to decouple the steps that install kubernetes/minikube itself, which many people would prefer to do outside of REANA using their packaging systems, and install the rest of the REANA component ecosystem.

The getting started overview for people would then look like:

$ pip install reana $ reana install-backend minikube # or skip this step if you have it already $ reana configure-backend minikube # or other clouds later... just an example $ reana configure-container docker # or singularity... just an example $ reana pull-components $ reana build-manifests $ reana initialise-backend $ reana run-hello-world ...

(hypothetical example, let's muse IRL)

hjhsalo · 2017-04-03T18:19:31Z

scripts/reana

+    case $1 in
+        install-minikube ) install_local_minikube
+                           ;;
+        clean-minikube)    minikube delete


I couldn't get minikube cluster to start after the first run without running minikube delete first.
I'm not sure how many times I started it before using reana install-minikube or did I mess around with Virtualbox, but I would bet that some of the minikube VM's were either in corrupted state or not present.
Maybe something about running reana clean-minikube should be added to the documentation? Maybe just a side note:

"NOTE: In case you run into error messages from Kubernetes related to local Kubernetes cluster (aka minikube), you can try to run reana clean-minikube to delete VMs created by previous minikube installation / initialization"

BTW, the error I got was:

Starting local Kubernetes cluster... Starting VM... E0403 20:06:00.759374 16086 start.go:113] Error starting host: Error getting state for host: machine does not exist.

Since our discussion last week, we decided to leave backend specific management up to the user. Just a minimal command for installing and running Kubernetes is provided. However, if it doesn't work (he have to investigate your problem to see if it is easy to fix) I would suggest to remove it from the CLI and stick to REANA specific operations.

* Adds local installation documentation and amends OpenStack installation. (closes reanahub#4) * Adds `minikube` specific installation option to REANA configuration script. (addresses reanahub#13) Signed-off-by: Diego Rodriguez <diego.rodriguez@cern.ch>

diegodelemos · 2017-04-04T08:04:27Z

As I specify in the code, it is not ready for non local installation (it is somehow, but hacks such as changing reana-resources-k8s configuration file by hand is needed 🙅 ) but since we are addressing #13, local development and quick tests by early adopters, we can leave it for later.

Specially in this case, differing from local to production could be easily specified by running pip install reana[minikube] or pip install reana[kubernetes] so a Python CLI (click) would be easier to implement (as discussed IRL with @tiborsimko ). On the other hand, I am not sure if now is the best moment to address it, probably using sh CLI for now and focusing in other work packages would be more useful. OFC we have to discuss IRL on Friday

tiborsimko · 2017-04-07T10:10:55Z

Thanks, let's merge now and let's improve CLI language and docs later.

diegodelemos added the Need: work label Mar 22, 2017

diegodelemos requested review from tiborsimko and hjhsalo March 22, 2017 08:47

diegodelemos added the Status: in review label Mar 22, 2017

jirikuncar reviewed Mar 22, 2017

View reviewed changes

diegodelemos force-pushed the 4-add-local-installation branch 5 times, most recently from 2427805 to ecb4b3b Compare March 23, 2017 23:14

hjhsalo requested changes Mar 24, 2017

View reviewed changes

diegodelemos force-pushed the 4-add-local-installation branch 2 times, most recently from 95a35c7 to ee74e44 Compare March 24, 2017 10:39

hjhsalo approved these changes Mar 24, 2017

View reviewed changes

diegodelemos force-pushed the 4-add-local-installation branch from ee74e44 to acd7f69 Compare March 24, 2017 15:01

tiborsimko requested changes Mar 28, 2017

View reviewed changes

diegodelemos self-assigned this Apr 3, 2017

diegodelemos force-pushed the 4-add-local-installation branch from acd7f69 to 0e26e88 Compare April 3, 2017 17:01

hjhsalo reviewed Apr 3, 2017

View reviewed changes

diegodelemos force-pushed the 4-add-local-installation branch from 0e26e88 to 3f0d713 Compare April 4, 2017 07:15

tiborsimko approved these changes Apr 7, 2017

View reviewed changes

tiborsimko merged commit 3f0d713 into reanahub:master Apr 7, 2017

tiborsimko removed the Status: in review label Apr 7, 2017

diegodelemos deleted the 4-add-local-installation branch April 10, 2020 13:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

installation: add `minikube` installation #15

installation: add `minikube` installation #15

diegodelemos commented Mar 22, 2017

jirikuncar Mar 22, 2017

diegodelemos Mar 22, 2017

hjhsalo left a comment

diegodelemos commented Mar 24, 2017

hjhsalo commented Mar 24, 2017

hjhsalo commented Mar 24, 2017

tiborsimko Mar 28, 2017

tiborsimko Mar 28, 2017

tiborsimko Mar 28, 2017

diegodelemos Apr 3, 2017

tiborsimko Mar 28, 2017

tiborsimko Mar 28, 2017

tiborsimko Mar 28, 2017

hjhsalo Apr 3, 2017

diegodelemos Apr 4, 2017

diegodelemos commented Apr 4, 2017

tiborsimko commented Apr 7, 2017


		.. code-block:: console

		$ export WORKFLOW_CONTROLLER_PORT=$(kubectl describe service workflow-controller \| grep 'NodePort:' \| cut -f 4 \| sed -e "s@/TCP@@")

installation: add minikube installation #15

installation: add minikube installation #15

Conversation

diegodelemos commented Mar 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hjhsalo left a comment

Choose a reason for hiding this comment

diegodelemos commented Mar 24, 2017

hjhsalo commented Mar 24, 2017

hjhsalo commented Mar 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diegodelemos commented Apr 4, 2017

tiborsimko commented Apr 7, 2017

installation: add `minikube` installation #15

installation: add `minikube` installation #15