RancherOS support

[djmaze]

TGT frontend doesn't work on RancherOS, but TCMU does (tried with these commands). Is there a way to tell the manager to use use TCMU mode?

[djmaze]

Okay, made it work by adjusting the hard-coded launch command. It would probably make sense to allow setting the frontend mode via the manager commandline?

[yasker]

Currently, TCMU is not as well tested as TGT, so we haven't recommended to use it.

We're working on it TGT support for RancherOS at the moment. We expect it to work in the near future.

[xinity]

any updates on this perhaps ? :)

[yasker]

Got to involve @SvenDowideit :)

[xinity]

Got to involve @SvenDowideit :)

he seems pretty busy as we can see in RancherOS github project :)
but yes i'd love to see him involved in this issue definitely :)

[SvenDowideit]

yup, I got so far as to add an open-iscsi service to the master services repo, and then things got busy :/

[xinity]

@SvenDowideit do you think it could be enough (open-iscsi service) to test longhorn with rancheros ?

[niusmallnan]

Now we should able to support RancherOS, longhorn/longhorn-manager@521920d

Anyone can refer to the following steps to experience:
Now can only support ubuntu-console

$ ros console switch ubuntu
$ ros engine switch docker-17.03.2-ce

#  install open-iscsi
$ apt install --no-install-recommends open-iscsi

# mkdir lock dir for iscsi
$ mkdir /run/lock
# update cloud-config in RancherOS
runcmd:
- [mkdir, /run/lock]

RKE

# download RKE
$ curl -o rke -L https://github.com/rancher/rke/releases/download/v0.1.7/rke_linux-amd64

# cluster.yml
nodes:
  - address: xxxxx
    user: rancher
    role: [controlplane,worker,etcd]

network:
  plugin: flannel

services:
  kube-api:
    extra_args:
      feature-gates: MountPropagation=true

  kubelet:
    extra_args:
      feature-gates: MountPropagation=true
      volume-plugin-dir: /opt/rke/var/lib/kubelet/volumeplugins

Update FLEXVOLUME_DIR before deploy longhorn

- name: FLEXVOLUME_DIR
  value: "/opt/rke/var/lib/kubelet/volumeplugins/"

[DeviantEng]

@niusmallnan Hey, can you provide any more guidance on this? Still not super clear to me.

I have 3 RancherOS nodes, all 3 running ubuntu console. open-iscsi installed, but still getting this error from longhorn-manager:

ERRO[0000] Failed environment check, please make sure you have iscsiadm/open-iscsi installed on the host
FATA[0000] Error starting manager: Environment check failed: Invalid mount namespace /host/proc/1/ns/mnt, error Failed to execute: nsenter [--mount=/host/proc/1/ns/mnt mount], output nsenter: failed to execute mount: No such file or directory

root@ros-01:~# ros console list
disabled alpine
disabled centos
disabled debian
disabled default
disabled fedora
current  ubuntu
root@ros-01:~# which iscsiadm
/usr/bin/iscsiadm
root@ros-01:~# dpkg-query -l 'open-iscsi'
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                    Version                  Architecture             Description
+++-=======================================-========================-========================-===================================================================================
ii  open-iscsi                              2.0.873+git0.3b4b4500-14 amd64                    iSCSI initiator tools

What exactly am I supposed to do with rke? (I'm not familiar with rke). And where am I suppoed to update that FLEXVOLUME_DIR value?

Thanks.

[yasker]

@DeviantEng We haven't had RancherOS support in a release yet. If you want to give it try, using deploy directory master branch of https://github.com/rancher/longhorn-manager . It's the development build so it's maybe broken and needs a clean new environment to be installed.

Starting the next release (v0.3) we will formally support RancherOS.

[DeviantEng]

@yasker Thanks for the replay. I'm still testing around with other things in Rancher 2, so I can wait for the next release (of Longhorn, I assume?). Is there an ETA on that currently?

With formal support fo ROS, will that only be with the ubuntu console, or will that be with all consoles (minor thing, but just curious)?

[yasker]

@DeviantEng We're targetting next month for the release.

Regarding the available consoles, we will support default console as well in the release.

[yasker]

Pending on default console support and (possible) user configuration-free experience.

[DeviantEng]

What's the latest on this? I would love to test Longhorn on my RancherOS Rancher cluster.

[MaxDiOrio]

Same here. Seems like a good alternative to NFS.

[yasker]

@DeviantEng @MaxDiOrio We cannot finish RancherOS support for v0.3.0 release in time. We're trying to get done in v0.3.1 release, but it all depends on our bandwidths. We definitely should able to support RancherOS in v0.4.0 release.

[krishofmans]

I was hoping to finally give this a try with the 0.4 release, but apparently this has moved. Will this be available by the time rancher 2.2 gets released?

[pagong]

@krishofmans:
Please have a look at the documentation of my setup of rook-nfs-0.9.2 on-top-of
longhorn-0.3.0 on-top-of RKE on-top-of rancheros-1.4.0 on-top-of KVM on-top-of openSUSE-15.0.
Maybe my write-up is helpful to you :-)

[yasker]

@krishofmans Sadly it won't be available by the time Rancher 2.2 released. RancherOS support is a priority for v0.5 release.

[yasker]

Fixed by longhorn/longhorn-manager#242