Kubernetes Deployable Image

[naisanza]

I read the documentation for integrating heketi onto a kubernetes host server, but I don't want to do all of that manual work

I'm looking for a heketi image that can be deployed by kubernetes with all the necessary configuration options be configured via environment variables.

I figured it's easier to create a gluster server in LXC, a heketi server as a kubernetes deployment, and then target the heketi server for dynamic volume creation in kubernetes

[MohamedAshiqrh]

Hi @naisanza,

I think you might be interested in gluster-kubernetes, We have made this deployment as better as possible. Below is the link for all the docs and required materials. Let us know If you want something else.

https://github.com/gluster/gluster-kubernetes

[naisanza]

@MohamedAshiqrh I did find that before ending up here. Though, I cannot have dedicated block devices assigned to glusterfs, as my glusterfs service is running within an LXC container on top of a filesystem that has all block devices optimized for random I/O operations

The mountpoints inside the LXC container looks like this:

containers/subvol-109-disk-1 on / type zfs (rw,noatime,xattr,posixacl)

Can the topography map handle LXC mountpoints?

[lpabon]

Heketi must provide glusterfs with logical volumes allocated from a PV which requires raw storage. If you want to run glusterfs on top of a file system, then you cannot use Heketi, but you can run it by yourself.

Please let me know if that answers your question.

[naisanza]

@lpabon This documentation states that heketi needs Disks registered with Heketi must be in raw format

You're able to create raw format disk images with qemu-img create -qf raw tinker.img 3.2G and then mount the image with losetup

Will that work with heketi?

Update: Also, another way to create a blank disk image http://askubuntu.com/questions/667291/create-blank-disk-image-for-file-storage

[naisanza]

@MohamedAshiqrh I've gotten pretty familiar with the gluster and heketi configuration, and the configuration on the heketi side is a bit much (https://github.com/heketi/heketi/wiki/Running-the-server), especially with the JWT authorization part.

I'm looking to simplify that

I'm looking to be able to just use the Docker image found on (https://hub.docker.com/r/heketi/heketi/) and just pass Environmental Variables to it, to tell it the following:

TOPOLOGY.NODE.HOSTNAMES.MANAGE = node0
TOPOLOGY.NODE.HOSTNAMES.STORAGE = 10.100.0.272
TOPOLOGY.NODE.DEVICES = /dev/sda, /dev/loop0, /dev/loop1
HEKETI_EXECUTOR = ssh
HEKETI_SSHEXEC.KEYFILE = /root/.ssh/id_rsa
HEKETI_SSHEXEC.USER = root

The Docker image will handle creating new tokens for JWT authorization and defaults for all other requirements (such as, location of its database).

[lpabon]

@naisanza I think that may work. I have never tried it because afaik there is a limit to the number of losetups you can do.

[MohamedAshiqrh]

@naisanza nice, will give it a try too. Thanks.

[phlogistonjohn]

From my reading, I think all topics raised in this issue have been resolved. In addition, the last activity on this issue occurred nearly a year ago. Please reopen if needed. Closing.