Deploy tinkerbell stack in a single step

[gauravgahlot]

Note: The setup has been tested on Ubuntu 18.04 and CentOS 7.

To setup tinkerbell stack in interactive mode using the following commands:

$ wget https://raw.githubusercontent.com/infracloudio/tink/deploy_stack/setup.sh && chmod +x setup.sh
$ ./setup.sh

You can also setup in declarative mode. To test the script I used:

• Server class: c3.small.x86
• Operating System: Ubuntu 18.04 and CentOS 7
• Region: Amsterdam, NL (AMS1)
• ENV variables:

 export TB_INTERFACE=enp1s0f1
 export TB_NETWORK=192.168.1.0/29
 export TB_IPADDR=192.168.1.1
 export TB_REGUSER=admin

Execute the setup with:

$ curl https://raw.githubusercontent.com/infracloudio/tink/deploy_stack/setup.sh | bash

Video samples:

• Setup on CentOS 7
• Setup on Ubuntu 18.04

[alexellis]

Is it worth retaining the manual instructions for local development? cc @nathangoulding

[alexellis]

It would be ideal if you could first try to read an ENV-var for this value and then only use read if nothing was given.

This is so that the script can be run on CI non-interactively.

[alexellis]

You may want to generate this instead, see how we do this in openfaas -> https://github.com/openfaas/faas-netes/tree/master/chart/openfaas#generate-basic-auth-credentials

The read statements ideally should read from env-vars or a similar mechanism that doesn't require user interaction.

piping or cat-ing a file is probably not going to be a robust option because it depends on the ordering of the file and is hard to document. It is also sensitive to timing bugs.

[alexellis]

Example:

if [ $TB_PASSWORD == "" ];
  read -sp 'Registry password [default admin]? ' TB_PASSWORD
fi
echo The password is $TB_PASSWORD

Then use it via:

# Interactive
curl https:// | bash
# inline value
curl https:// | TB_PASSWORD="test1234"  bash
# or
# Set on separate lines
export TB_PASSWORD="test1234"
curl https:// |  bash

[alexellis]

Good to see a one click install, however using read is problematic for CI and automation, I've made some other suggestions.

[DavidZisky]

Quick test run:

On fresh CentOS 7.6 it fails on:

ERROR: failed to enable NetworkManager

there is no "NetworkManager" in the system by default (and BTW there is no wget either - better use curl). I installed it (so you either have to check first if it's installed, install it yourself or use network package) and then it failed on:

failed to add IP address to interface

Turns out interface wasn't saved to envrc file as here:

Following network interfaces found on the system:
eth0
Which one would you like to use for with Tinkerbell?

I just hit enter assuming it will use eth0. It didn't, so I run again specifying it myself and then I ended up with:

INFO: tinkerbell stack setup completed successfully on centos server :)

I checked CentOS 8 as well and it, of course, failed on Docker installation.

[parauliya]

Quick test run:

On fresh CentOS 7.6 it fails on:

ERROR: failed to enable NetworkManager

there is no "NetworkManager" in the system by default (and BTW there is no wget either - better use curl). I installed it (so you either have to check first if it's installed, install it yourself or use network package) and then it failed on:

failed to add IP address to interface

Turns out interface wasn't saved to envrc file as here:

Following network interfaces found on the system:
eth0
Which one would you like to use for with Tinkerbell?

I just hit enter assuming it will use eth0. It didn't, so I run again specifying it myself and then I ended up with:

INFO: tinkerbell stack setup completed successfully on centos server :)

I checked CentOS 8 as well and it, of course, failed on Docker installation.

@DavidZisky , I would like to inform you that NetworkManger is required ( atleast for now ) to setup a private network on CentOS. So either we need to add it in the pre-requisite for CentOS 7 or install it thourght the script itself. But in that case as well we need to inform user that these are the things which will be installed on your machines.
Regarding CentOS 8, we have not tested on CentOS 8 since we are providing the support for CentOS 7 and Ubuntu 18.04 only for now. So it's better to raise an issue for the support of CentOS 8.

[nathangoulding]

I'd prefer to detect whether NM is enabled, and use if if so but not require it to be in place since NM is not universally used or loved (including at Packet).

[DavidZisky]

@DavidZisky , I would like to inform you that NetworkManger is required ( atleast for now ) to setup a private network on CentOS. So either we need to add it in the pre-requisite for CentOS 7 or install it thourght the script itself. But in that case as well we need to inform user that these are the things which will be installed on your machines.
Regarding CentOS 8, we have not tested on CentOS 8 since we are providing the support for CentOS 7 and Ubuntu 18.04 only for now. So it's better to raise an issue for the support of CentOS 8.

as far as I remember NM is definitely not required to set up a private network on CentOS 7. You should be able to do it by generating proper ifcfg-[interface] file in /etc/sysconfig/network-scripts/ and then bringing that interface up with:

ifup [interface]

But maybe I'm missing something - I didn't read the whole script yet.

Regarding CentOS8 - no worries. There is a known "bug" which makes the normal installation of docker impossible (at least for now) so it's expected that it fails. There is a workaround for it but since you are not supporting it officially then not worth the hassle.

[parauliya]

I'd prefer to detect whether NM is enabled, and use if if so but not require it to be in place since NM is not universally used or loved (including at Packet).

@nathangoulding if the NM is not installed on a machine we need to find another way to configure the network which we are still trying to figure out. I tried few ways like changing the ifcfg- file as well but it didn't work. I also think that we should not have the dependancy on NM but since we need to get this setup things done asap this was the only way working so we went for it. This will be changed soon.

[nathangoulding]

Yep, for sure. Let's create a couple issues for NM-less install, as well as CentOS 8 support.

[DavidZisky]

Just did a quick test and it works fine without NM (I just modified bit of your script which checks for NM, added generating the file and restarting interface):

3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 36:59:a8:4f:31:0b brd ff:ff:ff:ff:ff:ff
    inet 10.133.101.176/16 brd 10.133.255.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet 192.168.1.1/29 brd 192.168.1.7 scope global eth1:1
       valid_lft forever preferred_lft forever
    inet 192.168.1.2/29 scope global secondary eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::3459:a8ff:fe4f:310b/64 scope link 
       valid_lft forever preferred_lft forever

would have to be battle-tested in a more realistic scenario and if that interface actually works but that was just to check my idea

[invidian]

I'm testing it on Ubuntu 18.04 via Vagrant right now:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/bionic64"

  # config.disksize.size = '30GB'

  config.vm.provider :virtualbox do |v|
    v.check_guest_additions = false
    v.functional_vboxsf     = false
    v.cpus                  = 2
    v.memory                = 2048
    v.customize ['modifyvm', :id, '--paravirtprovider', 'kvm']
  end

  config.vm.network "private_network", ip: "192.168.50.2", auto_config: false
end

Couple of suggestions, I wasn't sure where they should land in the script:

• After selecting the interface, it would be cool if ecosystem CIDR and IP address would pick the default values from selected interface

• It should be noted somewhere, that script should be run as root, otherwise I get:

  Warning: Failed to create the file /usr/local/bin/docker-compose: Permission
  Warning: denied
  

  I see that sudo is being used in some places, perhaps this should be settled on either running as root and checking it or using sudo everywhere. Also the description of the PR use $ suggesting it should be run as non-root user.

• Ubuntu 18.04 run fails with ./setup.sh: line 209: ifdown: command not found. Solution: apt install ifupdown.

• Answers to the read needs to be provided every time to the script, a bit annoying

• ifdown enp0s8 fails for me:

  # ifdown enp0s8
  Unknown interface enp0s8
  

  IIRC write_iface_config replaces interface config, but does not handle the case where the interface was never configured via /etc/network/interfaces. I worked around it by executing: echo "iface enp0s8" >> /etc/network/interfaces

• ‘deploy.tar.gz` file is being downloaded multiple times on subsequent runs

• maybe mention, that the script downloads 1.3GB of assets and also how much disk space is needed to set it up? Default Vagrant disk size is 10G, I almost run out of space, had to do sudo rm /tmp/osie.tar.gz after it has been unpacked.

Other than the mentioned issues, the script succeeded and it seems everything is running. I need to figure out how to get CLI tool and configure it now.

[Ottovsky]

Hey there,

On the first run of setup.sh, it looks like the most recent version of cacher:
quay.io/packet/cacher workflow fe5226c5c975 5 days ago 145MB
overwrites certificates created during gen_certs(), which breaks communication between components.

[mikemrm]

this should be tabbed in