Bare-metal Linux VServer preparation script for BOA container #980

macmladen · 2016-06-12T20:46:34Z

Can you share some preparatory script that prepares bare-metal and puts up VServer to host BOA?

Also I'd be grateful if you can provide some practical tutorial (link) that could help me in the process as I was wondering how can I have BOA and Redmine, Gitlab, Mail all of which may run in their own containers (VServers).

I was looking for some solution to have Redmine and/or mail in separate containers so I wanted LXC as it seemed lightweight but then I was wondering if Docker is just toolset for LXC or does it offer more than just that.
#972

It's a good idea to open source this tiny and simple tool we use to set up VServer containers.

macmladen · 2016-06-26T11:57:02Z

As we started discussion in #972 I was wondering on virtualization techniques.

Debian was supporting linux-vserver but now it is deprecated,

Linux-VServer support is no longer present in Debian, starting from wheezy.

There are, let's say, three main virtualization techniques (or approaches based on needs):

Virtual machines (VMs) (kvm, qemu, vmware, virtualbox, virtuozzo, hyper-v)
Paravirtualization (xen, uml)
Virtualization on the OS level, a.k.a. containers virtualization

Linux VServer belongs to the third group which is probably the best for most use cases as that group presents the least overhead (in range from 1% to 3%). There are several operating system level virtualization implementations:

Virtuozzo / OpenVZ (2000/2005)
Linux VServer (2001)
LXC linux containers (2008)
Docker (2013)

OpenVZ required custom kernel and today is directly supported on CentOS. Linux VServer was supported on Debian before Wheezy. Docker is supposed to be used only for one process so it is not suited well for complete BOA stack.

Based on similarity between Linux VServer and LXC (which seem like more modern, current approach to virtualization), is there some specific reason not to go with LXC?

omega8cc · 2016-06-27T09:45:31Z

LXC is more complex to use, and got into kernel not because it is better or more modern than Linux VServer. It happened because of politics, better personal connections, etc. All that stuff which is always a warning sign for us. That is why we prefer Linux VServer, which is actively maintained despite being replaced upstream with LXC.

omega8cc · 2016-06-27T09:51:20Z

Also, LXC is supported by Canonical Ltd. and we don't like that fact, either. Plus, it is still largely a work in progress: https://linuxcontainers.org

EdNett · 2016-06-28T07:31:01Z

Hello, I am to understand then that BOA officially recommends linux-verser, which has been deprecated by debian in jessie? IS it being actively maintained on github - is that the place you suggest we take it from?

omega8cc · 2016-06-28T10:53:00Z

@EdNett -- We don't care what Debian mainstream kernel offers. If you are interested in VServer kernel details, please check:

http://linux-vserver.org/Frequently_Asked_Questions
http://linux-vserver.org/Communicate
http://www.psand.net/repo/

That said, don't ask us for help in installing it on your own. You should definitely not try this if you are not fluent in custom kernel management, broken grub config repair, broken machines recovery etc. Otherwise you can screw up things royally, and it is better to wait until installing VServer will be supported by BOA tools.

omega8cc · 2017-04-10T06:08:37Z

New installer has been added: https://github.com/omega8cc/boa/blob/master/docs/CLUSTER.txt#L77

It is a work in progress! For testing only, for now.

ar-jan · 2017-04-11T14:00:10Z

I'm unlikely to use this personally (anytime soon, anyway), but it's great to see this added. Thanks!

angel0d · 2017-04-15T11:54:11Z

@omega8cc tested the installed last night and everything seems to have worked fine, but ...

worked fine:

   $ cluster in-host server.example.com
   $ reboot
   $ cluster up-host
   $ cluster in-all head

it seems that there is no mySQL installed in the 3 vservers c1rdb0 c1rdb1 c1rdb2

c1rweb works fine with its local DB

but adding an additional octopus instance did not work $ cluster in-oct em@il o2 mini head

with error:

BOA [13:15:02] ==> INIT A: Shared platforms code v.001 will be used
BOA [13:15:02] ==> INIT A: Adding user...
mysqladmin: flush failed; error: 'Max connect timeout reached while reaching hostgroup 11 after 10000ms'
mysqladmin: reload failed; error: 'Max connect timeout reached while reaching hostgroup 11 after 10000ms'
SQL14 -h127.0.0.1 -P6033 --protocol=tcp -uroot
ERROR 9001 (HY000) at line 1: Max connect timeout reached while reaching hostgroup 11 after 10000ms
mysqladmin: reload failed; error: 'Max connect timeout reached while reaching hostgroup 11 after 10000ms'
BOA [13:15:59] ==> INIT B: Downloading drush micro-8-07-02-2017...
BOA [13:16:02] ==> INIT B: Installing Aegir Provision backend...
BOA [13:16:02] ==> INIT B: Downloading Drush and Provision extensions...
BOA [13:16:11] ==> INIT B: Running hostmaster-install, please wait...
/opt/tmp/boa/lib/functions/satellite.sh.inc: line 4765: cd: /data/disk/sol/aegir/distro/001: No such file or directory
BOA [13:16:25] ==> INIT B: FATAL ERROR: Required file does not exist:
BOA [13:16:25] ==> INIT B: FATAL ERROR: /data/disk/sol/aegir/distro/001/sites/sol.cluster.solebich.de/settings.php
BOA [13:16:25] ==> INIT B: FATAL ERROR: Aborting AegirSetupB installer NOW!
BOA [13:16:25] ==> INIT A: FATAL ERROR: AegirSetupB installer failed
BOA [13:16:25] ==> INIT A: FATAL ERROR: Aborting AegirSetupA installer NOW!
BOA [13:16:25] ==> FATAL ERROR: AegirSetupA installer failed
BOA [13:16:25] ==> FATAL ERROR: Aborting Octopus installer NOW!

boa info on c1rweb : https://gist.github.com/angel0d/ede19c4b4ac59943f6516aca595e4362

host is a dedicated on leaseweb with the following specs

Brand and Model	Hp - DL120 G6
Processor	1x Intel Quad-Core Xeon X3440 [ 4 Core(s) ]
RAM	16 GB
HDD	2x120 GB SSD 2.5 Inch
Hardware RAID	No

I understand this issue regards the in-vps part of the installation script. maybe #1136 or #502 would be more appropriate for feedback on the cluster part of the script ?

omega8cc · 2017-04-15T12:37:58Z

Yeah, there are two patches we forgot to apply, sorry about that! Should be fixed today.

angel0d · 2017-04-15T12:43:24Z

@omega8cc is it correct to assume that deleting the vservers would allow a new cluster in-all head to take place ?

or re-installing the host OS is a must ?

omega8cc · 2017-04-15T13:06:56Z

@angel0d You don't need to re-install anything, not even the guests. Once the patches are committed, it will be enough to re-install just the ProxySQL on the web node with:

cluster {in-pxy} {id} {ip}

which translates to:

cluster in-pxy c1rweb your-web-guest-ip

angel0d · 2017-04-15T14:36:48Z

@omega8cc just run the script but not sure how it could help

it seems to me that there is no mysql installed in the DB vservers

root@cluster:~# vserver c1rdb1 enter
root@c1rdb1:/# mysql
bash: mysql: command not found
root@c1rdb1:/# ps -A | grep mysql
root@c1rdb1:/#

I 'll re-try to re-install the vservers - is there a specfific log file that could help debug this further ?

omega8cc · 2017-04-15T14:48:48Z

@angel0d We have left most of the output to be printed on screen, it doesn't log anything in this initial version.

You should see errors if it couldn't install something.

We have tested it many times, and cluster in-all head just worked. No issues...

omega8cc · 2017-04-15T14:50:54Z

@angel0d Please check twice that the IPs are correctly listed and that they can be enabled by VServer (they should be assigned to the machine already, but don't need to be active before the installation).

Can be switched to _NOISE="-qq” to limit the noise further.

omega8cc · 2017-04-15T15:00:41Z

@angel0d Please use newer version of the script, we have reduced default noise cancellation level from -qq to -q

angel0d · 2017-04-15T15:06:44Z

We have left most of the output to be printed on screen, it doesn't log anything in this initial version.

I noticed that. will do the next install in screen so as to have all of it there for inspection

Please check twice that the IPs are correctly listed and that they can be enabled by VServer

I assumed that they should be enabled and working beforehand and I have added all public and private IPs as aliases on the primary interface

angel0d · 2017-04-15T15:07:42Z

btw I noticed that cluster up-all head seems to do nothing and returns in a second

omega8cc · 2017-04-15T15:21:50Z

@angel0d Please don't enable the IPs, they should be assigned, but not enabled.

omega8cc · 2017-04-15T15:24:19Z

@angel0d

btw I noticed that cluster up-all head seems to do nothing and returns in a second

That's correct:

upgrade_db_cluster() {
  check_config_cluster
  ### TODO
}

upgrade_web_node() {
  check_config_cluster
  ### TODO
}

angel0d · 2017-04-15T17:53:53Z

so I removed the IP aliases

after the base install on 1st vserver when it moves on with the galera setup it is not that verbose

I only got the following

Running post-install Galera VPS upgrade...
Removing systemd on jessie...
The c1rdb0 VPS installation is complete!
Installing Percona Cluster on VPS c1rdb0...
Warning: Permanently added '192.30.1.193' (ECDSA) to the list of known hosts.
bash: line 12: /etc/mysql/debian.cnf: No such file or directory
Warning: Permanently added '192.30.1.193' (ECDSA) to the list of known hosts.
bash: line 49: /etc/mysql/conf.d/galera.cnf: No such file or directory
Warning: Permanently added '192.30.1.193' (ECDSA) to the list of known hosts.
bash: line 113: /etc/mysql/my.cnf: No such file or directory
Warning: Permanently added '192.30.1.193' (ECDSA) to the list of known hosts.
mysql: unrecognized service

angel0d · 2017-04-15T18:02:13Z

although it installed the base system fine it seems that the c1rdb0 does not have outgoing access to the net

omega8cc · 2017-04-15T18:34:04Z

That could be your system/network issue, I guess?

angel0d · 2017-04-15T20:17:07Z

That could be your system/network issue, I guess?

i am trying to debug this but with no luck. What are the assumptions made for the private network ?

Is it supposed to have routed public access already ? Or should the host be set up so as to act as a NAT gateway ?

in my case private network is configured for a specific subnet. I see in cluster.sh

  vserver ${idn} build -n ${idn} -m debootstrap --i-know-its-there \
    --hostname ${hst} \
    --interface eth0:${vip}/32 \
    -- -d ${osx} -- --arch=amd64 --include=${incl} \
    --exclude=${excl}; &> /dev/null

maybe the prefix should be configurable as different providers/datacenters might have specific network settings ?

i tried to change the above line so as to be in accordance to the private network configuration (which is 10.30.1.192/27), use the clean script and re-run in-all but config in /usr/etc/vservers/c1rdb0/interfaces/0/ seems to be the same

omega8cc · 2017-04-15T20:28:20Z

The /32 netmask is not network specific, because it means "just this single IP address", and it works everywhere, no matter the server provider. At least, it works everywhere we have used it, at various data centers in the last 12 years. Example.

Thats said, the script was not tested with local IPs configured on interface other than eth0 and this perhaps should be configurable, at least for db nodes, because web node must use public IP address, no matter what.

Further testing and patches are welcome!

omega8cc · 2017-04-15T20:30:16Z

The /32 netmask should just work also with private IPs: http://jodies.de/ipcalc?host=10.30.1.192&mask1=32&mask2=

angel0d · 2017-04-19T09:52:27Z

just an update: with public IPs galera install proceeds and finishes fine. I am missing one IP though in this spare machine for c1rweb so as to fully test this

tried the standalone in-vps creation. it seems to install boa tools and populate a .barracuda.cnf. when running boa in-stable though in the newly created vps I noticed that it installs all PHP version. is this by design ?

other than that installation went flawlessly

great tools !!!

omega8cc · 2017-04-19T22:26:52Z

@angel0d Thanks for testing this and for the feedback!

We should test the installer with private IPs, obviously.

Yes, it is by design to install all PHP versions, so you don't need to edit .barracuda.cnf and run another upgrade, nor to use more complicated options on command line.

We should remove 5.5 from this list, though, but installing both 5.6 and 7.0 is probably a good idea, but suggestions on improvements of default configuration are always welcome!

EdNett · 2017-08-01T18:00:30Z

Hello, Is this supposed to be working currently in head? I tried just the first part:

Example for installing Linux VServer based BOA on a dedicated machine
without any cluster configuration, just a BOA VPS:

with one IPv4 - is that all that is required, one IPv4 ? (I imagine IPv6's don't work, right?)

It didn't install anything at all, just downloaded a few vserver scripts, but didn't install anything, much less barracua or octopus. I don't want to issue a full error report - as nothing happened. I just want to know are people successfully using this?

Thank you.

ed

omega8cc · 2017-08-18T07:31:09Z

Yes, cluster support works in head, although there are still missing (important) parts, including running upgrades in the correct order, plus handling machine reboot situation, so the guests don't get stuck, etc. The single VPS mode should work fine, but needs more testing, so people don't get into trouble due to some edge case (machine/vendor/etc) situations.

omega8cc · 2017-08-18T07:32:33Z

That said, bug reports related to features introduced in head, so not officially supported yet, are more than welcome!

angel0d · 2018-02-12T18:44:38Z

regarding the in-vps installer it needs to improve the logic around the use of VMFAMILY internal variable when it equals to 'VS'

while searching for VMFAMILY there were 20+ files/scripts returned - maybe the many checks can remain but find another way to initialize VMFAMILY in a way that future install with in-vps are not considered as 'VS'

angel0d · 2018-02-12T18:51:10Z

as of January 29th the installer would fail during the in-all install step when used like

$ cluster in-host server.example.com
$ reboot
$ cluster up-host
$ cluster in-all stable

with a lot of messages like ERROR 1045 (28000): ProxySQL Error: Access denied for user 'admin'@'' (using password: YES)

note in-all install step would fail with stable or head as install option

angel0d · 2018-03-13T16:12:33Z

Not sure if it should be a separate issue. I think csf is not properly install/working or I might be confusing something.

iptables and csf are install only on host system right ?

iptables --list returns an empty list of rules though

omega8cc · 2018-03-14T21:31:56Z

@angel0d Please open a separate issue for these problems, since there will be more of this kind to fix, I suspect.

omega8cc added type - feature request status - todo labels Jun 13, 2016

omega8cc modified the milestones: 3.3.0, 3.2.0 Jun 13, 2016

bserem mentioned this issue Jun 23, 2016

Wheezy to Jessie upgrade left bare-metal server unable to boot #972

Open

EdNett mentioned this issue Jun 25, 2016

MariaDb 10.x unexpectantly stopped - ? wheezy to jessie upgrade problem? #1004

Closed

omega8cc modified the milestones: 3.3.0, 3.2.0 Jul 27, 2016

omega8cc mentioned this issue Sep 22, 2016

Sites down after upgrades #1072

Closed

omega8cc added a commit that referenced this issue Apr 10, 2017

Percona XtraDB Cluster, ProxySQL, Linux VServer — #502 #980 #1009

dad4c87

omega8cc closed this as completed Apr 10, 2017

omega8cc added status - fixed and removed status - todo labels Apr 10, 2017

omega8cc reopened this Apr 15, 2017

omega8cc added status - needs work and removed status - fixed labels Apr 15, 2017

omega8cc added a commit that referenced this issue Apr 15, 2017

Use moderate noise cancellation mode in cluster #980

2307339

Can be switched to _NOISE="-qq” to limit the noise further.

omega8cc added a commit that referenced this issue Apr 15, 2017

Remove total silencing to help with debugging #980

1a46a81

omega8cc modified the milestones: 4.0.0, 4.x.0 Jan 3, 2019

omega8cc modified the milestones: 4.x.0, 4.2.0 Mar 15, 2021

omega8cc closed this as completed Mar 15, 2021

Bare-metal Linux VServer preparation script for BOA container #980

Bare-metal Linux VServer preparation script for BOA container #980

macmladen commented Jun 12, 2016

macmladen commented Jun 26, 2016

omega8cc commented Jun 27, 2016

omega8cc commented Jun 27, 2016

EdNett commented Jun 28, 2016

omega8cc commented Jun 28, 2016 •

edited

omega8cc commented Apr 10, 2017

ar-jan commented Apr 11, 2017

angel0d commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 15, 2017 •

edited

omega8cc commented Apr 15, 2017

omega8cc commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 15, 2017 •

edited

angel0d commented Apr 15, 2017

omega8cc commented Apr 15, 2017

omega8cc commented Apr 15, 2017 •

edited

angel0d commented Apr 15, 2017

angel0d commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 15, 2017 •

edited

omega8cc commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 19, 2017

omega8cc commented Apr 19, 2017

EdNett commented Aug 1, 2017

omega8cc commented Aug 18, 2017

omega8cc commented Aug 18, 2017

angel0d commented Feb 12, 2018

angel0d commented Feb 12, 2018

angel0d commented Mar 13, 2018

omega8cc commented Mar 14, 2018

Bare-metal Linux VServer preparation script for BOA container #980

Bare-metal Linux VServer preparation script for BOA container #980

Comments

macmladen commented Jun 12, 2016

macmladen commented Jun 26, 2016

omega8cc commented Jun 27, 2016

omega8cc commented Jun 27, 2016

EdNett commented Jun 28, 2016

omega8cc commented Jun 28, 2016 • edited

omega8cc commented Apr 10, 2017

ar-jan commented Apr 11, 2017

angel0d commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 15, 2017 • edited

omega8cc commented Apr 15, 2017

omega8cc commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 15, 2017 • edited

angel0d commented Apr 15, 2017

omega8cc commented Apr 15, 2017

omega8cc commented Apr 15, 2017 • edited

angel0d commented Apr 15, 2017

angel0d commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 15, 2017 • edited

omega8cc commented Apr 15, 2017

omega8cc commented Apr 15, 2017

angel0d commented Apr 19, 2017

omega8cc commented Apr 19, 2017

EdNett commented Aug 1, 2017

omega8cc commented Aug 18, 2017

omega8cc commented Aug 18, 2017

angel0d commented Feb 12, 2018

angel0d commented Feb 12, 2018

angel0d commented Mar 13, 2018

omega8cc commented Mar 14, 2018

omega8cc commented Jun 28, 2016 •

edited

angel0d commented Apr 15, 2017 •

edited

angel0d commented Apr 15, 2017 •

edited

omega8cc commented Apr 15, 2017 •

edited

angel0d commented Apr 15, 2017 •

edited