Completely replace existing MicroOS kiwi

[AvengerMoJo]

MicroOS using cutomized liveiso persistent script that is not easy to debug.
Since we are not using all the distribution profile in the MicroOS kiwi configuration.
I rewrite the kiwi config with the liveiso being persistent by the driver available storage

[kshtsk]

jenkins test leap

[kshtsk]

@AvengerMoJo kiwi-ng failed with unresolved dependecy on tumbleweed

[jecluis]

We many not want a Vagrant image. I don't think it's a good idea to do vagrant-specific things by default.

[AvengerMoJo]

I can put that into vagrant specific profile but do we need to add other user e.g.live iso and container as well? If that's the case we need some kind of user one way or another.

[jecluis]

I think this is a really bad idea to do by default. This should only happen if the image being built is Vagrant's.

[AvengerMoJo]

For the live-iso we still need another user, any suggestions?

[jecluis]

Instead of replace ceph distribution I'd say it wraps around Ceph.

[AvengerMoJo]

Sure, just to replacing microOS to something closer to what we are actually building.

[jecluis]

This feels something that we're never going to be using on a live/installation image. I understand you need it there to write the authorized_keys, but I don't think that's the way to go here. I don't think this is something we want to manage from a live boot image, which will force us to clean up this user on every single boot.

[AvengerMoJo]

oh now I see why you think differently ....from my experience I create default user for liveiso and installation as well so user can skip the so call "firstboot" process to input anything or adduser etc to reduce the adoption difficulty. My thought was if we are going to setup vagrant user may as well using the same. We can change it but I think we need to setup user by default for all images.

[jecluis]

Assuming we can change a user's password on first boot and persist that, I think we can have a default user to allow the debugging and other maintenance tasks should one want/need to. I think that is fine. In that case I'd add an aquarium user, with passwordless sudoers rights, and would not set up a root password at all.

As for vagrant, I really think this should only be available if we are running a Vagrant image - otherwise it makes little sense to have it around, even if just from a cosmetic point of view. But, more importantly, we can't be having a default authorized_keys with a ssh key that is known to everyone -- that is simply a security risk we can't afford on a real system. It's fine for a vagrant image running locally for testing, but not if we are running Aquarium on production.

Does that make sense?

[AvengerMoJo]

Yeah totally. I understand that. However, let listed all our images.
1 and 2 Both Install image and LiveISO should able to persistent password and configuration.
So user will update their password after install and using liveiso for long term and even production. (Discussion 1)
3 vagrant using ssh key also needed to be around and require to update once user wanted to use for long term. (Discussion 2)

For install and live iso both either require users to create the "first-boot" process to make a system working. However, that also makes it a little bit more difficult to use. To me they are two different approaches but not security-related which is how easy can someone with no background able to download and run it.

Appliance, download, start iso or dd USB, boot, running.
VS
Start LiveISO configuration, ask questions, setting .... etc then Running.

Meanwhile, this discussion can also apply to all the configurations being default to different settings e.g. I use DHCP for the first nic, if we have two interfaces, then I create a bond with two different subnets automatically to build a front and back subnet etc.

I can do either way, but pre-configuration default can be 1 of the multiple configurations we pick for Quick Start. And user can easily adopt and understand. Alternative may require us to create an installation process that may require lots of work we don't want to maintain in the long run. ( right now assume we can leverage yast tools which is ok, but that fail back to different distribution user may not used to the interface for system configuration tool)

[AvengerMoJo]

Discussion 2 I agreed I can create an aquarium user who is a better user than a vagrant user.
However, I was just thinking in terms of the regular users my ask ...why do you have multiple users for different images type and not the same as 1
So they need to remember which user to change the password after install and which platform using which user etc.
Therefore, I don't mind adding aquarium user for LiveISO and Install Profile. But adding more image type may end up adding more user role, should we consolidate into a single user.
For the root password I could disable it, I was testing the root password because microOS disable it and sudo didn't work. So I will remove the root login after user and sudo is working.

[jecluis]

1 and 2 Both Install image and LiveISO should able to persistent password and configuration.
So user will update their password after install and using liveiso for long term and even production. (Discussion 1)

Cool - I think that way we can indeed have a very basic default password (e.g., aquarium) for a default user aquarium, and have us change that through the frontend before bootstrapping. In the mean time, the user can use aquarium:aquarium however they see fit.

3 vagrant using ssh key also needed to be around and require to update once user wanted to use for long term. (Discussion 2)

I don't think the user will ever use the same image from Vagrant to a live system. They are two different images, and one will be running under vagrant and the other will be used on an actual machine. I don't think there's, nor should there be, a path from Vagrant -> Live ISO. So I don't think this is something we should care about.

For install and live iso both either require users to create the "first-boot" process to make a system working. However, that also makes it a little bit more difficult to use. To me they are two different approaches but not security-related which is how easy can someone with no background able to download and run it.

Appliance, download, start iso or dd USB, boot, running.
VS
Start LiveISO configuration, ask questions, setting .... etc then Running.

I think this is a fair point. We should make the experience as easy to the user as possible. Having a default user/password is not the end of the world, assuming we can indeed change it and persist it -- which I'm assuming is the case based on your previous comment.

Meanwhile, this discussion can also apply to all the configurations being default to different settings e.g. I use DHCP for the first nic, if we have two interfaces, then I create a bond with two different subnets automatically to build a front and back subnet etc.

For now, let's assume that the user can configure those things manually through the Live image. We may want to allow them to configure that, at some point, through the installer, but that's a stretch goal that is still a bit far away :)

I can do either way, but pre-configuration default can be 1 of the multiple configurations we pick for Quick Start. And user can easily adopt and understand. Alternative may require us to create an installation process that may require lots of work we don't want to maintain in the long run. ( right now assume we can leverage yast tools which is ok, but that fail back to different distribution user may not used to the interface for system configuration tool)

I'm not following here: do you mean have pre-built images with bonding enabled and such?

[jecluis]

Maybe add an archive with the ssh's authorized_keys on this profile?

[AvengerMoJo]

#316 (comment)

Fix the vagrant build but I never look at the image size before and somehow it size growth to the 20G+, need to look into why.

[jecluis]

#316 (comment)

Fix the vagrant build but I never look at the image size before and somehow it size growth to the 20G+, need to look into why.

That is because of the <size> tag in there. It allocates 24G for the image. This is sparse though, so the final vagrant image should actually be much smaller.

[kshtsk]

>>> SSH server on the guest doesn't support SCP. Please install the necessary
>>> software to enable SCP on your guest operating system.

The image still have some problems.

[AvengerMoJo]

image/config.sh: line 76: baseVagrantSetup: command not found

@kshtsk is it possible to update the kiwi version in the jenkins build environment?
This command is actually working in 9.23. or I will switch it back to script command

[kshtsk]

image/config.sh: line 76: baseVagrantSetup: command not found

@kshtsk is it possible to update the kiwi version in the jenkins build environment?
This command is actually working in 9.23. or I will switch it back to script command

to be precise it is not a jenkins environment, it is a case with Leap, the maximum version available on 15.2 is 9.21.23-lp152.5.13.1

If you can get the newer version get to the leap 15.2 repo or adopt it for current released Leap

[kshtsk]

Btw, the latest kiwi in not released Leap 15.3 is:
http://download.opensuse.org/distribution/leap/15.3/repo/oss/x86_64/python3-kiwi-9.21.23-3.30.2.x86_64.rpm

[kshtsk]

as alternative your could try and fix setup dev scripts to use kiwi from pypi instead of system packages.

[kshtsk]

Also as workaround in case of vagrant version lower then mentioned above, you could declare function taken from it:

#======================================
# baseVagrantSetup
#--------------------------------------
function baseVagrantSetup {
    # insert the default insecure ssh key from here:
    # https://github.com/hashicorp/vagrant/blob/master/keys/vagrant.pub
    mkdir -p /home/vagrant/.ssh/
    echo "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key" > /home/vagrant/.ssh/authorized_keys
    chmod 0600 /home/vagrant/.ssh/authorized_keys
    chown -R vagrant:vagrant /home/vagrant/

    # recommended ssh settings for vagrant boxes
    echo "UseDNS no" >> /etc/ssh/sshd_config
    echo "GSSAPIAuthentication no" >> /etc/ssh/sshd_config

    # vagrant assumes that it can sudo without a password
    # => add the vagrant user to the sudoers list
    SUDOERS_LINE="vagrant ALL=(ALL) NOPASSWD: ALL"
    if [ -d /etc/sudoers.d ]; then
        echo "$SUDOERS_LINE" >| /etc/sudoers.d/vagrant
        visudo -cf /etc/sudoers.d/vagrant
        chmod 0440 /etc/sudoers.d/vagrant
    else
        echo "$SUDOERS_LINE" >> /etc/sudoers
        visudo -cf /etc/sudoers
    fi

    # the default shared folder
    mkdir -p /vagrant
    chown -R vagrant:vagrant /vagrant

    # SSH service
    baseInsertService sshd
}

[AvengerMoJo]

@kshtsk I don't understand the jenkin OpenStack Cloud build fail. Do we need that?

[kshtsk]

jenkins run leap

[kshtsk]

@kshtsk I don't understand the jenkin OpenStack Cloud build fail. Do we need that?

OVH server creation failure, I triggered another run

[AvengerMoJo]

@kshtsk I don't understand the jenkin OpenStack Cloud build fail. Do we need that?

OVH server creation failure, I triggered another run

Thanks I didn't do anything and all the checks are passed!

[kshtsk]

Nevertheless the sudo systemctl status aquarium check passed, looks like this PR introduces the next failure:

vagrant@node01:~> sudo journalctl -u aquarium
-- Logs begin at Mon 2021-03-29 13:00:47 UTC, end at Mon 2021-03-29 13:10:04 UTC. --
Mar 29 13:01:17 localhost systemd[1]: Started Project Aquarium.
Mar 29 13:01:20 localhost uvicorn[1353]: INFO:     Started server process [1353]
Mar 29 13:01:20 localhost uvicorn[1353]: INFO:     Waiting for application startup.
Mar 29 13:01:20 localhost uvicorn[1353]: INFO:     2021-03-29 13:01:20 -- aquarium -- Aquarium startup!
Mar 29 13:01:20 localhost uvicorn[1353]: INFO:     2021-03-29 13:01:20 -- on -- Application startup complete.
Mar 29 13:01:20 localhost uvicorn[1353]: INFO:     2021-03-29 13:01:20 -- on -- Application startup complete.
Mar 29 13:01:20 localhost uvicorn[1353]: INFO:     2021-03-29 13:01:20 -- server -- Uvicorn running on http://0.0.0.0:1337 (Press CTRL+C to quit)
Mar 29 13:01:20 localhost uvicorn[1353]: INFO:     2021-03-29 13:01:20 -- server -- Uvicorn running on http://0.0.0.0:1337 (Press CTRL+C to quit)
Mar 29 13:01:45 node01 sudo[4712]:     root : PWD=/usr/share/aquarium ; USER=root ; COMMAND=/usr/sbin/cephadm gather-facts
Mar 29 13:01:45 node01 sudo[4712]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=0)
Mar 29 13:01:45 node01 sudo[4712]: pam_unix(sudo:session): session closed for user root
Mar 29 13:01:45 node01 uvicorn[1353]: ERROR:    2021-03-29 13:01:45 -- base_events -- Task exception was never retrieved
Mar 29 13:01:45 node01 uvicorn[1353]: future: <Task finished name='Task-102' coro=<Ticker.tick() done, defined at ./gravel/controllers/gstate.py:91> exception=CephadmError('error obtaining node info')>
Mar 29 13:01:45 node01 uvicorn[1353]: Traceback (most recent call last):
Mar 29 13:01:45 node01 uvicorn[1353]:   File "./gravel/cephadm/cephadm.py", line 161, in get_node_info
Mar 29 13:01:45 node01 uvicorn[1353]:     facts = await self.gather_facts()
Mar 29 13:01:45 node01 uvicorn[1353]:   File "./gravel/cephadm/cephadm.py", line 134, in gather_facts
Mar 29 13:01:45 node01 uvicorn[1353]:     raise CephadmError(stderr)
Mar 29 13:01:45 node01 uvicorn[1353]: gravel.cephadm.cephadm.CephadmError: ERROR: Unable to locate any of ['podman', 'docker']
Mar 29 13:01:45 node01 uvicorn[1353]: The above exception was the direct cause of the following exception:
Mar 29 13:01:45 node01 uvicorn[1353]: Traceback (most recent call last):
Mar 29 13:01:45 node01 uvicorn[1353]:   File "./gravel/controllers/gstate.py", line 101, in tick
Mar 29 13:01:45 node01 uvicorn[1353]:     await self._do_tick()
Mar 29 13:01:45 node01 uvicorn[1353]:   File "./gravel/controllers/resources/inventory.py", line 51, in _do_tick
Mar 29 13:01:45 node01 uvicorn[1353]:     await self.probe()
Mar 29 13:01:45 node01 uvicorn[1353]:   File "./gravel/controllers/resources/inventory.py", line 59, in probe
Mar 29 13:01:45 node01 uvicorn[1353]:     nodeinfo = await cephadm.get_node_info()
Mar 29 13:01:45 node01 uvicorn[1353]:   File "./gravel/cephadm/cephadm.py", line 164, in get_node_info
Mar 29 13:01:45 node01 uvicorn[1353]:     raise CephadmError("error obtaining node info") from e
Mar 29 13:01:45 node01 uvicorn[1353]: gravel.cephadm.cephadm.CephadmError: error obtaining node info

[AvengerMoJo]

@kshtsk missing the container runtime packages, adding that should fix the runtime error

[kshtsk]

jenkins test leap

[AvengerMoJo]

@jecluis if all the images types are working, I created a squashed version
https://github.com/AvengerMoJo/aquarium/tree/final_liveiso-persistent
for PR to keep the history clean.

[jecluis]

@AvengerMoJo alright, lets force push that branch to this branch so we can have the final result in this PR?

[AvengerMoJo]

@AvengerMoJo alright, lets force push that branch to this branch so we can have the final result in this PR?

I'm not sure what I did wrong but force pull created lot of merge request which is not exactly what I wanted so I will just close this and use the new one for PR #354 . Sorry about that.