Skip to content

Add support for deploying k3s#682

Merged
tserong merged 6 commits intoSUSE:masterfrom
tserong:wip-k3s
Mar 14, 2023
Merged

Add support for deploying k3s#682
tserong merged 6 commits intoSUSE:masterfrom
tserong:wip-k3s

Conversation

@tserong
Copy link
Copy Markdown
Member

@tserong tserong commented Mar 3, 2023

This does a couple of things:

  1. Lets us use sesdev to deploy k3s, just for the sake of having an easy way to create little VM clusters running k3s (think: arbitrary k3s dev/test playground)
  2. Adds the ability to deploy rook on top of k3s, for the sake of testing rook/ceph for ongoing SES maintenance.

@tserong
Copy link
Copy Markdown
Member Author

tserong commented Mar 3, 2023

AFAICT those remaining lint failures aren't actually in code I touched in this PR.

trociny
trociny reviewed Mar 3, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@trociny trociny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sesdev create k3s --deploy-ses failed for me with:

    master: ++ curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
    master: In order to verify checksum, openssl must first be installed.
    master: Please install openssl or set VERIFY_CHECKSUM=false in your environment.
    master: Failed to install helm

After I added export VERIFY_CHECKSUM=false into seslib/templates/k3s/provision.sh.j2 the deploy succeeded and the cluster is working.

Great job! Thanks!

@trociny
Copy link
Copy Markdown
Contributor

trociny commented Mar 3, 2023

Just a note. It might be useful to have the toolbox pod installed during deploy or have a hint message at the end of the deployment :

kubectl create -f ~/cluster/rook-ceph/examples/toolbox.yaml

@kshtsk
Copy link
Copy Markdown
Contributor

kshtsk commented Mar 3, 2023

AFAICT those remaining lint failures aren't actually in code I touched in this PR.

even though they must be fixed

@kshtsk
Copy link
Copy Markdown
Contributor

kshtsk commented Mar 3, 2023

@tserong btw, the deployment test is still failing how about to fix the repo issues or/and switch to the newer ses version in self test.

@tserong
Copy link
Copy Markdown
Member Author

tserong commented Mar 6, 2023 •
edited
Loading

@trociny I've updated this to explicitly install openssl to fix helm's VERIFY_CHECKSUM thing. I've also added toolbox deployment, because I agree that seems pretty useful. I've done that as separate commits so it's easy to see changes, but can squash later.

@kshtsk I've cherry picked your commits from #683 here to hopefully get a clean jenkins and tox run, but I'm happy to rebase this if you want to merge your PR before merging this one.

@kshtsk
Copy link
Copy Markdown
Contributor

kshtsk commented Mar 6, 2023

@tserong I gonna try this PR pretty soon

@kshtsk
Copy link
Copy Markdown
Contributor

kshtsk commented Mar 6, 2023

Looks interesting, could you please rebase and merge.

tserong added 5 commits March 7, 2023 17:37
@tserong
Add support for deploying k3s
9ddf6e2 
This adds `sesdev create k3s`, which by default will deploy a
master with four workers on openSUSE Tumbleweed.  It will also
deploy on SLE 15 SP3 if you specify `--os=sles-15-sp3`.

k3s is installed by running `curl https://get.k3s.io`.  Helm
is also installed on the master, again using `curl`.  If you
don't want the latest stable k3s, you can install a specific
version with the `--k3s-version` option.  If you want more or
less workers, use `--roles='[master],[worker],[...]`.

Signed-off-by: Tim Serong <tserong@suse.com>
@tserong
Add ability to deploy Ceph via Rook on k3s
1ede85a 
This adds the `--deploy-ses` option to `sesdev create k3s`,
which will pull registry.suse.com/ses/7.1/charts/rook-ceph,
then deploy Rook, which in turn will deploy Ceph, and have
it eat all the disks on all the workers.

Signed-off-by: Tim Serong <tserong@suse.com>
@tserong
README.md: Add k3s notes
5fdffe5 
Signed-off-by: Tim Serong <tserong@suse.com>
@tserong
k3s: install openssl for helm checksum verification
a3750a8 
When deploying on SLE 15, openssl is already installed as
it's a dependency of ca-certificates.  This is not true on
Tumblweed though, so let's just always install it.

Signed-off-by: Tim Serong <tserong@suse.com>
@tserong
k3s: deploy ceph toolbox container
96b696a 
Signed-off-by: Tim Serong <tserong@suse.com>
@tserong
Copy link
Copy Markdown
Member Author

tserong commented Mar 7, 2023

Rebased

@votdev votdev self-requested a review March 7, 2023 07:03
@kshtsk
Copy link
Copy Markdown
Contributor

kshtsk commented Mar 7, 2023 

    master: ++ ceph status
    master: 2023-03-07T10:11:23.938+0000 7faa9004e700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory
    master: 2023-03-07T10:11:23.938+0000 7faa9004e700 -1 AuthRegistry(0x7faa8805f158) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling cephx
    master: 2023-03-07T10:11:24.046+0000 7faa9004e700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory
    master: 2023-03-07T10:11:24.046+0000 7faa9004e700 -1 AuthRegistry(0x7faa9004cf50) no keyring found at /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling cephx
    master: 2023-03-07T10:11:24.246+0000 7faa8ddea700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [1]
    master: 2023-03-07T10:11:24.250+0000 7faa9004e700 -1 monclient: authenticate NOTE: no keyring found; disabled cephx authentication
    master: [errno 13] RADOS permission denied (error connecting to the cluster)
    master: +++ err_report 638
    master: +++ local hn
    master: +++ set +x
    master: Error in provisioner script trapped!

@kshtsk
Copy link
Copy Markdown
Contributor

kshtsk commented Mar 7, 2023

Just wonder why it wasn't failing before rebase, and btw, it is not reproducible on master now.

@kshtsk kshtsk self-requested a review March 7, 2023 10:31
kshtsk
kshtsk reviewed Mar 7, 2023
View reviewed changes
Comment thread sesdev/__init__.py Outdated
@tserong
k3s: add k3s_deploy_ses setting
be0bfab 
This avoids causing confusion by overloading the caasp_deploy_ses setting.

Signed-off-by: Tim Serong <tserong@suse.com>
@kshtsk
Copy link
Copy Markdown
Contributor

kshtsk commented Mar 13, 2023

so, gonna merge this?

@tserong tserong merged commit 1cf4353 into SUSE:master Mar 14, 2023
@tserong tserong deleted the wip-k3s branch March 14, 2023 02:04
@tserong
Copy link
Copy Markdown
Member Author

tserong commented Mar 14, 2023

so, gonna merge this?

Yep :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shaas shaas Awaiting requested review from shaas

@mgfritch mgfritch Awaiting requested review from mgfritch

@m-ildefons m-ildefons Awaiting requested review from m-ildefons

@votdev votdev Awaiting requested review from votdev

2 more reviewers

@trociny trociny trociny left review comments

@kshtsk kshtsk kshtsk approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@tserong @trociny @kshtsk