Solutions management #2279

alexandre-allard · 2020-03-03T07:16:13Z

Component: salt, scripts, solutions

Context:

Summary:

Acceptance criteria: Ability to import, activate and deploy a Solution in an Environment using the CLI solutions.sh

Closes: #2277

bert-e · 2020-03-03T07:16:15Z

Hello alexandre-allard-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

bert-e · 2020-03-03T07:16:21Z

Conflict

A conflict has been raised during the creation of
integration branch w/2.5/feature/solutions-management with contents from feature/solutions-management
and development/2.5.

I have not created the integration branch.

Here are the steps to resolve this conflict:

 $ git fetch
 $ git checkout -B w/2.5/feature/solutions-management origin/development/2.5
 $ git merge origin/feature/solutions-management
 $ # <intense conflict resolution>
 $ git commit
 $ git push -u origin w/2.5/feature/solutions-management

gdemonet · 2020-03-03T07:38:53Z

scripts/solutions.sh

+        return 1
+    fi
+
+    if ! jq -e ".[] | select(.version == \"$VERSION\")" \


Installing a package just for this single line seems a bit heavy... But that's OK I guess, jq is a nice tool to have at hand, could even make sense to put it in metalk8s-utils container image (see #2156).

FWIW, here's how you could have done it without external deps:

if ! python -c "import json, sys; \ available = any(v['version'] == '$VERSION' for v in json.loads('$solution_data')); \ sys.exit(0 if available else 1)"; then

Yeah, it's a bit heavy, but I think jq can still be useful to parse json as I did here, if one want to read the content of a configmap for example.

I've removed it and used the python solution.

scripts/bootstrap.sh.in

TeddyAndrieux

Some comment but you already know most of them I think

docs/developer/solutions/deploy.rst

TeddyAndrieux · 2020-03-05T13:09:40Z

scripts/solutions.sh

+}
+
+delete_solution() {
+    [[ ${NAMESPACE:-} ]] || NAMESPACE=$NAME


All namespaces used for a solution could be retrieved using label selector like in the salt module,
But IMO we don't need to do it in this PR

Yep, but the user can have 2 namespaces in the same environment, with the same Solution and only want to delete a specific Solution, not both.

Yes indeed but IMO by default we should remove all namespaces even if namespace is not equal to the environement name, but anyway lets not take care of this in the bash script

scripts/solutions.sh

salt/metalk8s/solutions/available.sls

Some objects are required for MetalK8s to manage Solutions, notably a Namespace and ConfigMap for Admin UIs, and the `Environment` CRD. Issue: #1852

A new field, currently named `active` (may be a poor choice, we'll see with first user interactions, `activeVersions` could be more explicit), is introduced to set which version of a Solution should be used for the cluster-wide components of this Solution (i.e. the Admin UI and CRDs for now). Validation of the file format is added, using the usual `kind` and `apiVersion` fields. Later improvements could use some OpenAPI schema. Fixes: #1582

Previous approached relied on a ConfigMap, which poses numerous issues: - it allows one to lie to the formula, potentially causing further problems down the line - the CM cannot always be up-to-date with the actual system state (due to errors when updating it, or manual operations outside of Salt) We change this approach by including basic heuristics in the `metalk8s_solutions` module, which can be refined in the future. Listing of mounted Solutions is made based on `mount.available` and a naive filtering on the mountpoints (necessarily under /srv/scality). Listing of active Solution versions is made by looking for UI Service objects in the `metalk8s-solutions` namespace. This approach relies on labels that we don't set ourselves, which is brittle, and does not account for Solutions that may want to ship without an Admin UI. Future iterations could improve those methods, but we are convinced it is still safer than the ConfigMap method used before. Issue: #1852

Making a Solution archive "available" to the cluster was split into two formulas: `metalk8s.solutions.mounted` and `.configured`. We group them into a single one, called `.available`. We also remove the `.unmounted` and `.unconfigured` formulas, since the source of truth is entirely held by the configuration file (/etc/metalk8s/solutions.yaml). A single formula is responsible for applying the desired state described in this configuration, adding and removing Solution archives from the cluster.

Once a Solution is "available" (i.e. mounted and its images exposed by the cluster registry), its cluster-wide components can be deployed. We rely on the `active` mapping of active versions in the config file to know which components to deploy or remove. Note that one can set a desired version to 'latest' in this mapping, which the orchestration will handle for using the highest version available for a Solution (in SemVer terms). Issue: #1852

We used `display_name` and `machine_id` to differenciate the expected consumers (user or machine). For simplicity (thanks to slaperche-scality), we rename them in `name` and `id` respectively. KISS!

Use Salt to read/write the Solutions config file

Environment CRD is not longer needed as we use directly a ConfigMap in each namespace

Add salt module to read solution environment ConfigMap and retrieve it in the solution ext_pillar

We re-instate the ConfigMap as both a way to track which version was made available (through the `deploy-components` orchestrate) and a way to share the information with MetalK8s UI users.

The setup of "environment" Namespaces and all the resources required to run a Solution instance in it is becoming too complex for it to be properly managed by the Solution Admin UIs, without mentioning the authz requirements on users of said Admin UIs to make it work. Instead, Admin UIs will now be deployed once per environment, making it easier to keep in sync with the same Operator version.

Solution ISOs can now contain a `config.yaml` file to describe the few specifics of its Operator and UI. We expose this information in the `metalk8s:solutions:available` pillar, for later consumption in orchestrations.

Salt is now able to "prepare" environments, based on two sources of information: - the environment *Namespace*s (denoted by some well-known labels) and their 'metalk8s-environment' *ConfigMap*s (which lists the Solution versions to deploy in this environment) - the *SolutionConfig*s stored in each Solution ISO which allows Salt to know what should be deployed Both sources of information are stored in the pillar.

Let's be consistent with the script's names, (we don't have bootstrap-manager, downgrade-manager, ...) the "-manager" suffix is not really useful anyway. Refs: #2277

We add two helpers: - first one to retrieve the saltenv - second one to retrieve the local minion ID These functions will be used in the script managing the Solutions Refs: #2277

We no longer try to reconfigure registry by including metalk8s.repo.installed in metalk8s.solutions.available which is wrong: We alter the pillar by mounting Solutions and the changes are not taken into account inside the repo sls, because Jjinja rendering has already been done at this point, so it reconfigures registry with outdated configuration. We now do this in 2 steps, first mount/unmount Solutions, then reconfigure registry. Refs: #2277

Adding of salt no-op to avoid having an empty sls file after rendering, otherwise it returns an exit code of 2 when using --retcode-passthrough, thus breaking the calling script. Refs: #2277

bert-e · 2020-03-06T14:29:50Z

Integration data created

I have created the integration data for the additional destination branches.

this pull request will merge feature/solutions-management into
development/2.4
w/2.5/feature/solutions-management will be merged into development/2.5
w/2.6/feature/solutions-management will be merged into development/2.6

The following branches will NOT be impacted:

development/1.0
development/1.1
development/1.2
development/1.3
development/2.0
development/2.1
development/2.2
development/2.3

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

bert-e · 2020-03-06T14:29:51Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

The following reviewers are expecting changes from the author, or must review again:

@TeddyAndrieux

TeddyAndrieux · 2020-03-06T14:48:44Z

salt/metalk8s/solutions/available.sls

@@ -0,0 +1,116 @@
+#


Maybe this state should be renamed because he not really make the solutions available now :/ maybe like for archives of metalk8s mounted.sls

Yeah, we'll change that later, we must first merge this asap.

TeddyAndrieux

LGTM

bert-e · 2020-03-06T14:56:38Z

Waiting for approval

The following approvals are needed before I can proceed with the merge:

the author
one peer

Peer approvals must include at least 1 approval from the following list:

alexandre-allard · 2020-03-06T15:10:15Z

/approve

bert-e · 2020-03-06T15:22:22Z

Build failed

The build for commit did not succeed in branch w/2.6/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-06T15:29:53Z

Build failed

The build for commit did not succeed in branch w/2.5/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-06T15:41:40Z

Build failed

The build for commit did not succeed in branch feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-06T16:21:56Z

Build failed

The build for commit did not succeed in branch w/2.6/feature/solutions-management.

The following options are set: approve

thomasdanan · 2020-03-06T16:34:43Z

docs/developer/solutions/deploy.rst

+    ./solutions.sh import --archive </path/to/solution1.iso> \
+      --archive </path/to/solution2.iso>
+
+Unimport a Solution


Unimport is probably not the right term, I would simply say remove

Yes, it's not the right term, it's not even a term. :)
Anyway this stuff will not stay as is and will be changed soon when we'll integrate this script in a centralized CLI.

will be changed soon

soon can mean far future or never in IT 😛 (except if you mean it'll be tackled in your next PR)

Almost my next PR, it's not "maybe, someday... if we have time" :)

bert-e · 2020-03-06T17:04:42Z

Build failed

The build for commit did not succeed in branch w/2.5/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-06T17:07:59Z

Build failed

The build for commit did not succeed in branch feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-06T18:42:39Z

Build failed

The build for commit did not succeed in branch w/2.6/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-06T18:48:40Z

Build failed

The build for commit did not succeed in branch w/2.5/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-06T18:49:42Z

Build failed

The build for commit did not succeed in branch feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-07T00:32:21Z

Build failed

The build for commit did not succeed in branch w/2.5/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-07T00:37:06Z

Build failed

The build for commit did not succeed in branch feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-09T10:40:07Z

Build failed

The build for commit did not succeed in branch w/2.5/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-09T11:16:26Z

Build failed

The build for commit did not succeed in branch w/2.6/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-09T11:19:28Z

Build failed

The build for commit did not succeed in branch w/2.5/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-09T12:16:42Z

Build failed

The build for commit did not succeed in branch w/2.6/feature/solutions-management.

The following options are set: approve

bert-e · 2020-03-09T14:21:24Z

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

✔️ development/2.4
✔️ development/2.5
✔️ development/2.6

The following branches will NOT be impacted:

development/1.0
development/1.1
development/1.2
development/1.3
development/2.0
development/2.1
development/2.2
development/2.3

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

Any commit you add on the source branch will trigger a new cycle after the
current queue is merged.
Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

bert-e · 2020-03-11T15:32:22Z

I have successfully merged the changeset of this pull request
into targetted development branches:

✔️ development/2.4
✔️ development/2.5
✔️ development/2.6

The following branches have NOT changed:

development/1.0
development/1.1
development/1.2
development/1.3
development/2.0
development/2.1
development/2.2
development/2.3

Please check the status of the associated issue None.

Goodbye alexandre-allard-scality.

gdemonet reviewed Mar 3, 2020

View reviewed changes

TeddyAndrieux reviewed Mar 3, 2020

View reviewed changes

scripts/bootstrap.sh.in Outdated Show resolved Hide resolved

alexandre-allard force-pushed the feature/solutions-management branch 2 times, most recently from 1732bca to 3c98eb4 Compare March 3, 2020 08:56

alexandre-allard marked this pull request as ready for review March 3, 2020 08:57

alexandre-allard requested a review from a team as a code owner March 3, 2020 08:57

alexandre-allard force-pushed the feature/solutions-management branch 6 times, most recently from 2fd3615 to 8fbee1c Compare March 3, 2020 17:22

TeddyAndrieux requested changes Mar 5, 2020

View reviewed changes

gdemonet and others added 15 commits March 5, 2020 17:38

salt/solutions: Deploy Solutions K8s dependencies

723abae

Some objects are required for MetalK8s to manage Solutions, notably a Namespace and ConfigMap for Admin UIs, and the `Environment` CRD. Issue: #1852

salt/solutions: Rename some attributes in pillar

5d9ea9b

We used `display_name` and `machine_id` to differenciate the expected consumers (user or machine). For simplicity (thanks to slaperche-scality), we rename them in `name` and `id` respectively. KISS!

salt/solutions: Manage config file from Salt

1d57d99

Use Salt to read/write the Solutions config file

salt: remove Environment CRD

6e1fd64

Environment CRD is not longer needed as we use directly a ConfigMap in each namespace

salt/solutions: environments in Pillar

b77574b

Add salt module to read solution environment ConfigMap and retrieve it in the solution ext_pillar

salt/solution: Reinstate ConfigMap

b38b5f7

We re-instate the ConfigMap as both a way to track which version was made available (through the `deploy-components` orchestrate) and a way to share the information with MetalK8s UI users.

salt/solution: Support SolutionConfig files

3ec19f2

Solution ISOs can now contain a `config.yaml` file to describe the few specifics of its Operator and UI. We expose this information in the `metalk8s:solutions:available` pillar, for later consumption in orchestrations.

scripts: shorten solution-manager script name

6490534

Let's be consistent with the script's names, (we don't have bootstrap-manager, downgrade-manager, ...) the "-manager" suffix is not really useful anyway. Refs: #2277

scripts: add helpers to common shell lib

3167e92

We add two helpers: - first one to retrieve the saltenv - second one to retrieve the local minion ID These functions will be used in the script managing the Solutions Refs: #2277

alexandre-allard added 2 commits March 5, 2020 17:38

salt: add a no-op if there is nothing to do in Solutions sls

ac115f9

Adding of salt no-op to avoid having an empty sls file after rendering, otherwise it returns an exit code of 2 when using --retcode-passthrough, thus breaking the calling script. Refs: #2277

alexandre-allard force-pushed the feature/solutions-management branch from 8fbee1c to ac115f9 Compare March 5, 2020 16:39

TeddyAndrieux reviewed Mar 6, 2020

View reviewed changes

TeddyAndrieux approved these changes Mar 6, 2020

View reviewed changes

thomasdanan reviewed Mar 6, 2020

View reviewed changes

bert-e merged commit ac115f9 into development/2.4 Mar 11, 2020

bert-e deleted the feature/solutions-management branch March 11, 2020 15:32

gdemonet mentioned this pull request Mar 17, 2020

Rewrite Solutions Management (Salt + CLI) #2277

Closed

Solutions management #2279

Solutions management #2279

Conversation

alexandre-allard commented Mar 3, 2020 • edited Loading

bert-e commented Mar 3, 2020

Hello alexandre-allard-scality,

bert-e commented Mar 3, 2020

Conflict

gdemonet Mar 3, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TeddyAndrieux left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bert-e commented Mar 6, 2020

Integration data created

bert-e commented Mar 6, 2020

Waiting for approval

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TeddyAndrieux left a comment

Choose a reason for hiding this comment

bert-e commented Mar 6, 2020

Waiting for approval

alexandre-allard commented Mar 6, 2020

bert-e commented Mar 6, 2020

Build failed

bert-e commented Mar 6, 2020

Build failed

bert-e commented Mar 6, 2020

Build failed

bert-e commented Mar 6, 2020

Build failed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bert-e commented Mar 6, 2020

Build failed

bert-e commented Mar 6, 2020

Build failed

bert-e commented Mar 6, 2020

Build failed

bert-e commented Mar 6, 2020

Build failed

bert-e commented Mar 6, 2020

Build failed

bert-e commented Mar 7, 2020

Build failed

bert-e commented Mar 7, 2020

Build failed

bert-e commented Mar 9, 2020

Build failed

bert-e commented Mar 9, 2020

Build failed

bert-e commented Mar 9, 2020

Build failed

bert-e commented Mar 9, 2020

Build failed

bert-e commented Mar 9, 2020

In the queue

bert-e commented Mar 11, 2020

alexandre-allard commented Mar 3, 2020 •

edited

Loading

gdemonet Mar 3, 2020 •

edited

Loading