Kubernetes configured via kubeconfig #46244

mcalmer · 2018-02-28T12:13:54Z

What does this PR do?

Newer versions of python-kubernetes (>2.0) has changed a lot in the way they want to have the configuration parameters. Also new types of authentication were added and more will come in near future.
All this make the current way to provide authentication data a bit useless.

The only future-proof way seems to be to provide the full kubeconfig file and let the python library
do the rest.

What issues does this PR fix or reference?

Mainly it support new python-kubernetes library as requested in #44701 .

Configuring via kubeconfig was requested in #43514 .

Last but not least this PR make it also possible to provide the full kubeconfig as data to overwrite configured kubeconfig like requested in #46086 .

Previous Behavior

Configuring the cluster access and authentication was possible via single values:

kubernetes.user
kubernetes.password
kubernetes.api_url
kubernetes.certificate-authority-data/file
kubernetes.client-certificate-data/file
kubernetes.client-key-data/file

New Behavior

This will be replaced by just configure via kubeconfig and context

kubernetes.kubeconfig
kubernetes.context

Tests written?

Yes (adapted the existing)

Commits signed with GPG?

No

Related PR

2017.7 - add warning about config option change : #46320

rallytime · 2018-02-28T18:59:53Z

@mcalmer Since Kubernetes is moving pretty quickly and this change will make it easier to maintain this module moving forward, I am fine with changing these configuration items in a feature release. However, I want to make sure that this is documented appropriately - there should be warnings at the top of the module that these configuration options have changed, and it probably would be good to add this to the Fluorine release notes.

It would also be prudent to add some deprecation warnings to the old configuration style in the 2017.7 branch.

mcalmer · 2018-02-28T19:19:24Z

@rallytime do you have an example for such a text? Just to use a similar format and wording.

isbm

@mcalmer could you please take a look at these changes? Thanks!

isbm · 2018-02-28T21:52:12Z

salt/modules/kubernetes.py

You don't need this (the whole elif).

isbm · 2018-02-28T21:53:53Z

salt/modules/kubernetes.py

You don't need kubeconfig_data_overwrite too.

isbm · 2018-02-28T21:54:10Z

salt/modules/kubernetes.py

You don't need this.

isbm · 2018-02-28T21:55:34Z

salt/modules/kubernetes.py

Just that:

kubeconfig_data = kwargs.get('kubeconfig_data') or __salt__['config.option']('kubernetes.kubeconfig-data')

Your suggestion is not directly working like it should. But I modify it to go in that direction.

isbm · 2018-02-28T22:02:44Z

salt/modules/kubernetes.py

No need to stat file here, you could just try/except and check for errno.ENOENT:

try: os.unlink(kubeconfig) except (IOError, OSError) as err: if err.errno != errno.ENOENT: log.exception()

If you think no logging should be happening, then the safe_rm all it does it try/except:pass. 😉 So you can simply safe_rm it as is w/o pre-checks.

BTW, interesting that you don't know what is inside kubeconfig... 😉

No, this is not working. I only want to remove it when it is a temp file created by the module itself.
So just unlink is not an option. I can either just check if I have something in kubeconfig or call
path.exists(). I like to check if kubeconfig is a "path" and if it exists before I remove it.

isbm · 2018-02-28T22:15:19Z

tests/unit/modules/test_kubernetes.py

Why copypaste the same thing all over again? It can be just a static method and you could do something like:

@staticmethod def kube_settings(name, value=None): data = {....} return data.get(name, value) def test......: with mock_kubernetes_library() ....: with patch.dict(kubernetes.__salt__, {'config.option': Mock(side_effect=KubernetesTestCase.kube_settings)}): ...

...and then just reuse it twice. I am not fan with multiple nested with statements (which you still can have multiple conditions in just one statement) and I would use @patch decorator instead, but for now it is OK.

Please refactor this.

isbm · 2018-02-28T22:21:45Z

tests/unit/modules/test_kubernetes.py

It can be just one statement, BTW.

isbm · 2018-03-01T10:16:07Z

salt/modules/kubernetes.py

In Salt core normally we just so: log.exception(err) — and it will do all the job for you.

isbm · 2018-03-01T10:18:29Z

@mcalmer just a little nitpick with the log.exception(...) but the rest is great and much simpler now! Thanks.

isbm · 2018-03-01T10:19:31Z

salt/modules/kubernetes.py

Typo: preferred.

isbm · 2018-03-01T10:20:25Z

salt/modules/kubernetes.py

isbm

@mcalmer thanks!

rallytime · 2018-03-02T19:21:13Z

@mcalmer I don't have any doc examples handy off the top of my head, but something along the lines of "The following configuration options have been removed in favor of x, y, z" and list out which items were removed (and why they were removed) and what to use instead. For example, if someone is using 2017.7.x or 2018.3.x and they upgrade and now their modules/states don't work, they need to know why and how to remediate the issue. This needs to be added to the Fluorine release notes.

I also would like to see a PR implementing the warn_util warnings in the 2017.7 branch so people will know that the old config options will be removed in Fluorine.

jryberg · 2018-03-03T10:24:36Z

Will overrides in states still be possible using this method? I'm connecting to multiple clusters from the same minion.

mcalmer · 2018-03-03T11:17:12Z

@rallytime I have adapted the warning message here, but I am not sure if using warn_until in older versions is a good idea. These messages would just pollute the logfiles with tons of warnings and
the admin has no chance to remove them because the new options are not available in the old version.

So I would like to go with just a message in the docs.

An alternative would be to backport the feature and try to make both options available - let's say in oxygen. Then warn_until would make sense because the admin could switch to the new config style and prevent the warnings.

What do you prefer?

mcalmer · 2018-03-03T11:18:10Z

@jryberg - yes. Overwrite is still possible. But now you use a full kubeconfig instead of single values out of it.

isbm · 2018-03-03T17:47:59Z

@mcalmer I would say, we should have indeed some sort of warning and a proper deprecation process. For this we actually using deprecation mechanism and it is just a decorator on top of a function. This decorator makes possible to retire old signatures without losing the API names. Full doc of which you can read here. This is something like:

def _stuff():
    'This has old signature. Underscore in front is hiding it initially'

@with_deprecated(globals(), "Beryllium")
def stuff(foo=None):
    'This has new signature'

Then, depending on what user wants to use (new or old Kubernetes), they just configure the minion accordingly.

mcalmer · 2018-03-03T18:06:24Z

@isbm - I fear it will not work as the signatures stay unchanged for all methods. Only configuration in the background will be different and you can provide different kwargs but they are optional.

isbm · 2018-03-03T18:30:29Z

@mcalmer technically, surely it will. It is just in this case it is equivalent to:

@with_deprecated(globals(), "Fluorine")
def foo(**kwargs):
    ... # all the real code

_foo = foo

The problem is here that due to the nature of the **kwargs that can be anything inside, it does not matter which you use: new or old, since it is technically the same function reference. So the workaround would be a bit of refactoring:

def _version_blocker(func, **kwargs):
    '''
    This will raise an exception if kubernetes is more than 2.0, for example.
    '''
    if _get_version_of_kubernetes() > (2, 0):  # Check what API we are linked to
        raise CommandExecutionError('Sticky bit become loose!')
    return func(**kwargs)

@with_deprecated(globals(), "Fluorine")
def foo(**kwargs):
    ... # all the real code

_foo = lambda **kwargs: _version_blocker(foo, **kwargs)

@with_deprecated(globals(), "Fluorine")
def bar(**kwargs):
    ... # all the real code

_bar = lambda **kwargs: _version_blocker(bar, **kwargs)

Well, you've got the idea.

rallytime · 2018-03-06T18:53:20Z

@mcalmer Regarding your questions about a proper deprecation process - what if we keep the old way of doing things for now with a warn_util message when the old configs are found, and then users can update over the next couple of feature releases and then we can remove the old behavior after the deprecation period has passed. This is what we usually do and I should have suggested doing that the first time around. My apologies for the run-around on that!

mcalmer · 2018-03-10T18:01:09Z

@rallytime - It is not easy as new python-kubernetes client lib do not support this old way.
So I bring just the old code back and try to catch all exceptions which might get thrown because of invlaud use of the API. Old configuration can only be used upto v2.0.0 . New style works with all.

…tead of single values Initializing kubernetes client using kubeconfig is future-proof because a lot of new auth methods were added recently and maybe added in future.

rallytime

👍

rallytime · 2018-03-12T18:42:53Z

re-run ubuntu

rallytime requested a review from a team February 28, 2018 19:00

isbm suggested changes Feb 28, 2018

View reviewed changes

isbm suggested changes Mar 1, 2018

View reviewed changes

salt/modules/kubernetes.py Outdated

Copy link

Contributor

isbm Mar 1, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Salt core normally we just so: log.exception(err) — and it will do all the job for you.

isbm suggested changes Mar 1, 2018

View reviewed changes

mcalmer force-pushed the k8s-with-kubeconfig branch from 5005e6e to fdf8fdb Compare March 1, 2018 16:49

isbm approved these changes Mar 2, 2018

View reviewed changes

mcalmer mentioned this pull request Mar 3, 2018

add warning about future config option change #46320

Merged

mcalmer force-pushed the k8s-with-kubeconfig branch from cd33e0c to 3d357a6 Compare March 10, 2018 18:00

mcalmer force-pushed the k8s-with-kubeconfig branch from 3d357a6 to ada08c2 Compare March 10, 2018 18:35

mcalmer added 9 commits March 11, 2018 11:28

change kubernetes access configuration to provide full kubeconfig ins…

e687aad

…tead of single values Initializing kubernetes client using kubeconfig is future-proof because a lot of new auth methods were added recently and maybe added in future.

adapt unit tests

f191b06

fix typo in description

14a2429

add warning about configuration options changed

46f88d5

simplify connection setup logic

453b3d2

extract settings function in unit tests

db91ed0

log error when removing of kubeconfig file failed

6a3983e

fix typos

ebf5ddd

pylint

f96e2f4

mcalmer added 4 commits March 11, 2018 11:28

enhance warning about configuration option change

af8a665

Get old style configuration back until Sodium release

572d543

template must not be None

08a3dcf

provide valid config for unit tests

7e39a95

mcalmer force-pushed the k8s-with-kubeconfig branch from 0f74c6c to 7e39a95 Compare March 11, 2018 10:29

rallytime approved these changes Mar 12, 2018

View reviewed changes

rallytime merged commit bcf8436 into saltstack:develop Mar 12, 2018

mcalmer mentioned this pull request Mar 20, 2018

Kubernetes python client version 4.0 causes errors #44701

Closed

mcalmer deleted the k8s-with-kubeconfig branch March 1, 2020 15:19

Kubernetes configured via kubeconfig #46244

Kubernetes configured via kubeconfig #46244

Uh oh!

Conversation

mcalmer commented Feb 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

What issues does this PR fix or reference?

Previous Behavior

New Behavior

Tests written?

Commits signed with GPG?

Related PR

Uh oh!

rallytime commented Feb 28, 2018

Uh oh!

mcalmer commented Feb 28, 2018

Uh oh!

isbm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

isbm Feb 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

isbm commented Mar 1, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

isbm left a comment

Choose a reason for hiding this comment

Uh oh!

rallytime commented Mar 2, 2018

Uh oh!

jryberg commented Mar 3, 2018

Uh oh!

mcalmer commented Mar 3, 2018

Uh oh!

mcalmer commented Mar 3, 2018

Uh oh!

isbm commented Mar 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcalmer commented Mar 3, 2018

Uh oh!

isbm commented Mar 3, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rallytime commented Mar 6, 2018

Uh oh!

mcalmer commented Mar 10, 2018

Uh oh!

rallytime left a comment

Choose a reason for hiding this comment

Uh oh!

rallytime commented Mar 12, 2018

Uh oh!

mcalmer commented Feb 28, 2018 •

edited

Loading

isbm Feb 28, 2018 •

edited

Loading

isbm commented Mar 1, 2018 •

edited

Loading

isbm commented Mar 3, 2018 •

edited

Loading

isbm commented Mar 3, 2018 •

edited

Loading