Add proper GCP config loader and refresher #22

mbohlool · 2017-07-21T21:53:33Z

This is based on #21. I added expiration mechanism as well as persisting config back so we don't need to refresh the token every time. The way it persisted back should be the same as kubectl.
Tests will fail for this PR unless kubernetes-client/python#302 is merged.

gabrielgbim · 2017-07-21T22:02:07Z

Thanks!

mbohlool · 2017-07-22T01:00:55Z

@gabrielgbim @pokoli This is a release blocker. Can you review it please.

codecov-io · 2017-07-23T10:21:00Z

Codecov Report

Merging #22 into master will decrease coverage by 1.05%.
The diff coverage is 87.61%.

@@            Coverage Diff             @@
##           master      #22      +/-   ##
==========================================
- Coverage   94.69%   93.64%   -1.06%     
==========================================
  Files           9       11       +2     
  Lines         698      803     +105     
==========================================
+ Hits          661      752      +91     
- Misses         37       51      +14

Impacted Files	Coverage Δ
config/dateutil_test.py	`100% <100%> (ø)`
config/kube_config.py	`93.17% <77.77%> (-3.41%)`	⬇️
config/dateutil.py	`88.88% <88.88%> (ø)`
config/kube_config_test.py	`92.49% <92.3%> (-0.07%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 00d2417...824c03c. Read the comment docs.

pokoli · 2017-07-24T07:37:57Z

config/kube_config.py

@@ -54,6 +59,17 @@ def _create_temp_file_with_content(content):
    return name


+def _is_expired(expiry):
+    tf = tf_from_timestamp(expiry)
+    n = time.time()


I don't think you need to store n as a variable.

pokoli · 2017-07-24T07:40:43Z

config/kube_config.py

@@ -308,21 +353,32 @@ def load_kube_config(config_file=None, context=None,
        from config file will be used.
    :param client_configuration: The kubernetes.client.ConfigurationObject to
        set configs to.
+    :param persist_config: If True and config changed (e.g. GCP token refresh)


Maybe better: "IF True config file will be updated when changed (e.g GCP token refresh)

pokoli · 2017-07-24T07:41:15Z

config/kube_config.py

    """

    if config_file is None:
        config_file = os.path.expanduser(KUBE_CONFIG_DEFAULT_LOCATION)

+    config_persister = None
+    if persist_config:
+        config_persister = lambda config_map, config_file=config_file: (


Why is required a lambda function here?

to pass the config_file to _save_kube_config. If there is a better way to do it, I am open to it.

AFAIU it should work with:
config_persister = _save_kube_config

the key here is the config_file=config_file: part that passes the local variable config_file as one of the parameters.config_persisteraccepts one parameter,_save_kube_config` needs two.

Ok, I didn't understand at all the code.

Then you should probably use the partial function from functools standard library.

This will make the code clearer.

I inline'ed _save_kube_config. it should be cleaner. is it good or do you still suggest partial function?

Ok, now the usage it's much clearer.

pokoli · 2017-07-24T07:49:23Z

config/kube_config_test.py

        actual = FakeConfig()
        KubeConfigLoader(
            config_dict=self.TEST_KUBE_CONFIG,
            active_context="gcp",
            client_configuration=actual,
-            get_google_credentials=lambda: TEST_ANOTHER_DATA_BASE64) \
+            get_google_credentials=lambda: "SHOULD NOT BE CALLED") \


maybe you can put a raise to ensure it's not called

Will fix this with some other tests.

pokoli · 2017-07-24T07:57:57Z

config/rfc3339.MD

@@ -0,0 +1 @@
+The (rfc3339.py)[rfc3339.py] file is copied from [this site](http://home.blarg.net/~steveha/pyfeed.html) because PyFeed is not available in PyPi.


What about https://pypi.python.org/pypi/rfc3339/5.2 (source code in https://bitbucket.org/henry/rfc3339/src)

looks like they only do one side of conversion? https://bitbucket.org/henry/rfc3339/issues/3/datetime-to-rfc-3339

I see a parse method but it is not exported.

Then probably it's worth to contribute one to the library :)

True. This is, however, a release blocker and I don't want to wait for that contribution to go through. I suggest we proceed with this and either contribute to the that one or even release the one I copied as a pypi package for all to use (I have a feeling by looking at the code that this one is better, but we can decide on that later).

I've looked at different options and decided to implement my own. Some options are too restrictive for our use and the others are too admissive. I've implemented a simple balanced one in dateutil.py.

Ok, as it's a release blocker I think the unique option we have is to include the code in the client.

For the record, I didn't copy code from any of those libraries. I've implemented it again by reading rfc. Of course my implementation can be influenced by those libraries, but I stick to the rules of rfc3339. None of the libraries I've checked were like that, they were either relaxer or stricter. Also our implementation forces a UTC timezone.

mbohlool · 2017-07-25T02:43:35Z

@pokoli I am going to add some tests for dateutil but this is basically ready for review and I already tested it for GKE. Please take another look. I would like to merge this as soon as possible to cut a new release.

pokoli

We should probably need some test for the dateutil module.

pokoli · 2017-07-25T07:18:11Z

config/kube_config.py

    """

    if config_file is None:
        config_file = os.path.expanduser(KUBE_CONFIG_DEFAULT_LOCATION)

+    config_persister = None
+    if persist_config:
+        config_persister = lambda config_map, config_file=config_file: (


AFAIU it should work with:
config_persister = _save_kube_config

mbohlool · 2017-07-25T09:34:32Z

Thanks for the review @pokoli I think I addressed all of your comments and tests are passing. Can you take a final look please?

pokoli

Despite two minor issues its a LGTM

pokoli · 2017-07-25T09:41:13Z

config/kube_config.py

@@ -308,21 +341,35 @@ def load_kube_config(config_file=None, context=None,
        from config file will be used.
    :param client_configuration: The kubernetes.client.ConfigurationObject to
        set configs to.
+    :param persist_config: IF True, config file will be updated when changed


pokoli · 2017-07-25T09:42:46Z

config/kube_config_test.py

@@ -36,6 +38,10 @@ def _base64(string):
    return base64.encodestring(string.encode()).decode()


+def _raise_exception(st):


Why not directly raising the exception?

Why do you mean? I am passing this function as a function pointer to config loader and expect the function pointer (that suppose to update GCE token) never been called in the test. I was using lambda syntax to return a dummy token before, but you cannot raise exception in lambda syntax.

Ok, I didn't now that you can not raise inside lambda (always learning from codereview).

Forget about it.

pokoli · 2017-07-25T09:44:01Z

@mbohlool for me you can merge once fixed.

mbohlool force-pushed the gce_config branch from a4d2b26 to 823f44e Compare July 21, 2017 21:55

mbohlool force-pushed the gce_config branch from 823f44e to b982d5e Compare July 22, 2017 00:59

mbohlool requested a review from pokoli July 22, 2017 01:00

mbohlool force-pushed the gce_config branch 2 times, most recently from f4da0c7 to 253d937 Compare July 23, 2017 10:15

pokoli reviewed Jul 24, 2017

View reviewed changes

mbohlool force-pushed the gce_config branch 2 times, most recently from 5d0b367 to 50f717d Compare July 25, 2017 03:54

pokoli reviewed Jul 25, 2017

View reviewed changes

mbohlool force-pushed the gce_config branch 3 times, most recently from c4b8c87 to bd507f4 Compare July 25, 2017 09:28

pokoli approved these changes Jul 25, 2017

View reviewed changes

Add proper GCP config loader and refresher

824c03c

mbohlool force-pushed the gce_config branch from bd507f4 to 824c03c Compare July 25, 2017 10:17

mbohlool mentioned this pull request Jul 25, 2017

Fixing get Google default credentials method #21

Closed

mbohlool merged commit ab3fc54 into kubernetes-client:master Jul 25, 2017

ofek mentioned this pull request Mar 18, 2019

Bug in rfc3339 implementation #123

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add proper GCP config loader and refresher #22

Add proper GCP config loader and refresher #22

mbohlool commented Jul 21, 2017

gabrielgbim commented Jul 21, 2017

mbohlool commented Jul 22, 2017

codecov-io commented Jul 23, 2017 •

edited

Loading

pokoli Jul 24, 2017

mbohlool Jul 24, 2017

pokoli Jul 24, 2017

mbohlool Jul 24, 2017

pokoli Jul 24, 2017

mbohlool Jul 25, 2017

pokoli Jul 25, 2017

mbohlool Jul 25, 2017 •

edited

Loading

pokoli Jul 25, 2017

mbohlool Jul 25, 2017

pokoli Jul 25, 2017

pokoli Jul 24, 2017

mbohlool Jul 25, 2017

mbohlool Jul 25, 2017

pokoli Jul 24, 2017

mbohlool Jul 24, 2017

pokoli Jul 24, 2017

mbohlool Jul 24, 2017

mbohlool Jul 25, 2017

pokoli Jul 25, 2017

mbohlool Jul 25, 2017 •

edited

Loading

mbohlool commented Jul 25, 2017

pokoli left a comment

pokoli Jul 25, 2017

mbohlool commented Jul 25, 2017

pokoli left a comment

pokoli Jul 25, 2017

pokoli Jul 25, 2017

mbohlool Jul 25, 2017 •

edited

Loading

pokoli Jul 25, 2017

pokoli commented Jul 25, 2017

		@@ -0,0 +1 @@
		The (rfc3339.py)[rfc3339.py] file is copied from [this site](http://home.blarg.net/~steveha/pyfeed.html) because PyFeed is not available in PyPi.

		@@ -36,6 +38,10 @@ def _base64(string):
		return base64.encodestring(string.encode()).decode()


		def _raise_exception(st):

Add proper GCP config loader and refresher #22

Add proper GCP config loader and refresher #22

Conversation

mbohlool commented Jul 21, 2017

gabrielgbim commented Jul 21, 2017

mbohlool commented Jul 22, 2017

codecov-io commented Jul 23, 2017 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbohlool Jul 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbohlool Jul 25, 2017 • edited Loading

Choose a reason for hiding this comment

mbohlool commented Jul 25, 2017

pokoli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbohlool commented Jul 25, 2017

pokoli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbohlool Jul 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pokoli commented Jul 25, 2017

codecov-io commented Jul 23, 2017 •

edited

Loading

mbohlool Jul 25, 2017 •

edited

Loading

mbohlool Jul 25, 2017 •

edited

Loading

mbohlool Jul 25, 2017 •

edited

Loading