feat(sdk): added sanitizing k8s object process and test. Fixes: #332 #345

jizg · 2020-10-27T01:27:59Z

Which issue is resolved by this Pull Request:
Resolves #332

Description of your changes:
Added process and testcases for recursively sanitizing k8s container object type based on the kubernetes.client.models definition. Currently primitive types of string, integer, boolean, list of string are all the types included in kubernetes.client.models.V1Container and its children attributes.

Environment tested:

Python Version (use python --version): 3.7.7
Tekton Version (use tkn version): 0.13.1
Kubernetes Version (use kubectl version): v1.16.15
OS (e.g. from /etc/os-release): Ubuntu 18.04

Checklist:

The title for your pull request (PR) should follow our title convention. Learn more about the pull request title convention used in this repository.

PR titles examples:
- fix(frontend): fixes empty page. Fixes #1234
  Use fix to indicate that this PR fixes a bug.
- feat(backend): configurable service account. Fixes #1234, fixes #1235
  Use feat to indicate that this PR adds a new feature.
- chore: set up changelog generation tools
  Use chore to indicate that this PR makes some changes that users don't need to know.
- test: fix CI failure. Part of #1234
  Use part of to indicate that a PR is working on an issue, but shouldn't close the issue when merged.
Do you want this pull request (PR) cherry-picked into the current release branch?

If yes, use one of the following options:
- (Recommended.) Ask the PR approver to add the cherrypick-approved label to this PR. The release manager adds this PR to the release branch in a batch update.
- After this PR is merged, create a cherry-pick PR to add these changes to the release branch. (For more information about creating a cherry-pick PR, see the Kubeflow Pipelines release guide.)

google-cla · 2020-10-27T01:28:03Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

kubeflow-bot · 2020-10-27T01:28:04Z

This change is

jizg · 2020-10-27T01:30:10Z

@googlebot I signed it!

google-cla · 2020-10-27T01:30:14Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

jizg · 2020-10-27T01:33:54Z

@Tomcli How to sign CLA? Shall I choose contribution on behalf of your Employer?

google-cla · 2020-10-27T12:16:29Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-10-27T12:22:26Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

google-cla · 2020-10-27T12:32:12Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

drewbutlerbb4 · 2020-10-27T15:54:13Z

Hi @jizg there is a process that you need to follow to join the IBM Google CLA group before you are covered under the CLA. I have sent you a slack message with the IBM internal link to the instructions.

Tomcli · 2020-10-27T17:29:47Z

Thanks @jizg for your contribution. This PR looks good to me. We can approve this PR once you signed the cla.

jizg · 2020-10-29T08:32:38Z

I have raised a request to join IBM contributor google group, and it should be soon to sign the CLA.

jizg · 2020-10-30T00:57:01Z

@googlebot I signed it!

google-cla · 2020-10-30T00:57:08Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

jizg · 2020-10-30T01:01:52Z

@googlebot I signed it!

jizg · 2020-10-30T01:09:01Z

@Tomcli I've signed the CLA. Would you please check whether it's ok to merge this PR? Thanks

Tomcli · 2020-10-30T17:43:45Z

thanks @jizg
/lgtm

drewbutlerbb4 · 2020-10-30T22:45:41Z

sdk/python/kfp_tekton/compiler/_k8s_helper.py

+
+
+def _to_str(s):
+    return None if s is None else str(s)


Thanks @jizg, quick question. Is there a reason we need to check for None here? Doesn't this get checked in sanitize_k8s_object? Or are there plans to use _to_str () outside this file in the future?

You are right, '_to_str()' is not necessary now. I firstly did the 'None' type check in '_to_str()', and then moved the check to sanitize_k8s_object but forgot to remove '_to_str()'. I have updated the codes. Thanks @drewbutlerbb4 .

drewbutlerbb4 · 2020-10-31T16:47:59Z

Thanks @jizg
/lgtm

Tomcli · 2020-11-02T22:07:02Z

thanks @jizg
/lgtm
/approve

k8s-ci-robot · 2020-11-02T22:07:12Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jizg, Tomcli

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Tomcli]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…sanitizing-k8s-objects * commit '3d3ecfb05fc13f400d4aec5ee5a0ca9b13b1566f': feat(sdk): added sanitizing k8s object process and test. Fixes: kubeflow#332 (kubeflow#345) add a warning label to kustomize page (kubeflow#350) make user guide prominent (kubeflow#349) feat(backend): ARTIFACT_COPY_STEP_TEMPLATE - custom template for copy step (kubeflow#336) Add KFP Tekton release deployment yaml (kubeflow#347) Fix UI merge conflict that blocks the UI image build. (kubeflow#346) KFP 1.0.4 Rebase (kubeflow#337) fix(sdk): Quick fix for sanitizing conditional variables (kubeflow#344) fix(backend): cover nil status race condition from Tekton (kubeflow#329) updated titanic-ml generated with kale 0.5.1 (kubeflow#330) Add s3 endpoint instructions for admin use case (kubeflow#323) Add best practice for tekton and other argo executors (kubeflow#338)

added sanitizing k8s object process and test

101a722

k8s-ci-robot requested review from ckadner and Tomcli October 27, 2020 01:28

k8s-ci-robot added the size/L label Oct 27, 2020

jizg changed the title ~~feat(sdk): added sanitizing k8s object process and test. Fixes: #322~~ feat(sdk): added sanitizing k8s object process and test. Fixes: #332 Oct 27, 2020

fix lint error by adding blank line at last

3b9e870

fix lint error by adding blank line at last

7c8a253

jizg added 2 commits October 27, 2020 20:28

fix lint error remove not used import

ef00a95

fix lint errors

36f8c7e

k8s-ci-robot assigned Tomcli Oct 30, 2020

k8s-ci-robot added the lgtm label Oct 30, 2020

drewbutlerbb4 reviewed Oct 30, 2020

View reviewed changes

removed unnecessary _to_str() and use str() directly

1a13407

k8s-ci-robot removed the lgtm label Oct 31, 2020

jizg requested a review from drewbutlerbb4 October 31, 2020 01:34

k8s-ci-robot assigned drewbutlerbb4 Oct 31, 2020

k8s-ci-robot added the lgtm label Oct 31, 2020

k8s-ci-robot added the approved label Nov 2, 2020

k8s-ci-robot merged commit 3d3ecfb into kubeflow:master Nov 2, 2020

animeshsingh mentioned this pull request Nov 9, 2020

Making Andrew Butler a committer in KFP-Tekton project #356

Merged

ckadner mentioned this pull request Nov 9, 2020

Update docs for KFP 1.0.4 and prepare for release 0.4.0 #357

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(sdk): added sanitizing k8s object process and test. Fixes: #332 #345

feat(sdk): added sanitizing k8s object process and test. Fixes: #332 #345

jizg commented Oct 27, 2020

google-cla bot commented Oct 27, 2020

kubeflow-bot commented Oct 27, 2020

jizg commented Oct 27, 2020

google-cla bot commented Oct 27, 2020

jizg commented Oct 27, 2020 •

edited

google-cla bot commented Oct 27, 2020

google-cla bot commented Oct 27, 2020

google-cla bot commented Oct 27, 2020

drewbutlerbb4 commented Oct 27, 2020

Tomcli commented Oct 27, 2020

jizg commented Oct 29, 2020

jizg commented Oct 30, 2020

google-cla bot commented Oct 30, 2020

jizg commented Oct 30, 2020

jizg commented Oct 30, 2020

Tomcli commented Oct 30, 2020

drewbutlerbb4 Oct 30, 2020

jizg Oct 31, 2020 •

edited

drewbutlerbb4 commented Oct 31, 2020

Tomcli commented Nov 2, 2020

k8s-ci-robot commented Nov 2, 2020

feat(sdk): added sanitizing k8s object process and test. Fixes: #332 #345

feat(sdk): added sanitizing k8s object process and test. Fixes: #332 #345

Conversation

jizg commented Oct 27, 2020

google-cla bot commented Oct 27, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

kubeflow-bot commented Oct 27, 2020

jizg commented Oct 27, 2020

google-cla bot commented Oct 27, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

jizg commented Oct 27, 2020 • edited

google-cla bot commented Oct 27, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

google-cla bot commented Oct 27, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

google-cla bot commented Oct 27, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

drewbutlerbb4 commented Oct 27, 2020

Tomcli commented Oct 27, 2020

jizg commented Oct 29, 2020

jizg commented Oct 30, 2020

google-cla bot commented Oct 30, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

jizg commented Oct 30, 2020

jizg commented Oct 30, 2020

Tomcli commented Oct 30, 2020

drewbutlerbb4 Oct 30, 2020

Choose a reason for hiding this comment

jizg Oct 31, 2020 • edited

Choose a reason for hiding this comment

drewbutlerbb4 commented Oct 31, 2020

Tomcli commented Nov 2, 2020

k8s-ci-robot commented Nov 2, 2020

jizg commented Oct 27, 2020 •

edited

jizg Oct 31, 2020 •

edited