[add] Support namespace scoped chaos #872

STRRL · 2020-09-06T05:58:55Z

What problem does this PR solve?

Make chaos-mesh controller could work without permission of cluster scope, like pod list from cluster scope.

Related issue: #487

What is changed and how does it work?

This PR makes three things:

Add new configurations to switch on namespace scope mode, and target namespace for chaos injecting within namespace scope mode.
Make controller manager only watch chaos objects belongs to target namespace; Make pod selector only list pods belongs to target namespace;
Create configuration validation and other logic to reject out-of-scoped behaviors.

Checklist

Todos:

kubebuilder controller manager support namespace scoped mode
K8sConfigMapWatcher support namespace scoped mode
helm configs
~~- [ ] enhance webhook (like, using namespaceSelector)~~
controller manager could skip chaos that does not belong to target namespace.
~~- [ ] e2e tests~~ It's hard to test as expected operation is NOOP.
new rbac yaml, as a mininum permission list, for controller serviceaccount
found the mininum permission list to install chaos-mesh
update the document

Tests

Unit test
E2E test
Manual test (add detailed scripts or steps below)
No code

Manually test steps:

WIP

Side effects

Breaking backward compatibility

Related changes

Need to update the documentation

Does this PR introduce a user-facing change?

No

NONE

… mode Signed-off-by: STRRL <str_ruiling@outlook.com>

…cope Signed-off-by: STRRL <str_ruiling@outlook.com>

cwen0 · 2020-09-07T03:19:07Z

@Yisaer PTAL

cmd/controller-manager/main.go

STRRL · 2020-09-07T11:47:41Z

I found that K8sConfigMapWatcher also need some patches, I will finish these changes very soon.

STRRL · 2020-09-07T12:12:02Z

Hi @cwen0 @Gallardot mentors, one more thing, shall we consider to split out namespace which inject chaos and namespace which running controller?

For example, running chaos-controller-manager in ns chaos-control-plane, and create a IOChaos in another ns chaos-testing.

Or we should play in ONLY ONE namespace.

cmd/controller-manager/main.go

pkg/utils/selector.go

Yisaer · 2020-09-08T07:45:36Z

Basically LGTM except the selector is too complex for now. Maybe we need to find a better way to refactor it. Adding an e2e test to verify the namespace-scoped would be better. @STRRL

STRRL · 2020-09-08T07:52:15Z

Hi @Yisaer , could you take a little while on this #872 (comment) ? Please give me your opinion about this question.

Thanks!

…ped mode or cluster scoped Signed-off-by: STRRL <str_ruiling@outlook.com>

…actions; Signed-off-by: STRRL <str_ruiling@outlook.com>

… and injecting chaos Signed-off-by: STRRL <str_ruiling@outlook.com>

STRRL · 2020-09-08T13:17:37Z

Hi @cwen0 @Gallardot mentors, one more thing, shall we consider to split out namespace which inject chaos and namespace which running controller?

For example, running chaos-controller-manager in ns chaos-control-plane, and create a IOChaos in another ns chaos-testing.

Or we should play in ONLY ONE namespace.

I am working on this, it could improve isolation of our system.

…e-scoped-mode

STRRL · 2020-09-09T04:31:41Z

Basically LGTM except the selector is too complex for now. Maybe we need to find a better way to refactor it.

Hi @Yisaer . I create new issue to track this. #886

I do not recommend refactoring while appending new features. :D

Yisaer · 2020-09-09T12:28:45Z

Basically LGTM except the selector is too complex for now. Maybe we need to find a better way to refactor it.

Hi @Yisaer . I create new issue to track this. #886

I do not recommend refactoring while appending new features. :D

Thanks for your contribution. I didn't mean refactoring selector in this request either. This should be tracked by a new issue and still need to be discussed.

Yisaer

I suggest using namespaceSelector for webhook while namespace-scoped is enabled.

As namespaceSelector need apiserver >= 1.15(need to be verified), the if-else template rendering could refer to here

helm/chaos-mesh/templates/controller-manager-deployment.yaml

helm/chaos-mesh/values.yaml

Yisaer · 2020-09-09T12:52:38Z

ning chaos-controller-manager in ns chaos-control-plane, and create a IOChaos in another ns chaos-testing.

Or we should play in ONLY ONE name

For this question, my answer is no (do not split the manager and chaos in different namespaces). The origin demand for this feature is to limit the range for the chaos mesh (namespace scoped is common for the most controllers while it is more important for chaos-mesh comparing to other controllers as we do need to control the effected range by chaos-mesh)

And this question also remind me that the manager's servceiaccount should be bound by role instead of clusterRole. The role is the only way to guarantee that the manager won't affect other namespaces.

STRRL · 2020-09-09T13:23:40Z

Hi @Yisaer mentor, I still do not figure out why. 😅

Deploying chaos-controller-manager and applications which are the target of chaos-testing into SAME namespace is DANGEROUS.

For example, if user using a bad selector, it might inject chaos into controller itself.

Other operators like prometheus-operator and openkruise have their own namespace which only running controller. I am very sure that your will NOT put your application into kube-prometheus orkube-system (doge.

And this question also remind me that the manager's servceiaccount should be bound by role instead of clusterRole. The role is the only way to guarantee that the manager won't affect other namespaces.

role and clusterRole is not the key, binding is the key. We could create a RoleBinding related to a ClusterRole. It is the way to grant a serviceaccount cross-namespace permissions.

It seems that we have opposite opinions, I am very pleasure to discuss this question on wechat or slack. :D

Yisaer · 2020-09-09T14:29:41Z

RoleBinding related to a ClusterRole is great! Thanks for pointing it out! @STRRL

In this way, deploying chaos-mesh into its own namespace and affecting other limit namespaces is ok to me.

Signed-off-by: STRRL <str_ruiling@outlook.com>

…e-scoped-mode

STRRL · 2020-09-25T05:17:47Z

HI, @Yisaer , PTAL

cwen0 · 2020-09-25T05:57:48Z

@Gallardot @Yisaer PTAL

Signed-off-by: STRRL <str_ruiling@outlook.com>

cwen0

LGTM

cwen0 · 2020-09-25T07:43:59Z

/merge

ti-srebot · 2020-09-25T07:46:06Z

/run-all-tests

ti-srebot · 2020-09-25T07:49:25Z

cherry pick to release-1.0 failed

Signed-off-by: STRRL <str_ruiling@outlook.com>

ti-srebot added the contribution label Sep 6, 2020

STRRL added 2 commits September 6, 2020 14:00

feat(controller-manager): add two configurations for namespace scoped…

db1eb17

… mode Signed-off-by: STRRL <str_ruiling@outlook.com>

improvement(controller-manager): configuration validation for cluterS…

66dafdc

…cope Signed-off-by: STRRL <str_ruiling@outlook.com>

STRRL force-pushed the faeature-namespace-scoped-mode branch from 21bf986 to 980cb96 Compare September 6, 2020 06:01

cwen0 requested a review from Yisaer September 7, 2020 03:18

Gallardot requested changes Sep 7, 2020

View reviewed changes

cmd/controller-manager/main.go Show resolved Hide resolved

STRRL marked this pull request as draft September 7, 2020 12:30

Yisaer reviewed Sep 8, 2020

View reviewed changes

cmd/controller-manager/main.go Outdated Show resolved Hide resolved

cmd/controller-manager/main.go Outdated Show resolved Hide resolved

pkg/utils/selector.go Show resolved Hide resolved

STRRL added 3 commits September 8, 2020 20:43

improvement(helm): configuration for CLUSTER_SCOPED for namespace sco…

d56f935

…ped mode or cluster scoped Signed-off-by: STRRL <str_ruiling@outlook.com>

improvement(controller-manager): prevent out of namespace scope list …

91c68e3

…actions; Signed-off-by: STRRL <str_ruiling@outlook.com>

feat(controller-manager): split out namespaces for running controller…

ad37f8a

… and injecting chaos Signed-off-by: STRRL <str_ruiling@outlook.com>

STRRL force-pushed the faeature-namespace-scoped-mode branch from 980cb96 to ad37f8a Compare September 8, 2020 15:03

Merge remote-tracking branch 'upstream/master' into faeature-namespac…

aca96db

…e-scoped-mode

STRRL force-pushed the faeature-namespace-scoped-mode branch 2 times, most recently from 0cb5f2d to a446991 Compare September 8, 2020 17:12

Yisaer reviewed Sep 9, 2020

View reviewed changes

helm/chaos-mesh/templates/controller-manager-deployment.yaml Show resolved Hide resolved

helm/chaos-mesh/values.yaml Show resolved Hide resolved

STRRL force-pushed the faeature-namespace-scoped-mode branch 2 times, most recently from d4dc2dc to 9d1349e Compare September 10, 2020 16:52

fix: fix broken tests

cdce6a1

Signed-off-by: STRRL <str_ruiling@outlook.com>

STRRL force-pushed the faeature-namespace-scoped-mode branch from 73e2092 to cccb5e9 Compare September 25, 2020 04:30

STRRL mentioned this pull request Sep 25, 2020

Namespace-scoped chaos mode should support multiple namespaces. #966

Closed

improvement: table-driven style test cases;

3af5f80

Signed-off-by: STRRL <str_ruiling@outlook.com>

STRRL force-pushed the faeature-namespace-scoped-mode branch from cccb5e9 to 3af5f80 Compare September 25, 2020 04:35

Merge remote-tracking branch 'upstream/master' into faeature-namespac…

c0f61c4

…e-scoped-mode

STRRL requested a review from Yisaer September 25, 2020 05:17

improvement: use table-driven style testcases

18b1a4a

Signed-off-by: STRRL <str_ruiling@outlook.com>

Gallardot approved these changes Sep 25, 2020

View reviewed changes

cwen0 approved these changes Sep 25, 2020

View reviewed changes

ti-srebot removed the status/LGT1 label Sep 25, 2020

ti-srebot approved these changes Sep 25, 2020

View reviewed changes

ti-srebot added the status/LGT2 label Sep 25, 2020

ti-srebot added the status/can-merge label Sep 25, 2020

Merge branch 'master' into faeature-namespace-scoped-mode

164fb4d

ti-srebot merged commit 212da2d into chaos-mesh:master Sep 25, 2020

STRRL added a commit to STRRL/chaos-mesh that referenced this pull request Sep 25, 2020

[add] Support namespace scoped chaos (chaos-mesh#872)

352493c

STRRL added a commit to STRRL/chaos-mesh that referenced this pull request Sep 25, 2020

[add] Support namespace scoped chaos (chaos-mesh#872)

1cbd328

Signed-off-by: STRRL <str_ruiling@outlook.com>

STRRL added a commit to STRRL/chaos-mesh that referenced this pull request Sep 25, 2020

[add] Support namespace scoped chaos (chaos-mesh#872)

cdfaee1

Signed-off-by: STRRL <str_ruiling@outlook.com>

STRRL added a commit to STRRL/chaos-mesh that referenced this pull request Sep 25, 2020

[add] Support namespace scoped chaos (chaos-mesh#872)

898cd6d

Signed-off-by: STRRL <str_ruiling@outlook.com>

cwen0 pushed a commit that referenced this pull request Sep 25, 2020

[add] Support namespace scoped chaos (#872) (#971)

7badda7

Signed-off-by: STRRL <str_ruiling@outlook.com>

Yiyiyimu mentioned this pull request Sep 26, 2020

website: change needed in Develop a New Chaos #985

Closed

Yiyiyimu mentioned this pull request Oct 12, 2020

fix bug in 'develop a new chaos' in website #1011

Merged

4 tasks

dcalvin mentioned this pull request Feb 5, 2021

promote committers #1492

Merged

STRRL deleted the faeature-namespace-scoped-mode branch February 23, 2021 06:18

sjwsl pushed a commit to sjwsl/chaos-mesh that referenced this pull request May 6, 2021

[add] Support namespace scoped chaos (chaos-mesh#872)

99397f6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[add] Support namespace scoped chaos #872

[add] Support namespace scoped chaos #872

STRRL commented Sep 6, 2020 •

edited

Loading

cwen0 commented Sep 7, 2020

STRRL commented Sep 7, 2020

STRRL commented Sep 7, 2020 •

edited

Loading

Yisaer commented Sep 8, 2020

STRRL commented Sep 8, 2020

STRRL commented Sep 8, 2020

STRRL commented Sep 9, 2020

Yisaer commented Sep 9, 2020

Yisaer left a comment

Yisaer commented Sep 9, 2020 •

edited

Loading

STRRL commented Sep 9, 2020

Yisaer commented Sep 9, 2020 •

edited

Loading

STRRL commented Sep 25, 2020

cwen0 commented Sep 25, 2020

cwen0 left a comment

cwen0 commented Sep 25, 2020

ti-srebot commented Sep 25, 2020

ti-srebot commented Sep 25, 2020

[add] Support namespace scoped chaos #872

[add] Support namespace scoped chaos #872

Conversation

STRRL commented Sep 6, 2020 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Checklist

Does this PR introduce a user-facing change?

cwen0 commented Sep 7, 2020

STRRL commented Sep 7, 2020

STRRL commented Sep 7, 2020 • edited Loading

Yisaer commented Sep 8, 2020

STRRL commented Sep 8, 2020

STRRL commented Sep 8, 2020

STRRL commented Sep 9, 2020

Yisaer commented Sep 9, 2020

Yisaer left a comment

Choose a reason for hiding this comment

Yisaer commented Sep 9, 2020 • edited Loading

STRRL commented Sep 9, 2020

Yisaer commented Sep 9, 2020 • edited Loading

STRRL commented Sep 25, 2020

cwen0 commented Sep 25, 2020

cwen0 left a comment

Choose a reason for hiding this comment

cwen0 commented Sep 25, 2020

ti-srebot commented Sep 25, 2020

ti-srebot commented Sep 25, 2020

STRRL commented Sep 6, 2020 •

edited

Loading

STRRL commented Sep 7, 2020 •

edited

Loading

Yisaer commented Sep 9, 2020 •

edited

Loading

Yisaer commented Sep 9, 2020 •

edited

Loading