Feat: component replication #4449

chivalryq · 2022-07-25T09:56:16Z

Description of your change

Fixes #4326
Added a replication policy. When using deploy workflow step, it could replicate components with keys. e.g.

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: app-replication-policy
spec:
  components:
    - name: hello-rep
      type: replica-webservice
      properties:
        image: crccheck/hello-world
      traits:
        - type: expose
          properties:
            port: [8000]
    - name: hello-no-rep
      type: webservice
      properties:
        image: crccheck/hello-world
      traits:
        - type: expose
          properties:
            port: [8001]

  policies:
    - name: comp-to-replicate
      type: override
      properties:
        selector: [ "hello-rep" ]
    - name: comp-not-replicate
      type: override
      properties:
        selector: [ "hello-no-rep" ]
    - name: target-default
      type: topology
      properties:
        clusters: [ "local" ]
    - name: replication-default
      type: replication
      properties:
        keys: ["beijing","hangzhou"]
        selector: ["hello-rep"]

  workflow:
    steps:
      - name: deploy-with-rep
        type: deploy
        properties:
          policies: ["comp-to-replicate","target-default","replication-default"]
      - name: deploy-without-rep
        type: deploy
        properties:
          policies: ["comp-not-replicate","target-default"]

Apply Order of Policies

Now, override, topology and replication can work with deploy workflow step. If they are used together, the apply order is:

topology -->  override -->  replication

Replication Spec

replication policy can NOT be used individually. Like topology, it need override to select target components to deploy.
The selector field of replication policy is used to choose which components to replicate. For example, you can specify three components with override policy and only replicate one of them. "comp2" and "comp3" will be dispatched without any replication

  policies:
    - name: select-three
      type: override
      properties:
        selector: [ "comp1", "comp2", "comp3" ]
    - name: replication-default
      type: replication
      properties:
        keys: [ "beijing","hangzhou" ]
        selector: [ "comp1" ]

Special Definition

Given that there aren't too many replication use case now. One want to use replication policy should modify the Component or Trait definition. A new context key replicaKey is provided when render the components. Here is an replica-webservice example in test/e2e-test/testdata/definition/replica-webservice.yaml

Fixes #

I have:

Read and followed KubeVela's contribution process.
Related Docs updated properly. In a new feature or configuration option, an update to the documentation is necessary.
Run make reviewable to ensure this PR is ready for review.
Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

Special notes for your reviewer

codecov · 2022-07-25T10:04:53Z

Codecov Report

Merging #4449 (6ef09ca) into master (6a9064b) will increase coverage by 0.15%.
The diff coverage is 93.22%.

@@            Coverage Diff             @@
##           master    #4449      +/-   ##
==========================================
+ Coverage   61.58%   61.74%   +0.15%     
==========================================
  Files         348      352       +4     
  Lines       34425    34773     +348     
==========================================
+ Hits        21201    21470     +269     
- Misses      10467    10501      +34     
- Partials     2757     2802      +45

Flag	Coverage Δ
apiserver-e2etests	`28.07% <2.04%> (+0.43%)`	⬆️
apiserver-unittests	`40.35% <ø> (+0.09%)`	⬆️
core-unittests	`56.41% <83.05%> (-0.11%)`	⬇️
e2e-multicluster-test	`20.17% <42.37%> (+0.41%)`	⬆️
e2e-rollout-tests	`23.27% <27.11%> (+0.33%)`	⬆️
e2etests	`30.25% <77.96%> (+0.85%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
apis/core.oam.dev/common/types.go	`79.31% <ø> (ø)`
apis/core.oam.dev/v1alpha1/policy_types.go	`100.00% <ø> (ø)`
pkg/appfile/appfile.go	`78.99% <ø> (+1.01%)`	⬆️
pkg/appfile/parser.go	`58.98% <ø> (-0.86%)`	⬇️
pkg/workflow/providers/oam/apply.go	`68.64% <ø> (+3.09%)`	⬆️
pkg/workflow/providers/multicluster/deploy.go	`78.72% <33.33%> (+1.19%)`	⬆️
pkg/policy/replication.go	`94.28% <94.28%> (ø)`
...ler/core.oam.dev/v1alpha2/application/generator.go	`87.93% <100.00%> (+0.65%)`	⬆️
...ller/core.oam.dev/v1alpha2/application/revision.go	`72.54% <100.00%> (+0.15%)`	⬆️
pkg/cue/definition/template.go	`67.64% <100.00%> (+4.84%)`	⬆️
... and 108 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Add e2e test Signed-off-by: qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

Signed-off-by: qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

wonderflow · 2022-07-25T13:49:00Z

pkg/oam/util/helper.go

+	if replicaKey == "" {
+		return componentName
+	}
+	return fmt.Sprintf("%s-%s", componentName, replicaKey)


what if a user define workload name in cue template?

It should be verified if the old style will overwrite the workload name as well.

apis/core.oam.dev/v1alpha1/policy_types.go

pkg/workflow/providers/oam/apply.go

pkg/workflow/providers/multicluster/deploy_test.go

pkg/policy/replication.go

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

apis/core.oam.dev/common/types.go

apis/core.oam.dev/v1alpha1/policy_types.go

pkg/oam/labels.go

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

Somefive

Great Job! LGTM

wonderflow · 2022-08-15T11:11:20Z

apis/core.oam.dev/v1alpha1/policy_types.go

+type ReplicationPolicySpec struct {
+	Keys []string `json:"keys,omitempty"`
+	// Selector is the subset of selected components which will be replicated.
+	Selector []string `json:"selector,omitempty"`


does the selector can only specify components?

Currently yes. It works similar to the selector field in override policy. We can make further extension in the future.

Yes, what else? Trait?

Maybe other component selection ways, like selecting components through regex?

But it's a string, it lacks flexible?

Yes, it lacks flexibility. The consideration here is:

It aligns with the design of override policy, although override policy also lacks flexibility, but it is easy to understand.

It satisfy the scenario we currently encountered by far.

We could have more extension ways in the future. For now although it would be nice to make it more extensible, I do not have too much good points due to the lack of more advanced feature requests.

Somefive · 2022-08-15T11:20:52Z

FYI, @chivalryq , a CUE file might need to be added to provide OpenAPI Schema for VelaUX and auto-generate documentations for kubevela.io. We can do it in another PR.

wonderflow · 2022-08-15T11:21:20Z

pkg/workflow/providers/multicluster/deploy.go

@@ -82,6 +84,10 @@ func (executor *deployWorkflowStepExecutor) Deploy(ctx context.Context, policyNa
 	if err != nil {
 		return false, "", err
 	}
+	components, err = pkgpolicy.ReplicateComponents(policies, components)


what will happen if the replicaComponent don't use the replicaKey in the definition template?

If the definition template relies on replicaKey field, an error will be thrown for the incorrect use of that component/trait.

If the definition template use the replicaKey field and has a fallback logic for handling the lack of replicaKey, it will be fine to use the definition with or without replicaKey.

If the definition template does not use the replicaKey, nothing will be happened while using the replicaKey. (The component will be repeated dispatched but there is no side effect.)

Then it depends on definitions. In definition it can check if context.replicaKey is nil. Just like I did in replica-webservice.yaml. But it's not recommended to mix component designed for replication and common ones.

what if one component use the replica key in the definition, but it's used in a fixed workload name? It will apply many times with the last config? What about next time? Does it cause any infinitely loop?

It is a problem. We can optimize it later. Like scanning the definition used when replicate.

Yes, it will apply many times but will not cause infinite loop. Replication does not mutate configurations. Multiple apply will be the same.

chivalryq · 2022-08-15T11:22:53Z

FYI, @chivalryq , a CUE file might need to be added to provide OpenAPI Schema for VelaUX and auto-generate documentations for kubevela.io. We can do it in another PR.

Got it.

test/e2e-test/application_test.go

wonderflow

please also add definition cue file with example markdown file (and some necessary explanation of the usage.)

It's really tricky when use it in definition.

Signed-off-by: qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

chivalryq · 2022-08-17T03:29:16Z

ping @wonderflow definition cue and example md added.

wonderflow

great job

chivalryq requested review from leejanee, FogDong, Somefive, barnettZQG and wonderflow as code owners July 25, 2022 09:56

chivalryq changed the title ~~Feature: component replication~~ Feat: component replication Jul 25, 2022

wonderflow added the backport release-1.5 add this label will automatically backport this PR to release-1.5 branch label Jul 25, 2022

chivalryq force-pushed the replication branch from 4a08186 to 813b329 Compare July 25, 2022 11:02

chivalryq added 2 commits July 25, 2022 20:26

Feat: component replication

309a341

Add e2e test Signed-off-by: qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

header

42fa723

Signed-off-by: qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

chivalryq force-pushed the replication branch from 813b329 to 42fa723 Compare July 25, 2022 12:26

wonderflow reviewed Jul 25, 2022

View reviewed changes

Somefive reviewed Jul 25, 2022

View reviewed changes

apis/core.oam.dev/v1alpha1/policy_types.go Outdated Show resolved Hide resolved

pkg/workflow/providers/oam/apply.go Show resolved Hide resolved

pkg/workflow/providers/multicluster/deploy_test.go Outdated Show resolved Hide resolved

charlie0129 reviewed Jul 26, 2022

View reviewed changes

pkg/policy/replication.go Show resolved Hide resolved

charlie0129 mentioned this pull request Jul 27, 2022

Self-nominate @charlie0129 as approver kubevela/community#58

Closed

Somefive removed the backport release-1.5 add this label will automatically backport this PR to release-1.5 branch label Jul 29, 2022

chivalryq marked this pull request as draft August 12, 2022 06:29

add replicaKey in ApplicationComponent, rollback some logic

170c184

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

chivalryq marked this pull request as ready for review August 14, 2022 16:40

chivalryq requested a review from jefree-cat as a code owner August 14, 2022 16:40

fix test

d4c2b7e

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

chivalryq force-pushed the replication branch from ac0bddd to d4c2b7e Compare August 15, 2022 07:34

Somefive reviewed Aug 15, 2022

View reviewed changes

apis/core.oam.dev/common/types.go Show resolved Hide resolved

Somefive reviewed Aug 15, 2022

View reviewed changes

apis/core.oam.dev/v1alpha1/policy_types.go Show resolved Hide resolved

Somefive reviewed Aug 15, 2022

View reviewed changes

pkg/oam/labels.go Outdated Show resolved Hide resolved

chivalryq added 2 commits August 15, 2022 17:49

Comments and labels

f9b8140

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

comments

5124fa8

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

Somefive approved these changes Aug 15, 2022

View reviewed changes

wonderflow reviewed Aug 15, 2022

View reviewed changes

test/e2e-test/application_test.go Show resolved Hide resolved

wonderflow reviewed Aug 15, 2022

View reviewed changes

chivalryq added 4 commits August 15, 2022 23:29

cue file

1381dab

Signed-off-by: qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

add doc

6b5e5a1

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

move doc

faf85c8

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

fix e2e test

6ef09ca

Signed-off-by: Qiaozp <qiaozhongpei.qzp@alibaba-inc.com>

wonderflow approved these changes Aug 17, 2022

View reviewed changes

chivalryq merged commit 924d553 into kubevela:master Aug 17, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: component replication #4449

Feat: component replication #4449

chivalryq commented Jul 25, 2022 •

edited

Loading

codecov bot commented Jul 25, 2022 •

edited

Loading

wonderflow Jul 25, 2022

chivalryq Jul 26, 2022

Somefive left a comment

wonderflow Aug 15, 2022

Somefive Aug 15, 2022

chivalryq Aug 15, 2022

Somefive Aug 15, 2022

wonderflow Aug 15, 2022

Somefive Aug 15, 2022

Somefive commented Aug 15, 2022

wonderflow Aug 15, 2022

Somefive Aug 15, 2022

chivalryq Aug 15, 2022 •

edited

Loading

wonderflow Aug 15, 2022

chivalryq Aug 15, 2022

Somefive Aug 15, 2022

chivalryq commented Aug 15, 2022

wonderflow left a comment

chivalryq commented Aug 17, 2022

wonderflow left a comment

Feat: component replication #4449

Feat: component replication #4449

Conversation

chivalryq commented Jul 25, 2022 • edited Loading

Description of your change

Apply Order of Policies

Replication Spec

Special Definition

How has this code been tested

Special notes for your reviewer

codecov bot commented Jul 25, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Somefive left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Somefive commented Aug 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chivalryq Aug 15, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chivalryq commented Aug 15, 2022

wonderflow left a comment

Choose a reason for hiding this comment

chivalryq commented Aug 17, 2022

wonderflow left a comment

Choose a reason for hiding this comment

chivalryq commented Jul 25, 2022 •

edited

Loading

codecov bot commented Jul 25, 2022 •

edited

Loading

chivalryq Aug 15, 2022 •

edited

Loading