[feature] integrate datasource of OpenSergo by OpenSergo Go SDK #489

jnan806 · 2022-10-14T01:40:51Z

Does this pull request fix one issue?

fixed #483

Special notes for reviews

~~### Prepare Environment for Test Version~~

- Prepare the OpenSergo GO SDK.
Because of the OpenSergo GO SDK has no published version, so should download the sourcecode of OpenSergo GO SDKjnan806/opensergo-go-sdk/tree/initial-version, and move it into you GOPATH.
- Rename the right version in directory name of OpenSergo GO SDK sourcecode.
Make sure the version in sourcecode directory name is the same with go.mod.
eg. $GOPATH/pkg/mod/github.com/opensergo/opensergo-go@v0.0.0-20220331070310-e5b01fee4d1c

The jnan806/opensergo-go-sdk/tree/initial-version has been merged into opensergo/opensergo-go-sdk.
So this step of Prepare the OpenSergo GO SDK` is unnecessary.
It's only need to add use sentinel-go with the opensergo-datasource.

Samples

datasource_opensergo_example.go

pkg/datasource/opensergo/opensergo.go

pkg/datasource/opensergo/demo/datasource_opensergo_example.go

pkg/datasource/opensergo/opensergo.go

pkg/datasource/opensergo/demo/datasource_opensergo_example.go

pkg/datasource/opensergo/opensergo_rule_aggregator.go

pkg/datasource/opensergo/opensergo.go

pkg/datasource/opensergo/mixed_property_adaptor.go

pkg/datasource/opensergo/opensergo.go

sczyh30

And please update with the latest Go SDK commit (with the new model).

pkg/datasource/opensergo/demo/datasource_opensergo_example.go

sczyh30 · 2022-12-03T03:56:23Z

@binbin0325 @louyuting @luckyxiaoqiang Any further suggestions?

pkg/datasource/opensergo/opensergo_faulttolerance_rule_subscriber.go

pkg/datasource/opensergo/opensergo_flowrule_strategy_subscriber.go

pkg/datasource/opensergo/rule_assembler_aggregator.go

sczyh30 · 2022-12-26T09:38:30Z

We might need to discuss the model of the OpenSergo data-source soon (to make it clear).

sczyh30

数据源实现看一下是不是可以完全实现 Sentinel DataSource 接口，做到标准化；
去除目前数据源内部的增量变更逻辑，后续可以和 Sentinel Go 社区一起来看这个设计；
订阅/取消订阅相关的 API 遵循 Sentinel 的语义；
可以添加一下熔断规则的支持，看看目前的模型是否满足多种规则转换，及多种 strategy 转到同种 Sentinel 规则的需要。
update apply 时做好更新范围管理

pkg/datasource/opensergo/opensergo.go

…oDataSouce Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

…fd77bd6ce Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

sczyh30 · 2023-01-09T09:53:26Z

cc @binbin0325 for update

pkg/datasource/opensergo/opensergo_sentinel_assembler.go

Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

jnan806 · 2023-01-11T01:03:18Z

@sczyh30

Cloud you help me to check and complete the // TODO list in file fix#483-datasource-opensergo -> opensergo_sentinel_assembler.go to fill filed-mapping between pb-message and sentinel rule.
And then delete the // TODO comment line.

You can commit into the branch jnan806/sentinel-golang/fix#483-datasource-opensergo directly.

pkg/datasource/opensergo/opensergo.go

pkg/datasource/opensergo/opensergo_rule_aggregator.go

pkg/datasource/opensergo/opensergo_const.go

pkg/datasource/opensergo/opensergo_rule_aggregator.go

binbin0325 · 2023-01-11T02:54:56Z

pkg/datasource/opensergo/opensergo.go

+func (ds *OpenSergoDataSource) ReadSource() ([]byte, error) {
+	// assemble updated MixedRule
+	mixedRule := new(MixedRule)
+	if ds.opensergoRuleAggregator.mixedRuleCache.updateFlagMap[RuleType_FlowRule] {


感觉这个mixedRuleCache 可以去掉，在doUpdate func 中接收更新的RuleType参数，在ReadSource根据更新的RuleType 参数设置到mixedRule即可。

我是这么想的，mixedRule 就仅作为 rule 的一个集合，不带业务属性比较好。用另外的mixedRuleCache这个结构体来包含更新标志，这样结构体的功能区分会更加清晰。

Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

jnan806 · 2023-01-11T05:10:30Z

pkg/datasource/opensergo/opensergo_sentinel_assembler.go

+	// TODO fill field-mapping between sentinel-rule and pb-message
+	flowRule.Threshold = float64(pbStrategy.Threshold)
+	flowRule.TokenCalculateStrategy = flow.Direct
+	flowRule.ControlBehavior = flow.Reject
+
+	return flowRule


@binbin0325
Cloud you help me to check and complete the field-mapping
And then delete the // TODO comment line.

You can commit into the branch jnan806/sentinel-golang/fix#483-datasource-opensergo directly.
I have already invite you into this repo.

jnan806 · 2023-01-11T05:11:21Z

pkg/datasource/opensergo/opensergo_sentinel_assembler.go

+	// TODO fill field-mapping between sentinel-rule and pb-message
+	flowRule.Threshold = float64(1000 / pbStrategy.MinIntervalMillisOfRequests)
+	flowRule.TokenCalculateStrategy = flow.Direct
+	flowRule.ControlBehavior = flow.Throttling
+	flowRule.MaxQueueingTimeMs = uint32(pbStrategy.QueueTimeoutMillis)
+
+	return flowRule


@binbin0325 the same as above

jnan806 · 2023-01-11T05:11:33Z

pkg/datasource/opensergo/opensergo_sentinel_assembler.go

+	// TODO fill field-mapping between sentinel-rule and pb-message
+	isolationRule.Threshold = uint32(pbStrategy.MaxConcurrency)
+	isolationRule.MetricType = isolation.Concurrency
+
+	return isolationRule


@binbin0325 the same as above

jnan806 · 2023-01-11T05:11:47Z

pkg/datasource/opensergo/opensergo_sentinel_assembler.go

+	// TODO fill field-mapping between sentinel-rule and pb-message
+	switch pbStrategy.Strategy {
+	case faulttolerancePb.CircuitBreakerStrategy_STRATEGY_SLOW_REQUEST_RATIO:
+		circuitbreakerRule.Threshold = float64(pbStrategy.SlowCondition.MaxAllowedRtMillis)
+		break
+	case faulttolerancePb.CircuitBreakerStrategy_STRATEGY_ERROR_REQUEST_RATIO:
+		circuitbreakerRule.Threshold = pbStrategy.TriggerRatio
+		break
+	default:
+		logging.Info("[OpenSergoDatasource] unknow CircuitBreakerStrategy.", "resourceName", circuitbreakerRule.Resource, "pbStrategy", pbStrategy)
+	}
+	circuitbreakerRule.MinRequestAmount = uint64(pbStrategy.MinRequestAmount)
+
+	return circuitbreakerRule


@binbin0325 the same as above

sczyh30 · 2023-01-18T02:54:23Z

pkg/datasource/opensergo/opensergo.go

+	logging.Info("[OpenSergoDatasource] un-subscribing OpenSergo ConcurrencyLimitStrategy.", "namespace", ds.namespace, "app", ds.app)
+}
+
+func (ds *OpenSergoDataSource) subscribeCircuitBreakerRule() {


These subscribe/unsubscribe methods for Sentinel rules should be public.

And please pay attention to error handling. Do not ignore the error here.

These subscribe/unsubscribe methods for Sentinel rules should be public.

i make it private because i have no idea about how to avoid re-invoking this functions.
The subscribe functions have been invoked in Initialize(). If make them public , users would invoke twice, also unsubscribe twice will be invoked twice.

sczyh30 added kind/feature Category issues or PRs related to feature request to-review PRs to review area/integrations Issue related to integrations with open-source components area/data-source Issues or PRs related to data-source extension labels Oct 14, 2022

jnan806 force-pushed the fix#483-datasource-opensergo branch 2 times, most recently from 217aa0c to 9990e57 Compare October 17, 2022 05:49

sczyh30 reviewed Oct 17, 2022

View reviewed changes

pkg/datasource/opensergo/opensergo.go Outdated Show resolved Hide resolved

sczyh30 requested review from binbin0325 and louyuting October 17, 2022 05:56

jnan806 force-pushed the fix#483-datasource-opensergo branch from 9990e57 to 1dd4b2d Compare October 17, 2022 05:59

binbin0325 reviewed Oct 17, 2022

View reviewed changes

pkg/datasource/opensergo/demo/datasource_opensergo_example.go Outdated Show resolved Hide resolved

pkg/datasource/opensergo/opensergo.go Show resolved Hide resolved

binbin0325 reviewed Oct 17, 2022

View reviewed changes

jnan806 force-pushed the fix#483-datasource-opensergo branch 4 times, most recently from 176b9ad to 7bcc31e Compare October 18, 2022 00:55

jnan806 marked this pull request as draft October 18, 2022 13:39

binbin0325 reviewed Oct 19, 2022

View reviewed changes

jnan806 force-pushed the fix#483-datasource-opensergo branch from f929da7 to a347f50 Compare October 27, 2022 14:14

jnan806 marked this pull request as ready for review November 30, 2022 00:18

jnan806 force-pushed the fix#483-datasource-opensergo branch from 6bd449c to 3afd07c Compare November 30, 2022 00:55

sczyh30 mentioned this pull request Dec 1, 2022

Umbrella issue for data-plane integrations with OpenSergo Go SDK opensergo/opensergo-go-sdk#1

Open

4 tasks

sczyh30 reviewed Dec 3, 2022

View reviewed changes

pkg/datasource/opensergo/demo/datasource_opensergo_example.go Show resolved Hide resolved

sczyh30 reviewed Dec 26, 2022

View reviewed changes

sczyh30 requested changes Jan 4, 2023

View reviewed changes

pkg/datasource/opensergo/opensergo.go Show resolved Hide resolved

jnan806 and others added 3 commits January 5, 2023 09:30

feat: integrate datasource of OpenSergo by OpenSergo Go SDK

50063e6

add mixed rule_type property use to parse and handle rule in Openserg…

a183ac7

…oDataSouce Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

re-format the code

1fbf0e3

Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

opensergo-go-sdk bump to the latest commit: v0.0.0-20230105054255-fe6…

ee33705

…fd77bd6ce Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

jnan806 force-pushed the fix#483-datasource-opensergo branch 3 times, most recently from df4a242 to f28ec80 Compare January 5, 2023 08:02

sczyh30 requested changes Jan 9, 2023

View reviewed changes

pkg/datasource/opensergo/opensergo_sentinel_assembler.go Outdated Show resolved Hide resolved

pkg/datasource/opensergo/opensergo_sentinel_assembler.go Show resolved Hide resolved

jnan806 force-pushed the fix#483-datasource-opensergo branch from f28ec80 to 20320de Compare January 10, 2023 01:40

support more sentinel-rules from OpenSergo

1d99c71

Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

jnan806 force-pushed the fix#483-datasource-opensergo branch from 20320de to 1d99c71 Compare January 11, 2023 01:01

binbin0325 reviewed Jan 11, 2023

View reviewed changes

polish code

7c50c3f

Signed-off-by: Jiangnan Jia <jnan0806@gmail.com>

jnan806 commented Jan 11, 2023

View reviewed changes

sczyh30 requested changes Jan 18, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feature] integrate datasource of OpenSergo by OpenSergo Go SDK #489

[feature] integrate datasource of OpenSergo by OpenSergo Go SDK #489

jnan806 commented Oct 14, 2022 •

edited

sczyh30 left a comment

sczyh30 commented Dec 3, 2022

sczyh30 commented Dec 26, 2022

sczyh30 left a comment

sczyh30 commented Jan 9, 2023

jnan806 commented Jan 11, 2023

binbin0325 Jan 11, 2023

jnan806 Jan 11, 2023 •

edited

jnan806 Jan 11, 2023 •

edited

jnan806 Jan 11, 2023

jnan806 Jan 11, 2023

jnan806 Jan 11, 2023

sczyh30 Jan 18, 2023

sczyh30 Jan 18, 2023

jnan806 Jan 18, 2023

[feature] integrate datasource of OpenSergo by OpenSergo Go SDK #489

Are you sure you want to change the base?

[feature] integrate datasource of OpenSergo by OpenSergo Go SDK #489

Conversation

jnan806 commented Oct 14, 2022 • edited

Does this pull request fix one issue?

Special notes for reviews

Samples

sczyh30 left a comment

Choose a reason for hiding this comment

sczyh30 commented Dec 3, 2022

sczyh30 commented Dec 26, 2022

sczyh30 left a comment

Choose a reason for hiding this comment

sczyh30 commented Jan 9, 2023

jnan806 commented Jan 11, 2023

binbin0325 Jan 11, 2023

Choose a reason for hiding this comment

jnan806 Jan 11, 2023 • edited

Choose a reason for hiding this comment

jnan806 Jan 11, 2023 • edited

Choose a reason for hiding this comment

jnan806 Jan 11, 2023

Choose a reason for hiding this comment

jnan806 Jan 11, 2023

Choose a reason for hiding this comment

jnan806 Jan 11, 2023

Choose a reason for hiding this comment

sczyh30 Jan 18, 2023

Choose a reason for hiding this comment

sczyh30 Jan 18, 2023

Choose a reason for hiding this comment

jnan806 Jan 18, 2023

Choose a reason for hiding this comment

jnan806 commented Oct 14, 2022 •

edited

jnan806 Jan 11, 2023 •

edited

jnan806 Jan 11, 2023 •

edited