Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: switch global ratelimit config provider to xds-grpc-sotw based server #1416

Merged
merged 18 commits into from
May 26, 2023

Conversation

shawnh2
Copy link
Contributor

@shawnh2 shawnh2 commented May 11, 2023

What type of PR is this?

feat(global-ratelimit): switch config provider to xds-grpc-sotw based

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #1254

Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
@arkodg
Copy link
Contributor

arkodg commented May 11, 2023

that was quick @shawnh2 ! will take another look when its out of draft

Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
@shawnh2 shawnh2 marked this pull request as ready for review May 13, 2023 08:46
@shawnh2 shawnh2 requested a review from a team as a code owner May 13, 2023 08:46
@codecov
Copy link

codecov bot commented May 13, 2023

Codecov Report

Merging #1416 (13996bf) into main (5dc86b8) will decrease coverage by 0.07%.
The diff coverage is 88.70%.

@@            Coverage Diff             @@
##             main    #1416      +/-   ##
==========================================
- Coverage   61.98%   61.92%   -0.07%     
==========================================
  Files          79       79              
  Lines       11388    11354      -34     
==========================================
- Hits         7059     7031      -28     
+ Misses       3870     3862       -8     
- Partials      459      461       +2     
Impacted Files Coverage Δ
internal/cmd/server.go 15.75% <ø> (ø)
...tructure/kubernetes/ratelimit/resource_provider.go 95.87% <0.00%> (-2.37%) ⬇️
internal/ir/zz_generated.deepcopy.go 12.68% <0.00%> (+0.26%) ⬆️
internal/xds/server/runner/runner.go 30.09% <0.00%> (ø)
internal/xds/translator/ratelimit.go 92.27% <97.43%> (ø)
...ternal/infrastructure/kubernetes/infra_resource.go 81.05% <100.00%> (-2.65%) ⬇️
...al/infrastructure/kubernetes/ratelimit/resource.go 94.85% <100.00%> (-0.26%) ⬇️

... and 2 files with indirect coverage changes

Copy link
Contributor

@zirain zirain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, defer to @arkodg

Copy link
Member

@Xunzhuo Xunzhuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits, generally LGTM.

internal/globalratelimit/runner/runner.go Outdated Show resolved Hide resolved
internal/globalratelimit/runner/runner.go Outdated Show resolved Hide resolved
internal/globalratelimit/runner/runner.go Show resolved Hide resolved
},
{
Name: ConfigGrpcXdsServerURL,
Value: net.JoinHostPort(XdsGrpcSotwConfigServerHost, strconv.Itoa(XdsGrpcSotwConfigServerPort)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check if XdsGrpcSotwConfigServerPort is a valid port ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right now the DefaultXdsServerPort does not have a check either.

// DefaultXdsServerPort is the default listening port of the xds-server.
DefaultXdsServerPort = 18000

maybe we can make these two ports configureable, and done as a follow-up?

Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
@zirain
Copy link
Contributor

zirain commented May 19, 2023

@shawnh2 I think this will speed up ratelimit e2e test, can you try to change following configuration, let's see what will happen.

time.Sleep(60 * time.Second)

Name: listener.Name,
Config: str,
}
ratelimitInfra.ServiceConfigs = append(ratelimitInfra.ServiceConfigs, c)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

curious why is the service config needed now that we are using xds ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because for ratelimitInfra, we only use it to update the configMap envoy-ratelimit:

for _, config := range r.infra.ServiceConfigs {
data[config.Name] = config.Config
}

that configMap envoy-ratelimit is used when we run ratelimit provider in FILE mode.

like the discuss above, someone may want to override the XDS mode to FILE manually. so if people does that, we can still make sure that ratelimit provider is working.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo we should remove all the configmap logic, its more maintenance burden for this project to support multiple config methods. afaik there is no use case to support both, users would like to configure ratelimiting and are not concerned with how ratelimiting is being implemented

Copy link
Contributor Author

@shawnh2 shawnh2 May 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed. I have removed the configmap logic, so only XDS mode is supported.

In the meantime, I've change the structure of ratelimitInfra, to make it simpler. we still need ratelimitInfra to publish the changes to ratelimit infra, so the svc and deployment resources will be created or deleted.

@arkodg
Copy link
Contributor

arkodg commented May 22, 2023

PR looks good ! just added some minor comments

@shawnh2
Copy link
Contributor Author

shawnh2 commented May 23, 2023

@shawnh2 I think this will speed up ratelimit e2e test, can you try to change following configuration, let's see what will happen.

time.Sleep(60 * time.Second)

@zirain hi, I shrink the sleep time down to 30s now, to see how it goes.

i notice that there're still some TODO in ratelimit e2e test, i can help with that as follow-up.

// TODO: find a better to make sure ratelimit load new configuration

Xunzhuo and others added 3 commits May 24, 2023 10:01
Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
Signed-off-by: Shawnh2 <shawnhxh@outlook.com>
Copy link
Member

@qicz qicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great job

@zirain zirain merged commit 1b565db into envoyproxy:main May 26, 2023
18 checks passed
@shawnh2 shawnh2 deleted the rls-switch-provider branch May 26, 2023 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ratelimit: switch to XdsGrpcSotwProvider
5 participants