Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(limiter): simplify the memory represent of limiter configuration #10591

Merged
merged 3 commits into from May 5, 2023

Conversation

lafirest
Copy link
Member

@lafirest lafirest commented May 4, 2023

Fixes EMQX-9777

  1. Simplify the memory representation of the limiter configuration

Before this PR, the representation of the default limiter config in a listener is:

#{bytes => #{burst => 0.0,initial => 0,rate => infinity},
  client =>
      #{bytes =>
            #{burst => 0.0,divisible => false,failure_strategy => force,
              initial => 0,low_watermark => 0,max_retry_time => 10000,
              rate => infinity},
        connection =>
            #{burst => 0.0,divisible => false,failure_strategy => force,
              initial => 0,low_watermark => 0,max_retry_time => 10000,
              rate => infinity},
        message_routing =>
            #{burst => 0.0,divisible => false,failure_strategy => force,
              initial => 0,low_watermark => 0,max_retry_time => 10000,
              rate => infinity},
        messages =>
            #{burst => 0.0,divisible => false,failure_strategy => force,
              initial => 0,low_watermark => 0,max_retry_time => 10000,
              rate => infinity}},
  connection => #{burst => 0.0,initial => 0,rate => 100.0},
  message_routing =>
      #{burst => 0.0,initial => 0,rate => infinity},
  messages => #{burst => 0.0,initial => 0,rate => infinity}}

After this PR, there will not be any default limiter configuration representation in the memory

  1. Use a container's first slot to store root/node level limiter tokens, so the root/node level can really work when the limiter configuration of a listener is omitted
  2. Return the most suitable instance for different settings

Summary

馃 Generated by Copilot at d0a7d64

This pull request refactors the emqx_limiter module and its related modules to use a shared root bucket for each limiter type, simplify the limiter configuration and schema, and improve the readability and testability of the code. It also renames some types and functions to avoid confusion and updates the test suite accordingly.

PR Checklist

Please convert it to a draft if any of the following conditions are not met. Reviewers may skip over until all the items are checked:

  • Added tests for the changes
  • Changed lines covered in coverage report
  • Change log has been added to changes/{ce,ee}/(feat|perf|fix)-<PR-id>.en.md files
  • For internal contributor: there is a jira ticket to track this change
  • If there should be document changes, a PR to emqx-docs.git is sent, or a jira ticket is created to follow up
  • Schema changes are backward compatible

Checklist for CI (.github/workflows) changes

  • If changed package build workflow, pass this action (manual trigger)
  • Change log has been added to changes/ dir for user-facing artifacts update

@lafirest lafirest requested a review from a team as a code owner May 4, 2023 07:38
@lafirest lafirest force-pushed the fix/simplify_limiter_client_cfg branch from d0a7d64 to c0ce727 Compare May 4, 2023 07:48
@lafirest lafirest marked this pull request as draft May 4, 2023 08:23
@lafirest lafirest marked this pull request as ready for review May 4, 2023 09:34
@lafirest lafirest force-pushed the fix/simplify_limiter_client_cfg branch from cf95257 to 9324b5d Compare May 4, 2023 10:07
@coveralls
Copy link
Collaborator

Pull Request Test Coverage Report for Build 4881534127

  • 52 of 56 (92.86%) changed or added relevant lines in 4 files are covered.
  • 42 unchanged lines in 15 files lost coverage.
  • Overall coverage decreased (-0.006%) to 81.612%

Changes Missing Coverage Covered Lines Changed/Added Lines %
apps/emqx/src/emqx_limiter/src/emqx_limiter_manager.erl 2 3 66.67%
apps/emqx/src/emqx_limiter/src/emqx_limiter_schema.erl 9 10 90.0%
apps/emqx/src/emqx_limiter/src/emqx_limiter_server.erl 40 42 95.24%
Files with Coverage Reduction New Missed Lines %
apps/emqx_authn/src/simple_authn/emqx_authn_http.erl 1 88.39%
apps/emqx_gateway_mqttsn/src/emqx_mqttsn_channel.erl 1 73.13%
apps/emqx_gateway/src/emqx_gateway_schema.erl 1 88.89%
apps/emqx/src/emqx_broker.erl 1 87.64%
apps/emqx/src/emqx_cm.erl 1 91.44%
apps/emqx/src/emqx_limiter/src/emqx_limiter_schema.erl 1 87.5%
apps/emqx/src/emqx_limiter/src/emqx_limiter_server.erl 1 90.91%
apps/emqx/src/emqx_alarm.erl 2 88.24%
apps/emqx/src/emqx_crl_cache.erl 2 92.31%
apps/emqx/src/emqx_limiter/src/emqx_limiter_bucket_ref.erl 2 76.47%
Totals Coverage Status
Change from base Build 4880440794: -0.006%
Covered Lines: 26106
Relevant Lines: 31988

馃挍 - Coveralls

changes/ce/perf-10591.en.md Outdated Show resolved Hide resolved
apps/emqx/src/emqx_limiter/src/emqx_limiter_schema.erl Outdated Show resolved Hide resolved
@lafirest lafirest force-pushed the fix/simplify_limiter_client_cfg branch from 9324b5d to 7a96a97 Compare May 5, 2023 08:27
@lafirest lafirest merged commit 335d948 into emqx:master May 5, 2023
141 of 146 checks passed
@lafirest lafirest deleted the fix/simplify_limiter_client_cfg branch May 5, 2023 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants