Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes the concurrency issue of calling the Next() proxy target on RRB #2409

Merged
merged 4 commits into from Feb 24, 2023

Conversation

bbasic
Copy link
Contributor

@bbasic bbasic commented Feb 23, 2023

Round Robin Balancer (RRB) Next() implementation did not properly use synchronization mechanisms to ensure right values are visible between concurrently executed code entering the same critical section.

Previous use of an atomic add to update the index value without also using an atomic load to read it is incorrect use of atomic synchronization (stale values are read in go-routines).

The index value, obviously, depends on the size of the targets slice.
If between index calculation and getting a value by index from the slice a target was removed and the index pointed to the last element then a panic due to out of bounds will be the result.

Hence, the logic must be guarded with the semaphore.

@codecov
Copy link

codecov bot commented Feb 23, 2023

Codecov Report

Base: 92.84% // Head: 92.87% // Increases project coverage by +0.02% 🎉

Coverage data is based on head (3164518) compared to base (47844c9).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2409      +/-   ##
==========================================
+ Coverage   92.84%   92.87%   +0.02%     
==========================================
  Files          39       39              
  Lines        4503     4519      +16     
==========================================
+ Hits         4181     4197      +16     
  Misses        234      234              
  Partials       88       88              
Impacted Files Coverage Δ
middleware/proxy.go 70.98% <100.00%> (+2.32%) ⬆️
middleware/recover.go 83.01% <0.00%> (+1.38%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@aldas
Copy link
Contributor

aldas commented Feb 23, 2023

@bbasic thanks. would it be too much to ask to tweak some other things there also?

  1. wider mutex scope in (b *commonBalancer) AddTarge method
  2. probably b.random = rand.New(rand.NewSource(int64(time.Now().Nanosecond()))) could be initialized in func NewRandomBalancer

- fixed concurrency issue in `AddTarget()`
- moved `rand.New()` to the random balancer initializer func.
- internal code reorganized eliminating unnecessary pointer redirection
- employing `sync.Mutex` instead of `RWMutex` which brings additional overhead of tracking readers and writers. No need for that since the guarded code has no long-running operations, hence no realistic congestion.
- added additional guards without which the code would otherwise panic (e.g., the case where a random value is calculation when targets list is empty)
- added descriptions for func return values, what to expect in which case.
@bbasic
Copy link
Contributor Author

bbasic commented Feb 24, 2023

Hi @aldas
I went over the code and did few more changes. Please review.

Copy link
Contributor

@aldas aldas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@aldas
Copy link
Contributor

aldas commented Feb 24, 2023

and thank you for doing the other parts too!

@aldas aldas merged commit 5b36ce3 into labstack:master Feb 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants