fix panic when under pressure #3098
Conversation
schoren
force-pushed
the
fix-pgueue-worker-leak
branch
from
84783f6
to
52bc8cf
Compare
| } | ||
|
|
||
| const enqueueTimeout = 500 * time.Millisecond | ||
| const enqueueTimeout = 5 * time.Minute |
There was a problem hiding this comment.
This line owes me two days of life and one year of therapy. 🙃
| subscribers := append(m.getSubscribers(resourceID), subscriber) | ||
| m.setSubscribers(resourceID, subscribers) |
There was a problem hiding this comment.
Don't you need to lock the mutex here to prevent read-write race conditions here? Go will panic in case of read-write concurrency: https://stackoverflow.com/questions/36167200/how-safe-are-golang-maps-for-concurrent-read-write-operations
Same for PublishUpdate
Edit: reproduced the issue here (you might have to run it multiple times to see it fail): https://go.dev/play/p/h_AIcsi2J-Z
fatal error: concurrent map read and map write
There was a problem hiding this comment.
Originally each function that needed to access the map would manage the lock itself. That code was causing issues for some reason, so I moved the usage of the lock to the getSubscribers function and the setSubscribers function, so effectively every read and write is lock protected
This PR fixes the performace issues detected by running k6 tests
Changes
Fixes
Checklist