Ensure context isn't exhausted via concurrent query as opposed to sentinel query #3334

gnpaone · 2025-04-05T13:56:19Z

Switch from sequential sentinel query to concurrent query to avoid context exhaustion and connect to new master redis after redis failover

gnpaone · 2025-04-05T13:57:09Z

Hey @ndyakov tried to fix your review points in #3174

ndyakov

Please add a test , it may be a setup of three sentinels, two of which are unreachable. Define the context in such a way, to get context.DeadlineExceeded error on GetMasterAddrByName. Such setup should fail prior to your PR and succeeded with the introduced change, correct?

sentinel.go

…onditions

kwenzh · 2025-04-10T12:06:31Z

Please add a test , it may be a setup of three sentinels, two of which are unreachable. Define the context in such a way, to get context.DeadlineExceeded error on GetMasterAddrByName. Such setup should fail prior to your PR and succeeded with the introduced change, correct?

The current problem is that the faulty sentinel node is in the front, which will cause context exhaustion. Even if the last sentinel can correctly provide the master address, don't we consider this situation?

kwenzh · 2025-04-11T03:01:40Z

I think that once any sentinel returns the master address correctly, it should be considered finished and other coroutines should be notified of the end. Even if the context timeout occurs afterwards, we should return the master address that has been queried, right?

kwenzh · 2025-04-11T03:26:50Z

I think that once any sentinel returns the master address correctly, it should be considered finished and other coroutines should be notified of the end. Even if the context timeout occurs afterwards, we should return the master address that has been queried, right?

with issue： #3172

gnpaone · 2025-04-11T04:10:35Z

I think that once any sentinel returns the master address correctly, it should be considered finished and other coroutines should be notified of the end. Even if the context timeout occurs afterwards, we should return the master address that has been queried, right?

with issue： #3172

Maybe should I use something like

ctx, cancel := context.WithCancel(ctx)
defer cancel()

...

cancel()

to stop goroutines early, once success is achieved ie sentinel returns the master address correctly?

kwenzh · 2025-04-11T08:06:09Z

once success is achieved ie sentinel returns the master address correctly?

yeah，once success returns the master address correctly，That's exactly what I meant！

gnpaone · 2025-04-13T12:03:35Z

Done

ndyakov · 2025-04-15T13:18:32Z

@gnpaone can you please add the issue in the description of this PR. I will review the latest changes later today and get back to you if there are any comments.

gnpaone · 2025-04-16T11:25:52Z

Updated the description @ndyakov

…tinel query (#3334)

This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) | require | minor | `v9.7.3` -> `v9.8.0` | --- ### Release Notes <details> <summary>redis/go-redis (github.com/redis/go-redis/v9)</summary> ### [`v9.8.0`](https://github.com/redis/go-redis/releases/tag/v9.8.0) [Compare Source](redis/go-redis@v9.7.3...v9.8.0) ### 9.8.0 (2025-04-30) #### 🚀 Highlights - **Redis 8 Support**: Full compatibility with Redis 8.0, including testing and CI integration - **Enhanced Hash Operations**: Added support for new hash commands (`HGETDEL`, `HGETEX`, `HSETEX`) and `HSTRLEN` command - **Search Improvements**: Enabled Search DIALECT 2 by default and added `CountOnly` argument for `FT.Search` #### ✨ New Features - Added support for new hash commands: `HGETDEL`, `HGETEX`, `HSETEX` ([#3305](redis/go-redis#3305)) - Added `HSTRLEN` command for hash operations ([#2843](redis/go-redis#2843)) - Added `Do` method for raw query by single connection from `pool.Conn()` ([#3182](redis/go-redis#3182)) - Prevent false-positive marshaling by treating zero time.Time as empty in isEmptyValue ([#3273](redis/go-redis#3273)) - Added FailoverClusterClient support for Universal client ([#2794](redis/go-redis#2794)) - Added support for cluster mode with `IsClusterMode` config parameter ([#3255](redis/go-redis#3255)) - Added client name support in `HELLO` RESP handshake ([#3294](redis/go-redis#3294)) - **Enabled Search DIALECT 2 by default** ([#3213](redis/go-redis#3213)) - Added read-only option for failover configurations ([#3281](redis/go-redis#3281)) - Added `CountOnly` argument for `FT.Search` to use `LIMIT 0 0` ([#3338](redis/go-redis#3338)) - Added `DB` option support in `NewFailoverClusterClient` ([#3342](redis/go-redis#3342)) - Added `nil` check for the options when creating a client ([#3363](redis/go-redis#3363)) #### 🐛 Bug Fixes - Fixed `PubSub` concurrency safety issues ([#3360](redis/go-redis#3360)) - Fixed panic caused when argument is `nil` ([#3353](redis/go-redis#3353)) - Improved error handling when fetching master node from sentinels ([#3349](redis/go-redis#3349)) - Fixed connection pool timeout issues and increased retries ([#3298](redis/go-redis#3298)) - Fixed context cancellation error leading to connection spikes on Primary instances ([#3190](redis/go-redis#3190)) - Fixed RedisCluster client to consider `MASTERDOWN` a retriable error ([#3164](redis/go-redis#3164)) - Fixed tracing to show complete commands instead of truncated versions ([#3290](redis/go-redis#3290)) - Fixed OpenTelemetry instrumentation to prevent multiple span reporting ([#3168](redis/go-redis#3168)) - Fixed `FT.Search` Limit argument and added `CountOnly` argument for limit 0 0 ([#3338](redis/go-redis#3338)) - Fixed missing command in interface ([#3344](redis/go-redis#3344)) - Fixed slot calculation for `COUNTKEYSINSLOT` command ([#3327](redis/go-redis#3327)) - Updated PubSub implementation with correct context ([#3329](redis/go-redis#3329)) #### 📚 Documentation - Added hash search examples ([#3357](redis/go-redis#3357)) - Fixed documentation comments ([#3351](redis/go-redis#3351)) - Added `CountOnly` search example ([#3345](redis/go-redis#3345)) - Added examples for list commands: `LLEN`, `LPOP`, `LPUSH`, `LRANGE`, `RPOP`, `RPUSH` ([#3234](redis/go-redis#3234)) - Added `SADD` and `SMEMBERS` command examples ([#3242](redis/go-redis#3242)) - Updated `README.md` to use Redis Discord guild ([#3331](redis/go-redis#3331)) - Updated `HExpire` command documentation ([#3355](redis/go-redis#3355)) - Featured OpenTelemetry instrumentation more prominently ([#3316](redis/go-redis#3316)) - Updated `README.md` with additional information ([#310ce55](redis/go-redis@310ce55)) #### ⚡ Performance and Reliability - Bound connection pool background dials to configured dial timeout ([#3089](redis/go-redis#3089)) - Ensured context isn't exhausted via concurrent query ([#3334](redis/go-redis#3334)) #### 🔧 Dependencies and Infrastructure - Updated testing image to Redis 8.0-RC2 ([#3361](redis/go-redis#3361)) - Enabled CI for Redis CE 8.0 ([#3274](redis/go-redis#3274)) - Updated various dependencies: - Bumped golangci/golangci-lint-action from 6.5.0 to 7.0.0 ([#3354](redis/go-redis#3354)) - Bumped rojopolis/spellcheck-github-actions ([#3336](redis/go-redis#3336)) - Bumped golang.org/x/net in example/otel ([#3308](redis/go-redis#3308)) - Migrated golangci-lint configuration to v2 format ([#3354](redis/go-redis#3354)) #### ⚠️ Breaking Changes - **Enabled Search DIALECT 2 by default** ([#3213](redis/go-redis#3213)) - Dropped RedisGears (Triggers and Functions) support ([#3321](redis/go-redis#3321)) - Dropped FT.PROFILE command that was never enabled ([#3323](redis/go-redis#3323)) #### 🔒 Security - Fixed network error handling on SETINFO (CVE-2025-29923) ([#3295](redis/go-redis#3295)) #### 🧪 Testing - Added integration tests for Redis 8 behavior changes in Redis Search ([#3337](redis/go-redis#3337)) - Added vector types INT8 and UINT8 tests ([#3299](redis/go-redis#3299)) - Added test codes for search_commands.go ([#3285](redis/go-redis#3285)) - Fixed example test sorting ([#3292](redis/go-redis#3292)) #### 👥 Contributors We would like to thank all the contributors who made this release possible: [@alexander-menshchikov](https://github.com/alexander-menshchikov), [@EXPEbdodla](https://github.com/EXPEbdodla), [@afti](https://github.com/afti), [@dmaier-redislabs](https://github.com/dmaier-redislabs), [@four_leaf_clover](https://github.com/four_leaf_clover), [@alohaglenn](https://github.com/alohaglenn), [@gh73962](https://github.com/gh73962), [@justinmir](https://github.com/justinmir), [@LINKIWI](https://github.com/LINKIWI), [@liushuangbill](https://github.com/liushuangbill), [@golang88](https://github.com/golang88), [@gnpaone](https://github.com/gnpaone), [@ndyakov](https://github.com/ndyakov), [@nikolaydubina](https://github.com/nikolaydubina), [@oleglacto](https://github.com/oleglacto), [@andy-stark-redis](https://github.com/andy-stark-redis), [@rodneyosodo](https://github.com/rodneyosodo), [@dependabot](https://github.com/dependabot), [@rfyiamcool](https://github.com/rfyiamcool), [@frankxjkuang](https://github.com/frankxjkuang), [@fukua95](https://github.com/fukua95), [@soleymani-milad](https://github.com/soleymani-milad), [@ofekshenawa](https://github.com/ofekshenawa), [@khasanovbi](https://github.com/khasanovbi) </details> --- ### Configuration 📅 **Schedule**: Branch creation - Between 12:00 AM and 03:59 AM ( * 0-3 * * * ) (UTC), Automerge - Between 12:00 AM and 03:59 AM ( * 0-3 * * * ) (UTC). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7739 Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org> Co-authored-by: Renovate Bot <forgejo-renovate-action@forgejo.org> Co-committed-by: Renovate Bot <forgejo-renovate-action@forgejo.org>

* Bump version to 9.8.0-beta1 Update README.md * Feature more prominently how to enable OpenTelemetry instrumentation (#3316) * Sync master with v9.8.0-beta.1 (#3322) * DOC-4464 examples for llen, lpop, lpush, lrange, rpop, and rpush (#3234) * DOC-4464 examples for llen, lpop, lpush, lrange, rpop, and rpush * DOC-4464 improved variable names --------- Co-authored-by: Vladyslav Vildanov <117659936+vladvildanov@users.noreply.github.com> Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com> * update pubsub.go (#3329) * use 8.0-RC1 (#3330) * drop ft.profile that was never enabled (#3323) * chore(deps): bump rojopolis/spellcheck-github-actions (#3336) Bumps [rojopolis/spellcheck-github-actions](https://github.com/rojopolis/spellcheck-github-actions) from 0.47.0 to 0.48.0. - [Release notes](https://github.com/rojopolis/spellcheck-github-actions/releases) - [Changelog](https://github.com/rojopolis/spellcheck-github-actions/blob/master/CHANGELOG.md) - [Commits](rojopolis/spellcheck-github-actions@0.47.0...0.48.0) --- updated-dependencies: - dependency-name: rojopolis/spellcheck-github-actions dependency-version: 0.48.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix FT.Search Limit argument and add CountOnly argument for limit 0 0 (#3338) * Fix Limit argument and add CountOnly argument * Add test and Documentation * Update search_commands.go --------- Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com> * fix add missing command in interface (#3344) * Use DB option in NewFailoverClusterClient (#3342) * DOC-5102 added CountOnly search example for docs (#3345) * Add integration tests for Redis 8 behavior changes in Redis Search (#3337) * Add integration tests for Redis 8 behavior changes in Redis Search * Undo changes in ft.search limit * Fix BM25 as the default scorer test * Add more tests and comments on deprecated params * Update search_commands.go * Remove deprication comment for nostopwords --------- Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com> * Use correct slot for COUNTKEYSINSLOT command (#3327) * Ensure context isn't exhausted via concurrent query as opposed to sentinel query (#3334) * fix: better error handling when fetching the master node from the sentinels (#3349) * Better error handling when fetching the master node from the sentinels * fix error message generation * close the errCh to not block * use len over errCh * docs: fix documentation comments (#3351) * DOC-5111 added hash search examples (#3357) * fix: Fix panic caused when arg is nil (#3353) * Update README.md, use redis discord guild (#3331) * use redis discord guild * add line in CONTRIBUTING.md * update with badges similar to rest of the libraries. update url * updated with direct invite link * fix discord link in CONTRIBUTING.md * fix stackoverflow tag --------- Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com> * update HExpire command documentation (#3355) * update HExpire command documentation * Apply suggestions from code review Format the links in the documentation. Add missing documentation. --------- Co-authored-by: Nedyalko Dyakov <1547186+ndyakov@users.noreply.github.com> * feat: func isEmptyValue support time.Time (#3273) * fix:func isEmptyValue support time.Time * fix: Improve HSet unit tests * feat: Improve HSet unit tests * fix: isEmptyValue Struct only support time.Time * test(hset): add empty custom struct test --------- Co-authored-by: Guo Hui <gh7396@gmail.com> Co-authored-by: Nedyalko Dyakov <nedyalko.dyakov@gmail.com> * fix: `PubSub` isn't concurrency-safe (#3360) * migrate golangci-lint config to v2 format (#3354) * migrate golangci-lint config to v2 format * chore: skip CI on migration [skip ci] * Bump golangci version * Address several golangci-lint/staticcheck warnings * change staticchecks settings * chore(ci): Use redis 8 rc2 image. (#3361) * chore(ci): Use redis 8 rc2 image * test(timeseries): fix duplicatePolicy check * feat(options): panic when options are nil (#3363) Client creation should panic when options are nil. * chore(release): Update version to v9.8.0 - update version in relevant places - add RELEASE-NOTES.md to keep track of release notes --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Nikolay Dubina <nikolay.dubina.pub@gmail.com> Co-authored-by: andy-stark-redis <164213578+andy-stark-redis@users.noreply.github.com> Co-authored-by: Vladyslav Vildanov <117659936+vladvildanov@users.noreply.github.com> Co-authored-by: Liu Shuang <liushuangbill@outlook.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: ofekshenawa <104765379+ofekshenawa@users.noreply.github.com> Co-authored-by: Bulat Khasanov <afti@yandex.ru> Co-authored-by: Naveen Prashanth <78990165+gnpaone@users.noreply.github.com> Co-authored-by: Glenn <alohaglenn@icloud.com> Co-authored-by: frankj <frankxjkuang@gmail.com> Co-authored-by: Elena Kolevska <elena-kolevska@users.noreply.github.com> Co-authored-by: Hui <gh73962@gmail.com> Co-authored-by: Guo Hui <gh7396@gmail.com> Co-authored-by: fukua95 <fukua95@gmail.com>

bug: ensure context isn't exhausted via concurrent query

6f22bf5

Update sentinel.go

6ec0cfd

ndyakov requested changes Apr 7, 2025

View reviewed changes

sentinel.go Outdated Show resolved Hide resolved

ndyakov and others added 3 commits April 7, 2025 13:46

Merge branch 'master' into patch-1

9b40ce4

Remove separate goroutine and just wait on the main WaitGroup

1601768

Need to immediately return in context canceled or deadline exceeded c…

84b2016

…onditions

Add test for proper sentinel resolution

d231fa0

Stop goroutines early after success is achieved

6d78214

gnpaone requested review from kwenzh and ndyakov April 13, 2025 12:03

gnpaone and others added 2 commits April 13, 2025 17:35

Update sentinel_test.go

bd27395

Merge branch 'master' into patch-1

7ab8412

Merge branch 'master' into patch-1

0f05272

Merge branch 'master' into patch-1

173d598

ndyakov approved these changes Apr 16, 2025

View reviewed changes

Merge branch 'master' into patch-1

67fe50d

ndyakov merged commit a4aea25 into redis:master Apr 16, 2025
16 checks passed

gnpaone deleted the patch-1 branch April 17, 2025 03:22

ndyakov pushed a commit that referenced this pull request Apr 30, 2025

Ensure context isn't exhausted via concurrent query as opposed to sen…

5067de7

…tinel query (#3334)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ensure context isn't exhausted via concurrent query as opposed to sentinel query #3334

Ensure context isn't exhausted via concurrent query as opposed to sentinel query #3334

gnpaone commented Apr 5, 2025 •

edited

Loading

gnpaone commented Apr 5, 2025

ndyakov left a comment

kwenzh commented Apr 10, 2025

kwenzh commented Apr 11, 2025

kwenzh commented Apr 11, 2025

gnpaone commented Apr 11, 2025

kwenzh commented Apr 11, 2025

gnpaone commented Apr 13, 2025

ndyakov commented Apr 15, 2025

gnpaone commented Apr 16, 2025

Ensure context isn't exhausted via concurrent query as opposed to sentinel query #3334

Ensure context isn't exhausted via concurrent query as opposed to sentinel query #3334

Conversation

gnpaone commented Apr 5, 2025 • edited Loading

gnpaone commented Apr 5, 2025

ndyakov left a comment

Choose a reason for hiding this comment

kwenzh commented Apr 10, 2025

kwenzh commented Apr 11, 2025

kwenzh commented Apr 11, 2025

gnpaone commented Apr 11, 2025

kwenzh commented Apr 11, 2025

gnpaone commented Apr 13, 2025

ndyakov commented Apr 15, 2025

gnpaone commented Apr 16, 2025

gnpaone commented Apr 5, 2025 •

edited

Loading