-
Notifications
You must be signed in to change notification settings - Fork 565
[MOD-10037] add default scorer in configuration #7065
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…eat-config-default-scorer
Co-authored-by: kei-nan <jonathan.keinan@redis.com>
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 2.8
git worktree add -d .worktree/backport-7065-to-2.8 origin/2.8
cd .worktree/backport-7065-to-2.8
git switch --create backport-7065-to-2.8
git cherry-pick -x b61c075a870e7b4252e2caf1edcbd6c1e6162354 |
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 2.6
git worktree add -d .worktree/backport-7065-to-2.6 origin/2.6
cd .worktree/backport-7065-to-2.6
git switch --create backport-7065-to-2.6
git cherry-pick -x b61c075a870e7b4252e2caf1edcbd6c1e6162354 |
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 2.10
git worktree add -d .worktree/backport-7065-to-2.10 origin/2.10
cd .worktree/backport-7065-to-2.10
git switch --create backport-7065-to-2.10
git cherry-pick -x b61c075a870e7b4252e2caf1edcbd6c1e6162354 |
|
Backport failed for Please cherry-pick the changes locally and resolve any conflicts. git fetch origin 8.2
git worktree add -d .worktree/backport-7065-to-8.2 origin/8.2
cd .worktree/backport-7065-to-8.2
git switch --create backport-7065-to-8.2
git cherry-pick -x b61c075a870e7b4252e2caf1edcbd6c1e6162354 |
* feat: add default scorer in configuration * fix and test config setters and getters * test: add flow tests checking scorer behavior * fix: fix as per cursor comments * fix: fix as per code review * fix: change the init config code * test: fix the tests * test: refactor tests * fix memory leak in getter * test: add explicit test BM25STD is the default since start * validate default scorer at module init * fix validation against extensions * apply changes as per review * Update tests/cpptests/test_cpp_forkgc.cpp * refactor small setter * try to fix sanitizer leak * test: Known leaks when onModuleLoad fails * Update src/module-init/module-init.c Co-authored-by: kei-nan <jonathan.keinan@redis.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> (cherry picked from commit b61c075)
* feat: add default scorer in configuration * fix and test config setters and getters * test: add flow tests checking scorer behavior * fix: fix as per cursor comments * fix: fix as per code review * fix: change the init config code * test: fix the tests * test: refactor tests * fix memory leak in getter * test: add explicit test BM25STD is the default since start * validate default scorer at module init * fix validation against extensions * apply changes as per review * Update tests/cpptests/test_cpp_forkgc.cpp * refactor small setter * try to fix sanitizer leak * test: Known leaks when onModuleLoad fails * Update src/module-init/module-init.c Co-authored-by: kei-nan <jonathan.keinan@redis.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> (cherry picked from commit b61c075)
* feat: add default scorer in configuration * fix and test config setters and getters * test: add flow tests checking scorer behavior * fix: fix as per cursor comments * fix: fix as per code review * fix: change the init config code * test: fix the tests * test: refactor tests * fix memory leak in getter * test: add explicit test BM25STD is the default since start * validate default scorer at module init * fix validation against extensions * apply changes as per review * Update tests/cpptests/test_cpp_forkgc.cpp * refactor small setter * try to fix sanitizer leak * test: Known leaks when onModuleLoad fails * Update src/module-init/module-init.c Co-authored-by: kei-nan <jonathan.keinan@redis.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> (cherry picked from commit b61c075)
* feat: add default scorer in configuration * fix and test config setters and getters * test: add flow tests checking scorer behavior * fix: fix as per cursor comments * fix: fix as per code review * fix: change the init config code * test: fix the tests * test: refactor tests * fix memory leak in getter * test: add explicit test BM25STD is the default since start * validate default scorer at module init * fix validation against extensions * apply changes as per review * Update tests/cpptests/test_cpp_forkgc.cpp * refactor small setter * try to fix sanitizer leak * test: Known leaks when onModuleLoad fails * Update src/module-init/module-init.c Co-authored-by: kei-nan <jonathan.keinan@redis.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> (cherry picked from commit b61c075)
* [MOD-11155] Create basic query pause debug command (#6750) * feat: add unified memory consumption checker for Redis * feat: implement unified memory consumption checker for Redis * refactor: move MIN_NOT_0 macro definition to redis_mem_info.c * fix: include minmax.h for MIN_NOT_0 macro definition * feat: add debug logging for memory usage check in ForkGC * feat: add QueryDebugCtx struct for debugging queries * fix: correct memory usage check and improve memory ratio calculation * fix: update debug command structures for improved clarity and functionality * reorder result_processor.h s.t Debug processors are at the end * reorder result_processor.c s.t Debug processors are at the end * fix: reorder ResultProcessorType enum values for clarity * fix: categorize debug-only result processors for better readability * add basic RP controller debug command * restructer parseAndCompile * basic query pause * remove condition as it might create a race * add RP before type function * remove problematic checks * Add basic pytests * Support insert debug rp before/after specific rp type * test insert debug rp and print rp stream * Add PRINT_RP_STREAM command to FT.DEBUG QUERY_CONTROLLER * Skip cluster test * fix error message * add active queries check * Don't forget the @ * cover invalid rp type * Remove redundant function declaration and add comment * fix TBD comment * change to return found * return error if RP is missing from stream, change to 1 return statement * add new options documentation to aggregate_debug.h * remove doube whitespace * space before var name * remove PAUSE_AFTER_N * change order of args in pause before/after * Test cluster * cluster test * confrim query in bg * Cover error when ran without workers * Guard against insert after Index RP * fixing edge case * fix leak * Apply suggestions from code review Co-authored-by: meiravgri <109056284+meiravgri@users.noreply.github.com> * Add error handling for resume command when no query is paused * Validate only 1 query is able to run at each time * comment * Add assertion in RPPauseAfterCount_Free to validate debugRP state before clearing * Enhance debug documentation for PAUSE_AFTER_RP_N and PAUSE_BEFORE_RP_N commands to clarify RP type restrictions and timeout behavior. * Remove unused field * add rp str to error msg * remvoe "IN" from error msg * fix test * Fix memory management in RPPauseAfterCount_Free by ensuring rm_free is called consistently. * Global debug ctx API * change expected doc number * Check active queries on all shards * Refactor test_query_controller_pause_and_resume to streamline index creation * Change mechanism in ft.aggregate in cluster * Test coord pause piepline * Enhance error handling for INTERNAL_ONLY usage in debug commands * Round 1 * Remove air conditionar * remove revach * cover invalid count * Move debug RPs to end of file, add debug pause RP (cherry picked from commit 583fee9) * Adjust to new context (cherry picked from commit ed69f6b) * remove duplicate * Re-add debug * remove not thread safe check --------- Co-authored-by: meiravgri <109056284+meiravgri@users.noreply.github.com> * remove readies from memcheck (#6868) * remove readies from memcheck * add colors * red * [MOD-11611] fix test_vecsim:TestTimeoutReached (#6904) * test: update KNN query to use large_k and remove redundant checks * revert unnecessary changes * MOD-10082 Rust Wildcard Iterator (#6846) * Rust Wildcard Iterator: WIP * tests * benchmarks * Avoid bench ffi overhead with Direct C impl * Remove overheaded-ffi tests from bench * Fix build * rename c bench file * PR: rename and fixes * PR: fix and remove redundant test * PR: provide benchmarks params as test inputs * PR: include inverted_index only for iterators bencher * PR: last comments * PR: set freq to 1 * PR: remove many redundant benchmarks * PR: scope wildcard iterator result lifetime * [MOD-11400] Add memory guardrail before AREQ_BuildPipeline (#6877) * Add guardrail * test failure on OOM * Refactor OOM tests to use common setup function * Update TODO comment for QUERY_EHYBRID_HYBRID_ALIAS in query_error.h * Apply guy suggestions (cherry picked from commit fd949e16466a627747cbfd7ab35cf497a3ed5110) * change goto error to return so no undeclared var issue * Add missing lines after feature branch merge (#6913) add missing lines after feature branch merge * [MOD-9916] Swap `Counter` result processor to rust (#6804) * fix typedef typedef <type> <name>; * swap to rust RPCounter, delete C code * port RPProfile count incrementing behavior * remove version * undo import formatting changes * address comments and (temporarily) fix tests - adds a `Context::parent` function instead of a `last_processor` - adds a `RPProfile_IncrementCount` mock function so that the rust expecting the C function to exist will still pass * cargo fmt --all * WIP * mock QueryProcessingCtx in Chain * comment * use Pin<Box<ffi::QueryProcessingCtx>> * comment * add unreachable!() in stub impl, simplify pointer cast * always set parent in each appended result_processor * fix miri error: UnsafeCell<*mut QueryProcessingCtx> * make fmt * put UnsafeCell inside QueryProcessingCtx * whoops * [MOD-11403] Add `revalidate` to the Rust inverted index reader (#6853) * Add GC marker to inverted index * Add GC marker to reader * FFI for reader revalidate * FFI for ii gc marker These will only be used by some C tests. * Take mutable parameter for updated variable * [MOD-11616] Fix SVS info test to properly handle Alpine platform compression fallback (#6914) * check if alpine * use distro.name().lower() * MOD-11172: Support WITHSCORES in Hybrid (#6858) * some bug fixes * add withscores test * remove print * fix tests * code review fixes * code review fix * [MOD-10076] - remove the FDs check in gc oom test (#6903) * add wait for gc * remove FDs check * add wait for gc * remove FDs check * Move `RSValue_SendReply` to `reply` module (#6803) * Move `RSValue_SendReply` to `reply` module Moves `RSValue_SendReply` to `reply` module, renaming it to `RedisModule_Reply_RSValue`. Furthermore, exposes `RSValue_NumToString` from `value.h`, as it's used in `RedisModule_Reply_RSValue`, adding a debug assertion that the pased `RSValue` is indeed a number, so as to make things safer. * Remove RSValue_SendReply_Collection Removes `RSValue_SendReply_Collection`, which was disabled, removing the last reference to the `reply` module from `value` * Clarify requirement of RSValue_NumToString parameter being a number * Use `RS_ASSERT` instead of `assert` in `RSValue_NumToString` Co-authored-by: GuyAv46 <guy.avimor@gmail.com> --------- Co-authored-by: GuyAv46 <guy.avimor@gmail.com> * MOD-11172: Require Param Count After RRF Keyword (#6902) * initial commit * code review fixes * fix compilation * code review comment fix + fix cpp tests * fix tests * fix tests * address code review comment * Fix The Unit Tests Script To Work With GDB And Compute BINDIR (#6852) * initial commit * check folders existence * revert some changes * improve the script * code review comments + group tests in github actions * code review fixes * code review fixes * code review fixes * Have Rust decode to a mut reference `RSIndexResult` (#6872) * Use mutable reference for decoding * Update tests * Update benchers * Update FFI * Fix safety comments * Flatten logic * Combine lifetimes on enum * Only decoding to return an new instance * Fix linting issue * Simplify return Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com> * Black box benchmark correctly --------- Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com> * MOD-11307 implement `RLookupRow::get` in Rust porting C function `RLookup_GetItem` (#6807) MOD-11307 implement RLookupRow::get Co-authored-by: Jonas Kruckenberg <iterpre@protonmail.com> * MOD-11043: FT.HYBRID - Support Score And Distance Aliasing (#6894) * try and support yield distance and score * support score alias in search sub query * fix compilation * add tests and fix issues they raised * fix cpp test * remove positional argument handling - doesn't fit the flow we are looking for * try and align branch to requested requirements * small changes to remove code which is not needed * code review comments * fix tests - add error code to arg parser to allow some error handling when combine id dealing with keywords while parsing combine keyword * code review comment - fix tests * fix test * revert YIELD_DISTANCE_AS support, move YIELD_SCORE_AS in combine to be part of RRF/LINEAR arg count * revert some changes * fix compilation * remove arg parse changes * revert some combine parsing code * change test names, revert some additional code * add error when failing to reserve alias * try and fix tests * fix some flow tests * fix flow test + code review comment * forgot to push a test fix * review comments * [MOD-10970] - Default Dialect for Hybrid commands should be at least 2 (#6885) * advance ac after finding the right definition * update the license for arg parser * if min and max are equal, use slice, otherwise use variable length args * code review comments * change how positional arguments are handled - we require the name to match and only then the value * update license * initial commit * update combine handling function * minor changes to parsing code - changes options types since some don't have count * fix options * post rebase fixes * post rebase fixes * fix CI * code review fixes * bit collision fix * fix tests * fix typo * changing constant arg definition to be a double * fix timeout parsing * remove temp variable * fix error messages * additional error fixes * free params dict on error * avoid double free * code review comments * remove constant range validation * improve error message, try to improve coverage * test: change test to reflect desired behavior * default dialect minimum 2 for Hybrid * test: fix unit test * fail if DIALECT is part of the query * do checks in different parts of query * fix DIALECT error messages * fix knn and range subqueries * test: fix tests failing because dialect overriden * update src/hybrid/parse_hybrid.c * apply suggestions from code review Co-authored-by: Omer Shadmi <76992134+oshadmi@users.noreply.github.com> * force subqueries same dialect Version as parent * test: fix tests failing in CI, no DIALECT in query * fix: fix according to review * Fix dialect stats, add test with EXPLAINSCORE * change the error message * test: fix test that was explicitly setting DIALECT --------- Co-authored-by: jonathan keinan <jonathan.keinan@redis.com> Co-authored-by: Omer Shadmi <76992134+oshadmi@users.noreply.github.com> Co-authored-by: oshadmi <omer.shadmi@redis.com> * [MOD-11646] - LIMIT and KNN parameters should behave independently (#6930) * LIMIT and KNN parameters should behave independently * test: fix tests showing default K independent from Limit * MOD-11090: Try marking DROP commands as touches-arbitrary-keys (#6795) * MOD-11090: Mark DROP commands as touches-arbitrary-keys * Fix spell check * changed compatible_redis_version to 8.0.0 * min_redis_version from 8.0.0 to 8.0 * change compatible_redis_version to 8.0.2 * change redis_compatible_version from 8.0.2 to 8.0 * Restore compatible_redis_version 99.99.99 * changed compatible_redis_version to 8.0 * changed redis_compatible_version to 8.2 * compatible_redis_version=8.0 * Restore compatible Redis version to 99.99.99 * Apply phrasing suggestion from @nafraf Co-authored-by: nafraf <nafraf@users.noreply.github.com> --------- Co-authored-by: galilev <galiwiner@gmail.com> Co-authored-by: nafraf <nafraf@gmail.com> Co-authored-by: nafraf <nafraf@users.noreply.github.com> * Inverted Index FFI preparation and tweaks (#6851) * Use getter for inverted index access * Only set numeric filter if not null * Numeric encoder can have duplicate entries * Raw doc id encoder should use first block id for base * Make helper inline static This is to get around multiple definitions compile errors. * Set wide schema flag on tested index These tests add records with a wide field mask but were not using the encoders which supports the wide field mask. So this fixes that. * Update include path * Have `NewIndexReader` take an owned decoder ctx * Remove calls to size of ii and index blocks These sizeof calls won't be possible once these are in Rust. * Remove access to block size constants These won't exist in the Rust move. I also don't know why the tests did not have these hardcoded in the first place. * Add comment to panic * Fix spelling * Fix tiny negatives numeric encoding These used to be incorrectly encoded as positive tiny numbers. * Write numeric records to a numeric index The Rust code will panic without this. * FFI for IndexBlock This is only needed by some C tests. * Fix comment Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com> --------- Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com> * MOD-10399 refactor: encapsulate SearchResult (#6808) * MOD-10399 encapsulate SearchResult type, removing direct field accesses * refactor: Make `indexResult` field of `SearchResult` immutable In sync with the work done in #6846 this change makes `SearchResult` hold onto an immutable point to the `RSIndexResult` instead of a mutable one. The change does the necessary (limited) cleanup to make this work. removes `SearchResult_GetIndexResultMut` * fix: crash revert SearchResult_Override * upgrade to macos15 (#6932) * upgrade to macos15 * try remove xlarge * print cpu information * fix print cpu * fix * [MOD-11188] apply window parameter in Hybrid linear commands (#6924) * apply window parameter in Hybrid linear commands * parse WINDOW when parsing Linear clause * test: test the behavior of windows setting w.r.t limit * test: add tests for parsing WINDOW in Linear * code review changes * test: fix tests * limit K by WINDOW also for Linear * update src/hybrid/parse_hybrid.c * update tests/cpptests/test_cpp_hybrid_defaults.cpp Co-authored-by: Itzikvaknin <82322982+Itzikvaknin@users.noreply.github.com> --------- Co-authored-by: Itzikvaknin <82322982+Itzikvaknin@users.noreply.github.com> * MOD-11416: Fix `total_result` Accuracy When Using FILTER (#6880) * fix total_result accuracy when using FILTER * correctly handle responses with errors * access total_result using recursive_index * code review fixes * another cursor code review comment fix * fix test * fix tests * code review comments * rebase + fix test * Sync tail RLookup with upstream RLookup after consuming results (#6937) * call RLookup_AddKeysFrom(sourceLookups[i], tailLookup) after consumiong that upstream * CR comment * CR comment * MOD-11660: Limit the size of the heap in search subquery SORTER (#6942) * apply window parameter in Hybrid linear commands * parse WINDOW when parsing Linear clause * test: test the behavior of windows setting w.r.t limit * test: add tests for parsing WINDOW in Linear * code review changes * test: fix tests * limit K by WINDOW also for Linear * update src/hybrid/parse_hybrid.c * update tests/cpptests/test_cpp_hybrid_defaults.cpp Co-authored-by: Itzikvaknin <82322982+Itzikvaknin@users.noreply.github.com> * propagate RRF and LINEAR window parameters to arrange step limit in search subquery. * Fix hybrid.py:_validate_results * Fix unit tests * Increase coverage --------- Co-authored-by: Joan Fontanals Martinez <jfontanalsmartinez@gmail.com> * chore: update hiredis (#6961) upgrade hiredis * [MOD-11587] (1/2) Refactor API of `value` C module (#6956) * Hide internals of `RSValue` Ensure code outside of the `value` C module does not access internals of the `RSValue` type, but instead uses several newly created constructors, setters, and getters. The `RS_DUOVAL_OTHER2VAL` is to be eliminated in a later commit that renames the `Duo` variant of `RSValueType` to `Trio`. `RSVALUE_MAP_KEYPOS` and `RSVALUE_MAP_VALUEPOS` will be eliminated in a redesign of the map creation functionality of `RSValue` * Rename `RSValue` `Duo` type to `Trio` In the past, `RSValue` has been adapted such that its `Duo` field actually holds three values. In preparation for the migration to Rust of the `value` module, this type is renamed to `Trio`, eliminating the `RS_DUOVAL_OTHER2VAL` macro in the process. * Refactor `RSValue` map API Add types `RSValueMap` and `RSValueMapEntry`, representing an `RSValue` map variant and its entries. Add functions `RSValueMap_Create_Uninit`, `RSValueMap_SetEntry`, and `RSValue_NewMap` to allow for creating maps and wrapping them in `RSValue`s. * Move `APIVERSION_RETURN_MULTI_CMP_FIRST` from `value.h` to `search_ctx.h` * Eliminate macros `RSVALUE_REPLACE` and `RSVALUE_CLEARVAR` * Rename `RSValue_Decref` to `RSValue_DecrRef` For consistency with `RSValue_IncrRef`, renames `RSValue_Decref` to `RSValue_DecrRef`. Also removes and updates outdated comments. * test: add tests for filter hybrid behavior (#6896) * test: add tests for filter hybrid behavior * test: improve testing * fix: fix test * test: test behavior of double filter pre and post * complete test with 2 filters no combine * Add 1 more test * test: clean comments * fix: fix problem after merge with master * change asserts --------- Co-authored-by: nafraf <nafraf@gmail.com> * Rust inverted index fixes (#6955) * Use correct flag to create tmp index Without this the tmp index created might not be the same as the original one and can therefore not be swapped for the original one to trigger a revalidation. * Fix index summary efficiency calculation The calculation should use all entries and not just the unique ones. * Allow empty ii reader * Defensive reset * [MOD-11610] tests: test total_results and results count when LIMIT applied with and without FILTER for FT.HYBRID (#6934) * test: add test showing num results are returned * test: add another piece to the test * test: use env.assertEqual * test: simplify * prove filter is not being applied properly * test: test also length of result list * fix: fix total_results value in Hybrid * test: add test combining filter and limit * fix: total_results represent matched documents, not affected by limit * test: remove useless test * [MOD-11587] (2/2) Further clean up and document the `value` C module (#6957) * Naming consistency in `value` - Rename variants of `RSValueType` and `RSStringType` so they are prefixed with their type name. - Rename constructors to clarify whether they allocate the newly created `RSValue` on the heap or not by suffixing heap-allocating constructors with `Alloc`, and constructors creating a static RSValue-backed instance with `Static`. Furthermore, ensure prefixing is consistent with other functions. * Reorder functions an definitions in `value` C module In `value.h` and `value.c`, the function definitions are ordered and grouped logically: constructors on top, getters/setters below that, and other functions and utilities at the bottom. * More naming consistency, add doc comments to constructors in `value` Add doc comments to the constructors defined in `value.h`, clarifying whether they are placed on the stack or on the heap and whether they should be freed or not. Apply consistent naming to `RSValue_ParseNumber`, `RSValue_NewArray`, `RSValue_VStringArray`, `RSValue_StringArray`, and `RSValue_StringArrayT` to clarify whether they allocate or not. * Rename constructors in `value` To enhance predictibility, the naming scheme of constructors defined in the `value` C module are renamed: - heap-allocating constructor names contain "New"`, the "Alloc" suffix is removed - stack-allocating constructors no longer contain "New" * Address review comments - Add doc comments to `RSValue_NewConstString`, and `RSValue_NewConstCString`. - Rename `RSValue_NewRedisString` to `RSValue_NewBorrowedRedisString` to clarify the difference with `RSValue_NewOwnedRedisString` - Have `convert_to_number` in `value.c` return a `bool` instead of `int` - Rename `RSValue_MallocString` to `RSValue_String` to clarify its similarities and differences with `RSValue_NewString` * MOD-11645: Support _NUM_SSTRING as arg of _FT.HYBRID (#6963) * Update parser to support _NUM_SSTRING as arg of _FT.HYBRID * Fix Test * Clear QueryError status * Test internal and external command * Split test * Uncomment test * Fix test and fix indentation * MOD-11654: FT.HYBRID - Merging Requires __key To Exists (#6941) * merge uses key to read doc key from rlookup - key is now mandatory in load if load is specified - must come from coordinator - fixed tests to pass key if they used LOAD * remove implicit loading step creation - try and get keyPtr from dmd - in single shard solution will work - in coordinator we will fallback to rlookup. * some error handling * code review comments * code review comments * more stuff to revert * fix typo * cursor code review comment * mark doc key as hidden * test: test hybrid filter additions (#6965) * [MOD-11233] - fix: empty string token should be counted in byteOffsets for highlighting (#6938) * initial commit * add newline at end of file * add debug prints * test: fix testing index being awaited * add test from the MOD-11233 * test: simplify tests to be easier to handle/debug * test: update test * test: improve test assertion * add test with some debug prints * fix: fix tokenization empty * remove useless field from tokenizer * fix: simpler fix * fix: improved behavior for forward index * fix spell check * change test name and var name * change test text * refactor: byteoffset handling independent * test: add more tests with highlighting and empty combination * apply changes as per code review --------- Co-authored-by: jonathan keinan <jonathan.keinan@redis.com> * MOD-10398 rlookup: port SearchResult (#6809) * MOD-10398 port SearchResult to Rust * apply review comments Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> --------- Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> * MOD-11277 implement `RLookupRow::write_key_by_name` porting C `RLookup_WriteKeyByName` function (#6839) * fix RLookupKey name lifetime issues Co-authored-by: Tim Janus <tim@janus.rs> * MOD-11277 implement `RLookupRow::write_key_by_name` porting C `RLookup_WriteKeyByName` function Co-authored-by: Tim Janus <tim@janus.rs> --------- Co-authored-by: Jonas Kruckenberg <iterpre@protonmail.com> * MOD-10721: Upload 99.99.99 beta version with unique suffix to beta subdir (#6979) * MOD-10721: Upload 99.99.99 version with unique suffix to beta subdir * Code review fix * Generate beta version in flow-build-artifacts (#6980) * Generate beta version per flow * Add force beta option * add vendor folder to .gitignore (#6994) add vendor folder to gitignore * Do not fanout FT.INFO command to replicas (#6960) The current behaviour of the coordinator is not correct, because FT.INFO command reports the number of docs as actual number of docs multiplied by number of replicas * [MOD-11410] Make QueryError Fields Private (#6962) * remove QueryError_Init, add QUERY_ERROR_DEFAULT c/c++ * make QueryError.code field private * use QueryError_HasError and QueryError_IsOk * make QueryError.message private * make QueryError.detail private, change two accesses There are two places in this commit where QueryError.detail were previously being accessed by consumers of the QueryError API. This probably shouldn't happen since I believe detail (and message) should only be read from and written to via the API. I'm not 100% sure though, and this should be carefully looked at when reviewed. * make QueryError.reachedMaxPrefixExpansions field private * replace brace initializations of query error with QUERY_ERROR_DEFAULT * change QUERY_ERROR_DEFAULT to a function * fix remaining references to status.code * fix memory leak on cloned query error detail string * fix upstream tests * Support multiple slot-ranges per shard - [MOD-11613] (#6898) * refactor MRCommand to hold a target node-id string instead of a target slot * minor improvement to redisStringsKeyDup of dict * refactor cluster topology to store array of ranges, to support multiple ranges per shard * refactor `SEARCH.CLUSTERREFRESH` to use `CLUSTER SHARDS` instead of `CLUSTER SLOTS` to generate the topology struct * refactor/fix `SEARCH.CLUSTERSET` to support multiple slot ranges per shard correctly * fix `SEARCH.CLUSTERINFO` to work with the new topology structs (no behavior changes, and still printing a single range per shard) * test fixes * minor fixes * fix test * revert using redis module strings to avoid fragile thread-safety rules * address build issues * ignore any port info and keep calling RedisModule_GetClusterNodeInfo to get it * revert and use port + tls-port of CLUSTER SHARDS * fix "SEARCH.CLUSTERINFO" to return an array of slot ranges (same behavior as CLUSTER SHARDS) * make the master node the first in the node's array of the shard * make the master node the first in the node's array of the shard - enterprise * fix tests * improve comment about reply structure * assert that the first node is the master * add first test * improve test * add basic cluster set test * small improvement * review fixes + remove multi-range support from `SEARCH.CLUSTERSET` * small style change * MOD-11752: Fix snapshot upload in rockylinux - use dnf instead of yum (#7011) * [MOD-11395] Handle shard returning OOM in Coord (#6921) * Handle OOM in Coordinator in FT.search and oom policy is fail * Handle OOM in Coordinator in FT.aggregate and oom policy is fail * Handle OOM in Coordinator when failure in shards * fix tests structure * remove static * remove pragma once from c file * Aggregate return partial * Prepare testing (cherry picked from commit 4f8cd0b) * Aggregate results if policy is fail * Draft for stable return cluster test * remove comments from test * Refactor threading helper function for consistency across tests * Add guardrail to ft.hybrid * test oom hybrid * Add missing module.h include in hybrid_exec.c * Move check to earlier * Free error reply before overriding in rpnetNext function * Enable ft.profile with resp2 * Enable ft.profile with resp3 * Reply warning in ft.profile and in RESP3 for search and aggregate * Profile AGG support * Fix empty warning output * add skip test * Optimize query error extraction * Improve error handling by passing the reply string as the error message in rpnetNext * remove !rows condition use env.con use TimeLimit remove unused function flatten loops Add assert on on ERR continue Add comment explaining why no ft.profile search in resp2 use number of keys instead of assertless remove unused variable * Make sure that agg uses request config * Reply error from shards in ft.profile * handle generic error * skip test on sanitizer * change to load @name * remove redundant else and change test skip * make first reply test more robust * Add wait for process * Adapt to new QueryError API * remove load from test + format and comments * [MOD-11756] - make sure all FT.SUG* are hashslot aware on Enterprise (#7016) fix: fix ramp enterprise so that all SUG* are hashslot aware * Fix command registration - [MOD-11766] (#7020) * register FT.ADD, FT.GET and FT.DEL with key * fix all test to correctly use ft.add ft.get and ft.del * revert test deletion * fix spellings * fix ACL and add a test * test SAFEADD as well * [MOD-11535] Define FFI crate and type definitions for rust port of `RSValue` (#7001) * Initial type layout for Rust `RsValue` and `SharedRsValue` - Move `RSValueTrait` and `RSValueFFI` in the `value` crate to a separate module, and put them behind a feature gate as the module depends on the `ffi` crate which conflicts with cbindgen when it attempts to generate bindings for the `value` crate. This feature is to be removed when `RsValueFFI` is eliminated in a later stage of the porting process. - Define `RsValue` and `SharedRsValue` types, both still incomplete and unoptimized, so that we can generate bindings referencing these types. `RsValue` is a stack-based type, whereas `SharedRsValue` is heap-allocated and reference counted. - Define `RsValueMap`, `RsValueTrio`, as well as `RsValueInternal` to represent the internals of `RsValue` and `SharedRsValue`. Signed-off-by: Henk Oordt <hd@oordt.dev> * Create `value_ffi` crate as the FFI specification for `value` Create `value_ffi` to mimic the C `value.h` interface. `RSValue_NewWithType`, `RSValue_WithType`, and `RSValue_NewStringWithType` are omitted intentionally, and are to be replaced with specialized constructors that fully initialized the produced value. Signed-off-by: Henk Oordt <hd@oordt.dev> --------- Signed-off-by: Henk Oordt <hd@oordt.dev> * Fix goto on RDB load (#7033) fix goto * fix bugs regarding misalignment and missing symbols (#7000) * fix bugs regarding misalignment and missing symbols - redis allocator mocks had misalignment issues - symbols were missing (e.g. RedisModule_Alloc) - ptrs vs objects This was causing issues with 'cargo test' on rqe_iterators_bencher on Linux, but it was all good on MacOS. * restore previous comments * fix license header * Add explanation about h1 * apply feedback from @LukeMathWalker * [MOD-10774] - test: add test validating crash from different timeout policies (#7017) * test: add test validating crash from different timeout policies * test: improve test as per comments * test: improve spped by using pipeline * handle smaller amount of docs * Update tests/pytests/test_cluster_aggregate_timeout.py Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com> --------- Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com> * test: fix issue FGC unit test (#7045) * ci: show weird test failure * fix: initialize keyspace notifciations when starting test * revert change on github * [MOD-11799] ci: add back MAX_WORKER_THREADS option to Makefile (#7039) * ci: add back MAX_WORKER_THREADS option to Makefile * ci: test in CMakeCache.txt * revert changes on actions * bump vecsim (#7052) * Update JSON LLAPI - [MOD-11606] (#6920) * preperations for the new version * remove any code path for JAPI<6 only * update API file * align with new API * some more cleanup * update JSON target branch when building * fix API usage * Apply API alignment * Merge feature-coordinator-hybrid branch to master (#7076) * WIP: Register Register DistHybridCommand() * WIP: Integrate hybrid_dispatcher * MOD-11172: Add Argument Parser Struct To Improve Parsing Experience (#6856) * advance ac after finding the right definition * update the license for arg parser * if min and max are equal, use slice, otherwise use variable length args * code review comments * change how positional arguments are handled - we require the name to match and only then the value * update license * Add dispatcher to rpnet * WIP: HybridRequest_BuildDistributedDepletionPipeline * WIP: Fixing HybridRequest_buildDistRPChain() * Intermediate work on coordinator FT.HYBRID * try to fix HybridRequest_buildMRCommand * Test HybridRequest_buildMRCommand() * Minor fixes * Intermediate work on dispatcher refactor * Update parseHybridCommand call to include internal arg * Intermediate work on dispatcher * Intermediate work on hybrid_cursor_mappings * initial commit * Intermediate work on RPnetNext in FT.HYBRID Timeout case is disabled at the moment * Working coordinator in case __key is loaded, Still very missing solution * Create dist_utils.c * Add timeout initialization for each subquery * Handle erros in processCursorMappingCallback() * Fix vector subquery flag * Fixing memory leak * Fixing HybridRequest_prepareForExecution() * Fixing memory leaks * Cleanup * MOD-11654: Deserialize Score If It Exists (#6968) * Update Feature Branch With Master (#6970) * MOD-11654: FT.HYBRID - Merging Requires __key To Exists (#6941) * MOD-11660: Coordinator upstreams sorters heap size (#6969) * Move coordinator subqueries areqs construction to hybridRequestSetupCoordinatorSubqueriesRequests * Fix hybridRequestSetupCoordinatorSubqueriesRequests implementation Add a flag indicating is coordinator subquery so getArrangeRP will add a sorter as expected * Small refactoring of RPHybridMerger_Accum (#6974) * MOD-11693: FT.HYBRID - Coordinator Gracefully Handle Errors (#6981) * initial commit * code review fixes * remove newline * code review fixes * fix typos * initialize count with current response count in case we already got some responses. * small fix * rephrase error messages * Update src/coord/hybrid/hybrid_cursor_mappings.c Typo --------- Co-authored-by: Itzikvaknin <82322982+Itzikvaknin@users.noreply.github.com> * MOD-11685: FT.HYBRID Fix Command Line Lifetime (#6977) * WIP: load key * initial commit * hybrid request - clone command line for lifetime correctness * small fix * additional fixes * remove logs * code review comments * sync the rpnet lookup with that of the merger * revert some changes * code review comments * code review comment * MOD-11685: Add a test for _ft.hybrid with the LOAD clause (#6976) * remove load clause handling from MR command * revert rlookup related code * additional revert of code * code review comment * small revert * code review comment --------- Co-authored-by: nafraf <nafraf@gmail.com> Co-authored-by: Itzikvaknin <82322982+Itzikvaknin@users.noreply.github.com> * MOD-11243: Change SORTBY 0 to NOSORT (#7009) * Change SORTBY 0 -> NOSORT in FT.HYBRID * New line EOF * Fix comments * Remove redundent callback * Add testSortbyNotEnoughArguments * Newline EOF * CR comment * Fix hybrid timeout (#7028) This PR includes several improvements and fixes: - Proper initialization of the time field in the search context for hybrid request flows. - Explicit setup of command properties in HybridRequest_prepareForExecution. - Updated iterStartCb private data to use a ref-counted data instead of a raw void*. - Fixed several memory leaks. Co-authored-by: Itzikvaknin <82322982+Itzikvaknin@users.noreply.github.com> * Change TestRealTimeouts queries to heavier ones (#7038) Change TestRealTimeouts_queries to heavier ones * MOD-11004: init hybrid areq time (#7051) * Add more timeout tests * Init time in HybridRequest_StartCursors() * UpdateTime in HybridRequest_BuildPipeline() * Clean up * Move setuptime to hybridCommandHandler * Remove unneeded SetupTime * Clean up test_hybrid_internal.py * Update RPDepleterTest * Merge master into feature-coordinator-hybrid (#7068) * MOD-11701: FT.HYBRID - Synchronize RLookup Moving Parts (#7006) * WIP: load key * initial commit * additional fixes * remove logs * code review comments * sync the rpnet lookup with that of the merger * revert some changes * code review comments * code review comment * remove load clause handling from MR command * initial commit * revert OpenLoadKeys function * bug fixes * fix typo * move back to Write in AddKeys, open docKey as read before initializing the hybrid lookup context * move sorting changes to only be in coordinator after distribute phase * remove MakeDefaultHybridUpstreams * cursor code review comments * code review comments * rename variable * fix compilation * modify aggpln serializatrion flow * fix buildMRCommand flow regarding adding __key to the LOAD clause * additional fixes * simplify the approach a bit - translate the result row as soon as we receive it - this allows us to use our own doc key to access the document key no need for a key array * rever some code to original state * code review fix * fix typos * code review comment * fix alias for search rlookup key not being opened in PLN_Distribute * fix cpp test compilation * small fix * fix * fix build mr tests * fix test leak * only free the array * fix aggregate flow * fix row leak * fix to fill upstream info function * handle cases where LOAD is already specified - since previous assert kept firiing * simulate previous behaviour to get tests to pass * fix * place the new load step at the end of the step arguments * fix typo * revert some of the code to a simpler way of serializing the additional steps * looks like LOAD can be specified multiple times * fix crash * remove free * change score key opening to write * fix compilation * fix cpptests - fallback to opening key as read if it already exists * open score as read for now to ensure consistent behaviour - less optimized * open the score key outside the unresolved ok scope * further refactoring of lookup flow * code review fixes * Fix depleter push to the coordinator upstream pipelines * Small fix to pushRP * Fix pointer address Fix typo --------- Co-authored-by: nafraf <nafraf@gmail.com> Co-authored-by: itzik.vaknin@redis.com <itzikvaknin6@gmail.com> * [MOD-11664] Handle _INDEX_PREFIXES in hybrid distributed queries (#7021) * feat: handle _INDEX_PREFIXES in hybrid distributed queries * Apply suggestions from code review * test: add test for the feature * fix: free allocated memory * test: fix unit tests * do not add empty _INDEX_PREFIXES to internal command if not needed * fix: do not directly cast args, use ArgCursor to traverse args cleanly * fix potential problem when no spec * initialize prefixes offsets for AREQ * fix _INDEX_PREFIXES handling and parsing * Fix memory issues * Prevent double-free * Fix mem leak * free prefixes in success path * Revert "free prefixes in success path" This reverts commit 2181baa. * fix: fix FGC tests * fix: empty prefix should be valid during the complete function scope * Code review fixes * fix by review comments * handle comments on review * use std vector as prefixes when working in tests * test: add happy path test * test: fix test --------- Co-authored-by: itzik.vaknin@redis.com <itzikvaknin6@gmail.com> Co-authored-by: nafraf <nafraf@users.noreply.github.com> Co-authored-by: nafraf <nafraf@gmail.com> --------- Co-authored-by: nafraf <nafraf@gmail.com> Co-authored-by: kei-nan <jonathan.keinan@redis.com> Co-authored-by: itzik.vaknin@redis.com <itzikvaknin6@gmail.com> Co-authored-by: Itzikvaknin <82322982+Itzikvaknin@users.noreply.github.com> Co-authored-by: nafraf <nafraf@users.noreply.github.com> Co-authored-by: Joan Fontanals <jfontanalsmartinez@gmail.com> * MOD-11884,MOD-11886: Support multiple load clauses in FT.HYBRID (#7075) (#7080) * Move load step handling in parseHybridCommand to a function handleLoadStepForHybridPipelines Support multiple load clauses * Add test for two load steps in FT.HYBRID * [MOD-10037] add default scorer in configuration (#7065) * feat: add default scorer in configuration * fix and test config setters and getters * test: add flow tests checking scorer behavior * fix: fix as per cursor comments * fix: fix as per code review * fix: change the init config code * test: fix the tests * test: refactor tests * fix memory leak in getter * test: add explicit test BM25STD is the default since start * validate default scorer at module init * fix validation against extensions * apply changes as per review * Update tests/cpptests/test_cpp_forkgc.cpp * refactor small setter * try to fix sanitizer leak * test: Known leaks when onModuleLoad fails * Update src/module-init/module-init.c Co-authored-by: kei-nan <jonathan.keinan@redis.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> * MOD-11669: Test hybrid response format (#7081) * Add simple test * Add test with APPLY, GROUPBY, YIELD_SCORE_AS --------- Co-authored-by: nafraf <nafraf@gmail.com> * Fix shifting shards order - [MOD-11833] (#7085) * remove incorrect assertion * sort shards in new topology RedisCluster_GetTopology, to have a stable order when the topology don't actually changes * [MOD-11542] Implement Inverted Index GC logic in Rust (#6878) * Repair an index block with deleted records * Repair an empty index block * Repair block when nothing was deleted * Repair block with some deletions * Remove num deletes * Simpler block creation * Have repair split big deltas correctly * Remove unwraps * Add GC scan to inverted index * Add comments * Test for scan * Add GC apply for ii * Add last block details to scan results * Skip the last block if new entries were added * Report on apply results * Derive Copy for unit structs * Use `smallvec` instead * Better name Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> * Use more of a zip / union for apply * Use `impl Fn` instead This will allow the use of closures at the FFI layer. * Add GC to wrappers and track unique docs correctly * Add repair callback * Clippy suggestions * Increment GC marker on apply * Improve comment Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> * Enable `union` feature for `smallvec` * Make it easier to see encoded block entries * Remove unneeded 0 entries check A block can never have zero entries outside of garbage collection so this check will never be hit. * Don't test empty blocks Nothing will result in empty block anyway. * Improve variable name Co-authored-by: meiravgri <109056284+meiravgri@users.noreply.github.com> * Check for last block only once * Ignore apply if deltas is empty * Use `u8` for delta * Document number in test * Make GC info fields public This will allow other tests (iterators) to access this information too. * Clippy suggestion * Add benchmark for GC patterns * Use variable to track changes * Faster unwrap This appears to be slightly faster than a pure `expect()`. * Run bench quicker This is done by setting custom measurement and warmup times. * Add an integration test This does a full cycle add_record, read, gc scan, and gc apply operations and tests the expected results. * Fix spelling --------- Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Co-authored-by: meiravgri <109056284+meiravgri@users.noreply.github.com> * MOD-11867: Fix ACL test to validate index creation (#7064) * Fix ACL test to validate index creation * Address review * Remove redundant test * copy test from razmon-fix_ACL_test_feature --------- Co-authored-by: nafraf <nafraf@gmail.com> --------- Signed-off-by: Henk Oordt <hd@oordt.dev> Co-authored-by: lerman25 <58445352+lerman25@users.noreply.github.com> Co-authored-by: meiravgri <109056284+meiravgri@users.noreply.github.com> Co-authored-by: BenGoldberger <ben.goldberger@redis.com> Co-authored-by: ofiryanai <ofiryanai1@gmail.com> Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com> Co-authored-by: Enrico Borba <enricozb@users.noreply.github.com> Co-authored-by: Pieter <pieter@chesedo.me> Co-authored-by: kei-nan <jonathan.keinan@redis.com> Co-authored-by: Henk Oordt <hd@oordt.dev> Co-authored-by: GuyAv46 <guy.avimor@gmail.com> Co-authored-by: Tim Janus <tim@janus.rs> Co-authored-by: Jonas Kruckenberg <iterpre@protonmail.com> Co-authored-by: Joan Fontanals <jfontanalsmartinez@gmail.com> Co-authored-by: Omer Shadmi <76992134+oshadmi@users.noreply.github.com> Co-authored-by: oshadmi <omer.shadmi@redis.com> Co-authored-by: galilev <galiwiner@gmail.com> Co-authored-by: Itzikvaknin <82322982+Itzikvaknin@users.noreply.github.com> Co-authored-by: Luca Palmieri <20745048+LukeMathWalker@users.noreply.github.com> Co-authored-by: Igor Malinovskiy <igor.malinovskiy@redis.com> Co-authored-by: Glen De Cauwsemaecker <contact@glendc.com> Co-authored-by: itzik.vaknin@redis.com <itzikvaknin6@gmail.com> Co-authored-by: Raz Monsonego <74051729+raz-mon@users.noreply.github.com>
(RediSearch/RediSearch#7076, RediSearch/RediSearch#6857) - Introducing `FT.HYBRID`, a new command that enables hybrid queries combining both text and vector search, with support for **RRF** and **LINEAR** result combination. This update enhances performance and reliability through a more efficient networking layer, smoother query execution, and improved overall stability. (RediSearch/RediSearch#7065) - Add `search-default-scorer` configuration to set the default text scorer across queries (by default it is BM25). RediSearch/RediSearch#7022 - Handle Atomic Slot Migration events upon moving slots from one node to another in a cluster mode. (RediSearch/RediSearch#6769, RediSearch/RediSearch#6828, RediSearch/RediSearch#6877, RediSearch/RediSearch#6921) - Introduce query time memory guardrails by adding a new `search-on-oom` configuration which defines the query engine behavior when OOM (Out Of Memory) is reached. OOM checks are applied to `FT.SEARCH`, `FT.AGGREGATE`, and `FT.HYBRID` commands. The behavior on OOM can be configured to one of three modes: `IGNORE`, `FAIL`, or `RETURN`. `IGNORE` - The default behavior, run queries anyway (not recommended for heavy queries returning a large result set). `FAIL` - Fail query execution immediately if any of the nodes are in OOM state when query execution starts. `RETURN` - A best effort appraoch to return partial results when OOM is detected in only some of the nodes in a cluster mode.
* [MOD-10037] add default scorer in configuration (#7065) * feat: add default scorer in configuration * fix and test config setters and getters * test: add flow tests checking scorer behavior * fix: fix as per cursor comments * fix: fix as per code review * fix: change the init config code * test: fix the tests * test: refactor tests * fix memory leak in getter * test: add explicit test BM25STD is the default since start * validate default scorer at module init * fix validation against extensions * apply changes as per review * Update tests/cpptests/test_cpp_forkgc.cpp * refactor small setter * try to fix sanitizer leak * test: Known leaks when onModuleLoad fails * Update src/module-init/module-init.c Co-authored-by: kei-nan <jonathan.keinan@redis.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> (cherry picked from commit b61c075) * fix: fix merge issues * fix: fix import in test * test: fix tests * fix: add bm25 std in query optimizier * Update src/module-init/module-init.c * test: fix tests * test: fix sanitizer errors in test_distagg * fix: protect with ENABLE_UNSTABLE_FEATURES * test: adapt test default scorer * Apply suggestions from code review Co-authored-by: nafraf <nafraf@users.noreply.github.com> * fix: fix after change of message * test: fix test --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> Co-authored-by: nafraf <nafraf@users.noreply.github.com>
* [MOD-10037] add default scorer in configuration (#7065) * feat: add default scorer in configuration * fix and test config setters and getters * test: add flow tests checking scorer behavior * fix: fix as per cursor comments * fix: fix as per code review * fix: change the init config code * test: fix the tests * test: refactor tests * fix memory leak in getter * test: add explicit test BM25STD is the default since start * validate default scorer at module init * fix validation against extensions * apply changes as per review * Update tests/cpptests/test_cpp_forkgc.cpp * refactor small setter * try to fix sanitizer leak * test: Known leaks when onModuleLoad fails * Update src/module-init/module-init.c Co-authored-by: kei-nan <jonathan.keinan@redis.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> (cherry picked from commit b61c075) * fix: add change from moved file * fix: protect default scorer by enable_unstable_features flag * fix: change as per code review * test: fix tests * change message * Update tests/pytests/test_config.py Co-authored-by: nafraf <nafraf@users.noreply.github.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> Co-authored-by: nafraf <nafraf@users.noreply.github.com>
* [MOD-10037] add default scorer in configuration (#7065) * feat: add default scorer in configuration * fix and test config setters and getters * test: add flow tests checking scorer behavior * fix: fix as per cursor comments * fix: fix as per code review * fix: change the init config code * test: fix the tests * test: refactor tests * fix memory leak in getter * test: add explicit test BM25STD is the default since start * validate default scorer at module init * fix validation against extensions * apply changes as per review * Update tests/cpptests/test_cpp_forkgc.cpp * refactor small setter * try to fix sanitizer leak * test: Known leaks when onModuleLoad fails * Update src/module-init/module-init.c Co-authored-by: kei-nan <jonathan.keinan@redis.com> --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> (cherry picked from commit b61c075) * test: fix tests * fix: remove file added by mistake * test: fix tests * Update src/query_optimizer.c * test: fix sanitizer errors in test_distagg * bring ENABLE_UNSTABLE_FEATURES in * test: fix the test config * Update src/config.h Co-authored-by: nafraf <nafraf@users.noreply.github.com> * fix: fix after change of message * test: fix test * test: fix flakiness in test --------- Co-authored-by: kei-nan <jonathan.keinan@redis.com> Co-authored-by: nafraf <nafraf@users.noreply.github.com>
Describe the changes in the pull request
Add DEFAULT_SCORER to configuration to be applied for every type of query
Note
Introduce a validated DEFAULT_SCORER config (via FT.CONFIG and Redis CONFIG) and apply it across SEARCH/AGGREGATE/HYBRID when no scorer is specified.
DEFAULT_SCORER(FT.CONFIG) andsearch-default-scorer(Redis CONFIG) with validation against registered scorers and exposure inINFOand config dumps.RSGlobalConfig.defaultScorer; getters/setters, load-time and runtime support; deprecation messages preserved.SCORERnot provided inFT.SEARCH/FT.AGGREGATE/FT.HYBRID(AREQ_ApplyContext, pipeline scorer, LLAPI iterator).BM25STDequivalently for scorer type.Extensions_InitDone(); validate configured default scorer after loading extensions; log/abort on invalid; free on unload.default_scorerin runtime info and formatted config string.test_default_scorer.py); updates to existing tests and redismock to init/cleanupdefaultScorer; minor hybrid tests adjusted to expect default scorer name.Written by Cursor Bugbot for commit 9110b7b. This will update automatically on new commits. Configure here.