Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Flaky tests in segment/test_vector #927

Closed
atroyn opened this issue Aug 3, 2023 · 0 comments · Fixed by #974
Closed

[Bug]: Flaky tests in segment/test_vector #927

atroyn opened this issue Aug 3, 2023 · 0 comments · Fixed by #974
Assignees
@HammadB
Labels
bug Something isn't working

Comments

@atroyn
Copy link
Contributor

atroyn commented Aug 3, 2023

What happened?

Occasionally, the test in chromadb/test/segment/test_vector.py will fail because it's possible for a vector to not be its own nearest neighbor.

We mitigate this in the property tests by setting the HNSW parameters such that this never happens.

chroma/chromadb/test/property/strategies.py

Lines 41 to 45 in c387e1d

test_hnsw_config = {
"hnsw:construction_ef": 128,
"hnsw:search_ef": 128,
"hnsw:M": 128,
}

Flaky tests are a liability. Since we have some known good parameters, we should share them wherever HNSW is used in a test context.

Versions

Chroma 0.4.x

Relevant log output

https://github.com/chroma-core/chroma/actions/runs/5756305311/job/15605458682
@atroyn atroyn added the bug Something isn't working label Aug 3, 2023
@HammadB HammadB self-assigned this Aug 12, 2023
HammadB added a commit that referenced this issue Aug 12, 2023
@HammadB
Fix #927 by adding hnsw params to test_vector segment tests
d76f009
This was referenced Aug 12, 2023
Merged
Merged
HammadB added a commit that referenced this issue Aug 14, 2023
@HammadB
[TST] Fix #927 by adding hnsw params to test_vector segment tests (#974)
1e30d13 
## Description of changes
Closes #927 
*Summarize the changes made by this PR.*
 - Improvements & Bug fixes
	 -Fix #927 by adding hnsw params to test_vector segment tests
 - New functionality
	 - N/A

## Test plan
These are tests 

## Documentation Changes
*Are all docstrings for user-facing APIs updated if required? Do we need
to make documentation changes in the [docs
repository](https://github.com/chroma-core/docs)?*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@HammadB HammadB

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[TST] Fix #927 by adding hnsw params to test_vector segment tests chroma-core/chroma
2 participants
@atroyn @HammadB