Skip to content

fix flaky recommender rerank_depth test #285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 15, 2025
Merged

Conversation

danyilq
Copy link
Contributor

@danyilq danyilq commented May 1, 2025

  • Please check if the PR fulfills these requirements
  • The commit message follows our guidelines
  • Tests for the changes have been added (for bug fixes/features)
  • Docs have been added / updated (for bug fixes / features)
  • What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

    • Bug fix
    • Documentation update
    • Minor test behavior improvement
  • What is the current behavior? (You can also link to an open issue here)

    • Some integration tests in py-marqo, particularly for recommender and facets, exhibit flakiness and inconsistent cleanup behavior.
    • The create-indexes=True parameter in cloud tests causes existing indexes to be deleted, which makes tests more annoying to deal with.
    • Test setup instructions are scattered and unclear.
  • What is the new behavior (if this is a feature change)?

    • Test flakiness related to recommender and facets tests is resolved by analyzing the roor cause and excepting different values depending on environment since cloud tests are multi-node.
    • One test in test_tensor_search has been updated for improved readability.
    • A new README.dev.md has been added under the tests/ directory, documenting how to properly run integration tests against both Open Source and Cloud environments, including GitHub Actions usage and cleanup procedures.
    • The create-indexes flag has been updated: if set to True and the indexes already exist, they are now skipped rather than deleted, allowing tests to proceed without destructive actions.
  • Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

  • Other information:

keshav
keshav previously approved these changes May 13, 2025
VitusAcabado
VitusAcabado previously approved these changes May 14, 2025
@danyilq danyilq dismissed stale reviews from VitusAcabado and keshav via fbc9497 May 14, 2025 05:40
keshav
keshav previously approved these changes May 14, 2025
@danyilq danyilq merged commit 9eac3e2 into mainline May 15, 2025
27 checks passed
@danyilq danyilq deleted the danyil/fix-flaky-test branch May 15, 2025 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants