Hybrid - Minimum viable results - acceptance threshold - skip very bad results

[sebawita]

Describe your feature request

Currently, hybrid search doesn't allow me to filter out really poor results – those where both keyword and vector search provide with very poor results.

It would be great if we could provide an acceptance threshold as part of a hybrid query, where we would require at least on of them to be satisfied to return an object i.e.

• max vector distance – accept results where distance is not more than x
• min keyword score – accept results where the score is at least y

This way we could eliminate results that are poor on both fronts (keyword and vector).

Example:

I have two objects:

• "a very big army helicopter"
• "a small light civilian plane"

If I search for "sandwich", this should result in poor results on both fronts, and I would expect no results back.
While a search for "aeroplane", should result in a vector match (and no match on BM25), and I should get back the two objects back.

import weaviate, os
import weaviate.classes.config as wc

client = weaviate.connect_to_local(
    headers={
        "X-OpenAI-Api-Key": os.environ["OPENAI_API_KEY"]
    }
)

test = client.collections.create(
    "Test",
    vectorizer_config=wc.Configure.Vectorizer.text2vec_openai()
)

test.data.insert_many([
    {
        "name":"a very big army helicopter"
    },
    {
        "name":"a small light civilian plane"
    },
])

Proposed solution

import weaviate.classes.query as wq 
response = test.query.hybrid(
    "sandwich",

    # The new param to remove poor results on both fronts - at least one should be satisfied to include in the result set
    acceptance_threshold: {
        min_hybrid_score: 0.1,
        max_vector_distance: 0.2
    },

    return_metadata=wq.MetadataQuery(score=True),
)

Note

BM25 alone doesn't return if no match was made ;)

If I run:

import weaviate.classes.query as wq 
response = test.query.bm25(
    "sandwich",
    return_metadata=wq.MetadataQuery(score=True),
)

The query returns 0 objects.
So, that is already a good indicator that we rely solely on the vector search. ;)

Code of Conduct

• I have read and agree to the Weaviate's Contributor Guide and Code of Conduct

[sebawita]

Note, the idea here is not to filter on the final hybrid score, but instead to filter out results based on the separate scores/distances before we get to calculate the final hybrid score.

[iamleonie]

/bounty $200

[algora-pbc]

💎 $200 bounty • Weaviate

Steps to solve:

1. Start working: Comment /attempt #4315 with your implementation plan
2. Submit work: Create a pull request including /claim #4315 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Additional opportunities:

• 🔴 Livestream on Algora TV while solving this bounty & earn $200 upon merge! Make sure to have your camera and microphone on. Comment /livestream once live

Thank you for contributing to weaviate/weaviate!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @dresslife-shbh	May 13, 2024, 12:29:58 PM	#4980
🔴 @Bhavyajain21	May 14, 2024, 10:15:31 AM	WIP
🟢 @hsm207	May 14, 2024, 10:18:57 AM	WIP

[dresslife-shbh]

/attempt #4315

Options

• Cancel my attempt

[Bhavyajain21]

/attempt

Algora profile	Completed bounties	Tech	Active attempts	Options
@Bhavyajain21	8 bounties from 3 projects	TypeScript, Rust, JavaScript & more		Cancel attempt

[hsm207]

/attempt

Options

• Cancel my attempt

[algora-pbc]

💡 @dresslife-shbh submitted a pull request that claims the bounty. You can visit your bounty board to reward.