search on all token during where searches

[philippebeaulieu]

We discovered a bug while using search.where with multi token criterias. With the previous logic, only the first token was used for search.

My following pull request includes a test that was failing without this fix.

ex: defining these 4 documents:

const id1 = await insert(db, {
    name: 'super coffee maker',
    rating: 5,
    price: 900,
    meta: {
      sales: 100,
      finish: 'black matte',
    },
  })

  const id2 = await insert(db, {
    name: 'washing machine',
    rating: 5,
    price: 900,
    meta: {
      sales: 100,
      finish: 'gloss black',
    },
  })

  const id3 = await insert(db, {
    name: 'coffee maker',
    rating: 3,
    price: 30,
    meta: {
      sales: 25,
      finish: 'gloss blue',
    },
  })

  const id4 = await insert(db, {
    name: 'coffee maker deluxe',
    rating: 5,
    price: 45,
    meta: {
      sales: 25,
      finish: 'blue matte',
    },
  })

And doing this search:

const result = await search(db, {
      where: {
        'meta.finish': 'black matte',
      },
    })

resulted in only 2 results. When tokenized the where criteria was converted to ["black", "matte"] and on line searchByWhereClause:479 only the first token was used resulting on only document 1 and 2 being returned.

Expected values: doc 1, 2 and 4.

This PR modifies the behavior to search on all tokens.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
orama-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Aug 31, 2023 7:04pm

[micheleriva]

that looks neat. Thanks! lgtm