Skip to content

v1.9.0 馃

Latest
Latest
Compare
Choose a tag to compare
@ManyTheFish ManyTheFish released this 01 Jul 05:51
v1.9.0
0df84bb
This commit was created on GitHub.com and signed with GitHub鈥檚 verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Meilisearch v1.9 includes performance improvements for hybrid search and the addition/updating of settings. This version benefits from multiple requested features, such as the new frequency matching strategy and the ability to retrieve similar documents.

馃О All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment happens between 4 to 48 hours after a new version becomes available.

Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that 鉂わ笍).

New features and updates 馃敟

Hybrid search updates

This release introduces multiple hybrid search updates.

Done by @dureuill and @irevoire in #4633 and #4649

鈿狅笍 Breaking change: Empty _vectors.embedder arrays

Empty _vectors.embedder arrays are now interpreted as having no vector embedding.

Before v1.9, Meilisearch interpreted these as a single embedding of dimension 0. This change follows user feedback that the previous behavior was unexpected and unhelpful.

鈿狅笍 Breaking change: _vectors field no longer present in search results

When the experimental vectorStore feature is enabled, Meilisearch no longer includes _vectors in returned search results by default. This will considerably improve performance.

Use the new retrieveVectors search parameter to display the _vectors field:

curl \
  -X POST 'http://localhost:7700/indexes/INDEX_NAME/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "SEARCH QUERY",
    "retrieveVectors": true
  }'

鈿狅笍 Breaking change: Meilisearch no longer preserves the exact representation of embeddings appearing in _vectors

In order to save storage and run faster, Meilisearch is no longer storing your vector "as-is". Meilisearch now returns the float in a canonicalized representation rather than the user-provided representation.

For example, 3 may be represented as 3.0

Document _vectors accepts object values

The document _vectors field now accepts objects in addition to embedding arrays:

{
  "id": 42,
  "_vectors": {
    "default": [0.1, 0.2 ],
      "text": {
        "embeddings": [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]],
        "regenerate": false
      },
      "translation": {
        "embeddings": [0.1, 0.2, 0.3, 0.4],
        "regenerate": true
      }
    }
}

The _vectors object may contain two fields: embeddings and regenerate.

If present, embeddings will replace this document's embeddings.

regenerate must be either true or false. If regenerate: true, Meilisearch will overwrite the document embeddings each time the document is updated in the future. If regenerate: false, Meilisearch will keep the last provided or generated embeddings even if the document is updated in the future.

This change allows importing embeddings to autoembedders as a one-shot process, by setting them as regenerate: true. This change also ensures embeddings are not regenerated when importing a dump created with Meilisearch v1.9.

Meilisearch v1.9.0 also improves performance when indexing and using hybrid search, avoiding useless operations and optimizing the important ones.

New feature: Ranking score threshold

Use rankingScoreThreshold to exclude search results with low ranking scores:

curl \
 -X POST 'http://localhost:7700/indexes/movies/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
    "q": "Badman dark returns 1",
    "showRankingScore": true,
    "limit": 5,
    "rankingScoreThreshold": 0.2
 }'

Meilisearch does not return any documents below the configured threshold. Excluded results do not count towards estimatedTotalHits, totalHits, and facet distribution.

鈿狅笍 For performance reasons, if the number of documents above rankingScoreThreshold is higher than limit, Meilisearch does not evaluate the ranking score of the remaining documents. Results ranking below the threshold are not immediately removed from the set of candidates. In this case, Meilisearch may overestimate the count of estimatedTotalHits, totalHits and facet distribution.

Done by @dureuill in #4666

New feature: Get similar documents endpoint

This release introduces a new AI-powered search feature allowing you to send a document to Meilisearch and receive a list of similar documents in return.

Use the /indexes/{indexUid}/similar endpoint to query Meilisearch for related documents:

curl \
  -X POST /indexes/:indexUid/similar
  -H 'Content-Type: application/json' \
  --data-binary '{
    "id": "23",
    "offset": 0,
    "limit": 2,
    "filter": "release_date > 1521763199",
    "embedder": "default",
    "attributesToRetrieve": [],
    "showRankingScore": false,
    "showRankingScoreDetails": false
  }'
  • id: string indicating the document needing similar results, required
  • offset: number of results to skip when paginating, optional, defaults to 0
  • limit: number of results to display, optional, defaults to 20
  • filter: string with a filter expression Meilisearch should apply to the results, optional, defaults to null
  • embedder: string indicating the embedder Meilisearch should use to retrieve similar documents, optional, defaults to "default"
  • attributesToRetrieve: array of strings indicating which fields Meilisearch will include in the response, optional, defaults to ["*"]
  • showRankingScore: boolean indicating if results should include ranking score information, optional, defaults to false
  • showRankingScoreDetails: boolean indicating if results should include detailed ranking score information, optional, defaults to false
  • rankingScoreThreshold: Excludes search results with a ranking score lower than the defined number, optional, defaults to null.

/indexes/{indexUid}/similar supports GET and POST routes. Use URL query parameters to configure your GET request, or include your parameters in the request body if using the POST route. Both offer identical functionality.

Done by @dureuill in #4647

New feature: frequency matching strategy

This release adds a new matching strategy, frequency. Use it to prioritize results containing the least frequent query terms:

curl \
 -X POST 'http://localhost:7700/indexes/{index_uid}/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
    "q": "cheval blanc",
    "matchingStrategy": "frequency"
 }'

Done by @ManyTheFish in #4667

Set distinctAttribute at search time

This release introduces a new search parameter: distinct which you can use to specify the distinct attribute at search time:

curl \
 -X POST 'http://localhost:7700/indexes/{index_uid}/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
    "q": "kefir le double poney",
    "distinct": "book.isbn"
 }'

If a distinct attribute is already defined in the settings it'll be ignored in favor of the one defined at search time.

Done by @Kerollmops in #4693

Improve indexing speed when updating/adding settings

Meilisearch now limits operations when importing settings by avoiding unnecessary writing operations in its internal database and reducing disk usage.

Additionally, when changing embedding settings, Meilisearch will now only regenerate the embeddings for the embedders whose settings have been modified, instead of for all embedders. When only the documentTemplate is modified, embeddings will only be regenerated for documents where this modification leads to a different text to embed.

Done by @irevoire, @Kerollmops, @ManyTheFish and @dureuill in #4646, #4680, #4631 and #4649

Other improvements

Fixes 馃悶

Misc

鉂わ笍 Thanks again to our external contributors:

Contributors

mosuka, Kerollmops, and 10 other contributors
Assets 8
Loading