Get and delete documents by filter #236

irevoire · 2023-05-04T14:11:54Z

Attention to reviewers

Associated issue: meilisearch/meilisearch#3477

Get/fetch implemented in Get documents by filter meilisearch#3570
Delete implemented in Delete documents by filter meilisearch#3550

We decided to merge both feature inside a single PR.
But some you can still see the initial review of the get/fetch documents by filter here: #234

Summary

Specify the new fetch and delete by filter route.

Changes

The GET /indexes/:uid/documents route has been updated with a new query parameter filter
A new POST /indexes/:uid/documents/fetch route has been created with all the parameters of the GET version above (limit, offset, fields, and filter)
A new error code has been introduced in case an error happens on the new filter fields: invalid_document_filter
Introduce a new POST /indexes/:index_uid/documents/delete route that lets you delete documents by filter
The payload must be an object only containing a filter field, i.e.: { "filter": "doggo = bernese" }
A new error has been introduced in case the specified filter is wrong or empty: invalid_document_delete_filter

Misc

Update OpenAPI specification file
Update telemetry datapoints

github-actions · 2023-05-04T14:12:34Z

🚨 Breaking API change detected:

Added (1)

POST /indexes/{indexUid}/documents/fetch

Modified (2)

GET /indexes/{indexUid}/documents
- Query parameter added: filter
[Breaking] POST /indexes/{indexUid}/documents/delete
- [Breaking] Header added: Content-Type
- [Breaking] Body added

Powered by Bump

gmourier · 2023-05-10T15:08:09Z

@irevoire is that one ready for review? I see it in draft state

irevoire · 2023-05-10T15:21:53Z

Hey, not really. I'm waiting for #234 to be merged before rebasing this one

gmourier · 2023-05-10T15:30:42Z

We merge branches on the release day (https://github.com/meilisearch/specifications#release-worfklow) to avoid spanning multiple PRs for the same specification scope if changes need to occur until the release.

What I've done in the past for that type of case is I rebase the last one to the parent one or mix changes relating to the same product scope under the same PR to avoid multiple rebase over-time 🥲

text/0060-tasks-api.md

gmourier · 2023-05-15T06:55:28Z

@irevoire can you rebase that one on #234? 🙇‍♂️

text/0124-documents-api.md

open-api.yaml

text/0124-documents-api.md

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

text/0034-telemetry-policies.md

text/0060-tasks-api.md

text/0061-error-format-and-definitions.md

text/0085-api-keys.md

text/0124-documents-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

3775: Last error code changes on the new get/delete documents routes r=dureuill a=irevoire # Pull Request ## Related issue Fixes #3774 ## What does this PR do? Following the specification: meilisearch/specifications#236 1. Get rid of the `invalid_document_delete_filter` and always use the `invalid_document_filter` 2. Introduce a new `missing_document_filter` instead of returning `invalid_document_delete_filter` (that’s consistent with all the other routes that have a mandatory parameter) 3. Always return the `original_filter` in the details (potentially set to `null`) instead of hiding it if it wasn’t used Co-authored-by: Tamo <tamo@meilisearch.com>

…e missing_document_filter

gmourier

Thank you @irevoire 🚂

open-api.yaml

472: Add delete_documents_with method for Meilisearch v1.2 r=bidoubiwa a=bidoubiwa following this spec: meilisearch/specifications#236 - Add a method `index.delete_documents_with` that lets you select a range of documents to delete based on the provided filters. Co-authored-by: meili-bot <74670311+meili-bot@users.noreply.github.com> Co-authored-by: Charlotte Vermandel <charlottevermandel@gmail.com> Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Bump OpenAPI version * CSV Content-Type — Add support for the boolean type in CSV (#228) * add support for the boolean type in csv * fix a typo * Update text/0028-indexing-csv.md --------- Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com> * Filter operators - `IS NULL` and `IS EMPTY` filter operators (#232) * Introduce the spec of the IS EMPTY filter * Introduce the spec of the IS NULL filter * Fix suggestions * Uppercase the IN operator Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com> * Fix a title number issue Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com> --------- Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com> * Search Algorithms Refactor: Relevancy Changes (#233) * Remove old exactness-criterion specification * Remove old asc-desc-criterion specification * Get and delete documents by filter (#236) * specify the new fetch documents route * update open-api * try to be nice with bumpsh * specify the new analytics of the get/fetch documents route * group the analytics behind already used groups * fix a placeholder text * group the get and fetch route under a common section * rename the section to contain the name of both routes * clarify how to send the parameter to both routes * fix the type of the parameters * add the new fetch route to the api keys actions * capitalize json Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * update open-api * Specify the new route to delete documents by filter * add the new analytics event * improve a comment * add the post - delete route to the api key actions * Fix the json payload Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com> * Update the details Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com> * delete the invalid_document_delete_filter error code and introduce the missing_document_filter * fix the open-api thingy --------- Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com> * Auto deletion of tasks (#238) * auto deletion of tasks * Update text/0060-tasks-api.md Co-authored-by: gui machiavelli <hey@guimachiavelli.com> * Update text/0060-tasks-api.md Co-authored-by: gui machiavelli <hey@guimachiavelli.com> --------- Co-authored-by: gui machiavelli <hey@guimachiavelli.com> * Experimental reduce indexing memory usage (#239) * Add the experimental-reduce-indexing-memory-usage option in the telemetry policies * Add the experimental-reduce-indexing-memory-usage option in the instance options * Update 0055-sort.md (#240) related to meilisearch/meilisearch#3749 * Update `payload_too_large` error `message` (#241) * Update 0061-error-format-and-definitions.md Update error message in the specification * Update open-api.yaml --------- Co-authored-by: Tamo <tamo@meilisearch.com> Co-authored-by: Clément Renault <renault.cle@gmail.com> Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com> Co-authored-by: gui machiavelli <hey@guimachiavelli.com> Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com>

irevoire requested a review from macraig May 4, 2023 14:11

irevoire marked this pull request as draft May 4, 2023 14:12

irevoire added Implemented Feature specification has been implemented. OpenAPI Update OpenAPI specification. labels May 4, 2023

irevoire mentioned this pull request May 4, 2023

Delete and get documents by filter meilisearch/meilisearch#3477

Closed

irevoire added 3 commits May 4, 2023 16:27

specify the new fetch documents route

174cafd

update open-api

6ef42de

try to be nice with bumpsh

5e2c6d8

macraig requested a review from gmourier May 8, 2023 19:11

macraig approved these changes May 8, 2023

View reviewed changes

brunoocasali mentioned this pull request May 10, 2023

Changes regarding Meilisearch v1.2.0 meilisearch/integration-guides#261

Closed

specify the new analytics of the get/fetch documents route

3141e26

maryamsulemani97 mentioned this pull request May 10, 2023

v1.2: Delete and get documents by filter meilisearch/documentation#2411

Closed

4 tasks

irevoire added 3 commits May 10, 2023 15:31

group the analytics behind already used groups

52ef6c6

fix a placeholder text

f50255b

group the get and fetch route under a common section

9af7b72

maryamsulemani97 mentioned this pull request May 11, 2023

v1.2: Telemetry table updates meilisearch/documentation#2414

Closed

3 tasks

maryamsulemani97 reviewed May 11, 2023

View reviewed changes

text/0060-tasks-api.md Outdated Show resolved Hide resolved

gmourier mentioned this pull request May 15, 2023

Release v1.2.0 #227

Merged

1 task

gmourier added v1.2.0 Telemetry Update the telemetry collect. labels May 15, 2023

bidoubiwa reviewed May 15, 2023

View reviewed changes

text/0124-documents-api.md Outdated Show resolved Hide resolved

text/0124-documents-api.md Outdated Show resolved Hide resolved

brunoocasali reviewed May 15, 2023

View reviewed changes

open-api.yaml Outdated Show resolved Hide resolved

gmourier added the Q2:2023 label May 16, 2023

maryamsulemani97 reviewed May 16, 2023

View reviewed changes

text/0124-documents-api.md Outdated Show resolved Hide resolved

irevoire force-pushed the delete-documents branch from 81eedec to b2931c5 Compare May 16, 2023 10:27

bidoubiwa reviewed May 16, 2023

View reviewed changes

text/0124-documents-api.md Outdated Show resolved Hide resolved

Fix the json payload

7d9c273

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

bidoubiwa mentioned this pull request May 16, 2023

Add filtering on deleteDocuments for Meilisearch v1.2 meilisearch/meilisearch-js#1492

Merged

irevoire changed the title ~~Delete documents by filter~~ Get and delete documents by filter May 22, 2023

irevoire mentioned this pull request May 22, 2023

Specify the new fetch documents route #234

Closed

2 tasks

bidoubiwa mentioned this pull request May 22, 2023

Add delete_documents_with method for Meilisearch v1.2 meilisearch/meilisearch-rust#472

Merged

alallema mentioned this pull request May 23, 2023

Delete documents only using primary keys, not conditional filtering meilisearch/meilisearch-go#438

Closed

gmourier suggested changes May 24, 2023

View reviewed changes

Apply suggestions from code review

57c403c

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

This was referenced May 24, 2023

Last error code changes on the new get/delete documents routes meilisearch/meilisearch#3774

Closed

Last error code changes on the new get/delete documents routes meilisearch/meilisearch#3775

Merged

Update the details

e37e55f

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

delete the invalid_document_delete_filter error code and introduce th…

2aa435d

…e missing_document_filter

alallema mentioned this pull request May 24, 2023

Add method DeleteDocumentsByFilter for Meilisearch v1.2 meilisearch/meilisearch-go#440

Merged

gmourier approved these changes May 29, 2023

View reviewed changes

gmourier removed the Ready For Review Feature specification must be reviewed. label May 29, 2023

ahmednfwela reviewed May 30, 2023

View reviewed changes

open-api.yaml Outdated Show resolved Hide resolved

ahmednfwela reviewed May 30, 2023

View reviewed changes

open-api.yaml Outdated Show resolved Hide resolved

fix the open-api thingy

98fbdcb

macraig merged commit 8b23eb8 into release-v1.2.0 Jun 5, 2023
2 checks passed

macraig deleted the delete-documents branch June 5, 2023 15:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Get and delete documents by filter #236

Get and delete documents by filter #236

irevoire commented May 4, 2023 •

edited

Loading

github-actions bot commented May 4, 2023

gmourier commented May 10, 2023

irevoire commented May 10, 2023

gmourier commented May 10, 2023 •

edited

Loading

gmourier commented May 15, 2023 •

edited

Loading

gmourier left a comment

Get and delete documents by filter #236

Get and delete documents by filter #236

Conversation

irevoire commented May 4, 2023 • edited Loading

Attention to reviewers

Summary

Changes

Misc

github-actions bot commented May 4, 2023

Added (1)

Modified (2)

gmourier commented May 10, 2023

irevoire commented May 10, 2023

gmourier commented May 10, 2023 • edited Loading

gmourier commented May 15, 2023 • edited Loading

gmourier left a comment

Choose a reason for hiding this comment

irevoire commented May 4, 2023 •

edited

Loading

gmourier commented May 10, 2023 •

edited

Loading

gmourier commented May 15, 2023 •

edited

Loading