Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Tasks API - Add task cancelation capability #195

Merged
merged 77 commits into from Nov 29, 2022

Conversation

alallema
Copy link
Contributor

@alallema alallema commented Oct 25, 2022

🤖 API Diff


Summary

This PR describes changes to welcome the task cancelation feature. It also adds consistency by specifying which task fields are displayed by default to null within a task resource.


Changes

  • A new endpoint: POST /tasks/cancel

    • A new task type: taskCancelation
    • A new API key action: tasks.cancel
    • A new task status canceled
  • Introduce a new error code missing_task_filters

  • uids filter. e.g /tasks?uids=1,2,3,4

    • Introduce a new error code invalid_task_uids_filter
  • canceledBy filter. e.g tasks?canceledBy=99,100

    • Introduce a new error code invalid_task_canceled_by_filter
  • Introduce date filters:

    • Introduce a new error code invalid_task_date_filter
    • beforeEnqueuedAt/afterEnqueuedAt filter.
    • beforeStartedAt/afterStartedAt filter.
    • beforeFinishedAt/afterFinishedAt filter.
  • Task error field is set to null by default.

  • Task canceledBy field is set to null by default.

  • Task details sub-fields are set to null when Meilsearch can't know the information depending on the task status. @Kerollmops please check the changes within the spec and suggests precision/correction, if any.

  • Add Tasks Canceled telemetry event

    • user_agent
    • filtered_by_uid
    • filtered_by_index_uid
    • filtered_by_status
    • filtered_by_type
    • filtered_by_canceled_by
    • filtered_by_before_enqueued_at
    • filtered_by_after_enqueued_at
    • filtered_by_before_started_at
    • filtered_by_after_started_at
    • filtered_by_before_finished_at
    • filtered_by_after_finished_at

Breaking Changes ⚠️

  • indexUid query parameter is renamed indexUids
  • type query parameter is renamed types
  • status query parameter is renamed statuses
  • invalid_task_type error is renamed invalid_task_types_filter
  • invalid_task_status error is renamed invalid_task_statuses_filter

Out Of Scope

Explain in clear way subjects that are out of scope and shouldn't be discussed by the reviewers


Attention To Reviewers

The delete tasks PR will be based on the cancel-tasks-feature branch because changes are mostly the same.


Misc

  • Update OpenAPI specification file
  • Update telemetry datapoints

@alallema alallema changed the base branch from main to release-v0.30.0 October 25, 2022 16:40
@alallema alallema changed the title Cancel Tasks Cancel Tasks API Oct 25, 2022
@gmourier gmourier mentioned this pull request Oct 25, 2022
1 task
@gmourier gmourier added OpenAPI Update OpenAPI specification. Q4:2022 v0.30 In Progress Feature specification is in elaboration. Important changes can still occurs in the specification. Telemetry Update the telemetry collect. labels Oct 26, 2022
@gmourier
Copy link
Member

@alallema

  • I created a new sub-section in #### 11. Filtering task resources and split into two sections ##### 11.1. Filtering task to get and ##### 11.2. Filtering task to cancel but maybe all this information should be mixed in the #### 11. Filtering task resources -> I agreed; it's a bit redundant. 👍
  • Auto-batching task cancelation should be added -> It could be added as a specific section for the POST - /tasks/cancel endpoint section or either be added in the auto-batching specification.

Copy link
Member

@gmourier gmourier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @alallema, here is a first draft review.

I think the specification can be extremely simplified if we explain all the filters listing and behaviors in one single place; since all tasks endpoints can use them there is no need to repeat that information for every endpoints.

  • I couldn't mark it as a suggestion but in some places, it is mentioned that dumpCreation is a global task. e.g. Summarized task Object for 202 Accepted. It would be worthwhile to make a section listing the types of tasks that are global and therefore have a null indexUid.

  • Don't forget to update the api-keys specification since we are adding a new action tasks.cancel

text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
alallema and others added 14 commits October 26, 2022 14:39
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
open-api.yaml Outdated Show resolved Hide resolved
text/0060-tasks-api.md Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
gmourier and others added 4 commits November 15, 2022 15:37
Co-authored-by: Clément Renault <renault.cle@gmail.com>
Co-authored-by: Clément Renault <renault.cle@gmail.com>
Copy link
Member

@Kerollmops Kerollmops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to our discussion on Slack, we must go with originalFilter because, technically, it is one filter expression and for consistency with the search route.

open-api.yaml Outdated Show resolved Hide resolved
text/0060-tasks-api.md Outdated Show resolved Hide resolved
gmourier and others added 2 commits November 21, 2022 14:29
Co-authored-by: Clément Renault <renault.cle@gmail.com>
Copy link
Member

@Kerollmops Kerollmops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 😃

text/0060-tasks-api.md Outdated Show resolved Hide resolved
gmourier and others added 3 commits November 23, 2022 15:02
* Describes task deletion feature additions

* Update open-api.yml

* Update telemetry-policies

* Update originalQuery to originalFilters

* Follow title structure

* Introduce review feedback

* Branch the right components

* Update text/0060-tasks-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

Co-authored-by: Clément Renault <renault.cle@gmail.com>
@gmourier gmourier merged commit 7f69193 into release-v0.30.0 Nov 29, 2022
@gmourier gmourier deleted the cancel-tasks-feature branch November 29, 2022 15:12
@gmourier gmourier added Implemented Feature specification has been implemented. and removed Ready For Review Feature specification must be reviewed. labels Nov 29, 2022
gmourier added a commit that referenced this pull request Nov 29, 2022
* Bump Meilisearch OpenAPI spec to v0.30.0

* Compression Headers (#178)

* Start specification

* Rename spec file

* Rephrase summary

* Rephrase receiving a compressed response section

* Add links to RFC9910

* Introduce missing_master_key error (#179)

* Update error message (#180)

* Specify the error variant (#181)

* replaces 127.0.0.1 by localhost (#184)

* Configuration File Support (#185)

* init spec file

* Fill-in spec

* Fix typing errors

* adds errors case

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Replace config file by configuration file in the error message

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Add infos.with_configuration_file to telemetry-policies

* Update text/0185-configuration-file.md

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0185-configuration-file.md

Co-authored-by: Tamo <tamo@meilisearch.com>

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>

* Telemetry - Missing events and options (#186)

* wip spec changes

* update telemetry

* add missing property to the full properties table

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Add distrinct_attribute.set property, Task Seen becomes an aggregated event, add missing sub settings properties to Settings Updated event

* Add requests.total_received property to Tasks Seen event

* add missing ranking rules metrics

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Update text/0034-telemetry-policies.md

Co-authored-by: Tamo <tamo@meilisearch.com>

* Precise max-index-size option (#187)

* Tasks API - Rename `receivedDocumentsIds` by `providedIds` (#188)

* Rename receivedDocumentsIds by matchedDocuments

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0060-tasks-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Add missing env vars, brings consistency (#190)

* Swap Indexes API (#192)

* Add new spec file as draft file

* Add specification file

* update spec

* draft open-api.yml changes

* Add errors

* fix typos

* Add Indexes Swapped event

* duplicate_index_found is synchronous

* Apply suggestions from code review

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>

* Add review suggestions

* Precis mention for tasks indexUid target swap in the history

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0061-error-format-and-definitions.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0191-swap-indexes-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Add snapshotCreation as a new task type to make it visible in the task history (#193)

* Tasks API - Add task priority order section (#194)

* Add 2.1 Task Priority section

* Precise other tasks priority ordering

* Search API - Page Selection Component (#196)

* Search API - numbered pagination

* Improve numbered pagination explaination

* Update text/0118-search-api.md

* Update text/0118-search-api.md

* Apply suggestions from code review

* Add links

* Update open api

* Add a dot at the end of the title

* Fix links

* Improve page and hitsperpage query params

* Refactor pagination guide

* Fix wording

* Add corner case

* Update open-api.yaml

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update open-api.yaml

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update open-api.yaml

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Apply wording changes from code review

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Apply suggestions from code review

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Bruno Casali <brunoocasali@gmail.com>

* Fix example

* Apply suggestions from code review

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update open-api.yaml

* Update text/0118-search-api.md

* Update links

* Add link to maxtotalhits

* Update text/0118-search-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Improve message in page reference

* Update text/0118-search-api.md

Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com>

* Add telemetry to know the most used search results navigation methods in a batch

* Update text/0118-search-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0118-search-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com>

* Tasks API - Add task cancelation capability (#195)

* Add cancel to task spec

* Remove missing error

* Fix typo on cancels

* Fix IndexUid always null

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Add canceledBy and missing_filters error

* Merge 11.1 filter iwth get and 11.2 filter with post

* Specified every date filter

* Change uid as integer and not as string

* Add auto-batching and error

* Add new route task/cancel api-key

* Add global task note

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0060-tasks-api.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Update text/0061-error-format-and-definitions.md

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>

* Modification due to review

* Modification due to review

* Modification due to review

* Modification due to review

* Fix global task link

* Fix format link

* Add open-api modification

* Remove trailing space

* Add clarification

* Add errors definition

* Update open-api.yml

* fix task uid query parameter name

* Fix collission between url path and query parameters

* fmt open-api.yml

* Add telemetry details

* Add 202_taskCancelation example

* Update originalQuery to originalFilters and update missing_task_filter to missing_task_filters

* matchedTasks is computed when the task is enqueued

* Apply suggestions from code review

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>

* Recall explanation for filter query parameter and mentions that limit and from are valid on

* Fix typo in property name

* Update query paramters name and related error codes when invalid

* Fix missing part of error msg

* Precise valid date formats

* Add supported format for date filters in open-api.yml

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0060-tasks-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* fix typo

* Update open-api.yaml

Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update text/0060-tasks-api.md

* Update text/0060-tasks-api.md

* Tasks API - Add task deletion capability (#198)

* Describes task deletion feature additions

* Update open-api.yml

* Update telemetry-policies

* Update originalQuery to originalFilters

* Follow title structure

* Introduce review feedback

* Branch the right components

* Update text/0060-tasks-api.md

Co-authored-by: Clément Renault <renault.cle@gmail.com>

Co-authored-by: Clément Renault <renault.cle@gmail.com>

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>
Co-authored-by: Clément Renault <renault.cle@gmail.com>

* Update the dump support table with the dumpv6 (#199)

* Settings API - Specify handling of null values when patching settings (#202)

* Specify behavior when patching a setting with partial values and null values.

- Partial value: the setting is unchanged.
- null value: the setting is reset to default.

* Settings: Update error messages to accept the null value

* Add missing task types

* Add canceledTasks details

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clément Renault <renault.cle@gmail.com>
Co-authored-by: cvermand <33010418+bidoubiwa@users.noreply.github.com>
Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
Co-authored-by: Many the fish <legendre.maxime.isn@gmail.com>
Co-authored-by: Amélie <alallema@users.noreply.github.com>
Co-authored-by: Louis Dureuil <louis.dureuil@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
⚠️ Breaking Introduce a breaking change. Implemented Feature specification has been implemented. OpenAPI Update OpenAPI specification. Q4:2022 Telemetry Update the telemetry collect. v0.30
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants