-
Notifications
You must be signed in to change notification settings - Fork 271
v1.12: New /batches route
#3072
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
e68d1fa
add new /batches route
guimachiavelli 541a5a7
add link to sidebar
guimachiavelli 8b36fea
add more mentions to batches in other pages, fix filename
guimachiavelli d43920f
fix wrong parameter name
guimachiavelli 7a6e010
first draft ready
guimachiavelli e84dc50
Apply suggestions from code review
guimachiavelli cae51e3
Apply suggestions from code review
guimachiavelli f0141e7
address reviewer feeedback
guimachiavelli 2597bdf
improve wording
guimachiavelli fafb93a
Update learn/async/asynchronous_operations.mdx
guimachiavelli e5a7fdb
Update reference/api/batches.mdx
guimachiavelli File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,250 @@ | ||
| --- | ||
| title: Batches — Meilisearch API reference | ||
| description: The /batches route allows you to monitor how Meilisearch is grouping and processing asynchronous operations. | ||
| --- | ||
|
|
||
| # Batches | ||
|
|
||
| The `/batches` route gives information about the progress of batches of [asynchronous operations](/learn/async/asynchronous_operations). | ||
|
|
||
| ## Batch object | ||
|
|
||
| ```json | ||
| { | ||
| "uid": 0, | ||
| "details": { | ||
| "receivedDocuments": 6, | ||
| "indexedDocuments": 6 | ||
| }, | ||
| "stats": { | ||
| "totalNbTasks": 1, | ||
| "status": { | ||
| "succeeded": 1 | ||
| }, | ||
| "types": { | ||
| "documentAdditionOrUpdate": 1 | ||
| }, | ||
| "indexUids": { | ||
| "INDEX_NAME": 1 | ||
| } | ||
| }, | ||
| "duration": "PT0.250518S", | ||
| "startedAt": "2024-12-10T15:20:30.18182Z", | ||
| "finishedAt": "2024-12-10T15:20:30.432338Z", | ||
| "progress": { | ||
| "steps": [ | ||
| { | ||
| "currentStep": "extracting words", | ||
| "finished": 2, | ||
| "total": 9, | ||
| }, | ||
| { | ||
| "currentStep": "document", | ||
| "finished": 30546, | ||
| "total": 31944, | ||
| } | ||
| ], | ||
| "percentage": 32.8471 | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### `uid` | ||
|
|
||
| **Type**: Integer<br /> | ||
| **Description**: Unique sequential identifier of the batch. Starts at `0` and increases by one for every new patch. | ||
|
|
||
| ### `details` | ||
|
|
||
| **Type**: Object<br /> | ||
| **Description**: Basic information on the types tasks in a batch. Consult the [task object reference](/reference/api/tasks#details) for an exhaustive list of possible values. | ||
|
|
||
| ### `progress` | ||
|
|
||
| **Type**: Object<br /> | ||
| **Description**: Object containing two fields: `steps` and `percentage`. Once Meilisearch has fully processed a batch, its `progress` is set to `null`. | ||
|
|
||
| #### `steps` | ||
|
|
||
| Information about the current operations Meilisearch is performing in this batch. A step may consist of multiple substeps. | ||
|
|
||
| | Name | Description | | ||
| | :-----------------| :------------------------------------------------- | | ||
| | **`currentStep`** | A string describing the operation | | ||
| | **`total`** | The total number of operations in the step | | ||
| | **`finished`** | The number of operations Meilisearch has completed | | ||
|
|
||
| <Capsule intent="tip" title=""> | ||
| If Meilisearch is taking longer than expected to process a batch, monitor the `steps` array. If the `finished` field of the last item in the `steps` array does not update, Meilisearch may be stuck. | ||
| </Capsule> | ||
|
|
||
| #### `percentage` | ||
|
|
||
| The percentage of completed operations, calculated from all current steps and substeps. This value is a rough estimate and may not always reflect the current state of the batch due to how different steps are processed more quickly than others. | ||
|
|
||
| ### `stats` | ||
|
|
||
| **Type**: Object<br /> | ||
| **Description**: Detailed information on the payload of all tasks in a batch. | ||
|
|
||
| #### `totalNbTasks` | ||
|
|
||
| Number of tasks in the batch. | ||
|
|
||
| #### `status` | ||
|
|
||
| Object listing the [status of each task](/reference/api/tasks#status) in the batch. Contains five keys whose values correspond to the number of tasks with that status. | ||
|
|
||
| #### `types` | ||
|
|
||
| List with the `types` of tasks contained in the batch. | ||
|
|
||
| #### `indexUids` | ||
|
|
||
| List of the number of tasks in the batch separated by the indexes they affect. | ||
|
|
||
| ### `duration` | ||
|
|
||
| **Type**: String<br /> | ||
| **Description**: The total elapsed time the batch spent in the `processing` state, in [ISO 8601](https://www.ionos.com/digitalguide/websites/web-development/iso-8601/) format. Set to `null` while the batch is processing tasks | ||
|
|
||
| ### `startedAt` | ||
|
|
||
| **Type**: String<br /> | ||
| **Description**: The date and time when the batch began `processing`, in [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt) format | ||
|
|
||
| ### `finishedAt` | ||
|
|
||
| **Type**: String<br /> | ||
| **Description**: The date and time when the tasks finished `processing`, whether `failed`, `succeeded`, or `canceled`, in [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt) format | ||
|
|
||
| ## Get batches | ||
|
|
||
| <RouteHighlighter method="GET" route="/batches"/> | ||
|
|
||
| List all batches, regardless of index. The batch objects are contained in the `results` array. | ||
|
|
||
| Batches are always returned in descending order of `uid`. This means that by default, **the most recently created batch objects appear first**. | ||
|
|
||
| Batch results are [paginated](/learn/async/paginating_tasks) and can be [filtered](/learn/async/filtering_tasks) with query parameters. | ||
|
|
||
| <Capsule intent="danger" title="`/batches` parameters and tasks"> | ||
| Some query parameters for `/batches`, such as `uids` and `statuses`, target tasks instead of batches. | ||
|
|
||
| For example, `?uids=0` returns a batch containing the task with a `taskUid` equal to `0`, instead of a batch with a `batchUid` equal to `0`. | ||
| </Capsule> | ||
|
|
||
| ### Query parameters | ||
|
|
||
| | Query Parameter | Default Value | Description | | ||
| | ---------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------- | | ||
| | **`uids`** | `*` (all task uids) | Select batches containing the tasks with the specified `uid`s. Separate multiple task `uids` with a comma (`,`) | | ||
| | **`batchUids`** | `*` (all batch uids) | Filter batches by their `uid`. Separate multiple batch `uids` with a comma (`,`) | | ||
| | **`indexUids`** | `*` (all indexes) | Select batches containing tasks affecting the specified indexes. Separate multiple `indexUids` with a comma (`,`) | | ||
| | **`statuses`** | `*` (all statuses) | Select batches containing tasks with the specified `status`. Separate multiple task `statuses` with a comma (`,`) | | ||
| | **`types`** | `*` (all types) | Select batches containing tasks with the specified `type`. Separate multiple task `types` with a comma (`,`) | | ||
| | **`limit`** | `20` | Number of batches to return | | ||
| | **`from`** | `uid` of the last created batch | `uid` of the first batch returned | | ||
| | **`reverse`** | `false` | If `true`, returns results in the reverse order, from oldest to most recent | | ||
| | **`beforeEnqueuedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `enqueuedAt` field | | ||
| | **`beforeStartedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `startedAt` field | | ||
| | **`beforeFinishedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `finishedAt` field | | ||
| | **`afterEnqueuedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `enqueuedAt` field | | ||
| | **`afterStartedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `startedAt` field | | ||
| | **`afterFinishedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `finishedAt` field | | ||
|
|
||
| ### Response | ||
|
|
||
| | Name | Type | Description | | ||
| | :------------ | :------ | :----------------------------------------------------------------------------------------------------------------------------- | | ||
| | **`results`** | Array | An array of [batch objects](#batch-object) | | ||
| | **`total`** | Integer | Total number of batches matching the filter or query | | ||
| | **`limit`** | Integer | Number of batches returned | | ||
| | **`from`** | Integer | `uid` of the first batch returned | | ||
| | **`next`** | Integer | Value passed to `from` to view the next "page" of results. When the value of `next` is `null`, there are no more tasks to view | | ||
|
|
||
| ### Example | ||
|
|
||
| <CodeSamples id="get_all_batches_1" /> | ||
|
|
||
| #### Response: `200 Ok` | ||
|
|
||
| ```json | ||
| { | ||
| "results": [ | ||
| { | ||
| "uid": 2, | ||
| "details": { | ||
| "stopWords": [ | ||
| "of", | ||
| "the" | ||
| ] | ||
| }, | ||
| "progress": null, | ||
| "stats": { | ||
| "totalNbTasks": 1, | ||
| "status": { | ||
| "succeeded": 1 | ||
| }, | ||
| "types": { | ||
| "settingsUpdate": 1 | ||
| }, | ||
| "indexUids": { | ||
| "INDEX_NAME": 1 | ||
| } | ||
| }, | ||
| "duration": "PT0.110083S", | ||
| "startedAt": "2024-12-10T15:49:04.995321Z", | ||
| "finishedAt": "2024-12-10T15:49:05.105404Z" | ||
| } | ||
| ], | ||
| "total": 3, | ||
| "limit": 1, | ||
| "from": 2, | ||
| "next": 1 | ||
| } | ||
| ``` | ||
|
|
||
| ## Get one batch | ||
|
|
||
| <RouteHighlighter method="GET" route="/batches/{batch_uid}"/> | ||
|
|
||
| Get a single batch. | ||
|
|
||
| ### Path parameters | ||
|
|
||
| | Name | Type | Description | | ||
| | :---------------- | :----- | :----------------------------------- | | ||
| | **`batch_uid`** * | String | [`uid`](#uid) of the requested batch | | ||
|
|
||
| ### Example | ||
|
|
||
| <CodeSamples id="get_batch_1" /> | ||
|
|
||
| #### Response: `200 Ok` | ||
|
|
||
| ```json | ||
| { | ||
| "uid": 1, | ||
| "details": { | ||
| "receivedDocuments": 1, | ||
| "indexedDocuments": 1 | ||
| }, | ||
| "progress": null, | ||
| "stats": { | ||
| "totalNbTasks": 1, | ||
| "status": { | ||
| "succeeded": 1 | ||
| }, | ||
| "types": { | ||
| "documentAdditionOrUpdate": 1 | ||
| }, | ||
| "indexUids": { | ||
| "INDEX_NAME": 1 | ||
| } | ||
| }, | ||
| "duration": "PT0.364788S", | ||
| "startedAt": "2024-12-10T15:48:49.672141Z", | ||
| "finishedAt": "2024-12-10T15:48:50.036929Z" | ||
| } | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.