From 275a26e817d6eebe727729ce67738654b470101d Mon Sep 17 00:00:00 2001 From: Tommaso Comparin <3862206+tcompa@users.noreply.github.com> Date: Fri, 3 May 2024 10:18:54 +0200 Subject: [PATCH 1/4] Remove warnings --- docs/version_2/image_list.md | 2 -- docs/version_2/index.md | 2 -- docs/version_2/tasks.md | 4 +--- 3 files changed, 1 insertion(+), 7 deletions(-) diff --git a/docs/version_2/image_list.md b/docs/version_2/image_list.md index 87b678a..7979234 100644 --- a/docs/version_2/image_list.md +++ b/docs/version_2/image_list.md @@ -2,8 +2,6 @@ layout: default --- -> **WARNING: This page is still in-progress, its contents may change based on the latest developments.** - While applying a processing workflow to a given dataset, Fractal keeps a list of all the OME-Zarr images it is processing. In this page we describe the concepts of [images](#images) and [filters](#filters) - see also the [examples section](#examples). ## Images diff --git a/docs/version_2/index.md b/docs/version_2/index.md index 3343e4e..1acfe5f 100644 --- a/docs/version_2/index.md +++ b/docs/version_2/index.md @@ -8,5 +8,3 @@ This section describes the main concepts introduced with the new Fractal version 1. The [new dataset image list](./image_list.md); 2. The [new definition of tasks](./tasks.md). - -> **WARNING: This page is still in-progress, its contents may change based on the latest developments.** \ No newline at end of file diff --git a/docs/version_2/tasks.md b/docs/version_2/tasks.md index 5e0f64c..8ac0a60 100644 --- a/docs/version_2/tasks.md +++ b/docs/version_2/tasks.md @@ -4,8 +4,6 @@ layout: default # Tasks -> **WARNING: This page is still in-progress, its contents may change based on the latest developments.** - Fractal v2 brings a large refactor to the task architecture to make tasks more flexible and allow for building more complex workflows, while also simplifying the task API. This page gives an overview over the different types of Fractal tasks, their input and output API and the elements that go into the Fractal task list. ## Task Types @@ -42,7 +40,7 @@ Tasks can optionally return updates to the image list and/or [new dataset filter For tasks that create new images or edit relevant image properties, `task_output` must include an `image_list_updates` property so the server can update its metadata about that image. -Task_outputs with image list updates are returned as a dictionary that contains the `image_list_updates` key and a list containing the updates to individual images. The updates need to be for unique `zarr_url`s and each update needs to contain the `zarr_url` of the image it’s providing an update for. Additionally, they can provide an `origin` key, an `attributes` key and a `types` key. The `origin` key describes the `zarr_url` of another image already in the image list and will take the existing attributes and types from that image. Attributes and types can also be directly set by a task. +Task outputs with image list updates are returned as a dictionary that contains the `image_list_updates` key and a list containing the updates to individual images. The updates need to be for unique `zarr_url`s and each update needs to contain the `zarr_url` of the image it’s providing an update for. Additionally, they can provide an `origin` key, an `attributes` key and a `types` key. The `origin` key describes the `zarr_url` of another image already in the image list and will take the existing attributes and types from that image. Attributes and types can also be directly set by a task. Here's an example of `task_output`: ```python From 570f34da4e9a02b3be3595673f78b60ba43297ac Mon Sep 17 00:00:00 2001 From: Tommaso Comparin <3862206+tcompa@users.noreply.github.com> Date: Fri, 3 May 2024 10:19:01 +0200 Subject: [PATCH 2/4] Add note --- docs/version_2/tasks.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/version_2/tasks.md b/docs/version_2/tasks.md index 8ac0a60..0a63fdc 100644 --- a/docs/version_2/tasks.md +++ b/docs/version_2/tasks.md @@ -40,6 +40,8 @@ Tasks can optionally return updates to the image list and/or [new dataset filter For tasks that create new images or edit relevant image properties, `task_output` must include an `image_list_updates` property so the server can update its metadata about that image. +> NOTE: if both `image_list_updates` and `image_list_removals` are empty, in the task output, then `fractal-server` includes all the filtered image list in `image_list_updates`, so that they are updated with the appropriate `types` (see also [the image-list page](./image_list.md#image-types)). + Task outputs with image list updates are returned as a dictionary that contains the `image_list_updates` key and a list containing the updates to individual images. The updates need to be for unique `zarr_url`s and each update needs to contain the `zarr_url` of the image it’s providing an update for. Additionally, they can provide an `origin` key, an `attributes` key and a `types` key. The `origin` key describes the `zarr_url` of another image already in the image list and will take the existing attributes and types from that image. Attributes and types can also be directly set by a task. Here's an example of `task_output`: From c2710e48e8ee9a3a28bb0a8f8ea7539bc0bf9699 Mon Sep 17 00:00:00 2001 From: Tommaso Comparin <3862206+tcompa@users.noreply.github.com> Date: Fri, 3 May 2024 10:19:49 +0200 Subject: [PATCH 3/4] Fix broken link --- docs/version_2/image_list.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/version_2/image_list.md b/docs/version_2/image_list.md index 7979234..3c03a43 100644 --- a/docs/version_2/image_list.md +++ b/docs/version_2/image_list.md @@ -10,7 +10,8 @@ Each entry in the image list is defined by a unique `zarr_url` property (the ful ### Image types -Image types are boolean properties that allow to split the image list into different sub-lists (e.g. the `is_3D` type for 3D/2D images, or the `illumination_corrected` type for raw/corrected images when illumination correction was not run in-place). Types can be set both by the task manifest (e.g. after an MIP task, the resulting images always have the type `is_3D` set to `False` - see [task-manifest section](#dataset-filters)) as well as from within an individual task (see [task-API/output section](#output-api)). +Image types are boolean properties that allow to split the image list into different sub-lists (e.g. the `is_3D` type for 3D/2D images, or the `illumination_corrected` type for raw/corrected images when illumination correction was not run in-place). Types can be set both by the task manifest (e.g. after an MIP task, the resulting images always have the type `is_3D` set to `False` - see [task-manifest section](#dataset-filters)) as well as from within an individual task (see [task-API/output section]( +./tasks.md#output-api)). Note: whenever applying filters to the image list, the absence of a type corresponds to false by default. From 4f306e24b9c2bb8d8b0c10c17d30a21627349eb0 Mon Sep 17 00:00:00 2001 From: Tommaso Comparin <3862206+tcompa@users.noreply.github.com> Date: Fri, 3 May 2024 10:20:40 +0200 Subject: [PATCH 4/4] Remove newline --- docs/version_2/image_list.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/version_2/image_list.md b/docs/version_2/image_list.md index 3c03a43..3cbce0d 100644 --- a/docs/version_2/image_list.md +++ b/docs/version_2/image_list.md @@ -10,8 +10,7 @@ Each entry in the image list is defined by a unique `zarr_url` property (the ful ### Image types -Image types are boolean properties that allow to split the image list into different sub-lists (e.g. the `is_3D` type for 3D/2D images, or the `illumination_corrected` type for raw/corrected images when illumination correction was not run in-place). Types can be set both by the task manifest (e.g. after an MIP task, the resulting images always have the type `is_3D` set to `False` - see [task-manifest section](#dataset-filters)) as well as from within an individual task (see [task-API/output section]( -./tasks.md#output-api)). +Image types are boolean properties that allow to split the image list into different sub-lists (e.g. the `is_3D` type for 3D/2D images, or the `illumination_corrected` type for raw/corrected images when illumination correction was not run in-place). Types can be set both by the task manifest (e.g. after an MIP task, the resulting images always have the type `is_3D` set to `False` - see [task-manifest section](#dataset-filters)) as well as from within an individual task (see [task-API/output section](./tasks.md#output-api)). Note: whenever applying filters to the image list, the absence of a type corresponds to false by default.