Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix in masked_loading ROI table reading in case that obs label ID is string of a float #783

Open
nrepina opened this issue Jul 9, 2024 · 0 comments · May be fixed by #786
Open

Bug fix in masked_loading ROI table reading in case that obs label ID is string of a float #783

nrepina opened this issue Jul 9, 2024 · 0 comments · May be fixed by #786
Assignees

Comments

@nrepina
Copy link

nrepina commented Jul 9, 2024

fractal_tasks_core/masked_loading.py line 112 assumes that the label ID is stored as a string of an integer (e.g. '1') in ROI table obs, however this might not be the case. The Cellpose task output indeed stores label ID in this format, but the RDCnet task stores as a string of a float (i.e. '1.0'). Thus table reading of an RDCnet output table throws the error below (see traceback below).
Tldr; int('1') = 1, whereas int('1.0') = ValueError: invalid literal for int() with base 10: '1.0'

To generalize, recommended fix is to change lines 112 to:

label_value = int(float(ROI_table.obs[column_name][ROI_positional_index]))

Traceback:


TASK ERROR: Task name: Rdcnet Segmentation 3d Task, position in Workflow: 5
TRACEBACK:
Traceback (most recent call last):
  File "/tungstenfs/landing/fractal/20230626_deployment_130/server-envs/fractal-server-2.1.0/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 116, in run_single_task
    raise e
  File "/tungstenfs/landing/fractal/20230626_deployment_130/server-envs/fractal-server-2.1.0/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 105, in run_single_task
    _call_command_wrapper(
  File "/tungstenfs/landing/fractal/20230626_deployment_130/server-envs/fractal-server-2.1.0/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 52, in _call_command_wrapper
    raise TaskExecutionError(err)
fractal_server.app.runner.exceptions.TaskExecutionError: 2024-07-09 10:29:26,473; INFO; START rdcnet_segmentation_3d task
2024-07-09 10:29:26,474; INFO; zarr_url='/tungstenfs/scratch/gliberal/Users/repinico/Yokogawa/20240419_DCFLEX/20240708_d3p5TestData/fractal_output/d3p5/dcflexr1.zarr/C/05/0'
2024-07-09 10:29:26,501; INFO; NGFF image has num_levels=5
2024-07-09 10:29:26,501; INFO; NGFF image has coarsening_xy=2
2024-07-09 10:29:26,501; INFO; NGFF image has full-res pixel sizes [0.6, 0.21666666666666667, 0.21666666666666667]
2024-07-09 10:29:26,501; INFO; NGFF image has level-0 pixel sizes (0.6, 0.21666666666666667, 0.21666666666666667)
2024-07-09 10:29:26,511; INFO; data_zyx.shape=(146, 3077, 7730)
2024-07-09 10:29:26,622; INFO; ROI table at /tungstenfs/scratch/gliberal/Users/repinico/Yokogawa/20240419_DCFLEX/20240708_d3p5TestData/fractal_output/d3p5/dcflexr1.zarr/C/05/0/tables/org_ROI_table has attrs: {'encoding-type': 'anndata', 'encoding-version': '0.1.0', 'fractal_table_version': '1', 'instance_key': 'label', 'region': {'path': '../labels/org'}, 'type': 'masking_roi_table'}
2024-07-09 10:29:26,622; INFO; ROI table can be used for masked loading
2024-07-09 10:29:26,622; INFO; use_masks=True
2024-07-09 10:29:26,650; INFO; Helper function `prepare_label_group` returned label_group=<zarr.hierarchy.Group '/labels/nuc'>
2024-07-09 10:29:26,650; INFO; Output label path: /tungstenfs/scratch/gliberal/Users/repinico/Yokogawa/20240419_DCFLEX/20240708_d3p5TestData/fractal_output/d3p5/dcflexr1.zarr/C/05/0/labels/nuc/0
2024-07-09 10:29:26,655; INFO; mask will have shape (146, 3077, 7730) and chunks ((1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1), (1000, 1000, 1000, 77), (1000, 1000, 1000, 1000, 1000, 1000, 1000, 730))
2024-07-09 10:29:26,655; INFO; 3D segmentation will be executed
2024-07-09 10:29:26.726435: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-07-09 10:29:28.264298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 30986 MB memory:  -> device: 0, name: Tesla V100-SXM2-32GB, pci bus id: 0000:1a:00.0, compute capability: 7.0
2024-07-09 10:29:29,599; INFO; Now starting loop over 12 ROIs
2024-07-09 10:29:29,599; INFO; Now processing ROI 1/12
2024-07-09 10:29:29,784; INFO; [_preprocess_input] image_array.shape=(1, 146, 48, 112)
2024-07-09 10:29:29,784; INFO; [_preprocess_input] region=(slice(0, 146, None), slice(0, 48, None), slice(7119, 7231, None))
2024-07-09 10:29:29,796; INFO; [_preprocess_input] ROI_table_path='/tungstenfs/scratch/gliberal/Users/repinico/Yokogawa/20240419_DCFLEX/20240708_d3p5TestData/fractal_output/d3p5/dcflexr1.zarr/C/05/0/tables/org_ROI_table'
2024-07-09 10:29:29,796; INFO; [_preprocess_input] attrs.asdict()={'encoding-type': 'anndata', 'encoding-version': '0.1.0', 'fractal_table_version': '1', 'instance_key': 'label', 'region': {'path': '../labels/org'}, 'type': 'masking_roi_table'}
Traceback (most recent call last):
  File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib/python3.9/site-packages/isit/fractal/rdcnet_segmentation_3d.py", line 101, in <module>
    run_fractal_task(
  File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/fractal_tasks_core/tasks/_utils.py", line 79, in run_fractal_task
    metadata_update = task_function(**pars)
  File "pydantic/decorator.py", line 40, in pydantic.decorator.validate_arguments.validate.wrapper_function
  File "pydantic/decorator.py", line 134, in pydantic.decorator.ValidatedFunction.call
  File "pydantic/decorator.py", line 206, in pydantic.decorator.ValidatedFunction.execute
  File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib/python3.9/site-packages/isit/fractal/rdcnet_segmentation_3d.py", line 77, in rdcnet_segmentation_3d
    return rdcnet_segmentation(
  File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/isit/fractal/rdcnet_segmentation.py", line 495, in rdcnet_segmentation
    new_label_img = seg_fun(**seg_fun_kwarg)
  File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/isit/fractal/rdcnet_segmentation.py", line 439, in _do_seg3d
    new_label_img = masked_loading_wrapper(
  File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/fractal_tasks_core/masked_loading.py", line 238, in masked_loading_wrapper
    ) = _preprocess_input(image_array, **preprocessing_kwargs)
  File "/tungstenfs/landing/fractal/20230626_deployment_130/FRACTAL_TASK_DIR/.fractal/isit0.1.3/venv/lib64/python3.9/site-packages/fractal_tasks_core/masked_loading.py", line 112, in _preprocess_input
    label_value = int(ROI_table.obs[column_name][ROI_positional_index])
ValueError: invalid literal for int() with base 10: '1.0'
@nrepina nrepina changed the title Bug fix in masked_loading ROI table reading in case that obs name is string not float Bug fix in masked_loading ROI table reading in case that obs label ID is string of a float Jul 9, 2024
@jluethi jluethi linked a pull request Jul 12, 2024 that will close this issue
1 task
@jluethi jluethi self-assigned this Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
2 participants