TypeError: list indices must be integers or slices, not str

[jcohenadad]

Issue description

Terminal output (from Wandb): output.log

log: log.log

version_info.log

Config file

{
    "command": "train",
    "gpu_ids": [4],
    "path_output": "seg_lesion_output",
    "model_name": "seg_lesion_model",
    "debugging": true,
    "object_detection_params": {
        "object_detection_path": null,
        "safety_factor": [1.0, 1.0, 1.0]
    },
    "wandb": {
        "wandb_api_key": "9095e2bc9e4ab445d478c9c8a81759ae908be8c6",
        "project_name": "basel-mp2rage-lesion",
        "group_name": "my_group",
        "run_name": "run-1",
        "log_grads_every": 100
    },
    "loader_parameters": {
        "path_data": ["/home/GRAMES.POLYMTL.CA/p101317/data_nvme_p101317/data_seg_mp2rage_20221212_175444/data_processed_lesionseg"],
        "subject_selection": {
            "n": [],
            "metadata": [],
            "value": []
        },
        "target_suffix": [["_lesion-manual", "_lesion-manual2"]],
        "extensions": [],
        "roi_params": {
            "suffix": null,
            "slice_filter_roi": null
        },
        "contrast_params": {
            "training_validation": ["UNIT1"],
            "testing": ["UNIT1"],
            "balance": {}
        },
        "slice_filter_params": {
            "filter_empty_mask": false,
            "filter_empty_input": false
        },
        "slice_axis": "axial",
        "multichannel": false,
        "soft_gt": false
    },
    "split_dataset": {
        "fname_split": null,
        "random_seed": 42,
        "center_test": [],
        "method": "per_patient",
        "balance": null,
        "train_fraction": 0.75,
        "test_fraction": 0.2
    },
    "training_parameters": {
        "batch_size": 4,
        "loss": {
            "name": "DiceLoss"
        },
        "training_time": {
            "num_epochs": 100,
            "early_stopping_patience": 200,
            "early_stopping_epsilon": 0.001
        },
        "scheduler": {
            "initial_lr": 1e-3,
            "lr_scheduler": {
                "name": "CosineAnnealingLR",
                "base_lr": 1e-5,
                "max_lr": 1e-3
            }
        },
        "balance_samples": {
            "applied": false,
            "type": "gt"
        },
        "mixup_alpha": null,
        "transfer_learning": {
            "retrain_model": null,
            "retrain_fraction": 1.0,
            "reset": true
        }
    },
    "default_model": {
        "name": "Unet",
        "dropout_rate": 0.3,
        "bn_momentum": 0.1,
        "depth": 4,
        "is_2d": false,
        "final_activation": "sigmoid"
    },
    "uncertainty": {
        "epistemic": false,
        "aleatoric": false,
        "n_it": 0
    },
    "postprocessing": {"binarize_prediction": {"thr": 0.5}},
    "evaluation_parameters": {},
    "transformation": {
        "Resample": {
            "wspace": 1.0,
            "hspace": 1.0,
            "dspace": 1.0
        },
        "CenterCrop": {
            "size": [32, 64, 128]
        },
        "RandomReverse": {
            "applied_to": ["im", "gt"],
            "dataset_type": ["training"]
        },
        "RandomAffine": {
            "degrees": 10,
            "scale": [0.2, 0.2, 0.2],
            "translate": [0.2, 0.2, 0.2],
            "applied_to": ["im", "gt"],
            "dataset_type": ["training"]
        },
        "ElasticTransform": {
            "alpha_range": [25.0, 35.0],
            "sigma_range": [3.5, 5.5],
            "p": 0.5,
            "applied_to": ["im", "gt"],
            "dataset_type": ["training"]
        },
        "RandomGamma": {
            "log_gamma_range": [-1.5, 1.5],
            "p": 0.5,
            "applied_to": ["im"],
            "dataset_type": ["training"]
        },
        "NumpyToTensor": {},
        "NormalizeInstance": {
            "applied_to": ["im"]
        }
    },
    "Modified3DUNet": {
        "applied": true,
        "length_3D": [32, 64, 128],
        "stride_3D": [32, 64, 128],
        "attention": false,
        "n_filters": 8
    }
}

Possibly related to #815

[jcohenadad]

Running debugging mode, the issue arrives here:

ivadomed/ivadomed/transforms.py

Line 719 in 9bb4e7f

metadata[MetadataKW.ROTATION] = [angle, axes]

Inspecting the variables:

[angle, axes]
[-0.05377395512467591, [0, 2]]
metadata[MetadataKW.ROTATION] = [angle, axes]
Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_exec2.py", line 3, in Exec
    exec(exp, global_vars, local_vars)
  File "<input>", line 1, in <module>
TypeError: list indices must be integers or slices, not str

[jcohenadad]

Interestingly there is no "str":

type(angle)
<class 'float'>
type(axes)
<class 'list'>
type(axes[0])
<class 'int'>
type(axes[1])
<class 'int'>

[jcohenadad]

Ah! wait, the error stems from the call to:

metadata[MetadataKW.ROTATION]

Because MetadataKW.ROTATION is a string.

[jcohenadad]

From #815, the culprit was the presence of the field "NumpyToTensor" in the data augmentation, which is indeed listed in my original config file (#1213 (comment)).

However, I tried with another config file, based on the latest ivadomed version, and I still observe the error.

More recent config file

{
    "command": "train",
    "gpu_ids": [4],
    "path_output": "model_seg_lesion_mp2rage_",
    "model_name": "model_seg_lesion_mp2rage",
    "debugging": true,
    "object_detection_params": {
        "object_detection_path": null,
        "safety_factor": [1.0, 1.0, 1.0]
    },
    "wandb": {
        "wandb_api_key": "9095e2bc9e4ab445d478c9c8a81759ae908be8c6",
        "project_name": "basel-mp2rage-lesion",
        "group_name": "my_group",
        "run_name": "run-3",
        "log_grads_every": 100
    },
    "loader_parameters": {
        "path_data": ["/Users/julien/temp/model_seg_basel/data_processed_lesionseg_small"],
        "subject_selection": {"n": [], "metadata": [], "value": []},
        "target_suffix": ["_lesion-manual", "_lesion-manual3"],
        "extensions": [".nii.gz"],
        "roi_params": {
            "suffix": null,
            "slice_filter_roi": null
        },
        "contrast_params": {
            "training_validation": ["UNIT1"],
            "testing": ["UNIT1"],
            "balance": {}
        },
        "slice_filter_params": {
            "filter_empty_mask": false,
            "filter_empty_input": true
        },
        "slice_axis": "axial",
        "multichannel": false,
        "soft_gt": false,
        "bids_validate": true
    },
    "split_dataset": {
        "fname_split": null,
        "random_seed": 42,
        "split_method" : "participant_id",
        "data_testing": {"data_type": null, "data_value":[]},
        "balance": null,
        "train_fraction": 0.6,
        "test_fraction": 0.2
    },
    "training_parameters": {
        "batch_size": 4,
        "loss": {
            "name": "DiceLoss"
        },
        "training_time": {
            "num_epochs": 50,
            "early_stopping_patience": 50,
            "early_stopping_epsilon": 0.001
        },
        "scheduler": {
            "initial_lr": 0.001,
            "lr_scheduler": {
                "name": "CosineAnnealingLR",
                "base_lr": 1e-5,
                "max_lr": 1e-3
            }
        },
        "balance_samples": {
            "applied": false,
            "type": "gt"
        },
        "mixup_alpha": null,
        "transfer_learning": {
            "retrain_model": null,
            "retrain_fraction": 1.0,
            "reset": true
        }
    },
    "default_model": {
        "name": "Unet",
        "dropout_rate": 0.3,
        "bn_momentum": 0.1,
        "final_activation": "sigmoid",
        "depth": 3
    },
    "FiLMedUnet": {
        "applied": false,
        "metadata": "contrasts",
        "film_layers": [0, 1, 0, 0, 0, 0, 0, 0, 0, 0]
    },
    "Modified3DUNet": {
        "applied": true,
        "length_3D": [32, 64, 128],
        "stride_3D": [32, 64, 128],
        "attention": false,
        "n_filters": 8
    },
    "uncertainty": {
        "epistemic": false,
        "aleatoric": false,
        "n_it": 0
    },
    "postprocessing": {
        "remove_noise": {"thr": -1},
        "keep_largest": {},
        "binarize_prediction": {"thr": 0.5},
        "uncertainty": {"thr": -1, "suffix": "_unc-vox.nii.gz"},
        "fill_holes": {},
        "remove_small": {"unit": "vox", "thr": 3}
    },
    "evaluation_parameters": {
        "target_size": {"unit": "vox", "thr": [20, 100]},
        "overlap": {"unit": "vox", "thr": 3}
    },
    "transformation": {
        "Resample":
        {
            "hspace": 1,
            "wspace": 1,
            "dspace": 1
        },
        "CenterCrop": {
            "size": [32, 64, 128]},
        "RandomAffine": {
            "degrees": 10,
            "scale": [0.2, 0.2, 0.2],
            "translate": [0.2, 0.2, 0.2],
            "applied_to": ["im", "gt"],
            "dataset_type": ["training"]
        },
        "ElasticTransform": {
			"alpha_range": [28.0, 30.0],
			"sigma_range":  [3.5, 4.5],
			"p": 0.1,
            "applied_to": ["im", "gt"],
            "dataset_type": ["training"]
        },
      "NormalizeInstance": {"applied_to": ["im"]}
    }
}

[mariehbourget]

From #815, the culprit was the presence of the field "NumpyToTensor" in the data augmentation, which is indeed listed in my original config file (#1213 (comment)).

However, I tried with another config file, based on the latest ivadomed version, and I still observe the error.

Based on #830, old config files with the "NympyToTensor" key are supposed to be retrocompatible, so I don't think this is the issue here.
Let me test it on my side to reproduce the issue and I'll get back to you asap.

[mariehbourget]

Ah! wait, the error stems from the call to:

metadata[MetadataKW.ROTATION]

Because MetadataKW.ROTATION is a string.

Thanks @jcohenadad for reporting this issue.

I was able to reproduce the error with a 3D training.
What happens is:

• metadata is a list of SampleMetadata object
• Because it's a list, it cannot be accessed with a string
• From my understanding, what we are trying to access here is metadata[0][MetadataKW.ROTATION] and not metadata[MetadataKW.ROTATION].
• However, this issue is not present with 2D training, where metadata is a SampleMetadata object itself (and not a list) at this point.

I have little experience with 3D training and transforms, so I don't know why there is a difference between 2D and 3D. I'll continue the investigation to find a fix and evaluate the implications, but it may take some time.

Note that the same error happens for other transforms, for example, if I switch the order of "RandomAffine" and "ElasticTransform", I get the same error for elastic transforms here:

ivadomed/ivadomed/transforms.py

Line 899 in 9bb4e7f

metadata[MetadataKW.ELASTIC] = [None, None]

File "/home/mhbourget/code/ivadomed/ivadomed/transforms.py", line 899, in __call__
    metadata[MetadataKW.ELASTIC] = [None, None]
TypeError: list indices must be integers or slices, not str

The only work around I have at this moment is to do the training without data augmentations i.e. without "RandomAffine" and "ElasticTransform" in this case.

[mariehbourget]

As a side note, we should make sure to add a test with the fix so we can catch this in the future.

[jcohenadad]

Thank you very much for the excellent detective work @mariehbourget 🙏 . I'm surprised this hasn't been caught before.

[dyt811]

Seems like more testing with 3D test pathway will be required. I will make a note to test that part as well in the Loader V2.

[mariehbourget]

Ok, I found where this is happening!
The issue is complicated but the fix should be easy.

In short:

• In 3D, the metadata variable in RandomAffine is a list of SampleMetadata object and directly cause the error message reported (as mentioned above).
• In 2D, it does not happen because the metadata variable is a SampleMetadata object (not a list).
• Up until the call for transforms (here in 3D and here in 2D) the metadata is a list.
• Looking closely to data augmentation transforms (ex: RandomAffine or ElasticTransform), many of them have the 2 following wrapper: multichannel_capable and two_dim_compatible.
• In multichannel_capable, when the sample is a list, the metadata is returned as a SampleMetadata object, whereas when the sample is not a list, the metadata is returned "as is" (as a list).
• In the call for transforms (again here in 3D and here in 2D), the sample is a numpy array in 3D and a list in 2D
• Conclusion: the difference of type for the sample is causing the difference in type for metadata when going through multichannel capable.

The bug is fairly recent and was introduced in PR #1169 in July.

• In 3D: when calling the transform here, stack_input is a numpy array (causing the issue) where before it was a list called seg_pair['input'].
• In 2D: the equivalent operation is done here where stack_input is converted to a list when calling the transform (hence no issue).

As I said, the fix should be simple, but I will require some time to test it properly in different scenarios (2D/3D, single/multi-channel and single/multi-rater).
I should be able to open a PR by the end of the day tomorrow.