Adapting 2D dataset

[danifranco]

Hello,

I'm trying to train nnU-Net with a custom 2D dataset and I get stuck when preprocessing. The dataset contains 165 2D .tif images, so I firstly adapted it to nnU-Net file format following: Task120_Massachusetts_RoadSegm.py. It creates succesfully the folder that contains the data. I called it Task130_Lucchi. The problem comes when running nnUNet_plan_and_preprocess command as follows:

$ nnUNet_plan_and_preprocess -t 130 --verify_dataset_integrity


Please cite the following paper when using nnUNet:

Isensee, F., Jaeger, P.F., Kohl, S.A.A. et al. "nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation." Nat Methods (2020). https://doi.org/10.1038/s41592-020-01008-z


If you have questions or suggestions, feel free to open an issue at https://github.com/MIC-DKFZ/nnUNet

Verifying training set
checking case training-0001
Traceback (most recent call last):
  File "/home/dfranco/anaconda3/envs/nnunet/bin/nnUNet_plan_and_preprocess", line 33, in <module>
    sys.exit(load_entry_point('nnunet', 'console_scripts', 'nnUNet_plan_and_preprocess')())
  File "/data2/dfranco/nnUNet/nnunet/experiment_planning/nnUNet_plan_and_preprocess.py", line 77, in main
    verify_dataset_integrity(join(nnUNet_raw_data, task_name))
  File "/data2/dfranco/nnUNet/nnunet/preprocessing/sanity_checks.py", line 131, in verify_dataset_integrity
    assert all([isfile(i) for i in
AssertionError: some image files are missing for case training-0001. Expected files:
 ['/data2/dfranco/datasets/nnunet/nnUNet_raw_data_base/nnUNet_raw_data/Task130_Lucchi/imagesTr/training-0001_0000.nii.gz', '/data2/dfranco/datasets/nnunet/nnUNet_raw_data_base/nnUNet_raw_data/Task130_Lucchi/imagesTr/training-0001_0001.nii.gz', '/data2/dfranco/datasets/nnunet/nnUNet_raw_data_base/nnUNet_raw_data/Task130_Lucchi/imagesTr/training-0001_0002.nii.gz']

It seems that something is wrong with the filenames as the command expects some files that are not present. Maybe it can be solved easily but I do not know where I failed.

Thanks a lot for your time!

Extra information:

2D dataset original files

Train

/data2/dfranco/datasets/nnunet/lucchi/training/output$ ls
training-0001.tif  training-0022.tif  training-0043.tif  training-0064.tif  training-0085.tif  training-0106.tif  training-0127.tif  training-0148.tif
training-0002.tif  training-0023.tif  training-0044.tif  training-0065.tif  training-0086.tif  training-0107.tif  training-0128.tif  training-0149.tif
training-0003.tif  training-0024.tif  training-0045.tif  training-0066.tif  training-0087.tif  training-0108.tif  training-0129.tif  training-0150.tif
training-0004.tif  training-0025.tif  training-0046.tif  training-0067.tif  training-0088.tif  training-0109.tif  training-0130.tif  training-0151.tif
training-0005.tif  training-0026.tif  training-0047.tif  training-0068.tif  training-0089.tif  training-0110.tif  training-0131.tif  training-0152.tif
training-0006.tif  training-0027.tif  training-0048.tif  training-0069.tif  training-0090.tif  training-0111.tif  training-0132.tif  training-0153.tif
training-0007.tif  training-0028.tif  training-0049.tif  training-0070.tif  training-0091.tif  training-0112.tif  training-0133.tif  training-0154.tif
training-0008.tif  training-0029.tif  training-0050.tif  training-0071.tif  training-0092.tif  training-0113.tif  training-0134.tif  training-0155.tif
training-0009.tif  training-0030.tif  training-0051.tif  training-0072.tif  training-0093.tif  training-0114.tif  training-0135.tif  training-0156.tif
training-0010.tif  training-0031.tif  training-0052.tif  training-0073.tif  training-0094.tif  training-0115.tif  training-0136.tif  training-0157.tif
training-0011.tif  training-0032.tif  training-0053.tif  training-0074.tif  training-0095.tif  training-0116.tif  training-0137.tif  training-0158.tif
training-0012.tif  training-0033.tif  training-0054.tif  training-0075.tif  training-0096.tif  training-0117.tif  training-0138.tif  training-0159.tif
training-0013.tif  training-0034.tif  training-0055.tif  training-0076.tif  training-0097.tif  training-0118.tif  training-0139.tif  training-0160.tif
training-0014.tif  training-0035.tif  training-0056.tif  training-0077.tif  training-0098.tif  training-0119.tif  training-0140.tif  training-0161.tif
training-0015.tif  training-0036.tif  training-0057.tif  training-0078.tif  training-0099.tif  training-0120.tif  training-0141.tif  training-0162.tif
training-0016.tif  training-0037.tif  training-0058.tif  training-0079.tif  training-0100.tif  training-0121.tif  training-0142.tif  training-0163.tif
training-0017.tif  training-0038.tif  training-0059.tif  training-0080.tif  training-0101.tif  training-0122.tif  training-0143.tif  training-0164.tif
training-0018.tif  training-0039.tif  training-0060.tif  training-0081.tif  training-0102.tif  training-0123.tif  training-0144.tif  training-0165.tif
training-0019.tif  training-0040.tif  training-0061.tif  training-0082.tif  training-0103.tif  training-0124.tif  training-0145.tif
training-0020.tif  training-0041.tif  training-0062.tif  training-0083.tif  training-0104.tif  training-0125.tif  training-0146.tif
training-0021.tif  training-0042.tif  training-0063.tif  training-0084.tif  training-0105.tif  training-0126.tif  training-0147.tif

Test

/data2/dfranco/datasets/nnunet/lucchi/testing/input$ ls
testing-0001.tif  testing-0020.tif  testing-0039.tif  testing-0058.tif  testing-0077.tif  testing-0096.tif  testing-0115.tif  testing-0134.tif  testing-0153.tif
testing-0002.tif  testing-0021.tif  testing-0040.tif  testing-0059.tif  testing-0078.tif  testing-0097.tif  testing-0116.tif  testing-0135.tif  testing-0154.tif
testing-0003.tif  testing-0022.tif  testing-0041.tif  testing-0060.tif  testing-0079.tif  testing-0098.tif  testing-0117.tif  testing-0136.tif  testing-0155.tif
testing-0004.tif  testing-0023.tif  testing-0042.tif  testing-0061.tif  testing-0080.tif  testing-0099.tif  testing-0118.tif  testing-0137.tif  testing-0156.tif
testing-0005.tif  testing-0024.tif  testing-0043.tif  testing-0062.tif  testing-0081.tif  testing-0100.tif  testing-0119.tif  testing-0138.tif  testing-0157.tif
testing-0006.tif  testing-0025.tif  testing-0044.tif  testing-0063.tif  testing-0082.tif  testing-0101.tif  testing-0120.tif  testing-0139.tif  testing-0158.tif
testing-0007.tif  testing-0026.tif  testing-0045.tif  testing-0064.tif  testing-0083.tif  testing-0102.tif  testing-0121.tif  testing-0140.tif  testing-0159.tif
testing-0008.tif  testing-0027.tif  testing-0046.tif  testing-0065.tif  testing-0084.tif  testing-0103.tif  testing-0122.tif  testing-0141.tif  testing-0160.tif
testing-0009.tif  testing-0028.tif  testing-0047.tif  testing-0066.tif  testing-0085.tif  testing-0104.tif  testing-0123.tif  testing-0142.tif  testing-0161.tif
testing-0010.tif  testing-0029.tif  testing-0048.tif  testing-0067.tif  testing-0086.tif  testing-0105.tif  testing-0124.tif  testing-0143.tif  testing-0162.tif
testing-0011.tif  testing-0030.tif  testing-0049.tif  testing-0068.tif  testing-0087.tif  testing-0106.tif  testing-0125.tif  testing-0144.tif  testing-0163.tif
testing-0012.tif  testing-0031.tif  testing-0050.tif  testing-0069.tif  testing-0088.tif  testing-0107.tif  testing-0126.tif  testing-0145.tif  testing-0164.tif
testing-0013.tif  testing-0032.tif  testing-0051.tif  testing-0070.tif  testing-0089.tif  testing-0108.tif  testing-0127.tif  testing-0146.tif  testing-0165.tif
testing-0014.tif  testing-0033.tif  testing-0052.tif  testing-0071.tif  testing-0090.tif  testing-0109.tif  testing-0128.tif  testing-0147.tif
testing-0015.tif  testing-0034.tif  testing-0053.tif  testing-0072.tif  testing-0091.tif  testing-0110.tif  testing-0129.tif  testing-0148.tif
testing-0016.tif  testing-0035.tif  testing-0054.tif  testing-0073.tif  testing-0092.tif  testing-0111.tif  testing-0130.tif  testing-0149.tif
testing-0017.tif  testing-0036.tif  testing-0055.tif  testing-0074.tif  testing-0093.tif  testing-0112.tif  testing-0131.tif  testing-0150.tif
testing-0018.tif  testing-0037.tif  testing-0056.tif  testing-0075.tif  testing-0094.tif  testing-0113.tif  testing-0132.tif  testing-0151.tif
testing-0019.tif  testing-0038.tif  testing-0057.tif  testing-0076.tif  testing-0095.tif  testing-0114.tif  testing-0133.tif  testing-0152.tif

Generated train nii.gz

/data2/dfranco/datasets/nnunet/nnUNet_raw_data_base/nnUNet_raw_data/Task130_Lucchi$ ls imagesTr
training-0001_0000.nii.gz  training-0029_0000.nii.gz  training-0057_0000.nii.gz  training-0085_0000.nii.gz  training-0113_0000.nii.gz  training-0141_0000.nii.gz
training-0002_0000.nii.gz  training-0030_0000.nii.gz  training-0058_0000.nii.gz  training-0086_0000.nii.gz  training-0114_0000.nii.gz  training-0142_0000.nii.gz
training-0003_0000.nii.gz  training-0031_0000.nii.gz  training-0059_0000.nii.gz  training-0087_0000.nii.gz  training-0115_0000.nii.gz  training-0143_0000.nii.gz
training-0004_0000.nii.gz  training-0032_0000.nii.gz  training-0060_0000.nii.gz  training-0088_0000.nii.gz  training-0116_0000.nii.gz  training-0144_0000.nii.gz
training-0005_0000.nii.gz  training-0033_0000.nii.gz  training-0061_0000.nii.gz  training-0089_0000.nii.gz  training-0117_0000.nii.gz  training-0145_0000.nii.gz
training-0006_0000.nii.gz  training-0034_0000.nii.gz  training-0062_0000.nii.gz  training-0090_0000.nii.gz  training-0118_0000.nii.gz  training-0146_0000.nii.gz
training-0007_0000.nii.gz  training-0035_0000.nii.gz  training-0063_0000.nii.gz  training-0091_0000.nii.gz  training-0119_0000.nii.gz  training-0147_0000.nii.gz
training-0008_0000.nii.gz  training-0036_0000.nii.gz  training-0064_0000.nii.gz  training-0092_0000.nii.gz  training-0120_0000.nii.gz  training-0148_0000.nii.gz
training-0009_0000.nii.gz  training-0037_0000.nii.gz  training-0065_0000.nii.gz  training-0093_0000.nii.gz  training-0121_0000.nii.gz  training-0149_0000.nii.gz
training-0010_0000.nii.gz  training-0038_0000.nii.gz  training-0066_0000.nii.gz  training-0094_0000.nii.gz  training-0122_0000.nii.gz  training-0150_0000.nii.gz
training-0011_0000.nii.gz  training-0039_0000.nii.gz  training-0067_0000.nii.gz  training-0095_0000.nii.gz  training-0123_0000.nii.gz  training-0151_0000.nii.gz
training-0012_0000.nii.gz  training-0040_0000.nii.gz  training-0068_0000.nii.gz  training-0096_0000.nii.gz  training-0124_0000.nii.gz  training-0152_0000.nii.gz
training-0013_0000.nii.gz  training-0041_0000.nii.gz  training-0069_0000.nii.gz  training-0097_0000.nii.gz  training-0125_0000.nii.gz  training-0153_0000.nii.gz
training-0014_0000.nii.gz  training-0042_0000.nii.gz  training-0070_0000.nii.gz  training-0098_0000.nii.gz  training-0126_0000.nii.gz  training-0154_0000.nii.gz
training-0015_0000.nii.gz  training-0043_0000.nii.gz  training-0071_0000.nii.gz  training-0099_0000.nii.gz  training-0127_0000.nii.gz  training-0155_0000.nii.gz
training-0016_0000.nii.gz  training-0044_0000.nii.gz  training-0072_0000.nii.gz  training-0100_0000.nii.gz  training-0128_0000.nii.gz  training-0156_0000.nii.gz
training-0017_0000.nii.gz  training-0045_0000.nii.gz  training-0073_0000.nii.gz  training-0101_0000.nii.gz  training-0129_0000.nii.gz  training-0157_0000.nii.gz
training-0018_0000.nii.gz  training-0046_0000.nii.gz  training-0074_0000.nii.gz  training-0102_0000.nii.gz  training-0130_0000.nii.gz  training-0158_0000.nii.gz
training-0019_0000.nii.gz  training-0047_0000.nii.gz  training-0075_0000.nii.gz  training-0103_0000.nii.gz  training-0131_0000.nii.gz  training-0159_0000.nii.gz
training-0020_0000.nii.gz  training-0048_0000.nii.gz  training-0076_0000.nii.gz  training-0104_0000.nii.gz  training-0132_0000.nii.gz  training-0160_0000.nii.gz
training-0021_0000.nii.gz  training-0049_0000.nii.gz  training-0077_0000.nii.gz  training-0105_0000.nii.gz  training-0133_0000.nii.gz  training-0161_0000.nii.gz
training-0022_0000.nii.gz  training-0050_0000.nii.gz  training-0078_0000.nii.gz  training-0106_0000.nii.gz  training-0134_0000.nii.gz  training-0162_0000.nii.gz
training-0023_0000.nii.gz  training-0051_0000.nii.gz  training-0079_0000.nii.gz  training-0107_0000.nii.gz  training-0135_0000.nii.gz  training-0163_0000.nii.gz
training-0024_0000.nii.gz  training-0052_0000.nii.gz  training-0080_0000.nii.gz  training-0108_0000.nii.gz  training-0136_0000.nii.gz  training-0164_0000.nii.gz
training-0025_0000.nii.gz  training-0053_0000.nii.gz  training-0081_0000.nii.gz  training-0109_0000.nii.gz  training-0137_0000.nii.gz  training-0165_0000.nii.gz
training-0026_0000.nii.gz  training-0054_0000.nii.gz  training-0082_0000.nii.gz  training-0110_0000.nii.gz  training-0138_0000.nii.gz
training-0027_0000.nii.gz  training-0055_0000.nii.gz  training-0083_0000.nii.gz  training-0111_0000.nii.gz  training-0139_0000.nii.gz
training-0028_0000.nii.gz  training-0056_0000.nii.gz  training-0084_0000.nii.gz  training-0112_0000.nii.gz  training-0140_0000.nii.gz

Generated test nii.gz

/data2/dfranco/datasets/nnunet/nnUNet_raw_data_base/nnUNet_raw_data/Task130_Lucchi$ ls imagesTs
testing-0001_0000.nii.gz  testing-0029_0000.nii.gz  testing-0057_0000.nii.gz  testing-0085_0000.nii.gz  testing-0113_0000.nii.gz  testing-0141_0000.nii.gz
testing-0002_0000.nii.gz  testing-0030_0000.nii.gz  testing-0058_0000.nii.gz  testing-0086_0000.nii.gz  testing-0114_0000.nii.gz  testing-0142_0000.nii.gz
testing-0003_0000.nii.gz  testing-0031_0000.nii.gz  testing-0059_0000.nii.gz  testing-0087_0000.nii.gz  testing-0115_0000.nii.gz  testing-0143_0000.nii.gz
testing-0004_0000.nii.gz  testing-0032_0000.nii.gz  testing-0060_0000.nii.gz  testing-0088_0000.nii.gz  testing-0116_0000.nii.gz  testing-0144_0000.nii.gz
testing-0005_0000.nii.gz  testing-0033_0000.nii.gz  testing-0061_0000.nii.gz  testing-0089_0000.nii.gz  testing-0117_0000.nii.gz  testing-0145_0000.nii.gz
testing-0006_0000.nii.gz  testing-0034_0000.nii.gz  testing-0062_0000.nii.gz  testing-0090_0000.nii.gz  testing-0118_0000.nii.gz  testing-0146_0000.nii.gz
testing-0007_0000.nii.gz  testing-0035_0000.nii.gz  testing-0063_0000.nii.gz  testing-0091_0000.nii.gz  testing-0119_0000.nii.gz  testing-0147_0000.nii.gz
testing-0008_0000.nii.gz  testing-0036_0000.nii.gz  testing-0064_0000.nii.gz  testing-0092_0000.nii.gz  testing-0120_0000.nii.gz  testing-0148_0000.nii.gz
testing-0009_0000.nii.gz  testing-0037_0000.nii.gz  testing-0065_0000.nii.gz  testing-0093_0000.nii.gz  testing-0121_0000.nii.gz  testing-0149_0000.nii.gz
testing-0010_0000.nii.gz  testing-0038_0000.nii.gz  testing-0066_0000.nii.gz  testing-0094_0000.nii.gz  testing-0122_0000.nii.gz  testing-0150_0000.nii.gz
testing-0011_0000.nii.gz  testing-0039_0000.nii.gz  testing-0067_0000.nii.gz  testing-0095_0000.nii.gz  testing-0123_0000.nii.gz  testing-0151_0000.nii.gz
testing-0012_0000.nii.gz  testing-0040_0000.nii.gz  testing-0068_0000.nii.gz  testing-0096_0000.nii.gz  testing-0124_0000.nii.gz  testing-0152_0000.nii.gz
testing-0013_0000.nii.gz  testing-0041_0000.nii.gz  testing-0069_0000.nii.gz  testing-0097_0000.nii.gz  testing-0125_0000.nii.gz  testing-0153_0000.nii.gz
testing-0014_0000.nii.gz  testing-0042_0000.nii.gz  testing-0070_0000.nii.gz  testing-0098_0000.nii.gz  testing-0126_0000.nii.gz  testing-0154_0000.nii.gz
testing-0015_0000.nii.gz  testing-0043_0000.nii.gz  testing-0071_0000.nii.gz  testing-0099_0000.nii.gz  testing-0127_0000.nii.gz  testing-0155_0000.nii.gz
testing-0016_0000.nii.gz  testing-0044_0000.nii.gz  testing-0072_0000.nii.gz  testing-0100_0000.nii.gz  testing-0128_0000.nii.gz  testing-0156_0000.nii.gz
testing-0017_0000.nii.gz  testing-0045_0000.nii.gz  testing-0073_0000.nii.gz  testing-0101_0000.nii.gz  testing-0129_0000.nii.gz  testing-0157_0000.nii.gz
testing-0018_0000.nii.gz  testing-0046_0000.nii.gz  testing-0074_0000.nii.gz  testing-0102_0000.nii.gz  testing-0130_0000.nii.gz  testing-0158_0000.nii.gz
testing-0019_0000.nii.gz  testing-0047_0000.nii.gz  testing-0075_0000.nii.gz  testing-0103_0000.nii.gz  testing-0131_0000.nii.gz  testing-0159_0000.nii.gz
testing-0020_0000.nii.gz  testing-0048_0000.nii.gz  testing-0076_0000.nii.gz  testing-0104_0000.nii.gz  testing-0132_0000.nii.gz  testing-0160_0000.nii.gz
testing-0021_0000.nii.gz  testing-0049_0000.nii.gz  testing-0077_0000.nii.gz  testing-0105_0000.nii.gz  testing-0133_0000.nii.gz  testing-0161_0000.nii.gz
testing-0022_0000.nii.gz  testing-0050_0000.nii.gz  testing-0078_0000.nii.gz  testing-0106_0000.nii.gz  testing-0134_0000.nii.gz  testing-0162_0000.nii.gz
testing-0023_0000.nii.gz  testing-0051_0000.nii.gz  testing-0079_0000.nii.gz  testing-0107_0000.nii.gz  testing-0135_0000.nii.gz  testing-0163_0000.nii.gz
testing-0024_0000.nii.gz  testing-0052_0000.nii.gz  testing-0080_0000.nii.gz  testing-0108_0000.nii.gz  testing-0136_0000.nii.gz  testing-0164_0000.nii.gz
testing-0025_0000.nii.gz  testing-0053_0000.nii.gz  testing-0081_0000.nii.gz  testing-0109_0000.nii.gz  testing-0137_0000.nii.gz  testing-0165_0000.nii.gz
testing-0026_0000.nii.gz  testing-0054_0000.nii.gz  testing-0082_0000.nii.gz  testing-0110_0000.nii.gz  testing-0138_0000.nii.gz
testing-0027_0000.nii.gz  testing-0055_0000.nii.gz  testing-0083_0000.nii.gz  testing-0111_0000.nii.gz  testing-0139_0000.nii.gz
testing-0028_0000.nii.gz  testing-0056_0000.nii.gz  testing-0084_0000.nii.gz  testing-0112_0000.nii.gz  testing-0140_0000.nii.gz

[TimChenyu]

Hi, May I ask how did you go around solving this problem? I am currently in a similar situation

[pushkarmutha]

What does your dataset.json look like? Looks like you only have 1 channel in your dataset but multiple in the dataset.json