在训练模型时报了这个错Error: Blocking queue is killed because the data reader raises an exception #308

shuhan666 · 2020-07-06T04:58:18Z

Error: Blocking queue is killed because the data reader raises an exception
[Hint: Expected killed_ != true, but received killed_:1 == true:1.] at (D:\1.8.1\paddle\paddle/fluid/operators/reader/blocking_queue.h:141)
[operator < read > error]
W0703 11:51:43.316778 16908 operator.cc:187] read raises an exception class std::future_error, no state
F0703 11:51:43.316778 16908 exception_holder.h:37] std::exception caught, no state
*** Check failure stack trace: ***
去群里问说吧batch sise调小一点，检查数据，但是我检查了还是这么报错，求求大佬帮帮忙

wuyefeilin · 2020-07-06T05:10:19Z

你修改过代码吗？你这是在什么情况下运行报错的，是训练，预测还是预测部署？然后你用的是哪个版本的

shuhan666 · 2020-07-06T05:22:29Z

你修改过代码吗？你这是在什么情况下运行报错的，是训练，预测还是预测部署？然后你用的是哪个版本的
我没有修改过代码，用的是那个快速入门里面的代码，然后数据是自己本地的，调整了参数，然后版本应该是0.5？这个是在训练的时候报错的，数据校验的时候好像没啥问题（我也不是特别懂）

shuhan666 · 2020-07-06T05:50:45Z

你修改过代码吗？你这是在什么情况下运行报错的，是训练，预测还是预测部署？然后你用的是哪个版本的

我没有修改过代码，用的是那个快速入门里面的代码，然后数据是自己本地的，调整了参数，然后版本应该是0.5？这个是在训练的时候报错的，数据校验的时候好像没啥问题（我也不是特别懂）

wuyefeilin · 2020-07-06T06:15:10Z

你是根据这个教程在自己电脑上运行的吗？然后你训练日志也提供一下

shuhan666 · 2020-07-06T07:20:45Z

你是根据这个教程在自己电脑上运行的吗？然后你训练日志也提供一下

是的，是这个教程，但是不好意思我不太清楚训练日志是什么（对不起我太菜了）

wuyefeilin · 2020-07-06T07:40:46Z

你先根据我们提供给的示例数据集跑一遍教程，初步了解我们的运行过程，然后你也看一下我们的数据结构，你再根据我们的示例数据的格式替换自己的数据集就行了。

shuhan666 · 2020-07-06T08:05:23Z

你先根据我们提供给的示例数据集跑一遍教程，初步了解我们的运行过程，然后你也看一下我们的数据结构，你再根据我们的示例数据的格式替换自己的数据集就行了。

啊对不起我在下载你们的数据时有报这个错，能问问这是为什么吗，是Python版本的问题吗

File "dataset/download_optic.py", line 1
SyntaxError: encoding problem: utf8

wuyefeilin · 2020-07-06T10:18:03Z

我这边运行正常，你把这一行删除了试试，不行的话你你直接自己下载解压数据集https://paddleseg.bj.bcebos.com/dataset/optic_disc_seg.zip

shuhan666 · 2020-07-10T13:27:17Z

我这边运行正常，你把这一行删除了试试，不行的话你你直接自己下载解压数据集https://paddleseg.bj.bcebos.com/dataset/optic_disc_seg.zip

对不起我照着你们给的例子修改了一下我的配置，还是报错，你们给的例子可以跑，但是我的数据集还是不行

shuhan666 · 2020-07-11T07:44:19Z

你先根据我们提供给的示例数据集跑一遍教程，初步了解我们的运行过程，然后你也看一下我们的数据结构，你再根据我们的示例数据的格式替换自己的数据集就行了。

您好，我还是没能解决问题，能再帮我看看吗

wuyefeilin · 2020-07-13T08:53:07Z

给我一下你的运行日志，你也可以跑一下python pdseg/check.py --cfg your.yaml 看一下数据是否有问题

shuhan666 · 2020-07-13T10:19:16Z

给我一下你的运行日志，你也可以跑一下python pdseg/check.py --cfg your.yaml 看一下数据是否有问题

您说的运行日志是这个吗
'FIX_RESIZE_SIZE': (718, 565),
'FLIP': False,
'FLIP_RATIO': 0.5,
'INF_RESIZE_VALUE': 500,
'MAX_RESIZE_VALUE': 600,
'MAX_SCALE_FACTOR': 2.0,
'MIN_RESIZE_VALUE': 400,
'MIN_SCALE_FACTOR': 0.5,
'MIRROR': True,
'RICH_CROP': {'ASPECT_RATIO': 0.33,
'BLUR': False,
'BLUR_RATIO': 0.1,
'BRIGHTNESS_JITTER_RATIO': 0.5,
'CONTRAST_JITTER_RATIO': 0.5,
'ENABLE': False,
'MAX_ROTATION': 15,
'MIN_AREA_RATIO': 0.5,
'SATURATION_JITTER_RATIO': 0.5},
'SCALE_STEP_SIZE': 0.25},
'BATCH_SIZE': 2,
'DATALOADER': {'BUF_SIZE': 256, 'NUM_WORKERS': 8},
'DATASET': {'DATA_DIM': 3,
'DATA_DIR': './dataset',
'IGNORE_INDEX': 255,
'IMAGE_TYPE': 'rgb',
'NUM_CLASSES': 1,
'PADDING_VALUE': [127.5, 127.5, 127.5],
'SEPARATOR': ' ',
'TEST_FILE_LIST': './dataset/test.txt',
'TEST_TOTAL_IMAGES': 7,
'TRAIN_FILE_LIST': './dataset/train.txt',
'TRAIN_TOTAL_IMAGES': 165,
'VAL_FILE_LIST': './dataset/val.txt',
'VAL_TOTAL_IMAGES': 7,
'VIS_FILE_LIST': './dataset/test.txt'},
'EVAL_CROP_SIZE': (718, 565),
'FREEZE': {'MODEL_FILENAME': 'model',
'PARAMS_FILENAME': 'params',
'SAVE_DIR': 'freeze_model'},
'MEAN': [0.5, 0.5, 0.5],
'MODEL': {'BN_MOMENTUM': 0.99,
'DEEPLAB': {'ASPP_WITH_SEP_CONV': True,
'BACKBONE': 'xception_65',
'DECODER_USE_SEP_CONV': True,
'DEPTH_MULTIPLIER': 1.0,
'ENABLE_DECODER': True,
'ENCODER_WITH_ASPP': True,
'OUTPUT_STRIDE': 16},
'DEFAULT_EPSILON': 1e-05,
'DEFAULT_GROUP_NUMBER': 32,
'DEFAULT_NORM_TYPE': 'bn',
'FP16': False,
'HRNET': {'STAGE2': {'NUM_CHANNELS': [40, 80], 'NUM_MODULES': 1},
'STAGE3': {'NUM_CHANNELS': [40, 80, 160],
'NUM_MODULES': 4},
'STAGE4': {'NUM_CHANNELS': [40, 80, 160, 320],
'NUM_MODULES': 3}},
'ICNET': {'DEPTH_MULTIPLIER': 0.5, 'LAYERS': 50},
'MODEL_NAME': 'unet',
'MULTI_LOSS_WEIGHT': [1.0],
'PSPNET': {'DEPTH_MULTIPLIER': 1, 'LAYERS': 50},
'SCALE_LOSS': 'DYNAMIC',
'UNET': {'UPSAMPLE_MODE': 'bilinear'}},
'NUM_TRAINERS': 1,
'SLIM': {'KNOWLEDGE_DISTILL': False,
'KNOWLEDGE_DISTILL_IS_TEACHER': False,
'KNOWLEDGE_DISTILL_TEACHER_MODEL_DIR': '',
'NAS_ADDRESS': '',
'NAS_IS_SERVER': True,
'NAS_PORT': 23333,
'NAS_SEARCH_STEPS': 100,
'NAS_SPACE_NAME': '',
'NAS_START_EVAL_EPOCH': 0,
'PREPROCESS': False,
'PRUNE_PARAMS': '',
'PRUNE_RATIOS': []},
'SOLVER': {'BEGIN_EPOCH': 1,
'CROSS_ENTROPY_WEIGHT': None,
'DECAY_EPOCH': [10, 20],
'GAMMA': 0.1,
'LOSS': ['softmax_loss'],
'LOSS_WEIGHT': {'BCE_LOSS': 1,
'DICE_LOSS': 1,
'LOVASZ_HINGE_LOSS': 1,
'LOVASZ_SOFTMAX_LOSS': 1,
'SOFTMAX_LOSS': 1},
'LR': 0.001,
'LR_POLICY': 'poly',
'LR_WARMUP': False,
'LR_WARMUP_STEPS': 2000,
'MOMENTUM': 0.9,
'MOMENTUM2': 0.999,
'NUM_EPOCHS': 10,
'OPTIMIZER': 'adam',
'POWER': 0.9,
'WEIGHT_DECAY': 4e-05},
'STD': [0.5, 0.5, 0.5],
'TEST': {'TEST_MODEL': './saved_model/unet_optic/final'},
'TRAIN': {'MODEL_SAVE_DIR': './saved_model/unet_optic/',
'PRETRAINED_MODEL_DIR': './pretrained_model/unet_bn_coco/',
'RESUME_MODEL_DIR': '',
'SNAPSHOT_EPOCH': 5,
'SYNC_BATCH_NORM': False},
'TRAINER_ID': 0,
'TRAIN_CROP_SIZE': (718, 565)}
#Device count: 1
batch_size_per_dev: 2
2020-07-13 18:14:50,070-INFO: If regularizer of a Parameter has been set by 'fluid.ParamAttr' or 'fluid.WeightNormParamAttr' already. The Regularization[L2Decay, regularization_coeff=0.000040] in Optimizer will
not take effect, and it will only be applied to other Parameters!
W0713 18:14:50.293848 9320 device_context.cc:252] Please NOTE: device: 0, CUDA Capability: 75, Driver API Version: 10.2, Runtime API Version: 10.0
W0713 18:14:50.357677 9320 device_context.cc:260] device: 0, cuDNN Version: 7.6.
Pretrained model dir ./pretrained_model/unet_bn_coco/ not exists, training from scratch...
Use multi-thread reader
W0713 18:14:55.208707 9320 build_strategy.cc:170] fusion_group is not enabled for Windows/MacOS now, and only effective when running with CUDA GPU.
2020-07-13 18:14:55,400-WARNING: Your reader has raised an exception!
Exception in thread Thread-3:
Traceback (most recent call last):
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\threading.py", line 917, in _bootstrap_inner
self.run()
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\paddle\fluid\reader.py", line 1156, in thread_main
six.reraise(*sys.exc_info())
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\six.py", line 703, in reraise
raise value
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\paddle\fluid\reader.py", line 1136, in thread_main
for tensors in self._tensor_reader():
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\paddle\fluid\data_feeder.py", line 196, in call
for each_sample in self.generator():
File "pdseg/train.py", line 195, in data_generator
batch_data = []
File "D:\work\try\pdseg\reader.py", line 81, in generator
yield self.process_image(line, self.data_dir, self.mode)
File "D:\work\try\pdseg\reader.py", line 253, in process_image
img, grt = aug.resize(img, grt, mode)
File "D:\work\try\pdseg\data_aug.py", line 44, in resize
grt = cv2.resize(grt, target_size, interpolation=cv2.INTER_NEAREST)
TypeError: Expected Ptrcv::UMat for argument 'src'

C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\paddle\fluid\executor.py:1070: UserWarning: The following exception is not an EOF exception.
"The following exception is not an EOF exception.")

C++ Call Stacks (More useful to developers):

Windows not support stack backtrace yet.

Python Call Stacks (More useful to users):

File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\paddle\fluid\framework.py", line 2610, in append_op
attrs=kwargs.get("attrs", None))
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\paddle\fluid\reader.py", line 1078, in _init_non_iterable
attrs={'drop_last': self._drop_last})
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\paddle\fluid\reader.py", line 976, in init
self._init_non_iterable()
File "C:\Users\DELL\AppData\Local\Programs\Python\Python37\lib\site-packages\paddle\fluid\reader.py", line 608, in from_generator
iterable, return_list, drop_last)
File "D:\work\try\pdseg\models\model_builder.py", line 142, in build_model
use_double_buffer=True)
File "pdseg/train.py", line 225, in train
data_loader, avg_loss, lr, pred, grts, masks = build_model(
File "pdseg/train.py", line 438, in main
print_info(pprint.pformat(cfg))
File "pdseg/train.py", line 451, in
sys.exit(1)

Error Message Summary:

Error: Blocking queue is killed because the data reader raises an exception
[Hint: Expected killed_ != true, but received killed_:1 == true:1.] at (D:\1.8.1\paddle\paddle/fluid/operators/reader/blocking_queue.h:141)
[operator < read > error]
W0713 18:14:55.431116 6320 operator.cc:187] read raises an exception class std::future_error, no state
F0713 18:14:55.432111 6320 exception_holder.h:37] std::exception caught, no state
*** Check failure stack trace: ***

shuhan666 · 2020-07-13T10:19:51Z

给我一下你的运行日志，你也可以跑一下python pdseg/check.py --cfg your.yaml 看一下数据是否有问题

然后我感觉数据校验是没问题的，那个detail文件内容如下：
{'AUG': {'AUG_METHOD': 'unpadding',
'FIX_RESIZE_SIZE': (718, 565),
'FLIP': False,
'FLIP_RATIO': 0.5,
'INF_RESIZE_VALUE': 500,
'MAX_RESIZE_VALUE': 600,
'MAX_SCALE_FACTOR': 2.0,
'MIN_RESIZE_VALUE': 400,
'MIN_SCALE_FACTOR': 0.5,
'MIRROR': True,
'RICH_CROP': {'ASPECT_RATIO': 0.33,
'BLUR': False,
'BLUR_RATIO': 0.1,
'BRIGHTNESS_JITTER_RATIO': 0.5,
'CONTRAST_JITTER_RATIO': 0.5,
'ENABLE': False,
'MAX_ROTATION': 15,
'MIN_AREA_RATIO': 0.5,
'SATURATION_JITTER_RATIO': 0.5},
'SCALE_STEP_SIZE': 0.25},
'BATCH_SIZE': 2,
'DATALOADER': {'BUF_SIZE': 256, 'NUM_WORKERS': 8},
'DATASET': {'DATA_DIM': 3,
'DATA_DIR': './dataset/optic_disc_seg/',
'IGNORE_INDEX': 255,
'IMAGE_TYPE': 'rgb',
'NUM_CLASSES': 2,
'PADDING_VALUE': [127.5, 127.5, 127.5],
'SEPARATOR': ' ',
'TEST_FILE_LIST': './dataset/optic_disc_seg/test.txt',
'TEST_TOTAL_IMAGES': 7,
'TRAIN_FILE_LIST': './dataset/optic_disc_seg/train.txt',
'TRAIN_TOTAL_IMAGES': 165,
'VAL_FILE_LIST': './dataset/optic_disc_seg/val.txt',
'VAL_TOTAL_IMAGES': 7,
'VIS_FILE_LIST': './dataset/optic_disc_seg/test.txt'},
'EVAL_CROP_SIZE': (718, 565),
'FREEZE': {'MODEL_FILENAME': 'model',
'PARAMS_FILENAME': 'params',
'SAVE_DIR': 'freeze_model'},
'MEAN': [0.5, 0.5, 0.5],
'MODEL': {'BN_MOMENTUM': 0.99,
'DEEPLAB': {'ASPP_WITH_SEP_CONV': True,
'BACKBONE': 'xception_65',
'DECODER_USE_SEP_CONV': True,
'DEPTH_MULTIPLIER': 1.0,
'ENABLE_DECODER': True,
'ENCODER_WITH_ASPP': True,
'OUTPUT_STRIDE': 16},
'DEFAULT_EPSILON': 1e-05,
'DEFAULT_GROUP_NUMBER': 32,
'DEFAULT_NORM_TYPE': 'bn',
'FP16': False,
'HRNET': {'STAGE2': {'NUM_CHANNELS': [40, 80], 'NUM_MODULES': 1},
'STAGE3': {'NUM_CHANNELS': [40, 80, 160],
'NUM_MODULES': 4},
'STAGE4': {'NUM_CHANNELS': [40, 80, 160, 320],
'NUM_MODULES': 3}},
'ICNET': {'DEPTH_MULTIPLIER': 0.5, 'LAYERS': 50},
'MODEL_NAME': 'unet',
'MULTI_LOSS_WEIGHT': [1.0],
'PSPNET': {'DEPTH_MULTIPLIER': 1, 'LAYERS': 50},
'SCALE_LOSS': 'DYNAMIC',
'UNET': {'UPSAMPLE_MODE': 'bilinear'}},
'NUM_TRAINERS': 1,
'SLIM': {'KNOWLEDGE_DISTILL': False,
'KNOWLEDGE_DISTILL_IS_TEACHER': False,
'KNOWLEDGE_DISTILL_TEACHER_MODEL_DIR': '',
'NAS_ADDRESS': '',
'NAS_IS_SERVER': True,
'NAS_PORT': 23333,
'NAS_SEARCH_STEPS': 100,
'NAS_SPACE_NAME': '',
'NAS_START_EVAL_EPOCH': 0,
'PREPROCESS': False,
'PRUNE_PARAMS': '',
'PRUNE_RATIOS': []},
'SOLVER': {'BEGIN_EPOCH': 1,
'CROSS_ENTROPY_WEIGHT': None,
'DECAY_EPOCH': [10, 20],
'GAMMA': 0.1,
'LOSS': ['softmax_loss'],
'LOSS_WEIGHT': {'BCE_LOSS': 1,
'DICE_LOSS': 1,
'LOVASZ_HINGE_LOSS': 1,
'LOVASZ_SOFTMAX_LOSS': 1,
'SOFTMAX_LOSS': 1},
'LR': 0.001,
'LR_POLICY': 'poly',
'LR_WARMUP': False,
'LR_WARMUP_STEPS': 2000,
'MOMENTUM': 0.9,
'MOMENTUM2': 0.999,
'NUM_EPOCHS': 10,
'OPTIMIZER': 'adam',
'POWER': 0.9,
'WEIGHT_DECAY': 4e-05},
'STD': [0.5, 0.5, 0.5],
'TEST': {'TEST_MODEL': './saved_model/unet_optic/final'},
'TRAIN': {'MODEL_SAVE_DIR': './saved_model/unet_optic/',
'PRETRAINED_MODEL_DIR': './pretrained_model/unet_bn_coco/',
'RESUME_MODEL_DIR': '',
'SNAPSHOT_EPOCH': 5,
'SYNC_BATCH_NORM': False},
'TRAINER_ID': 0,
'TRAIN_CROP_SIZE': (718, 565)}

Check train dataset...
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1217
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1316
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1307
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1395
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1389
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1336
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415

PASS ./dataset/optic_disc_seg/train.txt DATASET.SEPARATOR check

PASS dataset reading check
All images can be read successfully

PASS label gray check
All label images are gray

PASS label format check
total 165 label images are png format, 0 label images are not png format

Doing label pixel statistics:
(label class, total pixel number, percentage) = [(False, 52342026, 0.782), (True, 14593524, 0.218)]

PASS label class check!

PASS DATASET.IMAGE_TYPE check

WARNING: DATASET.IMAGE_TYPE is rgb but the type of all image is rgba

Doing max image size statistics:
max width and max height of images are (718,565)

PASS shape check
All images are the same shape as the labels

PASS loss check

Check val dataset...
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489

PASS ./dataset/optic_disc_seg/val.txt DATASET.SEPARATOR check

PASS dataset reading check
All images can be read successfully

PASS label gray check
All label images are gray

PASS label format check
total 7 label images are png format, 0 label images are not png format

Doing label pixel statistics:
(label class, total pixel number, percentage) = [(False, 2240087, 0.7888), (True, 599603, 0.2112)]

PASS label class check!

PASS DATASET.IMAGE_TYPE check

WARNING: DATASET.IMAGE_TYPE is rgb but the type of all image is rgba

Doing max image size statistics:
max width and max height of images are (718,565)

PASS shape check
All images are the same shape as the labels

PASS EVAL_CROP_SIZE check
satisfy current EVAL_CROP_SIZE: (718,565) >= AUG.FIX_RESIZE_SIZE: (718,565)

Check test dataset...
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1415
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1473
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1447
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1448
STREAM b'IHDR' 16 13
STREAM b'tIME' 41 7
b'tIME' 41 7 (unknown)
STREAM b'IDAT' 60 1489

PASS ./dataset/optic_disc_seg/test.txt DATASET.SEPARATOR check

PASS dataset reading check
All images can be read successfully

PASS label gray check
All label images are gray

PASS label format check
total 7 label images are png format, 0 label images are not png format

Doing label pixel statistics:
(label class, total pixel number, percentage) = [(False, 2240087, 0.7888), (True, 599603, 0.2112)]

PASS label class check!

PASS DATASET.IMAGE_TYPE check

WARNING: DATASET.IMAGE_TYPE is rgb but the type of all image is rgba

Doing max image size statistics:
max width and max height of images are (718,565)

PASS shape check
All images are the same shape as the labels

PASS EVAL_CROP_SIZE check
satisfy current EVAL_CROP_SIZE: (718,565) >= AUG.FIX_RESIZE_SIZE: (718,565)

wuyefeilin · 2020-07-13T13:29:41Z

两类的话应该是0，1,这边显示的是False, True。你在报错的这一行前面打印一下grt的数据类型

shuhan666 · 2020-07-14T09:21:24Z

两类的话应该是0，1,这边显示的是False, True。你在报错的这一行前面打印一下grt的数据类型

应该是none？对不起我不知道怎么在这个文件里打印（我太菜了）

wuyefeilin · 2020-07-14T11:05:09Z

print(grt.dtype)就行了

shuhan666 · 2020-07-14T14:35:21Z

print(grt.dtype)就行了

啊不好意思我是这么个情况，我在那个函数下面加了调用它报了这个错（不知道怎么办的我）

程序：
def resize(img, grt=None, mode=ModelPhase.TRAIN):
"""
改变图像及标签图像尺寸
AUG.AUG_METHOD为unpadding，所有模式均直接resize到AUG.FIX_RESIZE_SIZE的尺寸
AUG.AUG_METHOD为stepscaling, 按比例resize，训练时比例范围AUG.MIN_SCALE_FACTOR到AUG.MAX_SCALE_FACTOR,间隔为AUG.SCALE_STEP_SIZE，其他模式返回原图
AUG.AUG_METHOD为rangescaling，长边对齐，短边按比例变化，训练时长边对齐范围AUG.MIN_RESIZE_VALUE到AUG.MAX_RESIZE_VALUE，其他模式长边对齐AUG.INF_RESIZE_VALUE

Args：
    img(numpy.ndarray): 输入图像
    grt(numpy.ndarray): 标签图像，默认为None
    mode(string): 模式, 默认训练模式，即ModelPhase.TRAIN

Returns：
    resize后的图像和标签图

"""
if cfg.AUG.AUG_METHOD == 'unpadding':
    target_size = cfg.AUG.FIX_RESIZE_SIZE
    img = cv2.resize(img, target_size, interpolation=cv2.INTER_LINEAR)
    if grt is not None:
        print(grt.dtype)
        grt = cv2.resize(grt, target_size, interpolation=cv2.INTER_NEAREST)
elif cfg.AUG.AUG_METHOD == 'stepscaling':
    if mode == ModelPhase.TRAIN:
        min_scale_factor = cfg.AUG.MIN_SCALE_FACTOR
        max_scale_factor = cfg.AUG.MAX_SCALE_FACTOR
        step_size = cfg.AUG.SCALE_STEP_SIZE
        scale_factor = get_random_scale(min_scale_factor, max_scale_factor,
                                        step_size)
        img, grt = randomly_scale_image_and_label(
            img, grt, scale=scale_factor)
elif cfg.AUG.AUG_METHOD == 'rangescaling':
    min_resize_value = cfg.AUG.MIN_RESIZE_VALUE
    max_resize_value = cfg.AUG.MAX_RESIZE_VALUE
    if mode == ModelPhase.TRAIN:
        if min_resize_value == max_resize_value:
            random_size = min_resize_value
        else:
            random_size = int(
                np.random.uniform(min_resize_value, max_resize_value) + 0.5)
    else:
        random_size = cfg.AUG.INF_RESIZE_VALUE

    value = max(img.shape[0], img.shape[1])
    scale = float(random_size) / float(value)
    img = cv2.resize(
        img, (0, 0), fx=scale, fy=scale, interpolation=cv2.INTER_LINEAR)
    if grt is not None:
        grt = cv2.resize(
            grt, (0, 0),
            fx=scale,
            fy=scale,
            interpolation=cv2.INTER_NEAREST)
else:
    raise Exception("Unexpect data augmention method: {}".format(
        cfg.AUG.AUG_METHOD))

return img, grt

resize(img=None, grt=None, mode=ModelPhase.TRAIN)

然后报错是这样的：
Traceback (most recent call last):
File "D:/work/try/pdseg/data_aug.py", line 80, in
return img, grt
File "D:/work/try/pdseg/data_aug.py", line 40, in resize
target_size = cfg.AUG.FIX_RESIZE_SIZE
cv2.error: OpenCV(4.2.0) C:\projects\opencv-python\opencv\modules\imgproc\src\resize.cpp:4045: error: (-215:Assertion failed) !ssize.empty() in function 'cv::resize'
我就不是很清楚要怎么调用它然后让它打印出来

wuyefeilin · 2020-07-15T02:36:05Z

你就正常跑就行了，然后看打印出来的结果。而且你这边直接resize(img=None, grt=None)也没有数据传进。

另外你改动过代码吗？

把你的数据传几张给我看一下。

shuhan666 · 2020-07-15T03:54:55Z

你就正常跑就行了，然后看打印出来的结果。而且你这边直接resize(img=None, grt=None)也没有数据传进。

另外你改动过代码吗？

把你的数据传几张给我看一下。

我没有改动过（太菜了），然后正常跑之后我没有找到打印结果在哪（对不起），数据是这个样子的

wuyefeilin · 2020-07-15T06:24:06Z

你这是两类分类，类别标签应该是0和1，你看看是否符合要求

shuhan666 · 2020-07-15T06:32:23Z

你这是两类分类，类别标签应该是0和1，你看看是否符合要求

是把NUM_CLASSES设置为2吗，但是我试了还是不行呀（对不起实在麻烦了）

wuyefeilin · 2020-07-15T06:36:25Z

NUM_CLASSES为2. 另外你看看你标注图像是不是0和1,看你的图片，你的标注图像的数值像是0和255，把255改成1重新保存

shuhan666 · 2020-07-15T06:38:27Z

NUM_CLASSES为2. 另外你看看你标注图像是不是0和1,看你的图片，你的标注图像的数值像是0和255，把255改成1重新保存

啊是我现在应该去重新标注图像的意思吗（对不起这个图像是学姐帮忙标注的我就不是很懂）

wuyefeilin · 2020-07-15T06:44:42Z

不用重新标注，你直接读取然后重新保存就行了
像这样子进行重新赋值就行了：

import numpy as np
import cv2
label = cv2.imread('xxx.png', cv2.IMREAD_UNCHANGED)
label[label==255] =1
cv2.imwrite('xxx.png', label)

shuhan666 · 2020-07-15T07:14:29Z

不用重新标注，你直接读取然后重新保存就行了
像这样子进行重新赋值就行了：
import numpy as np
import cv2
label = cv2.imread('xxx.png', cv2.IMREAD_UNCHANGED)
label[label==255] =1
cv2.imwrite('xxx.png', label)

label[label==255] =1
TypeError: 'NoneType' object does not support item assignment
啊这是直接运行这段程序吗，我这这这报错惹（对不起实在麻烦啦）

wuyefeilin · 2020-07-15T07:33:43Z

不是直接运行， 'xxx.png'要替换成你的图片路径。参照我给你的脚本自己把标注图像进行读取修改数值保存。

如果你不清楚opencv关于图片读取与保存的话，可以上网搜一下学习学习哈。

shuhan666 · 2020-10-09T02:22:06Z

不是直接运行，'xxx.png'要替换成你的图片路径。参照我给你的脚本自己把标注图像进行读取修改数值保存。

如果您不清楚opencv关于图片读取与保存的话，可以上网搜一下学习学习哈。

您好，很抱歉延误了那么长时间还来问您，运行完这个脚本图像标注的部分就没有了（是没有了是吗），这样好像不可行

wuyefeilin · 2020-10-09T03:17:26Z

255改成1后，显示成黑的是正常的

shuhan666 · 2020-10-13T08:06:34Z

255改成1后，显示成黑的是正常的

非常感谢，上一个问题已经解决了，现在有个很尴尬的问题，报错是这样的

Error Message Summary:

ResourceExhaustedError:

Out of memory error on GPU 0. Cannot allocate 396.162354MB memory on GPU 0, available memory is only 182.887500MB.

Please check whether there is any other process using GPU 0.

If yes, please stop them, or start PaddlePaddle on another GPU.
If no, please decrease the batch size of your model.

at (D:\1.8.1\paddle\paddle\fluid\memory\allocation\cuda_allocator.cc:69)
*** Check failure stack trace: ***

batch size我已经调到2了，然后我其他的我不太清楚还能终止些什么程序，我现在还能怎么办呢（麻烦啦）

wuyefeilin · 2020-10-13T09:17:20Z

你跑的是什么模型，然后你的显存多大？

shuhan666 · 2020-10-13T10:26:34Z

你跑的是什么模型，然后你的显存多大？

跑的U-Net模型，然后显存这是4133m的意思吗（但是我报错显示只有170m左右可用）

wuyefeilin · 2020-10-13T11:24:44Z

那就是显存不足了，显存是随着模型的部署一步步增加的，这个是指这一步需要433m，但显存只剩下170m。
你可以减小图像尺寸和batch_size

shuhan666 · 2020-10-14T13:28:29Z

那就是显存不足了，显存是随着模型的部署一步步增加的，这个是指这一步需要433m，但显存只剩下170m。
你可以减小图像尺寸和batch_size

图像尺寸可能没办法，batchsize我也调到2了，还有别的办法吗（大哭）

shuhan666 · 2020-10-24T14:36:37Z

那就是显存不足了，显存是随着模型的部署一步步增加的，这个是指这一步需要433m，但显存只剩下170m。
你可以减小图像尺寸和batch_size

您好，我就我想问问咱们显卡编号是怎么设置的呀，集成显卡能用吗，要是实在不行我可能得放弃了（对不起麻烦你了）

shuhan666 · 2020-10-31T16:51:03Z

那就是显存不足了，显存是随着模型的部署一步步增加的，这个是指这一步需要433m，但显存只剩下170m。
你可以减小图像尺寸和batch_size

对不起我想问一下已经跑完的模型还占显存吗？我想试试其他模型但是突然显示显存不够了，报错大概是这个
fusion_group is not enabled for Windows/MacOS now, and only effective when running with CUDA GPU.

wuyefeilin · 2020-11-03T03:02:44Z

那就是显存不足了，显存是随着模型的部署一步步增加的，这个是指这一步需要433m，但显存只剩下170m。
你可以减小图像尺寸和batch_size

您好，我就我想问问咱们显卡编号是怎么设置的呀，集成显卡能用吗，要是实在不行我可能得放弃了（对不起麻烦你了）

集显用不了，得用独显。没有显卡的话可以上AIStudio试试，上面提供免费的V100显卡算力， 16G显存。

wuyefeilin · 2020-11-03T03:04:52Z

那就是显存不足了，显存是随着模型的部署一步步增加的，这个是指这一步需要433m，但显存只剩下170m。
你可以减小图像尺寸和batch_size

对不起我想问一下已经跑完的模型还占显存吗？我想试试其他模型但是突然显示显存不够了，报错大概是这个
fusion_group is not enabled for Windows/MacOS now, and only effective when running with CUDA GPU.

程序结束后会自动释放显存的，除非进程没有正常推出，这也是有可能。你显存比较小还是一个一个跑吧。跑完一个如果没有正常退出就手动kill一下

wuyefeilin self-assigned this Jul 17, 2020

michaelowenliu closed this as completed Nov 10, 2021

在训练模型时报了这个错Error: Blocking queue is killed because the data reader raises an exception #308

在训练模型时报了这个错Error: Blocking queue is killed because the data reader raises an exception #308

Comments

shuhan666 commented Jul 6, 2020

wuyefeilin commented Jul 6, 2020

shuhan666 commented Jul 6, 2020

shuhan666 commented Jul 6, 2020

wuyefeilin commented Jul 6, 2020

shuhan666 commented Jul 6, 2020

wuyefeilin commented Jul 6, 2020

shuhan666 commented Jul 6, 2020

wuyefeilin commented Jul 6, 2020

shuhan666 commented Jul 10, 2020

shuhan666 commented Jul 11, 2020

wuyefeilin commented Jul 13, 2020

shuhan666 commented Jul 13, 2020

C++ Call Stacks (More useful to developers):

Python Call Stacks (More useful to users):

Error Message Summary:

shuhan666 commented Jul 13, 2020

wuyefeilin commented Jul 13, 2020 • edited

shuhan666 commented Jul 14, 2020

wuyefeilin commented Jul 14, 2020

shuhan666 commented Jul 14, 2020

wuyefeilin commented Jul 15, 2020

shuhan666 commented Jul 15, 2020

wuyefeilin commented Jul 15, 2020

shuhan666 commented Jul 15, 2020

wuyefeilin commented Jul 15, 2020

shuhan666 commented Jul 15, 2020

wuyefeilin commented Jul 15, 2020

shuhan666 commented Jul 15, 2020

wuyefeilin commented Jul 15, 2020 • edited

shuhan666 commented Oct 9, 2020

wuyefeilin commented Oct 9, 2020

shuhan666 commented Oct 13, 2020

非常感谢，上一个问题已经解决了，现在有个很尴尬的问题，报错是这样的

Error Message Summary:

wuyefeilin commented Oct 13, 2020

shuhan666 commented Oct 13, 2020

wuyefeilin commented Oct 13, 2020

shuhan666 commented Oct 14, 2020

shuhan666 commented Oct 24, 2020

shuhan666 commented Oct 31, 2020

wuyefeilin commented Nov 3, 2020 • edited

wuyefeilin commented Nov 3, 2020

wuyefeilin commented Jul 13, 2020 •

edited

wuyefeilin commented Jul 15, 2020 •

edited

wuyefeilin commented Nov 3, 2020 •

edited