Customize a new dataset

[l763745823]

推荐使用英语模板 General question，以便你的问题帮助更多人。

首先确认以下内容

• 我已经查询了相关的 issue，但没有找到需要的帮助。
• 我已经阅读了相关文档，但仍不知道如何解决。

描述你遇到的问题

[I encountered this error when customizing my third dataset. This problem did not occur in the previous process of customizing the dataset. The format of the new dataset is even the same as before, but the number of pictures is increased]

相关信息

1. pip list | grep "mmcv\|mmcls\|^torch" 命令的输出
   [absl-py 1.0.0
   addict 2.4.0
   albumentations 1.1.0
   attrs 21.4.0
   cachetools 4.2.4
   certifi 2021.10.8
   charset-normalizer 2.0.9
   chumpy 0.70
   click 8.1.2
   colorama 0.4.4
   commonmark 0.9.1
   coverage 6.3.2
   cycler 0.11.0
   Cython 0.29.28
   efficientnet-pytorch 0.7.1
   flake8 4.0.1
   flatbuffers 2.0
   fonttools 4.28.4
   freetype-py 2.2.0
   google-auth 2.3.3
   google-auth-oauthlib 0.4.6
   grpcio 1.43.0
   idna 3.3
   imageio 2.16.2
   importlib-metadata 4.10.0
   iniconfig 1.1.1
   interrogate 1.5.0
   isort 4.3.21
   joblib 1.1.0
   json-tricks 3.15.5
   kiwisolver 1.3.2
   Markdown 3.3.6
   matplotlib 3.5.1
   mccabe 0.6.1
   mmcls 0.21.0 /home/lzx/mmclassification
   mmcv-full 1.4.4
   mmpose 0.25.0 /home/lzx/mmpose
   mmsegmentation 0.20.2 /home/lzx/mmsegmentation
   model-index 0.1.11
   motmetrics 1.2.5
   munkres 1.1.4
   networkx 2.8
   numpy 1.22.3
   oauthlib 3.1.1
   onnx 1.11.0
   onnxruntime 1.11.0
   opencv-python 4.5.4.60
   openmim 0.2.0
   ordered-set 4.1.0
   packaging 21.3
   pandas 1.3.5
   Pillow 8.4.0
   pip 21.2.4
   pluggy 1.0.0
   poseval 0.1.0
   prettytable 2.4.0
   protobuf 3.19.1
   py 1.11.0
   pyasn1 0.4.8
   pyasn1-modules 0.2.8
   pycodestyle 2.8.0
   pyflakes 2.4.0
   pyglet 1.5.23
   Pygments 2.12.0
   PyOpenGL 3.1.0
   pyparsing 3.0.6
   pyrender 0.1.45
   pytest 7.1.1
   pytest-runner 6.0.0
   python-dateutil 2.8.2
   pytz 2021.3
   PyWavelets 1.3.0
   PyYAML 6.0
   qudida 0.0.4
   requests 2.26.0
   requests-oauthlib 1.3.0
   rich 12.4.4
   rsa 4.8
   scikit-image 0.19.2
   scikit-learn 1.0.2
   scipy 1.8.0
   setuptools 58.0.4
   Shapely 1.8.1.post1
   six 1.16.0
   smplx 0.1.28
   tabulate 0.8.9
   tensorboard 2.7.0
   tensorboard-data-server 0.6.1
   tensorboard-plugin-wit 1.8.0
   threadpoolctl 3.1.0
   tifffile 2022.4.8
   timm 0.4.12
   toml 0.10.2
   tomli 2.0.1
   torch 1.9.0+cu111
   torchaudio 0.9.0
   torchvision 0.10.0+cu111
   tqdm 4.64.0
   trimesh 3.10.8
   typing_extensions 4.0.1
   urllib3 1.26.7
   wcwidth 0.2.5
   Werkzeug 2.0.2
   wheel 0.37.0
   xdoctest 1.0.0
   xmltodict 0.12.0
   xtcocotools 1.12
   yacs 0.1.8
   yapf 0.31.0
   zipp 3.6.0]
2. 如果你修改了，或者使用了新的配置文件，请在这里写明

[填写这里]

3. 如果你是在训练过程中遇到的问题，请填写完整的训练日志和报错信息
   [2022-07-07 07:03:45,526 - mmcls - INFO - Set random seed to 1708051488, deterministic: False
   2022-07-07 07:03:45,958 - mmcls - INFO - initialize ResNet with init_cfg [{'type': 'Kaiming', 'layer': ['Conv2d']}, {'type': 'Constant', 'val': 1, 'layer': ['_BatchNorm', 'GroupNorm']}]
   2022-07-07 07:03:46,499 - mmcls - INFO - initialize LinearClsHead with init_cfg {'type': 'Normal', 'layer': 'Linear', 'std': 0.01}
   Traceback (most recent call last):
   File "/home/lzx/miniconda3/envs/open-mmlab/lib/python3.8/site-packages/mmcv/utils/registry.py", line 52, in build_from_cfg
   return obj_cls(**args)
   File "/home/lzx/mmclassification/mmcls/datasets/base_dataset.py", line 42, in init
   self.data_infos = self.load_annotations()
   File "/home/lzx/mmclassification/mmcls/datasets/wild_clothing35781_fenge.py", line 16, in load_annotations
   samples = [x.strip().split(' ') for x in f.readlines()]
   File "/home/lzx/miniconda3/envs/open-mmlab/lib/python3.8/codecs.py", line 322, in decode
   (result, consumed) = self._buffer_decode(data, self.errors, final)
   UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb8 in position 8064: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "tools/train.py", line 188, in
main()
File "tools/train.py", line 161, in main
datasets = [build_dataset(cfg.data.train)]
File "/home/lzx/mmclassification/mmcls/datasets/builder.py", line 50, in build_dataset
dataset = build_from_cfg(cfg, DATASETS, default_args)
File "/home/lzx/miniconda3/envs/open-mmlab/lib/python3.8/site-packages/mmcv/utils/registry.py", line 55, in build_from_cfg
raise type(e)(f'{obj_cls.name}: {e}')
TypeError: function takes exactly 5 arguments (1 given)]
4. 如果你对 mmcls 文件夹下的代码做了其他相关的修改，请在这里写明
[填写这里]

[Ezra-Yu]

Can you show us your config?

[l763745823]

我已收到您的来信，十分感谢您的来信！

[l763745823]

base = [
'../base/models/resnet152.py', '../base/datasets/wild_clothing35781_fenge.py',
'../base/schedules/imagenet_bs256.py', '../base/default_runtime.py'
]
data = dict(
samples_per_gpu=128,
workers_per_gpu=8)

deprecation = dict(
expected='resnet152_8xb32_in1k.py',
reference='#508',
)

I just found that when defining a dataset class, in a dataset class, there is a difference between a value in CLASSESand the value of a directory in self.ann_file file, which may be the problem here, but there is a problem with my server temporarily, and it has not been verified

[okideal]

Can you show the details of cfg file ../base/datasets/wild_clothing35781_fenge.py and the Dataset class definition which you register

[l763745823]

Thank you. I've found the reason. It's because '-' cannot exist when naming datasets