New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error loading pkl file #5
Comments
Hi @VsionQing It will always load the soft label files. If you would like to train with the traditional one-hot label, you can replace "soft_label" with "target" at https://github.com/szq0214/FKD/blob/main/train_FKD.py#L383 and use "criterion_ce" instead of "criterion_sce". |
My code is |
@VsionQing, the downloaded soft label file is enough, no other pre-trained pkl file is needed in training. |
my code is |
It seems you are using the Windows system, not sure the current path structure is suitable for your system or not. |
I have revised the three suggestions you put forward |
@VsionQing Please make sure the two paths --softlabel_path and [imagenet-folder with train and val folders] are correct. |
This my code python train_FKD.py -a resnet50 --lr 0.1 --num_crops 4 -b 1024 --cos --softlabel_path G:\OPEN\FKD_soft_label_500_crops_marginal_smoothing_k_5.tar_7\FKD_soft_label_500_crops_marginal_smoothing_k_5 G:\OPEN\ILSVRC\Data\imagenet |
|
|
@VsionQing, I think the name of the file should be FKD_soft_label_500_crops_marginal_smoothing_k_5.tar.gz, instead of FKD_soft_label_500_crops_marginal_smoothing_k_5.tar_7.gz. Also, the size seems abnormal before uncompressing. |
Traceback (most recent call last):
File "E:\PythonFile\FKD-main\train_FKD.py", line 528, in
main()
File "E:\PythonFile\FKD-main\train_FKD.py", line 138, in main
main_worker(args.gpu, ngpus_per_node, args)
File "E:\PythonFile\FKD-main\train_FKD.py", line 328, in main_worker
train(train_loader, model, criterion_sce, optimizer, epoch, args)
File "E:\PythonFile\FKD-main\train_FKD.py", line 363, in train
for i, (images, target, soft_label) in enumerate(train_loader):
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 521, in next
data = self._next_data()
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 1229, in _process_data
data.reraise()
File "D:\Anconda\envs\pytorch\lib\site-packages\torch_utils.py", line 434, in reraise
raise exception
_pickle.UnpicklingError: Caught UnpicklingError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\utils\data_utils\fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "E:\PythonFile\FKD-main\utils_FKD.py", line 98, in getitem
label = torch.load(label_path, map_location=torch.device('cpu'))
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\serialization.py", line 608, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "D:\Anconda\envs\pytorch\lib\site-packages\torch\serialization.py", line 777, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\xff'.
Do not use the pre training file to directly train and report errors as above
The text was updated successfully, but these errors were encountered: