Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A Error in training #2

Closed
ytddd75 opened this issue Feb 28, 2023 · 4 comments
Closed

A Error in training #2

ytddd75 opened this issue Feb 28, 2023 · 4 comments

Comments

@ytddd75
Copy link

ytddd75 commented Feb 28, 2023

Thank you for your sharing. May I ask that after data preprocessing according to the code you provided, the training program reported an error:
File "/home/counting/SPDCN-CAC-main/datasets/dataset.py", line 56, in getSample
dotmap[0, points[:, 1], points[:, 0]] = 1
IndexError: index 384 is out of bounds for axis 1 with size 384

Elin24 added a commit that referenced this issue Mar 1, 2023
Clip point locations according to [issue-2](#2)
@Elin24
Copy link
Owner

Elin24 commented Mar 1, 2023

This happened because the point locations may be out of range. I add the clip() function to avoid it, as shown in line-56:

points[:, 1] = torch.clip(points[:, 1], min=0, max=h-1)
points[:, 0] = torch.clip(points[:, 0], min=0, max=w-1)

@ytddd75
Copy link
Author

ytddd75 commented Mar 1, 2023

Thanks for your thoughtful reply, the problem has been solved.
By the way, dose the code you provide allow for multi-GPU training?

@Elin24
Copy link
Owner

Elin24 commented Mar 1, 2023

I'm afraid the answer is no.

However, the batch size is 16, and the memory requirement is about 21G, so one 3090-GPU is enough to train this model. You can reduce the batch size and adjust the learning rate accordingly for training with limited resources.

Another way to reduce computation costs is using FP16, which may decrease the accuracy a little but save lots of memory.

@ytddd75
Copy link
Author

ytddd75 commented Mar 1, 2023

Thanks for your thoughtful reply.

@ytddd75 ytddd75 closed this as completed Mar 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants