New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[zero] add chunk init function for users #1729
Conversation
params_size = [p.numel() for p in model.parameters() if in_ddp(p)] | ||
params_size_arr = np.array(params_size) | ||
|
||
std = np.std(params_size_arr) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not use torch.std? is it necessary to conver it to np?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know. @ver217 wrote this code.
wasted_size /= 1024**2 | ||
|
||
if dist.get_rank() == 0: | ||
print("searching chunk configuration is completed in {:.2f} s.\n".format(span_s), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
print_rank_0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
print_rank_0 is not robust enough to print multiple strings. it is only able to print a single string.
No description provided.