Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

clear redundant timers #4308

Merged
merged 1 commit into from
Sep 14, 2023
Merged

clear redundant timers #4308

merged 1 commit into from
Sep 14, 2023

Conversation

starkhu
Copy link
Contributor

@starkhu starkhu commented Sep 12, 2023

No description provided.

@tjruwase
Copy link
Contributor

@starkhu, this does not seem redundant as it makes it possible for client code to probe timers on other ranks.

@starkhu
Copy link
Contributor Author

starkhu commented Sep 13, 2023 •
edited
Loading

@starkhu, this does not seem redundant as it makes it possible for client code to probe timers on other ranks.

hello, @tjruwase !
i just trying to clean up out-dated events on other ranks. in more details, it is no necessary to save all previous iteration events in the current iteration , otherwise, 'len(self.times('train_batch').event_times )' is gradually increasing on other ranks。

@tjruwase
Copy link
Contributor

@starkhu, I see your point. You are correct. Thanks!

@loadams loadams added this pull request to the merge queue Sep 14, 2023
Merged via the queue into microsoft:master with commit e20e4a9 Sep 14, 2023
16 checks passed
CurryRice233 pushed a commit to CurryRice233/DeepSpeed that referenced this pull request Sep 15, 2023
Merge remote-tracking branch 'origin/master'
8614f40 
* origin/master: (48 commits)
  Fix autotune to support Triton 2.1  (microsoft#4340)
  Fix skipped inference tests (microsoft#4336)
  Suppress noise (microsoft#4310)
  Fix a bug in the implementation of dequantization for inference (microsoft#3433)
  DS-Chat BLOOM: Fix Attention mask (microsoft#4338)
  clear redundant timers (microsoft#4308)
  Add release version checking (microsoft#4328)
  Fix Zero3 contiguous grads, reduce scatter false  accuracy issue (microsoft#4321)
  Clean up modeling code (microsoft#4320)
  Handle empty parameter groups (microsoft#4277)
  Update README.md (microsoft#4316)
  README update (microsoft#4303)
  Update release and bump patch versioning flow (microsoft#4286)
  added a bert-model check for triton (microsoft#4266)
  ZeRO-Inference v2 release
  bump to 0.10.4
  Update index.md (microsoft#4297)
  fix user args parsing of string with spaces on runner (microsoft#4265)
  ZeRO-Inference refresh (microsoft#4197)
  AMD Kernel Compatibility Fixes (microsoft#3180)
  ...
@starkhu starkhu deleted the pipe-engine-branch branch September 18, 2023 01:51
@nelyahu nelyahu mentioned this pull request Oct 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tjruwase tjruwase tjruwase approved these changes

@ShadenSmith ShadenSmith Awaiting requested review from ShadenSmith

@duli2012 duli2012 Awaiting requested review from duli2012

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@starkhu @tjruwase @loadams