Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix pipline dataloader when batch elements contain tuple #565

Merged
merged 4 commits into from
Aug 25, 2023
Merged

Fix pipline dataloader when batch elements contain tuple #565

merged 4 commits into from
Aug 25, 2023

Conversation

ghosthamlet
Copy link
Contributor

No description provided.

jeffra
jeffra approved these changes Dec 2, 2020
View reviewed changes
Copy link
Contributor

@jeffra jeffra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, /cc @ShadenSmith for another check though

@ShadenSmith
Copy link
Contributor

lgtm!

@ghosthamlet
Copy link
Contributor Author

Hi @jeffra, merge failed for test_pipe.py line 246, in _helper:

    assert rel_diff(base_avg, test_avg) < 0.03
AssertionError: assert 0.03816467364302331 < 0.03
 +  where 0.03816467364302331 = rel_diff(1.8875272107124328, 1.8154903507232667)

what can I do about it? it seems like not related to my simple commit.

@jeffra
Copy link
Contributor

jeffra commented Dec 3, 2020

Hi @jeffra, merge failed for test_pipe.py line 246, in _helper:

    assert rel_diff(base_avg, test_avg) < 0.03
AssertionError: assert 0.03816467364302331 < 0.03
 +  where 0.03816467364302331 = rel_diff(1.8875272107124328, 1.8154903507232667)

what can I do about it? it seems like not related to my simple commit.

Yeah, this is pretty strange actually. I was just talking with @ShadenSmith about this as well. We need to investigate further since this shouldn't have anything to do with your PR changes.

@ghosthamlet
Copy link
Contributor Author

Ok, thanks.

@rocm-mici
Copy link

Can one of the admins verify this patch?

@jeffra jeffra requested a review from duli2012 as a code owner June 23, 2023 21:31
@awan-10 awan-10 self-assigned this Aug 18, 2023
@awan-10 awan-10 enabled auto-merge August 25, 2023 16:56
@awan-10 awan-10 added this pull request to the merge queue Aug 25, 2023
Merged via the queue into microsoft:master with commit c69bd1f Aug 25, 2023
16 checks passed
conglongli pushed a commit to cctry/DeepSpeed that referenced this pull request Sep 14, 2023
@minjiaz
Updated the Megatron-DS section (microsoft#565)
15e7fd2 
* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix
conglongli added a commit that referenced this pull request Sep 18, 2023
@conglongli @cctry @minjiaz
DeepSpeed4Science (#569)
a5552a6 
* Integrating evoformer attention

* add cutlass version check

* Updaate error message

* add benchmark

* Update

* Update evoformer_attn.py

* Update run_evoformer_test.py

* Update evoformer_attn.py

* Update run_evoformer_test.py

* support more GPU archs

* add copyright

* add tests

* Fix bugs

* Update benchmark

* update

* Fix nvcc macro

* clean code

* fix formatting

* fix yaml import

* skip unit test when not compatible

* fix yaml requirement

* revert changes

* update tutorial

* update

* fix formatting

* fix format

* skip evoformer attn in pre-compile-ops

* revert changes

* update tutorial

* fix cutlass check

* update tutorial

* refactor tutorial

* revise

* Updated the Megatron-DS section (#565)

* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix

* separate evoformer tutorial

* Revised the ds4science landing page (#566)

* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix

* Revised the landing page

* Revised the landing page

* Removing unused file

* fix links image position

* modify main page

* fix doc

---------

Co-authored-by: Shiyang Chen <csycfl@gmail.com>
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
github-merge-queue bot pushed a commit that referenced this pull request Sep 18, 2023
@conglongli @HeyangQin
@zhiruiluo @cli99 @tjruwase @CaffreyR @jeffra @stephen-youn @styoun @arashb @yidoe @GuanhuaWang @cmikeh2 @awan-10 @mrwyattii @tohtana @loadams @jomayeri @rraminen @cctry @minjiaz
DeepSpeed4Science (#4357)
f876d81 
* zero++ tutorial PR (#3783)

* [Fix] _conv_flops_compute when padding is a str and stride=1 (#3169)

* fix conv_flops_compute when padding is a str when stride=1

* fix error

* change type of paddings to tuple

* fix padding calculation

* apply formatting check

---------

Co-authored-by: Cheng Li <pistasable@gmail.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

* fix interpolate flops compute (#3782)

* use `Flops Profiler` to test `model.generate()` (#2515)

* Update profiler.py

* pre-commit run --all-files

* Delete .DS_Store

* Delete .DS_Store

* Delete .DS_Store

---------

Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Cheng Li <pistasable@gmail.com>

* revert PR #3611 (#3786)

* bump to 0.9.6

* ZeRO++ chinese blog (#3793)

* zeropp chinese blog

* try better quality images

* make title larger

* even larger...

* various fix

* center captions

* more fixes

* fix format

* remove staging trigger (#3792)

* DeepSpeed-Triton for Inference (#3748)

Co-authored-by: Stephen Youn <styoun@microsoft.com>
Co-authored-by: Arash Bakhtiari <arash@bakhtiari.org>
Co-authored-by: Cheng Li <pistasable@gmail.com>
Co-authored-by: Ethan Doe <yidoe@microsoft.com>
Co-authored-by: yidoe <68296935+yidoe@users.noreply.github.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* ZeRO++ (#3784)

Co-authored-by: HeyangQin <heyangqin@microsoft.com>
Co-authored-by: GuanhuaWang <alexwgh333@gmail.com>
Co-authored-by: cmikeh2 <connorholmes@microsoft.com>
Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>

* adding zero++ to navigation panel of deepspeed.ai (#3796)

* Add ZeRO++ Japanese blog (#3797)

* zeropp chinese blog

* try better quality images

* make title larger

* even larger...

* various fix

* center captions

* more fixes

* fix format

* add ZeRO++ Japanese blog

* add links

---------

Co-authored-by: HeyangQin <heyangqin@microsoft.com>
Co-authored-by: Conglong Li <conglong.li@gmail.com>

* Bug Fixes for autotuner and flops profiler (#1880)

* fix autotuner when backward is not called

* fix format

---------

Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>

* Missing strided copy for gated MLP (#3788)

Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>

* Requires grad checking. (#3789)

Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* bump to 0.10.0

* Fix Bug in transform.cu (#3534)

* Bug fix

* Fixed formatting error

---------

Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>

* bug fix: triton importing error (#3799)

Co-authored-by: Stephen Youn <styoun@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>

* DeepSpeed4Science (#569)

* Integrating evoformer attention

* add cutlass version check

* Updaate error message

* add benchmark

* Update

* Update evoformer_attn.py

* Update run_evoformer_test.py

* Update evoformer_attn.py

* Update run_evoformer_test.py

* support more GPU archs

* add copyright

* add tests

* Fix bugs

* Update benchmark

* update

* Fix nvcc macro

* clean code

* fix formatting

* fix yaml import

* skip unit test when not compatible

* fix yaml requirement

* revert changes

* update tutorial

* update

* fix formatting

* fix format

* skip evoformer attn in pre-compile-ops

* revert changes

* update tutorial

* fix cutlass check

* update tutorial

* refactor tutorial

* revise

* Updated the Megatron-DS section (#565)

* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix

* separate evoformer tutorial

* Revised the ds4science landing page (#566)

* Updated the Megatron-DS section

* minor fix

* minor fix

* minor fix

* Revised the landing page

* Revised the landing page

* Removing unused file

* fix links image position

* modify main page

* fix doc

---------

Co-authored-by: Shiyang Chen <csycfl@gmail.com>
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>

---------

Co-authored-by: Heyang Qin <heyangqin@microsoft.com>
Co-authored-by: Bill Luo <50068224+zhiruiluo@users.noreply.github.com>
Co-authored-by: Cheng Li <pistasable@gmail.com>
Co-authored-by: Olatunji Ruwase <olruwase@microsoft.com>
Co-authored-by: Guorun <84232793+CaffreyR@users.noreply.github.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: stephen youn <13525892+stephen-youn@users.noreply.github.com>
Co-authored-by: Stephen Youn <styoun@microsoft.com>
Co-authored-by: Arash Bakhtiari <arash@bakhtiari.org>
Co-authored-by: Ethan Doe <yidoe@microsoft.com>
Co-authored-by: yidoe <68296935+yidoe@users.noreply.github.com>
Co-authored-by: GuanhuaWang <alexwgh333@gmail.com>
Co-authored-by: cmikeh2 <connorholmes@microsoft.com>
Co-authored-by: Ammar Ahmad Awan <ammar.awan@microsoft.com>
Co-authored-by: Michael Wyatt <michaelwyatt@microsoft.com>
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com>
Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
Co-authored-by: Joe Mayer <114769929+jomayeri@users.noreply.github.com>
Co-authored-by: Ramya Ramineni <62723901+rraminen@users.noreply.github.com>
Co-authored-by: Shiyang Chen <csycfl@gmail.com>
Co-authored-by: Minjia Zhang <33713995+minjiaz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ShadenSmith ShadenSmith ShadenSmith approved these changes

@jeffra jeffra jeffra approved these changes

@raghavthind2005 raghavthind2005 raghavthind2005 approved these changes

@arashashari arashashari Awaiting requested review from arashashari

@awan-10 awan-10 Awaiting requested review from awan-10

@cli99 cli99 Awaiting requested review from cli99

@conglongli conglongli Awaiting requested review from conglongli

@eltonzheng eltonzheng Awaiting requested review from eltonzheng

@minjiaz minjiaz Awaiting requested review from minjiaz

@niumanar niumanar Awaiting requested review from niumanar

@RezaYazdaniAminabadi RezaYazdaniAminabadi Awaiting requested review from RezaYazdaniAminabadi

@samyam samyam Awaiting requested review from samyam

@tjruwase tjruwase Awaiting requested review from tjruwase

@duli2012 duli2012 Awaiting requested review from duli2012 duli2012 is a code owner

Assignees

@awan-10 awan-10

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@ghosthamlet @ShadenSmith @jeffra @rocm-mici @raghavthind2005 @awan-10