Inference PP changes for neox#1899

Merged

jeffra merged 30 commits intomasterfrom

inference/pipechanges

Apr 26, 2022

Collaborator

jeffra commented Apr 19, 2022

No description provided.

Reza Yazdani and others added 8 commits

March 3, 2022 22:44


          Synchronize the GPUs for the text-generation inference test

64a58c7


          change pp to run inference

d153b09


          support ckpt conversion between pp<->seq

ec0edd5


          remove conversion code, its now moved to client side

a145129


          add the inference-api changes to support the GPT-Neox inference at de…

1aa5148

…epspeed-inference kernels


          fix the accuracy issue on the GPTNeoX inference

00d3be6


          fixing parameter copy

525e9e7


          add inference global

7e96581

jeffra requested review from RezaYazdaniAminabadi, ShadenSmith, awan-10, cli99, conglongli, eltonzheng, minjiaz, samyam and tjruwase as code owners

April 19, 2022 18:38


          Merge branch 'master' into inference/pipechanges

69b611e

jeffra commented

View reviewed changes

deepspeed/module_inject/replace_module.py Outdated Show resolved Hide resolved

jeffra commented

View reviewed changes

deepspeed/module_inject/replace_policy.py Outdated Show resolved Hide resolved

jeffra commented

View reviewed changes

deepspeed/module_inject/replace_policy.py Outdated Show resolved Hide resolved

jeffra commented

View reviewed changes

deepspeed/runtime/config.py Outdated Show resolved Hide resolved

jeffra commented

View reviewed changes

deepspeed/runtime/config.py Outdated Show resolved Hide resolved

jeffra commented

View reviewed changes

deepspeed/runtime/engine.py Outdated Show resolved Hide resolved

jeffra commented

View reviewed changes

deepspeed/runtime/pipe/engine.py Outdated Show resolved Hide resolved

jeffra commented

View reviewed changes

deepspeed/utils/nvtx.py Outdated Show resolved Hide resolved

jeffra changed the title ~~Inference/pipechanges~~ Inference PP changes for neox

jeffra added 3 commits

April 19, 2022 17:31


          clean-up

c271cc7


          cleanup

586c93f


          more clean-up

676023e

jeffra and others added 16 commits

April 19, 2022 17:38


          Merge branch 'master' into inference/pipechanges

da45d1a


          more clean-up/docs

73afc65


          formatting

a31a93d


          fix minor issue with injection

cecde18


          Merge branch 'inference/pipechanges' of github.com:microsoft/DeepSpee…

e3e0a9a

…d into inference/pipechanges


          refine the pipe module to get a map of the fwd_funcs to be able to in…

9e5e1ce

…ject inference kernels


          fix DSE

afdcac4


          add final_linear to state dict factory ckpt split

a16adb9


          Merge branch 'master' into inference/pipechanges

7b3a75e


          fix for circular import issue

a82ae85


          Merge branch 'inference/pipechanges' of github.com:microsoft/DeepSpee…

9eff79c

…d into inference/pipechanges


          comment out sanity check for now

3a3109c


          Merge branch 'master' into inference/pipechanges

346186b


          fix layernorm-param issue & compilation on newer version of torch

3f490ae


          change the default trainig_mp_size to 1

1891e30


          Merge branch 'master' into inference/pipechanges

8bd754f

Collaborator Author

jeffra commented Apr 26, 2022

need to fix -lcurand issue with amd build before merge

jeffra added 2 commits

April 26, 2022 10:03


          fix for curand on amd

5afbee9


          fix syntax error

8627db7

RezaYazdaniAminabadi approved these changes

View reviewed changes

jeffra merged commit b4fcd98 into master

jeffra deleted the inference/pipechanges branch

April 26, 2022 18:50

mrwyattii mentioned this pull request

[BUG] Inference Engine Error undefined symbol: curandCreateGenerator #1867

Closed

codertimo mentioned this pull request

[BUG] Floating Point Exception (core dump) at launch_attn_softmax_v2<float> #1925

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

samyam Awaiting requested review from samyam

tjruwase Awaiting requested review from tjruwase tjruwase is a code owner

ShadenSmith Awaiting requested review from ShadenSmith

conglongli Awaiting requested review from conglongli

awan-10 Awaiting requested review from awan-10

cli99 Awaiting requested review from cli99

eltonzheng Awaiting requested review from eltonzheng

minjiaz Awaiting requested review from minjiaz

1 more reviewer

RezaYazdaniAminabadi RezaYazdaniAminabadi approved these changes

Labels

None yet