Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bf16 Support to Adam #134

Closed
wants to merge 21 commits into from
Closed

Add bf16 Support to Adam #134

wants to merge 21 commits into from

Commits on Aug 2, 2023

  1. modified: bmtrain/optim/adam.py

    ycr0776 committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    3210632 View commit details
    Browse the repository at this point in the history
  2. modified: add bf16 in adam.py

    ycr0776 committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    4e4d32a View commit details
    Browse the repository at this point in the history
  3. modified: bmtrain/optim/_function.py

    	modified:   bmtrain/optim/adam.py
    	modified:   bmtrain/optim/adam_offload.py
    	modified:   csrc/cuda/adam_cuda.cu
    	modified:   csrc/include/adam_cpu.hpp
    ycr0776 committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    ef038ae View commit details
    Browse the repository at this point in the history
  4. modified: bmtrain/optim/_function.py

    	modified:   csrc/bind.cpp
    	modified:   csrc/cuda/adam_cuda.cu
    	modified:   csrc/include/adam_cpu.hpp
    	modified:   csrc/include/bind.hpp
    JerryYin777 committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    a1e0e42 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2023

  1. modified: add bf16.h to csrc/cuda/cross_entropy.cu

    modified: add bf16.h to csrc/cuda/has_inf_nan.cu
    modified: add bf16 support to bmtrain/optim/_function.py
    modified: add bf16 support to csrc/cuda/adam_cuda.cu(Todo)
    modified: add bf16.h to csrc/cuda/reduce.cuh
    modified: add bf16 support to csrc/include/adam_cpu.hpp
    JerryYin777 committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    efe3cdc View commit details
    Browse the repository at this point in the history
  2. modified: bmtrain/optim/_function.py

    	modified:   csrc/cuda/adam_cuda.cu
    	modified:   csrc/cuda/reduce.cuh
    	modified:   csrc/include/adam_cpu.hpp
    JerryYin777 committed Aug 3, 2023
    Configuration menu
    Copy the full SHA
    6125f39 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e2fdcc5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    00eee39 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    2eb6d72 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. modified: bmtrain/loss/_function.py

    	modified:   bmtrain/optim/optim_manager.py
    	modified:   csrc/bind.cpp
    	modified:   csrc/cuda/has_inf_nan.cu
    	modified:   csrc/include/adam_cpu.hpp
    	new file:   csrc/include/test.cpp
    JerryYin777 committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    6dfd739 View commit details
    Browse the repository at this point in the history
  2. modified: add bf16 to is_nan_inf()

    	modified:   bind bf16 functions
    	modified:   revise the logic of bf16 accum
    	modified:   add bf16 to is_nan_inf() of CUDA version
    	modified:   revise the logic of bf16 accum
    	modified:   bind bf16 functions
    	deleted:    csrc/include/test.cpp
    JerryYin777 committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    8a6f686 View commit details
    Browse the repository at this point in the history
  3. FIX: csrc/bind.cpp

    modified: revise logic of has_inf_nan.cu
    FIX:csrc/include/adam_cpu.hpp
    FIX:csrc/include/bind.hpp
    JerryYin777 committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    0b22ed1 View commit details
    Browse the repository at this point in the history
  4. modified: tests/test_has_inf_nan.py

    	new file:   tests/test_has_inf_nan_bf16.py
    JerryYin777 committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    f8885cf View commit details
    Browse the repository at this point in the history
  5. modified: bmtrain/optim/_function.py

    	modified:   csrc/bind.cpp
    	new file:   tests/test_optim_bf16.py
    	new file:   tests/test_optim_fp16.py
    JerryYin777 committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    9a9d526 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    77c3585 View commit details
    Browse the repository at this point in the history
  7. Update adam_cuda.cu

    JerryYin777 committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    5cc3611 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Configuration menu
    Copy the full SHA
    870c613 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2b414b8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    40441f8 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. FIX

    JerryYin777 committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    c5f7e49 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    148ed85 View commit details
    Browse the repository at this point in the history