Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix new arch context build check #441

Merged
merged 1 commit into from Jan 6, 2023
Merged

Fix new arch context build check #441

merged 1 commit into from Jan 6, 2023

Conversation

Kangmo
Copy link
Contributor

@Kangmo Kangmo commented Dec 24, 2022

LinearOp::forward is getting cublashandle before checking if the context is built.

problem :
LinearOp::forward is getting cublashandle without checking if context is built.
LinearOp::backward is checking if the context is built before getting cublashandle.

solution:
Modify LinearOp::forward to check if context is built before getting cublashandle.

@Kangmo
Fix problem of inconsistency while calling context->is_built between …
5e84cf3 
…forward/backward of LinearOp

problem :
    LinearOp::forward is getting cublashandle without checking if context is built.
    LinearOp::backward is checking if the context is built before getting cublashandle.

solution:
    Modify LinearOp::forward to check if context is built before getting cublashandle.
Taka152
Taka152 approved these changes Jan 6, 2023
View reviewed changes
Copy link
Contributor

@Taka152 Taka152 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your pr!

@hexisyztem
Copy link
Collaborator

Yes, the cublashandle will only be actually used after the context build is complete. In fact, we are dealing with multi-terminal compatibility later, such as an example in the figure below:
t1N965paUg

@Taka152 Taka152 changed the title LinearOp::forward is getting cublashandle before checking if the context is built. Fix new arch context build check Jan 6, 2023
@Taka152 Taka152 merged commit a25cc01 into bytedance:master Jan 6, 2023
@Kangmo
Copy link
Contributor Author

Kangmo commented Jan 6, 2023

Great! Thanks for the approval.

hexisyztem added a commit that referenced this pull request Feb 16, 2023
@hexisyztem @Kangmo @neopro12
Lsflow develop (#463)
1e2a422 
* Fix new arch context build check(#441)

problem :
    LinearOp::forward is getting cublashandle without checking if context is built.
    LinearOp::backward is checking if the context is built before getting cublashandle.

solution:
    Modify LinearOp::forward to check if context is built before getting cublashandle.

* fix config reference bug (#453)

* developing lsflow

* add split_head op and its test (#454)

* lsflow develop

* format note message

* lsflow tune

* add notes for context class

* add note for lsflow

* op example

* Update CODEOWNERS (#457)

* Gpt infer (#456)

* add split head for beam search

* alter checkin

* make launch_transform_0213 more clear (#459)

* fix operator compile

* make launch_transform_0213 more clear (#460)

* change max_shape to max_shape_size

* correct bias shape in split_head (#461)

* add unit test for x86 cpu kernel

---------

Co-authored-by: Kangmo Kim <kangmo.kim@gmail.com>
Co-authored-by: Ying Xiong <xiongying.taka@bytedance.com>
Co-authored-by: Xiaohui Wang <wangxiaohui.neo@bytedance.com>
hexisyztem added a commit that referenced this pull request Mar 2, 2023
@hexisyztem @Kangmo @neopro12
Multi device kernel (#473)
2ead283 
* multi device develop

* change build.sh

* cuda tools move into kernels

* multi device develop

* multi device develop

* multi device develop

* solve multi device develop

* solve arm compile error

* arm print_vec

* fix regress error

* solve cuda compile error

* solve multi device print vec

* split node class wit its derived class

* lightseq x86 unit test

* add x86 unit test

* Canonical Namespace

* add pybind compile

* Lsflow develop (#463)

* Fix new arch context build check(#441)

problem :
    LinearOp::forward is getting cublashandle without checking if context is built.
    LinearOp::backward is checking if the context is built before getting cublashandle.

solution:
    Modify LinearOp::forward to check if context is built before getting cublashandle.

* fix config reference bug (#453)

* developing lsflow

* add split_head op and its test (#454)

* lsflow develop

* format note message

* lsflow tune

* add notes for context class

* add note for lsflow

* op example

* Update CODEOWNERS (#457)

* Gpt infer (#456)

* add split head for beam search

* alter checkin

* make launch_transform_0213 more clear (#459)

* fix operator compile

* make launch_transform_0213 more clear (#460)

* change max_shape to max_shape_size

* correct bias shape in split_head (#461)

* add unit test for x86 cpu kernel

---------

Co-authored-by: Kangmo Kim <kangmo.kim@gmail.com>
Co-authored-by: Ying Xiong <xiongying.taka@bytedance.com>
Co-authored-by: Xiaohui Wang <wangxiaohui.neo@bytedance.com>

* fix sys.path

* fix sys.path (#466)

* jit build support pure cpu machine

* robust builder for x86 and cuda

* fix compile error

* develop test_ls_layer

* format

* fix training compile problem

* add mkl gemm for f32 and s8 (#470)

* test for encoder layer (#471)

* test encoder layer

* fix cuda free error

* lightseq multi device develop

* fix crf op error

* avoid import training directories

* fix strided_batch_gemm config data type

* add debug message

* convert shape from vector<int> to vector<size_t>

* change debug log format

* format

* remove useless dropout

* add sdpa layer into multi head attention layer

* fix conflict parameter: is_post_ln and pre_or_po...

* fix CMakeLists.txt compile

* multi kernel develop

* lightseq transformer.cu fix

* fix linear col/row major

* fix compile error

* fix lightseq post_ln network structure

* add transformer example & print error message

* add shape message

* format

* fix concat error

* add shape message

* fix beam search bug

---------

Co-authored-by: Kangmo Kim <kangmo.kim@gmail.com>
Co-authored-by: Ying Xiong <xiongying.taka@bytedance.com>
Co-authored-by: Xiaohui Wang <wangxiaohui.neo@bytedance.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Taka152 Taka152 Taka152 approved these changes

@neopro12 neopro12 Awaiting requested review from neopro12 neopro12 is a code owner

@godweiyang godweiyang Awaiting requested review from godweiyang godweiyang is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@Kangmo @hexisyztem @Taka152