Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

npu适配 #1250

Merged
merged 3 commits into from
Jul 3, 2024
Merged

npu适配 #1250

merged 3 commits into from
Jul 3, 2024

Conversation

baymax591
Copy link
Contributor

@baymax591 baymax591 commented Jun 17, 2024

What does this PR do?

We use this PR to adapt Opencompass to the NPU for huggingface side code. This change can help us evaluate the quality and effectiveness of NLP models on NPU.

By using mmengine.device to import is_npu_available, we can determine whether it is currently running on the NPU environment. That is to say, if is_npu_available() returns True, it indicates that there is an available NPU in the running environment. And then, we can use torch.npu.device_count() to determine the currently available devices.

What is Ascend and torch_npu

The following information is quoted from: Lightning-AI/pytorch-lightning#19498 (comment)

  1. Ascend is a full-stack AI computing infrastructure for industry applications and services based on Huawei Ascend processors and software. For more information about Ascend, see Ascend Community.
  2. torch_npu is an officially recognized pytorch integration plugin to support Ascend NPU using the pytorch framework(through key PRivateUse1), please see the PrivateUse1 tutorial here.
  3. Ascend is currently one of the premier members of the PyTorch Foundation.

@baymax591
Copy link
Contributor Author

In order to pass this PR, we generated log documentation on both NPU and GPU. Thanks for reviewing!

The following are my command and log screenshot:

  1. NPU:
    python run.py --models hf_qwen1_5_0_5b --datasets siqa_gen winogrande_gen
    NPU-qwen1 5-0 5b
  2. GPU:
    python run.py --models hf_qwen1_5_0_5b --datasets siqa_gen winogrande_gen
    GPU-qwen1 5-0 5b

@baymax591
Copy link
Contributor Author

cc @Leymore

@statelesshz
Copy link

cc @bittersweet1999

@Leymore Leymore merged commit 28eba6f into open-compass:main Jul 3, 2024
9 checks passed
Leymore added a commit to Leymore/opencompass that referenced this pull request Jul 12, 2024
* npu适配

* Add suport for Ascend NPU

* format

---------

Co-authored-by: baymax591 <14428251+baymax591@user.noreply.gitee.com>
Co-authored-by: Leymore <zfz-960727@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants