Add Ascend NPU support

[zhangsibo1129]

Why are these changes needed?

This PR integrates Ascend NPU hardware capabilities into the FastChat, and enables users to leverage the NPUs for training, inference and serving of LLMs.

Ascend NPU is an AI processor that support AI frameworks like PyTorch, TensorFlow, which has already integrated by huggingface, deepspeed and others popular LLM related software and tools.

Related issue number (if applicable)

N/A

Checks

• I've run format.sh to lint the changes in this PR.
• I've included any doc changes needed.
• I've made sure the relevant tests are passing (if applicable).

Use Cases

inference with cli test on Ascend device

$ python -m fastchat.serve.cli --model-path ~/vicuna-7b-v1.5-16k --device npu 

USER: hello
ASSISTANT: Hello! How can I assist you today?
USER: Provide three most famous deep learning frameworks
ASSISTANT: Sure, here are three of the most famous deep learning frameworks:

1. TensorFlow: Developed by Google, TensorFlow is an open-source deep learning framework that is widely used for building and deploying machine learning models. It is known for its flexibility and scalability, and is used by researchers and developers alike.
2. PyTorch: Developed by Facebook, PyTorch is another popular deep learning framework that is known for its ease of use and flexibility. It is often used for research purposes, as well as for building and deploying machine learning models.
3. Keras: Keras is a high-level deep learning framework that is built on top of TensorFlow. It is designed to be user-friendly and easy to use, making it a popular choice for beginners and researchers alike. Keras is also known for its ability to quickly prototype and test new ideas.
USER: 

Fine-tuning test with multi Ascend devices

torchrun --nproc_per_node=4 --master_port=20001 fastchat/train/train.py \
    --model_name_or_path ~/vicuna-7b-v1.5-16k  \
    --data_path data/dummy_conversation.json \
    --fp16 True \
    --output_dir output_vicuna \
    --num_train_epochs 3 \
    --per_device_train_batch_size 8 \
    --per_device_eval_batch_size 1 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2000 \
    --save_total_limit 200 \
    --learning_rate 1e-5 \
    --weight_decay 0. \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --fsdp "full_shard auto_wrap" \
    --fsdp_transformer_layer_cls_to_wrap 'LlamaDecoderLayer' \
    --tf32 False \
    --model_max_length 512 \
    --gradient_checkpointing True \
    --lazy_preprocess True | tee train.log

[2023-09-13 10:20:36,763] torch.distributed.run: [WARNING] 
[2023-09-13 10:20:36,763] torch.distributed.run: [WARNING] *****************************************
[2023-09-13 10:20:36,763] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
[2023-09-13 10:20:36,763] torch.distributed.run: [WARNING] *****************************************
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:12<00:00,  6.04s/it]
Loading checkpoint shards:  50%|███████████████████████████████████████████                                           | 1/2 [00:13<00:13, 13.34s/it][W LegacyTypeDispatch.h:74] Warning: AutoNonVariableTypeMode is deprecated and will be removed in 1.10 release. For kernel implementations please use AutoDispatchBelowADInplaceOrView instead, If you are looking for a user facing API to enable running your inference-only workload, please use c10::InferenceMode. Using AutoDispatchBelowADInplaceOrView in user code is under risk of producing silent wrong result in some edge cases. See Note [AutoDispatchBelowAutograd] for more details. (function operator())
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:17<00:00,  8.99s/it]
[W LegacyTypeDispatch.h:74] Warning: AutoNonVariableTypeMode is deprecated and will be removed in 1.10 release. For kernel implementations please use AutoDispatchBelowADInplaceOrView instead, If you are looking for a user facing API to enable running your inference-only workload, please use c10::InferenceMode. Using AutoDispatchBelowADInplaceOrView in user code is under risk of producing silent wrong result in some edge cases. See Note [AutoDispatchBelowAutograd] for more details. (function operator())
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:18<00:00,  9.24s/it]
[W LegacyTypeDispatch.h:74] Warning: AutoNonVariableTypeMode is deprecated and will be removed in 1.10 release. For kernel implementations please use AutoDispatchBelowADInplaceOrView instead, If you are looking for a user facing API to enable running your inference-only workload, please use c10::InferenceMode. Using AutoDispatchBelowADInplaceOrView in user code is under risk of producing silent wrong result in some edge cases. See Note [AutoDispatchBelowAutograd] for more details. (function operator())
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:49<00:00, 24.53s/it]
Loading data...
Formatting inputs...Skip in lazy mode
[W LegacyTypeDispatch.h:74] Warning: AutoNonVariableTypeMode is deprecated and will be removed in 1.10 release. For kernel implementations please use AutoDispatchBelowADInplaceOrView instead, If you are looking for a user facing API to enable running your inference-only workload, please use c10::InferenceMode. Using AutoDispatchBelowADInplaceOrView in user code is under risk of producing silent wrong result in some edge cases. See Note [AutoDispatchBelowAutograd] for more details. (function operator())
  0%|                                                                                                                        | 0/87 [00:00<?, ?it/s][W OpCommand.cpp:117] Warning: [Check][offset] Check input storage_offset[%ld] = 0 failed, result is untrustworthy16777216 (function operator())
[W OpCommand.cpp:117] Warning: [Check][offset] Check input storage_offset[%ld] = 0 failed, result is untrustworthy16777216 (function operator())
[W OpCommand.cpp:117] Warning: [Check][offset] Check input storage_offset[%ld] = 0 failed, result is untrustworthy16777216 (function operator())
[W OpCommand.cpp:117] Warning: [Check][offset] Check input storage_offset[%ld] = 0 failed, result is untrustworthy16777216 (function operator())
[W VariableFallbackKernel.cpp:40] Warning: The operator 'aten::linalg_vector_norm.out' is not currently supported on the NPU backend and will fall back to run on the CPU. This may have performance implications. (function operator())
[W VariableFallbackKernel.cpp:40] Warning: The operator 'aten::linalg_vector_norm.out' is not currently supported on the NPU backend and will fall back to run on the CPU. This may have performance implications. (function operator())
[W VariableFallbackKernel.cpp:40] Warning: The operator 'aten::linalg_vector_norm.out' is not currently supported on the NPU backend and will fall back to run on the CPU. This may have performance implications. (function operator())
[W VariableFallbackKernel.cpp:40] Warning: The operator 'aten::linalg_vector_norm.out' is not currently supported on the NPU backend and will fall back to run on the CPU. This may have performance implications. (function operator())
  1%|█▎                                                                                                           | 1/87 [02:38<3:47:11, 158.51s/it]{'loss': 0.2701, 'learning_rate': 9.996740476948386e-06, 'epoch': 0.03}                                                                             
{'loss': 0.9416, 'learning_rate': 9.986966157589751e-06, 'epoch': 0.07}                                                                             
{'loss': 0.2061, 'learning_rate': 9.970689785771798e-06, 'epoch': 0.1}                                                                              
{'loss': 0.3403, 'learning_rate': 9.947932582778188e-06, 'epoch': 0.14}                                                                             

// ... ...
                                                                          
{'loss': 0.1071, 'learning_rate': 5.206741722181385e-08, 'epoch': 2.86}                                                                             
{'loss': 0.1074, 'learning_rate': 2.9310214228202016e-08, 'epoch': 2.9}                                                                             
{'loss': 0.1088, 'learning_rate': 1.3033842410251074e-08, 'epoch': 2.93}                                                                            
{'loss': 0.1061, 'learning_rate': 3.2595230516152543e-09, 'epoch': 2.97}                                                                            
{'loss': 0.1076, 'learning_rate': 0.0, 'epoch': 3.0}                                                                                                
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 87/87 [2:19:10<00:00, 93.97s/it]{'train_runtime': 8350.1159, 'train_samples_per_second': 0.327, 'train_steps_per_second': 0.01, 'train_loss': 0.14017466147398128, 'epoch': 3.0}    
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 87/87 [2:19:16<00:00, 96.05s/it]

[Yikun]

LGTM, thanks, I also do a E2E test for vicuna model inference in my env with NPU device, it works.

@merrymercy Would you mind taking a look?

[statelesshz]

Just a comment about the training argument tf32. Otherwise all looks good!

[statelesshz]

Ascend NPU currently does not support tf32 regardless of whether you set it to true or false. I would prefer to remove the line 340: --fp32 True

[zhangsibo1129]

Removed

[merrymercy]

@zhangsibo1129 Thanks! It is merged.