| Platform | Build Status |
|---|---|
| AX650 |  |
AX-LLM 由 爱芯元智 主导开发。该项目用于探索业界常用 LLM(Large Language Model) 在已有芯片平台上落地的可行性和相关能力边界，方便社区开发者进行快速评估和二次开发自己的 LLM 应用。
- AX650A/AX650N
- TinyLLaMa-1.1B
- Qwen1.5-1.8B
- 递归 clone 本项目，确保所有
submodule正确 clonegit clone --recursive https://github.com/AXERA-TECH/ax-llm.git cd ax-llm - 仔细阅读
build.sh，并在build.sh正确修改BSP_MSP_DIR变量后，运行编译脚本./build.sh
- 正确编译后，
build/install/bin目录，应有以下文件（百度网盘中有预编译的可执行程序）$ tree install/bin/ install/bin/ ├── main ├── run_bf16.sh └── run_qwen_1.8B.sh
./run_phi3_mini.sh
[I][ Init][ 71]: LLM init start
100% | ████████████████████████████████ | 35 / 35 [28.39s<28.39s, 1.23 count/s] init post axmodel okremain_cmm(9045 MB))
[I][ Init][ 180]: max_token_len : 1023
[I][ Init][ 185]: kv_cache_size : 3072, kv_cache_num: 1023
[I][ Init][ 199]: LLM init ok
Type "q" to exit, Ctrl+c to stop current running
>>
>> who are you?
I am Phi, an AI developed by Microsoft, designed to assist and provide information to users across a wide range of topics. How can I assist you today?
[N][ Run][ 388]: hit eos,avg 4.40 token/s
>>
>> use c program language implement calculate sum 1-9
Certainly! To calculate the sum of numbers from 1 to 9 in C, you can use a simple loop. Here's a small program that does exactly that:
`c
#include <stdio.h>
int main() {
int sum = 0; // Initialize sum to 0
// Loop from 1 to 9 and add each number to sum
for(int i = 1; i <= 9; i++) {
sum += i;
}
printf("The sum of numbers from 1 to 9 is: %d\n", sum);
return 0;
}
`
This program initializes a variable `sum` to 0. It then iterates from 1 to 9, adding each number to `sum`. Finally, it prints the result.
The sum of numbers from 1 to 9 is 45, as calculated by the loop in the program.
[N][ Run][ 388]: hit eos,avg 4.37 token/s
tinyllama1.1.mp4
# ./run_bf16.sh
[I][ Init][ 71]: LLM init start
100% | ████████████████████████████████ | 25 / 25 [21.82s<21.82s, 1.15 count/s] init post axmodel okremain_cmm(3760 MB)
[I][ Init][ 162]: max_token_len : 1023
[I][ Init][ 167]: kv_cache_size : 256, kv_cache_num: 1023
[I][ Init][ 176]: LLM init ok
Type "q" to exit, Ctrl+c to stop current running
>> write a c++ program to calculate 1-9 sum
`c++
#include <iostream>
using namespace std;
int main() {
int sum = 0;
int num;
cout << "Enter a number: ";
cin >> num;
for (int I = 1; I <= num; i++) {
sum += i;
}
cout << "The sum of 1 to " << num << " is: " << sum << endl;
return 0;
}
`
this program prompts the user to enter a number, then calculates the sum of 1 to `num` using a loop. The loop iterates from 1 to `num`, adding each number to the sum variable `sum`. Finally, the program prints the sum of 1 to `num`.
[N][ Run][ 366]: hit eos,avg 10.14 token/s
>> where is shenzhen
Shenzhen is a city located in southern China, on the southern coast of the Pearl River Delta. It is the capital of Guangdong province and one of the most important economic and cultural centers in China. Shenzhen is known for its innovation, technology, and entrepreneurship, and it is home to many of China's largest companies, including Huawei, Lenovo, and Xiaomi. The city is also a hub for international trade and investment, with many multinational corporations and financial institutions setting up offices and operations in Shenzhen
[N][ Run][ 366]: hit eos,avg 10.16 token/s
qwen1.8.mp4
# ./run_qwen_1.8B.sh
[I][ Init][ 71]: LLM init start
100% | ████████████████████████████████ | 27 / 27 [23.68s<23.68s, 1.14 count/s] init post axmodel okremain_cmm(4140 MB)
[I][ Init][ 162]: max_token_len : 1023
[I][ Init][ 167]: kv_cache_size : 2048, kv_cache_num: 1023
[I][ Init][ 176]: LLM init ok
Type "q" to exit, Ctrl+c to stop current running
>> 给我写个C++代码计算1-9的和
以下是一个简单的C++代码，用于计算1-9的和：
`cpp
#include <iostream>
int main() {
int sum = 1;
for (int i = 1; i <= 9; i++) {
sum += i;
}
std::cout << "The sum of 1-9 is: " << sum << std::endl;
return 0;
}
`
在这个代码中，我们首先定义了一个变量`sum`，并将其初始化为1。然后，我们使用一个`for`循环，从1开始，每次增加1，直到10。在每次循环中，我们都会将当前的数`i`加到`sum`中。最后，我们在主函数中打印出`sum`的值，即1-9的和。
[N][ Run][ 366]: hit eos,avg 8.00 token/s
>> 深圳在哪
深圳位于中国广东省南部，珠江口东侧，东临大亚湾，西濒珠江口，南界深圳湾，北界香港特别行政区，是中国四大一线城市之一，也是中国经济最发达的城市之一。
[N][ Run][ 366]: hit eos,avg 8.44 token/s- Github issues
- QQ 群: 139953715
