一个轻量的推理框架
目前验证过Mistral-7B，Qwen-1.5及llama3-70B等类llama模型，examples里有完整代码
- Tensor定义及相关接口
- 基础CUDA算子
- GPTQ推理支持
- 基于nccl的分布式支持
- function trace及graph pattern match功能
- DTensor接口
持续开发中
计划支持大部分常见特性，包括分布式，量化，LoDTensor，trace等
随缘更新
-
Notifications
You must be signed in to change notification settings - Fork 1
GetUpEarlier/minit
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published