Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] 支持无apex、flash-attention的执行模式 #93

Closed
1 task
zhangxc11 opened this issue Jul 17, 2023 · 2 comments
Closed
1 task

[Feature] 支持无apex、flash-attention的执行模式 #93

zhangxc11 opened this issue Jul 17, 2023 · 2 comments
Assignees
@sunpengsdu
Labels
enhancement New feature or request planned issue submitted by InternLM team

Comments

@zhangxc11
Copy link
Contributor

描述该功能

目前的仓库有几个复杂的依赖库apex、flash-attention，需要可以在没有这些依赖的条件下，可以fallback执行。

  1. 提供无apex、flash-attention条件下的fallback执行模式
  2. 支持opencompass评测直接运行
  3. 支持pip安装

是否希望自己实现该功能？

  • 我希望自己来实现这一功能，并向 InternLM 贡献代码！
@zhangxc11 zhangxc11 added the enhancement New feature or request label Jul 17, 2023
@zhangxc11 zhangxc11 added the planned issue submitted by InternLM team label Jul 17, 2023
@dijkstra-mose
Copy link

没人维护这个网站。装了这两个后，原来的训练的环境慢了很多。

@yingtongxiong
Copy link
Collaborator

  1. 目前已支持非apex版本，已合入main分支。
  2. 非flash-attention版本已支持了attention、linearParallel算子的去flash化。但还是有部分其他算子，如loss等还在使用flash的版本。
  3. 已提供docker镜像，Standard and experiment docker #220

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sunpengsdu sunpengsdu

Labels
enhancement New feature or request planned issue submitted by InternLM team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@zhangxc11 @sunpengsdu @dijkstra-mose @yingtongxiong