huseinzol05 / context-parallelism Public

Notifications You must be signed in to change notification settings
Fork 4
Star 4

Context Parallelism using Flex Attention, support Ring Attention.

4 stars 4 forks Branches Tags Activity

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 27 Commits
benchmark		benchmark
context_parallelism		context_parallelism
playground		playground
test		test
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
requirements.txt		requirements.txt
setup.py		setup.py

Repository files navigation

context-parallelism

Support Ring Attention https://arxiv.org/abs/2310.01889 using Flex Attention.

About

Context Parallelism using Flex Attention, support Ring Attention.

Report repository

Releases

No releases published

Packages

No packages published

Languages