Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Min hardware requirement for 70B #96

Closed
dittops opened this issue Aug 4, 2023 · 2 comments
Closed

Min hardware requirement for 70B #96

dittops opened this issue Aug 4, 2023 · 2 comments
Labels
question Further information is requested

Comments

@dittops
Copy link

dittops commented Aug 4, 2023

What is the hardware requirement (GPU and CPU) for running 70B SFT without peft?
@HamidShojanazeri
Copy link
Contributor

@dittops please take a look this PR for low CPU usage, This has been tested on 2 nodes 8xA100 80 GB for FSDP only in case interested in PEFT + FSDP this should be more affordable with smaller settings as well.

@HamidShojanazeri HamidShojanazeri added the question Further information is requested label Aug 8, 2023
@dittops
Copy link
Author

dittops commented Aug 8, 2023

Thank you, that helps.

@dittops dittops closed this as completed Aug 8, 2023
@jshin49 jshin49 mentioned this issue Aug 15, 2023
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@HamidShojanazeri @dittops