Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pytest workflow and the Ray mode. #20

Merged
merged 16 commits into from
Jul 4, 2024
Merged

Add pytest workflow and the Ray mode. #20

merged 16 commits into from
Jul 4, 2024

Conversation

garrett4wade
Copy link
Contributor

@garrett4wade garrett4wade commented Jul 3, 2024

  1. Add a pytest github workflow to run pytest -m "not slow" upon each pull request.
  2. Support Ray mode. Quickstart commands with mode=ray will launch distributed jobs with ray. This mode assumes that the user has launched a ray cluster with ray start CLI. Launching ray clusters with srun is no longer supported because it increases the complexity.
  3. Move the CUDA GAE test from legacy code to the current repo.
  4. Add more comments in the scripts in the examples folder.
  5. Fix the unnecessary nan/inf check when loading/saving checkpoints. Now save & load is as fast as before Loading parameters takes exceptionally long time. #18 .
  6. Fix a generate bug when only the LM head resides on the last pipeline stage.

Future works:

  • Change the doc correspondingly.

@garrett4wade garrett4wade changed the title [WIP] Add pytest workflow and the Ray mode. Add pytest workflow and the Ray mode. Jul 3, 2024
@garrett4wade garrett4wade requested a review from nuzant July 3, 2024 07:31
Copy link
Collaborator

@nuzant nuzant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nuzant nuzant merged commit 56cd1a3 into main Jul 4, 2024
3 checks passed
@garrett4wade garrett4wade deleted the pytest branch July 4, 2024 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants