Skip to content

Conversation

ydshieh
Copy link
Collaborator

@ydshieh ydshieh commented Sep 15, 2025

What does this PR do?

This is added when we switched from T4 to A10. We tried to do it in a progressive way but ended up doing it in one-go (after a few days) because it was confusing about the results, in particular during the debug and fix phase.

That change also caused the fsdp/traiiner job not being run due to a mistake where

echo "runner_map=$(python3 ../utils/get_runner_map.py)" >> $GITHUB_OUTPUT

is not added to the elif branch below

        run: |
          if [ "${{ inputs.job }}" = "run_models_gpu" ]; then
            echo "folder_slices=$(python3 ../utils/split_model_tests.py --models '${{ inputs.models }}' --num_splits ${{ env.NUM_SLICES }})" >> $GITHUB_OUTPUT
            echo "slice_ids=$(python3 -c 'd = list(range(${{ env.NUM_SLICES }})); print(d)')" >> $GITHUB_OUTPUT
            echo "runner_map=$(python3 ../utils/get_runner_map.py)" >> $GITHUB_OUTPUT
          elif [ "${{ inputs.job }}" = "run_trainer_and_fsdp_gpu" ]; then
            echo "folder_slices=[['trainer'], ['fsdp']]" >> $GITHUB_OUTPUT
            echo "slice_ids=[0, 1]" >> $GITHUB_OUTPUT
          fi

Since we don't need this anymore, let's just remove it.

@ydshieh ydshieh marked this pull request as draft September 15, 2025 09:49
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ydshieh ydshieh force-pushed the remove_runner_map branch 3 times, most recently from c62ea61 to eba03f7 Compare September 15, 2025 13:11
@ydshieh ydshieh marked this pull request as ready for review September 15, 2025 13:11
@ydshieh ydshieh merged commit 270da89 into main Sep 16, 2025
22 of 23 checks passed
@ydshieh ydshieh deleted the remove_runner_map branch September 16, 2025 13:18
ErfanBaghaei pushed a commit to ErfanBaghaei/transformers that referenced this pull request Sep 25, 2025
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
vijayabhaskar-ev pushed a commit to vijayabhaskar-ev/transformers that referenced this pull request Oct 2, 2025
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
yuchenxie4645 pushed a commit to yuchenxie4645/transformers that referenced this pull request Oct 4, 2025
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants