deps: Update transformers to latest and skip broken prompt tuning tests #246
Conversation
Abhishek-TAMU
requested review from
anhuong,
Ssukriti and
alex-jw-brooks
as code owners
anhuong
changed the title
| @@ -168,108 +168,108 @@ def test_parse_arguments_peft_method(job_config): | |||
| ############################# Prompt Tuning Tests ############################# | |||
|
|
|||
|
|
|||
| def test_run_causallm_pt_and_inference(): | |||
There was a problem hiding this comment.
Instead of commenting out these tests, lets add a pytest skip instead like @pytest.mark.skip(reason="currently crashes before validation is done") as seen here and then provide the reason and error
There was a problem hiding this comment.
Thank you for reviewing the PR Anh. I have added this in all 3 test cases.
|
In addition, please address the linter error and run |
Signed-off-by: Abhishek <maurya.abhishek@ibm.com>
anhuong
changed the title
tuning/sft_trainer.py
Outdated
| @@ -322,6 +322,7 @@ def train( | |||
| # this validation, we just drop the things that aren't part of the SFT Config and build one | |||
| # from our object directly. In the future, we should consider renaming this class and / or | |||
| # not adding things that are not directly used by the trainer instance to it. | |||
| train_args.use_cpu=True | |||
There was a problem hiding this comment.
Please leave this change out of the PR, although useful note that transformers 4.42 requires this change to run unit tests on macs!
tox.ini
Outdated
| @@ -5,6 +5,7 @@ envlist = py, lint, fmt | |||
| description = run unit tests | |||
| deps = | |||
| pytest>=7 | |||
| setuptools | |||
There was a problem hiding this comment.
Is setuptools needed to run unittests?
There was a problem hiding this comment.
On my pc it is required, but I am not sure if its required in the build. Hence removing it for now.
Signed-off-by: Abhishek <maurya.abhishek@ibm.com>
Signed-off-by: Abhishek <maurya.abhishek@ibm.com>
Signed-off-by: Abhishek <maurya.abhishek@ibm.com>
Description of the change
Related issue number
Previously we had seen degraded performance in v4.41 of transformers as described in issue #201. After further evaluation described in the issue, the throughput degradation on llama3-8B was only 17%, only on 2 A100, and not at all on 4 A100, and that memory utilization was significantly improved by moving to 4.41.
How to verify the PR
Was the PR tested