Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix PABEE & PL CI failure #6433

Closed
wants to merge 12 commits into from
Closed

Fix PABEE & PL CI failure #6433

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
9fbcfa9
Debug PABEE CI failure
JetRunner Aug 12, 2020
f9e0424
Update test_run_glue_with_pabee.py
JetRunner Aug 12, 2020
f179260
Update test_run_glue_with_pabee.py
JetRunner Aug 12, 2020
1767a16
Update config.yml
JetRunner Aug 12, 2020
7136239
Update config.yml
JetRunner Aug 12, 2020
cd1ca4c
Update test_examples.py
JetRunner Aug 12, 2020
255ecdc
Update test_examples.py
JetRunner Aug 12, 2020
2e49a20
Update test_examples.py
JetRunner Aug 12, 2020
91bbd4d
Update test_examples.py
JetRunner Aug 12, 2020
0edaaa5
Update test_examples.py
JetRunner Aug 13, 2020
60203d1
Update test_examples.py
JetRunner Aug 13, 2020
22dc12f
Update test_examples.py
JetRunner Aug 13, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 1 addition & 2 deletions examples/bert-loses-patience/test_run_glue_with_pabee.py
View file
Open in desktop
Expand Up @@ -20,7 +20,6 @@ def get_setup_file():


class PabeeTests(unittest.TestCase):
@unittest.skip("Disable while Canwen investigates.")
def test_run_glue(self):
stream_handler = logging.StreamHandler(sys.stdout)
logger.addHandler(stream_handler)
Expand All @@ -33,7 +32,7 @@ def test_run_glue(self):
--task_name mrpc
--do_train
--do_eval
--output_dir ./tests/fixtures/tests_samples/temp_dir
--output_dir ./tests/fixtures/tests_samples/pabee_temp_dir
--per_gpu_train_batch_size=2
--per_gpu_eval_batch_size=1
--learning_rate=2e-5
Expand Down
16 changes: 4 additions & 12 deletions examples/test_examples.py
View file
Open in desktop
Expand Up @@ -90,10 +90,11 @@ def test_run_pl_glue(self):
--task mrpc
--do_train
--do_predict
--output_dir ./tests/fixtures/tests_samples/temp_dir
--output_dir ./tests/fixtures/tests_samples/pl_temp_dir
--train_batch_size=32
--learning_rate=1e-4
--num_train_epochs=1
--warmup_steps=3
--num_train_epochs=4
--seed=42
--max_seq_length=128
""".split()
Expand All @@ -103,16 +104,7 @@ def test_run_pl_glue(self):

with patch.object(sys, "argv", testargs):
result = run_pl_glue.main()
# for now just testing that the script can run to a completion
self.assertGreater(result["acc"], 0.25)
#
# TODO: this fails on CI - doesn't get acc/f1>=0.75:
#
# # remove all the various *loss* attributes
# result = {k: v for k, v in result.items() if "loss" not in k}
# for k, v in result.items():
# self.assertGreaterEqual(v, 0.75, f"({k})")
#
self.assertGreater(result["acc"], 0.75)

def test_run_language_modeling(self):
stream_handler = logging.StreamHandler(sys.stdout)
Expand Down