Skip to content

2357 Fixes unit test cases to run on GA100#3072

Merged
wyli merged 14 commits intodevfrom
fixes-a100
Oct 7, 2021
Merged

2357 Fixes unit test cases to run on GA100#3072
wyli merged 14 commits intodevfrom
fixes-a100

Conversation

@wyli
Copy link
Copy Markdown
Contributor

@wyli wyli commented Oct 5, 2021 •
edited
Loading

Fixes #2357

Description

set the atol/rtol for A100 tests

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Quick tests passed locally by running ./runtests.sh --quick --unittests.

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

1 similar comment
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

1 similar comment
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

2 similar comments
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 5, 2021

/build

@wyli wyli requested review from Nic-Ma and madil90 October 5, 2021 21:23
@wyli wyli marked this pull request as ready for review October 5, 2021 21:32
wyli added 7 commits October 6, 2021 09:47
@wyli
relax atol for a100
0b2f499 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
temp tests
b1b87f3 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
fixes #3071
b3a7279 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
remove temp tests
9a3b70e 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
fixes tests
a0a0905 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
update tests
d74565d 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
add tf32 tests
1c01c2a 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli wyli force-pushed the fixes-a100 branch 2 times, most recently from e47e0f7 to 137eb9a Compare October 6, 2021 09:11
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 6, 2021

/build

@wyli
fixes rtol wit tf32
22c4b53 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 6, 2021

/build

@wyli
detect tf32
29dd41c 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
wyli added 2 commits October 6, 2021 06:34
@wyli
update flag
15a673d 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
fixes testing util
2ab64d3 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 6, 2021

/build

@wyli wyli changed the title 2357 Fixes a100 2357 Fixes testing cases to run on GA100 Oct 6, 2021
@wyli wyli changed the title 2357 Fixes testing cases to run on GA100 2357 Fixes unit test cases to run on GA100 Oct 6, 2021
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 6, 2021

this is tested https://github.com/Project-MONAI/MONAI/actions/runs/1311658819 (premerge/135/) cc @madil90 @Nic-Ma

@wyli
Merge branch 'dev' into fixes-a100
7b7b085 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Oct 7, 2021

Hi @madil90 ,

Could you please help review this PR according to your test experience on A100 GPU?

Thanks in advance.

@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Oct 7, 2021

Hi @wyli ,

I want to double confirm that we will have new CI test server based on A100 GPU?

Thanks.

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 7, 2021

Hi @wyli ,

I want to double confirm that we will have new CI test server based on A100 GPU?

Thanks.

yes, now we have two A100 in use for the premerge tests. I'm interested in this fix because using this two additional nodes will accelerate the premerge pipelines.

@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Oct 7, 2021

Hi @wyli ,
I want to double confirm that we will have new CI test server based on A100 GPU?
Thanks.

yes, now we have two A100 in use for the premerge tests. I'm interested in this fix because using this two additional nodes will accelerate the premerge pipelines.

Great! Sounds good to me.

Thanks.

@wyli wyli enabled auto-merge (squash) October 7, 2021 18:35
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Oct 7, 2021

/build

@wyli wyli disabled auto-merge October 7, 2021 19:09
@wyli wyli merged commit 6a2086d into dev Oct 7, 2021
@wyli wyli deleted the fixes-a100 branch October 7, 2021 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Nic-Ma Nic-Ma Awaiting requested review from Nic-Ma Nic-Ma is a code owner

1 more reviewer

@madil90 madil90 madil90 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Tests failing on A100 due to tolerance

3 participants

@wyli @Nic-Ma @madil90