Skip to content

3621 fixes tests/test_handler_metrics_saver_dist.py#3673

Merged
wyli merged 11 commits intoProject-MONAI:devfrom
wyli:3621-adds-metrics-saver-barrier
Jan 25, 2022
Merged

3621 fixes tests/test_handler_metrics_saver_dist.py#3673
wyli merged 11 commits intoProject-MONAI:devfrom
wyli:3621-adds-metrics-saver-barrier

Conversation

@wyli
Copy link
Copy Markdown
Contributor

@wyli wyli commented Jan 18, 2022

Signed-off-by: Wenqi Li wenqil@nvidia.com

Fixes #3621

Description

adds a barrier to sync the processes

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@wyli wyli marked this pull request as ready for review January 20, 2022 11:17
@wyli wyli requested a review from Nic-Ma January 20, 2022 11:17
@Nic-Ma
Copy link
Copy Markdown
Contributor

Nic-Ma commented Jan 20, 2022

Hi @wyli ,

If barrier() works for this issue, could you please help add it to the DistCall test utility as @vfdev-5 suggested: #3641 (comment)
And I would suggest to change the get_rank() logic back to the original direct call.
What do you think?

Thanks in advance.

@wyli wyli enabled auto-merge (squash) January 24, 2022 14:55
@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Jan 24, 2022

/build

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Jan 25, 2022

I think this is good given the repeated tests are ok. I don't think the utility should be updated at the moment, unless we see different further issues... @Nic-Ma

Nic-Ma
Nic-Ma approved these changes Jan 25, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@Nic-Ma Nic-Ma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, looks good to me.

Thanks.

@wyli
Copy link
Copy Markdown
Contributor Author

wyli commented Jan 25, 2022

/build

@wyli wyli merged commit 3180c27 into Project-MONAI:dev Jan 25, 2022
wyli added a commit to wyli/MONAI that referenced this pull request Jan 25, 2022
@wyli
try to fix Project-MONAI#3621 (Project-MONAI#3673)
01e7f8d 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
wyli added a commit to wyli/MONAI that referenced this pull request Jan 26, 2022
@wyli
try to fix Project-MONAI#3621 (Project-MONAI#3673)
7b91be4 
Signed-off-by: Wenqi Li <wenqil@nvidia.com>
wyli added a commit that referenced this pull request Feb 3, 2022
@wyli @monai-bot
3697 add spatial resample (#3701)
5cce0af 
* temp spatial_resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes resampling

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes precisions

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update dict version

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds docs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* copy grid for resampling

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remove normalize coordinates

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* try to fix #3621 (#3673)

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes typing

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes grid_sample, interpolate URLs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* simplify norm_coords

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docstring

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update moveaxis

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* spatial sample tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* additional tests spatial resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* test invert saptial resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* rtol assert close

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes TF32 tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* smaller tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* skip when quick testing

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* comp tensor and ndarray

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* try to use torch.solve

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "fixes tests"

This reverts commit e532490.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes test_affined

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* default to float32 rotate/randrotate

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* workaround for #3752

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* default to float32 rotate/randrotate

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* temp test

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docstring

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
wyli added a commit that referenced this pull request Feb 4, 2022
@wyli @monai-bot
2823 enhance convert_data/dst_type typing (#3749)
816f413 
* temp spatial_resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes resampling

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes precisions

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update dict version

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds docs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* copy grid for resampling

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remove normalize coordinates

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* try to fix #3621 (#3673)

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes typing

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes grid_sample, interpolate URLs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* simplify norm_coords

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docstring

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update moveaxis

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* spatial sample tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* additional tests spatial resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* test invert saptial resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* rtol assert close

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes TF32 tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* smaller tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* skip when quick testing

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* comp tensor and ndarray

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* try to use torch.solve

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* temp updates

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* enhance typing

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* temp test

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "temp test"

This reverts commit 6200a38.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* enhance types

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update util

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* reverse workaround

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* formatting

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update type def.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* temp test

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* warn unused

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remote ignore

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "warn unused"

This reverts commit e645807.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* Revert "temp test"

This reverts commit ddc4770.

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

Co-authored-by: monai-bot <monai.miccai2019@gmail.com>
wyli added a commit that referenced this pull request Feb 7, 2022
@wyli
3595 3766 adds a base writer and an itk writer (#3674)
afcf593 
* temp spatial_resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes resampling

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes precisions

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update dict version

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds docs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* copy grid for resampling

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remove normalize coordinates

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* try to fix #3621 (#3673)

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes typing

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes grid_sample, interpolate URLs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* simplify norm_coords

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docstring

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update moveaxis

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* spatial sample tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* additional tests spatial resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* test invert saptial resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds a base writer and an itk writer

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docstrings

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remove return self

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds reorient_spatial_axes

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* sync 3701

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* try to fix #3766

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* revise docstring to be concise

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* 3765 Enhance `create_multigpu_supervised_XXX` for distributed (#3768)

* [DLMED] add check for devices

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <nma@nvidia.com>

* update to support dynamic spatial_size

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
wyli added a commit that referenced this pull request Feb 8, 2022
@wyli
3595 Adds nibabel/pil writers (#3772)
4d0baa0 
* temp spatial_resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes resampling

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes precisions

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update dict version

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds docs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* copy grid for resampling

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remove normalize coordinates

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* [MONAI] python code formatting

Signed-off-by: monai-bot <monai.miccai2019@gmail.com>

* try to fix #3621 (#3673)

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes typing

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes grid_sample, interpolate URLs

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* simplify norm_coords

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docstring

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update moveaxis

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* spatial sample tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* additional tests spatial resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* test invert saptial resample

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds a base writer and an itk writer

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update docstrings

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remove return self

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds reorient_spatial_axes

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* fixes unit tests

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* sync 3701

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* try to fix #3766

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* revise docstring to be concise

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* 3765 Enhance `create_multigpu_supervised_XXX` for distributed (#3768)

* [DLMED] add check for devices

Signed-off-by: Nic Ma <nma@nvidia.com>

* [DLMED] update according to comments

Signed-off-by: Nic Ma <nma@nvidia.com>

* update to support dynamic spatial_size

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* adds nibabel pil writers

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* remove unused ignore

Signed-off-by: Wenqi Li <wenqil@nvidia.com>

* update based on comments

Signed-off-by: Wenqi Li <wenqil@nvidia.com>
@wyli wyli deleted the 3621-adds-metrics-saver-barrier branch April 27, 2022 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Nic-Ma Nic-Ma Nic-Ma approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

test_handler_metrics_saver_dist error

2 participants

@wyli @Nic-Ma