Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize Header Handling when creating new nibabel object #758

Merged
merged 18 commits into from
Nov 6, 2021

Conversation

cakester
Copy link
Contributor

@cakester cakester commented Apr 15, 2021

Checklist

GitHub

  • I've given this PR a concise, self-descriptive, and meaningful title
  • I've linked relevant issues in the PR body
  • I've applied the relevant labels to this PR
  • I've assigned a reviewer

PR contents

Description

In PR #714 the way nibabel objects are created was changed, so we want to update the object creation in the following places as well

  • uncertainty
  • preprocessing
  • postprocessing
  • visualize
  • loader/adaptive
  • evaluation

Linked issues

Timeline of the related issues:

Resolve #715
Resolve #775

@cakester cakester requested a review from dyt811 April 15, 2021 14:03
@cakester
Copy link
Contributor Author

all changed except for

nib_image = nib.Nifti1Image(input_data, np.eye(4))

@coveralls
Copy link

coveralls commented Apr 15, 2021

Pull Request Test Coverage Report for Build 1427913172

  • 4 of 16 (25.0%) changed or added relevant lines in 7 files are covered.
  • 4 unchanged lines in 3 files lost coverage.
  • Overall coverage remained the same at 70.777%

Changes Missing Coverage Covered Lines Changed/Added Lines %
ivadomed/evaluation.py 0 1 0.0%
ivadomed/training.py 0 2 0.0%
ivadomed/visualize.py 0 2 0.0%
ivadomed/uncertainty.py 0 7 0.0%
Files with Coverage Reduction New Missed Lines %
ivadomed/evaluation.py 1 12.39%
ivadomed/visualize.py 1 47.92%
ivadomed/uncertainty.py 2 12.37%
Totals Coverage Status
Change from base Build 1427243580: 0.0%
Covered Lines: 4316
Relevant Lines: 6098

💛 - Coveralls

@cakester cakester changed the title copy header when creating new nibabel objects Copy header when creating new nibabel objects Apr 15, 2021
@dyt811 dyt811 self-assigned this May 2, 2021
@dyt811 dyt811 removed the request for review from jcohenadad May 2, 2021 22:29
@dyt811 dyt811 marked this pull request as draft May 2, 2021 22:32
@dyt811
Copy link
Member

dyt811 commented May 2, 2021

Going to need a bit more time to review this. Converting to draft for now.

Copy link
Member

@dyt811 dyt811 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great start. You caught majority of the places that require fixing already. There are still a few areas that would require patching up, I will detail them below.

@dyt811
Copy link
Member

dyt811 commented May 16, 2021

image

I think you just missed a few of the places when the NifTI images are instantiated, the header didn't get copied over.

I think once you address those, we can sit with @charleygros again to quickly ensure these changes makes sense. I do not see any major issues in terms of implementation as we are replicating what was done in Inference.py without changing anything significantly.

@jcohenadad
Copy link
Member

please see spinalcordtoolbox/spinalcordtoolbox#3295 (comment). I don't think it is the best strategy to implement the suggestion of #715 (comment) without strategizing on the potential consequences (e.g. nipy/nibabel#1012).

@dyt811
Copy link
Member

dyt811 commented May 17, 2021

@jcohenadad acknowledged. Will low prioritize this while monitoring that upstream thread. Thanks for the heads up. I somehow thought it had been resolved. Will update PR with the missing contextual details.

Just saw the latest update on the latest related issue and the closed upstream ticket. Sorry been missing context too long 😨. Will prioritize attempting the proposed resolution.

Current action plan:

  • Implement the header.get_best_affine() in place of none to affine parameter during nibabel instantiation process. (Completed by @cakester)
  • Ensure this is done for ALL cases listed in the pycharm search results area where nibabel objects were instantiated. (Completed by @cakester)
  • Replicate, reproduce the tutorial which generated the issue Affine matrix error in Cascaded architecture tutorial #775 and validate earlier issue did not reoccur.

@dyt811 dyt811 changed the title Copy header when creating new nibabel objects Standardize Header Handling when creating new nibabel object May 17, 2021
@cakester cakester marked this pull request as ready for review May 17, 2021 15:06
@dyt811 dyt811 marked this pull request as draft May 22, 2021 17:06
@pytest.mark.script_launch_mode('subprocess')
def test_cascaded_architecture_tutorial(script_runner):
download_dataset("data_example_spinegeneric")
file_config = os.path.join(__data_testing_dir__, 'cascaded_architecture_tutorial_config.json')
Copy link
Contributor Author

@cakester cakester May 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note for this to work you have to upload the config.json file to the data_functional_test directory just like automated_test_config

download_dataset("data_example_spinegeneric")
file_config = os.path.join(__data_testing_dir__, 'cascaded_architecture_tutorial_config.json')

ret = script_runner.run('ivadomed', '-c', f'{file_config}',
Copy link
Contributor Author

@cakester cakester May 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because of this function which gets executed in ivadomed main
`
def __get_commit(path_to_git_folder=None):
"""Get GIT ivadomed commit.

Args:
    path_to_git_folder (str): Path to GIT folder.
Returns:
    str: git commit ID, with trailing '*' if modified.
"""
if path_to_git_folder is None:
    path_to_git_folder = __ivadomed_dir__
else:
    path_to_git_folder = os.path.abspath(os.path.expanduser(path_to_git_folder))

p = subprocess.Popen(["git", "rev-parse", "HEAD"], stdout=subprocess.PIPE, stderr=subprocess.PIPE,
                     cwd=path_to_git_folder)

`
since the existance of os.path.abspath it assumes that current working directory is ivadomed root (which is not unless you run pytest inside the root directory) we can only execute this test inside ivadomed root directory or else it would flail...actually no it's the cwd argument for subprocess that's failing

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the existance of os.path.abspath it assumes that current working directory is ivadomed root (which is not unless you run pytest inside the root directory) we can only execute this test inside ivadomed root directory or else it would flail

good catch @cakester ! it would be great to solve that issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, places such as

    def write_derivatives_dataset_description(self, path_data):
        """Writes default dataset_description.json file if not found in path_data/derivatives folder
        """
        filename = 'dataset_description'
        deriv_desc_file = '{}/derivatives/{}.json'.format(path_data, filename)
        label_desc_file = '{}/derivatives/labels/{}.json'.format(path_data, filename)
        # need to write default dataset_description.json file if not found
        if not os.path.isfile(deriv_desc_file) and not os.path.isfile(label_desc_file):
            f = open(deriv_desc_file, 'w')
            f.write('{"Name": "Example dataset", "BIDSVersion": "1.0.2", "PipelineDescription": {"Name": "Example pipeline"}}')
            f.close()

assumes that path data is relative to the current running directory, which is not when we execute tests in the testing directory, that further prevents this test to be run inside testing dir

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

problem solved, we change directory before running the tests!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

problem solved, we change directory before running the tests!

is this directory change now implemented? is it done automatically in the code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's only done with this test, do you want it for all of ivadomed?

@cakester cakester marked this pull request as ready for review June 16, 2021 16:48
@cakester
Copy link
Contributor Author

someone approve this ivadomed/data_functional_testing#6

@cakester
Copy link
Contributor Author

note this test is very long compare to all other tests...

what test are you referring to? how long is it compare to other tests?

testing/functional_tests/test_cascaded_architecture_tutorial.py, around 23 min for this test...

@dyt811
Copy link
Member

dyt811 commented Aug 25, 2021

note this test is very long compare to all other tests...

what test are you referring to? how long is it compare to other tests?

testing/functional_tests/test_cascaded_architecture_tutorial.py, around 23 min for this test...

That is actually fine. As tutorial typically run very long. We will not be incorporating this as part of the regular unit testing but an optional integration testing that is run on either a weekly or more optinally triggered.

@mariehbourget
Copy link
Member

mariehbourget commented Aug 25, 2021

note this test is very long compare to all other tests...

what test are you referring to? how long is it compare to other tests?

testing/functional_tests/test_cascaded_architecture_tutorial.py, around 23 min for this test...

That is actually fine. As tutorial typically run very long. We will not be incorporating this as part of the regular unit testing but an optional integration testing that is run on either a weekly or more optinally triggered.

I have a suggestion for this. In the cascaded_architecture_tutorial_config.json config file for this test, the number of epochs of training is set to 100. If the purpose is only to make sure the training runs 1 epoch would be enough., a few epochs (for -t and -g options) would be enough.

@cakester
Copy link
Contributor Author

cakester commented Sep 7, 2021

note once #865 this got in, we don't need cascade tutorial test here anymore

@dyt811
Copy link
Member

dyt811 commented Oct 26, 2021

This change has been empirically validated to address bugs surfaced in Tutorial 2 and prevent its crash as documented in the tutorial. This needs to go in before the tutorial testing related changes.

@dyt811 dyt811 self-requested a review October 26, 2021 20:48
@dyt811
Copy link
Member

dyt811 commented Nov 5, 2021

This PR's change has been empirically validated in yd/tutorial_testing as of 2021-11-05T194421EST. This PR will be merged first as part of the Python 3.9/Torch 1.8 adoption. This addressed the issue in Tutorial 2 crash that is currently happening in the master branch, as mentioned in #715

ivadomed/postprocessing.py Outdated Show resolved Hide resolved
ivadomed/uncertainty.py Outdated Show resolved Hide resolved
ivadomed/uncertainty.py Outdated Show resolved Hide resolved
Copy link
Member

@dyt811 dyt811 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor adjustments are required before merging to ensure that headers are CONSISTENTLY copied in ALL places where we are invoking a Nifti1Image.

@dyt811 dyt811 merged commit 881f03f into master Nov 6, 2021
@dyt811 dyt811 deleted the hm/copy_header branch November 6, 2021 01:50
@dyt811 dyt811 removed their assignment Nov 6, 2021
@dyt811 dyt811 added this to the new release milestone Nov 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
5 participants