fix: metric flag while visualizing

[kanishk16]

Checklist

GitHub

• I've given this PR a concise, self-descriptive, and meaningful title
• I've linked relevant issues in the PR body
• I've applied the relevant labels to this PR
• I've assigned a reviewer

PR contents

• I've consulted ivadomed's internal developer documentation to ensure my contribution is in line with any relevant design decisions
• I've added relevant tests for my contribution
• I've updated the relevant documentation for my changes, including argparse descriptions, docstrings, and ReadTheDocs tutorial pages

Description

This PR resolves the bug while using the --metric flag in the visualize_and_compare_testing_models.py script. Furthermore, the datapoints within each violinplot are made fixed to be interactive as they earlier usesd to be. Although, the visualization works completely fine but fails in case of using a Jupyter Notebook or a Google Colab Notebook.

To test this PR:

1. Download & extract the zip file: viz_test.zip

2. run ivadomed_visualize_and_compare_testing_models as suggested in the docs without the metadata flag.
   To test the --metric flag, select any evaluation metric from evaluation_3Dmetrics.csv including from dice_class0. Also, the features in the docs should be enabled.

Linked issues

resolves #1138

[mariehbourget]

Thank you @kanishk16 for looking into this!
The metric argument parsing is working well now.

Unfortunately, I'm unable to review the visualization, as it is also not working on my setup (WSL) with the following error:

UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show(block=True)

I have little time to debug this, so I would recommend someone else review the visuazlization part.
Thanks again!

[kanishk16]

No issues... IIUC the visualization is bound to fail when the backend is agg as visible here:

ivadomed/ivadomed/scripts/visualize_and_compare_testing_models.py

Lines 25 to 28 in f7e9d9f

	if matplotlib.get_backend() == "agg":
	logger.warning("No backend can be used - Visualization will fail")
	else:
	logger.info(f"Using: {matplotlib.get_backend()} gui")

[lifetheater57]

@kanishk16 How should I test this PR?

[kanishk16]

@lifetheater57 My bad, I should've added the section how to test the PR earlier, nevertheless I've updated it.

[coveralls]

Pull Request Test Coverage Report for Build 3361273310

• 1 of 11 (9.09%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 69.04%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
ivadomed/scripts/visualize_and_compare_testing_models.py	0	10	0.0%

Totals
Change from base Build 3361122051:	0.0%
Covered Lines:	4237
Relevant Lines:	6137

---

💛 - Coveralls

[naga-karthik]

For some reason, I am getting this error on testing this PR:

what I ran: ivadomed_visualize_and_compare_testing_models --ofolders model1 model2 --metric dice_class0

Error Trace

I am not sure whether this is due to the PR (does seem like it though) and something do with my ivadomed version. Just a sanity check, I also ran the 2D UNet segmentation tutorial, and that started without any issues.
Any thoughts on what might be the issue here?

[kanishk16]

Could you try adding the complete path for the model1 and model2?

[naga-karthik]

Nope, getting the same error as above even with the full folder paths 😞

updated folder path

[kanishk16]

Ahh, the error observed is the original one which seems to have fixed... It primarily stems from how the --metric flag parses the argument (#1138 (comment))

Could you check if the error persists on running the file as a script?

[naga-karthik]

Sure! If you meant running it with python like (python visualize_and_compare_testing_models.py --ofolders ~/code/ivadomed/viz_test/model1 ~/code/ivadomed/viz_test/model2 --metric dice_class0) instead of ivadomed_visualize_and_compare_testing_models, then yes, I tried that and am still getting the same error.

What's surprising is that when I tried without the --metric argument (as suggested in the above PR comment), the code ran and I could see the output figure.

[naga-karthik]

Just tested it, works fine me! Just a note though, the plot is not interactive. I can just see the plots.

[kanishk16]

Idk why but the plots should be interactive... Why do I feel even those changes aren't there