Add MLCube wrapper for metrics API

[hasan7n]

Description: TBD

Proposed Changes

• Refactor cli/deploy.py to reuse code for deploying GaNDLF as a metrics calculator MLCube
• Add a gandlf_deployMetrics command. This command will accept a custom entrypoint script that do some preprocessing of inputs before calling GaNDLF's generate metrics command. One usecase (for MedPerf) is if input predictions and labels are expected to come from different MLCube mount points, the MLCube author can use gandlf_deployMetrics and pass to it a custom script that first combines predictions and labels to form a single data input csv file and then call the gandlf_generateMetrics command. Example scripts are provided in mlcube/metrics_mlcube directory.

Checklist

• I have read the CONTRIBUTING guide.
• My PR is based from the current GaNDLF master .
• Non-breaking change (does not break existing functionality): provide as many details as possible for any breaking change.
• Function/class source code documentation added/updated.
• Code has been blacked for style consistency.
• If applicable, version information has been updated in GANDLF/version.py.
• If adding a git submodule, add to list of exceptions for black styling in pyproject.toml file.
• Usage documentation has been updated, if appropriate.
• History has been updated, if appropriate.
• Tests added or modified to cover the changes; if coverage is reduced, please give explanation.
• If customized dependency installation is required (i.e., a separate pip install step is needed for PR to be functional), please ensure it is reflected in all the files that control the CI, namely: python-test.yml, and all docker files [1,2,3].

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[codecov]

Codecov Report

Merging #681 (d194a4b) into master (7a02223) will increase coverage by 0.00%.
The diff coverage is 96.82%.

@@           Coverage Diff           @@
##           master     #681   +/-   ##
=======================================
  Coverage   94.59%   94.60%           
=======================================
  Files         116      116           
  Lines        8017     8038   +21     
=======================================
+ Hits         7584     7604   +20     
- Misses        433      434    +1     

Flag	Coverage Δ
unittests	94.60% <96.82%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
GANDLF/cli/deploy.py	89.47% <96.36%> (-0.53%)	⬇️
GANDLF/cli/__init__.py	100.00% <100.00%> (ø)
testing/test_full.py	99.15% <100.00%> (+<0.01%)	⬆️

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[hasan7n]

@AlexanderGetka-cbica @sarthakpati Please start reviewing. I am left with adding a unittest and modifying documentation if necessary.

Also a question: do you know if Dev-COntainer CI check fails randomly? I think so.

[sarthakpati]

@AlexanderGetka-cbica @sarthakpati Please start reviewing. I am left with adding a unittest and modifying documentation if necessary.

Is it possible to combine gandf_deploy and gandlf_deployMetrics? I would imagine from a usability perspective, having them in the same script might be easier. What do you think?

Also a question: do you know if Dev-COntainer CI check fails randomly? I think so.

I have re-initiated the test. Haven't seen that happening before, TBH.

[sarthakpati]

Some minor changes, but the major one if gandlf_deployMetrics and gandlf_deploy can be combined.

[hasan7n]

Is it possible to combine gandf_deploy and gandlf_deployMetrics? I would imagine from a usability perspective, having them in the same script might be easier. What do you think?

I agree. I split them because I though it would be confusing for the user to see many optional args depending on the mlcube type they want to deploy. But on a second thought, I guess it should be fine with some good documentation

[hasan7n]

@sarthakpati @AlexanderGetka-cbica I think this PR is done from my side. I also tested a generated metrics MLCube with MedPerf and its compatibility with a model created from GaNDLF. All is working.

I see two tests failing: any idea?

[sarthakpati]

Minor changes, should be good to go once we figure out what's going on with devcontainers/features#598

[sarthakpati]

Minor changes, should be good to go once we figure out what's going on with devcontainers/features#598

#683 should take care of this.

[sarthakpati]

LGTM