Support MLflow along with Tensorboard for logging segmentation task visualizations during validation

[entn-at]

When using PyTorch-Lightning's MLFlowLogger instead of TensorBoardLogger during training of segmentation models, the current implementation fails because MLflow's experiment tracking client has a different method for logging figures than Tensorboard. Unfortunately, PyTorch-Lightning doesn't abstract away this logger API difference.

Tested with MLflow and Tensorboard.

[codecov]

Codecov Report

Base: 29.92% // Head: 29.91% // Decreases project coverage by -0.00% ⚠️

Coverage data is based on head (869067e) compared to base (01405fd).
Patch coverage: 20.00% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1209      +/-   ##
===========================================
- Coverage    29.92%   29.91%   -0.01%     
===========================================
  Files           62       62              
  Lines         3813     3817       +4     
===========================================
+ Hits          1141     1142       +1     
- Misses        2672     2675       +3     

Impacted Files	Coverage Δ
pyannote/audio/tasks/segmentation/mixins.py	48.02% <20.00%> (-0.54%)	⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[hbredin]

Thanks for the PR...

... but don't you think it would be better to contribute to pytorch-lightning directly?

[entn-at]

Pytorch-lightning purposefully removed its partial image logging support less than a year ago (see Lightning-AI/pytorch-lightning#11857). The current pyannote.audio code depends on the logger being an instance of TensorBoardLogger and will fail if it is not.

Alternatively, would it be an option to add a flag that controls whether figures are logged during the validation step? Just so that training doesn't fail.

[hbredin]

Thanks for the link to the lightning PR: I was missing this additionnal context to take a proper a decision.

Merging as it is. Thanks @entn-at, this will be part of next release.

[entn-at]

Thanks @hbredin!