You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
depends on the other estimates \hat s_{ik}(n) for k \ne j. Here is a quick example that shows the effect:
importmusdbimportmusevalimportnumpyasnpdefestimate_and_evaluate1(track):
""" Simple baseline system using mixture as estimate """estimates= {}
estimates['vocals'] =0.25*track.audioestimates['accompaniment'] =0.75*track.audioscores=museval.eval_mus_track(track, estimates, output_dir='.')
print('Score for `estimate_and_evaluate1`:')
print(scores)
returnestimatesdefestimate_and_evaluate2(track):#""" Modified baseline system, which sets the second half of `vocals` to zero """estimates= {}
estimates['vocals'] =0.25*track.audioestimates['accompaniment'] =0.75*track.audioestimates['vocals'] *=np.vstack((np.ones((track.audio.shape[0] //2, 2)),
np.zeros((track.audio.shape[0] -track.audio.shape[0] //2, 2))))
scores=museval.eval_mus_track(track, estimates, output_dir='.')
print('Score for `estimate_and_evaluate2`:')
print(scores)
returnestimatesdefestimate_and_evaluate3(track):#""" Modified baseline system, which sets the first half of `vocals` to zero """estimates= {}
estimates['vocals'] =0.25*track.audioestimates['accompaniment'] =0.75*track.audioestimates['vocals'] *=np.vstack((np.zeros((track.audio.shape[0] //2, 2)),
np.ones((track.audio.shape[0] -track.audio.shape[0] //2, 2))))
scores=museval.eval_mus_track(track, estimates, output_dir='.')
print('Score for `estimate_and_evaluate3`:')
print(scores)
returnestimatesmus=musdb.DB(root_dir='/speech/db/mul/separ4/sisec/data2018/', is_wav=True)
mus.run(estimate_and_evaluate1, estimates_dir=".", tracks=[mus.load_mus_tracks(subsets='test')[0]])
mus.run(estimate_and_evaluate2, estimates_dir=".", tracks=[mus.load_mus_tracks(subsets='test')[0]])
mus.run(estimate_and_evaluate3, estimates_dir=".", tracks=[mus.load_mus_tracks(subsets='test')[0]])
estimate_and_evaluate* are three simple systems that uses the mixture as estimate. Only vocals is modified for the different versions but also the BSSEval values for accompaniment are changed:
Currently, segments where one/several estimates are all-zero are not considered for the BSSEval computation
https://github.com/sigsep/sigsep-mus-eval/blob/05d52e4962660417801b78aa82ac598dd8c7b25a/museval/metrics.py#L300
This leads to the effect that the SDR value, which is defined for the
j
th instrument asSDR_j = 20\log_10 ( \sum_i,n s_{ij}(n)^2 ) / ( \sum_i,n (s_{ij}(n) - \hat s_{ij}(n))^2 )
depends on the other estimates
\hat s_{ik}(n)
fork \ne j
. Here is a quick example that shows the effect:estimate_and_evaluate*
are three simple systems that uses the mixture as estimate. Onlyvocals
is modified for the different versions but also the BSSEval values foraccompaniment
are changed:@faroit @aliutkus What do you think? Should this be changed for a future version of BSSEval?
The text was updated successfully, but these errors were encountered: