Dtype for Speechbrain

[Fhrozen]

Fix #5099

Only fixes the xvector_type=speechbrain.
Did not check for kaldi.

[codecov]

Codecov Report

Merging #5112 (6af7a7e) into master (ce13ba7) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #5112   +/-   ##
=======================================
  Coverage   75.06%   75.06%           
=======================================
  Files         615      615           
  Lines       54858    54858           
=======================================
  Hits        41177    41177           
  Misses      13681    13681           

Flag	Coverage Δ
test_integration_espnet1	66.29% <ø> (ø)
test_integration_espnet2	47.77% <ø> (ø)
test_python	65.36% <ø> (ø)
test_utils	23.28% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[sw005320]

Thanks, @Fhrozen!