Use cublasHgemm "back" for fp16 computation with Volta GPU

[pengwa]

Description: Use cublasHgemm for fp16 computation with Volta GPU

cublasHgemm is used for fp16 computation with Volta GPU before training code is merged into master. For historical reasons when we did master->old training branch internally merge, we commented out that path. So I used "back" in PR title to indicate this change is just re-enable existing path.

This change should do no harm for inference because it is the case already in master before training is merged.
This change bring perf improvement on training, tested on 32GV100.

The reasons exists here https://docs.nvidia.com/cuda/cublas/index.html#cublassetmathmode.
cublasGemmEx's computation type is CUDA_R_32F, though its main data inputs.outputs are CUDA_R_16F. cublasHgemm did CUDA_R_16F computation.

Motivation and Context

• Why is this change required? What problem does it solve?
• If it fixes an open issue, please link to the issue here.

[pengwa]

@SherlockNoMad feel free to take over this PR if we want it asap in latest benchmarking.

[weixingzhang]

Can you verify convergence? previously, the accumulation is done in FP32. But with this change, the accumulation will be done in FP16. For training, probably it is not a good idea to use hgemm, even it is OK for bert-l, but may not be OK for other big models.