nvidia-mgpu simulator is significantly slower than nvidia-fp64 for exp_pauli operations

[bmhowe23]

Required prerequisites

• Consult the security policy. If reporting a security vulnerability, do not report the bug using this form. Use the process described in the policy to report the issue.
• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker to verify that this hasn't already been reported. +1 or comment there if it has.
• If possible, make a PR with a failing test to give us a starting point to work on!

Describe the bug

Simulation of quantum programs using exp_pauli are currently more performant on the nvidia and nvidia-fp64 targets than on the nvidia-mgpu backend, even when using a single GPU. There is no good reason for this performance shortfall.

Steps to reproduce the bug

Run the test program here.

Results on my machine are:

Avg Time 10 qubits (nvidia-mgpu) = 1.036675715446472 [sec]
Avg Time 10 qubits (nvidia-fp64) = 0.07855632305145263 [sec]

Expected behavior

The simulation times should be similar or better when using nvidia-mgpu.

Is this a regression? If it is, put the last known working version (or commit) here.

Not a regression

Environment

• CUDA Quantum version: latest
• Python version:
• C++ compiler:
• Operating system:

Suggestions

We likely need to override applyExpPauli in the nvidia-mgpu simulator.