Unable to achieve the performance in experimental/whisper/speedup.ipynb

[S-GH]

First of all, thank you for a great project.

I have been trying to run the Jupyter notebook experimental/whisper/speedup.ipynb
However, I am unable to achieve the performance improvements that are described in the notebook.

My Result (whisper-base) :
Kernl speedup: 0.5X (0.4 VS 0.2 min) (i tested 100 samples)
different outputs: [1/100] (1.00%)

I also saw that experimenting with whisper large-v2 takes about 3+ hours to warm up, with no speedup. (1.0X)

nvidia-smi gives me :

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.78.01    Driver Version: 525.78.01    CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  On   | 00000000:21:00.0 Off |                  N/A |
|  0%   40C    P0    96W / 350W |     17MiB / 24576MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce ...  On   | 00000000:49:00.0 Off |                  N/A |
|  0%   36C    P8    39W / 350W |      6MiB / 24576MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

I experimented by opening the container with the provided docker file in the 3090GPU environment.

If you have any idea why this is happening, please advise. Thank you.