use nsight system inside containers

[Fizzbb]

Since all the workloads are deployed through container, we need to launch nsight system within the container.
procedure is like

1. in the Dockerfile, add command to install cuda toolkit, nsight system and compute will be installed in the /usr/local/cuda-11.4 folder.
   RUN apt-get install -y cuda-toolkit-11-4
2. mount a host path to the application container for passing nsight report file. when use kubectl cp, sometimes cause EOF error and corrupted file cannot be open by nsight
3. Run the profiling
   nsys profile -o report --stats=true --force-overwrite true python XXX.py
   If nsys is not found, you can use the complete path /usr/local/cuda-11.4/bin/nsys
4. Copy the report to the mount folder
5. SCP download the report.qdrep file to local machine with nsight GUI installed, and check out the timeline

Limitation is it cannot profile cross containers.

[pint1022]

hope it works on top of multiple containers.