LogoPlanner实时部署的耗时问题

[xxqs13]

服务端是部署在GeForce RTX 4090 ，然后用3090的客户端，相机是奥比相机，图片和深度图的分辨率输入也是640，480。
我的一次推理耗时有1.6s,

[PERF] Request parsing: 8.2 ms
[PERF] Image processing: 14.0 ms
[PERF] Depth dilation: 302.7 ms
[PERF] Model inference: 656.9 ms
[PERF] MPC setup: 24.3 ms
[PERF] MPC solve: 137.7 ms (20 steps)
[PERF] Trajectory plot: 439.1 ms
[PERF] Post-processing: 0.0 ms

[PERF] TOTAL: 1582.9 ms (0.63 Hz)

，就算不用Trajectory plot: 439.1 ms，也顶多是1hz,达不到作者说的。在配备 GeForce RTX 4090 显卡的本地机器上，我们的实时避障推理速度可达约 10Hz。
有可能是因为Warning, cannot find cuda-compiled version of RoPE2D,using a slow pytorch version instead 这个原因导致我模型推理得比较慢吗？
其他的参数设置跟官方案例保持一致

[steinate]

1. "在配备 GeForce RTX 4090 显卡的本地机器上，我们的实时避障推理速度可达约 10Hz"，这句话是针对NavDP说的，并不是LoGoPlanner的，LoGoPlanner我们自己本地4090机器推理时长在200ms左右，加上其他的耗时在300ms，实时性这个问题我们还在改进中，你也可以先尝试NavDP + 里程计的组合。
2. 这个warning是pi3的warning，是正常的
3. 个人经验上看你提供的这个耗时还是有些问题的，不光是网络推理时延，其他模块也比我们的耗时长很多，有可能CPU的性能不太行，MPC solve我们一般在20ms左右就可以搞定。你可以先运行下python policy_network.py单元测试一下耗时。以及测试下NavDP的耗时python policy_network.py，看看NavDP的耗时能不能跑在100ms以内

[xxqs13]

感谢您的回复！我整理了几组耗时数据供参考：
这是我在服务器正常运行时的第一次的推理耗时：

[PERF] Request parsing: 5.8 ms
[PERF] Image processing: 17.4 ms
[PERF] Depth dilation: 312.5 ms
[PERF] Model inference: 1588.5 ms
[PERF] MPC setup: 320.4 ms
[PERF] MPC solve: 195.1 ms (20 steps)
[PERF] Trajectory plot: 493.3 ms
[PERF] Post-processing: 0.0 ms

[PERF] TOTAL: 2933.0 ms (0.34 Hz)

，然后下面的是我单纯跑logoplanner的policy_network.py 的耗时：
with torch.cuda.amp.autocast(dtype=dtype):
[MODEL] Goal encoding: 104.6 ms
[MODEL] RGBD encoding: 552.7 ms
[MODEL] State encoding: 886.0 ms
[MODEL] State decoding: 1.7 ms
[MODEL] Diffusion denoising: 222.2 ms
[MODEL] Critic & selection: 286.5 ms
[MODEL] TOTAL: 2053.7 ms
All Trajectory Shape: (1, 16, 24, 3)
Critic Values Shape: (1, 16)
Positive Trajectory Shape: (1, 2, 24, 3)
Negative Trajectory Shape: (1, 2, 24, 3)

下面的是我服务器端4090跑NavDP的policy_network.py的耗时：
[1/3] Creating NavDP_Policy instance...
✓ Policy network created and moved to CUDA

[2/3] Creating test data...
Input images shape: torch.Size([1, 8, 224, 224, 3])
Input depths shape: torch.Size([1, 224, 224, 1])
Goal point: [[ 5. -2. 0.]]
✓ Test data created

[3/3] Testing predict_pointgoal_action with timing...
[MODEL] Encoding: 316.9 ms
[MODEL] Preparation: 1.1 ms
[MODEL] Diffusion denoising: 143.0 ms
[MODEL] Critic & selection: 99.9 ms
[MODEL] TOTAL: 560.9 ms

All Trajectory Shape: (1, 16, 24, 3)
Critic Values Shape: (1, 16)
Positive Trajectory Shape: (1, 2, 24, 3)
Negative Trajectory Shape: (1, 2, 24, 3)
Critic Values Range: [0.258, 0.701]

但是我在笔记本NVIDIA GeForce RTX 4060，上面说的3090是我记错了。

[1/3] Creating NavDP_Policy instance...
✓ Policy network created and moved to CUDA

[2/3] Creating test data...
Input images shape: torch.Size([1, 8, 224, 224, 3])
Input depths shape: torch.Size([1, 224, 224, 1])
Goal point: [[ 5. -2. 0.]]
✓ Test data created

[3/3] Testing predict_pointgoal_action with timing...
[MODEL] Encoding: 125.1 ms
[MODEL] Preparation: 0.5 ms
[MODEL] Diffusion denoising: 86.1 ms
[MODEL] Critic & selection: 44.3 ms
[MODEL] TOTAL: 256.0 ms

All Trajectory Shape: (1, 16, 24, 3)
Critic Values Shape: (1, 16)
Positive Trajectory Shape: (1, 2, 24, 3)
Negative Trajectory Shape: (1, 2, 24, 3)
Critic Values Range: [0.180, 0.411]

⏱️ Total inference time: 256.2 ms (3.90 Hz)
✓ Point goal prediction successful
因为我的笔记本的显存只有8Gb,所以测试不了logoplanner的模型推理的耗时，

[xxqs13]

补充在笔记本4060跑logoplanner的耗时：
with torch.cuda.amp.autocast(dtype=dtype):
[MODEL] Goal encoding: 15.7 ms
[MODEL] RGBD encoding: 173.1 ms
[MODEL] State encoding: 1338.4 ms
[MODEL] State decoding: 4.1 ms
[MODEL] Diffusion denoising: 58.4 ms
[MODEL] Critic & selection: 37.7 ms
[MODEL] TOTAL: 1627.2 ms
All Trajectory Shape: (1, 16, 24, 3)
Critic Values Shape: (1, 16)
Positive Trajectory Shape: (1, 2, 24, 3)
Negative Trajectory Shape: (1, 2, 24, 3)
应该是我的服务器的cpu性能比较差，导致某些模块耗时比较高，