[❔ other question] segfault on shutdown with pathreparam-optix7 branch ?

[vtuizat]

Summary

I am getting segfaults on shutdown when launching inverse rendering examples (invert_pose.py and invert_heightfield.py). I can get results from the optimization, I am using the reparam-optix7 branch.
Has anyone encountered the same problem ?

System configuration

• Platform: ubuntu 18.04
• Compiler: clang-9
• Python version: 3.6.9
• Mitsuba 2 version: reparam-optix7 branch
• Compiled variants:
  • scalar_rgb
  • gpu_rgb
  • gpu_autodiff_rgb
• Nvidia driver version : 465.27
• Optix 7.0
• GPU : tried on GTX 1070TI and RTX A6000

Description

I am experimenting with mitsuba in docker. The latest release version works fine with the container.

The reparam-optix7 branch seems to compile well with

mkdir mitsuba2/build && cd mitsuba2/build &&\
    cmake -GNinja -DMTS_ENABLE_OPTIX=ON .. && \
    ninja

I can launch both cpu and gpu rendering without issues.

I get this error when I launch invert_pose.py.

root@cb25d13e2bf8:/install/mitsuba2/docs/examples/10_inverse_rendering# python3 invert_pose.py  
2021-07-08 12:13:43 INFO  main  [optix_api.cpp:56] Dynamic loading of the Optix library ..  
2021-07-08 12:13:43 INFO  main  [xml.cpp:1182] Loading XML file "../../../resources/data/docs/examples/invert_pose/scene.xml"  ..
2021-07-08 12:13:43 INFO  main  [xml.cpp:1183] Using variant "gpu_autodiff_rgb" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/uniform.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/srgb.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/pathreparam.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/hdrfilm.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/gaussian.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/independent.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/perspective.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/smootharea.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/rectangle.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/diffuse.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/obj.so" ..
2021-07-08 12:13:43 INFO  main  [PluginManager] Loading plugin "plugins/checkerboard.so" ..
2021-07-08 12:13:43 INFO  main  [Scene] Building scene in OptiX ..
Write output/invert_pose/out_ref.exr
Iteration 099: error=0.00308545
1092.118392 ms per iteration
Segmentation fault (core dumped)

The optimization goes well, the frames get written and then the segfault seems to happen during either cuda_shutdown() or autodiff : shutdown.

I tried to compile mitsuba in debug mode to investigate, the compilation goes well, but I get encountered ray with nan or inf values errors when I run anything on the gpu. Very similar output to this issue : https://github.com/mitsuba-renderer/mitsuba2/issues/397

The error happened on different machines with different gpus.

Steps to reproduce

1. compile Mitsuba2 reparam-optix7 with the above parameters
2. run python3 invert_pose.py
3. you get a segfault at the end of the render