linux loader segfault while replaying gfxrecon traces

[lunarpapillo]

I'm putting this here at Lenny's request...

Trying to run gfxrecon traces on emmeria (Linux AMD), and using the snapshot layer to collect images for comparison purposes, I provoked a cryptic loader segfault:

        Program received signal SIGSEGV, Segmentation fault.
        0x00007ffff62dc104 in loader_validate_instance_extensions ()
           from /home/lunarg/sdks/1.1.101.0/x86_64/lib/libvulkan.so
        (gdb) bt
        #0  0x00007ffff62dc104 in loader_validate_instance_extensions ()
           from /home/lunarg/sdks/1.1.101.0/x86_64/lib/libvulkan.so
        #1  0x00007ffff62dee8d in vkCreateInstance ()
           from /home/lunarg/sdks/1.1.101.0/x86_64/lib/libvulkan.so
        #2  0x00000000004d51c0 in gfxrecon::decode::VulkanReplayConsumerBase::OverrideCreateInstance(VkInstanceCreateInfo const*, VkAllocationCallbacks const*, VkInstance_T**) ()
        #3  0x0000000000539542 in gfxrecon::decode::VulkanReplayConsumer::Process_vkCreateInstance(VkResult, gfxrecon::decode::StructPointerDecoder<gfxrecon::decode::Decoded_VkInstanceCreateInfo> const&, gfxrecon::decode::StructPointerDecoder<gfxrecon::decode::Decoded_VkAllocationCallbacks> const&, gfxrecon::decode::HandlePointerDecoder<VkInstance_T*> const&) ()
        #4  0x00000000004f836a in gfxrecon::decode::VulkanDecoder::Decode_vkCreateInstance(unsigned char const*, unsigned long) ()
        #5  0x00000000004cf8b6 in gfxrecon::decode::FileProcessor::ProcessFunctionCall(gfxrecon::format::BlockHeader const&, gfxrecon::format::ApiCallId) ()
        #6  0x00000000004d059f in gfxrecon::decode::FileProcessor::ProcessBlocks() ()
        #7  0x00000000004cd6a1 in gfxrecon::application::Application::Run() ()
        #8  0x00000000004cbe6a in main ()

No other message was given.

If you turn off the screenshot layer, the reason (or at least part of it) becomes clearer:

        $ unset VK_INSTANCE_LAYERS
        $ /home/lunarg/.jenkins/g0/Release64/gfxreconstruct/build/tools/replay/gfxrecon-replay /home/lunarg/.jenkins/g0/Release64/vulkaninfo/local-copy/vulkaninfo_20190220T162436.gfxr
        [gfxrecon] ERROR - API call (vkCreateInstance) returned value VK_ERROR_EXTENSION_NOT_PRESENT that does not match return value from capture file VK_SUCCESS.
        [gfxrecon] WARNING - API call (vkCreateInstance) returned error result VK_ERROR_EXTENSION_NOT_PRESENT: the specified extension does not exist
        Segmentation fault (core dumped)

So the problem looks like some issue when trying to create an instance using an extension that doesn't exist, while the snapshot layer is active.

Ideally, the loader should return some type of message in this situation.

[KarenGhavam-lunarG]

@lunarpapillo I think this issue should be in the loader project.

[lunarpapillo]

I think the intent was to first determine whether it was a loader issue or an issue with either the snapshot layer or gfxrecon, before moving it to a different project.

Karen suggested I move it to gfxrecon so Dustin can figure the first part out... but I haven't figured out how to move it yet (without re-entering everything)... Karen, I think you have to have higher permissions than I do to move issues around...

[dustin-lunarg]

A few notes on this issue:

1. Replay error handling has been updated such that an API call that succeeded on capture but failed on replay will now cause the tool to print a fatal error and stop playback, instead of printing an error but continuing playback and crashing. This addresses the last case presented in the original issue, running without the screenshot layer, but does not address the loader crash.
2. If you think that the loader crash is the result of invalid parameters in the capture file, you can enable the validation layers to validate parameters and generate messages for invalid cases.
3. If you think the loader crash is the result of a loader bug or screenshot layer bug, the issue needs to be reported to either the Vulkan-Loader project or VulkanTools project.