Vulkan Star Wars Force Unleashed Crash (Adreno GPUs)

[Blacob777]

After few minutes game crashes, opengl works fine

[hrydgard]

My guess would be an out-of-memory issue, which probably indicates a bug. I'm gonna add more logging so we can try to figure this out.

[hrydgard]

@Blacob777 Can you grab the latest build and try again, and post the new log? It should contain an important piece of information.

[Blacob777]

[Blacob777]

Build 846 crash before cutscene

[hrydgard]

I need the few lines JUST before that....

[Blacob777]

this is the full report, no more nothing

[hrydgard]

Please try again, when the next build shows up (the one with 2e9d59a ). It will contain the info this time :)

[hrydgard]

@Blacob777 new build is up.

[Blacob777]

Amm... 848 again that)

[hrydgard]

Well, at least we got one new piece of information: vkQueueSubmit returns -3, which is VK_ERROR_INITIALIZATION_FAILED, which is interesting. At least we didn't crash the driver or anything like that, but we clearly made it unhappy somehow. Googling did not find anything obvious...

Here are some more things to try and see if they make a difference:

* Lower rendering resolution to 1x
* Turn off "Simulate block transfer effects"

Also can you clarify exactly when this happens? Is it always at the first cutscene, right when starting the game or another cutscene later?

I'm gonna add a new debug option too that may help narrowing it down, stay tuned.

[hrydgard]

I've now added GfxDebugSplitSubmit under [Graphics], so open your PSP/SYSTEM/ppsspp.ini and set it to true under Graphics, like this:

[Graphics]
GfxDebugSplitSubmit = True

Then try again and post the crash log :)

[Blacob777]

it's not helping, got to the next cutscene on OpenGL it is played. on vulcan crash
crash just before cutscenes

[hrydgard]

It's not supposed to help, I just want the new crash log because if you enable the setting I mentioned, there will be more information. The new build isn't up yet though, please wait.

Again, please answer, which cutscene? The first before starting the game or later? Or any?

[Blacob777]

ok sry it's hard for me to understand my perfect English=)
all cutscene crashes with vulkan
im waiting new build and post new log

[blackman91]

Can confirm it crashes on Android 7 with latest ppsspp latest build, starting story mode and pressing circle to skip cutscene crashes the game also it crashes even if you wait, tried sending error report but it also crashes when you press that in the menu.

[hrydgard]

New builds are up. Can one of you guys please try what I suggested above with GfxDebugSplitSubmit , and report the resulting crash log?

[hrydgard]

And one more annoying question: Do you guys have the American or International version of the S7? It matters because of the different GPUs.

[Blacob777]

My Phone xiaomi mi5s plus, snap 821. Where that GfxDebugSplitSubmit? Build 859

[blackman91]

My phone is motog5 plus, same as blacob I don't have the option you said in my .ini.

[hrydgard]

It's a hidden option for debugging so it won't be added automatically, just add it yourself somewhere under [Graphics].

[Blacob777]

Add this option. You see changes?

[hrydgard]

Yes, and it contains a surprise! See the "Abort message:", that's the message I can send out. And it says that the error happens with the main command buffer. Every frame we send two command buffers to the GPU (if they contain anything), the "init" command buffer containing texture uploads and that kind of thing, and the main command buffer containing all the drawing. Turns out that the error happens when the GPU driver is processing the main one, and not the init one as I had expected.

Next thing we'll try is to "accept" the error and try to move on, and see what happens, if something goes missing or if it crashes even worse... I'll make a commit soon.

[Blacob777]

now app dont crash, now just freeze and im hear sound from cutscene

[hrydgard]

Okay, so ignoring the error doesn't work. A crash is better than freezing graphics, so I'm reverting that.

[hrydgard]

@Blacob777 What's that app you use to display the crash dump? Something MIUI specific? I'd like to recommend users to download it as it seems easier than using ADB..

[Blacob777]

@hrydgard MIUI bro

[blackman91]

So this game only crashes on adreno gpus?

[blackman91]

Still crashing in latest dev with motog5 plus, here is a log:

Default settings.

[hrydgard]

"Resource deadlock would occur" is a common result of various errors in Adreno drivers it seems, from some googling. I believe this would be a driver bug that's likely fixed by later drivers. I think you're gonna have to stick with OpenGL for this game until your device gets an update, if ever..

[blackman91]

That's sad to hear, the game runs very slow on gl on my device. I also tested the usa iso and the same crash happens with vulkan.

[weihuoya]

There are 2 frames vulkan steps will cause adreno error, PC just black screen frames.
skip these steps, game is playable for adreno.
remove either of the Clear command in two steps, adreno not crash.

=======================================
LogSteps begin size: 2
    RenderPass Begin(63dfa600), commands: 6
        BindPipeline(6551a200)
        Scissor(0, 0, 1920, 1088)
        Viewport(0.000000, 0.000000, 1920.000000, 1088.000000, 0.375000, 0.625004)
        Stencil(ref=0, compare=255, write=255)
        DrawIndexed(96)
        Clear
    RenderPass End(63dfa600)
    RenderPass Begin(0), commands: 9
        Scissor(0, 0, 1934, 1088)
        Viewport(0.000000, 0.000000, 1934.000000, 1088.000000, 0.000000, 1.000000)
        BindPipeline(6a835a00)
        PushConstants(36)
        Draw(4)
        Scissor(0, 0, 1934, 1088)
        Viewport(0.000000, 0.000000, 1934.000000, 1088.000000, 0.000000, 1.000000)
        BindPipeline(6f1e0400)
        Draw(288)
    RenderPass End(0)
LogSteps end


=======================================
LogSteps begin size: 2
    RenderPass Begin(6673b900), commands: 6
        BindPipeline(6551a200)
        Scissor(0, 0, 1920, 1088)
        Viewport(0.000000, 0.000000, 1920.000000, 1088.000000, 0.375000, 0.625004)
        Stencil(ref=0, compare=255, write=255)
        DrawIndexed(96)
        Clear
    RenderPass End(6673b900)
    RenderPass Begin(0), commands: 9
        Scissor(0, 0, 1934, 1088)
        Viewport(0.000000, 0.000000, 1934.000000, 1088.000000, 0.000000, 1.000000)
        BindPipeline(6a835a00)
        PushConstants(36)
        Draw(4)
        Scissor(0, 0, 1934, 1088)
        Viewport(0.000000, 0.000000, 1934.000000, 1088.000000, 0.000000, 1.000000)
        BindPipeline(6f1e0400)
        Draw(288)
    RenderPass End(0)
LogSteps end

this game use a lot of drawcall, and opengl backend is more efficient than vulkan on my phone(snapdragon 820), almost full speed 60 frames.

[blackman91]

Nice, so that means this issue can be fixed now?

[blackman91]

Just saw the edit, good job man, is this going to be merged guys?

[unknownbrackets]

That's great debugging - very nice.

Let's log more about the clear - there's probably something wrong with it? Or maybe it's mad that it's the last draw. I'm thinking:

ILOG("  Clear(%d, %08x, %f, %02x)", cmd.clear.clearMask, cmd.clear.clearColor, cmd.clear.clearZ, cmd.clear.clearStencil);

Maybe the float value is invalid somehow, like outside range.

-[Unknown]

[weihuoya]

   Clear(7, 00000000, 0.625000, 00)
   Clear(7, 00000000, 0.625000, 00)

[unknownbrackets]

Oh, so it's literally clearing everything right after the draw. That's a pretty unusual thing as far as Vulkan goes, so maybe the driver crashes due to a bug.

Perhaps if a render pass has a clear with 7 in it, we should discard all previous actions in that render pass? I'm thinking this:

void VulkanRenderManager::Clear(uint32_t clearColor, float clearZ, int clearStencil, int clearMask) {
	_dbg_assert_(G3D, curRenderStep_ && curRenderStep_->stepType == VKRStepType::RENDER);
	if (!clearMask)
		return;
	// If this is the first drawing command or clears everything, merge it into the pass.
	int allAspects = VK_IMAGE_ASPECT_COLOR_BIT | VK_IMAGE_ASPECT_DEPTH_BIT | VK_IMAGE_ASPECT_STENCIL_BIT;
	if (curRenderStep_->render.numDraws == 0 || clearMask == allAspects) {
		curRenderStep_->render.clearColor = clearColor;
		curRenderStep_->render.clearDepth = clearZ;
		curRenderStep_->render.clearStencil = clearStencil;
		curRenderStep_->render.color = (clearMask & VK_IMAGE_ASPECT_COLOR_BIT) ? VKRRenderPassAction::CLEAR : VKRRenderPassAction::KEEP;
		curRenderStep_->render.depth = (clearMask & VK_IMAGE_ASPECT_DEPTH_BIT) ? VKRRenderPassAction::CLEAR : VKRRenderPassAction::KEEP;
		curRenderStep_->render.stencil = (clearMask & VK_IMAGE_ASPECT_STENCIL_BIT) ? VKRRenderPassAction::CLEAR : VKRRenderPassAction::KEEP;

		// In case there were commands already.
		curRenderStep_->render.numDraws = 0;
		curRenderStep_->commands.clear();
	} else {
		VkRenderData data{ VKRRenderCommand::CLEAR };
		data.clear.clearColor = clearColor;
		data.clear.clearZ = clearZ;
		data.clear.clearStencil = clearStencil;
		data.clear.clearMask = clearMask;
		curRenderStep_->commands.push_back(data);
	}
}

-[Unknown]

[weihuoya]

nice, fx crash

[hrydgard]

Thanks weihouya for nice debugging and unknown for spotting that suspicious clear :) I was focusing on the readbacks...