Black Rock Shooter (NPUH10126) randomly hangs at the Now Loading screen after a certain point during a certain story mission in all PPSSPP builds since v0.7.6-1157-g7d566b9 (updated, responsible commit found)

[solarmystic]

(Update 2:- The single commit responsible for this issue is oioitff@5fb92f6 from #2107 (Bug fix for _AtracSetData) that was merged into master sometime between 0.7.6-1140 and 0.76-1157) #2608 (comment)

Issue is as described in the title.

Firstly, the issue was hard to repro due to the random nature of the hang (some playthroughs it would hang, some it wouldn't). Once you can get it to hang, however, subsequent hangs in the same ppsspp instance are easier(?) to reproduce.

Secondly, I will state that I couldn't get a complete log out of this issue due to the fact that the game will never hang when ppsspp is in logging mode (via double clicking the Log.bat icon) (Update:- Due to the random nature of it, I managed to grab one anyway. it was just a lot harder to repro the hang with Logging mode on)

After much repeated testing, bisecting and elimination, it was discovered that the first build to consistently (can I even use this word in this context?) produce these random hang ups is v0.7.6-1157-g7d566b9

https://github.com/hrydgard/ppsspp/commit/v0.7.6-1157-g7d566b9

The last, definitive build that did not hang at all at the specified point is ppsspp-v0.7.6-1140-gbe483a1

There are many unaccounted for builds in between those two that couldn't be tested due to the lack of a downloadable build on the Orphis build bot. (Update: responsible commit found, see above)

The story mission is "Release the Golden Gate Gate" (sic,copied verbatim from the in-game description) from Stage 1-5.

The game will hang with the BGM continually playing in the background (only if you have the at3+ dll and at3+ audio enabled in the options menu) after you've managed to hook up the 4 power connectors that were broken throughout the map, ,returned back to your companions at the gate and unlocked the gate.

Savefile for problem reproduction:-
http://www.mediafire.com/?h8z39gzj5tb72sk

Use default settings for problem reproduction.

Problem reproduction (for those without a savefile at that bit, and/or don't want to kill the mobs again.)

1. Load up the game using any PPSSPP build since v0.7.6-1157-g7d566b9
2. Load up the 2nd slot of my provided savefile
3. Head back to the gate (from the save point, head north)
4. Press X at the gate
5. Let the cutscenes play out, which will be followed by the hang
6. If you fail to repro it the first time, just stop the game, and restart the game again in the same ppsspp instance. It takes an average of 3 attempts to reproduce the hang on an affected build.

Conditions:-

1. Can happen in both 32 and 64 bit builds.
2. Hang ups can occur with OR without at3+ audio enabled and the dll located in the directory.
3. Has a higher chance of NOT happening when PPSSPP is in logging mode. Grabbing a complete log when it hangs is next to impossible. (Wrong, it was a random chance that it didn't happen).
4. Happens at random, which means it has a chance of not occuring on a first boot up of the game.

Log Console segment when it hangs:-

Logfile from an affected build when it hangs:-

https://gist.github.com/solarmystic/5922609

https://gist.github.com/solarmystic/5923477

Credit to ShedoSurashu from the ppsspp.org forums for bringing the matter to my attention in the first place.

[solarmystic]

Logfile from an affected debug build (started up the game, and played until hang, then stopped the game):-

http://www.mediafire.com/download/qj9vzu6k87i3p3e/ppsspp_(black_rock_shooter).7z

(Seems to point to context switching when it hangs)

[solarmystic]

After even more bisecting and learning how to use git, Visual studio C++ express, merges, git checkouts, submodule updates and hard resets thanks to @thedax, I have finally found the individual commit responsible for this random hang.

This is it. This is the responsible individual commit between 0.7.6-1140 and 0.7.6-1157 that's causing the random hang as described in the main issue title.

The #2107 oioitff@5fb92f6 pull request/commit that got merged into the master is the one.

All the other individual commits that I made individual builds for to test the issue seperately checked out fine.

Now I can go to sleep in peace.

EDIT:- If anyone's interested in reproducing the issue quickly, within a few tries, use this build that I compiled to include the offending commit. Thanks again to @thedax for teaching me the basics of github, Visual Studio C++ and how to do hard resets, and submodule updates used for making older unlisted builds to use for debugging.

http://www.mediafire.com/?n58ljdm4kiau1ih

Remember, if you don't succeed the first time, try it a couple more times, to elicit the hang. Only takes 3 quick attempts using the savefile I provided in the main issue post.

[unknownbrackets]

The trouble with scheduling issues is it really can be anything. That commit could be totally right, and just have exposed an issue where another function doesn't reschedule. I'm also sure there are other scheduling issues.

-[Unknown]

[solarmystic]

@unknownbrackets

I thought that one of your six commits in your #2106 mega pull request was responsible at first. I comipiled six individual builds in Visual Studio C++ to represent each suspected commit, each separately compiled with the git reset hard function and a quick submodule update in between each build in Visual Studio C++ and then tested them each 10 times to reproduce the hang.

I was wrong, they were all clear. (10 attempts and all of them cleared the test, with no producible hang ups)

I then proceeded to check out the other potentially responsible commits in that timeframe, (between 0.7.6-1140 and 0.7.6-1157), and came up with two more from oioitff, thanks to @thedax teaching me how to do a git compare between commit ids.

I compiled two more builds to test out those commits individually and it turned out that one of them ended up being the culprit. The one I mentioned earlier. oioitff@5fb92f6 #2107

[unknownbrackets]

Even so, the commit you pointed to is still a scheduling related commit. It was about the hleDelayResult() iirc.

Also, I definitely recommend reading about git bisect. That command is awesome. It can save a lot of time when searching for a commit that broke something, if you have the game. Even if there's a range of 1000 commits, you should be able to find the culprit in 10 tries by bisecting, and git bisect automates the whole efficient process for you.

-[Unknown]

[oioitff]

Hmm, I guess the issue should be something about:

sceAtracSetMOutHalfwayBufferAndGetID(00000001, 092a6c50, 00005be4)

I'm sure it's wrong compare to:

int sceAtracSetMOutHalfwayBufferAndGetID(u32 halfBuffer, u32 readSize, u32 halfBufferSize)

@solarmystic How about the logs for those working build? Does the Mout voice work fine in the working build?

[hrydgard]

Yup, those parameters must be wrong.

[solarmystic]

@oioitff

You mean the voices of the characters? Yes they do, they work okay in the working build. (EDIT:- I misunderstood your comment about the MOut thingy, but I think they do work fine in 1140)

The only thing faulty about those builds (11xx+) was that they had scratchy audio for the BGM (which you fixed later on in the 12xx builds)

Anyway, here's a logfile for the last working debug build (0.7.6-1140) covering the exact same location:-

(Loaded the game, loaded the savefile and headed to the gate, went through the ingame cutscene with the characters, then stopped and exit)

(Debug build logfile was too big to be put on gist or pastebin, so had to be zipped and uploaded to MF)

http://www.mediafire.com/?pz9n9bkhf1xbo7r

Here's another logfile for the last working release build (0.7.6-1140) covering the exact same location and procedure as above:-

https://gist.github.com/solarmystic/5956772

[oioitff]

Yep, it should be a a scheduling related issue.
And it seems that issue was caused by my commits #2102, which aimed to do schedule in some atrac *GetID functions, and that should be right. So maybe we are just missing scheduling in sceAtracDecodeData.

[solarmystic]

Issue has been resolved in v0.8.1-511-gd5719f0 thanks to @oioitff

Case closed.