Correctly fetch correct pitch and depth for textures from RSX

[kd-11]

Texture pitch and depth are fetched from the registers without indexing, causing all texture layers to contain info for texture layer 0. Thanks to @jarveson for pointing out the problem.

Should fix issue #1495

[mention-bot]

By analyzing the blame information on this pull request, we identified @DHrpcs3, @vlj and @Nekotekina to be potential reviewers

[kd-11]

I'm not sure how the pitch per index should be fetched. The other values are set in the init() function but not for depth and pitch. Someone more familiar with the core RSX should confirm.

[kd-11]

Updated to also affect vertex textures.
Fixes alot of texture alignment issues on openGL backend.

[vlj]

Nice catch.

I think gcm.cpp code correctly parses control 2 and 3, I had an issue where some texture related opcode had a stride of 8 and some other a stride of 1.

Dx12 doesn't use texture pitch from the register and computes it from texture width. It'd likely not accurate but I didn't know how to interpret the pitch value from the register when I wrote the code: sometimes it's 0 and sometimes the pitch value is wrong.
Actually the pitch value is ignored if texture is swizzled. Textures and their mipmaps level are tightly packed. If texture is linear stored then pitch is used for textures and mipmap levels. Since mipmap width is halved at every level it takes more storage space with unused memory gap (which is sometimes used to store other texture like in after burner climax)
Depth is almost always 1. As far as I know rsx doesn't support texture array so depth is needed only in 3d texture case which are rare. The only game that uses a 3d texture so far is Resogun in the first level.
That's likely why it wasn't spotted before.

[kd-11]

@jarveson Indeed; there is a rather large gap between 583-640 (base 10) so I agree with your analysis about the indexing having a stride of 8. Will update accordingly.

[kd-11]

Updated vertex texture pitch and depth index stride.

[vlj]

Is travis-ci stuck ?

[tambry]

Travis seems to be triggered now. Sadly there's no way to redeliver Travis payloads.

[kd-11]

Sorry. Bad push. @vlj Requires approval again.

[danilaml]

@tambry where should be rebuild/cancel build option for those in the team (same as appveyor).

[tambry]

@danilaml There is, but I can't do that, when Travis didn't receive the build request. Ie. Github was waiting for Travis build to start for hours.