rsx/vk: Fix some spec violations

[kd-11]

Handles some spec violations caught by the vulkan validation layers

• Enable stencil export extension before compiling shaders that utilize this feature . This feature is used to accelerate depth-stencil unresolve on all non-nvidia hw when using MSAA.
• Rewrite memory detection and initialization. Avoids duplicate heap bindings into a memory type descriptor and should avoid unintentional memory spilling and/or crashing.
• Rework attachment clears to avoid touching aspects that do not exist.
• Enable mirror-clamp-to-edge sampler addressing mode extension before using it in actual samplers.
• Fix some extent miscalculation for mipmapped 3D textures.

Partially handles #9495

[legend800]

This broke Motorstorm gfx: #10936