SDL_yuv_mmx.c cause segmentation fault.

[SDLBugzilla]

This bug report was migrated from our old Bugzilla tracker.

Reported in version: 1.2.15
Reported for operating system, platform: Linux, x86

Comments on the original bug report:

On 2012-04-07 02:27:52 +0000, Anton Yarth wrote:

mmx code of SDL_yuv_mmx.c cause segmentation fault with smpeg:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb55abb70 (LWP 3796)]
0x00162071 in ColorRGBDitherYV12MMX1X (colortab=0x163f8c, rgb_2_pix=0x8725510,
lum=0x8726518 "\377a", cr=0x870a708 "", cb=0x8717e08 "", out=0x871b3c8 "",
rows=141379880, cols=172, mod=320) at ./src/video/SDL_yuv_mmx.c:94
94 asm volatile (

patch for "correction" by turning off optimization in SDL_yuv_sw.c http://pastebin.com/AsGmpwwN

On 2012-04-15 00:27:44 +0000, Anton Yarth wrote:

Same error on windows build.

On 2012-08-04 02:04:56 +0000, Andrew Church wrote:

The inline assembly in the routine appears to be fundamentally broken because it manipulates the stack and also tries to take arguments from the stack:

0xb7251b09 <+73>: push $0x0
0xb7251b0b <+75>: push %ebx
0xb7251b0c <+76>: mov 0x2c(%esp),%ebx
0xb7251b10 <+80>: mov %ebx,0x4(%esp)
...
0xb7251c40 <+384>: add $0x2,%esi
0xb7251c43 <+387>: addl $0x4,0xc(%esp)
0xb7251c48 <+392>: cmp %eax,0xc(%esp)
0xb7251c4c <+396>: jl 0xb7251b18 <ColorRGBDitherYV12MMX1X+88>
(and more instances)

I suspect this only worked in the past because previous versions of GCC may have used %ebp instead of %esp to reference function arguments.

On 2012-08-29 12:38:35 +0000, Steffen Pankratz wrote:

I see a similar problem with the exact same use case.
Disabling the optimization fixes the problem for me too.

Here is what I get:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xf5432b40 (LWP 25948)]
0xf7f0a724 in Color565DitherYV12MMX1X (colortab=0xf7f0c485, rgb_2_pix=0x85b03c0, lum=0x85b13c8 "", cr=0xf5e69008 '\020' <repeats 200 times>...,
cb=0xf5eb4008 "\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200"...,
out=0xf5ec6c08 "\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200\200"..., rows=-190410744, cols=480, mod=640) at src/video/SDL_yuv_mmx.c:255
255 asm volatile(

On 2013-03-10 21:54:34 +0000, Andreas Schiffler wrote:

Tried to contact the original author of the MMX code (via email in the sources) without success.
mvogt@rhrk.uni-kl.de: host mx2.uni-kl.de[131.246.120.219] said: 550 # 5.1.0
Address rejected. (in reply to RCPT TO command)

Not sure if it is worth to maintain this MMX code and recommend to remove/disable this code from the SDL2.0 codebase for now.

On 2015-08-25 09:38:20 +0000, Ryan C. Gordon wrote:

Hello, and sorry if you're getting several copies of this message by email, since we are closing many bugs at once here.

We have decided to mark all SDL 1.2-related bugs as RESOLVED ENDOFLIFE, as we don't intend to work on SDL 1.2 any further, but didn't want to mark a large quantity of bugs as RESOLVED WONTFIX, to clearly show what was left unattended to and make it easily searchable.

Our current focus is on SDL 2.0.

If you are still having problems with an ENDOFLIFE bug, your absolute best option is to move your program to SDL2, as it will likely fix the problem by default, and give you access to modern platforms and tons of super-cool new features.

Failing that, we will accept small patches to fix these issues, and put them in revision control, although we do not intend to do any further official 1.2 releases.

Failing that, please feel free to contact me directly by email (icculus@icculus.org) and we'll try to find some way to help you out of your situation.

Thank you,
--ryan.