New crash in codec_unittest OSX/Release build only

[ethanhugg]

This is new today 9/25.

lldb gives this on release build (doesn't happen on debug build):

Process 52803 stopped

• thread doesn't work on windows 8 64bits #1: tid = 0x11ac18, 0x00000001002f2850 codec_unittestWelsI16x16LumaPredH_sse2, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x1002f2850) frame #0: 0x00000001002f2850 codec_unittestWelsI16x16LumaPredH_sse2
  codec_unittest`WelsI16x16LumaPredH_sse2:
  -> 0x1002f2850: pushq %rcx
  0x1002f2851: movslq %edx, %rdx
  0x1002f2854: decq %rsi
  0x1002f2857: movzbq (%rsi), %rcx
  (lldb)

Failed Travis run is here:
https://travis-ci.org/ethanhugg/openh264/builds/36270912

[ethanhugg]

Made a build with BUILDTYPE=Debug and USE_ASM=Yes to get a full stack:

Process 53254 stopped

• thread doesn't work on windows 8 64bits #1: tid = 0x11c2fd, 0x0000000100428c20 codec_unittestWelsI16x16LumaPredH_sse2, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x100428c20) frame #0: 0x0000000100428c20 codec_unittestWelsI16x16LumaPredH_sse2
  codec_unittest`WelsI16x16LumaPredH_sse2:
  -> 0x100428c20: pushq %rcx
  0x100428c21: movslq %edx, %rdx
  0x100428c24: decq %rsi
  0x100428c27: movzbq (%rsi), %rcx
  (lldb) bt
• thread doesn't work on windows 8 64bits #1: tid = 0x11c2fd, 0x0000000100428c20 codec_unittest`WelsI16x16LumaPredH_sse2, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=2, address=0x100428c20)
  • frame #0: 0x0000000100428c20 codec_unittestWelsI16x16LumaPredH_sse2 frame #1: 0x0000000100317e0e codec_unittestWelsEnc::WelsMdI16x16(pFunc=0x0000000103001440, pCurDqLayer=0x0000000102d01900, pMbCache=0x0000000102d01ac0, iLambda=10) + 750 at svc_base_layer_md.cpp:401
    frame Please either add source, or remove this project #2: 0x000000010031a6ea codec_unittestWelsEnc::WelsMdIntraMb(pEncCtx=0x0000000102d00a90, pWelsMd=0x00007fff5fbfe358, pCurMb=0x00000001009b9080, pMbCache=0x0000000102d01ac0) + 58 at svc_base_layer_md.cpp:967 frame #3: 0x0000000100322c3a codec_unittestWelsEnc::WelsISliceMdEnc(pEncCtx=0x0000000102d00a90, pSlice=0x0000000102d01ac0) + 394 at svc_encode_slice.cpp:514
    frame Provide x264 binaries with patent licenses #4: 0x000000010032451f codec_unittestWelsEnc::WelsCodeOneSlice(pEncCtx=0x0000000102d00a90, kiSliceIdx=0, kiNalType=5) + 543 at svc_encode_slice.cpp:748 frame #5: 0x00000001002f9992 codec_unittestWelsEnc::WelsEncoderEncodeExt(pCtx=0x0000000102d00a90, pFbi=0x0000000103000038, pSrcPic=0x0000000102d00140) + 2850 at encoder_ext.cpp:3199
    frame Add a basic build environment for OSX. #6: 0x00000001003328f0 codec_unittestWelsEnc::CWelsH264SVCEncoder::EncodeFrameInternal(this=0x0000000102d000e0, pSrcPic=0x0000000102d00140, pBsInfo=0x0000000103000038) + 48 at welsEncoderExt.cpp:414 frame #7: 0x0000000100332896 codec_unittestWelsEnc::CWelsH264SVCEncoder::EncodeFrame(this=0x0000000102d000e0, kpSrcPic=0x0000000102d00140, pBsInfo=0x0000000103000038) + 134 at welsEncoderExt.cpp:395
    frame Assembly enable #8: 0x0000000100189c08 codec_unittestEncoderInterfaceTest_EncoderAdditionalOptionSetTest_Test::TestBody(this=0x0000000103000000) + 2024 at EncUT_EncoderExt.cpp:165 frame #9: 0x00000001002b3ed3 codec_unittestvoid testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(object=0x0000000103000000, method=0x0000000000000021, location=0x00000001003c5d73)(), char const_) + 131 at gtest.cc:2078
    frame reserved identifier violation #10: 0x000000010029e897 codec_unittestvoid testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(object=0x0000000103000000, method=0x0000000000000021, location=0x00000001003c5d73)(), char const_) + 119 at gtest.cc:2114 frame #11: 0x0000000100277fc5 codec_unittesttesting::Test::Run(this=0x0000000103000000) + 197 at gtest.cc:2150
    frame Completion of error handling #12: 0x000000010027929b codec_unittesttesting::TestInfo::Run(this=0x0000000100a38010) + 219 at gtest.cc:2326 frame #13: 0x000000010027a1c7 codec_unittesttesting::TestCase::Run(this=0x0000000100a38120) + 231 at gtest.cc:2444
    frame Rebase of EKR's patches for tracing and frame size with makefile updates. #14: 0x0000000100286cb8 codec_unittesttesting::internal::UnitTestImpl::RunAllTests(this=0x0000000100a03b70) + 952 at gtest.cc:4315 frame #15: 0x00000001002b0d53 codec_unittestbool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(object=0x0000000100a03b70, method=0x0000000100286900, location=0x00000001003c6432)(), char const*) + 131 at gtest.cc:2078
    frame Fix gtest detection #16: 0x00000001002a10a7 codec_unittestbool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(object=0x0000000100a03b70, method=0x0000000100286900, location=0x00000001003c6432)(), char const*) + 119 at gtest.cc:2114 frame #17: 0x0000000100286876 codec_unittesttesting::UnitTest::Run(this=0x000000010043bd78) + 422 at gtest.cc:3926
    frame fix VS build issue: can't find typedefs.h in console project #18: 0x00000001002656d1 codec_unittestRUN_ALL_TESTS() + 17 at gtest.h:2288 frame #19: 0x00000001002656ba codec_unittestmain(argc=1, argv=0x00007fff5fbffa60) + 170 at simple_test.cpp:26
    frame Consistently use unix newlines, remove trailing whitespace #20: 0x00007fff8be7d5fd libdyld.dylib`start + 1

[ethanhugg]

This commit is the likely culprit:
8945348

[ruil2]

#1389 pull request fixed this issue.

[ethanhugg]

Got a clean travis run on OSX . Thanks again.