Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wasm][ci] Intermittent failures with ChakraCore Tests #13195

Closed
kjpou1 opened this issue Feb 26, 2019 · 13 comments · Fixed by #14653
Labels

Comments

@kjpou1
Copy link
Member

@kjpou1 kjpou1 commented Feb 26, 2019

The tests using Chakra Javascript Runtime are aborting intermittently.

There are no stack traces produced with only Abort trap: 6 being output. This does not happen all the time and eventually passes.

This is happening with the following CI tests:

run-ch-mini
run-ch-system-core

This also happens locally on Mac 1 out of 10 runs.

@kjpou1 kjpou1 changed the title [wasm][ci] Intermittent failures with ChakraTests [wasm][ci] Intermittent failures with ChakraCore Tests Feb 26, 2019
kjpou1 added a commit to kjpou1/jsvu that referenced this issue Feb 27, 2019
@vargaz

This comment has been minimized.

Copy link
Member

@vargaz vargaz commented Feb 28, 2019

It seems to randomly hit this assert (chakra 9360fcfb8d8b6397201b353c6a2814aa6c50ded0):

    frame #0: 0x000000010045d8ae ch`IRBuilderAsmJs::InsertLabels(this=0x000070000b3edaf8) at IRBuilderAsmJs.cpp:1044
   1041	        uint offset = reloc->GetOffset();
   1042	        uint const branchOffset = reloc->GetBranchOffset();
   1043	
-> 1044	        Assert(!IsLoopBody() || offset <= GetLoopBodyExitInstrOffset());
   1045	
   1046	        if (branchInstr->m_opcode == Js::OpCode::MultiBr)
   1047	        {

  * frame #0: 0x000000010045d8ae ch`IRBuilderAsmJs::InsertLabels(this=0x000070000b3edaf8) at IRBuilderAsmJs.cpp:1044
    frame #1: 0x000000010042833a ch`IRBuilderAsmJs::Build(this=0x000070000b3edaf8) at IRBuilderAsmJs.cpp:192
    frame #2: 0x00000001002a83ca ch`Func::TryCodegen(this=0x000070000b3edf80) at Func.cpp:408
    frame #3: 0x00000001002a7da4 ch`Func::Codegen(alloc=0x000070000b3ee550, workItem=0x000000010e081030, threadContextInfo=0x0000000109800058, scriptContextInfo=0x000000010a00d258, outputData=0x000070000b3eea40, epInfo=0x000000010e53d980, runtimeInfo=0x0000000000000000, polymorphicInlineCacheInfo=0x000000010e224020, codeGenAllocators=0x000000010a00fc58, codeGenProfiler=0x0000000000000000, isBackgroundJIT=true) at Func.cpp:323
    frame #4: 0x0000000100648aad ch`NativeCodeGenerator::CodeGen(this=0x0000000108a00f38, pageAllocator=0x0000000108a016f8, workItemData=0x000000010dd3f3a0, jitWriteData=0x000070000b3eea40, foreground=false, epInfo=0x000000010e53d980) at NativeCodeGenerator.cpp:894
    frame #5: 0x000000010064b459 ch`NativeCodeGenerator::CodeGen(this=0x0000000108a00f38, pageAllocator=0x0000000108a016f8, workItem=0x000000010dd3f378, foreground=false) at NativeCodeGenerator.cpp:1011
    frame #6: 0x000000010064ee79 ch`NativeCodeGenerator::Process(this=0x0000000108a00f38, job=0x000000010dd3f380, threadData=0x0000000108a016d8) at NativeCodeGenerator.cpp:1895
    frame #7: 0x000000010070d631 ch`JsUtil::BackgroundJobProcessor::Process(this=0x0000000108a01098, job=0x000000010dd3f380, threadData=0x0000000108a016d8) at Jobs.cpp:1037
    frame #8: 0x000000010070dc20 ch`JsUtil::BackgroundJobProcessor::Run(this=0x0000000108a01098, threadData=0x0000000108a016d8) at Jobs.cpp:1135
    frame #9: 0x000000010070aeef ch`JsUtil::BackgroundJobProcessor::StaticThreadProc(lpParam=0x0000000108a016d8) at Jobs.cpp:1319
    frame #10: 0x00000001000c7da2 ch`CorUnix::CPalThread::ThreadEntry(pvParam=0x000000010a00e800) at pal_thread.cpp:1605
    frame #11: 0x00007fff718042eb libsystem_pthread.dylib`_pthread_body + 126
    frame #12: 0x00007fff71807249 libsystem_pthread.dylib`_pthread_start + 66
    frame #13: 0x00007fff7180340d libsystem_pthread.dylib`thread_start + 13
@vargaz

This comment has been minimized.

Copy link
Member

@vargaz vargaz commented Feb 28, 2019

The method where the assert happens seems to be random as well:
(lldb) p this->m_func->GetJITFunctionBody()->m_bodyData.displayName
(WCHAR *const) $1 = 0x0000000106dd1b70 u"wasm-function[3337]"
(lldb) p this->m_func->GetJITFunctionBody()->m_bodyData.displayName
(WCHAR *const) $3 = 0x0000000106fdee40 u"wasm-function[3557]"

@vargaz

This comment has been minimized.

Copy link
Member

@vargaz vargaz commented Mar 4, 2019

Running a debug build of chakra with the -Prejit argument makes the failure deterministic.

@vargaz

This comment has been minimized.

Copy link
Member

@vargaz vargaz commented Mar 4, 2019

Testcase:

typedef struct _MonoType2 MonoType2;

struct _MonoType2 {
	union {
		MonoType2 *type;
	} data;
	int type;
};

int
foo3 (MonoType2 *t)
{
	return t->type;
}

typedef enum {
	MONO_TYPE2_PTR        = 0x0f,       /* arg: <type> token */
	MONO_TYPE2_VALUETYPE  = 0x11,       /* arg: <type> token */
	MONO_TYPE2_CLASS      = 0x12,       /* arg: <type> token */
	MONO_TYPE2_VAR	     = 0x13,	   /* number */
	MONO_TYPE2_ARRAY      = 0x14,       /* type, rank, boundsCount, bound1, loCount, lo1 */
	MONO_TYPE2_GENERICINST= 0x15,	   /* <type> <type-arg-count> <type-1> \x{2026} <type-n> */
	MONO_TYPE2_SZARRAY    = 0x1d,       /* 0-based one-dim-array */
	MONO_TYPE2_MVAR	     = 0x1e,       /* number */
} MonoTypeEnum2;

int
foo2 (MonoType2 *t)
{
	switch (t->type) {
	case MONO_TYPE2_VAR:
	case MONO_TYPE2_MVAR:
		return 1;
	case MONO_TYPE2_SZARRAY:
		return foo2 (t->data.type);
	case MONO_TYPE2_ARRAY:
		return foo2 (t->data.type);
	case MONO_TYPE2_PTR:
		return foo2 (t->data.type);
	case MONO_TYPE2_GENERICINST:
		return t->data.type->type;
	case MONO_TYPE2_CLASS:
	case MONO_TYPE2_VALUETYPE:
		return foo2 (t->data.type);
	default:
		return 0;
	}
}

int
main (int argc, char **argv)
{
	MonoType2 t;

	t.data.type = 0;
	t.type = argc;
	return foo2 (&t);
}

Compile with:
emcc -O2 -s WASM=1 -s ASSERTIONS=1 -g4 bug.c
Run with:
~/ChakraCore/out/Debug/ch -PreJit a.out.js

@vargaz

This comment has been minimized.

Copy link
Member

@vargaz vargaz commented Mar 4, 2019

@vargaz

This comment has been minimized.

Copy link
Member

@vargaz vargaz commented Mar 6, 2019

@kjpou1

This comment has been minimized.

Copy link
Member Author

@kjpou1 kjpou1 commented Mar 15, 2019

microsoft/ChakraCore#6037 seems to be fixed now

@kjpou1

This comment has been minimized.

Copy link
Member Author

@kjpou1 kjpou1 commented Apr 9, 2019

@kjpou1

This comment has been minimized.

Copy link
Member Author

@kjpou1 kjpou1 commented Apr 10, 2019

So far using chakra build from chakra repo: https://github.com/Microsoft/ChakraCore with the fix microsoft/ChakraCore#6072 applied all the tests are running through. Have executed tests locally with the build multiple times.

@kjpou1

This comment has been minimized.

Copy link
Member Author

@kjpou1 kjpou1 commented Apr 10, 2019

@kjpou1

This comment has been minimized.

Copy link
Member Author

@kjpou1 kjpou1 commented Apr 11, 2019

Update release version link still points to 1.11.7: microsoft/ChakraCore#6093

@kjpou1

This comment has been minimized.

Copy link
Member Author

@kjpou1 kjpou1 commented Apr 12, 2019

microsoft/ChakraCore#6093 was fixed so we now have access to 1.11.8. Even though testing repo source build of ChakraCore with the fix microsoft/ChakraCore#6072 applied never ran into an abort with a minimum of 20 runs for each test that is run through our CI for Chakra Core.

Unfortunately with the 1.11.8 release I am experiencing multiple aborts still. Will need to report this again. :-(

@lewing ^

@kjpou1 kjpou1 added the target-wasm label Apr 12, 2019
@kjpou1

This comment has been minimized.

Copy link
Member Author

@kjpou1 kjpou1 commented Apr 16, 2019

It looks like the 1.11.8 release did not contain the fix for this issue. May 14th is the next release date and should contain the fix for microsoft/ChakraCore#6037.

@lewing ^

kjpou1 added a commit to kjpou1/mono that referenced this issue May 27, 2019
- Chakra 1.11.9 release is now out which fixes microsoft/ChakraCore#6037

Closes mono#13195
kjpou1 added a commit that referenced this issue May 28, 2019
- Chakra 1.11.9 release is now out which fixes microsoft/ChakraCore#6037

Closes #13195
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.