[BUG] pthread case hang when running glibc test with simulation mode

[jessehui]

Describe the bug

pthread case hangs when running glibc test in simulation mode. It is not always happening but can still be easily reproduced if running for within 10 times.

To reproduce

Steps to reproduce the behavior:

1. Install NGO master (a999922) in release mode
2. TESTS=pthread OCCLUM_LOG_LEVEL=trace TEST_CPUS=2 SGX_MODE=SIM taskset -c 0-1 make test-glibc times=100

Expected behavior

All tests can pass.

Logs

2022-02-16T11:18:36.748Z][TRACE][C:1][T39][#5][···Futex] ret = 0x1
[2022-02-16T11:18:36.749Z][ERROR][C:1][T39][#6] Error = ENOSYS (#38, Function not implemented): Invalid system call number (28) [line = 726, file = src/entry/syscall.rs]
[2022-02-16T11:18:36.749Z][TRACE][C:1][T39][#7][····Exit] Syscall { num = Exit, exit_status = 0 }
[2022-02-16T11:18:36.749Z][DEBUG][C:1][T39][#7][····Exit] exit: 0
[2022-02-16T11:18:36.749Z][DEBUG][C:1][T39][#7][····Exit] futex_wake_bitset addr: 0x7fff9fddb9d0, max_count: 1, bitset: 0xffffffff
[2022-02-16T11:18:36.749Z][DEBUG][C:1][T39][#7][····Exit] wake the rubust_list: 0x7fff9fddb9e0
[2022-02-16T11:18:36.749Z][DEBUG][C:1][T39][#7][····Exit] futex_wake_bitset addr: 0x7fff751703f0, max_count: 1, bitset: 0xffffffff
[2022-02-16T11:18:36.749Z][TRACE][C:1][T39][#7][····Exit] ret = 0x0
[2022-02-16T11:18:36.749Z][TRACE][C:1][T41][#4][···Futex] ret = 0x0
[2022-02-16T11:18:36.749Z][TRACE][C:1][T41][#5][···Futex] Syscall { num = Futex, futex_addr = 0x7fff9f3ffc80, futex_op = 129, futex_val = 1, timeout = 0, futex_new_addr = 0x0, bitset = 2690502023 }
[2022-02-16T11:18:36.750Z][DEBUG][C:1][T41][#5][···Futex] futex_wake_bitset addr: 0x7fff9f3ffc80, max_count: 1, bitset: 0xffffffff
[2022-02-16T11:18:36.750Z][TRACE][C:1][T41][#5][···Futex] ret = 0x0
[2022-02-16T11:18:36.751Z][ERROR][C:1][T41][#6] Error = ENOSYS (#38, Function not implemented): Invalid system call number (28) [line = 726, file = src/entry/syscall.rs]
[2022-02-16T11:18:36.751Z][TRACE][C:1][T41][#7][····Exit] Syscall { num = Exit, exit_status = 0 }
[2022-02-16T11:18:36.751Z][DEBUG][C:1][T41][#7][····Exit] exit: 0
[2022-02-16T11:18:36.751Z][DEBUG][C:1][T41][#7][····Exit] futex_wake_bitset addr: 0x7fffa05dd9d0, max_count: 1, bitset: 0xffffffff
[2022-02-16T11:18:36.751Z][DEBUG][C:1][T41][#7][····Exit] wake the rubust_list: 0x7fffa05dd9e0
[2022-02-16T11:18:36.751Z][DEBUG][C:1][T41][#7][····Exit] futex_wake_bitset addr: 0x7fff751703f0, max_count: 1, bitset: 0xffffffff
[2022-02-16T11:18:36.751Z][TRACE][C:1][T41][#7][····Exit] ret = 0x0
[2022-02-16T11:18:37.012Z][DEBUG][C:0][T32][#88] handle interrupt
[2022-02-16T11:18:37.012Z][DEBUG][C:0][T40][#0] Thread #40 is executed as task #40
[2022-02-16T11:18:37.012Z][TRACE][C:0][T40][#1][SetRobustList] Syscall { num = SetRobustList, list_head_ptr = 0x7fffa01dc9e0, len = 24 }
[2022-02-16T11:18:37.012Z][DEBUG][C:0][T40][#1][SetRobustList] set_robust_list: list_head_ptr: 0x7fffa01dc9e0, len: 24
[2022-02-16T11:18:37.012Z][TRACE][C:0][T40][#1][SetRobustList] ret = 0x0
[2022-02-16T11:18:37.012Z][TRACE][C:0][T40][#2][···Futex] Syscall { num = Futex, futex_addr = 0x7fff9f3ffc80, futex_op = 128, futex_val = 2, timeout = 0, futex_new_addr = 0x7fff9f3ffc80, bitset = 2686303623 }
[2022-02-16T11:18:37.012Z][DEBUG][C:0][T40][#2][···Futex] futex_wait_bitset addr: 0x7fff9f3ffc80, val: 2, timeout: None, bitset: 0xffffffff
[2022-02-16T11:18:37.012Z][TRACE][C:0][T32][#89][···Futex] Syscall { num = Futex, futex_addr = 0x7fff9f3ffc80, futex_op = 128, futex_val = 2, timeout = 0, futex_new_addr = 0x7fff9f3ffc80, bitset = 2671770800 }
[2022-02-16T11:18:37.012Z][DEBUG][C:0][T32][#89][···Futex] futex_wait_bitset addr: 0x7fff9f3ffc80, val: 2, timeout: None, bitset: 0xffffffff
[2022-02-16T11:18:46.621Z][DEBUG][C:0] Timer Wheel: timeout
[2022-02-16T11:18:46.621Z][DEBUG][C:0] Timer Wheel: will sleep 10s
[2022-02-16T11:18:56.621Z][DEBUG][C:0] Timer Wheel: timeout
[2022-02-16T11:18:56.621Z][DEBUG][C:0] Timer Wheel: will sleep 10s
[2022-02-16T11:19:06.621Z][DEBUG][C:0] Timer Wheel: timeout
[2022-02-16T11:19:06.621Z][DEBUG][C:0] Timer Wheel: will sleep 10s

Then the timer sleeps forever.

Environment

• HW: [e.g. SGX1, SGX2] SGX2
• OS: [e.g. Ubuntu18.04, CentOS8.1] ubuntu 18.04
• Occlum version: [e.g. 0.17.0] NGO master (a999922) release mode

Additional context

Running pthread musl test with all other variables the same can't reproduce this behavior in 100 times.
Running pthread glibc test in hardware mode with all other variables the same can't reproduce this behavior in 100 times.
Not sure if this is related to libc or SGX mode.