Cannot start observer, ERROR malloc in log

[zhangysh1995]

Describe the bug (must be provided)

$ obd cluster start testdb
Get local repositories and plugins ok
Start observer ok
observer program health check ok
Connect to observer x

Your Environments (must be provided)

• OS: CentOS 7, 3.10.0-1160.el7.x86_64
• Compliler: gcc/g++ 4.8.5
• CPU: Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz
• Commit id (e.g. a3ffc7d8): 12676080774af2940e15911b9f2658f59c52899a

How To Reproduce(must be provided)
Followed #87 to fix devname and followed #19 to fix warning https://github.com/oceanbase/oceanbase/issues/87

Expected behavior

Connect to the server

Additional context

......
./observer.log.wf:22427:[2021-06-07 19:38:53.589284] ERROR [LIB] ob_malloc (ob_malloc.h:38) [79518][226][YB427F000001-0005C42B7A0EFBCA] [lt=5] [dc=0] allocate memory fail(attr=tenant_id=1, label=MysqlRequesReco, ctx_id=0, prio=0, nbyte=2088960) BACKTRACE:0x90a107e 0x90090c1 0x24d1eef 0x24d1b3b 0x24d1900 0x24d1552 0x24d130f 0x24dfe00 0x8692c6a 0x49ae651 0x86d10dd 0x86d2fd9 0x86d9194 0x8f8f342 0x6404e55 0x68b2a45 0x68880bd 0x4ff1a5d 0x4ff47ee 0x501630c 0x501482e 0x505dfc8 0x505b4d5 0x41b49f4 0x2a55b14 0x8ebc475 0x8eb9e4e 0x8eb46ef
./observer.log.wf:22454:[2021-06-07 19:38:53.648402] ERROR [LIB] ob_malloc (ob_malloc.h:38) [80252][1440][YB427F000001-0005C42B79FEFDC2] [lt=3] [dc=0] allocate memory fail(attr=tenant_id=1, label=MysqlRequesReco, ctx_id=0, prio=0, nbyte=2088960) BACKTRACE:0x90a107e 0x90090c1 0x24d1eef 0x24d1b3b 0x24d1900 0x24d1552 0x24d130f 0x24dfe00 0x8692c6a 0x49ae651 0x86d10dd 0x86d2fd9 0x86d9194 0x8f8f342 0x6402731 0x68a72db 0x68a6dfc 0x6886091 0x4ff14ed 0x4ff3609 0x5004638 0x5005acd 0x5007e30 0x500993f 0x8725d90 0x8727cce 0x8727196 0x8726dc7 0x872798d 0x41b49f4 0x2a55b14 0x8ebc475 0x8eb9e4e 0x8eb46ef
./observer.log.wf:22487:[2021-06-07 19:38:53.708831] ERROR [LIB] ob_malloc (ob_malloc.h:38) [80252][1440][YB427F000001-0005C42B79FEFDC3] [lt=4] [dc=0] allocate memory fail(attr=tenant_id=1, label=MysqlRequesReco, ctx_id=0, prio=0, nbyte=2088960) BACKTRACE:0x90a107e 0x90090c1 0x24d1eef 0x24d1b3b 0x24d1900 0x24d1552 0x24d130f 0x24dfe00 0x8692c6a 0x49ae651 0x86d10dd 0x86d2fd9 0x86d9194 0x8f8f342 0x6402731 0x68a72db 0x68a6dfc 0x6886091 0x4ff14ed 0x4ff3609 0x5004638 0x5005acd 0x5007e30 0x500993f 0x8725d90 0x8727cce 0x8727196 0x8726dc7 0x872798d 0x41b49f4 0x2a55b14 0x8ebc475 0x8eb9e4e 0x8eb46ef
./observer.log.wf:22519:[2021-06-07 19:38:53.769141] ERROR [LIB] ob_malloc (ob_malloc.h:38) [80252][1440][YB427F000001-0005C42B79FEFDC4] [lt=3] [dc=0] allocate memory fail(attr=tenant_id=1, label=MysqlRequesReco, ctx_id=0, prio=0, nbyte=2088960) BACKTRACE:0x90a107e 0x90090c1 0x24d1eef 0x24d1b3b 0x24d1900 0x24d1552 0x24d130f 0x24dfe00 0x8692c6a 0x49ae651 0x86d10dd 0x86d2fd9 0x86d9194 0x8f8f342 0x6402731 0x68a72db 0x68a6dfc 0x6886091 0x4ff14ed 0x4ff3609 0x5004638 0x5005acd 0x5007e30 0x500993f 0x8725d90 0x8727cce 0x8727196 0x8726dc7 0x872798d 0x41b49f4 0x2a55b14 0x8ebc475 0x8eb9e4e 0x8eb46ef
./observer.log.wf:22552:[2021-06-07 19:38:53.829479] ERROR [LIB] ob_malloc (ob_malloc.h:38) [80252][1440][YB427F000001-0005C42B79FEFDC5] [lt=3] [dc=0] allocate memory fail(attr=tenant_id=1, label=MysqlRequesReco, ctx_id=0, prio=0, nbyte=2088960) BACKTRACE:0x90a107e 0x90090c1 0x24d1eef 0x24d1b3b 0x24d1900 0x24d1552 0x24d130f 0x24dfe00 0x8692c6a 0x49ae651 0x86d10dd 0x86d2fd9 0x86d9194 0x8f8f342 0x6402731 0x68a72db 0x68a6dfc 0x6886091 0x4ff14ed 0x4ff3609 0x5004638 0x5005acd 0x5007e30 0x500993f 0x8725d90 0x8727cce 0x8727196 0x8726dc7 0x872798d 0x41b49f4 0x2a55b14 0x8ebc475 0x8eb9e4e 0x8eb46ef
./observer.log.wf:22584:[2021-06-07 19:38:53.889796] ERROR [LIB] ob_malloc (ob_malloc.h:38) [80252][1440][YB427F000001-0005C42B79FEFDC6] [lt=4] [dc=0] allocate memory fail(attr=tenant_id=1, label=MysqlRequesReco, ctx_id=0, prio=0, nbyte=2088960) BACKTRACE:0x90a107e 0x90090c1 0x24d1eef 0x24d1b3b 0x24d1900 0x24d1552 0x24d130f 0x24dfe00 0x8692c6a 0x49ae651 0x86d10dd 0x86d2fd9 0x86d9194 0x8f8f342 0x6402731 0x68a72db 0x68a6dfc 0x6886091 0x4ff14ed 0x4ff3609 0x5004638 0x5005acd 0x5007e30 0x500993f 0x8725d90 0x8727cce 0x8727196 0x8726dc7 0x872798d 0x41b49f4 0x2a55b14 0x8ebc475 0x8eb9e4e 0x8eb46ef
./observer.log.wf:22615:[2021-06-07 19:38:53.950120] ERROR [LIB] ob_malloc (ob_malloc.h:38) [80252][1440][YB427F000001-0005C42B79FEFDC7] [lt=3] [dc=0] allocate memory fail(attr=tenant_id=1, label=MysqlRequesReco, ctx_id=0, prio=0, nbyte=2088960) BACKTRACE:0x90a107e 0x90090c1 0x24d1eef 0x24d1b3b 0x24d1900 0x24d1552 0x24d130f 0x24dfe00 0x8692c6a 0x49ae651 0x86d10dd 0x86d2fd9 0x86d9194 0x8f8f342 0x6402731 0x68a72db 0x68a6dfc 0x6886091 0x4ff14ed 0x4ff3609 0x5004638 0x5005acd 0x5007e30 0x500993f 0x8725d90 0x8727cce 0x8727196 0x8726dc7 0x872798d 0x41b49f4 0x2a55b14 0x8ebc475 0x8eb9e4e 0x8eb46ef
./observer.log.wf:22647:[2021-06-07 19:38:54.010839] ERROR [LIB] ob_malloc (ob_malloc.h:38) [80252][1440][YB427F000001-0005C42B79FEFDC8] [lt=6] [dc=0] allocate memory fail(attr=tenant_id=1, label=MysqlRequesReco, ctx_id=0, prio=0, nbyte=2088960) BACKTRACE:0x90a107e 0x90090c1 0x24d1eef 0x24d1b3b 0x24d1900 0x24d1552 0x24d130f 0x24dfe00 0x8692c6a 0x49ae651 0x86d10dd 0x86d2fd9 0x86d9194 0x8f8f342 0x6402731 0x68a72db 0x68a6dfc 0x6886091 0x4ff14ed 0x4ff3609 0x5004638 0x5005acd 0x5007e30 0x500993f 0x8725d90 0x8727cce 0x8727196 0x8726dc7 0x872798d 0x41b49f4 0x2a55b14 0x8ebc475 0x8eb9e4e 0x8eb46ef
......

My configuration was given in #87.

[nroskill]

system_memory is 256G...

use free -g to see use physical memory, and reset system_memory.
system_memory should be less than memory_limit, such as 4G according to your memory_limit(8G).

[objunyue]

system_memory means observer internal system memory

[zhangysh1995]

I would like to suggest adding the variable description if possible, because I understood it as my system memory so I set it to the maximum.