vizier-pem crashed when running "px run px/cluster" #368
Description
Describe the bug
vizier-pem crashed when running "px run px/cluster"
Screenshots
[root@vm183185 pixie_cli]# px get viziers
Pixie CLI
*******************************
* IN TESTING MODE
* PL_CLOUD_ADDR=dev.withpixie.dev
*******************************
Table ID: viziers
CLUSTERNAME ID K8S VERSION VIZIER VERSION LAST HEARTBEAT PASSTHROUGH STATUS STATUS MESSAGE
kubernetes-admin@kubernetes b548fe5c-5397-4459-8901-9e7759479a12 v1.22.4 0.9.11 now true CS_UNHEALTHY PEMs are experiencing a high crash rate. Your Pi
[root@vm183185 pixie_cli]# kubectl logs vizier-pem-zrh8p -n pl
F1130 02:05:39.455119 40597 registration.cc:48] Timeout waiting for registration ack
*** Check failure stack trace: ***
@ 0x5058dff google::LogMessageFatal::~LogMessageFatal()
@ 0x42e3703 std::_Function_handler<>::_M_invoke()
@ 0x4926b86 uv__run_timers
@ 0x49279ab uv_run
@ 0x42d4b0a px::vizier::agent::Manager::Run()
@ 0x13eb6d2 main
@ 0x7fc84b932e0b __libc_start_main
@ 0x13ea8ee _start
E1130 02:05:39.487874 40597 signal_action.cc:63] Caught Aborted, suspect faulting address 0x9e95. Trace:
**************************
PC: @ 0x505d05c (unknown) google::DumpStackTraceAndExit()
@ 0x5056df2 (unknown) google::LogMessage::SendToLog()
@ 0x5057093 (unknown) google::LogMessage::Flush()
@ 0x5058dff (unknown) google::LogMessageFatal::~LogMessageFatal()
@ 0x42e3703 (unknown) std::_Function_handler<>::_M_invoke()
@ 0x4926b86 (unknown) uv__run_timers
@ 0x49279ab (unknown) uv_run
@ 0x42d4b0a (unknown) px::vizier::agent::Manager::Run()
@ 0x13eb6d2 (unknown) main
@ 0x7fc84b932e0b (unknown) __libc_start_main
@ 0x13ea8ee (unknown) _start
**************************
Logs
pixie_logs_20211130021227.zip.log
App information (please complete the following information):
- Pixie version master
- K8s cluster version 1.22.4
- Node Kernel version 4.18
- Browser version