MME restart several times an hour due to memory related errors

[Balaji-Jatla]

This Issue is related to stateless operations on RedisDB when ICS response is not coming up in MME.

[moinuddin1980]

Following are the traceback from the syslogs
Oct 16 02:36:10 ps1-agw-02 sessiond[153958]: I1016 02:36:10.746078 153967 SessionEvents.cpp:132] Could not log session_created event {"mac_addr":"","user_location":" 82 09 f1 07 00 0a 09 f1 07 00 46 bf 03","spgw_ip":"10.18.1.4","imei":"3587741004156301","pdp_start_time":1665912965,"session_id":"IMSI901700000032556-330103","apn":"hwinternet","msisdn":"","charging_characteristics":"","ipv6_addr":"","ip_addr":"100.64.64.231","imsi":"IMSI901700000032556"}, Error Message: Could not connect to FluentBit locally, Details: [Errno 111] Connection refused
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #0 0x5602632d8977 in magma::lte::NasStateConverter::esm_ebr_timer_data_to_proto(esm_ebr_timer_data_s const&, magma::lte::oai::EsmEbrTimerData*) /home/vagrant/magma/lte/gateway/c/core/oai/tasks/nas/nas_state_converter.cpp:300
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #1 0x5602632d9243 in magma::lte::NasStateConverter::esm_ebr_context_to_proto(esm_ebr_context_s const&, magma::lte::oai::EsmEbrContext*) /home/vagrant/magma/lte/gateway/c/core/oai/tasks/nas/nas_state_converter.cpp:420
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #2 0x560263052053 in magma::lte::MmeNasStateConverter::bearer_context_to_proto(bearer_context_s const&, magma::lte::oai::BearerContext*) /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_state_converter.cpp:208
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #3 0x5602630522f7 in magma::lte::MmeNasStateConverter::bearer_context_list_to_proto(ue_mm_context_s const&, magma::lte::oai::UeContext*) /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_state_converter.cpp:279
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #4 0x5602630537bc in magma::lte::MmeNasStateConverter::ue_context_to_proto(ue_mm_context_s const*, magma::lte::oai::UeContext*) /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_state_converter.cpp:523
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #5 0x5602630546dc in magma::lte::MmeNasStateConverter::ue_to_proto(ue_mm_context_s const*, magma::lte::oai::UeContext*) /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_state_converter.cpp:763
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #6 0x560262f5b738 in magma::lte::StateManager<mme_app_desc_s, ue_mm_context_s, magma::lte::oai::MmeNasState, magma::lte::oai::UeContext, magma::lte::MmeNasStateConverter>::write_ue_state_to_db(ue_mm_context_s const*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&) /home/vagrant/magma/lte/gateway/c/core/oai/include/state_manager.hpp:175
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #7 0x560262f3e647 in put_mme_ue_state /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_state.cpp:73
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #8 0x560262f1ef76 in handle_message /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_main.c:492
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #9 0x7f3f74c9d5c6 in zloop_start (/lib/x86_64-linux-gnu/libczmq.so.4+0x295c6)
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #10 0x560262f1f7c5 in mme_app_thread /home/vagrant/magma/lte/gateway/c/core/oai/tasks/mme_app/mme_app_main.c:516
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #11 0x7f3f75766608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608)
Oct 16 02:36:10 ps1-agw-02 mme[154167]: #12 0x7f3f746ac132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)
Oct 16 02:36:10 ps1-agw-02 mme[154167]: LeakSanitizer can not provide additional info.
Oct 16 02:36:10 ps1-agw-02 mme[154167]: SUMMARY: LeakSanitizer: SEGV /home/vagrant/magma/lte/gateway/c/core/oai/tasks/nas/nas_state_converter.cpp:300 in magma::lte::NasStateConverter::esm_ebr_timer_data_to_proto(esm_ebr_timer_data_s const&, magma::lte::oai::EsmEbrTimerData*)
Oct 16 02:36:10 ps1-agw-02 mme[154167]: ==154167==ABORTING

[moinuddin1980]

https://app.zenhub.com/files/170803235/b4962d87-a6c2-4e4d-a039-708a94b32aa6/download