ignore FRR error messages on 202311 branch

[lipxu]

Description of PR

Summary:
Fixes # (issue)
28150848

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• Test case(new/improvement)

Back port request

• 201911
• 202012
• 202205
• 202305
• 202311

Approach

What is the motivation for this PR?

Many cases failed during teardown due to loganalyzer with below FRR related error messages.
The failure only happen on the device which runs slim image.

bgp#zebra[36]: [SHWNK-NWT5S][EC 100663304] No such command on config line 10: no fpm use-next-hop-groups
bgp#zebra[36]: [SHWNK-NWT5S][EC 100663304] No such command on config line 12: fpm address 127.0.0.1
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[50]: [MVZKX-EG443][EC 33554452] bgp_process_packet: BGP OPEN receipt failed for peer: XXX
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[50]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[49]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[49]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#zebra[36]: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: No route to host, type=RTM_NEWROUTE(24), seq=95412, pid=3576249171
ERR bgp#bgpd[65]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#zebra[36]: [WVJCK-PPMGD][EC 4043309093] netlink-dp (NS 0) error: No route to host, type=RTM_NEWROUTE(24), seq=95495, pid=3576249171
ERR bgp#bgpd[65]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[65]: [XETTR-D5MR0][EC 100663316] Attempting to process an I/O event but for fd: 66(8) no thread to handle this!
ERR bgp#bgpd[53]: [H4B4J-DCW2R][EC 33554455] XXX [Error] bgp_read_packet error: Connection reset by peer
ERR bgp#zebra[62]: [SHWNK-NWT5S][EC 100663304] No such command on config line 10: no fpm use-next-hop-groups
ERR bgp#zebra[62]: [SHWNK-NWT5S][EC 100663304] No such command on config line 12: fpm address 127.0.0.1
ERR bgp#zebra[62]: [RFREB-PAV4B][EC 100663299] vty_read: read error on vty client fd 26, closing: Connection reset by peer
ERR bgp#zebra[36]: [SHWNK-NWT5S][EC 100663304] No such command on config line 10: no fpm use-next-hop-groups
ERR bgp#zebra[36]: [SHWNK-NWT5S][EC 100663304] No such command on config line 12: fpm address 127.0.0.1
ERR bgp#bgpd[51]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[49]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#bgpd[51]: [VCGF0-X62M1][EC 100663301] INTERFACE_STATE: Cannot find IF PortChannel999 in VRF 0
ERR bgp#bgpd[51]: [VCGF0-X62M1][EC 100663301] INTERFACE_STATE: Cannot find IF PortChannel999 in VRF 0
ERR bgp#bgpd[48]: [P3GYW-PBKQG][EC 33554466] XXX [FSM] unexpected packet received in state OpenSent
ERR bgp#zebra[36]: [SHWNK-NWT5S][EC 100663304] No such command on config line 10: no fpm use-next-hop-groups
ERR bgp#zebra[36]: [SHWNK-NWT5S][EC 100663304] No such command on config line 12: fpm address 127.0.0.1

How did you do it?

Slim image's rsyslog config has been modified by sonic-net/sonic-buildimage#17905 by mistake.
So the FRR error message would output into syslog and cause teardown failure.
Created a github issue sonic-net/sonic-buildimage#19047 to track the issue and ignore these related error message until the issue fixed to make the nightly build more stable.

r, ".* ERR bgp#bgpd.* unexpected packet received in state OpenSent"
r, ".* ERR bgp#bgpd.* INTERFACE_STATE: Cannot find IF .*"
r, ".* ERR bgp#bgpd.* bgp_process_packet: BGP OPEN receipt failed for peer.*"
r, ".* ERR bgp#bgpd.* bgp_read_packet error: Connection reset by peer.*"
r, ".* ERR bgp#bgpd.* Attempting to process an I/O event but for fd.*"
r, ".* ERR bgp#zebra.* No such command on config line .*"
r, ".* ERR bgp#zebra.* error: No route to host.*"
r, ".* ERR bgp#zebra.* read error on vty client fd .*"

How did you verify/test it?

NA

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[StormLiangMS]

hi @lipxu could we give more background about why we ignore those err messages? For these err messages, could we open a github issue or ADO to track the progress to investigate?

[StormLiangMS]

LGTM