SYNCD exits with SAI Failures while onboarding Dell EMC S5212 to Community SONiC

[thaj-deen]

Description

SYNCD exits with SAI Failures while onboarding new platform - Dell EMC S5212 to Community SONiC

Dell S5212 is based on BRCM TD3.X5. Community SONiC's SAI Support for BCM56771 is available in SAI versions > 4.x

My cloned repo (label #7996) is running 5.0.0.1

Steps to reproduce the issue:

1. Cloned Community code label Fix sonic-cfggen contextlib err #7996 on June 29th
2. Made changes for onboarding S5212 and built the image
3. Ports failed to come up. syslog's point to _brcm_sai_dm_init () failure

BRCM SAI ver: [5.0.0.1], OCP SAI ver: [1.8.1], SDK ver: [sdk-6.5.21] CANCUN ver: [6.4.1]

Describe the results you received:

1. "show interface transceiver presence" was empty
2. Went over the syslog and able to find that SWSS & SYNCD failed to initialize the chip

Jul 13 01:59:08.663055 sonic ERR syncd#syncd: [none] SAI_API_HOSTIF:_brcm_sai_create_trap_resource:9249 field group config create failed with error Feature unavailable (0xfffffff0).
Jul 13 01:59:08.663147 sonic ERR syncd#syncd: [none] SAI_API_HOSTIF:_brcm_sai_alloc_hostif:10227 create trap resource failed with error -2.
Jul 13 01:59:08.663195 sonic CRIT syncd#syncd: [none] SAI_API_SWITCH:_brcm_sai_dm_init:24877 initializing hostif state failed with error -2.
Jul 13 01:59:08.663241 sonic CRIT syncd#syncd: [none] SAI_API_SWITCH:brcm_sai_create_switch:1386 initializing modules data failed with error -2.
Jul 13 01:59:08.663288 sonic ERR syncd#syncd: :- sendApiResponse: api SAI_COMMON_API_CREATE failed in syncd mode: SAI_STATUS_NOT_SUPPORTED
Jul 13 01:59:08.666357 sonic ERR swss#orchagent: :- create: create status: SAI_STATUS_NOT_SUPPORTED
Jul 13 01:59:08.666442 sonic ERR swss#orchagent: :- main: Failed to create a switch, rv:-2
Jul 13 01:59:08.666865 sonic INFO teamd#supervisord 2021-07-13 01:59:08,665 INFO spawned: 'teammgrd' with pid 19
Jul 13 01:59:08.666992 sonic NOTICE swss#orchagent: :- uninitialize: begin
Jul 13 01:59:08.667098 sonic NOTICE swss#orchagent: :- uninitialize: begin
Jul 13 01:59:08.667202 sonic NOTICE swss#orchagent: :- ~RedisChannel: join ntf thread begin
Jul 13 01:59:08.667305 sonic NOTICE swss#orchagent: :- ~RedisChannel: join ntf thread end
Jul 13 01:59:08.667363 sonic NOTICE swss#orchagent: :- clear_local_state: clearing local state
Jul 13 01:59:08.667425 sonic NOTICE swss#orchagent: :- meta_init_db: begin
Jul 13 01:59:08.667627 sonic NOTICE swss#orchagent: :- meta_init_db: end
Jul 13 01:59:08.667746 sonic NOTICE swss#orchagent: :- uninitialize: end
Jul 13 01:59:08.667810 sonic NOTICE swss#orchagent: :- stopRecording: stopped recording
Jul 13 01:59:08.667866 sonic NOTICE swss#orchagent: :- stopRecording: closed recording file: sairedis.rec
Jul 13 01:59:08.667928 sonic NOTICE swss#orchagent: :- uninitialize: end
Jul 13 01:59:08.674825 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_INIT_SWITCH: true
Jul 13 01:59:08.674909 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_FDB_EVENT_NOTIFY: 0x559e54b4bdb0
Jul 13 01:59:08.674961 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_PORT_STATE_CHANGE_NOTIFY: 0x559e54b4bdc0
Jul 13 01:59:08.675012 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_SWITCH_SHUTDOWN_REQUEST_NOTIFY: 0x559e54b4bdd0
Jul 13 01:59:08.675061 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_SRC_MAC_ADDRESS: 3C:2C:30:20:F8:00

Describe the results you expected:

All ports should be UP and port attributes should be present correctly

Output of show version:

admin@sonic:~$ show version

SONiC Software Version: SONiC.s5212-comm-onb.0-dirty-20210706.131822
Distribution: Debian 10.10
Kernel: 4.19.0-12-2-amd64
Build commit: 101b1fa
Build date: Tue Jul 6 20:26:17 UTC 2021
Built by: tnawabjan@dell-vm-82

Platform: x86_64-dellemc_s5212f_c3538-r0
HwSKU: DellEMC-S5212f-P-25G
ASIC: broadcom
ASIC Count: 1
Serial Number: None
Model Number: None
Hardware Revision: None
Uptime: 02:44:23 up 3 days, 23:19, 1 user, load average: 1.14, 1.14, 1.15

Docker images:
REPOSITORY TAG IMAGE ID SIZE
docker-syncd-brcm latest bed50370ec08 705MB
docker-syncd-brcm s5212-comm-onb.0-dirty-20210706.131822 bed50370ec08 705MB
docker-snmp latest 9a320c706461 454MB
docker-snmp s5212-comm-onb.0-dirty-20210706.131822 9a320c706461 454MB
docker-teamd latest 7c70590d0a94 424MB
docker-teamd s5212-comm-onb.0-dirty-20210706.131822 7c70590d0a94 424MB
docker-nat latest 0c32603d6f9b 427MB
docker-nat s5212-comm-onb.0-dirty-20210706.131822 0c32603d6f9b 427MB
docker-platform-monitor latest a89710d730de 627MB
docker-platform-monitor s5212-comm-onb.0-dirty-20210706.131822 a89710d730de 627MB
docker-lldp latest 1b9912b51c79 453MB
docker-lldp s5212-comm-onb.0-dirty-20210706.131822 1b9912b51c79 453MB
docker-dhcp-relay latest 54c7da112c68 420MB
docker-dhcp-relay s5212-comm-onb.0-dirty-20210706.131822 54c7da112c68 420MB
docker-database latest fc187948f01f 413MB
docker-database s5212-comm-onb.0-dirty-20210706.131822 fc187948f01f 413MB
docker-router-advertiser latest b82847ee7830 413MB
docker-router-advertiser s5212-comm-onb.0-dirty-20210706.131822 b82847ee7830 413MB
docker-orchagent latest fe29165907f7 442MB
docker-orchagent s5212-comm-onb.0-dirty-20210706.131822 fe29165907f7 442MB
docker-macsec latest 0bfb6554a730 427MB
docker-macsec s5212-comm-onb.0-dirty-20210706.131822 0bfb6554a730 427MB
docker-sonic-telemetry latest e5ef93a9bdf0 501MB
docker-sonic-telemetry s5212-comm-onb.0-dirty-20210706.131822 e5ef93a9bdf0 501MB
docker-sonic-mgmt-framework latest 789bcb7e4885 570MB
docker-sonic-mgmt-framework s5212-comm-onb.0-dirty-20210706.131822 789bcb7e4885 570MB
docker-fpm-frr latest a186f7541d60 442MB
docker-fpm-frr s5212-comm-onb.0-dirty-20210706.131822 a186f7541d60 442MB
docker-sflow latest a28a39fe7fc1 425MB
docker-sflow s5212-comm-onb.0-dirty-20210706.131822 a28a39fe7fc1 425MB

admin@sonic:~$
sonic_dump_sonic_20210714_024447.tar.gz

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

[zhangyanzhao]

already tracked with mail thread offline. No action from community so far