[Mellanox] make sure shared storage with syncd is cleared on restarts

[stepanblyschak]

Why I did it

Sharing the storage of syncd with other proprietary application extensions allows them to communicate with syncd in differnt ways.
If one container wants to pass some information to syncd then shared storage can be used. However, today the shared storage isn't cleaned on restarts making it possible for syncd to read out-of-date information generated in the past.

NOTE: No plans to use it for standard SONIC dockers and we are working on removing the SDK dependency from PMON docker

How I did it

Implemented new service to clean the shared storage.

How to verify it

Do reboot/fast-reboot/warm-reboot/config-reload/systemctl restart swss and verify /tmp/ is cleaned after each restart in syncd container.

Which release branch to backport (provide reason below if selected)

• 201811
• 201911
• 202006
• 202012
• 202106
• 202111
• 202205
• 202211

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

[liat-grozovik]

@stepanblyschak does it have any impact of upgrades between different version on the same branch and between branches?

[liat-grozovik]

/azp run Azure.sonic-buildimage

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[liat-grozovik]

@lguohan would you like to review or assign someone to review? LGTM

[lguohan]

frankly, i do not like the idea of having additional channel for communication between dockers. there are lots of challenges of dependency between dockers and very hard to debug. I would really like to understand why this is needed, what is the data shared between syncd docker and other dockers.

[lguohan]

@vaibhavhd and @yxieca for more comments.

[stepanblyschak]

@lguohan This is not new communication channel. It exists for quite some time and used by pmon, what-just-happened to communicate with SDK directly as not all features are now supported by SAI. This PR is not about adding new communication channel but making the shared location is cleared on restarts.

[yxieca]

This change has an risk: since the shared folder is used by syncd and other dockers:

1. all services uses the share folder should start after the service creates / removes the share folder.
2. all service uses the shared folder have to stop access the shared folder in order for the service to be able to remove the shared folder.

[stepanblyschak]

@yxieca This is guernteed by systemd. What is the risk?

[stepanblyschak]

@yxieca @liat-grozovik @lguohan Clarified the purpose in PR description.

[yxieca]

@yxieca This is guernteed by systemd. What is the risk?

I am not sure that is fully guaranteed. e.g. if syncd is stopping while pmon is accessing the shared memory. Then the syncd won't be able to remove the shared memory folder. As result, it could see stale information after next start. Or the service stop/start could fail?

[stepanblyschak]

@yxieca Pmon is instructed to start After=syncd -

sonic-buildimage/files/build_templates/pmon.service.j2

Line 6 in fe773e2

After=syncd.service

.

That means, on shutdown the reverse order is used. -

When two units with an ordering dependency between them are shut down, the inverse of the start-up order is applied. I.e. if a unit is configured with After= on another unit, the former is stopped before the latter if both are shut down

If syncd service is about to stop (no matter if gracefully by request or abnormally) systemd first stops pmon service.

[liat-grozovik]

@lguohan could you please help to review/approve?

[lguohan]

@yxieca , i see this note.

NOTE: No plans to use it for standard SONIC dockers and we are working on removing the SDK dependency from PMON docker

looks like it is only used between mellanox syncd docker and pmon docker, and they are working on removing that dependency, so that will be removed in the future. is this ok?

[yxieca]

@yxieca , i see this note.

NOTE: No plans to use it for standard SONIC dockers and we are working on removing the SDK dependency from PMON docker

looks like it is only used between mellanox syncd docker and pmon docker, and they are working on removing that dependency, so that will be removed in the future. is this ok?

I believe @stepanblyschak is checking warm reboot scenario, we can merge on his signal.

[stepanblyschak]

@yxieca We are good to merge it.

[mssonicbld]

Cherry-pick PR to 202305: #16046