Changes to support config-setup service for multi-npu #4609

abdosi · 2020-05-15T22:24:01Z

- Why I did it
Changes to support config-setup service for multi-npu platforms.
we are using python commands config reload/save/load-minigraph
so that there is common code. when being called from this script
we will not stop/start docker services. This PR is related to this:
sonic-net/sonic-utilities#919
For Multi-npu we are not supporting as of
now config initialization and ZTP. It will support creating
config db from minigraph if present or using config db from previous
file system

Submodule update PR for sonic-utilities is : #4622
Signed-off-by: Abhishek Dosi abdosi@microsoft.com

- How I did it
Updated the service to handle multiple config db in case of multi-npu platforms

- How to verify it
Manual testing by running /usr/bin/config-setup on both single and multi-npu
platforms.

platforms. For Multi-npu we are not supporting as of now config initializtion and ZTP. It will support creating config db from minigraph or using config db from previous file system Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

files/image_config/config-setup/config-setup

yxieca

Suggested some minor changes. none of them are blocking.

lguohan · 2020-05-18T18:45:56Z

files/image_config/config-setup/config-setup

@@ -99,28 +103,119 @@ run_hookdir() {
    return $exit_status
 }

+initalize_configdb()


can instead call config reload -y instead of implementing here? it is code duplication. it can cause code maintain problem.

@lguohan I was thinking other way when making change. config reload/save command does stop/start services which is not needed during config-setup script.
We can update those commands to call config-setup shell script with maintaining stop/start of services.
That way code will be present only in config-setup for config db generation.

the reason the other way is preferred is that we can add more unit tests for the python script, but not bash script.

@lguohan. Python script will still exist but internally will call shell script command. Here is the reference PR: sonic-net/sonic-utilities#919

@lguohan Updated to use python commands.

files/image_config/config-setup/config-setup

yxieca · 2020-05-18T20:53:30Z

You have a mixture of [...] and [[...]]. Suggest unify to [[....]]

abdosi · 2020-05-18T22:07:22Z

You have a mixture of [...] and [[...]]. Suggest unify to [[....]]

@yxieca Done for new changes. However there are many places where we have this mismatch so we might need to revisit for complete update.

config save/config reload from shell scripts so that we don't duplicate code. Also while running from shell we will skip stop/start services done by those commands.

abdosi · 2020-05-20T16:28:28Z

retest vsimage please

abdosi · 2020-05-20T16:29:09Z

retest mellanox please

abdosi · 2020-05-20T17:47:38Z

@lguohan Can you please review the PR.

* Changes to support config-setup service for multi-npu platforms. For Multi-npu we are not supporting as of now config initializtion and ZTP. It will support creating config db from minigraph or using config db from previous file system Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Address Review Comments. * Address Review comments * Address Review Comments of using pyhton based config load_minigraph/ config save/config reload from shell scripts so that we don't duplicate code. Also while running from shell we will skip stop/start services done by those commands. * Updated to use python command so no code duplication.

* [201911][devices] skip_fancontrol for wedge 100 barefoot platforms (sonic-net#4528) * [device] DellEMC s5232f 50G hwsku support (sonic-net#4525) * [device] DellEmc S5232 support for new hwsku C8D48 8 100G ports and 48 50G ports * 10G ports update for S5232 hwsku-C8D48 Signed-off-by: Srideep Devireddy <srideep_devireddy@dell.com> * DellEMC S6000 updated sensors.conf (sonic-net#4568) Change PSU MAX temperature to 80 degree Change tmp75 sensors default temperature value from 25/50 to 70/80 degree. * [sonic-slave-stretch]: install same version for docker-ce and docker-ce-cli difference versions can cause compatibility issue between the server and client Signed-off-by: Guohan Lu <lguohan@gmail.com> * [baseimage]: install same version for docker-ce and docker-ce-cli Signed-off-by: Guohan Lu <lguohan@gmail.com> * [FRR]: Update frr to latest 7.2.1-s3 (sonic-net#4294) - Updated to latest frr 7.2.1 from the master. - Updated patches accordingly * [sonic-buildimage] updated minigraph for ACL Table data and ACL Interface Binding for Multi-NPU platforms (sonic-net#4491) * [sonic-buildimage] updated minigraph for ACL Table data and ACL Interface binding update for multu-npu platform based on subrole as "Frontend" or "Backend". For backend npu no ACL table is associated. For frontend npu only front-panel interface are associated. Updated with test case and fix typo in sample-mingraph for npu Address Review comments Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Fixed the logic as per preview comment. Interface Filter logic only applies to Everflow/Mirror tables. * Address Review Comments. * Changes for LLDP docker to support multi-npu platforms (sonic-net#4530) * Changes for LLDP for Multi NPU Platoforms:- a) Enable LLDP for Host namespace for Management Port b) Make sure Management IP is avaliable in per asic namespace needed for LLDP Chassis configuration c) Make sure chassis mac-address is correct in per asic namespace d) Do not run lldp on eth0 of per asic namespace and avoid chassis configuration for same e) Use Linux hostname instead from Device Metadata for lldp chassis configuration since in multi-npu platforms device metadata hostname will be differnt Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Address Review Comment with following changes: a) Use Device Metadata hostname even in per namespace conatiner. updated minigraph parsing for same to have hostname as system hostname and add new key for asic name b) Minigraph changes to have MGMT_INTERFACE Key in per asic/namespace config also as needed for LLDP for setting chassis management IP. Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Address Review Comments * Moved utility functions for multi-npu platforms from sonic-utilities to sonic_device_util.py (sonic-net#4559) * Moved utility functions for multi-npu platforms from sonic-utilities config/main.py to here so that they can be used any module * Fix the issue with test run during compilation with acl-uploader PR#908 of sonic-utilities. * Fix get_num_npu as it was retuning string and not int * Address Review Comments * Address Review Comments * Fix for issue where image is compile with flag ENABLE_DHCP_GRAPH_SERVICE (sonic-net#4573) and then we load image and reboot even if there was existing config_db.json we will look for DHCP Service. we should disbale update_graph in such cases. This behaviour is silimar to what we have in 201811 image. * Change to enable redistribute connected on Frontend asics instead of backend asics (sonic-net#4588) Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com> * [DellEMC] S6000 Disable Low power mode by default (sonic-net#4592) * [BFN] Updated Barefoot SDK to 2020-05-07 (sonic-net#4566) Signed-off-by: Andriy Kokhan <akokhan@barefootnetworks.com> * [minigraph] Add tags for egress mirror tables (sonic-net#4526) Signed-off-by: Danny Allen <daall@microsoft.com> * [Submodule update] sonic-utlities with PR's [201911][show] Fix abbreviations for 'show ip bgp ...' commands (sonic-net#909) Changes to support acl-loader and mirror-session config commands for multi-npu platforms. (sonic-net#908) Changes to commands config reload/load-minigraph (sonic-net#919) Stop/Start restapi server upon config reload (sonic-net#911) [config] Add 'interface transceiver' subgroup with 'lpmode' and 'reset' subcommands (sonic-net#904) * [minigraph] Support FECDisabled in minigraph parser (sonic-net#4556) (sonic-net#4624) Signed-off-by: Qi Luo <qiluo-msft@users.noreply.github.com> * [ntp] enable/disable NTP long jump according to reboot type (sonic-net#4577) * [ntp] enable/disable NTP long jump according to reboot type - Enable NTP long jump after cold reboot. - Disable NTP long jump after warrm/fast reboot. Signed-off-by: Ying Xie <ying.xie@microsoft.com> * fix typo * further refactoring * use sonic-db-cli instead * [arista]: remove the soc property disabling sram scan (sonic-net#4623) * Changes to support config-setup service for multi-npu (sonic-net#4609) * Changes to support config-setup service for multi-npu platforms. For Multi-npu we are not supporting as of now config initializtion and ZTP. It will support creating config db from minigraph or using config db from previous file system Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Address Review Comments. * Address Review comments * Address Review Comments of using pyhton based config load_minigraph/ config save/config reload from shell scripts so that we don't duplicate code. Also while running from shell we will skip stop/start services done by those commands. * Updated to use python command so no code duplication. * [config]: Fix the device type and internal bgp session status for multi NPU platforms (sonic-net#4600) * The following changes for multi-npu platforms are done - Set the type in device_metadata for asic configuration to be same as host - Set the admin-status of internal bgp sessions as up Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com> * Adding new BGP peer groups PEER_V4_INT and PEER_V6_INT. (sonic-net#4620) * Adding new BGP peer groups PEER_V4_INT and PEER_V6_INT. The internal BGP sessions will be added to this peer group while the external BGP sessions will be added to the exising PEER_V4 and PEER_V6 peer group. * Check for "ASIC" keyword in the hostname to identify the internal neighbors. * [submodule update] sonic-swss with PR [vnet] Fix IP2ME route creation logic for BITMAP VNET interface (sonic-net#1284) * [submodule update] sonic-util Revert "[config] Add 'interface transceiver' subgroup with 'lpmode' and 'reset' subcommands (sonic-net#904)" Multi-asic changes for config bgp commands and utilities. (sonic-net#910) * [submodule update] sonic-rest API's PR#39 Setup module versioning Add support for get all Vlans (#37) * Update golang version for 1.11.5 to 1.14.2 (sonic-net#4520) Co-authored-by: Myron Sosyak <49795530+msosyak@users.noreply.github.com> Co-authored-by: Srideep <srideep_devireddy@dell.com> Co-authored-by: paavaanan <paavaanan_t_n@dell.com> Co-authored-by: Guohan Lu <lguohan@gmail.com> Co-authored-by: pavel-shirshov <pavelsh@microsoft.com> Co-authored-by: abdosi <58047199+abdosi@users.noreply.github.com> Co-authored-by: arlakshm <55814491+arlakshm@users.noreply.github.com> Co-authored-by: Santhosh Kumar T <53558409+santhosh-kt@users.noreply.github.com> Co-authored-by: Andriy Kokhan <43479230+akokhan@users.noreply.github.com> Co-authored-by: Danny Allen <daall@microsoft.com> Co-authored-by: Abhishek Dosi <abdosi@microsoft.com> Co-authored-by: Qi Luo <qiluo-msft@users.noreply.github.com> Co-authored-by: Ying Xie <yxieca@users.noreply.github.com> Co-authored-by: Samuel Angebault <staphylo@arista.com> Co-authored-by: judyjoseph <53951155+judyjoseph@users.noreply.github.com>

abdosi force-pushed the multi-npu-config-setup branch 2 times, most recently from 123d37b to dcf433a Compare May 15, 2020 22:36

abdosi requested review from lguohan, jleveque, judyjoseph and arlakshm May 15, 2020 22:37

abdosi added Multi-NPU 🔢 Request for 201911 Branch labels May 15, 2020

abdosi requested a review from yxieca May 15, 2020 23:00

abdosi force-pushed the multi-npu-config-setup branch from dcf433a to 215cb7b Compare May 16, 2020 21:05

yxieca reviewed May 17, 2020

View reviewed changes

files/image_config/config-setup/config-setup Outdated Show resolved Hide resolved

yxieca reviewed May 17, 2020

View reviewed changes

files/image_config/config-setup/config-setup Outdated Show resolved Hide resolved

yxieca reviewed May 17, 2020

View reviewed changes

files/image_config/config-setup/config-setup Outdated Show resolved Hide resolved

yxieca reviewed May 17, 2020

View reviewed changes

files/image_config/config-setup/config-setup Outdated Show resolved Hide resolved

yxieca reviewed May 17, 2020

View reviewed changes

files/image_config/config-setup/config-setup Outdated Show resolved Hide resolved

yxieca previously approved these changes May 17, 2020

View reviewed changes

Address Review Comments.

a4c8ea8

abdosi dismissed yxieca’s stale review via a4c8ea8 May 18, 2020 16:52

lguohan reviewed May 18, 2020

View reviewed changes

judyjoseph reviewed May 18, 2020

View reviewed changes

files/image_config/config-setup/config-setup Show resolved Hide resolved

judyjoseph reviewed May 18, 2020

View reviewed changes

files/image_config/config-setup/config-setup Outdated Show resolved Hide resolved

judyjoseph reviewed May 18, 2020

View reviewed changes

files/image_config/config-setup/config-setup Show resolved Hide resolved

Address Review comments

d0f07b0

Address Review Comments of using pyhton based config load_minigraph/

2a64192

config save/config reload from shell scripts so that we don't duplicate code. Also while running from shell we will skip stop/start services done by those commands.

abdosi force-pushed the multi-npu-config-setup branch from 8e21b83 to 2a64192 Compare May 20, 2020 00:14

abdosi mentioned this pull request May 20, 2020

Changes to commands config reload/load-minigraph sonic-net/sonic-utilities#919

Merged

Updated to use python command so no code duplication.

1bcafef

lguohan approved these changes May 20, 2020

View reviewed changes

abdosi merged commit a44fc07 into sonic-net:master May 20, 2020

abdosi deleted the multi-npu-config-setup branch May 20, 2020 23:32

abdosi added the Included in 201911 Branch label May 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Changes to support config-setup service for multi-npu #4609

Changes to support config-setup service for multi-npu #4609

abdosi commented May 15, 2020 •

edited

yxieca left a comment •

edited

lguohan May 18, 2020 •

edited

abdosi May 18, 2020

lguohan May 19, 2020

abdosi May 19, 2020

abdosi May 20, 2020 •

edited

yxieca commented May 18, 2020

abdosi commented May 18, 2020

abdosi commented May 20, 2020

abdosi commented May 20, 2020

abdosi commented May 20, 2020

Changes to support config-setup service for multi-npu #4609

Changes to support config-setup service for multi-npu #4609

Conversation

abdosi commented May 15, 2020 • edited

yxieca left a comment • edited

Choose a reason for hiding this comment

lguohan May 18, 2020 • edited

Choose a reason for hiding this comment

abdosi May 18, 2020

Choose a reason for hiding this comment

lguohan May 19, 2020

Choose a reason for hiding this comment

abdosi May 19, 2020

Choose a reason for hiding this comment

abdosi May 20, 2020 • edited

Choose a reason for hiding this comment

yxieca commented May 18, 2020

abdosi commented May 18, 2020

abdosi commented May 20, 2020

abdosi commented May 20, 2020

abdosi commented May 20, 2020

abdosi commented May 15, 2020 •

edited

yxieca left a comment •

edited

lguohan May 18, 2020 •

edited

abdosi May 20, 2020 •

edited