[DHCP Relay]: Support Multiple VLANs (Separate DHCP Relay Agents, One Per VLAN) #999

jleveque · 2017-09-28T23:54:32Z

- What I did

Spawn one DHCP relay agent for each VLAN that is configured as requiring one or more DHCP servers.
Also resolves Failed DHCP relay test #985

- How I did it

docker-dhcp-relay entrypoint is a new script, docker_init.sh, which is responsible for generating the supervisord configuration file dynamically from a Jinja2 template. docker_init.sh then starts supervisord via exec so that supervisord still runs as PID 1
Dynamically-configured supervisord config now supports spawning a variable number of DHCP relay agent processes, based on VLAN configuration
Due to above changes, we no longer rely on /etc/default/isc-dhcp-relay config file, so the related Jinja2 template and shell script have been removed
Update isc-dhcp-relay patch such that it no longer relies on calling sonic-cfggen to retrieve port aliases with each packet (slow). Now parses the aliases from port_config.ini file directly once at startup and loads them into memory
Prevent relaying of already-relayed packets using -m discard parameter
Change name of supervisord config file and template from generic supervisord.conf to more specific (albeit long) docker-dhcp-relay.supervisord.conf for better comprehension and also to avoid conflicts when copying to /usr/share/sonic/templates if we need to templatize other supervisord config files. We may also want to consider creating subdirectories under /usr/share/sonic/templates to further avoid other conflicts.
Update minigraph.py parser to parse out new <DhcpRelays> tag
Update sonic-cfggen test T0 minigraphs to contain new <DhcpRelays> tag
Remove executable permission and extra trailing newline from wait_for_intf.sh.j2 template file

- How to verify it

Generate a T0 minigraph with two or more VLANs, each configured with one or more DHCP servers (using the <DhcpRelays> tag.
Boot up SONiC using the minigraph created in step 1.
Verify that there are multiple dhcrelay processes running, one per VLAN as configured in step 1.
Verify that DHCP requests from devices on VLANs under the SONiC T0 are forwarded to all the DHCP servers configured in the minigraph, and verify all responses from DHCP servers are forwarded back to the appropriate device under the SONiC T0.

- Description for the changelog

[DHCP Relay]: Support Multiple VLANs (Separate DHCP Relay Agents, One Per VLAN)

- A picture of a cute animal (not mandatory but encouraged)

       _                        
       \`*-.                    
        )  _`-.                 
       .  : `. .                
       : _   '  \               
       ; *` _.   `*-._          
       `-.-'          `-.       
         ;       `       `.     
         :.       .        \    
         . \  .   :   .-'   .   
         '  `+.;  ;  '      :   
         :  '  |    ;       ;-. 
         ; '   : :`-:     _.`* ;
[bug] .*' /  .*' ; .*`- +'  `*' 
      `*-*   `*-*  `*-*'

taoyl-ms · 2017-09-28T23:59:29Z

src/sonic-config-engine/tests/pc-test-graph.xml

@@ -108,6 +108,7 @@
          <NoDhcpRelay>False</NoDhcpRelay>
          <StaticDHCPRelay>0.0.0.0/0</StaticDHCPRelay>
          <Type i:nil="true"/>
+	  <DhcpRelays>192.0.0.1;192.0.0.2</DhcpRelays>


Seems that indenting does not match here?

Thanks. Looks like I wound up with tabs instead of spaces in all of the test minigraphs for some reason.

taoyl-ms · 2017-09-29T00:02:36Z

dockers/docker-dhcp-relay/docker_init.sh

+
+# Generate supervisord config file
+mkdir -p /etc/supervisor/conf.d/
+sonic-cfggen -m /etc/sonic/minigraph.xml -t /usr/share/sonic/templates/docker-dhcp-relay.supervisord.conf.j2 > /etc/supervisor/conf.d/docker-dhcp-relay.supervisord.conf


"-d" instead of "-m /etc/sonic/minigraph.xml"

taoyl-ms · 2017-09-29T00:07:23Z

dockers/docker-dhcp-relay/docker-dhcp-relay.supervisord.conf.j2

+
+{# If our configuration has VLANs... #}
+{% if VLAN %}
+[group:isc-dhcp-relay]


If there are VLANs but non of them contains a "dhcp_server" attribute, there will be a [group: isc-dhcp-relay] with no programs specified. Will that work?

It does work, but it isn't the cleanest solution.

I could iterate all VLANs first and check whether any of them require a DHCP relay, but that adds more complexity to what is already a fairly complex template.

Well, it required 8 lines of code and 2 lines of comments to implement the check. I think it's still fairly readable, so I made the change, but I wish it could be simpler.

taoyl-ms · 2017-09-29T00:10:28Z

Does this mean DHCP_Server field in minigraph - device metadata will no longer be used?

jleveque · 2017-09-29T00:29:28Z

@taoyl-ms: Are you referring to the DhcpResources field?

taoyl-ms · 2017-09-29T00:32:02Z

Ah Yes.

jleveque · 2017-09-29T01:00:51Z

@taoyl-ms: The DhcpResources field is global for the device. The new <DhcpRelays> tag is a child of each VLAN, so this tag should be used to specify DHCP servers for each VLAN.

lguohan · 2017-09-30T18:31:45Z

src/isc-dhcp/isc-dhcp-4.3.1_dhcrelay-custom-circuit_id-remote_id-and-bridge-iface-support.patch

+-			       inet_ntoa(to.sin_addr));
+			//log_debug("Forwarded BOOTREPLY for %s to %s",
+			//          print_hw_addr(packet->htype, packet->hlen, packet->chaddr),
+			//          inet_ntoa(to.sin_addr));


remove the commented lines?

I would have liked to leave them uncommented, but unfortunately, the behavior of log_debug() is to print to stdout, and we have configured supervisord to forward stdout messages to the syslog, so leaving these log_debug() calls in spams the syslog.

I considered removing these altogether, but they may come in handy for future debugging, so I thought leaving them commented out is better for posterity's sake than having to rewrite them again, even if it leaves a bit of clutter.

If you still prefer I remove them, I will. Just let me know.

lguohan · 2017-09-30T18:32:09Z

src/isc-dhcp/isc-dhcp-4.3.1_dhcrelay-custom-circuit_id-remote_id-and-bridge-iface-support.patch

+-			       inet_ntoa(sp->to.sin_addr));
+			//log_debug("Forwarded BOOTREQUEST for %s to %s",
+			//          print_hw_addr(packet->htype, packet->hlen, packet->chaddr),
+			//          inet_ntoa(sp->to.sin_addr));


remove commented lines?

lguohan · 2017-09-30T18:45:16Z

dockers/docker-dhcp-relay/docker-dhcp-relay.supervisord.conf.j2

+{% for vlan_name in VLAN -%}
+{%- if VLAN[vlan_name]['dhcp_servers'] -%}
+[program:isc-dhcp-relay-{{ vlan_name }}]
+command=/usr/sbin/dhcrelay -d -c 1 -a %%h:%%p %%P -i {{ vlan_name }}


/usr/sbin/dhcrelay -d -c 1 [](start = 8, length = 26)

we are going to forward the already relayed packet?

-m append|replace|forward|discard ?

how do you control the behavior for already relayed packet?

In reply to: 142012491 [](ancestors = 142012491)

I am preventing the relay of previously-relayed packets by limiting the max-hop-count to 1 (-c 1), under the assumption that we should only ever relay requests from devices immediately below the ToR (i.e., hop count of 0). If this assumption is incorrect, I will investigate the behavior of the -m flag.

lguohan · 2017-09-30T18:54:28Z

src/isc-dhcp/isc-dhcp-4.3.1_dhcrelay-custom-circuit_id-remote_id-and-bridge-iface-support.patch

+	int i = 0;
+	FILE *fp = NULL;
+	char line[MAX_PORT_CONFIG_LINE_LEN] = { 0 };
+	char port_config_file_path[] = "/usr/share/sonic/hwsku/port_config.ini";


it is better to give an option to specify which file to load. Also, you do not need to load port_config.ini file, you can use cfggen to generate your own port mapping file and load that port mapping file. In this case, we do not need to parse the port_config.ini, you can rely on cfggen's logic to get the correct port name mapping.

I considered that solution, but felt as though writing the output from cfggen to a file and reading that file back in was a bit excessive. However, it does prevent the need for maintaining a separate port_config.ini parser here.

That said, with my current solution, if the format of port_config.ini were to change, this parser would have to be updated. However, if we go with the cfggen solution you suggested and the cfggen output format ever changes, we would need to update the parsing of the cfggen output and the writing of the port mapping file here. I guess the decision comes down to the question: which format is less likely to change over time?

Done. Now parsing output from sonic-cfggen and writing to a file in docker_init.sh, then isc-dhcp-relay reads that file at startup. Also added file path as a parameter to isc-dhcp-relay.

lguohan

as comments.

…e VLANs

… agent -- only parse if present

…fied DHCP servers

…e no longer generate that file

…s maps into memory once at start instead of calling sonic-cfggen on each packet we relay

…h a hop count of zero

… template

…n lieu of parsing port_config.ini directly

…h already contain agent info

lguohan

jleveque · 2017-10-05T00:54:26Z

Retest this please.

* msft_github/master: [DHCP Relay]: Support Multiple VLANs (Separate DHCP Relay Agents, One Per VLAN) (sonic-net#999) [build]: sonic-utilities package depends on swsssdk; build as wheel and add build dependency (sonic-net#1011) Make swssconfig status FATAL when it fails (sonic-net#1009) [swss]: Update swss-common/sairedis/swss submodules (sonic-net#1008) [config-engine]: Fix bug multiple ports connecting to same neighbor (sonic-net#1005)

Common functions for show CLI support on multi ASIC (#999)

* src/sonic-utilities d5fdd74...17fb378 (7): > [sonic-installer] Import re module (#1061) > [fast-reboot]: Fix fail to execute fast-reboot problem (#1047) > [config] Reduce Calls to SONiC Cfggen (#1052) > [filter-fdb] Call Filter FDB Main From Within Test Code (#1051) > [sflow_test.py]: Fix show sflow display. (#1054) > Change fast-reboot script to use swss and radv service script (#1036) > Common functions for show CLI support on multi ASIC (#999)

* [BFN] Add support pcied daemon for Montara and Newport (sonic-net#5199) Signed-off-by: Petro Bratash <petrox.bratash@intel.com> * [cfggen] Allow Write To Redis DB With Template/Batch Mode (sonic-net#5203) Argument to write to config-db is not allowed when using template. This PR allows cfggen to write to redis db when using template mode. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [submodule]: Advance sonic-snmpagent. (sonic-net#5213) Update sonic-snmpagent submodule to include below commits: 1a2b62a [Namespace]: Fix SAI_ID key used in cpfcIfTable and csqIfQosGroupStatsTable implementation (sonic-net#138) d06f00c [pytest/coverage]: add coverage support (sonic-net#156) 90e9f2e [Namespace]: Simplify sync_d functions to use higher order (sonic-net#154) b5815d9 [LLDP]: Modify OID index of LLDPRemTableUpdater MIB (sonic-net#155) d5f2b92 [Multiasic]: Provide namespace support for ipNetToMediaPhysAddress (sonic-net#129) 166c221 [Namespace]: Fix interface counters in RFC 1213 (sonic-net#145) Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com> * [cfggen] Conform With Python 3 Syntax (sonic-net#5154) Preparing sonic-cfggen for migration to Python 3. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [redis-dump-load] Update submodule (sonic-net#5215) * src/redis-dump-load 832a645...7585497 (2): > Merge pull request sonic-net#63 from jleveque/update_gitignore > Merge pull request sonic-net#59 from breser/redis-load-empty * [services] Fix Delay Start of SNMP And Telemetry (sonic-net#5211) SNMP and Telemetry services are not critical to switch startup. They also cause fast-reboot not to meet timing requirements. In order to delay start those service are associated with systemd timer units, however when hostcfgd initiate service start, it start the service and not the timer. This PR fixes this issue by starting the timer associated with systemd unit. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [sonic-py-common][multi ASIC] API to get a list of frontend ports (sonic-net#5221) * [sonic-py-common][multi ASIC] utility to get a list of frontend ports from a given list of ports * [sonic-config-engine] Update .gitignore (sonic-net#5223) - Ignore directories generated by building Python wheel package - Move all sonic-config-engine ignores from the root .gitignore to src/sonic-config-engine/.gitignore * Advance swss-common submodule. (sonic-net#5222) 9a7c9d Dbconnector namespace support (sonic-net#376) c32f0b5 add state db entry for fgnhg route entry (sonic-net#374) * [caclmgrd] Add support for multi-ASIC platforms (sonic-net#5022) * Support for Control Plane ACL's for Multi-asic Platforms. Following changes were done: 1) Moved from using blocking listen() on Config DB to the select() model via python-swsscommon since we have to wait on event from multiple config db's 2) Since python-swsscommon is not available on host added libswsscommon and python-swsscommon and dependent packages in the base image (host enviroment) 3) Made iptables programmed in all namespace using ip netns exec Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Address Review Comments Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Fix Review Comments * Fix Comments * Added Change for Multi-asic to have iptables rules to accept internal docker tcp/udp traffic needed for syslog and redis-tcp connection. Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Fix Review Comments * Added more comments on logic. * Fixed all warning/errors reported by http://pep8online.com/ other than line > 80 characters. * Fix Comment Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Verified with swsscommon package. Fix issue for single asic platforms. * Moved to new python package * Address Review Comments. Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Address Review Comments. * Add support to VS platform for platform.json and DPB CLI Tests (sonic-net#5192) - Reverts commit 457674c - Creates "platform.json" for vs docker - Adds test case for port breakout CLI - Explicitly sets admin status of all the VS interfaces to down to be compatible with SWSS test cases, specifically vnet tests and sflow tests Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com> * [iccpd] Fix uninitialized variable. (sonic-net#5112) To declare *tb[] but do not initialize it, it might be very risky. We get iccpd exception during processing arp/nd event. Initialize it to {0}; * Fix unwanted python exception in syslog during database container (sonic-net#5227) startup when doing redis PING since database_config.json getting generated from jinja2 template is still not ready. Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * [hostcfgd] Handle Both Service And Timer Units (sonic-net#5228) Commit e484ae9 introduced systemd .timer unit to hostcfgd. However, when stopping service that has timer, there is possibility that timer is not running and the service would not be stopped. This PR address this situation by handling both .timer and .service units. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [arista] Update driver submodules (sonic-net#5147) - fix watchdog timeout units - fix import path for thermal_manager - remove arista bind mounts for docker-snmp - improve arista bind mounts for pmon * [docker-radv] Fix startup issues (sonic-net#5230) **- Why I did it** PR sonic-net#4599 introduced two bugs in the startup of the router advertiser container: 1. References to the `wait_for_intf.sh` script were changed to `wait_for_link.sh`, but the actual script was not renamed 2. The `ipv6_found` Jinja2 variable added to the supervisor config file goes out of scope before it is read. **- How I did it** 1. Rename the `wait_for_intf.sh` script to `wait_for_link.sh` 2. Use the Jinja2 "namespace" construct to fix the scope issue **- How to verify it** Ensure all processes in the radv container start properly under the correct conditions (i.e., whether or not there is at least one VLAN with an IPv6 address assigned). * [sonic-utilities] Update submodule (sonic-net#5233) * src/sonic-utilities d5fdd74...17fb378 (7): > [sonic-installer] Import re module (sonic-net#1061) > [fast-reboot]: Fix fail to execute fast-reboot problem (sonic-net#1047) > [config] Reduce Calls to SONiC Cfggen (sonic-net#1052) > [filter-fdb] Call Filter FDB Main From Within Test Code (sonic-net#1051) > [sflow_test.py]: Fix show sflow display. (sonic-net#1054) > Change fast-reboot script to use swss and radv service script (sonic-net#1036) > Common functions for show CLI support on multi ASIC (sonic-net#999) * [sonic-host-service]: Add SONiC Host Services infrastructure (sonic-net#4840) - Why I did it When SONiC is configured with the management framework and/or telemetry services, the applications running inside those containers need to access some functionality on the host system. The following is a non-exhaustive list of such functionality: Image management Configuration save and load ZTP enable/disable and status Show tech support - How I did it The host service is a Python process that listens for requests via D-Bus. It will then service those requests and send a response back to the requestor. This PR only introduces the host service infrastructure. Applications that need access to the host services must add applets that will register on D-Bus endpoints to service the appropriate functionality. - How to verify it - Description for the changelog Add SONiC Host Service for container to execute select commands in host Signed-off-by: Nirenjan Krishnan <Nirenjan.Krishnan@dell.com> * Add common functions applicable to single/multi asic platforms (sonic-net#5224) * Add common functions applicable to single/multi asic platforms * Raise exception if invalid namespace is given as input. * [sonic-swss] Update submodule (sonic-net#5231) * src/sonic-swss d2bab10...c4949a2 (34): > [dvs] Add new common issues and TOC to DVS README (sonic-net#1405) > Avoid adding loopback interface (ip link add) when setting nat zone on loopback interface (sonic-net#1411) > [portsorch] add buffer drop FC group (sonic-net#1368) > [dvs/chassis] Bring up SONiC interfaces in virtual chassis (sonic-net#1410) > [chassis/dvs] Add support for virtual chassis to DVS testbed (sonic-net#1345) > [sonic-swsss] Fix the issue of field "next_hop_ip" not getting updated in state DB in ERSPAN Mirror (sonic-net#1375) > [intfmgr] Fix OA crash issue due to link local configurations (sonic-net#1195) > Fix the issue when persistent DVS is used to run pytest which has number of front-panel ports < 32 (sonic-net#1373) > [dvs] Refactor AsicDbValidator (sonic-net#1402) > [fec] Get FEC mode when port is already admin down (sonic-net#1403) > [fec] added logic that put port down before applying fec onfiguration (sonic-net#1399) > [dvs] Add performance test for adding and deleting routes (sonic-net#1392) > Ignore IPv6 link-local and multicast entries as Vnet routes (sonic-net#1401) > [vlanmgr] Support Jumbo Frame By Default (sonic-net#1393) > Fix log/syslog not being correct when last test fails for given module (sonic-net#1395) > Get initial speed from ASIC DB (sonic-net#1390) > [dvs] Add options to limit CPU usage (sonic-net#1394) > [intfsorch] Retrieve Port object before setting NAT zone on router interfaces. (sonic-net#1372) > [.gitignore] Ignore gearsyncd binary (sonic-net#1381) > Added Max Nexthopgroup/ECMP Count supported by device into State DB. (sonic-net#1383) > [dvs] Upload logs even if failure occurs during startup (sonic-net#1389) > [rates] fix issue with rates init (sonic-net#1387) > [dvs] Validate that SWSS is ready to receive input before starting tests (sonic-net#1385) > [dvs] Convert sflow and speed tests to use dvslib (sonic-net#1382) > [dvs_acl] Refactor and document dvs_acl library (sonic-net#1378) > [dvs] Fix install instructions in README (sonic-net#1379) > [dvs] Update README with new flags, options, and known issues (sonic-net#1380) > swss: gearsyncd should return 0 on exit (sonic-net#1376) > Remove 00-copp.config.json from swss debian package. (sonic-net#1366) > fix undefined var in rates lua scripts (sonic-net#1365) > [fdborch] Fixed Orchagent crash in FDB flush on port disable. (sonic-net#1369) > [tlm_teamd]: Try to add LAG again, when teamd is not ready first time (sonic-net#1347) > [vs] Incorporate python3 best practices into DVSLib (sonic-net#1357) > [dvs] Mark unstable tests as xfail (sonic-net#1356) * [arista/aboot]: Zero out 1st MB before repartitioning (sonic-net#5220) The first partition starting point was changed to be 1M as part of this commit: 6ba2f97. On systems that are misaligned before conversion (partition start is the first sector), the relica partition that is left in the first MB can cause problems in Aboot and result in corruption of the filesystem on the new aligned partition. Zeroing this old relica makes sure that there is nothing left of the old partition lying around. There won't be any risk of having Aboot corrupt the new filesystem because of the old relica. Signed-off-by: Baptiste Covolato <baptiste@arista.com> * [sonic-py-common] Add unit test framework (sonic-net#5238) **- Why I did it** To install the framework for adding unit tests to the sonic-py-common package and report coverage. ** How I did it ** - Incorporate pytest and pytest-cov into sonic-py-common package build - Updgrade version of 'mock' installed to version 3.0.5, the last version which supports Python 2. This fixes a bug where the file object returned from `mock_open()` was not iterable (see https://bugs.python.org/issue32933) - Add support for Python 3 setuptools and pytest in sonic-slave-buster environment - Add tests for `device_info.get_machine_info()` and `device_info.get_platform()` functions - Also add a .gitignore in the root of the sonic-py-common directory, move all related ignores from main .gitignore file, and add ignores for files and dirs generated by pytest-cov * Add switch for synchronous mode (sonic-net#5237) Add a master switch so that the sync/async mode can be configured. Example usage of the switch: 1. Configure mode while building an image `make ENABLE_SYNCHRONOUS_MODE=y <target>` 2. Configure when the device is running Change CONFIG_DB with `sonic-cfggen -a '{"DEVICE_METADATA":{"localhost": {"synchronous_mode": "enable"}}}' --write-to-db` Restart swss with `systemctl restart swss` * [enable counters] Enable port buffer drops by default and update MLNX SAI submodule (sonic-net#5059) * Enable port buffer drops by default * [Mellanox] Update SAI_Implementation Signed-off-by: Mykola Faryma <mykolaf@mellanox.com> * Platform monitor changes in daemon_base for multi_asic (sonic-net#4932) Adding namespace support for db connect API. Co-authored-by: Petro Bratash <68950226+bratashX@users.noreply.github.com> Co-authored-by: Tamer Ahmed <tamer.ahmed@microsoft.com> Co-authored-by: SuvarnaMeenakshi <50386592+SuvarnaMeenakshi@users.noreply.github.com> Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com> Co-authored-by: Mahesh Maddikayala <10645050+smaheshm@users.noreply.github.com> Co-authored-by: judyjoseph <53951155+judyjoseph@users.noreply.github.com> Co-authored-by: abdosi <58047199+abdosi@users.noreply.github.com> Co-authored-by: Sangita Maity <sangitamaity0211@gmail.com> Co-authored-by: Kelly Chen <kelly_chen@edge-core.com> Co-authored-by: Samuel Angebault <staphylo@arista.com> Co-authored-by: nirenjan <nirenjan@users.noreply.github.com> Co-authored-by: Baptiste Covolato <b.covolato@gmail.com> Co-authored-by: shi-su <67605788+shi-su@users.noreply.github.com> Co-authored-by: Mykola F <37578614+mykolaf@users.noreply.github.com>

* [BFN] Add support pcied daemon for Montara and Newport (sonic-net#5199) Signed-off-by: Petro Bratash <petrox.bratash@intel.com> * [cfggen] Allow Write To Redis DB With Template/Batch Mode (sonic-net#5203) Argument to write to config-db is not allowed when using template. This PR allows cfggen to write to redis db when using template mode. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [submodule]: Advance sonic-snmpagent. (sonic-net#5213) Update sonic-snmpagent submodule to include below commits: 1a2b62a [Namespace]: Fix SAI_ID key used in cpfcIfTable and csqIfQosGroupStatsTable implementation (sonic-net#138) d06f00c [pytest/coverage]: add coverage support (sonic-net#156) 90e9f2e [Namespace]: Simplify sync_d functions to use higher order (sonic-net#154) b5815d9 [LLDP]: Modify OID index of LLDPRemTableUpdater MIB (sonic-net#155) d5f2b92 [Multiasic]: Provide namespace support for ipNetToMediaPhysAddress (sonic-net#129) 166c221 [Namespace]: Fix interface counters in RFC 1213 (sonic-net#145) Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com> * [cfggen] Conform With Python 3 Syntax (sonic-net#5154) Preparing sonic-cfggen for migration to Python 3. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [redis-dump-load] Update submodule (sonic-net#5215) * src/redis-dump-load 832a645...7585497 (2): > Merge pull request sonic-net#63 from jleveque/update_gitignore > Merge pull request sonic-net#59 from breser/redis-load-empty * [services] Fix Delay Start of SNMP And Telemetry (sonic-net#5211) SNMP and Telemetry services are not critical to switch startup. They also cause fast-reboot not to meet timing requirements. In order to delay start those service are associated with systemd timer units, however when hostcfgd initiate service start, it start the service and not the timer. This PR fixes this issue by starting the timer associated with systemd unit. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [sonic-py-common][multi ASIC] API to get a list of frontend ports (sonic-net#5221) * [sonic-py-common][multi ASIC] utility to get a list of frontend ports from a given list of ports * [sonic-config-engine] Update .gitignore (sonic-net#5223) - Ignore directories generated by building Python wheel package - Move all sonic-config-engine ignores from the root .gitignore to src/sonic-config-engine/.gitignore * Advance swss-common submodule. (sonic-net#5222) 9a7c9d Dbconnector namespace support (sonic-net#376) c32f0b5 add state db entry for fgnhg route entry (sonic-net#374) * [caclmgrd] Add support for multi-ASIC platforms (sonic-net#5022) * Support for Control Plane ACL's for Multi-asic Platforms. Following changes were done: 1) Moved from using blocking listen() on Config DB to the select() model via python-swsscommon since we have to wait on event from multiple config db's 2) Since python-swsscommon is not available on host added libswsscommon and python-swsscommon and dependent packages in the base image (host enviroment) 3) Made iptables programmed in all namespace using ip netns exec Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Address Review Comments Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Fix Review Comments * Fix Comments * Added Change for Multi-asic to have iptables rules to accept internal docker tcp/udp traffic needed for syslog and redis-tcp connection. Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Fix Review Comments * Added more comments on logic. * Fixed all warning/errors reported by http://pep8online.com/ other than line > 80 characters. * Fix Comment Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Verified with swsscommon package. Fix issue for single asic platforms. * Moved to new python package * Address Review Comments. Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * Address Review Comments. * Add support to VS platform for platform.json and DPB CLI Tests (sonic-net#5192) - Reverts commit 457674c - Creates "platform.json" for vs docker - Adds test case for port breakout CLI - Explicitly sets admin status of all the VS interfaces to down to be compatible with SWSS test cases, specifically vnet tests and sflow tests Signed-off-by: Sangita Maity <sangitamaity0211@gmail.com> * [iccpd] Fix uninitialized variable. (sonic-net#5112) To declare *tb[] but do not initialize it, it might be very risky. We get iccpd exception during processing arp/nd event. Initialize it to {0}; * Fix unwanted python exception in syslog during database container (sonic-net#5227) startup when doing redis PING since database_config.json getting generated from jinja2 template is still not ready. Signed-off-by: Abhishek Dosi <abdosi@microsoft.com> * [hostcfgd] Handle Both Service And Timer Units (sonic-net#5228) Commit e484ae9 introduced systemd .timer unit to hostcfgd. However, when stopping service that has timer, there is possibility that timer is not running and the service would not be stopped. This PR address this situation by handling both .timer and .service units. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [arista] Update driver submodules (sonic-net#5147) - fix watchdog timeout units - fix import path for thermal_manager - remove arista bind mounts for docker-snmp - improve arista bind mounts for pmon * [docker-radv] Fix startup issues (sonic-net#5230) **- Why I did it** PR sonic-net#4599 introduced two bugs in the startup of the router advertiser container: 1. References to the `wait_for_intf.sh` script were changed to `wait_for_link.sh`, but the actual script was not renamed 2. The `ipv6_found` Jinja2 variable added to the supervisor config file goes out of scope before it is read. **- How I did it** 1. Rename the `wait_for_intf.sh` script to `wait_for_link.sh` 2. Use the Jinja2 "namespace" construct to fix the scope issue **- How to verify it** Ensure all processes in the radv container start properly under the correct conditions (i.e., whether or not there is at least one VLAN with an IPv6 address assigned). * [sonic-utilities] Update submodule (sonic-net#5233) * src/sonic-utilities d5fdd74...17fb378 (7): > [sonic-installer] Import re module (sonic-net#1061) > [fast-reboot]: Fix fail to execute fast-reboot problem (sonic-net#1047) > [config] Reduce Calls to SONiC Cfggen (sonic-net#1052) > [filter-fdb] Call Filter FDB Main From Within Test Code (sonic-net#1051) > [sflow_test.py]: Fix show sflow display. (sonic-net#1054) > Change fast-reboot script to use swss and radv service script (sonic-net#1036) > Common functions for show CLI support on multi ASIC (sonic-net#999) * [sonic-host-service]: Add SONiC Host Services infrastructure (sonic-net#4840) - Why I did it When SONiC is configured with the management framework and/or telemetry services, the applications running inside those containers need to access some functionality on the host system. The following is a non-exhaustive list of such functionality: Image management Configuration save and load ZTP enable/disable and status Show tech support - How I did it The host service is a Python process that listens for requests via D-Bus. It will then service those requests and send a response back to the requestor. This PR only introduces the host service infrastructure. Applications that need access to the host services must add applets that will register on D-Bus endpoints to service the appropriate functionality. - How to verify it - Description for the changelog Add SONiC Host Service for container to execute select commands in host Signed-off-by: Nirenjan Krishnan <Nirenjan.Krishnan@dell.com> * Add common functions applicable to single/multi asic platforms (sonic-net#5224) * Add common functions applicable to single/multi asic platforms * Raise exception if invalid namespace is given as input. * [sonic-swss] Update submodule (sonic-net#5231) * src/sonic-swss d2bab10...c4949a2 (34): > [dvs] Add new common issues and TOC to DVS README (sonic-net#1405) > Avoid adding loopback interface (ip link add) when setting nat zone on loopback interface (sonic-net#1411) > [portsorch] add buffer drop FC group (sonic-net#1368) > [dvs/chassis] Bring up SONiC interfaces in virtual chassis (sonic-net#1410) > [chassis/dvs] Add support for virtual chassis to DVS testbed (sonic-net#1345) > [sonic-swsss] Fix the issue of field "next_hop_ip" not getting updated in state DB in ERSPAN Mirror (sonic-net#1375) > [intfmgr] Fix OA crash issue due to link local configurations (sonic-net#1195) > Fix the issue when persistent DVS is used to run pytest which has number of front-panel ports < 32 (sonic-net#1373) > [dvs] Refactor AsicDbValidator (sonic-net#1402) > [fec] Get FEC mode when port is already admin down (sonic-net#1403) > [fec] added logic that put port down before applying fec onfiguration (sonic-net#1399) > [dvs] Add performance test for adding and deleting routes (sonic-net#1392) > Ignore IPv6 link-local and multicast entries as Vnet routes (sonic-net#1401) > [vlanmgr] Support Jumbo Frame By Default (sonic-net#1393) > Fix log/syslog not being correct when last test fails for given module (sonic-net#1395) > Get initial speed from ASIC DB (sonic-net#1390) > [dvs] Add options to limit CPU usage (sonic-net#1394) > [intfsorch] Retrieve Port object before setting NAT zone on router interfaces. (sonic-net#1372) > [.gitignore] Ignore gearsyncd binary (sonic-net#1381) > Added Max Nexthopgroup/ECMP Count supported by device into State DB. (sonic-net#1383) > [dvs] Upload logs even if failure occurs during startup (sonic-net#1389) > [rates] fix issue with rates init (sonic-net#1387) > [dvs] Validate that SWSS is ready to receive input before starting tests (sonic-net#1385) > [dvs] Convert sflow and speed tests to use dvslib (sonic-net#1382) > [dvs_acl] Refactor and document dvs_acl library (sonic-net#1378) > [dvs] Fix install instructions in README (sonic-net#1379) > [dvs] Update README with new flags, options, and known issues (sonic-net#1380) > swss: gearsyncd should return 0 on exit (sonic-net#1376) > Remove 00-copp.config.json from swss debian package. (sonic-net#1366) > fix undefined var in rates lua scripts (sonic-net#1365) > [fdborch] Fixed Orchagent crash in FDB flush on port disable. (sonic-net#1369) > [tlm_teamd]: Try to add LAG again, when teamd is not ready first time (sonic-net#1347) > [vs] Incorporate python3 best practices into DVSLib (sonic-net#1357) > [dvs] Mark unstable tests as xfail (sonic-net#1356) * [arista/aboot]: Zero out 1st MB before repartitioning (sonic-net#5220) The first partition starting point was changed to be 1M as part of this commit: 6ba2f97. On systems that are misaligned before conversion (partition start is the first sector), the relica partition that is left in the first MB can cause problems in Aboot and result in corruption of the filesystem on the new aligned partition. Zeroing this old relica makes sure that there is nothing left of the old partition lying around. There won't be any risk of having Aboot corrupt the new filesystem because of the old relica. Signed-off-by: Baptiste Covolato <baptiste@arista.com> * [sonic-py-common] Add unit test framework (sonic-net#5238) **- Why I did it** To install the framework for adding unit tests to the sonic-py-common package and report coverage. ** How I did it ** - Incorporate pytest and pytest-cov into sonic-py-common package build - Updgrade version of 'mock' installed to version 3.0.5, the last version which supports Python 2. This fixes a bug where the file object returned from `mock_open()` was not iterable (see https://bugs.python.org/issue32933) - Add support for Python 3 setuptools and pytest in sonic-slave-buster environment - Add tests for `device_info.get_machine_info()` and `device_info.get_platform()` functions - Also add a .gitignore in the root of the sonic-py-common directory, move all related ignores from main .gitignore file, and add ignores for files and dirs generated by pytest-cov * Add switch for synchronous mode (sonic-net#5237) Add a master switch so that the sync/async mode can be configured. Example usage of the switch: 1. Configure mode while building an image `make ENABLE_SYNCHRONOUS_MODE=y <target>` 2. Configure when the device is running Change CONFIG_DB with `sonic-cfggen -a '{"DEVICE_METADATA":{"localhost": {"synchronous_mode": "enable"}}}' --write-to-db` Restart swss with `systemctl restart swss` * [enable counters] Enable port buffer drops by default and update MLNX SAI submodule (sonic-net#5059) * Enable port buffer drops by default * [Mellanox] Update SAI_Implementation Signed-off-by: Mykola Faryma <mykolaf@mellanox.com> * Platform monitor changes in daemon_base for multi_asic (sonic-net#4932) Adding namespace support for db connect API. * [py-swsssdk] Submodule Update (sonic-net#5249) Change: c25d492 Merge pull request sonic-net#83 from tahmed-dev/taahme/add-redis-pipeline-operation 198d143 review comments - part of [configdb] Add Ability to Query/Update Redis Using Pipelines 994851c review comments - part of [configdb] Add Ability to Query/Update Redis Using Pipelines 2d2b7e1 making lgtm happy - part of [configdb] Add Ability to Query/Update Redis Using Pipelines fa9093c [configdb] Add Ability to Query/Update Redis Using Pipelines signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> * [cfggen] Use Redis Pipeline (sonic-net#5250) This PR enables cfggen to readr/write from Redis DB using pipelines. Pipelines enables batch read/write from/to Redis DB. signed-off-by: Tamer Ahmed <tamer.ahmed@microsoft.com> Co-authored-by: Petro Bratash <68950226+bratashX@users.noreply.github.com> Co-authored-by: Tamer Ahmed <tamer.ahmed@microsoft.com> Co-authored-by: SuvarnaMeenakshi <50386592+SuvarnaMeenakshi@users.noreply.github.com> Co-authored-by: Joe LeVeque <jleveque@users.noreply.github.com> Co-authored-by: Mahesh Maddikayala <10645050+smaheshm@users.noreply.github.com> Co-authored-by: judyjoseph <53951155+judyjoseph@users.noreply.github.com> Co-authored-by: abdosi <58047199+abdosi@users.noreply.github.com> Co-authored-by: Sangita Maity <sangitamaity0211@gmail.com> Co-authored-by: Kelly Chen <kelly_chen@edge-core.com> Co-authored-by: Samuel Angebault <staphylo@arista.com> Co-authored-by: nirenjan <nirenjan@users.noreply.github.com> Co-authored-by: Baptiste Covolato <b.covolato@gmail.com> Co-authored-by: shi-su <67605788+shi-su@users.noreply.github.com> Co-authored-by: Mykola F <37578614+mykolaf@users.noreply.github.com>

* src/sonic-utilities d5fdd74...17fb378 (7): > [sonic-installer] Import re module (sonic-net#1061) > [fast-reboot]: Fix fail to execute fast-reboot problem (sonic-net#1047) > [config] Reduce Calls to SONiC Cfggen (sonic-net#1052) > [filter-fdb] Call Filter FDB Main From Within Test Code (sonic-net#1051) > [sflow_test.py]: Fix show sflow display. (sonic-net#1054) > Change fast-reboot script to use swss and radv service script (sonic-net#1036) > Common functions for show CLI support on multi ASIC (sonic-net#999)

Common changes will be used to support SONiC CLIs for multi ASIC - New MultiAsic class to support not displaying of internal object - Common CLI options which needed for multi ASIC platforms - a new decorator to execute a function on all namespaces Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>

Signed-off-by: Ze Gan <ganze718@gmail.com> b9337dc (HEAD, origin/master, origin/HEAD) [vslib]: Fix MACsec bug in SCI and XPN (#1003) edbceb9 [syncd][vslib] Keep new warm boot discovered SERDES objects (#985) af5c156 Fix build issues on gcc-10 (#999) 1445cd5 update SAI submoule (#1001) 48fe704 [ci] pipeline fixes for VS test (#1002) f484cf9 Enable SAI_SWITCH_ATTR_UNINIT_DATA_PLANE_ON_REMOVAL attribute (#975) 5d0b22d Enable SAI_SWITCH_ATTR_UNINIT_DATA_PLANE_ON_REMOVAL attribute (#975) 1b8ce97 (origin/202111) [pipeline] Download swss common artifact in a separated directory (#995) 7a2e096 Change sonic-buildimage.vs artifact source from CI build to official build. (#992)

5331ecd [vslib]: Fix MACsec bug in SCI and XPN (#1003) ac04509 Fix build issues on gcc-10 (#999) 1b8ce97 [pipeline] Download swss common artifact in a separated directory (#995) 7a2e096 Change sonic-buildimage.vs artifact source from CI build to official build. (#992) d5866a3 [vslib]: fix create MACsec SA error (#986) f36f7ce Added Support for enum query capability of Nexthop Group Type. (#989) 323b89b Support for MACsec statistics (#892) 26a8a12 Prevent other notification event storms to keep enqueue unchecked and drained all memory that leads to crashing the switch router (#968) 0cb253a Fix object availability conversion (#974)

jleveque requested review from lguohan and taoyl-ms September 28, 2017 23:54

jleveque self-assigned this Sep 28, 2017

taoyl-ms suggested changes Sep 29, 2017

View reviewed changes

taoyl-ms approved these changes Sep 29, 2017

View reviewed changes

lguohan reviewed Sep 30, 2017

View reviewed changes

lguohan requested changes Sep 30, 2017

View reviewed changes

stcheng added the Enhancement ➕ label Oct 2, 2017

jleveque added 15 commits October 2, 2017 23:28

[DHCP Relay]: Support new <DhcpRelays> minigraph tag; support multipl…

5a8f2e3

…e VLANs

Don't start dhcrelay in quiet mode so as to get startup output in syslog

36a0b65

Update sonic-cfggen tests to support new '<DhcpRelays>' tag

4e76084

<DhcpRelays> tag is only present for VLANs which require a DHCP relay…

afa2fb3

… agent -- only parse if present

Don't attempt to configure a DHCP relay agent for VLANs without speci…

06f903d

…fied DHCP servers

Modify to work with Taoyu's minigraph/DB changes (#942)

57101fb

Reduce number of DHCP servers in sonic-cfggen unit tests from 4 to 2

39b4c60

Remove isc-dhcp-relay sample output file from sonic-cfggen test, as w…

9893597

…e no longer generate that file

Update Option 82 isc-dhcp-relay patch to load all interface name-alia…

ef93c68

…s maps into memory once at start instead of calling sonic-cfggen on each packet we relay

Remove executable permission from Jinja2 template

69d6fa6

Set max hop count to 1 so that DHCP relay will only relay packets wit…

39d0301

…h a hop count of zero

Replace tabs with spaces

bc1550f

Modify overlooked sonic-cfggen call, use Config DB instead of minigraph

a97fff2

Also ensure > 1 VLAN requires a DHCP relay agent before outputting to…

ba4a06b

… template

Generate port name-alias map file using sonic-cfggen and parse that i…

7bb46d1

…n lieu of parsing port_config.ini directly

No longer drop packets with hop count > 0; Instead, drop packets whic…

486d0a5

…h already contain agent info

lguohan approved these changes Oct 4, 2017

View reviewed changes

lguohan merged commit 1d16a37 into sonic-net:master Oct 5, 2017

jleveque deleted the dhcp_relay_multiple_vlans branch October 5, 2017 17:10

jleveque added the DHCP Relay label Jul 9, 2020

abdosi added a commit that referenced this pull request Aug 16, 2020

[submodule update] sonic-utilities

87fa880

Common functions for show CLI support on multi ASIC (#999)

jleveque mentioned this pull request Aug 21, 2020

[sonic-utilities] Update submodule #5233

Merged

Pterosaur mentioned this pull request Feb 22, 2022

[submodule]: Update sonic-sairedis #10061

Merged

6 tasks

AidanCopeland pushed a commit to Metaswitch/sonic-buildimage that referenced this pull request Apr 14, 2022

Fix build issues on gcc-10 (sonic-net#999)

af5c156

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DHCP Relay]: Support Multiple VLANs (Separate DHCP Relay Agents, One Per VLAN) #999

[DHCP Relay]: Support Multiple VLANs (Separate DHCP Relay Agents, One Per VLAN) #999

jleveque commented Sep 28, 2017 •

edited

taoyl-ms Sep 28, 2017 •

edited

jleveque Sep 29, 2017

taoyl-ms Sep 29, 2017

taoyl-ms Sep 29, 2017

jleveque Sep 29, 2017 •

edited

jleveque Sep 29, 2017 •

edited

taoyl-ms commented Sep 29, 2017

jleveque commented Sep 29, 2017

taoyl-ms commented Sep 29, 2017

jleveque commented Sep 29, 2017

lguohan Sep 30, 2017

jleveque Sep 30, 2017

lguohan Sep 30, 2017

lguohan Sep 30, 2017

lguohan Sep 30, 2017

jleveque Sep 30, 2017

lguohan Sep 30, 2017

jleveque Sep 30, 2017

jleveque Oct 4, 2017 •

edited

lguohan left a comment

lguohan left a comment

jleveque commented Oct 5, 2017

[DHCP Relay]: Support Multiple VLANs (Separate DHCP Relay Agents, One Per VLAN) #999

[DHCP Relay]: Support Multiple VLANs (Separate DHCP Relay Agents, One Per VLAN) #999

Conversation

jleveque commented Sep 28, 2017 • edited

taoyl-ms Sep 28, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jleveque Sep 29, 2017 • edited

Choose a reason for hiding this comment

jleveque Sep 29, 2017 • edited

Choose a reason for hiding this comment

taoyl-ms commented Sep 29, 2017

jleveque commented Sep 29, 2017

taoyl-ms commented Sep 29, 2017

jleveque commented Sep 29, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jleveque Oct 4, 2017 • edited

Choose a reason for hiding this comment

lguohan left a comment

Choose a reason for hiding this comment

lguohan left a comment

Choose a reason for hiding this comment

jleveque commented Oct 5, 2017

jleveque commented Sep 28, 2017 •

edited

taoyl-ms Sep 28, 2017 •

edited

jleveque Sep 29, 2017 •

edited

jleveque Sep 29, 2017 •

edited

jleveque Oct 4, 2017 •

edited