Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Apply commits from master onto R6.0-WIP
db_resync_done lock is used to indicate the amqp thread to wait for resync to complete. In this case, when we call SchemaTransformer.destroy_instance() due to Casandra connection failure, this lock remains locked blocking destroy_instance. destroy_instance calls _vnc_subscribe_callback to drain the amqp queue which waits infinitely for db_resync_done lock to be released. This fix releases db_resync_done lock so that destroy_instance doesnt get blocked. Closes-Bug: #1801474 [DM] Hitless image upgrade implementation Closes-Bug: #1799322 Provisioner for the devicemanager node. usage: from /opt/contrail/utils python provision_devicemgr_node.py --host_name aio --host_ip 10.87.82.2 --oper add --admin_user admin --admin_password contrail123 --admin_tenant_name admin --openstack_ip 10.87.82.2 --api_server_ip 10.87.82.2 Closes-Bug: #1805303 CFM: Changes for onboarding L3PNF - Add new platform SRX240 - Add L3PNF subnet is schema - Add new namespace, VN and IPAM for L3PNF during brownfield onboarding Closes-Bug: 1800701 Add entrypoint to vrouter-agent service on Windows Introduce entrypoint for agent similar in design to that from microservice deployment. For now it will only start agent, actual features will be added in following changes. Partial-Bug: #1806677 Check build dependencies for tbb, SimpleAmqpClient and rabbitmq Closes-Bug: #1806719 Make agent's entrypoint update agent's config on Windows In future we will generate the whole config from scratch as on Linux, but for now we only update the vhost's ifname. It's the only field that can change upon restart. Partial-Bug: #1806677 bgp-peer selection support for bgpaas 1. Listener BgpRouterConfig is added for BgpRouter and ControlNodeZone 2. BgpRouterConfig builds BgpRouterTree and ControlNodeZoneTree from IFMapNode 3. BGPaaS gets BgpRouter for configured ControlNodeZone from BgpRouterConfig and Updates bgp-peer-ip and bgp-peer-port in the flow. 4. Step 3 is followed for xmpp based peer-selection also. 5. BGPaaS sandesh is updated with primary_control_node_zone, secondary_control_node_zone, bgp_peer_ip and bgp_peer_port Partial-bug: #1775872 [DM] Inside-outside workflow - lag/mH 1. Change the exisiting business logic to adhere to the new data model for lag/mH workflow 2. Multi-vlan support Partial-Bug: #1799329 Rework nodemgr before fixing ntp issue - move windows/linux code to separate classes instead of same condition through the code - simplify main.py - remove copy duplication Closes-Bug: 1800704 [fabric] Added playbook retry support to job manager 1) When playbook return retry_devices in the output, job manager will retry the playbooks against those devices 2) remove obsolete playbooks from 5.0 3) remove obsolete ansible roles from 5.0 4) added a warning log on missing loopback interface when creating bgp router 5) import os version to prouter object 6) Fixed pep8 warnings and errors Partial-Bug: #1805237 nodemgr: add checking time sync by crhony Closes-Bug: #1800704 Dont include "lost+found" folder in /var/crashes If /var/crashes is mounted, fsck can create "lost+found" folder in it. The core list reported by nodemgr will contain this folder, even though its not a core file. This assumes there is no core file reported by the name "lost+found" and it can also report other folders/files reported in var/crashes. Closes-Bug: 1784069 Adding Introspect for mesos manager Fix for bringing introspect for mesos-manager objects on port 8109 Partial-bug: #1795295 Agent changes for inter-as option c part1 1. added new table int.3 to process labelled inet routes 2. added new nexthop type labelled tunnel nh 3. vpn routes with tunnel type as mpls depends on inet.3 routes to resolve its nexthop partial-Bug: #1743517 Mesos api server not responing to 127.0.0.1 ip Setting api server ip to be accessed through the node ip Closes-Bug: #1807302 Adding default as_path for 2 byte asn neighbor With 4 byte asn changes, by default as_path with 4 byte asn was being added. It should be as_path with 2 byte asn if the peer does not support 4 byte asn. Closes-Bug: #218663 build fix for inter-as changes Partial-Bug: #1743517 Account for flow count on interface if flow is ingress flow , increase flow count by 2 for interface one for forward and another for reverse. If zero is set on vmi take value from virtual network. Closes-Bug: #1805337 Closes-Bug: #1806866 Closes-Bug: #1804823 Agent crash @ assert in VxlanRoutingManager::BridgeVnNotify Porting the fix from R5.0 - removed the assert Closes-bug: #1767044 Split flow management to multiple files Partial-Bug: #1800894 [Fabric] Enhancements for Generic Device Operations Included the following changes on the controller: 1. Included the output_ui_schema in the predef paylaods.json for job_template_object 2. Included the actual output_ui_schema in the schema.json files 3. included the output_ui_schema in load_data under api-server 4. Resolved merge conflicts and changed description to job_template_description in predef_payloads.json 5. Changed description in schema for better understanding Closes-Bug: #1806508 Fix for multiple interfaces in pod showing in ECMP endpoints This addresses the issue of multiple interfaces showing up as ECMP endpoints when a loadbalancer is created for a multi interface pod. With this fix only the default interface is added as an endpoint Closes-Bug: 1791460 Make XMPP talk TLSv1.2 Use SSLv23 for TLS connections and then set the context options to exclude SSLv1, SSLv2 and TLSv1 thereby forcing the connection to use TLS v1.2 Partial-Bug: #1807296 Fix most of level 1 warnings on Windows - Invalid escape sequence - Invalid unused field and getter - Invalid preprocessor usage - Wrong type of field - Missing return in non-void returning functions Partial-Bug: #1800894 Fix warning related to not called destructor Partial-Bug: #1800894 [DM] Hitless image upgrade Files and roles added to support hitless image upgrade Closes-Bug: #1803626 Revert "Fix for active TCP flows not transitioning" This reverts commit c0c4425. Closes-Bug: #1786924 schema-transfomer: don't update RT list when SI init isn't done * When RoutingInstanceST are initialized, static routes are updated. * This update trigger an update of route targets list which should not be updated in the case where the RI is owned by an SNAT SI and that latter has not been initialized yet in the reinit function. * Prevent the schema to remove RT of SNAT's RI and so prevent to lose SNAT traffic temporarily when restarting schema-transformer. Closes-Bug: #1787371 (cherry picked from commit 182f705) Asn should be treated as uint With 4 byte ASN, introspect is displaying higher asn as -ve number because those are defined as u32 in sandesh files. They should be defined as u32 instead. Also Rtarget should take into account 4 byte asn properly. If asn <= 0xFFFF, we'll create rtarget with 2 byte asn value, otherwise, rtarget will have 4 byte asn Partial-Bug: #218680 [DM] Add device health check for hitless upgrade Closes-Bug: 1803614 bgp-peer selection support for bgpaas 1. Added introspect to BgpRouter and ControlNodeZone 2. Added Unit tests to validate secondary control-node-zone Partial-bug: #1775872 Partial-jira-bug: #JCB-184435 [Config] Add provisioning scripts for Config Database and WEBUI Inorder to support the following feature https://github.com/Juniper/contrail-specs/blob/master/5.1/ansible-delete-roles-and-nodes.md we need to add support to register two new nodes, WEBUI and CONFIG DB. In this change, added two files for provisioning webui and config database nodes. These files are similar to provision_database_node.py Closes-Bug: #1799269 Closes Jira Bug: JCB-216383 Support DNS reverse zones for IPv6 Automatically generate reverse zones for IPv6 addresses OpenContrail assigns to tenant VMs. Also, make contrail-api accept ip6.arpa names alongside in-addr.arpa ones, and adapt provision tools to create them. Finally, support having NS records in reverse zones to delegate a sub-zone to another nameserver. Closes-Bug: #1730584 Signed-off-by: Valentin Sinitsyn <valesini@yandex-team.ru> Signed-off-by: Alexey I. Froloff <raorn@raorn.name> Adding support to add timer interval through config Changing hard coded logic of forever syncing to time based which can be acquired by conf file. Closes Jira Bug:JCB-168187 Remove unused variables Partial-Bug: #1800894 Fixed boundary condtion check for ksync bulk msgs Agent assert is seen when the size of the bulk message sent to vrouter is equal to 4096 as the message size should be less than 4096.The code allows to form bulk message upto 4096. Fixed this boundary condition issue. Closes-Bug: #218722 Active TCP flows not moving to deny upon SG attach The bug manifested coz we reflexively update a reverse flow, when a forward flow is updated. We call UpdateReflexiveFlow() for reverse flow whenever a forward flow is updated. After the SG attach(allows only egress) to Server port, the forward flow moves to 'D' and reverse flow to 'H'. When TcpAck from server port hit the reverse flow, flow is reevaluated and it becomes F(since egress is allowed and this is TcpAck). And this reverse flow becomes forward and previous forward flow(which is 'D') becomes reverse. Now the new reverse flow is updated reflexively and copies the sg_action_summary from Forward flow to be toggled to 'F', which leads to the reported problem. This will allow back traffic from client to server. Fix here has been to not reflexively update the reverse flow for forward flows, triggered because of TcpAck packets. This allows the keepalives from server to work, while allowing the C->S flow to be dropped. The change has been made in ResyncFlow since this is called first for reverse flow and then for the forward flow(expecting the reverse flow to be updated again as part of the ResyncFlow code). Closes-Bug: #1786924 [DM] Add strategy playbook for hitless upgrade Closes-Bug: 1805981 [DM] Include LAG configuration for maintenance mode Update jinja template to include LAG configuration to put a leaf into maintenance mode closes-jira-bug: CF-29 Mark graceful_restart_flap_all_test7 as flaky Partial-Bug: 1733446 schema-transfomer: clean stale route-target properly When RouteTargets are reinitialized, the stale route targets (route targets which doesn't have logical router backref or routing instance back ref) are getting removed. But its not removed properly. Route target remains on cls.__dict. Call the 'delete_vnc_obj' method which cleans the route target properly. Closes-Bug: #1808518 Firewall: traffic filter between two AGs as EP fails We have made changes for address group labels to do 'OR' operation instead 'AND' between endpoints Closes-Bug: #1770518 DCI: DCI UT (covers API tests) and global system config Loopback NS update issue Partial-Bug: #1794010 Don't overwrite createvrf flag for mirror config updates Agent sets create vrf flag to true if it creates VRF as part of mirror entry addition. when mirror entry is deleted , Agent checks vreate vrf flag, if it is set then deletes teh VRF. In this scenario, mirror config updates are overwriting the flag to false and that blocks the VRF deletion made changes not to overwrite the flag if it set. Closes-bug: #1803264 vnc_cassandra: encode keys before calling pycassa Closes-Bug: #1808132 (cherry picked from commit d7cad14) Disable log level checks for flows The flow messages are sent at LOG_INFO by default. If the configured log level is anything more flow messages will get dropped. Default configured log level is SYS_NOTICE. For flow messages to be written to the syslog, the configured log level has to be SYS_INFO; but this can cause a huge churn. By disabling this check only for flow messages, they will be logged as long as the destination is set as Syslog. Partial-bug: #1811001 Partial-jira-bug: #CBS-7 bgp-peer selection support for bgpaas Added missing link delete in cleanup Partial-bug: #1775872 Partial-jira-bug: JCB-184435 Add analytics-alarm and alalytics-snmp nodemgr. Adding provisiong code for alarm/snmp nodes Partial-Bug: #1784493 [DM] Set the IRB IP as the default gateway IP for ERB closes-jira-bug: CF-41 Job Manager and ZTP refactoring - Moved Job Manager to DM - DM manages ZTP - Using DNSMASQ as DHCP and TFTP server - Using Rabbit MQ for Job Manager and ZTP communication - Implemented ZTP config and TFTP update in DM - Added Kombu AMQP client - Added APIs to api-server for AMQP Implements: https://blueprints.launchpad.net/juniperopenstack/+spec/dm-pod Closes-Bug: #1780580 Closes-Bug: #1793811 Closes-Bug: #1794887 Closes-Bug: #1796762 Fat flow AggrDst: IPv6: ICMP6 Flow in hold state - This is a day1 issue where fat flow for ICMP6 was not supported/working. Agent Changes (Also see vrouter changes) :- 1) ICMPv6 rule will be stored with proto 1 and port 0 (similar to ICMP rule) 2) Rule lookup for ICMPv6 will happen using proto 1. Additional fix for IPv6 rule match when src/dst prefix is empty. Closes Jira Bug:JCB-206313 Chaning the index of route target if going beyond boudary With 4 byte ASN, index field in route target can only be 2 bytes. However, schema generates default rtargets with 4 byte value. Real fix would have been to change those values in schema itself but it has its own challenges. Here, adding a hack in control node to change index of a rtarget if asn is 4 byte and index is also 4 byte (only if in the range of auto generated rtargets) Partial-Bug: #1807123 Manage rtarget routes associated with bgpaas sessions With bgpaas static configuration for associated control-nodes, it is possible that bgpaas sessions can connect to a control-node in a routing-instance, even though there is no explicit xmpp agent subscription for the same. If this happens, then as is, control-node will not attract any routes of the instance if route-target-filtering is enabled (which is the default configuration) It is only when agents subscribe do we add import-target to the bgp.target.0 table so that instance routes are attracted in bgp.l3vpn.0 table. With this change, when ever bgpaas session comes up in a routing-instance, then all its associated import route targets are added to bgp.rtarget.0 table with source peer as bgpaas peer pointer. This is similar to what we do when an agent subscribes. On the other hand, when bgpaas session goes down, then those routes are removed from the bgp.rtarget.0 table. Also, current set of route-targets configured for an instance are tracked so that in case of configuration change to the import route targets list, routes are correctly updated. TODO: Add more unit tests to cover routing-instance configuration changes. Partial-bug: #1775872 [Config] Add support to log remote IP address As per the current implementation, WEBUI host IP is sent as remote IP in the logs sent to Analytics. This change will add actual remote IP address of the end user when sending the log information to Analytics. To do this, a WEBUI change will be done seperately. This code review is to handle the changes in vnc_cfg only. Closes-Bug: #1808574 closes-jira-bug: JCB-189711 [DM] Improve hitless upgrade error handling closes-jira-bug: CEMDA-60 pass ssl options to all instances of cassandra's client partial-jira-bug: JCB-218753 [DM] Job concurrency support Added zookeeper lock based fabric and device level concurrency control for jobs closes-jira-bug: CF-25 Added a defensive check in casting NH in AddReceiveRoute AddReceiveRoute assumes the active path to have an NH with interface. This resulted in agent coring when the active path pointed to composite NH. Adding a defensive check to proceed only if NH is INTERFACE or RECEIVE. Closes-Bug: #1795090 (cherry picked from commit 879df7e) [DM] Vendor agnostic changes - initial checkin 1. Initial checkin for framework testing 2. Device import tested for vendor agnostic device querying / command execution 3. Device import tested for filter plugin invoking 4. Device import tested for functionality with refactoring 5. Device import including debug logging functionality 6. Topology discovery tested for vendor agnostic device querying / command execution 7. Topology discovery tested for filter plugin invoking 8. Topology discovery tested for functionality with refactoring 9. Topology discovery including debug logging functionality 10.Unified logging across all modules into a single instance 11.Included device name in debug logs also 12.Included validators at the beginning of parsing 13. Included schema checks. 14. To Modify parsing methods and include some annotations Patch Set #23 changes: 1. Resolved merge conflicts Patch Set #24 changes: 1. restructured files into different folder strucure 2. deleted unwanted files Patch Set #28 changes: 1. Restructured directory organization and file names using appropriate naming convention 2. Included doc strings for modules 3. Removed vnc_util and added functionality in JobVncApi Patch Set #32 changes: 1. Changed lo0 ip address tracking to list in order to save the value in jinja processing Patch Set #35 changes: 1. Resolved merge conflicts Closes-Bug: #1785127 Closes-Bug: #1784968 Closes-Bug: #1786319 Metadata requests should use fqdn instead IP partial-jira-bug: JCB-218839 [DM] Use container label to find and restart closes-jira-bug: JCB-218842 [DM] Simplified plugin manager logic partial-jira-bug: JCB-218842 [APISERVER] - Adding 'None' (str) check for gateway and service address while creating Subnet object Closes-Jira-Bug: JCB-211810 [CFM] enable multi-homing unit test Partial-jira-Bug: CF-39 Add Support for: Server Discovery, Server Import and Node Profile Import Servers and Node Profiles can be imported from YAML or JSON Files Patches: Fixed Job Ctx to use Single Auth token end to end Fixed Try Catch Blocks Removed whitespaces Removed auth vars from group_vars/all.yml -> Should be in test code Can import nodes without ports IPMI Port Range format (Min Max) Fixes for Auth Token Added decription for schema Removed Ironic in Import schema (not needed) Added YAML/JSON enum for validation of format Removed failed import Partial-Bug: #1807762 Fixing some flaky test in agent There were some tests marked as flaky. Fixing them so that there is better code coverage for agent code. closes-jira-bug:JCB-218754 Added validation check for BgpRouter Object in Contrail-api Control-Node-Zone should be linked only BgpRouter type of "control-node". Added check to validate the same in BgpRouter pre_dbe_create() and pre_dbe_update() Closes-jira-bug: CNET-75 Use semicolons properly with 'while (false)' in macros Partial-Bug: #1800894 get inet labeled address family enable info partial changes to get inet labeled address family info from bgp router config. workaround for UI issue: setting tunnel encapsulation type as MPLS over MPLS if labeld inet is enabled, will remove these changes once UI fix is in. Closes Jira Bug:JCB-218795 [DM] fix regression issue in device job manager closes-jira-bug: CEMDA-89 [DM] Device states not updated properly during hitless image upgrade closes-jira-bug: CEMDA-69 enable SSL for cassandras' clients partial-jira-bug: JCB-218753 Agent EVPN Multicast Changes partial-jira-bug: JCB-218799 increase version of db_manage.py script partial-jira-bug: JCB-218753 Fixed flaky test-cases in test_sg_flow and test_sg_tcp_flow Fixes in test_sg_flow.cc 1. XML parsing code had extra white-spaces, hence removed it. 2. Call AddIPAM() from Init() and remove DelIPAM() from TearDown() as it is already getting called in Shutdown. This way it will be similar to test_sg_flowv6 which is already working. Fixes in test_sg_tcp_flow.cc 1. XML parsing code had extra white-spaces, hence removed it. 2. We were not deleting bgp_peer_ Updated SConscript to make above test-cases as non-flaky. Closes Jira Bug:JCB-218833 AMQP Client: Fixed typo in keyfile attribute closes-jira-bug: JCB-218897 [vnc openstack] Add support of Neutron FWaaS v2 API Add support to the Neutron FWaaS v2 API extension [1] as described in the spec [2]. [1] https://developer.openstack.org/api-ref/network/v2/?expanded=id341-detail,id346-detail#fwaas-v2-0-current-fwaas-firewall-groups-firewall-policies-firewall-rules [2] https://github.com/Juniper/contrail-specs/blob/master/neutron_FWaaSv2.md Partial-Jira-bug: JCB-204217 Route Reflector code clean up in ST Cleaned up route reflector related code that got added previously. It was causing some unexpected behaviour like calling update peering on all the bgp routers on an update to any one of the bgp routers even when it's not required. Closes-Jira-Bug: JCB-218900 [DM] Filter logic for hitless upgrade flag closes-jira-bug: CEMDA-59 [DM] Regression in logging - Vendor Agnostic Checkin Refactoring logging to a single file was missed out for some places in fabric.py for role_assignment. Also included the refactored model of logging for hitless_upgrade_filters.py and write_to_file_filter.py. closes-jira-bug: CEMDA-113 Packaging Ironic and Contrail Command Plugins These were missing and causing import error when any fabric job is called closes-jira-bug: CNET-101 Check for consumers before draining events If the AMQP client proceeds with draining events without having any consumers that could delay the consumer creation and loss of messages closes-jira-bug: JCB-218915 db_manage: Make Individual connection timeout and buffer size user configurable closes-jira-bug: JCB-218813 [DM] Hitless Upgrade Health Check Improvements closes-jira-bug: CEMDA-117 FWAAS support Partial-Jira-bug: JCB-204217 Add agent changes to support FWAAS policies Added UT to test the FWAAS gate [DM] Create QFX-specific basic config template closes-jira-bug: CEMDA-119 To enable api-server SSL, enable bottle ssl. The following parameters need to be passed from deployer to the api-server configuration file: 'config_api_ssl_enable': <boolean> 'config_api_ssl_keyfile': 'config_api_ssl_certfile': 'config_api_ssl_ca_cert': the webui_job_1 & webui_web_1 uses VncApi client, it need to be configured to use ssl as well. closes-jira-bug: JCB-218811 AS4PATH was not being processed for 2 byte neighbours AS4PATH attribute was added to handle cases where 2byte node interacts with 4 byte node but somehow it was not added in the list of attributes to be handle for 2 byte neighbours. Closes-Jira-Bug: JCB-218927 [CFM] Ansible filter to read from db directly Helps in reading objects from DB directly Closes-jira-bug: CEMDA-31 Prefer EBGP over IBGP first before looking at path-id During BGP best-path selection, EBGP paths should be preferred over IBGP and only then should we look at the path-id (which is typically the nexthop address which is pretty much a random selction) Because BGPaaS peer based ebgp route-target route was not preferred as the best path, route target filtering caused routes not to get advertised Closes-Jira-Bug: CNET-121 [DM] Populate default annotations on fabric object - Example annotations listed in bug CF-44 comments - Currently annotations only used for hitless upgrade and fabric creation closes-jira-bug: CF-44 - Changes for schema transformer to be aware of fabric when handling mp-bgp peering. - Added UT to handle the fabric aware schema transformer mp-bgp peering closes-jira-bug: JCB-218878 Enabling agent UT test script. Closes Jira Bug:JCB-218757 Fix type conversion related warnings for common/base/task* Partial-Bug: #1800894 [DM] Enhance information provided to the user during job execution - Fixes bug with JobAnnotations json import closes-jira-bug: CEMDA-44 [DM][fabric] IPv6 support - Updated abstract config to have family and gateway for every IP address - Deprecated 'ip_list' - Updated Jinja templates to use address family from abstract config closes-jira-bug: CEMDA-142 Set psutil version to last working one for nodemgr Psutil==5.5.0 throws exceptions in nodemgr on Windows. This issue was reported to the maintainers and should be fixed in future versions. Closes Jira Bug: WIND-158 [fabric] Fixed JSON error closes-jira-bug: JCB-218973 [fabric] Enable IGMP snooping for ERB closes-jira-bug: JCB-218983 [DM] Deletion of an onboarded fabric device fails closes-jira-bug: JCB-218968 [DM] Error handling for maintenance mode During any failure through the entire workflow, deactivate MM if it is in the active state closes-jira-bug: JCB-218949 Fix HA issue with VncApi usage Use api server ip list instead of a single ip while creating the vnc api client. Randomize the list to ensure load balancing Closes-jira-bug: JCB-218962 [DM] Check VN is valid when setting IRB closes-jira-bug: JCB-209995 Mark graceful_restart_flap_some_test2 as flaky (for now) Partial Jira Bug: JCB-218991 [DM] change LinkAggregationGroup to VirtualPortGroup Partial-jira-Bug: CF-39 [DM] Hitless upgrade health report printing problems closes-jira-bug: CEMDA-149 Kubemanager should wait till Global vrouter object is created by VNC API server Closes-Jira-Bug: JCB-218957 Rtarget should be valid when attaching to RI Whenever a route target is attached to a routing instance, its validity should be checked, especially with 4 byte ASN. Closes-Jira-Bug: JCB-219010 add missing files/folder to fabric-ansible packaging closes-jira-bug: CNET-101 [DM] import lldp playbook bails out if there an interface has multiple neighbors This bug has been fixed for shared interfaces with multiple neighbors. closes-jira-bug: JCB-218966 Marking igmp_test as flaky Mark igmp_test UT as flaky for now. Will be addressed as part of cleaning up UTs Closes Jira Bug: JCB-219033 [fabric] Enable igmp for IRB and set multicast-replication mode closes-jira-bug: JCB-219035 closes-jira-bug: JCB-219036 [DM] Remove rb_routing roles from hitless upgrade logic closed-jira-bug: CEMDA-156 [fabric] Use FilterLog for logging in db_filter closes-jira-bug: JCB-219027 Core file deletion in nodemgr The nodemgr instances delete the core files of processes it controls. If the core files belonging to a process exceeds 4, nodemgr deletes the oldest entry. At any point in time only 4 core files for each process that nodemgr controls in allowed to exist. Partial-Bug: 1776170 [DM] Percentage calculation change to include two modes Percentage change to include both hitless and non-hitles mode closes-jira-bug: CEM-299 [DM][fabric] Fixed error in job status notification closes-jira-bug: JCB-219030 closes-jira-bug: JCB-219031 [config] Split resource API server code Split the huge 'vnc_cfg_types.py' file in one file per Contrail resources type. All new file will follow pep8 recommendations and the patch adds that pep8 checks in the scons test target to be sure pep8 compliance will not be broken. it also moves from testr to stestr as testr is not anymore maintained. Closes-jira-bug: JCB-218755 Partial-jira-bug: JCB-218756 nexthops are not added correctly in vrouter. Made changes in nexthop ksync compare function to compare transport label for labelled tunnel nexthops. changes to support tunnel type change( MPLS over GRE/MPLS over UDP) for MPLS over MPLS tunnel based on global tunnel encapsulation priority list. Closes Jira Bug:JCB-218985 [vnc openstack] Add config knob to enable FWaaS on VNC OpenStack API When the config flag 'NEUTRON.fwaas_enabled' is set to true, the default firewall group is automatically provisioned in project and applied to VM port by default. Partial-Jira-bug: JCB-204217 [fabric] Add ipv6 static route under rib for MX closes-jira-bug: CEM-305 [DM] Use updated docker api Partial-Jira-Bug: JCB-218848 Reverting the changes related to db_filter Removing the db_filter since we cannot use the vnc_db_client within the ansible process space. Closes-jira-bug: JCB-219027 [DM] Hitless image upgrade device_list processing closes-jira-bug: CEM-144 [schema transformer] Clean RT when a VN is deleted API server is now in charge of the creation and deletion of the primary RI of a VN but the schema stills in charge to allocate a unique route target number and create/delete the corresponding RT. But actually, when a VN is deleted, schema tries to delete all referenced RI (in memory and on the VNC API for all the RI and only in memory for the primary RI). But frequently, it deletes the primary RI before the VNC API have deleted it and it tries to delete the allocated RT but that failed as it still referenced by the primary RI not yet deleted. Then when the primary RI is deleted, the schema ignore the notification as that RI is not anymore in its memory and the RT remains (lock in zk was removed and could be re-used). That patch changes the schema to not delete the primary RI when a VN is deleted and lets the notification of the primary RI deletion warns the schema to delete the RI and its allocated RT. Closes-Jira-Bug: JCB-219068 [DM] Move vrf import reject maintenance mode to evpn overlay closes-jira-bug: CEM-306 parse string to boolean flag for api_ssl_enable option partial-jira-bug: JCB-218849 [DM] Handle untagged vlan for lag/mH Closes-jira-bug: JCB-219032 Closes-jira-bug: JCB-218974 Fixed flaky test-cases in test_vrf_assign_acl.cc 1. XML parsing code had extra white-spaces, hence removed it. 2. While verifying flow action we need checking if particular flag bit is set. Made it same as class VerifyFlowAction. 3. Updated SConscript to make above test-case as non-flaky. Closes Jira Bug:JCB-218833 Agent support for Multicast Policy defined in schema. Implementation of schema field "virtual-network-multicast-policy" in contrail agent. Also includes changes required for integration with IGMP module for <S,G> policy checks. partial-jira-bug: JCB-218799 [config] Add validation check for SubnetType Validate any create or update of all SubnetType attribute with netaddr python library. Closes-Jira-Bug: JCB-218787 support new interface of python-docker library in nodemgr nodemgr can work with both interface with this fix. later old interface support can be removed and code can be rerworked to use new features. Partial-Jira-Bug: JCB-218848 Added dependency path in bgp-as-a-service Dependency path was missing between bgpaas-control-node-zone and bgpaas-virtual-machine-interface in bgp-as-a-service. Due to that bgpaas-virtual-machine-interface event got missed in virtual-machine-interface node which caused the issue. Added missed dependency path in bgp-as-a-service. Closes-jira-bug: JCB-219072 [DM] - Added API server validation while creating service appliance object - Added logic to allocate VLAN, each left and right per service instance - Fix build compilation - Fix unit tests - Allocate two internal VN's for management of RT's for left and right service VRF - Fix VLAN allocation logic for service instance creation - Fix physical interface ref creation during service appliance creation - Fix port tuple deletion errors - Added logic to allocate IRB units for spine device using virtual network ID - Added logic to create dummy loopback interface for PNF device to be used for BGP peering on PNF device - Fix build error - First clean-up, removing un-wanted code - Added pre dbe create validation for service instance - Handled most of Suresh review comments - Handle Edouard review comments Closes-Bug: #1807426 fixes for getloadavg using wmi Partial-Bug: #1789754 [DM] Missing error handler in Sconscript closes-jira-bug: CEM-325 [DM] - Updating local cache for new VNC objects - Defining new PNF plugin for PNF physical role - Updated the reaction map based on new logic and data-model changes - Fix DM integration issues - Add initial logic for rendering PNF abstract config - Add business logic for QFX abstract config - Added a check to check for PNF physical role for PR's with no BGP router - Handled Suresh review comments - Added checks while generating abstract config, so that config push happens when all required parameters are present Closes-Bug: #1800701 CFM: PNF Service chaining ansible playbooks Closes-Bug: 1800701 [Config] Fix for lost update problem in Security Group Rules When updating a security group in parallel to add/delete multiple rules, we see that there's a lost update problem leading to undefined behavior. This change is to fix this problem by implementing a Zookeeper lock inside neutron_plugin_db of API Server. Closes-Jira-Bug: JCB-218976 Closes-Jira-Bug: JCB-181802 Fixed flaky test-cases in test_flow_error 1. On setting block_msg_processing_ as true flow_stats_manager was not able to collect short flows for deletion and hence it was not getting deleted. 2. Flows are actually not deleted, we just reset them. Hence fe->deleted flag is reset to false and hence assertion happens. So we need to delete check for fe->deleted(). 3. Added a check to see that flow count should be 0. 4. Added a check to see if it is shortflow. 5. Updated SConscript to make above test-case as non-flaky. Closes Jira Bug:JCB-218833 [DM] Fix hitless upgrade file Fix the typo in hitless upgrade file that was changed during percentage update. closes-jira-bug: CEM-337 master: slow vrouter convergence with BFD health check In BFD Poll phase, BFD Detection time is not computed properly. 1) Detection time Fix After the BFD session is UP, when remotes bfd.DesiredminTxmin decreased due the detection time is not recomputed. Earlier code was only computing if the bfd.DesiredminTxmin is greater than its previous advertised value but not when the value is decreased. 2) Also fixed if in poll phase if the remote session's minRxInterval is changed. If the value is increased, the new value will be used when recomputing the SendTimer(). 3) Added UT BasicSingleHop_CfgChange_DetectionTime to validate the detectiontime. Closes-jira-bug: CEM-294 (cherry picked from commit beb3396) Server-Disocvery: PI-Port Ref, EndSystem-Node-Profile ref fixes Partial-Bug: #1805237 1. on create , create ref_link 2. on update, remove old link and create new link 3. for deleting port, ref link needs to be deleted by client. 4. Moved from Node to EndSystem 5. if node_profile_ref is found in EndSystem, remove old tag ref from EndSystem-Ports and then re-create new ref to tag 6. create port-groups if specificed in node-profile 7. remove port-groups refs before creating new ones updated code based on newer hierarchy and recent schema changes changes to comply with pep8 fixed any unit-test failures fixed pep8 failures as well do not call keystone_connect from extension init it leads to exception because keystone is not ready yet. and due to excpetion the extension becomes not loaded. and config-api tells that keystone connection is not ready until config-api is not restarted. closes-jira-bug: JCB-176427 DM: Use vlan-id none for irb interfaces in the case of Spines. We were assigning vn-id as vlna-id which is not recommended. https://junipernetworks.sharepoint.com/sites/EMC-multi-cloud-NPI-docs-site/Shared%20Documents/InterLock/EVPN-VXLAN-Configuration-Examples-for-Quick-Referance.pdf close-jira-bug: JCB-210290 Set key name in routing-instance UVE during deletion Because key was not, UVE remains in the cache locally and in the remote collector Closes-Jira-Bug: JCB-218965 Fixed kombu client consuming and hearbeat logic - Run hearbeat check only if there are consumers - Handle connection status explicitly - Wait for consumers in consumer greenlet - Payload check when publishing - Fixed error in docker api usage partial-jira-bug: JCB-219115 ISSU run docker under root - in case of OSP for ssh heat-admin user is used. Partial-Bug: #1801956 Move the multi_device flag from the job template to playbook Use this flag to control the spawning of greenlets within the job manager instead of the device json. Closes-jira-bug: CEMDA-31 [DM] Read leases file line by line partial-jira-bug: JCB-219115 Support comma separated list of interfaces As per the multi-interface spec, annotations for networks can now be a comma separated list. Also fixes un-necessary update of networks and display formatting for network-status annotation Closes-jira-bug: JCB-219125 For local ECMP NHs, pick the vif from the child member NH while creating fat flows Today, in case of local ECMP NHs, the flow gets created using the ECMP NH as Key NH because of which the vif interface couldn't be retrieved to apply fat flow config. With this fix, we will check for local ECMP NH, and if so we will pick the vif from the first member NH. If the first member vif interface has fat flow configured, then it will be applied. closes-jira-bug: JCB-219075 [config] Fix vitual DNS resource name typo During the refactoring code patch, the Virtual DNS resource type name was not correctly wrote. Resource type name is 'virtual_DNS' instead of 'virtual_dns'. Closes-Jira-Bug: JCB-219182 Check for non-FDN vrouter name if FQDN names do not match. closes-jira-bug: JCB-219162 FWAAS support Partial-Jira-bug: JCB-204217 Add agent changes to support FWAAS policies Missed to checkin flow_mgmt changes If Kubernetes API server watch is not set during event process, call watch register again. closes-jira-bug: JCB-219181 Crash with sub-interface on VM side only. Sub-interface not created on contrail side. VN IPAM is not picked resulting in ipam being NULL. Hence the crash. putting a check for ipam being NULL. Unexpected use-case, but should be handled. partial-jira-bug: JCB-219107 [DM] Removing routing instances delete as part of fabric deletion Removed explicit deletion of Routing Instances Also fixed one of validate role assignment checks closes-jira-bug: JCB-219195 [Fabric] - Changing int to str before concatenation closes-jira-bug: JCB-219172 Change-Id: I0fd2d010fe5448d9ced5061690acbc85a31bc3f5 Depends-on: Icd111bf92e4656fe7187726cd95ceabb80b9415f Partial-jira-bug: JBE-925
- Loading branch information
1 parent
9c3d4e4
commit 77df3b5