Bug fixes in Junos-related components #55824

rsmekala · 2020-01-09T09:20:45Z

What does this PR do?

Multiple bug fixes were done for Junos-related components in salt.

What issues does this PR fix or reference?

Fixes install_config times out on longer commit checks Juniper/salt#85, Salt execution module junos.cli with invalid dest="" throws exception FileNotFoundError: [Errno 2] No such file or directory Juniper/salt#86, Wrong documentation under junos.diff CLI section Juniper/salt#88, Saltstack: salt execution module junos.rpc options format and dest works only with get-config rpc Juniper/salt#89, Saltstack:salt execution module junos.rpc get-config invalid filter throws SyntaxError exception Juniper/salt#90, salt execution module junos.rollback CLI example syntax is not correct (salt 'device_name' junos.rollback 10) Juniper/salt#93, salt execution module junos.load and junos.install_config with invalid config files throws exception AttributeError: 'bool' object has no attribute 'read Juniper/salt#94, Applying salt state file with function junos.diff fails with TypeError: diff() missing 1 required positional argument: 'd_id' Juniper/salt#96, Applying salt state file with funciton junos.rpc fails with TypeError: rpc() takes at most 2 arguments (3 given) Juniper/salt#99, Exception emitted on salt-master [ERROR ] Unhandled exception running status.proxy_reconnect after junos.zeroize . Juniper/salt#100, salt execution module junos.install_config with invalid mode throws exception ValueError: unsupported action: test Juniper/salt#103, salt execution module junos.install_config with mode dynamic throws exception RpcError: RpcError(severity: error, bad_element: None, message: error: statement creation failed: version Juniper/salt#104, salt execution module junos.install_os with invalid image name throws exception and help information is not dispalyed Juniper/salt#109, Saltstack:salt execution module junos.install_os default installs the image on Master RE , options are not available to select dual RE , back up RE Juniper/salt#110, salt execution module junos.rpc CLI example syntax is not correct (salt 'device' junos.rpc get-interface-information /home/user/interface.xml) Juniper/salt#112, Only one config action has to be supported, currently multiple config actions are supported Juniper/salt#113, Add merge option to load() to match PyEZ capabilities Juniper/salt#114, salt executiom module junos.install_os with dev_timout fails with Installation failed due to: "pkgadd() got multiple values for keyword argument 'dev_timeout'" Juniper/salt#117

Tests written?

Yes

Fixes multiple bugs found by test team

…tall_config

Fixes multiple PRs raise by test team

…ities

sagetherage · 2020-04-15T15:59:42Z

sagetherage I have changed the file requirements/static/darwin.txt and linux.txt version for junos-eznc to pick 2.4.0
https://github.com/saltstack/salt/pull/55824/files#diff-5b7107aa6e0c14fa491d8ffb575c4264R11

But it still picks 2.2.0 https://jenkinsci.saltstack.com/job/pr-ubuntu1604-py3/job/PR-55824/15/consoleFull and hence test cases fail for missing package.

Do I need to change into py3.5, py3.6, and py3.7 folder files also?

@vnitinv Getting some help here, but wanted you to know I am working on this. thank you!

dmurphy18

Also need to now use pre-commit and black for PR submissions. They will be run as part of the PR test etc. but good to catch issues beforehand. Check https://github.com/saltstack/salt-enhancement-proposals/blob/master/accepted/0015-black.md where it was discussed.

Apart from changing published interfaces it looks good, published interfaces have to go through a deprecation cycle (used to be two major releases) if the changes are not backwards compatible, that is, can extend them, but should not affect existing parameters.

salt/modules/junos.py

dmurphy18 · 2020-04-15T23:52:53Z

salt/states/junos.py

@@ -33,16 +33,15 @@ def wrapper(*args, **kwargs):


 @resultdecorator
-def rpc(name, dest=None, format="xml", args=None, **kwargs):
+def rpc(name, **kwargs):


changing an interface for a function which has been published without a deprecation cycle having been done is not allowed

Salt initially made changes in Junos-salt modules 93ee5ee#diff-15ad308a12ea5b6d5188dd4a94eab4a0R490 93ee5ee#diff-15ad308a12ea5b6d5188dd4a94eab4a0R400. This commit caused all our state module to break Juniper#77.

@dmurphy18 Hi David, The issue was caused as salt changed positional arguments(*args) in our modules function definition to **kwargs. But, the state modules were not modified in those commit. we had changed it to **kwargs to keep it in line with changes done by salt.
Also we have neither changes any function name of removed any parameters, just that they are consumed via **kwargs.

@vnitinv I have checked back as far as 2017.7.0 and the function signature is the same as that remove by your change. I can understand that this may not line up with Juniper's own functionality but it is an interface which has been published for a few years and it MUST go through a deprecation cycle before it can be changed. And I understand that the kwargs can handle the changes but the positional arguments MUST remain until the deprecation cycle for an interface change has completed. Note can add a new interface and mark the old interface for deprecation. Existing customers cannot have the rug pulled out from underneath them without time to adjust.

@dmurphy18 I have reverted to old function definition itself.

dmurphy18 · 2020-04-15T23:54:43Z

salt/modules/junos.py

@@ -149,7 +160,7 @@ def facts():


 @timeoutDecorator
-def rpc(cmd=None, dest=None, **kwargs):
+def rpc(cmd=None, **kwargs):


changing an interface for a function which has been published without a deprecation cycle having been done is not allowed

same as above

see reply for salt/states/junos.py about deprecation cycle and changing interfaces

@dmurphy18 I have reverted to old code itself.

salt/modules/junos.py

dmurphy18 · 2020-04-16T00:01:25Z

salt/states/junos.py

@@ -158,15 +152,14 @@ def commit(name, **kwargs):


 @resultdecorator
-def rollback(name, id, **kwargs):
+def rollback(name, **kwargs):


Changing an interface for a function which has been published without a deprecation cycle having been done is not allowed

same as above

see reply for salt/states/junos.py about deprecation cycle and changing interfaces

@dmurphy18 I have reverted to old code itself.

dmurphy18 · 2020-04-16T00:03:48Z

salt/states/junos.py

    return ret


 @resultdecorator
-def diff(name, d_id):
+def diff(name, **kwargs):


Changing an interface for a function which has been published without a deprecation cycle having been done is not allowed. You could extend the interface

def diff(name, d_id, **kwargs)

and do a versionadded, but the interface has to be backwards compatible unless a deprecation cycle is performed

same as above, also to add

For diff, the documentation clearly stated that it expects "id" as the parameter name but the name of the argument was "d_id". This is a bug and we don't see a reason to support "d_id". Also, this will go inline to changes done at 93ee5ee#diff-15ad308a12ea5b6d5188dd4a94eab4a0R490

see reply for salt/states/junos.py about deprecation cycle and changing interfaces
The label of a positional argument doesn't really matter here compared to a documentation typo/error, but the positional argument is required otherwise the interface is changing without a deprecation cycle for the interface

@dmurphy18 I have reverted to the old code function definition itself.

salt/states/junos.py

tests/unit/modules/test_junos.py

vnitinv · 2020-04-16T08:41:52Z

Also need to now use pre-commit and black for PR submissions. They will be run as part of the PR test etc. but good to catch issues beforehand. Check https://github.com/saltstack/salt-enhancement-proposals/blob/master/accepted/0015-black.md where it was discussed.

Apart from changing published interfaces it looks good, published interfaces have to go through a deprecation cycle (used to be two major releases) if the changes are not backwards compatible, that is, can extend them, but should not affect existing parameters.

Used pre-commit tools and also done suggested changes like a single quote to double.
Also added dependencies list in Linux and darwin files.

dmurphy18

Interfaces cannot change without deprecation cycles, new interfaces can be added to address desired changes and the old interfaces marked for deprecation, which typically are removed after two major release cycles. Existing customers cannot have interfaces changes without time to accommodate them.

dmurphy18 · 2020-04-17T18:03:04Z

salt/states/junos.py

@@ -158,15 +152,14 @@ def commit(name, **kwargs):


 @resultdecorator
-def rollback(name, id, **kwargs):
+def rollback(name, **kwargs):


see reply for salt/states/junos.py about deprecation cycle and changing interfaces

dmurphy18 · 2020-04-17T18:04:54Z

salt/states/junos.py

    return ret


 @resultdecorator
-def diff(name, d_id):
+def diff(name, **kwargs):


see reply for salt/states/junos.py about deprecation cycle and changing interfaces
The label of a positional argument doesn't really matter here compared to a documentation typo/error, but the positional argument is required otherwise the interface is changing without a deprecation cycle for the interface

vnitinv · 2020-04-20T13:06:45Z

@dmurphy18 Check if the changes are good now?

vnitinv · 2020-04-24T05:49:54Z

Hi @sagetherage @cro @dmurphy18, Please help in getting this to closure. We want this to be part of Sodium release. Let us know if anything is to be done from our side.

sagetherage · 2020-04-24T17:10:52Z

I have followed up internally as well @vnitinv with David Murphy and he will respond by no later than end of day, Monday 2020-04-27, he may respond sooner.

dmurphy18

Apart from adding versionchanged for 'def diff' for the new extra kwargs, everything looks good, review wise.

salt/states/junos.py

dmurphy18

Thanks for the work

dmurphy18 · 2020-04-27T23:47:49Z

@vnitinv Some failing tests that need to be fixed before can merge

vnitinv · 2020-04-28T06:26:20Z

@dmurphy18 I have fixed pre-commit failures.

For ci/py3/macosxmojave looks like there is something that needs to be done from salt backend. Check the existing issue related to enum failure of Mac. #56603

rsmekala and others added 30 commits March 18, 2019 12:06

Merged junos-specific changes on 2019.2

c2b425b

Merged junos-specific changes on 2019.2

f70aed5

Fixes FileNotFound exception with invalid dest option in junos.cli

ffb1b59

Fixed documentation for junos.diff

4b80926

Fixed improper handling of dest and format options in rpc modules

e241cf8

Implement exception handling around XML filter parsing

f8f38a6

Porting fix from develop to 2019.2

90e3f76

Fixed documentation

403e197

Fixed documentation

704b376

Merge pull request #87 from rsmekala/2019.2

4fb35fa

Fixes multiple bugs found by test team

Fixed rollback docstring: CLI example

d9e762f

Merge remote-tracking branch 'upstream1/2019.2' into 2019.2

7bec402

Implemented exception handling around template rendering in junos.load

4b4b50d

Implemented exception handling around template rendering in junos.ins…

a34846d

…tall_config

Fixes broken state module junos.diff

5203b8f

Fixes broken state module junos.rollback

20fdfa4

Fixes broken state module junos.rpc

6b78415

Adds documentation about zeroize functionality for non-root user

4a8818c

Adds error message for install_config in case invalid mode provided

e20b06d

Handle LockError case in install_config gracefully

c97e9b5

Replace string concatenation with format

9678f7a

Revert replacing cli command with rpc: junos.zeroize

89ca74a

Merge pull request #95 from rsmekala/2019.2

e6be5be

Fixes multiple PRs raise by test team

Support dynamic config mode

65e7bb3

Changed CLI example to conform with new rpc module definition

a20863a

Added exception handling for invalid image path for install_os module

153aa54

Extended load() support to config_action:merge, to match PyEZ capabil…

80d3188

…ities

Added check to allow only 1 config_action in load()

afa1491

Fixed reboot=True scenario for install_os

26fda1c

Added comments to explain the behaviour in reboot scenario

2d0d56c

dmurphy18 suggested changes Apr 16, 2020

View reviewed changes

dmurphy18 self-assigned this Apr 16, 2020

fix as per comment from @dmurphy18 and pre-commit changes

e99f139

import line changes as per pre-commit

fb18ed4

dmurphy18 suggested changes Apr 17, 2020

View reviewed changes

Nitin Kumar added 2 commits April 20, 2020 16:58

changes as per comment from salt team regarding interface syntax

d51e7c7

changes as per comment from salt team regarding interface syntax

95c4fff

when called from junos.rpc from test, dest is intacct

be75ccf

dmurphy18 suggested changes Apr 27, 2020

View reviewed changes

salt/states/junos.py Show resolved Hide resolved

added versionchanged salt/states/junos.py->diff

9bc4f8c

dmurphy18 previously approved these changes Apr 27, 2020

View reviewed changes

pre-commit checks for darwin

2d21b7c

vnitinv dismissed dmurphy18’s stale review via 2d21b7c April 28, 2020 05:06

pre-commit check for linux onf py 3.8 and 3.9

f274d8b

Merge branch 'master' into 2019.2

86c5e3e

dmurphy18 approved these changes May 5, 2020

View reviewed changes

dmurphy18 added 4 commits May 5, 2020 10:50

Merge branch 'master' into 2019.2

056f719

Merge branch 'master' into 2019.2

419faa9

Merge branch 'master' into 2019.2

ca9a480

Merge branch 'master' into 2019.2

4025063

dmurphy18 added the Merge Ready label May 7, 2020

twangboy approved these changes May 7, 2020

View reviewed changes

dwoz merged commit d0150f4 into saltstack:master May 7, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug fixes in Junos-related components #55824

Bug fixes in Junos-related components #55824

rsmekala commented Jan 9, 2020

sagetherage commented Apr 15, 2020 •

edited

Loading

dmurphy18 left a comment

dmurphy18 Apr 15, 2020

vnitinv Apr 16, 2020 •

edited

Loading

dmurphy18 Apr 17, 2020 •

edited

Loading

vnitinv Apr 20, 2020

dmurphy18 Apr 15, 2020

vnitinv Apr 16, 2020

dmurphy18 Apr 17, 2020 •

edited

Loading

vnitinv Apr 20, 2020

dmurphy18 Apr 16, 2020

vnitinv Apr 16, 2020

dmurphy18 Apr 17, 2020

vnitinv Apr 20, 2020

dmurphy18 Apr 16, 2020

vnitinv Apr 16, 2020

dmurphy18 Apr 17, 2020

vnitinv Apr 20, 2020

vnitinv commented Apr 16, 2020

dmurphy18 left a comment

dmurphy18 Apr 17, 2020

dmurphy18 Apr 17, 2020

vnitinv commented Apr 20, 2020

vnitinv commented Apr 24, 2020

sagetherage commented Apr 24, 2020

dmurphy18 left a comment

dmurphy18 left a comment

dmurphy18 commented Apr 27, 2020

vnitinv commented Apr 28, 2020

Bug fixes in Junos-related components #55824

Bug fixes in Junos-related components #55824

Conversation

rsmekala commented Jan 9, 2020

What does this PR do?

What issues does this PR fix or reference?

Tests written?

sagetherage commented Apr 15, 2020 • edited Loading

dmurphy18 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vnitinv Apr 16, 2020 • edited Loading

Choose a reason for hiding this comment

dmurphy18 Apr 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dmurphy18 Apr 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vnitinv commented Apr 16, 2020

dmurphy18 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vnitinv commented Apr 20, 2020

vnitinv commented Apr 24, 2020

sagetherage commented Apr 24, 2020

dmurphy18 left a comment

Choose a reason for hiding this comment

dmurphy18 left a comment

Choose a reason for hiding this comment

dmurphy18 commented Apr 27, 2020

vnitinv commented Apr 28, 2020

sagetherage commented Apr 15, 2020 •

edited

Loading

vnitinv Apr 16, 2020 •

edited

Loading

dmurphy18 Apr 17, 2020 •

edited

Loading

dmurphy18 Apr 17, 2020 •

edited

Loading