Cisco IOS support and file transfer mechanism

[ktbyers]

cc: @GGabriele

Wanted a place to have a discussion on the Cisco IOS implementation in NAPALM related to the following methods:

load_merge_candidate(filename, config):
compare_config():
discard_config():
commit_commit():
rollback():

I have been thinking for Cisco IOS that we should do an SCP file transfer mechanism where the file gets transferred to the box (to flash: or whatever is appropriate probably user option).

transfer new file via SCP to remote network device

load_merge_candidate()

'show archive config differences' between running-config and uploaded file (assumes

complete configuration, however, could handle partial config by doing "more" on the uploaded file)

compare_config()

just don't do the merge/config replace (or potentially delete transferred file)

discard_config()

perform 'configure merge' or 'configure replace' depending on what is appropriate

commit_config()

I think rollback should really be a "configure replace" back to a previous known state

rollback()

Here are some issues that I see with the current method:

1. Large config changes will be exceptionally slow using Netmiko (and probably any SSH screen scraping mechanism).
2. Rollback using a 'no' to the command is very problematic (a lot of bad side effects will happen because of this).
3. IMO the file transfer mechanism is more consistent with the NAPALM paradigm.

I have done proof of concept of a lot of the above, but definitely more work is needed.

I do have concerns about how reliable 'configure replace' is. How transparent is this to the network when you do this?

Wanted to see what people's thoughts are on this?

[dbarrosop]

I think it makes sense. My main concern is keeping it consistent; uploading a file to a place present on all platforms. You mentioned a proof of concept. Could you share it via a gist? Is there any easy way to test it? Regarding reliability of "configure replace", I think it's just a matter of adding some input on the "caveats" section if we think something is fishy.

By the way, I am curious about using *miko vs scp. Do you think it's going to be faster? I am using paramiko with pyfg and it's extremely fast applying changes and parsing the configuration. And fortigate can easily have thousands of lines of configuration ;) I am curious if you have some experience with this that makes you believe SCP can be faster.

[ktbyers]

@dbarrosop Let me create a new branch and generate a proof of concept on it. I did work on this earlier, but most of it was outside of NAPALM (i.e. similar ideas but not using NAPALM specific methods).

Yes, I have IOS devices in my environment so I can test.

I glanced at some of the pyfg code. It looks like you batch the commands and then send them down the channel in a single exec_command(). I would have to test if interacting this way would work with Cisco IOS. There definitely are concerns that I have on how Cisco IOS would behave using this method (and it is not how Netmiko is currently built).

So I expect using SCP would not be meaningfully faster for you with the pyfg code, but using SCP will be a lot faster than the current NAPALM Cisco IOS implementation for anything but very small config changes (since NAPALM Cisco IOS relies on Netmiko which I know is slow for all but very small config changes).

I probably need to look at a faster way to do multiline config changes inside Netmiko. Up to now I have been more concerned with having automation that works reliably.

[plucena24]

@ktbyers -> "configure merge"? I haven't seen these semantics on IOS before. Do you mean "copy <scp'd file> running-config" - which will essentially merge?

The only downside to doing "config replace" is that a lot matters on the type of config being sent (downtime, etc). The good thing about it as you mentioned is that the caller is forced to only deal with full configs each time you need to make a change using NAPALM.

On the other hand, the copy scpd-file running method has the disadvantage of merging configs - and the potential for leaving bits of old config behind.

My personal preference from an operational perspective would be to do a "copy scpd-file running" - and make it a responsibility of the user of NAPALM to take care of side effects by properly crafting the config file.

Even better would be to delete startup, copy scpd file to startup, and reboot =). This would be the cleanest and least error prone on IOS - but not possible in many cases =)

[ktbyers]

@plucena24 Yes, I just meant 'copy flash:candidate_config.txt running-config' (i.e a merge of the candidate_config.txt file into the running-config).

The only downside to doing "config replace" is that a lot matters on the type of config being sent (downtime, etc).

Do you just mean by this that the caller has to get their configuration right (or something else)?

[ktbyers]

@plucena24 I think it will support both the replace and merge operation.

def load_replace_candidate() # will do 'configure replace'

def load_merge_candidate() # will do the merge into running-config

[plucena24]

@ktbyers yes - I meant that the caller has to get their configs right. For example, if you are going to change a port from "access" to "trunk" by just setting:

switchport mode trunk
switchport trunk encap dot1q

In the config, then all of the other "access port" config will still be present in the config. Another example, if your intent is to replace a BGP peer's address (currently 1.1.1.1 - changing to 2.2.2.2), and the new config being merge only does the following:

router bgp X
neighbor 2.2.2.2 remote-as XX

Then the old peer will still remain in the running config after the operation.

What I mean by "the caller has to get their config right" is that the caller has to take care of deleting old config that will no longer be needed - if they are going to be merging from scp'd file to running.

I think having both use cases available is a good option.

load_replace_candidate can be used when the caller is going to be strictly dealing with full config files during each change.

load_merge_candidate can be used when the caller wants to do small additions to the config (adding a peer, adding a user, adding snmp, etc).

[ktbyers]

@plucena24 Yes, I think we are saying the same thing :-)

[dbarrosop]

Yeah, I think we are all in the same page.

[ktbyers]

@dbarrosop @plucena24 @GGabriele

Okay, I have an implementation of this here:

https://github.com/ktbyers/napalm/tree/cisco_ios_scp

It should be pretty good. I haven't updated the unit tests, but I have my own set of tests that I have been running it against.

I pretty much re-wrote these methods:

    def load_replace_candidate(self, filename=None, config=None):
    def load_merge_candidate(self, filename=None, config=None):
    def compare_config(self, file1=None, file2='running-config',
    def commit_config(self, filename=None):
    def discard_config(self):
    def rollback(self, filename=None):

I also added some other methods to assist the above operations:

    def scp_file(self, source_file, dest_file, file_system):
    def _enable_confirm(self):
    def _disable_confirm(self):
    def gen_full_path(self, filename, file_system=None):
    def _gen_rollback_cfg(self):
    def _check_file_exists(self, cfg_file):

I will provide some more details on how it operates, caveats, and work on the tests.

Once I get the unit tests updated I will submit a PR on it.

[dbarrosop]

I will look into this during the holidays when I have the time. I talked with @GGabriele and he mentioned he would take a look as well.

[ktbyers]

No worries. I am out traveling so I probably won't look at it for another
week.

Kirk

On Tuesday, December 22, 2015, David Barroso notifications@github.com
wrote:

I will look into this during the holidays when I have the time. I talked
with @GGabriele https://github.com/GGabriele and he mentioned he would
take a look as well.

—
Reply to this email directly or view it on GitHub
#119 (comment).

Kirk Byers
CCIE #6243 emeritus
ktbyers@twb-tech.com
http://pynet.twb-tech.com http://pynet.twb-tech.com/blog
(415) 264-6270
Simplify your network management through automation

[GGabriele]

Yes sorry @ktbyers, I've not found the time to test it yet. I'll definitely do it in coming days :)

[t2d]

I have one more question about def load_replace_candidate(). What is supposed to happen with commands, which are completely excluded? Let's say, I don't want to manage my version in the template. Would it still be safe to just omit it?

[ktbyers]

@t2d I didn't understand your question here? Can you add some more detail on what you are trying to do?

[ktbyers]

The new Cisco IOS solution is included in NAPALM 0.51