How to establish NETCONF sessions using SSH authentication in AWS ?

[khalilmebarkia]

I'm running docker Yang Development Kit for python ydk-py in my remote server Linux Ubuntu. I would like to Establish a connection using with the remote server and my AWS EC2 instance that runs CSR 1000v (SSH authentication)

I used to access my router using the following ssh command:

ssh -i "ssh-key.pem" ec2-user@ec2-xx-xx-xx-xxx.us-west-2.compute.amazonaws.com
Where ec2-xx-xx-xx-xxx.us-west-2.compute.amazonaws.com is the hostname, ec2-user is the username and the ssh key ssh-key.pem is for authentification.

As the first step, I want to run the given example in here ydk-py samples

This is the creation of NETCONF session in the given example:

    provider = NetconfServiceProvider(address="10.0.0.1",
                                      port=830,
                                      username="admin",
                                      password="admin",
                                      protocol="ssh")

I have tried this

provider = NetconfServiceProvider(address="ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com", 
username= "ec2-user", 
 public_key_path="mykey.pem")

I have got this error

Traceback (most recent call last):
  File "hello-ydk.py", line 18, in <module>
    private_key_path="mykey.pem")
TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. ydk_.providers.NetconfServiceProvider(repo: ydk_.path.Repository, address: unicode, username: unicode, password: unicode, port: int=830L, protocol: unicode=u'ssh', on_demand: bool=True, timeout: int=-1L)
    2. ydk_.providers.NetconfServiceProvider(address: unicode, username: unicode, password: unicode, port: int=830L, protocol: unicode=u'ssh', on_demand: bool=True, common_cache: bool=False, timeout: int=-1L)
    3. ydk_.providers.NetconfServiceProvider(repo: ydk_.path.Repository, address: unicode, username: unicode, private_key_path: unicode, public_key_path: unicode, port: int=830L, on_demand: bool=True, timeout: int=-1L)
    4. ydk_.providers.NetconfServiceProvider(address: unicode, username: unicode, private_key_path: unicode, public_key_path: unicode, port: int=830L, on_demand: bool=True, common_cache: bool=False, timeout: int=-1L)

Invoked with: 'ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com', 'ec2-user'; kwargs: repo=None, public_key_path='mykey.pem'

I kept going through all possibility where I found in the README file in here Read me the running a sample app is the following:

Unless specified by the app, all basic apps take two command line arguments. An optional argument (-v | --verbose) to enable logging and a mandatory argument in URL format that describes the connection details to the networking device (ssh://user:password@device:port):

$ ./nc-read-xr-ip-ntp-oper-10-ydk.py ssh://admin:admin@10.0.0.1
So in my case, it should be like this, right?

$ ./hello-ydk.py ssh://ec2-user:ec2-user@ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com

But still did not work, and here's what I got

from: can't read /var/mail/ydk.services
from: can't read /var/mail/ydk.providers
from: can't read /var/mail/ydk.models.cisco_ios_xr
from: can't read /var/mail/datetime
./hello-ydk.py: 13: ./hello-ydk.py: Syntax error: "(" unexpected (expecting "then")

[khalilmebarkia]

I tired the following.

provider = NetconfServiceProvider(address="ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com",
 username="ec2-user", 
 private_key_path="/home/server/shared_files/mykey.pem")

I tried to debug the python script and it turns out that there is a problem with argument type which is private_key_path for the code above

-> username="ec2-user",
(Pdb) next
> /home/server/shared_files/hello-ydk.py(15)<module>()
-> private_key_path="/home/server/shared_files/mykey.pem")
(Pdb) next
TypeError: "__init__(): incompatible constructor arguments. The following argument types are supported:\n .../home/server/shared_files/mykey.pem', address='ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com'"

How can I solve this issue?

[vulcan25]

Can you ssh into this 1000v instance either on port 22 or port 830, without ydk-py in the mix, and sucessfully gain console access?

Critically, can you do it with the same mykey.pem?

Can you post that ssh command here, if you can do this sucessfully. That would isolate whether it's a problem with gaining access to the instance in general, or ydk-py's implementation of NetConfServiceProvider.

Also the official docs on making an ssh connection to this don't specifically mentioned port 830, so one could assume this may run on 22 by default in this AMI image (which I am assuming is this one from the marketplace?).

So it may be worth trying to specify port=22 when you create an instance of NetconfServiceProvider in your code above. Otherwise it defaults to 830.

Obviously to test the raw connection via ssh on linux, you'd be leaving -p22 out as this is the default. The string would litterally look like what the docs say:

ssh -i pem-file-name ec2-user @[public-ipaddress | DNS-name ] 

[ghost]

Can you try with providing the private_key argument?

[khalilmebarkia]

Can you ssh into this 1000v instance either on port 22 or port 830, without ydk-py in the mix, and sucessfully gain console access?
Critically, can you do it with the same mykey.pem?

Yes, I could. I think the default port is 22 to access to EC2 instance by simply using ssh -i "mykey.pem" ec2-user@ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com

Can you post that ssh command here, if you can do this sucessfully. That would isolate whether it's a problem with gaining access to the instance in general, or ydk-py's implementation of NetConfServiceProvider.

here it is ssh -i "mykey.pem" ec2-user@ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com.
I think the problem is the implementation itself, since I did debug of that python script, the problem was in private_key_path argument in my comment here.

Also the official docs on making an ssh connection to this don't specifically mentioned port 830, so one could assume this may run on 22 by default in this AMI image (which I am assuming is this one from the marketplace?).

Actually, I deployed this is one

So it may be worth trying to specify port=22 when you create an instance of NetconfServiceProvider in your code above. Otherwise it defaults to 830.

Well, I just tried with 22 port and still the same issue.

Obviously to test the raw connection via ssh on linux, you'd be leaving -p22 out as this is the default. The string would litterally look like what the docs say:

ssh -i pem-file-name ec2-user @[public-ipaddress | DNS-name ] 

Yes, it is like that to access to EC2 instance.

[khalilmebarkia]

Can you try with providing the private_key argument?

Well, I did and it did not work.
The same error while debugging

-> private_key="/home/server/shared_files/mykey.pem")
(Pdb) n
TypeError: "__init__(): incompatible constructor arguments. The following argument types are supported:\n    ...ver/shared_files/mykey.pem', port=22, address='ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com'"
> /home/server/shared_files/hello-ydk.py(15)<module>()
-> private_key="/home/server/shared_files/mykey.pem")

[ghost]

Can you post your full script? You need all the arguments: hostname, username, private_key, public_key. See example

[khalilmebarkia]

Her

Can you post your full script? You need all the arguments: hostname, username, private_key, public_key. See example

Here it is.

from ydk.services import CRUDService
from ydk.providers import NetconfServiceProvider
from ydk.models.cisco_ios_xr import Cisco_IOS_XR_shellutil_oper \
    as xr_shellutil_oper
from datetime import timedelta


if __name__ == "__main__":
    """Main execution path"""

    # create NETCONF session
    provider = NetconfServiceProvider(address="ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com", 
       username="ec2-user",                          
       private_key_path="/ydk-py/mykey.pem", #this path is for the private key from amazon**
       public_key_path="/root/.ssh/id_rsa.pub", #this path is for the public generated key from my server and I uploaded it to amazon**
                          port=22)

    # create CRUD service
    crud = CRUDService()

    # create system time object
    system_time = xr_shellutil_oper.SystemTime()

    # read system time from device
    system_time = crud.read(provider, system_time)

    # print system uptime
    print("System uptime is " +
          str(timedelta(seconds=system_time.uptime.uptime)))

    exit()

And I'm still getting the same error.

[vulcan25]

#this path is for the public generated key from my server and I uploaded it to amazon**

Can you expand on this? IS this definately the public half of the same key. You can generate that from mykey.pem, with something like: ssh-keygen -f mykey.pem -y > id_rsa.pub (
According to this answer)

If this is all correct, the only other thing I can think of is: does the private key have a passphrase? If that's the case perhaps NetConfServiceProvider is throwning the error because it can't handle the requred passphrase input.

Perhaps try generating your own (passphraseless) keypair, and upload the public section to amazon, then put the newly generated priv/pub pair into your script. That's assuming amazon doesn't allow you to generate a key without a passphrase within their interface.

EDIT: Actually just checked this on AWS EC2, it appears generating a keypair doesn't have the option to add a passphrase. The only way you could have a passphrase is if you generated the keys locally with a passphrase, then uploaded the public segment to AWS. So if you are actually using a private key created in the AWS EC2 console, then generating the public half with the command in this post shouldn't introduce any passphrase issue.

[khalilmebarkia]

#this path is for the public generated key from my server and I uploaded it to amazon**
Can you expand on this? IS this definately the public half of the same key. You can generate that from mykey.pem, with something like: ssh-keygen -f mykey.pem -y > id_rsa.pub (
According to this answer)

I tried but still nothing, it did not work and same error as before. I generated the public key exactly as you told me, uploaded it to Amazon.

If this is all correct, the only other thing I can think of is: does the private key have a passphrase? If that's the case perhaps NetConfServiceProvider is throwning the error because it can't handle the requred passphrase input.

Well, while creating EC2 instance, the private key is generated automatically.

Perhaps try generating your own (passphraseless) keypair, and upload the private section to amazon, then put the newly generated priv/pub pair into your script. That's assuming amazon doesn't allow you to generate a key without a passphrase within their interface.

You mean, I generate a new private key right ? the problem is mykey.pem is attached to all my instances in that VPC and having a new generated private key might get me in trouble because I don't want to loose the configuration in the routers. It happened to me once, and I had to redo everything over.

[vulcan25]

Sorry just to recap... Usually when connecting to something else with SSH keyauth, the client needs the private key, and the server has the public key. The client doesn't need the public key but this can be derived from the private key anyway. As @abhikeshav correctly pointed out, with NetConfServiceProvider you need both arguments, because one depends on the other. As far as I can tell this seems to be a quirk of this implementation of NetConfServiceProvider and nothing else.

There are two methods of creating a keypair on AWS:

• 1 Create Key Pair button, which immediatelly downloads the PEM file to your client/device, and retains the public key within AWS. With the PEM file on your device you can use it as a private or public key.
• 2 Import Key Pair button which requires you to upload a public key which is retained in AWS. If you've done this you should already have the private/PEM segment on the client/device.

I'm assuming you have followed method 1 in which case you wouldn't need to upload the public key to AWS, as you've already got mykey.pem and can extract the public key from that. This keypair could be auto-assigned to the instance when you create it.

Perhaps I'm degressing, I just wanted to be sure this wasn't a case of jumbled up keys.

---

The error you refer to earlier:

TypeError: "__init__(): incompatible constructor arguments. The following argument types are supported:
...ver/shared_files/mykey.pem', port=22, address='ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com'"

Has line 2 of this been manually snipped? If so could you possibly post the full trackback alongside latest set of arguments?

[ghost]

Her

Can you post your full script? You need all the arguments: hostname, username, private_key, public_key. See example

Here it is.

from ydk.services import CRUDService
from ydk.providers import NetconfServiceProvider
from ydk.models.cisco_ios_xr import Cisco_IOS_XR_shellutil_oper \
    as xr_shellutil_oper
from datetime import timedelta


if __name__ == "__main__":
    """Main execution path"""

    # create NETCONF session
    provider = NetconfServiceProvider(address="ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com", 
       username="ec2-user",                          
       private_key_path="/ydk-py/mykey.pem", #this path is for the private key from amazon**
       public_key_path="/root/.ssh/id_rsa.pub", #this path is for the public generated key from my server and I uploaded it to amazon**
                          port=22)

    # create CRUD service
    crud = CRUDService()

    # create system time object
    system_time = xr_shellutil_oper.SystemTime()

    # read system time from device
    system_time = crud.read(provider, system_time)

    # print system uptime
    print("System uptime is " +
          str(timedelta(seconds=system_time.uptime.uptime)))

    exit()

And I'm still getting the same error.

Can you please post the full error log from commandline?

[khalilmebarkia]

Can you please post the full error log from commandline?

@vulcan25 @abhikeshav Here it is.

 # create NETCONF session
    provider = NetconfServiceProvider(address="ec2-35-166-239-202.us-west-2.compute.amazonaws.com",
      username="ec2-user", port=22, public_key_path="/home/server/shared_files/id_rsa.pub",
      private_key_path="/home/server/shared_files/mykey.pem")

and the error log.

server@zsz:~/shared_files$ python hello-ydk.py 
Traceback (most recent call last):
  File "hello-ydk.py", line 15, in <module>
    private_key_path="/home/server/shared_files/mykey.pem")
RuntimeError: YClientError: Could not connect to ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com

and @vulcan25 you are right. Something wrong maybe with the implementation itself.

[ghost]

Can you please post the full error log from commandline?

@vulcan25 @abhikeshav Here it is.

 # create NETCONF session
    provider = NetconfServiceProvider(address="ec2-35-166-239-202.us-west-2.compute.amazonaws.com",
      username="ec2-user", port=22, public_key_path="/home/server/shared_files/id_rsa.pub",
      private_key_path="/home/server/shared_files/mykey.pem")

and the error log.

server@zsz:~/shared_files$ python hello-ydk.py 
Traceback (most recent call last):
  File "hello-ydk.py", line 15, in <module>
    private_key_path="/home/server/shared_files/mykey.pem")
RuntimeError: YClientError: Could not connect to ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com

and @vulcan25 you are right. Something wrong maybe with the implementation itself.

Thanks. Can you enable debug log and post the debug log?

import logging
log = logging.getLogger('ydk')
log.setLevel(logging.DEBUG)
handler = logging.StreamHandler()
formatter = logging.Formatter(("%(asctime)s - %(name)s - %(levelname)s - %(message)s"))
handler.setFormatter(formatter)
log.addHandler(handler)

[khalilmebarkia]

Can you please post the full error log from commandline?

Same error after putting the logging lib, and nothing is shown.

 server@zsz:~/shared_files$ python hello-ydk.py 
 Traceback (most recent call last):
  File "hello-ydk.py", line 15, in <module>
  private_key_path="/home/server/shared_files/mykey.pem")
 RuntimeError: YClientError: Could not connect to ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com
> > ```

[vulcan25]

EDIT: of course the following probably doesn't effect it because the cannot connect error is before any of this is called, but maybe something to be aware of for the next step.

---

Stab in the dark here: What version of the bundle did you install? Going by the Quick Install guidance:

You get a fully operational YDK environment by installing the cisco-ios-xr and/or cisco-ios-xe bundle(s) (depending on whether you're developing for an IOS XR or IOS XE platform)

The AMI you linked uses XE according to the product description:

This AMI runs Cisco IOS XE technology features (ASR1000 and ISR4000 series)

So you should have installed the compatible bundle:

pip install ydk-models-cisco-ios-xe

This should also be reflected in your script though. The sample appears to use XR.

from ydk.services import CRUDService
from ydk.providers import NetconfServiceProvider
from ydk.models.cisco_ios_xr import Cisco_IOS_XR_shellutil_oper \
    as xr_shellutil_oper
from datetime import timedelta

Maybe change this to:

from ydk.services import CRUDService
from ydk.providers import NetconfServiceProvider
from ydk.models.cisco_ios_xe import Cisco_IOS_XE_shellutil_oper \
    as xe_shellutil_oper
from datetime import timedelta

(I've just changed xr to xe here).

Then update the system_time line:

system_time = xe_shellutil_oper.SystemTime()

(Again, I'm assuming the naming convention is identical between XE/XE versions).

Maybe try a connect after this?

[ghost]

Can you please post the full error log from commandline?

Same error after putting the logging lib, and nothing is shown.

 server@zsz:~/shared_files$ python hello-ydk.py 
 Traceback (most recent call last):
  File "hello-ydk.py", line 15, in <module>
  private_key_path="/home/server/shared_files/mykey.pem")
 RuntimeError: YClientError: Could not connect to ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com
> > ```

Can you connect to your netconf server via command line (not using YDK)?
ssh user@address -p 830 -s netconf

[vulcan25]

@abhikeshav This AMI has console access on port 22, not 830. @khalilmebarkia confirmed earlier that he was able to access this via ssh.

I was so curious to know about this I spun up one of these CSR 1000v instances myself to check. Connecting on port 22 definitely takes you to an IOS console, and is accessible using the pem key of the keypair assigned when launching the EC2 instance. I'm still to get ydk compiled, then will test that portion myself.

[ghost]

@abhikeshav This AMI has console access on port 22, not 830. @khalilmebarkia confirmed earlier that he was able to access this via ssh.

I was so curious to know about this I spun up one of these CSR 1000v instances myself to check. Connecting on port 22 definitely takes you to an IOS console, and is accessible using the pem key of the keypair assigned when launching the EC2 instance. I'm still to get ydk compiled, then will test that portion myself.

Connecting to ssh terminal is different. Netconf is usually accessed on ssh via port 830. If he is able to connect to netconf on port 830 using the command line arguments above, then YDK should also work. See the RFC for details.

[khalilmebarkia]

from ydk.services import CRUDService from ydk.providers import NetconfServiceProvider from ydk.models.cisco_ios_xe import Cisco_IOS_XE_shellutil_oper \ as xe_shellutil_oper from datetime import timedelta

It is so weird because I deleted the line of calling XE libraries and deleted the functions below, and still the same error.

server@zsz:~/shared_files$ python hello-ydk.py 
Traceback (most recent call last):
  File "hello-ydk.py", line 13, in <module>
    private_key_path="/home/server/shared_files/mykey.pem")
RuntimeError: YClientError: Could not connect to ec2-35-166-239-202.us-west-2.compute.amazonaws.com
server@zsz:~/shared_files$ ```

[khalilmebarkia]

ssh user@address -p 830 -s netconf

I tried it and here what I wrote
ssh -i "oregon_ssh-key.pem" ec2-user@ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com -p 830 -s netconf
then I got
ssh: connect to host ec2-35-166-239-202.us-west-2.compute.amazonaws.com port 830: Connection refused
and with port 22
Line has invalid autocommand "netconf"Connection to ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com closed by remote host.

[vulcan25]

@abhikeshav I can also confirm this AMI image is defininately not serving anything on port 830 by default:

$ ssh ec2-user@<snipped-ip> -i ~/.ssh/tmp/TESTt.pem -p830 -s netconf
ssh: connect to host <snipped-ip> port 830: Connection refused

$ nmap <snipped-ip>
Starting Nmap 7.70 ( https://nmap.org ) at 2019-01-24 22:57 GMT
Nmap scan report for instanceID.eu-west-1.compute.amazonaws.com (<snipped-ip>)
Host is up (0.094s latency).
Not shown: 904 closed ports, 93 filtered ports
PORT    STATE SERVICE
22/tcp  open  ssh
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 2.76 seconds

Are there any Cisco commands I could run at the terminal to launch netconf on 830?

[ghost]

Please make sure you have netconf configured on your router. Try the below config

 
conf t
ssh server netconf vrf default
ssh server logging
netconf-yang agent
ssh
!
commit

[khalilmebarkia]

Please make sure you have netconf configured on your router. Try the below config

 
conf t
ssh server netconf vrf default
ssh server logging
netconf-yang agent
ssh
!
commit

Apparently, These commands are incorrect.

ip-172-0-1-9(config)#ssh?
% Unrecognized command

as well as the others

ip-172-0-1-9(config)#ssh server netconf vrf default
                      ^
% Invalid input detected at '^' marker.
ip-172-0-1-9(config)#netconf-yang 
ip-172-0-1-9(config)#netconf-yang ssh?
ssh  
ip-172-0-1-9(config)#netconf-yang agent
                                  ^
% Invalid input detected at '^' marker.

[khalilmebarkia]

Well, I decided to go for RESTCONF because of that auth problem, and in case of RESTCONF it goes over HTTP in which HTTP/HTTP port can be opened in AWS console.
For first step, I wanted to make sure to send a simple request using POSTMAN.

The simple sent request is the following:
https://xx.xx.xx.xx:430/restconf/api/config/native/interface/GigabitEthernet/
I added the private_key and CRT_file in Postman settings certificates, as well as the host.
I allowed HTTP/HTTPs traffic in Security Group of the VPC, and here what I have got:

<errors xmlns="urn:ietf:params:xml:ns:yang:ietf-restconf">
    <error>
        <error-tag>access-denied</error-tag>
        <error-type>protocol</error-type>
    </error>
</errors>

Obviously, the auth is still in here, too.

[111pontes]

For IOS XE, you should need at least the following configuration:

netconf-yang
ip ssh version 2
!
crypto key generate rsa

The router configuration mentioned earlier is for IOS XR devices.

Once the router is properly configured, make sure you can manually establish an SSH session on port 830 from your client and that the netconf subsystem is started:

$ ssh user@host -p830 -s netconf

You should receive a HELLO xml message with the device capabilities.

[khalilmebarkia]

netconf-yang ip ssh version 2 ! crypto key generate rsa

Did not work, I followed the configuration as you mentioned and when I tried to establish the SSH session, here what I got:

$ ssh -i mykey.pem ec2-user@ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com -p 830 -s netconf
ec2-user@ec2-xx-xx-xx-xx.us-west-2.compute.amazonaws.com's password:

I have to enter a password, and since I'm using a private key to access the router. It is the same case when I don't pass the private key.