Ubiquiti EdgeOS Authentication - no acceptable ecdsa/rsa-pss (almost there)

[0x-2a]

Update: was able to get it partially working to authenticate and receive only (under 15mbps). Still a work in progress.

I'm nearly able to authenticate ikev2 StrongSwan client (Linux strongSwan U5.6.3/K4.9.79-UBNT) from EdgeOS on Ubiquiti routers, which I believe is a Debian Linux variant similar to Vyatta/VyOS. Having read much of the docs and many issues here, I have nearly achieved connectivity but am stumped by issues similar to #930. Though the solution there doesn't work here (authby=sha1).

My goal is to make a wiki doc with how to achieve an all-traffic vpn from these routers. Given the growing popularity of Ubiquiti routers along with many unresolved forum discussions on community.ui.com, reddit.com, serverfault.com, and others trying to figure out similar vpn client configs with little success. Hopefully this post can also be a reference point for others.
Below is my current work in progress doc . This is between the EdgeOS v2.0.9 router (Ubiquiti ER-4) and an AWS instance running Amazon Linux 2 AMI.

EdgeOS -> VPN Server Tunnel (IKEv2) Setup Steps

Assumptions (change these for yourself if you follow this guide)

• VPN Server IP (AWS instance) :99.99.99.99
• Router Internals
  • WAN IP: 22.22.22.22
  • Router LAN IP: 192.168.1.1
  • LAN Subnet: 192.168.1.0/24
  • VPN tunnel-assigned local ip 192.168.43.11
  • Interfaces
    • eth0 WAN to internet
    • eth1 lan (not bridged)
  • VPN Instance Creds: ec2-user@99.99.99.99

# Pull latest build as of May 20, 2022
wget https://get.vpnsetup.net -nv -O vpn.sh 

sudo \
VPN_IPSEC_PSK='foo-bar-key-here' \
VPN_USER='foo-user' \
VPN_PASSWORD='foo-bar-password-here' \
sh vpn.sh

# Edit /etc/ipsec.d/ikev2.conf
vi /etc/ipsec.d/ikev2.conf
  # search for conn ikev2-cp section and add

  authby=rsa-sha1

# Restart ipsec
sudo service ipsec restart

# Extract the RSA X509 certs for the ipsec service, no password added
openssl pkcs12 -in vpnclient.p12 -cacerts -nokeys -out ikev2vpnca.cer
openssl pkcs12 -in vpnclient.p12 -clcerts -nokeys -out vpnclient.cer
openssl pkcs12 -in vpnclient.p12 -nocerts -nodes  -out vpnclient.key

Now copy the keys to local machine

scp ec2-user@99.99.99.99:vpnclient.cer ~/Desktop/.
scp ec2-user@99.99.99.99:ikev2vpnca.cer ~/Desktop/.
scp ec2-user@99.99.99.99:vpnclient.key ~/Desktop/.

# Test mobileconfig on mac/iphone first, works OK!

And copy the certs to the router

scp ~/Desktop/vpnclient.cer admin@192.168.1.1:/config/.
scp ~/Desktop/ikev2vpnca.cer admin@192.168.1.1:/config/.
scp ~/Desktop/vpnclient.key admin@192.168.1.1:/config/.

Now ssh into the router via ssh admin@192.168.1.1

cd /config

# restrict perms
sudo chown root.root ikev2vpnca.cer vpnclient.cer vpnclient.key
sudo chmod 600 ikev2vpnca.cer vpnclient.cer vpnclient.key

# Install certs
sudo mv ikev2vpnca.cer /etc/ipsec.d/cacerts/.
sudo mv vpnclient.cer /etc/ipsec.d/certs/.
sudo mv vpnclient.key /etc/ipsec.d/private/.

Make a new file at /config/ipsec-updown.sh and add the following. Use vim vi or whatever text editor you like that's on the os.

export VTI_IFACE="vti0"

case "${PLUTO_VERB}" in
    up-client)
        echo "Creating tunnel interface ${VTI_IFACE}"
        ip tunnel add "${VTI_IFACE}" local "${PLUTO_ME}" remote "${PLUTO_PEER}" mode vti

        echo "Activating tunnel interface ${VTI_IFACE}"
        ip link set "${VTI_IFACE}" up

        echo "Adding ${PLUTO_MY_SOURCEIP} to ${VTI_IFACE}"
        ip addr add "${PLUTO_MY_SOURCEIP}" dev "${VTI_IFACE}"

        echo "Disabling IPsec policy (SPD) for ${VTI_IFACE}"
        sysctl -w "net.ipv4.conf.${VTI_IFACE}.disable_policy=1"

        echo "Identified default route as ${DEFAULT_ROUTE}"
        echo "Adding route: ${PLUTO_PEER} via ${DEFAULT_ROUTE} dev ${PLUTO_INTERFACE}"
        ip route add "${PLUTO_PEER}" via "${DEFAULT_ROUTE}" dev "${PLUTO_INTERFACE}"
        ;;
    down-client)
        echo "Deleting interface ${VTI_IFACE}"
        ip tunnel del "${VTI_IFACE}"

        echo "Deleting route for ${PLUTO_PEER}"
        ip route del "${PLUTO_PEER}"
        ;;
esac

Make the above file executable.

chmod +x /config/ipsec-updown.sh

Later should see the echos above in

cat /var/log/charon.log

Make a new file /config/ipsec.conf and add the following. I'll make an attempt at trying to explain what I think these options mean.

# https://wiki.strongswan.org/projects/strongswan/wiki/ConnSection
# Left is local, right is remote
# 
# Always `sudo ipsec restart` after edits. 
conn vpnclient
 keyexchange=ikev2
 left=22.22.22.22             # requests IP from remote
 leftid=vpnclient       # matches id in ipsec.secrets below
 leftsourceip=%config4  # requests tunnel IP
 leftsubnet=0.0.0.0/0  # restrict participant mask, allow all
 leftauth=pubkey        # uses certs
 leftcert=vpnclient.cer # the public key
 leftsendcert=always    # don't wait to be asked for it
 right=99.99.99.99       # remote pub net interface
 rightid=99.99.99.99     # hardcoded for static ip
 rightsubnet=0.0.0.0/0  # wide open
 rightauth=pubkey       # uses certs
 rightdns=8.8.8.8,8.8.4.4 # google dns
 esp=aes128gcm16        # from https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/ikev2-howt
 # TODO there is no hardware support for aes128gcm16, test other methods
 # see https://help.ui.com/hc/en-us/articles/115006567467-EdgeRouter-Hardware-Offloading
 auto=add             # bring up manually with `sudo ipsec up vpnclient`
 dpdaction=clear        # remove conns when they break
 dpddelay=10s           # how frequently to send keepalives
 leftupdown=/config/ipsec-updown.sh # complete the ip tunneling wiring eth1 to vti0

In the router's /config/ipsec.secrets put the following. vpnclient references the leftid above.

# https://wiki.strongswan.org/projects/strongswan/wiki/Ipsecsecrets
vpnclient : RSA vpnclient.key

Alter the EdgeOS vpn, firewall, interfaces, and services to direct all traffic to the vpn.

configure

# Register our certs and conf with ipsec.
set vpn ipsec auto-firewall-nat-exclude enable
set vpn ipsec include-ipsec-conf /config/ipsec.conf
set vpn ipsec include-ipsec-secrets /config/ipsec.secrets
set vpn ipsec global-config "charon.install_routes := no"
set vpn ipsec global-config "charon.install_virtual_ip := no"

# Make a virtual tunnel
set interfaces vti vti0 address 192.168.43.11/32 # the expected VPN tunnel client IP, may change
set interfaces vti vti0 mtu 1436 # different subnet than LAN

# Route all traffic to the tunnel. 
set protocols static table 1 interface-route 0.0.0.0/0 next-hop-interface vti0

commit ;

# Set firewall rules
set firewall modify SOURCE_ROUTE rule 1 action modify
set firewall modify SOURCE_ROUTE rule 1 description "IPSec-only policy"
set firewall modify SOURCE_ROUTE rule 1 modify table 1
set firewall modify SOURCE_ROUTE rule 1 source address 192.168.1.0/24

# Interface using IPsec tunnel
set interfaces ethernet eth1 firewall in modify SOURCE_ROUTE 
#set interfaces ethernet eth2 firewall in modify SOURCE_ROUTE 
#set interfaces ethernet eth3 firewall in modify SOURCE_ROUTE 
#set interfaces bridge br0 firewall in modify SOURCE_ROUTE # TODO if you did bridge? 

commit ;

# Drop connections if not VPN tunneled
set firewall name WAN_OUT default-action accept
set firewall name WAN_OUT description "Outgoing connections"
set firewall name WAN_OUT rule 1 action drop
set firewall name WAN_OUT rule 1 description VPN_FAIL_DENY
set firewall name WAN_OUT rule 1 log disable
set firewall name WAN_OUT rule 1 protocol all
set firewall name WAN_OUT rule 1 source address 192.168.1.0/24
commit ;


# Set NAT rules
set service nat rule 5000 description "masquerade for vti0"
set service nat rule 5000 log disable
set service nat rule 5000 outbound-interface vti0
set service nat rule 5000 protocol all
set service nat rule 5000 type masquerade
commit ;

# Enable hardware offloading
set system offload ipsec enable

commit ; save ; exit

# A reboot may be necessary if save didn't apply immediately, you'll know below if listcerts doesn't show your cert.
sudo reboot

Get ready to connect. Out on the AWS server, start watching logs in a different ssh session to debug.

ssh ec2-user@99.99.99.99

sudo tail /var/log/secure | grep --line-buffered pluto

Update ipsec, run, and test the connection

# Make sure ipsec pulls in new certs
sudo ipsec rereadall

# Check that the certs were loaded
sudo ipsec listcerts
sudo ipsec listcacerts

# Restart ipsec to pick up changes in ipsec.conf
sudo ipsec restart
sudo ipsec up vpnclient # not necessary if you have auto=start above
sudo ipsec statusall

---

Where I'm currently stuck (most recent update)

I've got the tunnel working with the above for just receiving only. And it is rather slow (max 15mbps).

Feel free to contribute, otherwise I'll leave this here for others to work off of.

Stuck on no acceptable edsa rsa (update, got this part working)

With all the above, the resulting error is no acceptable signature hash.

  proposal 1:IKE=AES_CBC_256-HMAC_SHA2_256-HMAC_SHA2_256_128-MODP2048 chosen from remote proposals
  sent IKE_SA_INIT reply {cipher=AES_CBC_256 integ=HMAC_SHA2_256_128 prf=HMAC_SHA2_256 group=MODP2048}
  processing decrypted IKE_AUTH request: SK{IDi,CERT,N,CERTREQ,AUTH,CP,SA,TSi,TSr,N,N,N,N}
  ignoring CERTREQ payload that is not ASN1
  switched to "ikev2-cp"[65] <my pub ip>

- authentication failed: no acceptable ECDSA/RSA-PSS ASN.1 signature hash proposal included for rsasig
- responding to IKE_AUTH message (ID 1) from <my pub ip>:1024 with encrypted notification AUTHENTICATION_FAILED

I have tried setting authby=sha1, however that flag seems invalid (from #930)

Starting strongSwan 5.6.3 IPsec [starter]...
# bad value: authby=sha1
  bad argument value in conn 'vpnclient'
# ignored conn 'vpnclient' due to 1 parsing error
### 1 parsing error (0 fatal) ###

I have tried opening up ike and esp to more ciphers, however no change (from #1122).

# Couldn't find docs on allowed values, maybe https://docs.strongswan.org/docs/5.9/config/IKEv2CipherSuites.html
# or https://github.com/nm-l2tp/NetworkManager-l2tp/blob/main/README.md
vi /config/ipsec.conf
  ...
  esp=aes128gcm16 # it seems the VPN server likes this value, otherwise I'll see `INVALID_KE_PAYLOAD requesting MODP2048`
  ike=aes256-sha2_256-modp2048,aes256-sha2_256-modp1536,aes256-sha2_256-modp1024,aes256-sha1-modp2048,aes256-sha1-modp1536,aes256-sha1-modp1024,aes256-sha1-ecp384,aes128-sha1-modp1024,aes128-sha1-ecp256,3des-sha1-modp2048,3des-sha1-modp1024
  ...
  :wq
sudo ipsec restart
sudo ipsec up vpnclient
...

...

Where to from here?

---

Resources

Docs

• https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/clients.md#debian-1110-kernel

• https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/ikev2-howto.md#android

• https://github.com/hwdsl2/setup-ipsec-vpn/blob/master/docs/ikev2-howto.md#routeros

Related Issues

• Document how to setup an ikev2 client on Linux using the CLI #1060

• no Rx traffic but there is Tx traffic (IKEV2) #1122

• Ubuntu client cannot use IKEv2 VPN DNS #1077

• AUTHENTICATION_FAILED: No acceptable ECDSA/RSA-PSS ASN.1 signature hash proposal included for rsasig in I2 Auth Payload #930

External

• ProtonVPN IKEv2 client config https://community.ui.com/questions/ProtonVPN-IKEv2-client-configuration-for-EdgeRouter/c2b64fa4-9eac-4aa4-b854-9ec9949a7e11

• Debian strongswan tunneling https://blog.sys4.de/routing-based-vpn-with-strongswan-de.html

• EdgeOS with AlgoVPN (doesn't work anymore) https://github.com/kiratp/algo/blob/master/docs/client-edgemax.md

• Ubiquiti forums ikev2 for nordvpn (with 11k views) https://community.ui.com/questions/IPSec-to-NordVPN-IKEv2-EAP-MSCHAPv2/9b2615c4-960b-4563-afa2-aeeefee3fd48#answer/900f60de-fcad-45d5-a9e4-ccac764332a9

• Ubiquiti site-to-site VTI over IKEv2 https://help.ui.com/hc/en-us/articles/115012305347-EdgeRouter-Route-Based-Site-to-Site-VPN-to-Azure-VTI-over-IKEv2-IPsec-

• MARC Strong Swang boards https://marc.info/?l=strongswan-users&m=147066200415239&w=2

• Reddit ramblings, unhelpful mixed bag

  • https://www.reddit.com/r/Ubiquiti/comments/acsex2/ikev2ipsec_vpn_on_edgerouter/

  • https://www.reddit.com/r/Ubiquiti/comments/somr5s/edgerouter_ike2ipsec_nordvpn_client/

  • https://www.reddit.com/r/Ubiquiti/comments/93pgj1/edgerouter_x_setting_up_vpn/

  • https://www.reddit.com/r/meraki/comments/9kzyqu/use_ubiquiti_edgerouter_x_to_get_ikev2/

  • https://www.reddit.com/r/Ubiquiti/comments/cui1wj/edgerouter_4_mdnsrepeater_ipsec_sitetosite_vpn/

[hwdsl2]

@y3sh Hello! Thank you for the detailed issue description. The error you mentioned, authentication failed: no acceptable ECDSA/RSA-PSS ASN.1 signature hash proposal included for rsasig, can be fixed using authby=rsa-sha1. This option should be added on your VPN server, not on the client.

Edit /etc/ipsec.d/ikev2.conf on your VPN server. Append authby=rsa-sha1 to the end of conn ikev2-cp section, indented by two spaces. Save the file and run sudo service ipsec restart.

Let us know if this fixes the issue.

[0x-2a]

@COI-03 I just use the diff keyword after three back-ticks

```diff
    - minus is red
    + is green
```

[0x-2a]

Thanks @hwdsl2

It seems to be connected now, with ipsec status showing vpnclient[1]: ESTABLISHED 20 minutes ago .... However, on the server it throws the error:

"ikev2-cp"[18] x.x.x.x #30: established IKE SA; authenticated using RSA with SHA1 and peer certificate 'CN=vpnclient, O=IKEv2 VPN' issued by CA 'CN=IKEv2 VPN CA, O=IKEv2 VPN'
"ikev2-cp"[18] x.x.x.x #31: proposal 1:ESP=AES_GCM_C_128-DISABLED SPI=c4360aab chosen from remote proposals 1:ESP:ENCR=AES_GCM_C_128;ESN=DISABLED[first-match] 2:ESP:ENCR=AES_CBC_128;ENCR=AES_CBC_192;ENCR=AES_CBC_256;INTEG=HMAC_SHA2_256_128;INTEG=HMAC_SHA2_384_192;INTEG=HMAC_SHA2_512_256;INTEG=HMAC_SHA1_96;INTEG=AES_XCBC_96;ESN=DISABLED
"ikev2-cp"[18] x.x.x.x #31: established Child SA using #30; IPsec tunnel [0.0.0.0-255.255.255.255:0-65535 0] -> [192.168.43.10-192.168.43.10:0-65535 0] {ESPinUDP=>0xc4360aab <0x210769d5 xfrm=AES_GCM_16_128-NONE NATD=x.x.x.x:4500 DPD=active}
"ikev2-cp"[18] x.x.x.x #31: ESP traffic information: in=0B out=0B
"ikev2-cp"[18] x.x.x.x: kernel: xfrm XFRM_MSG_DELPOLICY for flow %discard(discard)(in) encountered unexpected policy
--ERROR: "ikev2-cp"[18] x.x.x.x #31: kernel: xfrm XFRM_MSG_DELPOLICYdelete(UNUSED) response for flow (in): No such file or directory (errno 2)

While this appears to be connected, I'm unable to route traffic through the tunnel. I'm not sure what the cause is yet -- likely I misconfigured a NAT or firewall policy. If there's something else obvious here I could try it, otherwise we can probably close the ticket as there is more investigation to do on my end.

[hwdsl2]

@y3sh Thanks for the update. The error you highlighted, kernel: xfrm XFRM_MSG_DELPOLICYdelete(UNUSED) response for flow (in): No such file or directory (errno 2), is a known issue with Libreswan 4.6. It generally should not affect the functionality of the VPN, and is to be fixed in the upcoming Libreswan version. For the other issue you mentioned, "unable to route traffic through the tunnel", it could be your firewall policy or IPTables issues on the VPN server (less likely). Please investigate further yourself.

[0x-2a]

@hwdsl2 I was able to get the error resolved by changing left=%any to have my WAN IP instead of %any. Because the "NO Child SA" error exits with code 1, the leftupdown script in ipsec.conf never gets run, preventing the final ip tunnel config connecting ethernet to the vti.

A couple new issues remain though, am only able to receive and rather slowly. Both I feel are still probably out of scope of this repo though.