UPnP port map fails on Mikrotik CHR v7.10 with a UPnPError #8364
Comments
|
cc @andrew-d |
|
Would you be able to fetch http://10.0.0.1:2828/gateway.xml on your local network and attach it here? |
DentonGentry
added
L1 Very few
|
Of course! Apologies for the massive delay, I had forgotten all about it. <?xml version="1.0"?>
<root xmlns="urn:schemas-upnp-org:device-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<device>
<deviceType>urn:schemas-upnp-org:device:InternetGatewayDevice:1</deviceType>
<friendlyName>MikroTik Router</friendlyName>
<manufacturer>MikroTik</manufacturer>
<manufacturerURL>https://www.mikrotik.com/</manufacturerURL>
<modelName>Router OS</modelName>
<UDN>uuid:UUID-MIKROTIK-INTERNET-GATEWAY-DEVICE-</UDN>
<iconList>
<icon>
<mimetype>image/gif</mimetype>
<width>16</width>
<height>16</height>
<depth>8</depth>
<url>/logo16.gif</url>
</icon>
<icon>
<mimetype>image/gif</mimetype>
<width>32</width>
<height>32</height>
<depth>8</depth>
<url>/logo32.gif</url>
</icon>
<icon>
<mimetype>image/gif</mimetype>
<width>48</width>
<height>48</height>
<depth>8</depth>
<url>/logo48.gif</url>
</icon>
</iconList>
<serviceList>
<service>
<serviceType>urn:schemas-microsoft-com:service:OSInfo:1</serviceType>
<serviceId>urn:microsoft-com:serviceId:OSInfo1</serviceId>
<SCPDURL>/osinfo.xml</SCPDURL>
<controlURL>/upnp/control/oqjsxqshhz/osinfo</controlURL>
<eventSubURL>/upnp/event/cwzcyndrjf/osinfo</eventSubURL>
</service>
</serviceList>
<deviceList>
<device>
<deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType>
<friendlyName>WAN Device</friendlyName>
<manufacturer>MikroTik</manufacturer>
<manufacturerURL>https://www.mikrotik.com/</manufacturerURL>
<modelName>Router OS</modelName>
<UDN>uuid:UUID-MIKROTIK-WAN-DEVICE--1</UDN>
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANCommonIFC1</serviceId>
<SCPDURL>/wancommonifc-1.xml</SCPDURL>
<controlURL>/upnp/control/ivvmxhunyq/wancommonifc-1</controlURL>
<eventSubURL>/upnp/event/mkjzdqvryf/wancommonifc-1</eventSubURL>
</service>
</serviceList>
<deviceList>
<device>
<deviceType>urn:schemas-upnp-org:device:WANConnectionDevice:1</deviceType>
<friendlyName>WAN Connection Device</friendlyName>
<manufacturer>MikroTik</manufacturer>
<manufacturerURL>https://www.mikrotik.com/</manufacturerURL>
<modelName>Router OS</modelName>
<UDN>uuid:UUID-MIKROTIK-WAN-CONNECTION-DEVICE--1</UDN>
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:WANIPConnection:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANIPConn1</serviceId>
<SCPDURL>/wanipconn-1.xml</SCPDURL>
<controlURL>/upnp/control/yomkmsnooi/wanipconn-1</controlURL>
<eventSubURL>/upnp/event/veeabhzzva/wanipconn-1</eventSubURL>
</service>
</serviceList>
</device>
</deviceList>
</device>
<device>
<deviceType>urn:schemas-upnp-org:device:WANDevice:1</deviceType>
<friendlyName>WAN Device</friendlyName>
<manufacturer>MikroTik</manufacturer>
<manufacturerURL>https://www.mikrotik.com/</manufacturerURL>
<modelName>Router OS</modelName>
<UDN>uuid:UUID-MIKROTIK-WAN-DEVICE--7</UDN>
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:WANCommonInterfaceConfig:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANCommonIFC1</serviceId>
<SCPDURL>/wancommonifc-7.xml</SCPDURL>
<controlURL>/upnp/control/vzcyyzzttz/wancommonifc-7</controlURL>
<eventSubURL>/upnp/event/womwbqtbkq/wancommonifc-7</eventSubURL>
</service>
</serviceList>
<deviceList>
<device>
<deviceType>urn:schemas-upnp-org:device:WANConnectionDevice:1</deviceType>
<friendlyName>WAN Connection Device</friendlyName>
<manufacturer>MikroTik</manufacturer>
<manufacturerURL>https://www.mikrotik.com/</manufacturerURL>
<modelName>Router OS</modelName>
<UDN>uuid:UUID-MIKROTIK-WAN-CONNECTION-DEVICE--7</UDN>
<serviceList>
<service>
<serviceType>urn:schemas-upnp-org:service:WANIPConnection:1</serviceType>
<serviceId>urn:upnp-org:serviceId:WANIPConn1</serviceId>
<SCPDURL>/wanipconn-7.xml</SCPDURL>
<controlURL>/upnp/control/xstnsgeuyh/wanipconn-7</controlURL>
<eventSubURL>/upnp/event/rscixkusbs/wanipconn-7</eventSubURL>
</service>
</serviceList>
</device>
</deviceList>
</device>
</deviceList>
<presentationURL>http://10.0.0.1/</presentationURL>
</device>
<URLBase>http://10.0.0.1:2828</URLBase>
</root> |
Unfortunately in the test we can't reproduce the failure seen
in the real system ("SOAP fault: UPnPError")
Updates #8364
Signed-off-by: Denton Gentry <dgentry@tailscale.com>
|
The error condition has changed slightly with later versions of tailscale: |
Unfortunately in the test we can't reproduce the failure seen
in the real system ("SOAP fault: UPnPError")
Updates #8364
Signed-off-by: Denton Gentry <dgentry@tailscale.com>
|
@tylerjwatson - One more question, if you don't mind: could you run |
|
Done mate. Doesn't appear that the log contains any sensitive information so I'll also attach it here. Another interesting tidbit is the tw@pve:~$ ip route
default via 10.0.0.1 dev vmbr1 proto kernel onlink
10.0.0.0/24 dev vmbr1 proto kernel scope link src 10.0.0.6
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev br-903ffb19a348 proto kernel scope link src 172.18.0.1 May be triggering the 404 on the mikrotik side? |
Yeah, that's not right 😓 What happens if you specify the correct IP/gateway manually? (or whatever your real self IP is) |
|
No discernable change. |
|
@tylerjwatson First off, thanks for the log and the route table; this actually surfaced an older bug that appears to be unrelated, but still isn't great (fix in #10467).
As for your actual issue... this is pretty puzzling to me. Your Mikrotik is telling us that the UPnP endpoint is at |
|
Will do soon. Feel free to flick me a public key if you want to have a play around for yourself, could probably just port forward 2828 over the tunnel and see what's going on if it helps |
Unfortunately in the test we can't reproduce the failure seen
in the real system ("SOAP fault: UPnPError")
Updates #8364
Signed-off-by: Denton Gentry <dgentry@tailscale.com>
|
I spent tonight hacking on It appears to choose an IGD by probing it with a <?xml version="1.0"?>
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:AddPortMapping xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1">
<NewRemoteHost></NewRemoteHost>
<NewExternalPort>41641</NewExternalPort>
<NewProtocol>UDP</NewProtocol>
<NewInternalPort>41641</NewInternalPort>
<NewInternalClient>10.0.0.6</NewInternalClient>
<NewEnabled>1</NewEnabled>
<NewPortMappingDescription>libminiupnpc</NewPortMappingDescription>
<NewLeaseDuration>0</NewLeaseDuration>
</u:AddPortMapping>
</s:Body>
</s:Envelope>To which the server responds: <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<s:Body>
<u:AddPortMappingResponse xmlns:u="urn:schemas-upnp-org:service:WANIPConnection:1"></u:AddPortMappingResponse>
</s:Body>
</s:Envelope>what's interesting is tailscale chooses a different endpoint. In the router I've only set one external interface (which has the public address on it) so that is quite perplexing as to why there are two. |
Sure, if you don't mind:
Yeah, it looks like it first tries to check whether a given UPnP "device" is connected ( tailscale/net/portmapper/upnp.go Lines 248 to 256 in 137e9f4 Looks like something we could add without too much trouble, I think, assuming I can actually test it 😅 Access to a test device would be super helpful, if that's okay with you! |
Previously, we would select the first WANIPConnection2 (and related) client from the root device, without any additional checks. However, some routers expose multiple UPnP devices in various states, and simply picking the first available one can result in attempting to perform a portmap with a device that isn't functional. Instead, mimic what the miniupnpc code does, and prefer devices that are (a) reporting as Connected, and (b) have a valid external IP address. For our use-case, we additionally prefer devices that have an external IP address that's a public address, to increase the likelihood that we can obtain a direct connection from peers. Finally, we split out fetching the root device (getUPnPRootDevice) from selecting the best service within that root device (selectBestService), and add some extensive tests for various UPnP server behaviours. Updates #8364 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I71795cd80be6214dfcef0fe83115a5e3fe4b8753
Previously, we would select the first WANIPConnection2 (and related) client from the root device, without any additional checks. However, some routers expose multiple UPnP devices in various states, and simply picking the first available one can result in attempting to perform a portmap with a device that isn't functional. Instead, mimic what the miniupnpc code does, and prefer devices that are (a) reporting as Connected, and (b) have a valid external IP address. For our use-case, we additionally prefer devices that have an external IP address that's a public address, to increase the likelihood that we can obtain a direct connection from peers. Finally, we split out fetching the root device (getUPnPRootDevice) from selecting the best service within that root device (selectBestService), and add some extensive tests for various UPnP server behaviours. Updates #8364 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I71795cd80be6214dfcef0fe83115a5e3fe4b8753
Previously, we would select the first WANIPConnection2 (and related) client from the root device, without any additional checks. However, some routers expose multiple UPnP devices in various states, and simply picking the first available one can result in attempting to perform a portmap with a device that isn't functional. Instead, mimic what the miniupnpc code does, and prefer devices that are (a) reporting as Connected, and (b) have a valid external IP address. For our use-case, we additionally prefer devices that have an external IP address that's a public address, to increase the likelihood that we can obtain a direct connection from peers. Finally, we split out fetching the root device (getUPnPRootDevice) from selecting the best service within that root device (selectBestService), and add some extensive tests for various UPnP server behaviours. Updates #8364 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I71795cd80be6214dfcef0fe83115a5e3fe4b8753
You'll find some goodies in the home dir. If you need more resources in the container let me know. |
Previously, we would select the first WANIPConnection2 (and related) client from the root device, without any additional checks. However, some routers expose multiple UPnP devices in various states, and simply picking the first available one can result in attempting to perform a portmap with a device that isn't functional. Instead, mimic what the miniupnpc code does, and prefer devices that are (a) reporting as Connected, and (b) have a valid external IP address. For our use-case, we additionally prefer devices that have an external IP address that's a public address, to increase the likelihood that we can obtain a direct connection from peers. Finally, we split out fetching the root device (getUPnPRootDevice) from selecting the best service within that root device (selectBestService), and add some extensive tests for various UPnP server behaviours. Updates #8364 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I71795cd80be6214dfcef0fe83115a5e3fe4b8753
|
@tylerjwatson Great! Just confirmed that the changes in #10489 work for your Mikrotik; it was able to probe both URLs and pick the working one, then successfully obtain a portmapping. I left the log in the homedir as Once that PR is reviewed + merged, I'll build an unstable release and leave another comment here, and it'll end up in the stable release 1.58, which we'll build sometime in early 2024. Also: I really appreciate the help with debugging this; it was great. Folks like you are a maintainers' dream–so thanks 😃 |
Previously, we would select the first WANIPConnection2 (and related) client from the root device, without any additional checks. However, some routers expose multiple UPnP devices in various states, and simply picking the first available one can result in attempting to perform a portmap with a device that isn't functional. Instead, mimic what the miniupnpc code does, and prefer devices that are (a) reporting as Connected, and (b) have a valid external IP address. For our use-case, we additionally prefer devices that have an external IP address that's a public address, to increase the likelihood that we can obtain a direct connection from peers. Finally, we split out fetching the root device (getUPnPRootDevice) from selecting the best service within that root device (selectBestService), and add some extensive tests for various UPnP server behaviours. Updates #8364 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I71795cd80be6214dfcef0fe83115a5e3fe4b8753
Compliment much appreciated - a bit of team work always works out the best, and likewise; It's easy to just ignore annoyance issues like these but it was picked up straight away. Massive kudos to you. Edit: I'll leave the container open for now in case the story develops as the unstable release gets ...well, released. Happy to run a dev build in the meantime. |
Previously, we would select the first WANIPConnection2 (and related) client from the root device, without any additional checks. However, some routers expose multiple UPnP devices in various states, and simply picking the first available one can result in attempting to perform a portmap with a device that isn't functional. Instead, mimic what the miniupnpc code does, and prefer devices that are (a) reporting as Connected, and (b) have a valid external IP address. For our use-case, we additionally prefer devices that have an external IP address that's a public address, to increase the likelihood that we can obtain a direct connection from peers. Finally, we split out fetching the root device (getUPnPRootDevice) from selecting the best service within that root device (selectBestService), and add some extensive tests for various UPnP server behaviours. RELNOTE=Improve UPnP portmapping on devices with multiple interfaces Updates #8364 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I71795cd80be6214dfcef0fe83115a5e3fe4b8753
Previously, we would select the first WANIPConnection2 (and related) client from the root device, without any additional checks. However, some routers expose multiple UPnP devices in various states, and simply picking the first available one can result in attempting to perform a portmap with a device that isn't functional. Instead, mimic what the miniupnpc code does, and prefer devices that are (a) reporting as Connected, and (b) have a valid external IP address. For our use-case, we additionally prefer devices that have an external IP address that's a public address, to increase the likelihood that we can obtain a direct connection from peers. Finally, we split out fetching the root device (getUPnPRootDevice) from selecting the best service within that root device (selectBestService), and add some extensive tests for various UPnP server behaviours. RELNOTE=Improve UPnP portmapping when multiple UPnP services exist Updates #8364 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I71795cd80be6214dfcef0fe83115a5e3fe4b8753
Previously, we would select the first WANIPConnection2 (and related) client from the root device, without any additional checks. However, some routers expose multiple UPnP devices in various states, and simply picking the first available one can result in attempting to perform a portmap with a device that isn't functional. Instead, mimic what the miniupnpc code does, and prefer devices that are (a) reporting as Connected, and (b) have a valid external IP address. For our use-case, we additionally prefer devices that have an external IP address that's a public address, to increase the likelihood that we can obtain a direct connection from peers. Finally, we split out fetching the root device (getUPnPRootDevice) from selecting the best service within that root device (selectBestService), and add some extensive tests for various UPnP server behaviours. RELNOTE=Improve UPnP portmapping when multiple UPnP services exist Updates #8364 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I71795cd80be6214dfcef0fe83115a5e3fe4b8753
|
@tylerjwatson - We just built an unstable Tailscale build (1.57.25) which contains the fix for this issue; when you get a moment, would you mind installing that and checking if it works? If so, we can close this issue as completed 😄 |
|
Can do, I will have to replicate the exact conditions tonight when I get home, as my WANscape(ugh) has changed slightly Reading over the patches, come to think of it, instead of selecting the first external gateway, wouldn't it be better to add portmaps to all endpoints that are offered and externally reachable in the nodemap? |
Yep, that's something I want to work on down the road. It's a bit complicated right now since much of the codebase assumes we have a single port mapping, so there's a bit of refactoring required. That's definitely the "right" longer-term solution though. |
What is the issue?
UPNP fails to map ports from a Mikrotik CHR using PnP fails with a UPnP error:
Steps to reproduce
No response
Are there any recent changes that introduced the issue?
No response
OS
Linux, macOS, Windows, Android
OS version
Archlinux
Tailscale version
1.42.0-dev20230524
Other software
No response
Bug report
BUG-ee4bbbf9cbe5d3499aea8dc1c3bb73ea626a554b7838ef8a0aa6b988964f7bf4-20230617120209Z-eae3a7b4231b6bf8
The text was updated successfully, but these errors were encountered: