CNetwork - implement IPv6 #7030

mk01 · 2015-04-29T00:13:16Z

rewrite of network class(es) for abstracted universal ipv6/v4 support

xbmc/network/Network.cpp

@@ -140,6 +141,25 @@ CNetwork::~CNetwork()
  CApplicationMessenger::Get().NetworkMessage(SERVICES_DOWN, 0);
 }

+//Convert a struct sockaddr address to a string, IPv4 and IPv6:
+char *CNetwork::GetIpStr(const struct sockaddr *sa, char *s, size_t maxlen)


FernetMenta · 2015-04-30T06:19:37Z

I'd like to consider this for Isengard. Objections?

Jalle19 · 2015-04-30T07:23:01Z

I don't mind

Paxxi · 2015-04-30T12:21:58Z

I've only gone over this quickly but I'm a bit concerned about how it affects CURL::Parse(we try to find port number by searching for : from the start of the string) and URIUtils::IsHostOnLAN . Has this been tested properly? Also are there changes needed on the windows side to match? I only see changes for NetworkLinux and general
This is not my area of expertise but I vote no for isengard.(yes for J* though)

FernetMenta · 2015-04-30T12:49:24Z

@Paxxi so no expert judgment :)

Paxxi · 2015-04-30T13:13:08Z

nope, I leave that to the experts :)

MartijnKaijser · 2015-05-17T19:35:42Z

@FernetMenta @Jalle19 is this still acceptable for 15.0?
jenkins build this please

Jalle19 · 2015-05-18T05:05:26Z

@Paxxi's comments haven't been answered yet so I vote no.

mk01 · 2015-05-18T10:44:08Z

@Jalle19 now I'm not really sure if mine involvement was expected/asked or not. your last comment triggers such feelings from my side but personally I was still waiting for the 'experts judgement' by a 3rd party - not myself.

as the original comment was saying - that's bare minimum for start for you (multi-platform code) - but is enough for deploying linux distros - thus I would not be rewriting the rest because I feel bored.

I'll be happy to continue and work out other aspects too, just naturally won't be doing that on my own without prior discussion with maintainer's and setting/agreeing on some basics (before other and other code will be rewritten)

Memphiz · 2015-05-18T13:27:44Z

I think it might be a bit too late for isengard at this point.

It looks like it alters behavior of name resolution
It doesn't compile on the other platforms yet:
android arm&x86:
NetworkLinux.cpp:31:23: fatal error: ifaddrs.h: No such file or directory
#include <ifaddrs.h>

iOS & OSX:
/Users/Shared/jenkins/workspace/IOS-ATV2/xbmc/network/linux/NetworkLinux.cpp:128:45: error: use of undeclared identifier 'IFF_LOWER_UP'
m_ifa_flags & (IFF_UP | IFF_LOWER_UP)));

Windows:
DNSNameCache.cpp(64): error C3861: 'inet_aton': identifier not found

After it compiled it might need some more user testing (i mean - we should merge it after I* release and see how feedback is over some months).

mk01 · 2015-05-18T21:53:00Z

@Memphiz

thanks for taking the lead.

I can look (and will) on to osx and (at least code edit without ability to test) the x86/android. hat is
(what is the x86 you build on ? as ifaddrs.h is default part of libc6-dev and deployed in /usr/include. this is valid for wheezy(any arch), jessie(any arch), ubuntu since 12.04...)

(are there some specialties to install for XCode to get master compiling? according to readme.osx it is "just working")

in regards to Windows - unfortunately my last know how is from dos3 and gwbasic.
I just did some MS docs reading before I used those new function calls and type defs. According to it they are supported almost 1:1 in something called "ansi" mode. I looked on examples attached too - some differences in types looked like it is all interfaced somehow to the vc++ containers / native calls. so I assumed win32 guy should be fine.

I will ping again when I push some updates.
btw - if you are fine with the new approach in NetworkLinux.cpp I would re-check & re-adapt whole class (where needed). Speaking of Curl interfacing - curl internals are fine parsing ipv6 ips.
the ipv6 "style" putting ip between [] was chosen to avoid conflicts with ipv4 :ports definitions (and make vintage code work without changes) but I wasn't sure about that when quick looking at CURL::Parse (already pointed before). I will follow up on this.

Honestly - I don't know EXACT workflows in the Network code (yet) - specifically if CURL::Parse is something used for ANY fs/net/xx "destination" parsing or not (meaning what to test in details) - but speaking about it because I already rolled out this commit into stable XBian builds those 20 days ago. that means ANY conflicts/problems will for sure be reported sooner or later.
Also it was merged into Sam's OSMC R2 or R3. ((the message told here is not that code is already bullet-prove, but that it is undergoing wider test already than me and you Memphis)).

mk01 · 2015-05-18T22:00:40Z

ach forgot the important - quoting you

1. It looks like it alters behavior of name resolution

I was targeting no change to outside. Can you go into details what I missed ?

Paxxi · 2015-05-19T06:16:42Z

The choice of going with [] for ipv6 addresses is correct I think as it's commonly used. our CURL class should be fixed in a separate pr I think to add ipv6 parsing and unit testing for it before this PR goes in.

@mk01 we have CURL and Curl, CURL is our path/url class that handles local and remote paths of any kind. Curl is well it's curl as we all know and love :) just to clarify which one we're talking about it's our internal CURL that I'm worried about and it needs ipv6 parsing anyhow.

I can help out with whatever win32 changes are needed once this is in shape to be tested.

To further clarify the win32api has two methods of anything that takes a string one ending with A(ansi) and one with W(unicode/widestring), When docs mention ANSI they mean the method takes string in ascii/oem codepage and the network methods should function mostly as on other platforms but they likely live in another header.

Memphiz · 2015-05-19T06:54:35Z

you only need latest xcode and you need to try to follow the readme as close as possible - that should get it build for osx (and ios)
The diff looks like the shortcut with the comment "// first see if this is already an ip address" is missing now from lookup
android arm&x86 both refers to android. We are compiling using the android native development kit (ndk) - it might have different / subset of headers available only
I already had an adaption of CURL for parsing ipv6 somewhere - trying to dig it up fyi

mk01 · 2015-05-19T08:48:41Z

@Paxxi

thx for the quick tour. with the w32 then you will be fine, all by me introduced functions I found as directly supported - normally I would try to adapt the includes for w32 too, but (really) it was so different and with no installation to check against I simply left it as is...

mk01 · 2015-05-19T08:53:14Z

@Memphiz

indeed that code I missed initially and force pushed later, so maybe you have link for the obsolete first push?
mk01@741df4a?diff=unified

can you check this please - if we are looking at the same diffo

((what is the requirement of API version to compile on for osx?))

Memphiz · 2015-05-19T11:50:21Z

Yes we are looking at the same diff.

Former approach:

try to convert hostname to ip address - on success craft a string from the ip address
New approach
try to convert hostname to ip or ip6 address - on success return the hostname itself as ip address

The new approach is fine for ipv4 addresses i think (as there is only one allowed ip format) - but the ipv6 approach might give some issues (might be out of scope of this PR though as its a new use case for dns lookup here). The former code ensured that the returned ip was properly formatted, the new approach returns the hostname if it contains an ipv6 address. Based on the man page of inet_pton there are multiple supported ipv6 formats. So whatever the format was in hostname will be the format in the returned ip address. Not sure if we would need to return a generic format here. Or even add the "[" "]" to it already - so the caller can use the returned address inside of URLs.

Example:

CDNSNameCache::Lookup("192.168.1.100", strIp);
string myNewUrl = "nfs://" + strIp + "/myexport";

For ipv4 this is fine.

CDNSNameCache::Lookup("2607:f8b0:4010:801:0:0:0:1018", strIp6);
string myNewUrl = "nfs://" + strIp6 + "/myexport";

For ipv6 this would be wrong i think (missing brackets)...

mk01 · 2015-05-19T22:14:21Z

@Memphiz

the += is probably some kind of (evidently) wrong habit when assigning value to std::string. thanks for pointing.

what particular part makes you say that hostname is returned (instead of IP) ?

apropo IPs. I was just going through NetworkLinux.cpp to check/align all it's content and realised there are functions defined with u32 as IP parameter.
maybe we could agree on how to pass "host" parameter for the future.
I can even imagine a super tiny class as abstract wrapper around sockaddr_in/in6 respectively.

mk01 · 2015-05-19T23:52:23Z

@Memphiz @Paxxi

just pushed few changes.

code should now compile with all platforms (but w32)
unfortunately it looks that Android NDK doesn't provide implementation of ifaddrs yet

via google I came over it's implementation https://github.com/kmackay/android-ifaddrs which looks to be used on Android for this purpose. it is integrated in the commit - although I'm not sure about XBMC's policy with such injections - it is there for now to be able moving forward.

@Memphiz
I tested compilation for osx, but not roid/x86. can you ask Jenkins to do so ?

Memphiz · 2015-05-20T06:41:05Z

Look at my example again and then i mean this assignment

mk01@a18e65c#diff-3c53759a9e37920d7538e7aa60223fcaR74

In this case hostname was already an ipv6 and gets assigned 1:1 to the ip address. But there is no common format guaranteed for the ipv6 in that case. (no issue for ipv4 - this is something new because v6 can have multiple formats)

up to @koying if its ok to backport lonux code to droid

jenkins build this please

koying · 2015-05-20T08:46:57Z

I have no idea on the status of IPV6 in android/NDK and have my hands full, so I'd suggest to ifdef out android for now, if possible, to not be blocking.

Memphiz · 2015-05-20T10:33:33Z

@mk01 you stated you compiled this successfully on osx? Well jenkins says no.

Just search for "NetworkLinux.cpp" in the logs:

http://jenkins.kodi.tv/job/OSX-64/4503/console
http://jenkins.kodi.tv/job/OSX-32/4320/console
http://jenkins.kodi.tv/job/IOS-ATV2/4052/console

mk01 · 2015-05-20T13:30:37Z

@Memphiz

compiled - and forgot to include it's changes into commit (because having different source dirs for different platforms). doing it right away.

only interfaces with flag IFF_UP (interface has (some) configuration and is UP - bind() will not fail) (this means non empty interfaces list is sufficient prerequisite to start network services without errors)

specific handler in LinuxNetwork code. Prepare for easy w32 watcher integration.

- fix FreeBSD/OSX MacAddr fetch - fix NetworkInterfaceLinux::IsConnected()

announcements. This avoids unnecessary reconfigurations when IFs are changing back to IFF_UP as part of resuming process.

- remove network api calls from wakeonlan, use new CNetwork API - move HostToIp() to CDNSNameCache / rename to Lookup(), adapt calls accordingly - remove Lookup() failure logging from WakeOnLane (as this is logged in CDNSNameCache itself)

(Upstream commit 254d0d8)

…ockaddr_storage down to socks layer

…ready creating lock.

…lve before.

- remove announcer use from CDNSNameCache - flush name cache from Network on services start/restart (cherry picked from commit 36666d3173633f125f7d3494a8e3c90f365a9d54)

razzeee · 2017-04-13T07:13:43Z

We might want to move ahead with this now, as we're in pre-alpha status right now.

razzeee · 2017-09-16T12:01:45Z

I would still like to see this merged, as it's early in the v18 cycle and even if we break nightlies for some days, so be it.

FernetMenta · 2017-09-16T12:28:43Z

this needs more than just a single additional round. first it needs a rebase, ...

jenkins4kodi · 2017-11-19T15:20:28Z

@mk01 this needs a rebase

MartijnKaijser · 2018-06-24T06:52:34Z

I will close this PR. If there's interest please open a new one (with required changes if needed).

Jalle19 reviewed Apr 29, 2015
View reviewed changes

Jalle19 added the Type: Improvement non-breaking change which improves existing functionality label Apr 29, 2015

PeterPablo mentioned this pull request Apr 29, 2015

class CDNSNameCache native IPV6 support #7024

Closed

mk01 force-pushed the master_par branch from 4113d27 to 741df4a Compare April 29, 2015 22:37

MartijnKaijser added the v16 Jarvis label May 18, 2015

mk01 force-pushed the master_par branch from 741df4a to cc25443 Compare May 19, 2015 23:38

mk01 force-pushed the master_par branch from cc25443 to a18e65c Compare May 19, 2015 23:56

mk01 force-pushed the master_par branch from a18e65c to c5cc339 Compare May 20, 2015 13:31

mk01 and others added 21 commits January 8, 2017 02:37

[NetworkLinux] - align the behaviour with W32 implementation and load

038fa7c

only interfaces with flag IFF_UP (interface has (some) configuration and is UP - bind() will not fail) (this means non empty interfaces list is sufficient prerequisite to start network services without errors)

[CNetwork] - update IsAvailable()

bbf2685

[NetWatch] - move the core code into CNetwork, keep only NetLink

84192f2

specific handler in LinuxNetwork code. Prepare for easy w32 watcher integration.

Removed interface - count as change too.

d827e74

[NetworkLinux] - OSX/FreeBSD fixes

68251fa

- fix FreeBSD/OSX MacAddr fetch - fix NetworkInterfaceLinux::IsConnected()

[CNetwork] - stop and start NetworkUpdater thread with OnSleep/OnWake

abbea6d

announcements. This avoids unnecessary reconfigurations when IFs are changing back to IFF_UP as part of resuming process.

[CNetwork, WakeOnLan, CDNSNameCache]

1b1edfa

- remove network api calls from wakeonlan, use new CNetwork API - move HostToIp() to CDNSNameCache / rename to Lookup(), adapt calls accordingly - remove Lookup() failure logging from WakeOnLane (as this is logged in CDNSNameCache itself)

CNetworkLinux::GetNameServers(): also return ipv6 nameservers

f28c402

(Upstream commit 254d0d8)

[NetworkLinux] GetNameServers() - merge DARWIN & LINUX code.

28a589d

[Network] fix/introduce a way to pass size(addr) from unisex struct s…

a959b68

…ockaddr_storage down to socks layer

[CNetworkLinux] remove lock, as all callers to DeleteRemoved() are al…

2378735

…ready creating lock.

cleanup CNetworkInterfaceLinux::GetHostMacAddress(...)

e6a7fec

add CNetwork::PrefixLength(...) and CNetwork::CompareAddresses(...)

94f1f8c

rework CNetworkInterfaceLinux

225e6e4

remove CNetwork::PrefixLengthIPv6(...)

f406e6f

[WOL] CNetwork functions expect IP address, not HostName. Try to reso…

772daec

…lve before.

[NetworkLinux] suppress ping's output with >/dev/null 2>&1

3cbded6

[CDNSNameCache] Update to master

ad449ea

- remove announcer use from CDNSNameCache - flush name cache from Network on services start/restart (cherry picked from commit 36666d3173633f125f7d3494a8e3c90f365a9d54)

CNetworkWin32::GetNameServers(): also return IPv6 nameservers

913f05e

fix after m_bStop type change (volatile bool -> std::atomic<bool>)

c531f6d

[CNetwork] make use of shared_ptr, make sa_len static class method

585364e

mk01 force-pushed the master_par branch from dc9a249 to 585364e Compare January 8, 2017 01:57

jenkins4kodi added the Rebase needed PR that does not apply/merge cleanly to current base branch label Nov 19, 2017

MartijnKaijser added this to the Abandoned, obsolete or superseeded milestone Jun 24, 2018

MartijnKaijser closed this Jun 24, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CNetwork - implement IPv6 #7030

CNetwork - implement IPv6 #7030

mk01 commented Apr 29, 2015 •

edited

Loading

This comment was marked as spam.

FernetMenta commented Apr 30, 2015

Jalle19 commented Apr 30, 2015

Paxxi commented Apr 30, 2015

FernetMenta commented Apr 30, 2015

Paxxi commented Apr 30, 2015

MartijnKaijser commented May 17, 2015

Jalle19 commented May 18, 2015

mk01 commented May 18, 2015

Memphiz commented May 18, 2015

mk01 commented May 18, 2015

mk01 commented May 18, 2015

Paxxi commented May 19, 2015

Memphiz commented May 19, 2015

mk01 commented May 19, 2015

mk01 commented May 19, 2015

Memphiz commented May 19, 2015

mk01 commented May 19, 2015

mk01 commented May 19, 2015

Memphiz commented May 20, 2015

koying commented May 20, 2015

Memphiz commented May 20, 2015

mk01 commented May 20, 2015

razzeee commented Apr 13, 2017

razzeee commented Sep 16, 2017

FernetMenta commented Sep 16, 2017

jenkins4kodi commented Nov 19, 2017

MartijnKaijser commented Jun 24, 2018

CNetwork - implement IPv6 #7030

CNetwork - implement IPv6 #7030

Conversation

mk01 commented Apr 29, 2015 • edited Loading

This comment was marked as spam.

FernetMenta commented Apr 30, 2015

Jalle19 commented Apr 30, 2015

Paxxi commented Apr 30, 2015

FernetMenta commented Apr 30, 2015

Paxxi commented Apr 30, 2015

MartijnKaijser commented May 17, 2015

Jalle19 commented May 18, 2015

mk01 commented May 18, 2015

Memphiz commented May 18, 2015

mk01 commented May 18, 2015

mk01 commented May 18, 2015

Paxxi commented May 19, 2015

Memphiz commented May 19, 2015

mk01 commented May 19, 2015

mk01 commented May 19, 2015

Memphiz commented May 19, 2015

mk01 commented May 19, 2015

mk01 commented May 19, 2015

Memphiz commented May 20, 2015

koying commented May 20, 2015

Memphiz commented May 20, 2015

mk01 commented May 20, 2015

razzeee commented Apr 13, 2017

razzeee commented Sep 16, 2017

FernetMenta commented Sep 16, 2017

jenkins4kodi commented Nov 19, 2017

MartijnKaijser commented Jun 24, 2018

mk01 commented Apr 29, 2015 •

edited

Loading