Not able to compile in Ubuntu #3
Comments
|
I haven't been back to testing on Linux in some time (main development is happening on FreeBSD), so this isn't surprising. This looks like something pretty fundamental. Try adding
to lib/libuinet/uinet_if_pcap_host.c and see if that improves things for you. |
|
Sorry... I updated this while you were adding the comments. |
|
What version of Ubuntu are you using? |
|
I am using 12.04 uname -a I have done the following changes to uinet_host_interface.h +#include <stdint.h> Now, when I make with the following command, I get the following error - peter: |
|
OK, let me try to reproduce. btw, when putting code in these comments, enclose it in backticks, otherwise the markdown processing isn't friendly (note the include file in your last comment is blank) |
|
Sure. Will take a note of it. I am using Ububtu 12.04. I followed the following steps -
And as I said, I have just included "stdint.h" in "uinet_host_interface.h" |
|
I've cleaned things up a bit for Linux based on what was going on with Ubuntu 12.04. Have another go and let me know how it works out. |
|
I was able to compile. Many thanks. I will test now and update you on the results. |
|
getting a coredump for all the sample programs while running in ubuntu. peter:~/src/libuinet/bin/echo++$ ./echo++ -i eth0 -l 192.168.4.107 -p 8090 stack trace: |
|
The above issue was because mmap was failing. Looks like we can't call mmap with MAP_ANONYMOUS using fd -1. Once I changed the fd to fd = open("/dev/zero", O_RDWR); and flag to MAP_SHARED. There is no crash. But, I am not able to connect to the example server. |
|
although there is no crash observed but I am not able to connect to the example server. I see the below failures while running echo++. INTERNAL: Allocating one item from UMA Zones(0x72e680) command and eth0 interface: command i used: i see you have already linux specific code for this in function if_netmap_ethtool_set_flag(). is there anything I am missing |
|
Hi, At present I am stuck with the two issues, I need your inputs on addressing them.
Please let me know. Thanks....Peter. |
|
Hi, Many thanks.... ~Peter |
|
The actual mmap() issue was that neither MAP_SHARED nor MAP_PRIVATE was being specified in the call to uhi_mmap() in uinet_vm_kern.c, and one of them is required, at least on this system. The use of an fd of -1 for anonymous mappings is fine (the fd is ignored on this system), and it is actually suggested for portability by the mmap man page on this system. The fix for this has been comitted. |
|
The fix for the ethtool issue has been comitted. |
|
Thanks for the fix. I picked up your latest changes. I don't see the error messages. But, once I start the echo program, I am not able to ping to that machine. I looked at the interface and I see that once I start the program, it changes to "PROMISC" mode. Please see the outputs below - Before starting the echo program !!!peter$ ifconfig eth0 sudo ./echo -i eth0 -l 169.254.8.13 -p 8090 configstr is eth0 "tshark" capture: to show that ARP is being received but no reply being sent37.160342 28:d2:44:1c:ee:1b -> Broadcast ARP 60 Who has 169.254.8.13? Tell 169.254.6.238 After the echo program !!!peter$ ifconfig eth0 Some more info:Breakpoint 1, ether_input_internal (ifp=0x79bf90, m=0x7ffff7fee600) It comes to the function " " to resolve the ARP and goes to "drop". gdb) n Breakpoint 1, in_arpinput (m=0x7ffff7fee500) at /home/peter/libuinet/lib/libuinet/../../sys/netinet/if_ether.c:514 Breakpoint 2, in_arpinput (m=0x7ffff7fee500) at /home/peter/libuinet/lib/libuinet/../../sys/netinet/if_ether.c:614 (gdb) p ifp->if_addrhead->tqh_first->ifa_addr Look like the IP address is not getting added to the interface causing the ARP request getting dropped. Could you please point me what is going wrong. Also the value of sa_family is not AF_INET. |
|
It would help if I understood your goal a bit better. Absent such an understanding, I can give you some general information that may be of assistance in understanding what you are observing. The main purpose of the echo program as it is now is to allow exercise of the promiscuous sockets functionality that I added to the TCP/IP stack in libuinet. 'Promiscuous' here refers to the ability to bind sockets to any VLAN tags, MAC addresses, and IP addresses. When using promiscuous sockets on a given interface, that interface is necessarily put into promiscuous mode so that no L2 filtering is performed by the hardware in order to allow the stack such visibility into and control over L2 and L3 addressing. Also, when using promiscuous sockets, no IP addresses are assigned to the interface. Not only would doing so present a scaling issue, but more importantly, such an approach doesn't apply generally to how promiscuous sockets can be used. Consider, for example, that with promiscuous sockets, you can listen on 0.0.0.0:0 (that is, listen for connections to any IP address and port) - what address you add to the interface in that case? The libuinet stack can be used to do regular, that is, non-promiscuous, socket operations (in which case you will have to make additional libuinet calls to add IP addresses to your interfaces), however the echo program is not currently using the stack in that way. Further, you may want to consider that using the libuinet stack with netmap on a given interface causes libuinet to completely take over that interface - none of the inbound packets on that interface will go anywhere but that libuinet instance while it is running. If eth0 is your primary interface, you probably want to use a separate interface for libuinet experimentation. Also, if you are going to play with promiscuous listens that can capture connections to addresses in use elsewhere on your network (say, listening on 0.0.0.0:0), you would probably be better off testing on an isolated network. Consider that echo -i eth0 -l 0.0.0.0 -p 0 might see a SYN sent from one machine on your network to another (even in a switched environment, due to MAC address table misses in the switches), and it will respond with a SYN|ACK and try to establish the connection. Finally, note that in your debugging session, the value of sa_famliy you see in the interface address list is AF_LINK, because the only entry on that list will be the MAC address for the interface. |
|
Thank you for the explanation. I must say, you have put a detailed great explanation. I was trying to see the performance of libuinet in term of PPS and CPS such that if possible I can use with Nginx (of course it will need some modification). But, my first aim is to evaluate how much performance I get using libuinet. I am still not clear about the promiscuous mode, I am going to study a bit more about this. What I understand from your email is from the client machine I will not be able to ping the echo server. But, I should be able to connect. But, I was thinking how is this working. Before connect, should not the client try to resolve the ARP? Also, do you have any sample program, where you are using socket operations in non-promiscuous mode? Also, I was not able to find any document on libuinet. If you have some writeup somewhere, can you please point them to me, please. I know that libuinet stack with netmap on a given interface causes libuinet to completely take over that interface. For which I was configuring multiple interface, one for data and one for management etc... Thank you once again. ~Peter |
|
Thank you for describing your intended use for libuinet. libuinet is a young project, and as you've noticed there is currently little in the way of documentation. As you may have also noticed, there isn't even a version number nor has anything resembling a release been made. Understanding where libuinet came from may aid you in understanding where it is now. libuinet was initially created so that it could be used to build transparent proxies, with a specific requirement that it scale beyond 64k simultaneous connections. Transparent proxy functionality (essentially, being able to accept connections to, and initiate connections from, arbitrary {VLAN, IP, port} tuples) required something like promiscuous sockets. The FreeBSD stack was chosen for extending with promiscuous sockets functionality, and it was ported to user space in order to avoid dealing with file descriptors, provide tighter coupling to the applications, have easy multi-instance scaling, allow porting to other operating systems, etc. When using promiscuous sockets in a transparent proxy application, there isn't one universally right answer on what to do with ARP requests (depending on the goals and configuration of the proxy, you may want to ignore some, reply locally to some, proxy some to somewhere else, etc). In some cases, the proxy is part of a larger system in which ARP traffic is handled elsewhere. The likely future direction for ARP handling when using promiscuous mode sockets is to allow the application to install an ARP handler so it can determine whether to ignore/reply/other each request. I have updated bin/echo so that it will use non-promiscuous sockets unless the listen address is 0.0.0.0, the listen port is 0, or the the -P option is given. If none of those conditions are met, then the specified listen address will be added to the given interface, a non-promiscuous listen will be performed, and ARP requests for the given listen address should be handled. As far as libuinet performance is concerned, the initial focus has been on scalability (that is, handling large numbers of connections). Measurement and improvement of time-performance metrics such as throughput, connection rates, etc. is just beginning. Note that as the initial focus has been on correctness of new features and non-time performance metrics, everything is currently being built by default with -O0 to facilitate quick debugging. If you are going to measure throughput and connection rates, you'll want to adjust the compiler flags in the relevant makefiles first. As far as Nginx integration issues are concerned, I can't comment in detail as I haven't examined the Nginx internals. However, given that Nginx is event driven, the libuinet API is meant for integration into event systems, and the libuinet API has been integrated with two different event systems to date (WANProxy and libev), I would expect there to be a reasonable integration path. I will be giving an ~hour talk on libuinet at BSDCan next week (https://www.bsdcan.org/2014/schedule/events/447.en.html), and following that the slides should be posted online. If you aren't already planning on being there, the slides should provide a more complete picture of the current state of the project and anticipated future directions. |
|
Thanks. It is clear to me now. I will not be able to attend your tech talk, looking forward for your slides after that. |
|
I was measuring the performance of libuinet on a KVM VM (which uses virtio) with one core. I performed two tests -
Next,
I am not sure why the performance get reduced when I run with libuinet. I am planning to run callgrind and see where in the libuinet we take time. But, I expected the libuinet will get me better performance. I can't attach my simple TCP programs here. But, if you can drop me an email at peter.gsnm@gmail.com, where I can share my sample programs. Other thing is to get the 200K PPS (without libuinet), I made the sysctl changes as follows. Not sure if I need to modify some of your header files for the below mentioned sysctl changes. Can you please point me to where I need to modify. fs.file-max = 5000000 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_synack_retries = 2 net.ipv4.tcp_fin_timeout = 7 net.ipv4.tcp_slow_start_after_idle = 0 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_low_latency = 1 Please let me know what you think. Thank you. ~Peter |
|
Thank you for the detailed measurements; I don't have any specific suggestions or thoughts at present but wanted to suggest you create a new issue for this specific set of performance issues. I can't speak for pkelsey, but I think it may be worth creating a new issue on the performance front rather than continuing to deal with it in this one, for ease of tracking and to avoid having to read through a number of different conversations to get to the current one. GitHub makes it very cheap to open and close issues. You wouldn't want the performance concerns to fall by the wayside just because the Ubuntu build is fixed. |
|
Yes, please open a new issue for this. |
|
Thank you. I am closing this issue as I am able to build use libuinet on my Ubuntu 12.04 LTS. I am able to test the sample programs provided by pkelsey and was also able to create my sample program on libuinet too. Many thanks to pkelsey for all his help on this. |
peter:
/UTCP/libuinet/lib/libuinet$ make NETMAP_INCLUDES=/home/peter/UTCP/netmap/sys//UTCP/libuinet/lib/libuinet$cc -c -O2 -frename-registers -pipe -fno-strict-aliasing -std=c99 -ggdb -O0 -Wall -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wno-pointer-sign -Wmissing-include-dirs -fdiagnostics-show-option -fno-common -finline-limit=8000 --param inline-unit-growth=100 --param large-function-growth=1000 -DUINET=1 -D_BSD_SOURCE -fstack-protector -I. -I/home/peter/UTCP/libuinet/lib/libuinet/api_include -I/home/peter/UTCP/netmap/sys/ -Werror uinet_if_netmap_host.c
In file included from /home/peter/UTCP/netmap/sys/net/netmap_user.h:70:0,
from uinet_if_netmap_host.c:61:
/usr/include/net/if.h:45:5: error: expected identifier before numeric constant
/usr/include/net/if.h:112:8: error: redefinition of ‘struct ifmap’
/usr/include/linux/if.h:137:8: note: originally defined here
/usr/include/net/if.h:127:8: error: redefinition of ‘struct ifreq’
/usr/include/linux/if.h:171:8: note: originally defined here
/usr/include/net/if.h:177:8: error: redefinition of ‘struct ifconf’
/usr/include/linux/if.h:220:8: note: originally defined here
uinet_if_netmap_host.c: In function ‘if_netmap_ethtool_set_flag’:
uinet_if_netmap_host.c:320:16: error: assignment from incompatible pointer type [-Werror]
uinet_if_netmap_host.c: In function ‘if_netmap_ethtool_set_discrete’:
uinet_if_netmap_host.c:353:16: error: assignment from incompatible pointer type [-Werror]
uinet_if_netmap_host.c: In function ‘if_netmap_set_promisc’:
uinet_if_netmap_host.c:454:19: error: ‘IFF_PROMISC’ undeclared (first use in this function)
uinet_if_netmap_host.c:454:19: note: each undeclared identifier is reported only once for each function it appears in
cc1: all warnings being treated as errors
make: *** [uinet_if_netmap_host.o] Error 1
peter:
The text was updated successfully, but these errors were encountered: