Support for AF_LOCAL domain sockets in mtcp

[arunkumarsit]

Hi Team,
I can understand that the mtcp is mainly to support AF_INET sockets and to accelerate the networking speed across systems.
But why can't I create AF_LOCAL sockets with mtcp stack? Is there any specific reason the AF_LOCAL socket creation blocked by mtcp?
Also the mtcp_epoll_wait() and mtcp_epoll_ctl() API's have any restrictions for other socket domains like AF_LOCAL?

The reason for my question is,

• I am using an application which has Kernel sockets with AF_LOCAL and AF_INET domain. I converted the AF_INET domain kernel sockets to mtcp supported sockets.
• But I have only one thread which is receiving the events via epoll_wait() call for both AF_LOCAL and AF_INET fd's.
• When I converts this epoll_wait() API to mtcp_epoll_wait() API, the local socket events cannot be handled by mtcp API. so I need to use epoll_wait() for AF_LOCAL sockets and mtcp_epoll_wait() for mtcp sockets.
• This becomes tough as I have only one thread to receive the events from all the fd's.

Any ideas/suggestion on this would be great !

[ajamshed]

I think your first sentence kind of answers why we do not provide support of AF_LOCAL sockets yet. I think you would probably need to spawn a thread that handles epoll_wait() for normal AF_LOCAL sockets. By the way, even if we do eventually extend mTCP to support AF_LOCAL sockets, an mTCP-based AF_LOCAL socket won't be able to communicate with a Linux/BSD AF_LOCAL socket as its underlying implementation is architecturally different (mTCP sockets don't use Linux process file descriptors).

[arunkumarsit]

Thanks for your reply !

I thought of 2 solutions to my problem.

1. One is to instantiate new thread for mtcp sockets and keep the existing thread to process local sockets
2. Second one is to use mtcp sockets for all kind of domains in order to avoid 2 different threads (as I have to check for mutex locks for the variable access)

I am interested to try both the solutions

In case of solution-2:
I know that mtcp is not using Linux socket descriptors and uses its own file descriptors. Also communication to Linux descriptor is not possible via mtcp stack.
But can I create all the sockets in mtcp context and poll for events using mtcp_epoll_wait() ?
ultimately I have to find out a trigger point for the events to trigger for mtcp supported AF_LOCAL sockets. AF_INET socket in mtcp triggers events in the SYN_RECVD (TCP_ST_SYN_RECVD) state from the TCP stack. similarly I have to find out a way to trigger the events in event queue for mtcp_AF_LOCAL sockets.
In this case I can have only one thread which will wait for events in mtcp queue for both AF_LOCAL and AF_INET sockets.

Is this work? Any suggestions/comments from your side ?

[ajamshed]

I think solution 1 will be faster to implement. I am not sure how difficult it will be to integrate support for solution 2 in mTCP stack. I suggest that you first try solution 1 and see if you get satisfactory performance with it.

[johalun]

Reviving this old issue. I'm in the same situation. Plus, I have a table with file descriptors for AF_LOCAL and AF_INET sockets which all have to be unique. Did you find a good solution to this?

Edit: I'm thinking maybe go with suggestion 1 and have the background mtcp thread trigger an eventfd in the main thread's epoll event loop. As for the fd, I'm thinking dup'ing a dummy fd for each mtcp connection to use as connection identifier for the application.