Design of named pipes support on windows

[Darksonn]

The purpose of this issue is to flesh out a good API for supporting named pipes on windows. To help out with this effort, here are some good starting points for discussion:

1. What are the use cases we need to support?
2. What is prior art? Please post some links to prior Rust implementations, as well as good implementations in other languages.
3. Does it make sense to provide an API that mirrors UnixStream/UnixListener?
4. Does it make sense to provide an API in the spirit of AsyncFd? If so, how would it differ from AsyncFd? Or does the raw named pipes API look totally different?
5. Is the raw API cumbersome enough to warrant exposing both an UnixStream/UnixListener mirror and a more raw API?
6. Regarding mirroring, what about UnixDatagram?
7. What should the types be named?

[blackbeam]

@carllerche,

Would you be able to try to expand on these use cases over there?

Well. My only use case is to be able to connect to MySql on Windows. Named pipes show significantly better performance over TCP.

• an ability to control pipe access rights and audit rules

This feature was mentioned as a must-have by @faern here. As far as I can see it'll have no impact on API design. We could simply add it in a form similar to this one.

I found no safe rust wrappers for SECURITY_ATTRIBUTES. The closest thing I found is windows-acl, but it wouldn't enough.

• an ability to choose the direction (in, out, in-out)

I have nothing to say about use cases.

• an ability to impersonate a client (quite dangerous if misused)

It's a powerful feature that is assumed to allow a priveleged server to temporarily lower its privileges down to a client's but may be used in other direction which is quite dangerous especially in the context of Named Pipe Instance Creation Race Condition. Please read this small document for more info.

• an ability to set the maximum number of server instances

I believe this could be exposed as a function that returns a bunch of servers, as mentioned by @simonbuchan. The upper bound is 254.

• anonymous pipes (implemented in miow)

Seems like anonymous pipes and named pipes are unrelated to each other and anonymous pipes are out of scope.

• the message transmission mode

I have nothing to say about use cases. Anyway, it should not affect the API design.

@Darksonn,

What is prior art? Please post some links to prior Rust implementations, as well as good implementations in other languages.

I've used this implementation as a source of inspiration for #3388, but not from the design perspective. The design mimics UnixListener/UnixStream API. The good rust implementation I can think of is the one from miow.

Does it make sense to provide an API that mirrors UnixStream/UnixListener?
Regarding mirroring, what about UnixDatagram?

Even though #3388 mostly mimics UnixStream/UnixListener, I have to admit that the API is somewhat incompatible with named pipes. The problem lays in the fact, that there is no way to bind to a pipe name without creating an instance. I see two options to solve this:

1. Omit FILE_FLAG_FIRST_PIPE_INSTANCE, but this makes impl vulnerable to the Named Pipe Instance Creation Race Condition (again, see here).
2. Create the first instance using the FILE_FLAG_FIRST_PIPE_INSTANCE flag and always keep at least one instance (this approach was chosen in Add initial named pipes support #3388), but this will work flawlessly only with a continuous flow of clients since the order in which the operating system chooses which server instance to make the connection to is undocumented (as far as I know).

I believe that it's possible to come up with an ad-hoc solution to this, but I'd rather choose to avoid it from the API design side, if possible.

[simonbuchan]

I'm not sure I understand AsyncFd properly, but an AsyncHandle should allow random clients to implement async ConnectNamedPipe and proper overlapped read and writes: in general getting overlapped results on random handles (file, network, pipes, FS change notifications, theres a few), so if that is possible, I'm all for that. I'm a bit confused about the Insterest::READABLE/WRITABLE and poll functions on AsyncFd, but perhaps that's simply the way poll() works? With IOCP AsyncHandle should probably be very thin and the interesting bits in an OVERLAPPED wrapper (so maybe OverlappedHandle would be clearer?). That seems to be the easiest thing that could completely unblock this case (for wrapping libraries, at least, it would still be fairly annoying code to write), and in fact pretty much every case on windows without potentially locking the API into some design issue.

If that is a nightmare for some reason, I'm pretty mixed. Cloning UnixStream actually seems to work great if you just need a windows alternative, so I would be fine with it, but I can see it potentially being a problem. Perhaps whatever algorithm has some edge case in heavily loaded servers, or the like?

The point on safely implementing SECURITY_ATTRIBUTES is interesting, std::fs::File / std::os::windows::fs::* doesn't support it either. There they have the FromRawHandle escape hatch, which AsyncHandle would also allow.

@blackbeam

I believe this could be exposed as a function that returns a bunch of servers, as mentioned by @simonbuchan. The upper bound is 254.

There's the (optional!) OS-enforced limit in nMaxInstances (for the first successful call) and then the actual API for creating the multiple instances, which can be on demand. Limit seems like a clear match for OpenOptions, but actually creating them could be explicit or internally handled by tokio, as with your branch. The difference could be in performance, but this may be over-optimizing.

I find the comment

there is no way to bind to a pipe name without creating an instance

interesting: to my understanding bind() is the point of creating a FS entry, and CreateNamedPipe() is the point of creating a \\?\pipe entry, and then both require an accept call to actuall establish a connection. The main weird API difference I know of is that you can create and wait for connections on multiple instances on different threads, and re-use instances after the connection is closed. This seems to functionally behave the same as UDS, but possibly allow greater optimization?

[faern]

When used as the Windows equivalent of UDS (normally for local IPC) there are at least two things to think about with regards to access permissions:

• By default Windows named pipes accept connections over the network. Very dangerous when the intention is local IPC. PIPE_REJECT_REMOTE_CLIENTS must be passed correctly when creating the pipe. https://docs.rs/miow/0.3.6/miow/pipe/struct.NamedPipeBuilder.html#method.accept_remote

• SECURITY_ATTRIBUTES must be supported in order to be able to grant unprivileged processes access to named pipes created by privileged processes etc. Common in a system service <-> unpriv client software relationship. https://docs.rs/miow/0.3.6/miow/pipe/struct.NamedPipeBuilder.html#method.with_security_attributes

I think whatever API tokio exposes has to allow setting these two things on pipe creation. Maybe not in a super nice way, maybe not safely, but it must be possible. Otherwise the API becomes somewhat useless from an IPC standpoint.

Does it make sense to provide an API that mirrors UnixStream/UnixListener?

Yes. At least for our use case. The only thing I have ever used named pipes for is for the equivalent of IPC via UDS on Windows.

[pimeys]

I've always been a bit confused about the difference between named pipes, and the named shared memory protocol. I'm mostly staying on the Linux side to have a deep knowledge about Microsoft's protocols, but I'd like to know are these two protocols the same? SQL Server allows you to explicitly enable the named pipe protocol, and by default only allows the shared memory protocol.

Could we use the NamedPipe implementation in @blackbeam's PR for both, or are they two completely different things?

[simonbuchan]

@pimeys Roughly, named pipes is UDS, named shared memory is unix shared memory - literally sharing a raw hunk of memory between multiple processes, bring your own synchronization.

[udoprog]

I've put together a preliminary working prototype in #3760 based on the proposal in #3388. See the PR for motivation and design.

[Darksonn]

We would be happy to hear feedback on #3760. If you have time to look at it, please post your thoughts.

[Darksonn]

Closing as #3760 was merged.

[pronebird]

@Darksonn although #3760 is merged and I don't have all the insight into the inner workings of named pipes, what @blackbeam spoke about. However helpers akin to UnixListener/UnixStream for named pipes would still make sense in terms of ergonomics. Right now it's very simple to get started with UDS, easy to wrap it with tokio-stream and then toss it towards tonic server or anything else. Very different story with named pipes and practically no available solutions, I am only aware of mostly abandoned parity-tokio-ipc crate, but I think windows folks would greatly benefit from having equally great support for pipes in tokio.

[Darksonn]

The current API matches what the OS provides, so we need to use that API to allow full flexibility for users. And we generally, do not have any other case where Tokio provides two versions of the same API, with one being a wrapper of the other that is more convenient.

So my gut feeling is that this belongs in its own crate.

If you want to discuss this further, then open a new issue.

[simonbuchan]

We've had our own wrapping API over tokio UDS and Named Pipes since before tokio added this; it's not too hard to get running (probably ~50-100 lines from years old memory...), but it's tricky to completely paper over the path and locking semantics, so I'd say it's probably not a great idea to try to hide those differences.

Specifically, we wanted exactly one server to be the "canonical" one using whichever process can serve on the socket but it's pretty ugly to get that to be reliable on Unix. (There's probably a much nicer application level approach)

[pronebird]

@Darksonn ok agree.

@simonbuchan I took time and figured out how to implement the wrapper myself. My primary issue was making my own stream which ultimately I solved with try_stream! from tokio async-stream. The rest was relatively straightforward.