Windows support

[promag]

I think this should be referenced here:

Supporting windows, not just unix systems. The existing socket code is already cross-platform, so the only windows-specific code that needs to be written is code spawning a process and passing a socket descriptor. This can be implemented with CreateProcess and WSADuplicateSocket. Example: https://memset.wordpress.com/2010/10/13/win32-api-passing-socket-with-ipc-method/.

From https://github.com/bitcoin/bitcoin/blob/master/doc/multiprocess.md

Is there anything else worth mentioning? I could give it a try.

[ryanofsky]

This is interesting since that I think comment was written before libmultiprocess existed as a separate library.

I think basically the comment is still applicable and can be used to implement windows versions of mp::SpawnProcess and mp::WaitProcess (with the latter calling WaitForSingleObject and GetExitCodeProcess).

Function signatures in mp/util.h should also change slightly to take generic socket and process handles instead of using ints. Maybe should look something like:

#ifdef WIN32
using SocketId = HANDLE;
using ProcessId = HANDLE;
#else 
using SocketId = int;
using ProcessId = int;
#endif

using FdToArgsFn = std::function<std::vector<std::string>(SocketId fd)>;
int SpawnProcess(ProcessId& pid, FdToArgsFn&& fd_to_args);
int WaitProcess(ProcessId pid);

[ryanofsky]

Another part of this is that signatures of mp::ConnectStream and mp::ServeStream should change to accept windows handles:

template <typename InitInterface>
std::unique_ptr<ProxyClient<InitInterface>> ConnectStream(EventLoop& loop, SocketId socket);

template <typename InitInterface, typename InitImpl>
void ServeStream(EventLoop& loop, SocketId socket, InitImpl& init);

I think this should only require replacing the int type with the generic SocketId though.

[ryanofsky]

Also finally some KJ_SYSCALL uses in mp/proxy.cpp might have to be replaced:

https://github.com/chaincodelabs/libmultiprocess/blob/805eb73e570fa509e2bb043dc822151a34b67bb0/src/mp/proxy.cpp#L132-L135
https://github.com/chaincodelabs/libmultiprocess/blob/805eb73e570fa509e2bb043dc822151a34b67bb0/src/mp/proxy.cpp#L183
https://github.com/chaincodelabs/libmultiprocess/blob/805eb73e570fa509e2bb043dc822151a34b67bb0/src/mp/proxy.cpp#L196-L197
https://github.com/chaincodelabs/libmultiprocess/blob/805eb73e570fa509e2bb043dc822151a34b67bb0/src/mp/proxy.cpp#L211-L212

I think there are straightforward replacements for all of these. Replace socketpair with maybe 2
CreatePipe calls, replace close with CloseHandle write with WriteFile.

EDIT: Should only need to replace socketpair with one CreatePipe call not two. I was thinking 2 because socketpair is bidirectional, but this code is only using the socket for signalling just writing to one end, so even on unix I think this could just use plain pipe.

[promag]

I'm having some issues trying to build on windows.

I downloaded and built capnproto-c++-0.8.0 with msvc2017. Then I set CapnProto_DIR and ran cmake. Then building the solution I have the following error:

C:\Users\joao\Projects\libmultiprocess\build>msbuild Libmultiprocess.sln  /property:Configuration=Release /property:Platform=Win32

[...]

CustomBuild:
  Compiling Cap'n Proto schema include/mp/proxy.capnp
  C:\Users\joao\Projects\capnproto-c++-0.8.0\build\src\capnp\Release\capnp.exe compile: -I C:/Users/joao/Projects/include: no such directory
  Try 'C:\Users\joao\Projects\capnproto-c++-0.8.0\build\src\capnp\Release\capnp.exe compile --help' for more information.

Looks like it's using C:/Users/joao/Projects to search for include/mp/proxy.capnp?

[ryanofsky]

Hmm, that custom build rule comes from this line in CMakeLists:

https://github.com/chaincodelabs/libmultiprocess/blob/805eb73e570fa509e2bb043dc822151a34b67bb0/CMakeLists.txt#L45

It's possible prefixing include/mp/proxy.capnp as ${CMAKE_CURRENT_SOURCE_DIR}/include/mp/proxy.capnp could help, but this is a guess, and it would probably be good to see if there's way to get more verbose build output (complete command line) to see what could be happening.

In longer run, I assume the best way to get these various packages working together on windows will be with vcpkg, but I don't have any experience using vcpkg yet. I'm expecting there are more build issues waiting after this one. My previous comments above were about porting library code, and I think porting build code should be simpler and require fewer changes than porting library code, but the problems are probably less predictable and may take more time to debug.

[promag]

Same here, not used to code on windows, I'll try your suggestion just to have it build so I can work on the actual code change.

[promag]

Issue is related to CAPNP_INCLUDE_DIRECTORY , not with include/mp/proxy.capnp. Tomorrow I'll try to see why it's getting that value, otherwise I'll set it manually to the correct one just to move forward.

[ryanofsky]

Some more recent notes on implementing windows support:

• On current compilation failures: windows build failure #114
• On implementing the -ipcfd option: From Multiprocess bitcoin bitcoin/bitcoin#10102 (comment) "The -ipcfd option isn't described in the command line documentation because it isn't meant to be used externally. Also it will have slightly different behavior on windows where it will contain the path to a named pipe, and the then the child process will create a socket and call WSADuplicateSocket to send one side of the socket to the parent over the named pipe."
• Chatgpt code suggestions: https://chatgpt.com/share/67f7ffdd-61b0-800a-8891-a0069ca140c9