w32 struct return values

[twall]

If JNA is compiled with a different compiler than your target library, you may see a crash when returning a struct of size greater than 8 bytes. This is because GCC, mingw-gcc, and MSVC don't necessarily agree on how to handle such a scenario.

This turned up when building and testing with mingw32 (x86 target) calling a callback with a returned struct. There is some accommodation within libffi to handle GCC vs MSVC calling conventions, but the JNA code should really test all permutations to ensure it works properly:

Simple call to native function returning struct:

• Java -> JNA (mingw) -> native (MSVC)
• Java -> JNA (MSVC) -> native (mingw)
• Java -> JNA (MSVC) -> native (MSVC) (tested, OK)
• Java -> JNA (mingw) -> native (mingw) (tested, OK)

Call to native function returning struct returned by a provided callback. Note that the calling and callback handling glue (libffi) will always be compiled by the same compiler:

• Java -> JNA (mingw) -> native (mingw) -> JNA (callback mingw) (tested, fails)
• Java -> JNA (MSVC) -> native (MSVC) -> JNA (callback MSVC) (tested, OK)
• Java -> JNA (MSVC) -> native (mingw) -> JNA (callback MSVC)
• Java -> JNA (mingw) -> native (MSVC) -> JNA (callback mingw)

[twall]

This is caused by a merge error in libffi proper, filed an issue.

[twall]

The FFI_MS_CDECL ABI will only have an effect in closures. Its usage in ffi_call() is irrelevant, since ffi_call generally ignores what the callee does to the stack, manually providing storage for a returned structure.