Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need calling convention for stdcall methods (different from functions!) #1342

Open
retep998 opened this Issue Oct 28, 2015 · 6 comments

Comments

Projects
None yet
5 participants
@retep998
Copy link
Member

retep998 commented Oct 28, 2015

The crux of the matter is that stdcall methods on Windows using msvc handle struct returns differently than stdcall functions. This is particularly noticeable in COM, where although most of COM doesn't use struct returns, a few of them do, causing incorrect behavior when you attempt to call them from Rust (or even C for the matter).

Thus I propose that we add a new calling convention specifically for stdcall methods. When faced with this new calling convention Rust should do what Clang does:

(2:07:35 PM) rnk: right, this is sret vs. this
(2:07:47 PM) rnk: clang handles this by swapping the order of the arguments
(2:07:54 PM) rnk: and moving the sret attribute to the second parameter

Relevant discussion in #LLVM https://gist.github.com/retep998/604861ea06aa984ee6c7
Relevant discussion on discourse https://internals.rust-lang.org/t/need-custom-calling-convention-for-com/2389

gist demonstrating the calling convention difference: https://gist.github.com/retep998/9503145841a61551d3c6

@eefriedman

This comment has been minimized.

Copy link
Contributor

eefriedman commented Oct 28, 2015

extern "stdcall_method" and friends seems reasonable to me.

@mzabaluev

This comment has been minimized.

Copy link
Contributor

mzabaluev commented Oct 29, 2015

Notably, the MIDL compiler seems to have no problem annotating __stdcall in the plain-C bindings for COM vtables. If it turns out that on -msvc targets rustc or LLVM does not do what MSVC does for any C function, then it's simply a matter of fixing "stdcall".

@retep998

This comment has been minimized.

Copy link
Member Author

retep998 commented Oct 29, 2015

MSVC for C actually suffers from the same issue. I tested it myself. Functions in COM that return structs have incorrect behavior when called from the official Microsoft C bindings using the MSVC compiler. I'm assuming nobody at Microsoft noticed or cared about this since very few people use C when working with COM.

@retep998 retep998 referenced this issue Mar 7, 2016

Open

Rust, Windows, and MSVC #1061

18 of 47 tasks complete
@DemiMarie

This comment has been minimized.

Copy link

DemiMarie commented Jul 10, 2016

What about thiscall? That is what LLVM calls it I believe.

@retep998

This comment has been minimized.

Copy link
Member Author

retep998 commented May 26, 2017

@DemiMarie thiscall is not the same as an stdcall method.

@retep998

This comment has been minimized.

Copy link
Member Author

retep998 commented Jul 6, 2017

It turned out that the workaround I use for this in winapi was actually slightly wrong. When the function takes parameters beyond this, those parameters need to go after the return value parameter. This just goes to show how important language support for this is, because it would remove the need to implement hacky workarounds that might be broken.
retep998/winapi-rs@6754bdd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.