Native Apple Silicon (M1) Support

[voldyman]

The project doesn't build on apple silicone yet, i think it's mainly because of the lack of assembly files.

[progrium]

Right you are, I’m not going to be able to do that anytime soon so hopefully somebody else will take a swing at an M1 implementation for variadic.

[voldyman]

I tried doing it but am not familiar with the internal information or arm/x86 assembly but looks fun to learn.

I might give it another go, no guarantees

[stephanwesten]

Hi,

I m running on Mac m1 and get the following error when I make the pomodoro example (main, 8d2ca95):

make pomodoro
go run ./examples/pomodoro
# github.com/progrium/macdriver/objc
objc/typeinfo.go:64:15: undefined: typeInfoForType
objc/typeinfo.go:73:14: undefined: typeInfoForType
objc/typeinfo.go:77:15: undefined: typeInfoForType
objc/class.go:114:36: undefined: methodCallTarget

I do not understand how this compile error is m1 related...

[dmitshur]

@stephanwesten It has to do with build constraints and the fact that GOARCH is arm64 instead of amd64 on M1. typeInfoForType is defined in a file named typeinfo_amd64.go, which is only selected when GOARCH is amd64. There isn't a a typeinfo_arm64.go file, which this issue is about.

[progrium]

missing files from build constraints for diff platforms is part of it, but afaik the differences are minimal. the real missing piece is the implementation of the workaround in variadic for arm and then the rest should be pretty straightforward. really its all straightforward but somebody that knows or is willing to learn arm architecture will have to port variadic.

[dolmen]

https://github.com/progrium/macdriver/tree/main/misc/variadic

[progrium]

@dolmen awesome, thank you for making it easy to find from here

[helperShang]

@stephanwesten

GOOS=darwin GOARCH=amd64 CGO_CFLAGS="-arch x86_64" CGO_ENABLED=1 go run -x  github.com/progrium/macdriver/examples/largetype

[progrium]

see also https://gist.github.com/progrium/b286cd8c82ce0825b2eb3b0b3a0720a0

but these are workarounds.

[mgood]

I'm currently working on setting things up with Rosetta as described, but maybe I'll get a chance to come back to this later.

Some Apple docs that are likely to be useful:
https://developer.apple.com/documentation/xcode/writing-arm64-code-for-apple-platforms
https://developer.apple.com/documentation/apple-silicon/addressing-architectural-differences-in-your-macos-code

[progrium]

yep yep. remember go has no problem compiling to arm64 already, but the variadic package that every call in this library depends on needs to have an arm64 implementation since it drops into assembly. and it doesn't do much! all it's doing is making an alternative way to do a function call.

[mgood]

(making note of a few more findings)

It looks like it's not just variadic, but some of the necessary code is in objc/call_amd64.* and objc/msg_amd64.go for initializing the procedures calls for AMD64. We'll see if I have time to fully dig into the implementation, but ideally I would like to see if there's a way to more cleanly delineate between the stuff that's specific to ObjcC, and the portion that's setting up the architecture-specific procedure calls.

For ARM, this describes the process for initializing the registers & memory to pass parameters to the procedure:
https://github.com/ARM-software/abi-aa/blob/2bcab1e3b22d55170c563c3c7940134089176746/aapcs64/aapcs64.rst#parameter-passing

The Apple docs here about calling objc_msgSend were also enlightening:
https://developer.apple.com/documentation/apple-silicon/addressing-architectural-differences-in-your-macos-code#Enable-Strict-Type-Enforcement-for-Dynamic-Method-Dispatching

So, the way objc_msgSend is implemented, it expects args to be passed like "normal" positional parameters. They give this example for creating a type-safe pointer to call objc_msgSend with the expected parameter types for the underlying function definition:

// Declare a type-safe function pointer.
void (* didSaveDispatcher)(id,SEL,NSDocument *,BOOL,void *) = 
       (void(*)(id,SEL,NSDocument *,BOOL,void *))objc_msgSend;

So, this does NOT use the conventional C-style "variadic" parameter passing where you use va_list and va_start as described here:
https://github.com/ARM-software/abi-aa/blob/2bcab1e3b22d55170c563c3c7940134089176746/aapcs64/aapcs64.rst#appendix-variable-argument-lists

Per Apple:

Because objc_msgSend declares your method as variadic, the compiler places the method’s parameters on the stack, in accordance with the calling conventions for the arm64 architecture. However, the original method declaration contains fixed parameters, not variable parameters. As a result, the method’s implementation looks for its parameters in registers, which is where the compiler places fixed parameters for arm64. This mismatch causes the method call to generate undefined results.

[mgood]

In #52 I've added code that could be used to reimplement Send based on NSInvocation in a cross-platform way (just based on CGo code and the Obj-C runtime). This would address the objc/msg_<arch> and variadic implementations.

The other part is objc/call_<arch> which is used by AddMethod to send Obj-C method calls back to the Go implementation. Looking into things a bit more, it looks like we should also be able address those with NSInvocation by implementing forwardInvocation instead:
https://developer.apple.com/documentation/objectivec/nsobject/1571955-forwardinvocation

IIUC we can use that hook to create objects that instead of registering each method directly, call a common CGo implementation for forwardInvocation. In there we can call back into Go to look up the Go method implementation and pass along the arguments.

A couple of the files (e.g. objc/typeinfo_<arch>) are more generally for any 32- vs 64-bit platform (rather than a specific CPU instruction set) so we can make those use build tags to target the appropriate platforms instead.

[vtolstov]

Can somebody write current status of this problem?

[progrium]

its being solved mostly by way of code generation which is pretty close

On Fri, Sep 17, 2021 at 7:52 AM Vasiliy Tolstov ***@***.***> wrote: Can somebody write current status of this problem? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAFBZC5LEN4ASVMVCKIE3UCM2XXANCNFSM4V3V2VJA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

-- Jeff Lindsay http://progrium.com

[vtolstov]

very nice, last question - when ? =)
and last another question - what about support File Provider stuff ? ( https://developer.apple.com/documentation/fileprovider/macos_support )

[mgood]

There's no set timeline for this. For now your best bet is still to install an amd64 Go build and run with Rosetta.

I'm working on the generator, which will expand the breadth of APIs included, and reduces the dependencies on the dynamic Send calls which are currently X86-only. Though after the generator is merged there will still be some more work to support M1. We could disable compiling Send on M1, though macdriver/cocoa also depends on AddMethod which would also need ported to work on M1.

I'm not aware of anyone looking into File Provider support for macdriver yet, but it would be interesting.

[vtolstov]

thank you

[sparky4pro]

Pull request #100 by mkrautz adds basic arm64 support.

Thank you mkrautz, this is wonderful.

[mkrautz]

@sparkylein You're welcome. Have you tried it? Does it work to your liking?
As the PR says, right now it only implements parameter passing and returns in registers, not on the stack. I don't know if any of the APIs that macdriver calls require enough arguments as to require the implementation to do more than it does now or not.

[sparky4pro]

@mkrautz I only tried to create a minimalistic window which worked.

However, it's strange that it works, because here https://github.com/below/HelloSilicon#listing-9-1 it is said that for variadic functions Apple silicon diverges from the ARM64 standard ABI. Where Linux will accept arguments passed in the registers, for Darwin the arguments must be passed on the stack.

I'm not an assembly coder myself, so I don't know why what you did works...
It could also very well be that I'm mixing-up stuff, and the gentleman from HelloSilicon is talking about something different.
In that case, I apologize for my ignorance :P

[mkrautz]

It works because on darwin/arm64, Apple redefined objc_msgSend to use the regular calling convention.

objc_msgSend is defined as (void)objc_msgSend(void) in the headers, and if you must call it directly, you cast it to have the signature you require. That way, the compiler generates a non-variadic call, which objc_msgSend on arm64 requires.

That's also why the package name 'variadic' is a bit misleading with the new PR. Perhaps a better name would be 'objcffi'.

[sparky4pro]

Interesting. Thanks for the info.
Maybe this is the reason why this https://github.com/hsiafan/cocoa works without any assembly magic :)
(uses only some Objective-C glue...)

[mgood]

I guess just to clarify now that this is based on the https://github.com/hsiafan/cocoa package mentioned, it's using libffi to handle the function calls instead. It's still doing something equivalent to the prior assembly implementation, but now that's abstracted to use a common library for that architecture-specific ABI translation instead of reimplementing it here.