Improve Objective-C interfacing

[habemus-papadum]

libcmt seems both useful but also difficult to maintain as Apple makes changes to the Metal.framework. It might make sense to come up with a libcmt plan.

I would suggest:

• Minor fixes
  • rename some functions e.g mtNewCommandBufferWithDescriptor -> mtCommandBufferWithDescriptor to better indicate what should be MRR and what is autoreleased
  • add support for autoreleasepools
  • make cmake more pedantic -fno-objc-arc
• Audit libcmt for other memory problems
• Build knowledge
  • add metalcpp alongside cmt, and then create examples to show how one can do similar things with both and also show how to run apps to debug memory e.g. OBJC_DEBUG_* and xctrace record
  • Using this to improve memory management in 'libcmt`
• Replace libcmt with a pure Julia implementation
  • use low-level Objective-C runtime functions (e.g. sel_getName,objc_msgSend, etc) in the way metalcpp does
    • explain why this is not a performance issue (Note: the Objective-C is highly dynamic; much of what clang is doing for objective-C is ensuring correctness and not statically optimizing, though it does do some performance optimizations as well based on its understanding of the semantics of the Objective-C runtime. )
  • change Clang generator to generate pure julia binding
  • create a standalone tools that goes from framework to julia package
    • need to support obj-c blocks...
    • creating Objective classes in julia (e.g. for delegates)
    • Most of this has been worked out to some extent in ObjectiveC.jl

Probably before spending too much time on revamping libcmt it could be useful to agree a scope for Metal.jl. I would be interested in support being complete enough to write a game engine in Julia e.g.

• Support render pipeline and also compiling render shaders written in Julia
• Support for raytracing in compute pipeline
• Support for debug annotations, gpu events, etc to profile and benchmark fully using Instruments

[maleadt]

Great write-up, thanks.

Replace libcmt with a pure Julia implementation

Although I'd very much like a native FFI, aren't we going to be running into the same memory-management problems then, and wouldn't we avoid those by using metalcpp? Or can you parse that info from the headers?
Of course, C++ FFI is probably much harder to accomplish than ObjC FFI.

add metalcpp alongside cmt

You mean, as an alternative back-end for Metal.jl? Which wrapping solution would you propose to use for that? CxxWrap.jl is pretty laborious, and Cxx.jl isn't functional.

Probably before spending too much time on revamping libcmt it could be useful to agree a scope for Metal.jl.

Although the focus is currently on compute, as you probably noticed by the bits of Metal I didn't care to wrap in libcmt or the MTL wrappers, I see no problem in adding more broad functionality to Metal.jl. The scope should be to offer Julia support for whatever the Metal libraries have to offer, or at least the bits that somebody cared to wrap and test. For me, that's only the compute parts, but feel free to work on other areas.

[habemus-papadum]

Thanks for reading! I will add a PR that concretely tries to clarify in code some of the confusing points of my writeup.

[maleadt]

With #117 merged, this should now be much easier to contribute to. In that PR, I've just copied whatever memory management we were doing, so those points remain. The Clang generator is also still very relevant, although we could probably get most of the benefits already by just having it generate enum definitions instead of the whole Objective-C call signatures (which are both difficult to generate, and might still evolve in ObjectiveC.jl).

@habemus-papadum Since you have a better understanding of the exact memory management semantics in ObjectiveC, feel free to propose changes on that front! I don't have plans for any other large-scale refactors right now, so hopefully we can stabilize the interface a bit.

[habemus-papadum]

Sounds good -- I was going to write up (& work on) a proposal for a clang-based MacOS Framework to Julia package generator. It would discuss memory management, testing strategy, syntax, api availability, edge cases, etc -- I think it is better suited to start that conversation in Objective-C.jl. If you agree, I think we close this issue as complete, and revisit in Metal.jl after a generator exists.

[maleadt]

Fine for me.