Finish support for devirtualizing the HAL API.

[benvanik]

The HAL C interface was designed to be devirtualized in size-optimized builds where only a single HAL backend is linked in. The implementation was not completed, though, and needs finishing.

The intent is that all of the vtables used within the HAL like iree_hal_buffer_vtable_t are stripped and dispatches to the vtables that all route through IREE_HAL_VTABLE_DISPATCH become direct function calls.

Instead of having a vtable that references some implementation function like iree_hal_heap_buffer_map_range, an alias for iree_hal_buffer_map_range_direct that points at iree_hal_heap_buffer_map_range would be created. Then any dispatch to the vtable through that macro becomes a direct call to type_prefix##method_name#_direct. The vtable is then never used, stripped, and LTO takes care of stripping unused HAL implementations and such.

This would reduce total binary size as only those HAL methods used would be required, improves call overhead as direct functions are used vs indirections through the vtables, and reduces binary size as no per-type vtables need to be stored.

There's a few gotchas that were never finished - like we still may need distinct vtable-like identifiers for type checking - but they can be unique byte memory locations vs. a full vtable with function pointers such that simple pointer equality works. We'll also need something that emits the _direct aliases - either automatically, via copy/paste, or macro magic around vtable definition. There may be some easier techniques for that too like making the vtable static and having _direct calls go through the vtable - in theory a compiler should be plenty smart to devirtualize that for us (so we still have the vtable struct but it's only ever used from within the compilation unit).

[ergawy]

Hello,
As mentioned on #34, I would like to start working on this issue.

I had a look under the hal directory to better understand what's there. However, since this is huge, I still miss the starting point. Since you stated devirtualization was started but not finished, can you point me to the relevant parts of the code where devirtualization is currently implemented and the test code that exercises it (if any)?

(note: I am new the runtime world so I hope you bear with the possibly stupid questions I might have :))

[benvanik]

Did some tests with the current style and the struct handle style: https://godbolt.org/z/3Wr6Ko5ET
clang is impressive O_O

We'd need to analyze our usage to see if this optimization is able to kick in - if it does work then we're done 🎉
This does require LTO, but anyone caring about performance or binary size needs to use LTO anyway.

[benvanik]

The static const on the vtable is load-bearing to DCE - for some reason extern const is used in IREE right now - LTO still kills it but it'd be nice to fix that.

update: c++ needs it to be extern; I can fix up all the C impls though

[benvanik]

Not going to pursue this for now. Making the vtables smaller would be more worthwhile.