Proposed VK_EXT_metal_objects extension to expose Metal objects to Vulkan apps

[billhollings]

Two calls:

• vkGetMetalObjectsEXT() to retrieve one or more metal objects from the associated Vulkan objects (each 1:1).
• vkSetMetalObjectsEXT() to set one or more metal objects into the associated Vulkan objects (each 1:1).

Each call makes use of a set of structs chained to the pNext of the VkMetalObjectsInfoEXT, each of which connects one Metal object with a Vulkan object (reading/writing the Metal object from/to the Vulkan object).

Multiple objects can be read/written with each call if multiple pNext structs are included.

#define VK_EXT_metal_objects 1
#ifdef __OBJC__
@protocol MTLDevice;
typedef id<MTLDevice> MTLDevice_id;
#else
typedef void* MTLDevice_id;
#endif

#ifdef __OBJC__
@protocol MTLCommandQueue;
typedef id<MTLCommandQueue> MTLCommandQueue_id;
#else
typedef void* MTLCommandQueue_id;
#endif

#ifdef __OBJC__
@protocol MTLBuffer;
typedef id<MTLBuffer> MTLBuffer_id;
#else
typedef void* MTLBuffer_id;
#endif

#ifdef __OBJC__
@protocol MTLTexture;
typedef id<MTLTexture> MTLTexture_id;
#else
typedef void* MTLTexture_id;
#endif

typedef struct __IOSurface* IOSurfaceRef;
#ifdef __OBJC__
@protocol MTLSharedEvent;
typedef id<MTLSharedEvent> MTLSharedEvent_id;
#else
typedef void* MTLSharedEvent_id;
#endif

#define VK_EXT_METAL_OBJECTS_SPEC_VERSION 1
#define VK_EXT_METAL_OBJECTS_EXTENSION_NAME "VK_EXT_metal_objects"
typedef struct VkGetMetalObjectsInfoEXT {
    VkStructureType    sType;
    const void*        pNext;
} VkGetMetalObjectsInfoEXT;

typedef struct VkGetMetalDeviceInfoEXT {
    VkStructureType    sType;
    const void*        pNext;
    MTLDevice_id       mtlDevice;
} VkGetMetalDeviceInfoEXT;

typedef struct VkGetMetalCommandQueueInfoEXT {
    VkStructureType       sType;
    const void*           pNext;
    VkQueue               queue;
    MTLCommandQueue_id    mtlCommandQueue;
} VkGetMetalCommandQueueInfoEXT;

typedef struct VkGetMetalBufferInfoEXT {
    VkStructureType    sType;
    const void*        pNext;
    VkBuffer           buffer;
    MTLBuffer_id       mtlBuffer;
    VkDeviceSize       mtlBufferOffset;
} VkGetMetalBufferInfoEXT;

typedef struct VkGetMetalTextureInfoEXT {
    VkStructureType       sType;
    const void*           pNext;
    VkImage               image;
    VkImageView           imageView;
    VkBufferView          bufferView;
    VkImageAspectFlags    aspectMask;
    MTLTexture_id         mtlTexture;
} VkGetMetalTextureInfoEXT;

typedef struct VkGetMetalIOSurfaceInfoEXT {
    VkStructureType    sType;
    const void*        pNext;
    VkImage            image;
    IOSurfaceRef       ioSurface;
} VkGetMetalIOSurfaceInfoEXT;

typedef struct VkGetMetalEventInfoEXT {
    VkStructureType      sType;
    const void*          pNext;
    VkEvent              event;
    MTLSharedEvent_id    mtlSharedEvent;
} VkGetMetalEventInfoEXT;

typedef struct VkBufferCreateMetalBufferInfoEXT {
    VkStructureType    sType;
    const void*        pNext;
    MTLBuffer_id       mtlBuffer;
    VkDeviceSize       mtlBufferOffset;
} VkBufferCreateMetalBufferInfoEXT;

typedef struct VkImageCreateMetalTextureInfoEXT {
    VkStructureType       sType;
    const void*           pNext;
    VkImageAspectFlags    aspectMask;
    MTLTexture_id         mtlTexture;
} VkImageCreateMetalTextureInfoEXT;

typedef struct VkImageCreateMetalIOSurfaceInfoEXT {
    VkStructureType    sType;
    const void*        pNext;
    IOSurfaceRef       ioSurface;
} VkImageCreateMetalIOSurfaceInfoEXT;

typedef void (VKAPI_PTR *PFN_vkGetMetalObjectsEXT)(VkDevice device, VkGetMetalObjectsInfoEXT* pMetalObjectsInfo);

#ifndef VK_NO_PROTOTYPES
VKAPI_ATTR void VKAPI_CALL vkGetMetalObjectsEXT(
    VkDevice                                    device,
    VkGetMetalObjectsInfoEXT*                   pMetalObjectsInfo);
#endif

[billhollings]

Hi Everyone… Based on the discussion on last week’s portability call, I’ve updated the definition <#34> of the VK_EXT_metal_objects extension. It’s now a Device extension, and VkDevice is now the dispatchable object on the get/set calls. The type and struct definitions are all now functional. I’ve built a MoltenVK branch <https://github.com/KhronosGroup/MoltenVK/tree/vk-ext-metal-objects> that supports the extension, and have tested it. Now that the definition is complete and working, I’m going to move onto the documentation components of the spec for this extension. I’d appreciate any feedback on the definitions and functionality. …Bill

…

On Nov 18, 2021, at 5:05 PM, Bill Hollings ***@***.***> wrote: Two calls: vkGetMetalObjectsEXT() to retrieve one or more metal objects from the associated Vulkan objects (each 1:1). vkSetMetalObjectsEXT() to set one or more metal objects into the associated Vulkan objects (each 1:1). Each call makes use of a set of structs chained to the pNext of the VkMetalObjectsInfoEXT, each of which connects one Metal object with a Vulkan object (reading/writing the Metal object from/to the Vulkan object). Multiple objects can be read/written with each call if multiple pNext structs are included. #define VK_EXT_metal_objects 1 #define VK_EXT_METAL_OBJECTS_SPEC_VERSION 1 #define VK_EXT_METAL_OBJECTS_EXTENSION_NAME "VK_EXT_metal_objects" typedef struct VkMetalObjectsInfoEXT { VkStructureType sType; const void* pNext; } VkMetalObjectsInfoEXT; typedef struct VkMetalDeviceInfoEXT { VkStructureType sType; const void* pNext; id<MTLDevice>* pMTLDevice; } VkMetalDeviceInfoEXT; typedef struct VkMetalCommandQueueInfoEXT { VkStructureType sType; const void* pNext; VkQueue queue; id<MTLCommandQueue>* pMTLCommandQueue; } VkMetalCommandQueueInfoEXT; typedef struct VkMetalBufferInfoEXT { VkStructureType sType; const void* pNext; VkBuffer buffer; id<MTLBuffer>* pMTLBuffer; } VkMetalBufferInfoEXT; typedef struct VkMetalTextureInfoEXT { VkStructureType sType; const void* pNext; VkImage image; VkImageAspectFlags aspectMask; id<MTLTexture>* pMTLTexture; } VkMetalTextureInfoEXT; typedef struct VkMetalIOSurfaceInfoEXT { VkStructureType sType; const void* pNext; VkImage image; IOSurfaceRef* pIOSurface; } VkMetalIOSurfaceInfoEXT; typedef void (VKAPI_PTR *PFN_vkGetMetalObjectsEXT)(VkPhysicalDevice physicalDevice, const VkMetalObjectsInfoEXT* pMetalObjectsInfo); typedef VkResult (VKAPI_PTR *PFN_vkSetMetalObjectsEXT)(VkPhysicalDevice physicalDevice, const VkMetalObjectsInfoEXT* pMetalObjectsInfo); #ifndef VK_NO_PROTOTYPES VKAPI_ATTR void VKAPI_CALL vkGetMetalObjectsEXT( VkPhysicalDevice physicalDevice, const VkMetalObjectsInfoEXT* pMetalObjectsInfo); VKAPI_ATTR VkResult VKAPI_CALL vkSetMetalObjectsEXT( VkPhysicalDevice physicalDevice, const VkMetalObjectsInfoEXT* pMetalObjectsInfo); #endif — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#34>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACXAMV45VXR6CKQ2A4YRW3UMV2DHANCNFSM5IKWSWEQ>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[kvark]

Are the pointers guaranteed to be valid id<> after these calls?
There are cases where a Vulkan on Metal implementation doesn't have a corresponding object 1:1.
For example, CPU-visible buffer may be sub-allocated, or a CPU-visible texture may be backed by a buffer.

[billhollings]

CPU-visible buffer may be sub-allocated

Good point. Thanks. I've added VkMetalBufferInfoEXT:: pMTLBufferOffset to return the offset of the VkBuffer in the returned MTLBuffer.

CPU-visible texture may be backed by a buffer

Not sure if this matters, unless there is a use case for retrieving buffer memory underlying the texture? Current use cases focus on retrieving the MTLTexture in order to be able to have it interact with other Apple API's. Are there use cases where it would matter that it is backed by a buffer?

[billhollings]

Do we need to extract a MTLTexture from VkImageView, in addition to VkImage?

[billhollings]

Do we need to extract a MTLTexture from VkImageView, in addition to VkImage?

This has been included now, along with MTLTexture from VkBufferView.

[kanerogers]

This extension has now been implemented: https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_EXT_metal_objects.html