VK_EXT_metal_objects: alias VkImage to MTLBuffer

[blapalp]

Hi,

I have to interface with a library exporting image data only in MTLBuffer format, device-local.

The doc seems to only provide for MTLTexture import into VkImages, being unclear whether this would also work with MTLBuffers.
I tried importing the external metal handle via the VkImages' VkMemoryAllocateInfo.pNext = VkImportMetalBufferInfoEXT (VkImageCreateInfo expects a texture, not a metal buffer) and binding, but this silently fails.
OTOH, importing the MTLBuffer into a VkBuffer as described in the doc and subsequently doing a vkCopyBufferToImage works, but is not an option for performance reasons.
Same with [mtlBuffer newTextureWithDescriptor:textureDescriptor offset:0 bytesPerRow:pitch]: works, but also involves a copy as far as I know.

So, is this simply not supported or a bug? And what would my options be to do in-place memory interop MTLBuffer->VkImage.

[billhollings]

I'm not sure what your export-import frequency is, but have you verified that a buffer-to-texture copy actually does slow things down unacceptably?

Going the other route, MTLBuffer newTextureWithDescriptor:offset:bytesPerRow: does not involve a copy. It creates a MTLTexture as a buffer view, which shares the same memory, and which you could import into a VkImage in MoltenVK.

Since they share memory, changes are dynamic. You could establish the connection once, and then if your library changes the contents of the MTLBuffer, it would be reflected immediately in the MTLTexture (and VkImage) in MoltenVK.

[blapalp]

Oh, that's great, seems I got fooled by ChatGPT. So direct MTLBuffer-VkImage aliasing is UB atm?
(Copying large video frames is indeed a performance issue in my case).

[billhollings]

Importing a MTLBuffer into a VkImage is not so much UB, as currently impossible. 😉

The VK_EXT_metal_objects extension supports importing either a MTLTexture or an IOSurface into a VkImage. We'd need to add another Vulkan extension to define an API to import a MTLBuffer into a VkImage, and internally create the MTLTexture view on the MTLBuffer.

But as discussed above, you can do that last part outside MoltenVK, using newTextureWithDescriptor:offset:bytesPerRow:, and submit the resulting MTLTexture in a VkImportMetalTextureInfoEXT struct.

[blapalp]

Ok, thanks for the clarification! Btw, might be useful to add some validation or error to vkBindImageMemory, as this just fails silently (VK_SUCCESS).