WIP: Implementing Acceleration Structures #1967

AntarticCoder · 2023-07-06T15:28:06Z

This PR provides an implementation of the VK_KHR_acceleration_structure extension which provides a gateway to ray queries and ray tracing pipelines. This PR is still very WIP due to not being anywhere close to done. The reason for opening this PR so early on is to allow for more concrete discussion of the implementation of acceleration structures, and also keeps people up to date on the implementation.

This PR is related to:
#427
#1953 - Not directly related but may have some slight discussion on Acceleration Structures
#1956

Just setup for acceleration structures by adding the definitions of the extension where it is needed. I also added the physical device features and properties that are needed.

This commit adds a few items which are: * A list of functions that are needed to be implemented * An implementation of the vkGetAccelerationStructureBuildSizesKHR function * Fixed the parameters for the create and destroy acceleration structure in MVKDevice * Added the current functions in vulkan.mm

AntarticCoder · 2023-07-06T22:22:36Z

Acceleration Structure and Raytracing in general does not seem to be supported before MacOS 11, so Xcode 11.7 will always fail.

cdavis5e · 2023-07-06T22:26:33Z

Acceleration Structure and Raytracing in general does not seem to be supported before MacOS 11, so Xcode 11.7 will always fail.

Then the parts of MoltenVK that deal with Acceleration Structures need to be inside MVK_XCODE_12 blocks.

billhollings · 2023-07-07T16:12:55Z

Acceleration Structure and Raytracing in general does not seem to be supported before MacOS 11, so Xcode 11.7 will always fail.

Then the parts of MoltenVK that deal with Acceleration Structures need to be inside MVK_XCODE_12 blocks.

Agreed. But before forcing that, we should discuss whether that makes sense at this point. Xcode 11 is now 4 years old, and at some point, we must give up support for it, from a practicality perspective (like this one). Retaining support for Xcode 11 was added a couple of years ago because some devs required it for their internal processes.

A few months ago, I reached out to the community about this exact question, and received no responses. Unless we can determine a good reason for maintaining Xcode 11, maybe now is the time to drop support for it.

AntarticCoder · 2023-07-07T16:35:58Z

@billhollings MacOS 11 seems to have support from devices as old as 2013 and newer. So it's a matter of dropping support of these pre-2013 devices, as well as that, some people stay on MacOS 10 for support of 32 bit applications and other reasons. This is just something to take into consideration.

billhollings · 2023-07-07T16:39:42Z

A few months ago, I reached out to the community about this exact question, and received no responses. Unless we can determine a good reason for maintaining Xcode 11, maybe now is the time to drop support for it.

I have added a ping post to that feedback request thread.

@AntarticCoder Hold off wrapping your code in any MVK_XCODE_12 guards while this PR remains a WIP. When this PR is ready to go, based on any feedback we receive to my query ping, we can decide whether we need to actually implement those guard wraps, or abandon Xcode 11.

AntarticCoder · 2023-07-07T16:41:19Z

@billhollings Alright, I'll hold off on the MVK_XCODE_12 guards. Thanks

billhollings · 2023-07-07T16:44:26Z

@billhollings MacOS 11 seems to have support from devices as old as 2013 and newer. So it's a matter of dropping support of these pre-2013 devices, as well as that, some people stay on MacOS 10 for support of 32 bit applications and other reasons. This is just something to take into consideration.

The MVK_XCODE_12 guard is strictly for API compilation during MoltenVK builds (ie- will it build with the Metal API supported by Xcode 11). Support for older OS runtimes is handled independently, through things like respondsToSelector:.

AntarticCoder · 2023-07-07T16:48:52Z

@billhollings Ah, yes my mistake. Also, just a thought but only about 120 people actually watch this repository, so I'm not sure how many people will see your message.

This commit adds: * A .h and .mm file for Acceleration Structure commands * An acceleration structure command encoder into `MVKCommandBuffer` * An actual acceleration structure handle * And some other items that are not complete, or need to removed

Fixed the missing symbol for getPoolType in MVKCmdBuildAccelerationStructure by including it in MVKCommandPool.h. I also added the Build Acceleration structure command into definitions file.

Finished up what was needed for the MVKCmdBuildAccelerationStructure. The only 2 issues at the moment are the scratch buffer and the scratch buffer offset, to which a solution has been proposed. I plan to discuss this in the PR thread before trying out anything.

AntarticCoder · 2023-07-10T13:09:41Z

@billhollings @cdavis5e An issue I've run into during this PR, is accessing the provided scratch buffer, via the provided device address. To solve this, I got a reply from @K0bin in issue #1956, which is as followed.

@AntarticCoder @rcaridade145 The contents function will just give you a CPU pointer to the data of a shared buffer. That's not useful here unless you want to copy all the data around on the CPU every time. (which would also involve a GPU sync)

What you have to do is basically maintain a map that maps BDA VAs to their original buffer objects. Keep in mind that this VA map has to be extremely fast and should minimize locking as much as possible. An example for that can be found in vkd3d-Proton: https://github.com/HansKristian-Work/vkd3d-proton/blob/master/libs/vkd3d/va_map.c

Basically create a map from scratch that is fast, and thread safe, and when you call vkGetBufferDeviceAddress, we could push the address along with buffer. I just wanted to ask if this is a good idea, and what you would change about it.

This commit adds the copy acceleration structure, but does not add the commands that copy memory to and from an acceleration structure. As well as that I've added 2 files for a map that will store the device address along with the buffer. This map will also come in handy when getting the device address for the acceleration structure

K0bin · 2023-07-10T15:02:13Z

and when you call vkGetBufferDeviceAddress, we could push the address along with buffer

It's probably better to do that at buffer creation time and keep vkGetBufferDeviceAddress fast.

AntarticCoder · 2023-07-10T15:04:31Z

@K0bin But not every created buffer will be used via the device address. So if you pushed it on vkGetBufferDeviceAddress, you would effectivly be keeping uneeded buffers out of the map.

K0bin · 2023-07-10T15:32:57Z

Base it off of VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT.

AntarticCoder · 2023-07-10T15:44:06Z

Okay then, I'll get started on the implementation. Thanks @K0bin

A half done implementation of MVKMap. MVKMap aims to use the same API as std::unordered_map, and I used MVKSmallVector as an example of how to write MVKMap. I hope there aren't any bugs however, I'll probably do some tests off of the repository once I'm done

billhollings · 2023-07-10T21:30:25Z

Base it off of VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT.

Search for VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT in existing MoltenVK code. There is already an MVKSmallVector containing a list of these in MVKDevice::_gpuAddressableBuffers. Perhaps this could be modified to use a std::unordered_map, to be used to serve both purposes?

AntarticCoder · 2023-07-10T22:22:05Z

@billhollings That seems like a good idea, I'll go ahead and use that for now, and we can change it in the future if it's not getting the job done.

This commit finished off the build acceleration structure command. This is because in MVKDevice, we are now using a std::unordered_map instead of a custom map implementation.

AntarticCoder · 2023-07-11T14:32:46Z

6/12 commands have been implemented, so I'm halfway there. 🎉

MoltenVK/MoltenVK/Commands/MVKCmdAccelerationStructure.mm

This commit fixes a glaring issue with copying accelerations structures to memory. Previously, I would copy the acceleration structure to another acceleration structure, however the proper thing to do was to copy a buffer to an acceleration structure. This however has not fixed copy acceleration structure to memory.

This commit quickly fixes copying acceleration structures to memory, however I'm not sure if my implementation is right.

cdavis5e · 2023-07-18T21:07:39Z

MoltenVK/MoltenVK/Commands/MVKCmdAccelerationStructure.mm

+        return;
+    }
+
+    memcpy(_dstBuffer->getDeviceMemory()->getHostMemoryAddress(), (void*)_srcAccelerationStructure, sizeof(_srcAccelerationStructure));


The copy is supposed to be performed on the device, especially since it is possible that one or both of the resources are in device-local (i.e. MTLStorageModePrivate) memory. The -[MTLBlitCommandEncoder copyFromBuffer:sourceOffset:toBuffer:destinationOffset:size:] method is what you want here.

Also, you'll need a method on the MVKAccelerationStructure to get its underlying MVKBuffer. And, you'll need to make sure that the MTLAccelerationStructure and MTLBuffer underlying both objects have their underlying GPU storage aliased to each other for this to work right. The only way I can think of to ensure that is to use a placement MTLHeap. Unfortunately, this means this functionality will now be limited to macOS Ventura or iOS 16, in which the -[MTLHeap newAccelerationStructureWithDescriptor:offset:] method you need for that was introduced.

Note that, although MoltenVK has support for using placement MTLHeaps, it's currently disabled due to some rendering artifacts with render targets in heaps. (I really need to check if that's been fixed.) But that doesn't matter, because you need to use an MTLHeap even in the dedicated allocation case.

If I understand this correctly, you're proposing to implement Acceleration Structure copies using memory aliasing with a buffer and copying the buffer contents.

Why not just AccelerationStructureCommandEncoder::copy?

This particular copy is for copying (i.e. serializing) acceleration structures to buffers.

Oh, in that case your suggestion makes a lot of sense.

cdavis5e · 2023-07-18T21:09:34Z

MoltenVK/MoltenVK/Commands/MVKCmdAccelerationStructure.mm

+    MVKAccelerationStructure* mvkSrcAccStruct = (MVKAccelerationStructure*)serializedAccStruct;
+    id<MTLAccelerationStructure> srcAccelerationStructure = mvkSrcAccStruct->getMTLAccelerationStructure();
+
+    [accStructEncoder


Er, uh, you're still assuming the destination resource has a VkAccelerationStructure created at that address. You'll need to make an analogous change here to the one you need to make for vkCmdCopyAccelerationStructureToMemory() above.

MoltenVK/MoltenVK/Commands/MVKCmdAccelerationStructure.mm

This commit adds the functionality for a bottom and top level acceleration structure, which are not quite finished, but I'm pushing this because I'm unable to stash this.

This commit does not do much, however I'm updating to the next macos update, so I'd like to push so I don't lose everything.

billhollings · 2023-11-03T16:07:32Z

@AntarticCoder @cdavis5e

This PR seems to have stalled. I've received a request from a game studio using MoltenVK who would like to see VK_KHR_acceleration_structure completed, and is willing to fund that work.

If either of you have spare time, and are interested in receiving compensation to work on finishing VK_KHR_acceleration_structure, let me know and I'll put you in touch with them. Their schedule is not rushed, so this is something that could start anytime in the next month or two, and be something that could be fit in part-time.

This sponsor is actually interested in seeing general ray tracing added, so if you (or anyone else out there), is interested in working on a funded project to see the following completed (in order or priority, and similar schedule behavior as above), please let me know:

VK_KHR_acceleration_structure
VK_KHR_ray_tracing_pipeline
VK_KHR_ray_query
VK_KHR_pipeline_library

AntarticCoder · 2023-11-04T14:30:45Z

@billhollings

I’m sorry, I’ve been busy with since I have began school and never got around to finishing this PR. I am interested in receiving compensation for finishing up VK_KHR_acceleration_structure. I also would be interested in working on general ray-tracing as well. Could you somehow get me into contact with the game studio?

Thanks

cdavis5e · 2023-11-04T18:25:52Z

I'm interested in this. I've talked with Holochip, and they're also interested.

billhollings · 2023-11-06T22:42:27Z

@billhollings

I’m sorry, I’ve been busy with since I have began school and never got around to finishing this PR. I am interested in receiving compensation for finishing up VK_KHR_acceleration_structure. I also would be interested in working on general ray-tracing as well. Could you somehow get me into contact with the game studio?

Thanks

@AntarticCoder

I think it definitely makes sense to have you working on completing this PR. Can you shoot me an email at support@brenwill.com, and we'll sort things out. On your email, can you quote an hourly rate you'd like, how much time you have available, and where you are located (for how to best get you actually paid), please?

AntarticCoder · 2023-11-08T00:46:10Z

@billhollings

Just sent you an email.

zmarlon · 2023-12-26T19:03:35Z

Is there any news regarding this PR? I now own an M3 Max Macbook and could test if this is of interest.

kanerogers · 2024-05-27T04:55:14Z

Our game studio is interested in cross-platform ray tracing with Vulkan, wondering whether there's been any progress here.

Still not finished, just quickly saving my work on get build sizes

K0bin · 2024-05-30T04:56:32Z

How do you intend to work around the fact that Metal needs a list of all bottom level acceleration structures to build the TLAS while Vulkan only needs a GPU buffer address that contains that data?

You'll probably have to maintain a list that has every single BLAS and use that when creating the Metal TLAS.
Then in vkCmdBuildAccelerationStructure you prepare some kind of hashmap on the CPU for BLAS VkDeviceAddress -> uint32_t index. Then you run a compute shader that prepares the actual MTLAccelerationStructureInstanceDescriptors by doing a hashmap lookup for each instance to get the index.
Not great, maybe you can come up with a simpler solution.

This commit is pretty small and just adds AABBs to be allowed to be pushed to the acceleration structure.

AntarticCoder added 3 commits July 5, 2023 19:12

Setup for Implementing Acceleration Structures

72902a9

Just setup for acceleration structures by adding the definitions of the extension where it is needed. I also added the physical device features and properties that are needed.

Merge branch 'main' into khr-acceleration-structures

a65c358

AntarticCoder mentioned this pull request Jul 6, 2023

Raytracing support #427

Open

AntarticCoder force-pushed the khr-acceleration-structures branch from 898e09d to 5e5c4a7 Compare July 7, 2023 16:51

AntarticCoder force-pushed the khr-acceleration-structures branch from 5e5c4a7 to a1b0961 Compare July 7, 2023 16:55

AntarticCoder added 2 commits July 8, 2023 12:02

Fixed missing symbol for getPoolType in MVKCmdBuildAccelerationStructure

d409fbf

Fixed the missing symbol for getPoolType in MVKCmdBuildAccelerationStructure by including it in MVKCommandPool.h. I also added the Build Acceleration structure command into definitions file.

billhollings mentioned this pull request Jul 10, 2023

MoltenVK Enhancement Roadmap #1975

Open

Using std::unordered_map to store the Buffer Device Addresses

43a987c

This commit finished off the build acceleration structure command. This is because in MVKDevice, we are now using a std::unordered_map instead of a custom map implementation.

AntarticCoder force-pushed the khr-acceleration-structures branch from 542c3f8 to 9aae084 Compare July 11, 2023 14:31

cdavis5e requested changes Jul 16, 2023

View reviewed changes

MoltenVK/MoltenVK/Commands/MVKCmdAccelerationStructure.mm Outdated Show resolved Hide resolved

MoltenVK/MoltenVK/Commands/MVKCmdAccelerationStructure.mm Outdated Show resolved Hide resolved

AntarticCoder added 2 commits July 17, 2023 15:43

Fixed Copy Acceleration Structure to Memory

89a92fb

This commit quickly fixes copying acceleration structures to memory, however I'm not sure if my implementation is right.

AntarticCoder requested a review from cdavis5e July 18, 2023 13:50

cdavis5e requested changes Jul 18, 2023

View reviewed changes

AntarticCoder added 2 commits July 18, 2023 19:58

Acceleration structures with Levels

c6d97c2

This commit adds the functionality for a bottom and top level acceleration structure, which are not quite finished, but I'm pushing this because I'm unable to stash this.

Added command uses and MTLHeaps

1cf021e

This commit does not do much, however I'm updating to the next macos update, so I'd like to push so I don't lose everything.

AntarticCoder force-pushed the khr-acceleration-structures branch from edf5303 to 1cf021e Compare November 12, 2023 17:35

AntarticCoder added 3 commits November 12, 2023 12:36

Corrected Some Simple Build Errors

333f41b

Merge branch 'main' into khr-acceleration-structures

6662762

Correcting Build Errors due to Merge

1696c18

xirreal mentioned this pull request Nov 17, 2023

WIP: MacOS support (tracking PR) MCRcortex/vulkanite#36

Draft

AntarticCoder and others added 2 commits December 3, 2023 11:42

Merge branch 'KhronosGroup:main' into khr-acceleration-structures

607cde1

Added Function Definitions for Overwritten Functions

3937822

AntarticCoder force-pushed the khr-acceleration-structures branch 2 times, most recently from c688f5e to 3937822 Compare January 2, 2024 14:11

AntarticCoder and others added 4 commits January 2, 2024 09:12

Merge branch 'main' into khr-acceleration-structures

22766de

Quick Save

4664dad

Merge branch 'main' into khr-acceleration-structures

65dea0c

Merge branch 'main' into khr-acceleration-structures

2b560f1

Buggy Implementation on Get Build Sizes

4362ea1

Still not finished, just quickly saving my work on get build sizes

Implemented AABB Geometry for Build and Get Sizes

3f1fbde

This commit is pretty small and just adds AABBs to be allowed to be pushed to the acceleration structure.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Implementing Acceleration Structures #1967

WIP: Implementing Acceleration Structures #1967

AntarticCoder commented Jul 6, 2023 •

edited

Loading

AntarticCoder commented Jul 6, 2023

cdavis5e commented Jul 6, 2023

billhollings commented Jul 7, 2023

AntarticCoder commented Jul 7, 2023 •

edited

Loading

billhollings commented Jul 7, 2023

AntarticCoder commented Jul 7, 2023

billhollings commented Jul 7, 2023

AntarticCoder commented Jul 7, 2023 •

edited

Loading

AntarticCoder commented Jul 10, 2023 •

edited

Loading

K0bin commented Jul 10, 2023

AntarticCoder commented Jul 10, 2023

K0bin commented Jul 10, 2023

AntarticCoder commented Jul 10, 2023

billhollings commented Jul 10, 2023 •

edited

Loading

AntarticCoder commented Jul 10, 2023

AntarticCoder commented Jul 11, 2023

cdavis5e Jul 18, 2023

K0bin Sep 14, 2023

cdavis5e Sep 14, 2023

K0bin Sep 14, 2023

cdavis5e Jul 18, 2023

billhollings commented Nov 3, 2023 •

edited

Loading

AntarticCoder commented Nov 4, 2023

cdavis5e commented Nov 4, 2023

billhollings commented Nov 6, 2023

AntarticCoder commented Nov 8, 2023

zmarlon commented Dec 26, 2023

kanerogers commented May 27, 2024

K0bin commented May 30, 2024 •

edited

Loading

WIP: Implementing Acceleration Structures #1967

Are you sure you want to change the base?

WIP: Implementing Acceleration Structures #1967

Conversation

AntarticCoder commented Jul 6, 2023 • edited Loading

AntarticCoder commented Jul 6, 2023

cdavis5e commented Jul 6, 2023

billhollings commented Jul 7, 2023

AntarticCoder commented Jul 7, 2023 • edited Loading

billhollings commented Jul 7, 2023

AntarticCoder commented Jul 7, 2023

billhollings commented Jul 7, 2023

AntarticCoder commented Jul 7, 2023 • edited Loading

AntarticCoder commented Jul 10, 2023 • edited Loading

K0bin commented Jul 10, 2023

AntarticCoder commented Jul 10, 2023

K0bin commented Jul 10, 2023

AntarticCoder commented Jul 10, 2023

billhollings commented Jul 10, 2023 • edited Loading

AntarticCoder commented Jul 10, 2023

AntarticCoder commented Jul 11, 2023

cdavis5e Jul 18, 2023

Choose a reason for hiding this comment

K0bin Sep 14, 2023

Choose a reason for hiding this comment

cdavis5e Sep 14, 2023

Choose a reason for hiding this comment

K0bin Sep 14, 2023

Choose a reason for hiding this comment

cdavis5e Jul 18, 2023

Choose a reason for hiding this comment

billhollings commented Nov 3, 2023 • edited Loading

AntarticCoder commented Nov 4, 2023

cdavis5e commented Nov 4, 2023

billhollings commented Nov 6, 2023

AntarticCoder commented Nov 8, 2023

zmarlon commented Dec 26, 2023

kanerogers commented May 27, 2024

K0bin commented May 30, 2024 • edited Loading

AntarticCoder commented Jul 6, 2023 •

edited

Loading

AntarticCoder commented Jul 7, 2023 •

edited

Loading

AntarticCoder commented Jul 7, 2023 •

edited

Loading

AntarticCoder commented Jul 10, 2023 •

edited

Loading

billhollings commented Jul 10, 2023 •

edited

Loading

billhollings commented Nov 3, 2023 •

edited

Loading

K0bin commented May 30, 2024 •

edited

Loading