Pixel-Perfect Occlusion Culling (PPOC)
Note; i’m not sure if this was done before. It is not revolutionary and maybe somebody else done it earlier, but i was not able to find a trace so far.
The Birth
Not so long ago, and while simplyfing the pipeline and overall feature set of Delusion and getting rid of stuff not needed anymore that were carried over from Mirage (the older engine), it sometimes becomes hard decision to get rid of something. There are things that takes no time to decide to throw out of the window, these things such as 3rd party libraries, because there will be either a better one to replace or becuase simply i’m not using much of the library, and will write my own solution that fits only my needs. 3rd party integration is not a big effort to worry about, this is why for me they’re simple to throw away. But there are things that took sometime to think about or write and polish from scratch, these are the type of things that i like to give enough time to take the finall call, to keep or not to keep.
One of those things, was what so called the ID attachment/rendertarget, it is not something unique or special. I’ve added it long time ago in Mirage for object selection or hovering, and that was carried over to Delusion and it is tangled in many places (hovering, highlighting, outlines,..etc.). When questioning a feature to keep or not to keep like this, i ask myself, few questions:
– Do i really need it?
– Can it be replaced with something else?
– Can i use it to do more, so i have more reasons to not remove it?
And for this ID pass feature, the first two questions, had the “Yes” answer without hesitation, but that last question sticked in my mind for few days. i know quite a few visual things that can be done with that additional rendertarget, things like outlines or some special highights effects and such. But can i use this ID pass to do more? Something that is beyond effects or post processing? Somethings that may save milliseconds not somthing that adds more milliseconds?
i left the questions in my head to process, and forgot about them, just a few side neurons running this task at the back of my head, and they will poke me back if an answer came (just a fancy way to decribe throwing it to the back of my head to be forgetted till it resurface naturally again if the subconscious mind did its magic and found an answer).
Till one day and out of nowhere, while was breaking down one of the games for a potential Behind the Pretty Frames future article, and while looking into culling workflow, i just resurfaced naturally in a sudden in the form of “Why not using the IDs to occlusion cull & draw only what we actually sees?”
And the rest is history!!
Few hours of thinking to get the entire thing visualized in my head, and then i put a sequence of tasks in my backlog, with the hope to revisit them as soon as i can to start implementing it, and to test it against Hi-Z culling (this was the technique i was investigating when the idea came to me).
Well, it is always the case with new names, PPOC or SCOC, pick the one you want!
i started thinking about renaming to SCOC after testing with brutally lower resoltuions, because it became no more Pixel-Perfect
And everything went to bed since….at the back of my head i know i want to have some time to try it out, but no time was suitable enough since…it slept deep for several months till..
The Kiss
One day i was twittering and saw that nice tweet of a milestone from Panagiotis of AnKi 3D, and didn’t hesitate to take a moment to admire how much i find IDs buffer very useful…i left a hint saying “multiple purposes” but didn’t want to elaborate further till i do test the idea myself.
And there we go again, it went to its pre allocated bed at the back of my head room where all other ideas are sleeping and waiting for a prince’s kiss to come back to life.
Fast forwared few few more months (almost nearing a year), and i was impacted by layoffs, and finally all the time in the world (well 24h a day) are mine again to do anything, and it took not time to decided without hesitation to dive into implementing the long waited PPOC as a refreshment & return to my forgotten engine API, knowing that Delusion already had 1/2 of the idea implemented in the pipeline (IDs for the sake of objects selection) made it a great candidate as a half feature to be worked on. After the completion, i carried it forward to another renderer as part of testing re-integrating it from scratch in an alien API, shaders were very simple anyways and the entire trick or workflow is in the pipelines setup & resources mangement.
The Workings
Idea is stupid & simple, and it goes as follow:
1. A pixel shader does a simple pass where it draw integer IDs per object/primitive/instance and of course, that pipeline need to benefits from depth tests during that pass. (i went with drawing on RG16_UINT @ full target resolution at beginning but went with 1/2 target res evntually, but can be reduced later where needed to more brutally smaller resolutions & still okay-ish)[0][6][7].
2. A compute shader executes right after that, this dispatch reads that previous IDs attachment, and then add any unique values found of IDs into a visibilities table (a UAV RWBuffer of uints).
3. When start drawing geometry for shading, either in the Vertex Shader or Geometry Shader (depends where you intially generated IDs at step 1), only process IDs that match values in the buffer table, anything else just early-out/return[8].
4. After drawing, make sure to run a shader that emptys the visibilities tables. (important, or will see some fun artifacts due to drawing or skipping things that shouldn’t).
5. Add barriers where needed, you guessed that!
much simpler than Hi-Z, little faster than Hi-Z, sounds very naive, but most importantly, re-using existing resouces instead of generating new ones (if you already using IDs rendertarget). And of course visiblities are calculated on one go for the entire frame, not at two steps to update visiblities between previous frame & current frame.
Now while this sounds easy & simple few steps, but there maybe some issues arise as edge cases, or perhaps few question going in mind like,..
What if submissions are not all done at once (not everything draw once, there are multiple submits in queue), what would be the best course of action?—this was a case i hit when i tried to integrate it outside Delusion
At my case, i decided to read the table everytime there will be draws, but also still free the table only once after all draw queues are done and most preferably by the end of the frame. It may make more sense for you to empty the table after each queue submits, and it makes sense logically, but so far for my use cases there were no matching IDs between the different queues, so i can live with it like that, single clear regardless the amount of draw submits, and just keeping an eye on it, after all moving where the clear happens won’t be too much of an effort (i ended up leaving it as a flag that can be toggled anytime, but still defaulting to end of frame) .
Or, What would be the best way to handle instanced drawing without doing any type of CPU side work?
At my case, i decided to use the G to store the instance index, this is why i went with 2 channel initially, with that additional channels, we can expand the possible IDs per single given ID.[4]
You maybe asking, wouldn’t this add to the total cost of the frame?
Yes of course, any shader code or compute invocations adds to the total cost, but the idea here was to find another use case for the already existing IDs buffer, and culling was a great option considering the culling cost regardless what method. Even if at worst case scenario the culling cost through IDs buffer is costing the exact same as Hi-Z culling, then it is still a win for me to cull through IDs buffer instead of Hi-Z, because now i can handle the occlusion culling through IDs buffer (existing step/cost), and get rid of the Hi-Z entirley. In another word, why would i’ve Hi-Z cost/resources for culling in addition IDs cost for selection, highlighting and outlines, where i can have only one of both costs that handle all cases (IDs cost for culling + selection/highlighting/outlines still less than Hi-Z at initial tests)[1][5].
The only change in the IDs workflow now, that i needed to move the IDs buffer from being a pass taking place after everything else (after CPU or Hi-Z culling) to be the first thing at the begining of the frame, because everything else will count on its result.
Now i want to leave you with this two gifs, at left is very basic setup, no culling enabled, just pass vert & frag shader, and anything renders, will show the wireframe, you can see a lot of geometry processed already behind the larger one that obsecures half of the frame. At the other hand with the IDs buffer culling (or Pixel-Perfect Occlusion Culling), you will see wireframes (draws) only for things that actaully visible in the final frame, anything culled by objects, it just skips early at the vertex shaders level before we reaches the lighing calculations [2][3], so if there are moments in that PPOC gif where nothing happens, basically because culling happens.
And for IDs example
The Notes
[0] Drawing the IDs image can be direct or indirect, up to you and your preferences and how it would fit inside your pipeline.
[1] The main goal here was not to make a silver bullet occlusion culling solution that fit all cases (it can though with more refinement), but to make use of what i already have in the pipeline and try to reduce extra work (building depth pyramid or doing bounding box tests). Most importantly, it need to fit with my engine/game than it becomes a solution for all sorts of games.
[2] This can be improved by moving the culling from Vertex Shader to the Geometry Shader, and benefit from something like SV_PrimitiveID. But i wanted it that way for my engine because i try to reduce utilization of geometry shaders where possible. If the move will gain me 0.1ms or more, i’ll do it, otherwise, wills tick with vertex shaders.
[3] This can be even handled partly in the CPU to avoid reaching the Vertex/Geometry Shader stage invocation (why call invoke VS if we going to skip it, valid point), but i went that route because i wanted it to be fully on the GPU. But keep in mind, if moved to CPU, then instances would need to handled in a different way, more buffers to bind, more back & forth,..etc. and i was satisfied with that existing results vs simplified workflow. Whatever i can throw to the GPU i’ll throw to the GPU.
[4] Instead of using a custom math calculation to store the IDs in the visibility table, can rely on an additional UAV buffer with atomic adds to work as a counter buffer. It is nice known technique and an option here, but i didn’t like to go with it here, not needed for my use case and i prefer as less buffers as possible. But i’ve explored it as a future option if my maps/worlds got larger enought to make the current implementation fall apart.
[5] The main point behind this method is to use what is already existing, and the core focus is to reduce the lighting calculations in the fragment shader as much as possible without relying on CPU occlusion culling, or per object bounds checks (regardless CPU or GPU) and try to benefit from something already exist that went through the hardware rasterization already like depth tests (i refer to the IDs shader), and try to finding a cheaper alternate to something like Hi-Z occlusion culling that works for my engine.
[6] The rendertarget size is a good optimization, but at certain reduction resolution there maybe some artifacts. So far rendering at 1/2 or 1/4 worked fine, but less can be problematic.
[7] The rendertarget format itself is negotiatable of course, can be at 8 bit not 16 or 32, depends on the target project/game and map density, and can be just a single channel format like R8_UINT, R16_UINT, R32_UINT or such, itt depends on use case and many factors, but can be tailored, and you can even do some encoding to store more colors or store colors in a different way. But the reason i went with RG16_UINT for my intial implementation was to make sure instaced draw store unique IDs per instance easily, as initially all instances of a single draw were storing the same ID, and that would kill the overall idea, i needed to cull at instance level too. And the other reason was to avoid encoding in & back colors if possible.
[8] Yes at cases where geometry will be culled, you already paid the CPU cost of invoking a vertex/geo shader that will later early-out, but this cost is much less than doing any sort of CPU sorting/culling pre-invoking the shader.
-m