Jekyll2023-01-02T23:45:54+00:00/feed.xmlEric MellinoRenderDoc Integration in Veldrid2019-01-23T06:00:00+00:002019-01-23T06:00:00+00:00/graphics/2019/01/23/renderdoc-integration-in-veldrid<p><em>This post describes an upcoming feature in Veldrid 4.6.0</em></p>
<p>Due to the complexity of modern graphics APIs and techniques, it can often be difficult to identify the source of rendering bugs. API’s like Vulkan include validation layers which help alert you to invalid usage at runtime. However, there is still a large class of programming errors which can result in problems like the dreaded “black screen”.</p>
<p><img src="/images/black-screen.png" alt="Black Screen" class="img-responsive" /></p>
<p><em>Where’s my triangle?</em></p>
<p>Tools like <a href="https://renderdoc.org">RenderDoc</a> let you inspect fine-grained details about the state of your system at each draw call, which can save a lot of valuable time that might otherwise be spent agonizing over the details of your code. Problems like a bad depth test, bad face culling, or an invalid scissor rectangle can easily be identified and dealt with.</p>
<p>Although most of its functionality is easily accessible through the graphical UI, RenderDoc also exposes an <a href="https://renderdoc.org/docs/in_application_api.html">in-application API</a>. This allows you to configure and trigger RenderDoc captures from inside any part of your application. While RenderDoc’s graphical UI can only capture entire frames at once (which may include lots of extraneous information), the in-application API is more flexible, and can be used to capture a small portion of a frame, or some process that doesn’t occur strictly within a frame: perhaps a one-time render pass for an environment map (performed during startup), or a periodic compute pass. Capturing a smaller part of your code, and having the flexibility to start and stop the capture when you want, makes it much easier to isolate potentially problematic areas from each other.</p>
<h2 id="veldrid-support">Veldrid Support</h2>
<p>The upcoming Veldrid 4.6.0 release includes a new helper library, Veldrid.RenderDoc, which is a simple .NET wrapper for RenderDoc’s in-application API. With it, you can configure, collect, and save RenderDoc captures from inside your application. Additionally, you can launch and manage the “Replay UI” from within your app to quickly debug a capture.</p>
<p>Veldrid.RenderDoc relies on the RenderDoc shared library being available on the library load path, or its path can be passed in explicitly:</p>
<pre><code class="language-C#">public class RenderDoc
{
public static bool Load(out RenderDoc renderDoc);
public static bool Load(string renderDocLibPath, out RenderDoc renderDoc);
}
</code></pre>
<p>Once a <code class="language-plaintext highlighter-rouge">RenderDoc</code> instance is obtained, a variety of settings can be tweaked, and then captures can be taken quite easily:</p>
<pre><code class="language-C#">RenderDoc.Load(out RenderDoc rd); // Load RenderDoc from the default locations.
rd.SetCaptureSavePath("my/special/path"); // Save captures into a particular folder.
rd.TriggerCapture(); // Capture the next frame.
rd.StartFrameCapture(); // Start capturing.
// Submit graphics or compute work to a Veldrid.GraphicsDevice.
rd.EndFrameCapture(); // Stop capturing and save.
rd.LaunchReplayUI(); // Launch the replay UI, with previous captures already loaded in.
</code></pre>
<p>In order for RenderDoc to successfully hook into the graphics API being used, a <code class="language-plaintext highlighter-rouge">RenderDoc</code> instance must be created before the <code class="language-plaintext highlighter-rouge">Veldrid.GraphicsDevice</code> that you wish to debug has been created. If your application allows it, you can re-create your GraphicsDevice after loading RenderDoc in order to allow it to hook the necessary functions. The Veldrid sample application does this, and allows RenderDoc to be loaded at any time, even after startup.</p>
<p><img src="/images/renderdoc-load-fade.gif" alt="Loading RenderDoc" class="img-responsive" /></p>
<p><a href="/images/renderdoc-capture-fades.gif"><img src="/images/renderdoc-capture-fades.gif" alt="Debugging with RenderDoc" /></a></p>
<h2 id="platform-support">Platform Support</h2>
<p>Veldrid.RenderDoc can be used on Windows and Linux. On Windows, <code class="language-plaintext highlighter-rouge">RenderDoc.Load</code> will attempt to load the shared library from the standard global install path, if it exists. On Linux, librenderdoc.so should be on the library load path, or the <code class="language-plaintext highlighter-rouge">Load</code> overload that accepts a full path should be used.</p>
<p>RenderDoc currently supports capturing and debugging Vulkan, Direct3D, OpenGL, and OpenGL ES applications. When using Veldrid, the Metal backend is the only one that RenderDoc cannot capture and debug.</p>This post describes an upcoming feature in Veldrid 4.6.0Overhauling ImGui.NET2018-10-12T01:00:00+00:002018-10-12T01:00:00+00:00/graphics/2018/10/12/imgui-net-overhaul<p>I’m happy to finally release the new, overhauled version of <a href="https://github.com/mellinoe/imgui.net">ImGui.NET</a>. The library has been re-built from the ground up, utilizing the new auto-generated <a href="https://github.com/cimgui/cimgui">cimgui</a> library and its associated tools. Previously, updating ImGui.NET was a very time-consuming and elaborate process, done by hand, and it often lagged behind official Dear ImGui releases. Instead of continuing that, I’ve implemented a code generator which processes cimgui’s pre-parsed data files and spits out a bunch of C# code automatically. There are a lot of benefits to this new approach, and it will allow me to keep the library up to date much more easily and painlessly. I’ve also taken this opportunity to improve the usability of the library a great deal, and to line up the C# interface and versioning more closely with C++.</p>
<h2 id="layers-and-design">Layers and Design</h2>
<p>As before, there are two layers to ImGui.NET: the raw, unsafe native layer (<code class="language-plaintext highlighter-rouge">ImGuiNative</code>), and the safe, C#-friendly layer (<code class="language-plaintext highlighter-rouge">ImGui</code>). Previously, a very common problem was some functionality being available to the low-level layer, but not to the friendly high-level layer. Since both layers are automatically generated now, there are no gaps between the two. Additionally, new features added to Dear ImGui will automatically surface in both places in ImGui.NET. In most cases, users of ImGui.NET should never need to touch unsafe code, and the high-level <code class="language-plaintext highlighter-rouge">ImGui</code> class will suffice. For advanced scenarios, low-level access may still be more convenient, and the option to drop down to <code class="language-plaintext highlighter-rouge">ImGuiNative</code> remains.</p>
<p>It is also painless to utilize the auto-generation machinery to create a different version of ImGui.NET for an experimental branch of Dear ImGui – for example, <a href="https://github.com/ocornut/imgui/issues/2109">the docking branch</a>. Pointing ImGui.NET’s code generator at the processed output from that branch will give you a fully-usable library exposing all of the new functions and types, including safe wrappers.</p>
<h2 id="safety">Safety</h2>
<p>Some care has been taken to automatically generated safe wrapper code for the library. In many places, Dear ImGui expects that you will interact with it through various pointers to structures. In order to simplify and protect these patterns in C#, I’ve introduced a number of “Ptr”-suffixed structures, each of which represent a specific typed pointer. <code class="language-plaintext highlighter-rouge">ImGui.GetIO</code>, for example, now returns an <code class="language-plaintext highlighter-rouge">ImGuiIOPtr</code>, which is a safe struct wrapper over the native <code class="language-plaintext highlighter-rouge">ImGuiIO*</code> that the function returns. It provides safe access to all of that type’s members and functions, and requires no unsafe code to use. Unlike the previous version, it allocates no garbage-collected memory at all, and does not make any copies when fields are accessed, utilizing managed references instead. These “Ptr” structures should be viewed as a thin wrapper over a pointer, and are implicitly convertible back and forth with those native types.</p>
<p>Previously-opaque structures like <code class="language-plaintext highlighter-rouge">ImVector<T></code> are also much more friendly to C# than they were before, and give you safe, copy-free access to individual elements inside the vector.</p>
<h2 id="breaking-changes">Breaking Changes</h2>
<p>ImGui.NET 1.65.0 introduces some breaking changes if you are upgrading from 0.4.7. Many structure, enum, method, and parameters have been renamed so that they are identical to their C++ counterparts. These should be viewed as one-time breaks; future versions of ImGui.NET will continue to match the C++ naming as closely as possible.</p>
<p>As a result of the bump to Dear ImGui 1.65, there are also several functions that have been removed or deprecated. This category of breaking change is documented well in Dear ImGui’s release notes.</p>
<h2 id="versioning">Versioning</h2>
<p>Going forward, ImGui.NET will use a versioning scheme that more closely lines up with native Dear ImGui. To start, this initial NuGet package will be versioned 1.65.0, corresponding to <a href="https://github.com/ocornut/imgui/releases/tag/v1.65">v1.65 of Dear ImGui</a>.</p>
<p>Note that previous releases of ImGui.NET were versioned from 0.1.0+. Version 1.65.0 contains breaking changes from 0.4.7 (the last release of the previous series), and will not necessarily maintain compatibility between updates. Going forward, I intend to inherit the deprecations and removals from Dear ImGui itself, rather than maintain strict binary compatibility between versions of ImGui.NET.</p>I’m happy to finally release the new, overhauled version of ImGui.NET. The library has been re-built from the ground up, utilizing the new auto-generated cimgui library and its associated tools. Previously, updating ImGui.NET was a very time-consuming and elaborate process, done by hand, and it often lagged behind official Dear ImGui releases. Instead of continuing that, I’ve implemented a code generator which processes cimgui’s pre-parsed data files and spits out a bunch of C# code automatically. There are a lot of benefits to this new approach, and it will allow me to keep the library up to date much more easily and painlessly. I’ve also taken this opportunity to improve the usability of the library a great deal, and to line up the C# interface and versioning more closely with C++.Veldrid Support for SPIR-V Shaders2018-06-25T19:00:00+00:002018-06-25T19:00:00+00:00/graphics/2018/06/25/veldrid-spirv<p><a href="https://mellinoe.github.io/veldrid-docs/">Veldrid</a> is a low-level graphics library written in C# that allows you to create GPU-accelerated applications targeting wide variety of platforms, without dealing with platform-specific graphics APIs. Although Veldrid aims to be as portable as possible, one pain point has always been shader code, which differs between platforms. Writing your shaders multiple times is error-prone, limits your portability, and can quickly become a big hassle. Other portable graphics libraries and game engines take different approaches to tackling this problem. Many libraries support a single “official” shading language (often HLSL or a variant, but occasionally a custom shading language) and translate it into a number of shading languages, depending on the graphics APIs being targeted. My <a href="https://github.com/mellinoe/ShaderGen/">ShaderGen</a> project can be seen as one such custom shading language.</p>
<p>A very promising new option for portable shaders is the <a href="https://www.khronos.org/registry/spir-v/">Khronos Group’s SPIR-V language</a>.</p>
<blockquote>
<p>SPIR-V is a binary intermediate language for representing graphical-shader stages and compute kernels.</p>
</blockquote>
<p>SPIR-V is a simple bytecode language for graphics and compute that can be targeted from several languages (including GLSL and HLSL), with more in development. There are also a variety of post-processing tools, optimizers, and debugging utilities available for it. Overall, it is a well-supported language with a very healthy and productive ecosystem developing around it.</p>
<p>Today, I’m releasing a Veldrid extension library called Veldrid.SPIRV, which provides support for loading SPIR-V shaders on all of Veldrid’s supported backends. Veldrid.SPIRV is built on top of <a href="https://github.com/KhronosGroup/SPIRV-Cross">SPIRV-Cross</a>, a library for translating SPIR-V bytecode into several high-level shading languages. With Veldrid.SPIRV, you can write your shaders in any language targeting SPIR-V and use them easily with Veldrid.</p>
<p>Veldrid.SPIRV is available on NuGet.org: <a href="https://www.nuget.org/packages/Veldrid.SPIRV"><img src="https://img.shields.io/nuget/v/Veldrid.SPIRV.svg" alt="NuGet" /></a></p>
<h2 id="veldrid-support">Veldrid Support</h2>
<p>Veldrid.SPIRV exposes several extension methods on ResourceFactory which allow you to create Shaders from SPIR-V bytecode. In order to create a Veldrid Pipeline from SPIR-V, you need to provide the bytecode for all shader stages being used. This is because Veldrid.SPIRV needs to be aware of the full set of shader resources (Buffers, Textures, and Samplers) used by a Pipeline in order to assign the correct “slots” for each resource.</p>
<p>Based on the type of ResourceFactory passed in, Veldrid.SPIRV will figure out which target language is needed, and will automatically generate the appropriate shader code and compile it for you. Most people will just need these two extension methods:</p>
<figure class="highlight"><pre><code class="language-c#" data-lang="c#"><span class="c1">// Create a set of Shaders usable in a graphics Pipeline.</span>
<span class="k">public</span> <span class="k">static</span> <span class="n">Shader</span><span class="p">[]</span> <span class="nf">CreateFromSpirv</span><span class="p">(</span>
<span class="k">this</span> <span class="n">ResourceFactory</span> <span class="n">factory</span><span class="p">,</span>
<span class="n">ShaderDescription</span> <span class="n">vertexShaderDescription</span><span class="p">,</span>
<span class="n">ShaderDescription</span> <span class="n">fragmentShaderDescription</span><span class="p">,</span>
<span class="n">CrossCompileOptions</span> <span class="n">options</span><span class="p">);</span>
<span class="c1">// Create a Shader usable in a compute Pipeline.</span>
<span class="k">public</span> <span class="k">static</span> <span class="n">Shader</span> <span class="nf">CreateFromSpirv</span><span class="p">(</span>
<span class="k">this</span> <span class="n">ResourceFactory</span> <span class="n">factory</span><span class="p">,</span>
<span class="n">ShaderDescription</span> <span class="n">computeShaderDescription</span><span class="p">,</span>
<span class="n">CrossCompileOptions</span> <span class="n">options</span><span class="p">);</span></code></pre></figure>
<h2 id="specialization-constants">Specialization Constants</h2>
<p>SPIR-V and Vulkan have support for “Specialization Constants”, which are an interesting feature providing greater flexibility to shaders. Specialization Constants are constants within a shader program that can be substituted with new values when a Pipeline is created. Likewise, Metal shaders can contain “function constants”, which serve roughly the same purpose. I’ve added support for both of these concepts to Veldrid through a new SpecializationConstant type. When constructing a new Pipeline, you can provide an array of SpecializationConstants which will influence the behavior of your shaders.</p>
<p>Here is an example fragment shader which contains several Specialization Constants. When you create one or more Pipelines with this shader, you can override these values without generating new SPIR-V bytecode or re-compiling your shader at all.</p>
<figure class="highlight"><pre><code class="language-glsl" data-lang="glsl"><span class="cp">#version 450
</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">set</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="n">binding</span> <span class="o">=</span> <span class="mi">0</span><span class="p">)</span> <span class="k">uniform</span> <span class="n">texture2D</span> <span class="n">Tex</span><span class="p">;</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">set</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="n">binding</span> <span class="o">=</span> <span class="mi">1</span><span class="p">)</span> <span class="k">uniform</span> <span class="n">sampler</span> <span class="n">Smp</span><span class="p">;</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">constant_id</span> <span class="o">=</span> <span class="mi">0</span><span class="p">)</span> <span class="k">const</span> <span class="kt">bool</span> <span class="n">UseTexture</span> <span class="o">=</span> <span class="nb">false</span><span class="p">;</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">constant_id</span> <span class="o">=</span> <span class="mi">1</span><span class="p">)</span> <span class="k">const</span> <span class="kt">bool</span> <span class="n">FlipTexture</span> <span class="o">=</span> <span class="nb">false</span><span class="p">;</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">constant_id</span> <span class="o">=</span> <span class="mi">2</span><span class="p">)</span> <span class="k">const</span> <span class="kt">float</span> <span class="n">RedChannel</span> <span class="o">=</span> <span class="mi">0</span><span class="p">.</span><span class="mi">1</span><span class="n">f</span><span class="p">;</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">constant_id</span> <span class="o">=</span> <span class="mi">3</span><span class="p">)</span> <span class="k">const</span> <span class="kt">float</span> <span class="n">GreenChannel</span> <span class="o">=</span> <span class="mi">0</span><span class="p">.</span><span class="mi">1</span><span class="n">f</span><span class="p">;</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">constant_id</span> <span class="o">=</span> <span class="mi">4</span><span class="p">)</span> <span class="k">const</span> <span class="kt">float</span> <span class="n">BlueChannel</span> <span class="o">=</span> <span class="mi">0</span><span class="p">.</span><span class="mi">1</span><span class="n">f</span><span class="p">;</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">location</span> <span class="o">=</span> <span class="mi">0</span><span class="p">)</span> <span class="k">in</span> <span class="kt">vec2</span> <span class="n">fsin_TexCoords</span><span class="p">;</span>
<span class="k">layout</span> <span class="p">(</span><span class="n">location</span> <span class="o">=</span> <span class="mi">0</span><span class="p">)</span> <span class="k">out</span> <span class="kt">vec4</span> <span class="n">fsout_Color0</span><span class="p">;</span>
<span class="kt">void</span> <span class="nf">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="n">UseTexture</span><span class="p">)</span>
<span class="p">{</span>
<span class="kt">vec2</span> <span class="n">uv</span> <span class="o">=</span> <span class="n">fsin_TexCoords</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="n">FlipTexture</span><span class="p">)</span> <span class="p">{</span> <span class="n">uv</span><span class="p">.</span><span class="n">y</span> <span class="o">=</span> <span class="mi">1</span> <span class="o">-</span> <span class="n">uv</span><span class="p">.</span><span class="n">y</span><span class="p">;</span> <span class="p">}</span>
<span class="n">fsout_Color0</span> <span class="o">=</span> <span class="n">texture</span><span class="p">(</span><span class="kt">sampler2D</span><span class="p">(</span><span class="n">Tex</span><span class="p">,</span> <span class="n">Smp</span><span class="p">),</span> <span class="n">uv</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">else</span>
<span class="p">{</span>
<span class="n">fsout_Color0</span> <span class="o">=</span> <span class="kt">vec4</span><span class="p">(</span><span class="n">RedChannel</span><span class="p">,</span> <span class="n">GreenChannel</span><span class="p">,</span> <span class="n">BlueChannel</span><span class="p">,</span> <span class="mi">1</span><span class="p">.</span><span class="mi">0</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span></code></pre></figure>
<p><a href="http://shader-playground.timjones.io/26c9a6d6acbb7013abf960955eb4e465"><em>Click here to see the compiled SPIR-V bytecode for this shader.</em></a></p>
<p>If you want to enable the “UseTexture” and “FlipTexture” flags and substitute different color channels in, you can write code like the following:</p>
<figure class="highlight"><pre><code class="language-c#" data-lang="c#"><span class="n">ShaderSetDescription</span> <span class="n">shaderSetDesc</span> <span class="p">=</span> <span class="k">new</span> <span class="nf">ShaderSetDescription</span><span class="p">(</span>
<span class="n">vertexLayoutDescriptions</span><span class="p">,</span>
<span class="k">new</span> <span class="n">Shader</span><span class="p">[]</span> <span class="p">{</span> <span class="n">vertexShader</span><span class="p">,</span> <span class="n">fragmentShader</span> <span class="p">},</span>
<span class="k">new</span> <span class="n">SpecializationConstant</span><span class="p">[]</span>
<span class="p">{</span>
<span class="k">new</span> <span class="nf">SpecializationConstant</span><span class="p">(</span><span class="m">0</span><span class="p">,</span> <span class="k">true</span><span class="p">),</span> <span class="c1">// UseTexture = true</span>
<span class="k">new</span> <span class="nf">SpecializationConstant</span><span class="p">(</span><span class="m">1</span><span class="p">,</span> <span class="k">true</span><span class="p">),</span> <span class="c1">// FlipTexture = true</span>
<span class="k">new</span> <span class="nf">SpecializationConstant</span><span class="p">(</span><span class="m">2</span><span class="p">,</span> <span class="m">0.95f</span><span class="p">),</span> <span class="c1">// RedChannel = 0.95f</span>
<span class="k">new</span> <span class="nf">SpecializationConstant</span><span class="p">(</span><span class="m">3</span><span class="p">,</span> <span class="m">0.0f</span><span class="p">),</span> <span class="c1">// GreenChannel = 0f</span>
<span class="k">new</span> <span class="nf">SpecializationConstant</span><span class="p">(</span><span class="m">4</span><span class="p">,</span> <span class="m">0.5f</span><span class="p">),</span> <span class="c1">// BlueChannel = 0.5f</span>
<span class="p">});</span></code></pre></figure>
<p>If this ShaderSetDescription is used to create a Vulkan or Metal Pipeline, then the SpecializationConstant values listed in the array will replace the pre-defined constants in the shader. It is therefore trivial to create another Pipeline which substitutes different constant values by passing in a different array. SPIR-V Specialization Constants always contain default values, so providing SpecializationConstants is optional. You may override a subset (or none) of the Specialization Constants defined in the shader.</p>
<p>Unfortunately, HLSL and OpenGL-style GLSL do not support any kind of specialization constants. All constant values used in the shader must be baked into the shader itself when it is compiled. However, Veldrid.SPIRV allows you to substitute new values in for each Specialization Constant before the shader is translated from SPIR-V into the target language. In practice, this allows you to use SPIR-V shaders with all of Veldrid’s backends and still take advantage of the flexibility of Specialization Constants. If you want to produce GLSL or HLSL (bytecode) at build-time for your application (e.g. to improve load time), you will need to manage the “specialization matrix” yourself.</p>
<h2 id="extras">Extras</h2>
<p>Veldrid.SPIRV also supports compiling GLSL code into SPIR-V, by wrapping Google’s <a href="https://github.com/google/shaderc">shaderc</a> compiler library. This gives you even more flexibility at runtime with your shaders. It’s possible to defer all shader compilation til runtime and still maintain full portability with Veldrid.</p>
<h2 id="limitations">Limitations</h2>
<p>Veldrid.SPIRV relies on a native shared library (libveldrid-spirv), currently packaged for Windows, Linux, and macOS. This is a fairly small component with few dependencies, and is not difficult to compile for additional platforms.</p>Veldrid is a low-level graphics library written in C# that allows you to create GPU-accelerated applications targeting wide variety of platforms, without dealing with platform-specific graphics APIs. Although Veldrid aims to be as portable as possible, one pain point has always been shader code, which differs between platforms. Writing your shaders multiple times is error-prone, limits your portability, and can quickly become a big hassle. Other portable graphics libraries and game engines take different approaches to tackling this problem. Many libraries support a single “official” shading language (often HLSL or a variant, but occasionally a custom shading language) and translate it into a number of shading languages, depending on the graphics APIs being targeted. My ShaderGen project can be seen as one such custom shading language.Writing a Portable CPU/GPU Ray Tracer in C#2018-05-19T17:07:00+00:002018-05-19T17:07:00+00:00/graphics/2018/05/19/writing-a-portable-cpu-gpu-ray-tracer-in-c<p><img src="/images/BookSceneOutput.png" alt="Book Scene - 64 Samples Per Pixel" class="img-responsive" /></p>
<p>Ray tracing is getting a lot of hype lately. Lots of advanced rendering techniques are emerging that involve scene tracing of some kind, there are several frameworks for high-performance ray tracing, D3D12 is getting <a href="https://blogs.msdn.microsoft.com/directx/2018/03/19/announcing-microsoft-directx-raytracing/">integrated support for ray tracing</a>, and there’s a <a href="http://on-demand.gputechconf.com/gtc/2018/presentation/s8521-advanced-graphics-extensions-for-vulkan.pdf">proposed Vulkan extension</a> for the same. I’ve written (bad) ray tracers in the past, but it’s been a while and I wanted to get back up to speed with how things are done. Lots of people are following along with <a href="http://in1weekend.blogspot.com/2016/01/ray-tracing-in-one-weekend.html">Ray Tracing in One Weekend</a> by Peter Shirley, which is a nice, short book going over the fundamentals of ray tracing. The structure of my ray tracer is adapted roughly from the C++ code in that book.</p>
<p>However, I wanted to try something more interesting than just a simple translation to C#. Although .NET is quite fast these days, it still can’t compare to how fast a GPU will chew through computations for a ray tracer. Enter <a href="https://github.com/mellinoe/shadergen">ShaderGen</a>, a project which lets you author portable shader code in C#. You give it regular C# structures and methods, and it automatically converts them to HLSL, GLSL SPIR-V, and Metal shaders. I wanted to see how far I could get by pushing my C# code through ShaderGen and using the resulting compute shaders with <a href="https://mellinoe.github.io/veldrid-docs/">Veldrid</a>. In theory, I can use the same C# code to run my ray tracer on the CPU <em>and</em> the GPU, across a bunch of graphics API’s – Vulkan, Direct3D, Metal, and OpenGL.</p>
<h2 id="goals">Goals</h2>
<p>Ahead of time, I knew of a few snags that I was going to hit when doing this, but these were my primary goals:</p>
<ul>
<li>Write all of the ray tracing logic in C#, including the compute shaders running on the GPU.</li>
<li><em>As much as possible</em>, use the same C# code on the CPU and GPU.</li>
<li>Use the same code to run on Vulkan, Direct3D, Metal, and OpenGL.</li>
</ul>
<h2 id="gpu-friendly-tracing">GPU-friendly Tracing</h2>
<p>The structure in Peter Shirley’s book is a good starting point, but it needs to be changed a bit to run in a compute shader. The original code’s “Sphere” class contains a pointer to an abstract “Material” object which has a virtual function controlling how instances behave. This pattern won’t work in a compute shader. In my version, a Material is another simple, flat structure, and there’s an array of them sitting next to the array of Spheres – one Material per Sphere. Instead of a virtual function, the Material struct just contains an enumerated value identifying its type, and the tracing logic switches off of that. All of these patterns work perfectly fine in C#, but you might not reach for them if you were writing a regular CPU ray tracer.</p>
<p>Another obvious limitation is that GPU code cannot recurse. A simple ray tracer will recurse for each reflection and refraction ray, up to a max depth, because it’s elegant and clean. In a GPU tracer, you’ll instead need to use an explicit loop, bounded by the max depth, which accumulates the color of reflected and refracted rays.</p>
<p><img src="/images/ToyPathTracerOutput.png" alt="ToyPathTracer Scene - 64 Samples Per Pixel" class="img-responsive" /></p>
<h2 id="results">Results</h2>
<p>All of the code is available in the <a href="https://github.com/mellinoe/veldrid-raytracer">Veldrid Ray Tracer repo</a> on GitHub. Check out the README file there for instructions on how to run the program.</p>
<p>Ultimately, I was able to share most of the code between the two versions. There were some limitations (see below), some of which I was aware of already, on how much code could be shared. Obviously, there is some baseline “entry point” code that differs between a compute shader and a C# application. A compute shader runs inside a “thread” set up by the graphics API and drivers. Thread scheduling and dispatch is handled automatically – all that’s needed is the code that grabs the predefined dispatch ID, converts it to a screen coordinate, and traces a ray from it. In the C# app, though, job scheduling is handled manually. For simplicity, I’ve just used the built-in Parallel.For method to loop over each row of pixels in the output texture. Each job then loops over its row of pixels and traces rays through the scene in the same way as the compute shader. There’s not much of a difference here, and this is only a small part of the ray tracer.</p>
<p>I set up two test scenes: one from the final chapter of Peter Shirley’s book, and the other from Aras Pranckevičius’s <a href="https://github.com/aras-p/ToyPathTracer">ToyPathTracer project</a> (image above). I’ve run both scenes on a couple of my machines, on the CPU and on several graphics API’s. I wasn’t really trying to squeeze the best performance out of this project, but I think it’s interesting to see how the code runs in these different contexts. It’s a “brute-force” ray tracer, and there’s some obvious optimizations that aren’t in place that would make it a lot faster. Also, ShaderGen doesn’t yet support some code patterns that could have made the CPU version quite a bit faster, like readonly structs and “in” parameters.</p>
<h3 id="book-scene-100-spheres-1280x720-4-samples-per-pixel">Book scene: 100 spheres, 1280x720, 4 samples per pixel</h3>
<p>Windows 10, Nvidia GTX 770, Core i7-4770K</p>
<ul>
<li>Vulkan: <strong>96</strong> million rays / sec</li>
<li>Direct3D 11: <strong>98</strong> million rays / sec</li>
<li>OpenGL: <strong>85</strong> million rays / sec</li>
<li>CPU: <strong>2</strong> million rays / sec</li>
</ul>
<p>macOS 10.13, Intel Iris Plus 640, Intel Core i5 2.3 GHz</p>
<ul>
<li>Metal: <strong>58</strong> million rays / sec</li>
<li>CPU: <strong>1.15</strong> million rays / sec</li>
</ul>
<h3 id="toypathtracer-scene-46-spheres-1280x720-4-samples-per-pixel">ToyPathTracer scene: 46 spheres, 1280x720, 4 samples per pixel</h3>
<p>Windows 10, Nvidia GTX 770, Core i7-4770K</p>
<ul>
<li>Vulkan: <strong>196</strong> million rays / sec</li>
<li>Direct3D 11: <strong>182</strong> million rays / sec</li>
<li>OpenGL: <strong>165</strong> million rays / sec</li>
<li>CPU: <strong>3.5</strong> million rays / sec</li>
</ul>
<p>macOS 10.13, Intel Iris Plus 640, Intel Core i5 2.3 GHz</p>
<ul>
<li>Metal: <strong>118</strong> million rays / sec</li>
<li>CPU: <strong>2.4</strong> million rays / sec</li>
</ul>
<p>Each graphics API on Windows seems to be in roughly the same ballpark, with OpenGL underperforming a little bit, as expected. All versions are using 100% of my GPU or CPU. I was actually surprised at how well the Metal version runs, since my MacBook has a pretty weak GPU.</p>
<h2 id="limitations">Limitations</h2>
<p>One of my goals was “use as much of the same code as possible on the CPU and GPU”. In any ray tracer, you need to loop over multiple collections of structures: shapes, materials, lights, etc. In my code, these are stored in Veldrid “StructuredBuffer” objects. These correspond to StructuredBuffers in HLSL, storage buffer blocks in GLSL, and buffer-backed “constant pointers” in Metal. All of these shader concepts are roughly equivalent and work well for fine-grained methods operating on individual elements. However, there’s no way to pass one of these buffer blocks around as a method parameter in GLSL, unless it is “fixed-size”. This doesn’t work so well for my ray tracer, unless I lock the size in at compile time. I’d really like to be able to resize these buffers and pass them around freely to different chunks of tracing logic in individual functions, but GLSL doesn’t support it.</p>
<p>What this means is that I wasn’t able to share <em>all</em> of the tracing logic. Any methods that access the scene data need to be duplicated between my CPU and GPU code, at least until I figure out a clever solution to the above. Luckily, in my current version, the only method that this affects is the top-level “Color” method. That’s the one that actually loops over all of the spheres and determines which one intersects with the current ray. Once the sphere and its “Material” are identified, the remaining logic doesn’t need to worry about any of the other objects in the scene. I have two versions of this method which are roughly identical. However, things would get ugly if I wanted to add a collection of lights or light-emitting spheres, for example. Even after I know which object is hit, I would then still need to loop over the collection of lights (or emissive objects) to determine how each affects the sphere. I’d have to pull out even more code into GPU- and CPU-specific chunks.</p>
<p>The “OpenGL ES flavor” of GLSL has another quirky limitation – you can’t read and write to a Texture unless it has a single-channel pixel format. Perhaps this limitation could be worked around, but for the time being my shaders don’t work on OpenGL ES.</p>
<h2 id="practicality">Practicality</h2>
<p>So how useful would this be in an actual game? Well, I’m not sure, but the possibilities are interesting. For most problems utilizing a compute shader, a CPU fallback won’t be fast enough. Algorithms designed for a GPU are also likely to be different from the optimal algorithm for a CPU, at least for more complex problems. For lighter-weight applications, like a very simple particle system, maybe it would work well enough on both sides. For systems that don’t support compute shaders (OpenGL on macOS, for example), you could fall back to updating your particle buffers on the CPU, with the same code powering both versions. If your particles aren’t doing anything particularly taxing, then the performance could be acceptable, and you would no longer need to maintain two versions of the same particle system code.</p>
<p>From another angle, it’s a lot easier to debug regular C# than it is to debug anything running on your GPU. Having the option to debug problematic areas from the regular VS debugger could be valuable in itself.</p>
<h2 id="related">Related</h2>
<ul>
<li>
<p>Most of the code in my project is based on <a href="http://in1weekend.blogspot.com/2016/01/ray-tracing-in-one-weekend.html">Ray Tracing in One Weekend</a>, which is a great little intro book on ray tracing.</p>
</li>
<li>
<p>Aras Pranckevičius has a fun series on ray tracing <a href="https://github.com/aras-p/ToyPathTracer">here</a>. In it, he has built a C# tracer which is much more optimized than mine, as well as various versions in other languages.</p>
</li>
</ul>