Speaker 1: Hey there, and welcome back to the deep Dive. Today,
Speaker 1: we're taking a journey into the intricate world of modern
Speaker 1: three DE graphics, that magical realm powering everything from your
Speaker 1: favorite Triple A games to cutting edge augmented reality experiences.
Speaker 1: We've got an incredible set of sources for this deep
Speaker 1: dive excerpts from the Vulcan three D Graphics Rendering Cookbook,
Speaker 1: second Edition. And this is just any book. It's penned
Speaker 1: by real industry titans. We're talking Sergei Kosarewski formally rendering
Speaker 1: lad at Ubisoft red Links now leading Vulcan development at Meta,
Speaker 1: and Alexi Medvedev ar tech lead at Meta and also
Speaker 1: the Kronos Chair for three D Formats.
Speaker 2: Yeah, these are folks who've been in the trenches ship
Speaker 2: massive games, designed real time rendering tech for years, really
Speaker 2: know their stuff exactly.
Speaker 1: So our mission for you today is a shortcut, a
Speaker 1: shortcut to understanding the core pillars of three D graphics rendering.
Speaker 1: We'll unpack how engineers achieve that astonishing performance, how they
Speaker 1: craft that jaw dropping realism, and what it takes to
Speaker 1: build these experiences across all sorts of different platforms. Consider
Speaker 1: this your fast track to being truly well informed about
Speaker 1: the incredible engineering under the hood. So let's jump right in. Okay,
Speaker 1: first up, the book immediately highlights Vulcan. Now for anyone
Speaker 1: following graphics its name is absolutely everywhere. But what makes
Speaker 1: this API so significant?
Speaker 2: Why Vulcan, Well, what's fascinating here is that Vulcan's popularity.
Speaker 2: It's not just hype, you know, it's a real foundational shift.
Speaker 2: As Anton Kaplanion from Intel points out in the forward,
Speaker 2: it really boils down the two main things, incredible performance
Speaker 2: and truly being cross platform. It's designed to give developers
Speaker 2: really granular control over the GPU, much more than older
Speaker 2: APIs like OpenGL or directex eleven.
Speaker 1: And that control is the key.
Speaker 2: It's absolutely essential. It lets developers push the absolute limits
Speaker 2: of graphics of efficiency. Doesn't matter if you're targeting Windows, Linux,
Speaker 2: high end Android phones, even consoles. Now it fundamentally changed
Speaker 2: how developers could talk to the hardware. Really lets them
Speaker 2: squeeze out every last drop of performance.
Speaker 1: Okay, that granular control, that high performance sounds like a
Speaker 1: developer's dream, definitely, but maybe also a bit of a
Speaker 1: nightmare to set up. Initially, where does someone even begin
Speaker 1: to build a dev environment for something that powerful must
Speaker 1: require pretty specific toolkit.
Speaker 2: Right, You're right, it has its requirements, but the book
Speaker 2: actually outlines a very clear path. You'd start with the
Speaker 2: standard toolkit basically Microsoft Visual Studio twenty twenty two for
Speaker 2: C plus plus A. You need to get for version control,
Speaker 2: C MAKE for building projects, Python, maybe for scripting, pretty
Speaker 2: standard dev stuff. Okay, but here's the critical part. For Vulcan.
Speaker 2: You need the full Vulcan SDK, not just the header
Speaker 2: files you might find lying.
Speaker 1: Reund full SDK.
Speaker 2: Why is that because the SDK gives you essential validation layers.
Speaker 2: These are absolute life savers for debugging when you're working
Speaker 2: at such a low level. They catch mistakes. And it
Speaker 2: includes the GLSL shader compiler, which you obviously need. And
Speaker 2: of course, always make sure you have the latest GPU
Speaker 2: drivers installed. That's crucial.
Speaker 1: Got it. So, validation layers, shader compiler, latest drivers. Without
Speaker 1: those specific Vulcan bits, you're kind of flying blind pretty much. Yeah,
Speaker 1: so I've got the foundational tools sorted. But even with
Speaker 1: those building blocks, Vulcan has a reputation for being well complex,
Speaker 1: a steep learning curve. Are there any of those aha tools?
Speaker 1: Or libraries mentioned. That really helps simplify things, especially when
Speaker 1: you're just starting out, make life a bit easier.
Speaker 2: Yeah, that's a really important point. The book highlights two
Speaker 2: standout libraries, GLFW and Taskflow.
Speaker 1: GLFW, what's that handle?
Speaker 2: GLFW is fantastic because it handles all that operating system
Speaker 2: level boilerplate, you know, creating windows, managing the graphics context,
Speaker 2: capturing keyboard and mouse input. It just hides all that complexity,
Speaker 2: lets you focus on the actual Vulcan rendering code instead
Speaker 2: of fighting with the OS.
Speaker 1: Okay, that makes sense, takes away the platform specific headaches
Speaker 1: and taskflow.
Speaker 2: Taskflow is different. It's a C plus plus header only library,
Speaker 2: sumer easy to integrate, and it's a game changer for
Speaker 2: multi threading. Think of it like conducting an orchestra of CPU. Course.
Speaker 2: It helps you write parallel programs, manage really complex task
Speaker 2: dependencies efficiently. Dependencies like like in modern rendering pipelines, you
Speaker 2: often have these frame graphs where one rendering pass depends
Speaker 2: on another finishing or maybe you want to generate your
Speaker 2: Vulcan command buffers across multiple CPU core simultaneously.
Speaker 1: Ah okay, So distributing.
Speaker 2: The work exactly. Task flow helps manage all that parallelism.
Speaker 2: It's really about leveraging every bit of processing power your
Speaker 2: CPU has available, super important for performance.
Speaker 1: Right makes sense. Okay, so we've got our environment, we've
Speaker 1: got tools to help manage complexity. Now the fun part
Speaker 1: making things look real? How do modern graphics achieve that
Speaker 1: stunning level of realism we see in today's games and
Speaker 1: you know advanced dar It feels like it's gone way
Speaker 1: beyond just slapping on a texture.
Speaker 2: Oh. Absolutely. The answer largely is physically based rendering or PBR.
Speaker 1: PBR heard that term a lot. What's the core idea?
Speaker 2: The core principle is energy conservation. It sounds technical, but
Speaker 2: it's a fundamental law from physics. It basically asserts that
Speaker 2: the amount of light hitting any point on a surface
Speaker 2: has to equal the total amount of light that gets reflected,
Speaker 2: transmitted through it, or absorbed by it. No energy is
Speaker 2: magically created or destroyed.
Speaker 1: Okay, energy conservation. Why is that so transformative for graphics?
Speaker 2: Because it forces assets, models, materials to behave realistically and
Speaker 2: crucially consistently under any lighting condition. It prevents those weird,
Speaker 2: unnatural kind of glowing or two dark results you sometimes
Speaker 2: saw in older engines. It grounds the visuals in reality,
Speaker 2: so it moves.
Speaker 1: Away from artists just guessing how light should look.
Speaker 2: Pretty much, it's like shifting from artistic interpretation to applying
Speaker 2: the actual laws of physics. It leads to much more
Speaker 2: believable and predictable results, which also makes artists' lives easier
Speaker 2: in the long run.
Speaker 1: So, sticking with that energy conservation idea, when light actually
Speaker 1: hits a surface, how does PBR model the different ways
Speaker 1: it can interact.
Speaker 2: Well, we mainly observe two key types of light interaction
Speaker 2: in PBR models. First, there's diffuse light. That's when light
Speaker 2: penetrates the surface just a little bit, scatters around inside,
Speaker 2: and then re emerges at different points. Think of a
Speaker 2: matte surface like rough plastic or dry wall.
Speaker 1: Okay, so it scatters. What's the other type?
Speaker 2: The other is specular reflection. That's when light bounces directly
Speaker 2: off the surface, more like a mirror. This is dominant
Speaker 2: on smooth, polished areas, metals plastics.
Speaker 1: That sort of thing, right, shiny versus matt.
Speaker 2: Exactly and how much light does which is governed by
Speaker 2: the material's properties, is it metal or non metal and
Speaker 2: its microfacets.
Speaker 1: Microfacets tiny details on the surface.
Speaker 2: Precisely, these are tiny, often microscopic bumps and grooves on
Speaker 2: the surface. They dictate how light scatters at a very
Speaker 2: fine level, giving materials their unique look, whether it's a
Speaker 2: blurry reflection or a sharp one. We can even simulate
Speaker 2: really nuanced effects like anisotropic reflection. Yeah, like on brushed
Speaker 2: metal or velvet, where the reflection stretches or changes depending
Speaker 2: on the viewing angle and the orientation of the surface details.
Speaker 2: PBR can moodel that too.
Speaker 1: Wow. Okay, that's a lot of physics detail. For a
Speaker 1: developer actually working with this, does it mean they need
Speaker 1: a PhD in optics or is there a more practical
Speaker 1: way to handle it? Uh?
Speaker 2: Huh No PhD required. Thankfully, standardization is absolutely key here.
Speaker 2: The glTF PBR specification is a great example. It approaches
Speaker 2: material definition focusing on realism, yes, but also efficiency and
Speaker 2: consistency across different renderers.
Speaker 1: So it gives artists and developers standard knobs to turn exactly.
Speaker 2: While the physics underneath is complex. The spec provides standardized
Speaker 2: parameters like base color, roughness, metallic value, specular value, things
Speaker 2: artists can understand and control and to make all those
Speaker 2: complex calculations actually run in real time, like sixty times
Speaker 2: a second.
Speaker 1: Yeah, how do they do that?
Speaker 2: They use clever tricks like pre computing things. A big
Speaker 2: one is using BRDF lookup tables.
Speaker 1: Or LUT lookup tables like a cheat sheet.
Speaker 2: Sort of. Think of it as a precalculated table stored
Speaker 2: in a two D texture using some advanced math, often
Speaker 2: involving things like Monte Carlo estimation, run on a compute shader. Beforehand,
Speaker 2: they basically bake the results of the complex physics equations
Speaker 2: into this simple texture. Then in the game, the shader
Speaker 2: just needs to do a quick lookup in the texture
Speaker 2: based on things like surface roughness and viewing angle to
Speaker 2: get the physically correct lighting results.
Speaker 1: Ah clever. So you do the heavy math once offline,
Speaker 1: then use a fast lookup at runtime.
Speaker 2: Precisely, it makes complex physics feasible in real time. The
Speaker 2: book even points to resources like Brian Carris's work on
Speaker 2: Unreal Engine four shading for a deeper dive into the
Speaker 2: math behind it.
Speaker 1: That's really interesting. So this standardized PBR approach like glTF,
Speaker 1: can it be extended for even more specific effects?
Speaker 2: Oh? Absolutely, It's designed to be extensible. For instance, the
Speaker 2: book mentions extensions like KHR materials clearcoat, clear coat like
Speaker 2: on a car exactly. It adds a thin reflective layer
Speaker 2: on top of the base material, perfect for simulating car paint,
Speaker 2: lacquered wood, polished shoes, that kind of thing.
Speaker 1: Cool.
Speaker 2: Any other examples, Yeah, there's KHR material sheen. This one
Speaker 2: simulates how light interacts with tiny fibers on a surface,
Speaker 2: giving that soft, fuzzy look you see on velvet or
Speaker 2: certain fabrics.
Speaker 1: Wow.
Speaker 2: So these extensions show how you can layer effects onto
Speaker 2: the base PBR model to get incredibly nuanced and realistic materials.
Speaker 2: It allows artists huge flexibility.
Speaker 1: Okay, so realistic materials are sorted, but modern game scenes,
Speaker 1: architectural visualizations, they can be absolutely massive, thousands, maybe millions
Speaker 1: of objects. How do engineers manage all that complexity? How
Speaker 1: do they structure the scene data and still keep performance high?
Speaker 1: Are their common mistakes people make?
Speaker 2: That's a huge challenge, and scene management is critical. The
Speaker 2: book specifically calls out how not to do it?
Speaker 1: Oh, what's the big don't.
Speaker 2: The naive traditional object oriented approach class based scene graphs
Speaker 2: where each know they might have a virtual render method
Speaker 2: and you traverse it recursively.
Speaker 1: That sounds intuitive. Maybe why is it bad?
Speaker 2: It sounds intuitive, but it quickly becomes a performance nightmare.
Speaker 2: Lots of virtual function calls cash misses because related data
Speaker 2: isn't stored together in memory. Plus it makes managing things.
Speaker 2: It's like rendering order for transparency really complicated and prone
Speaker 2: to errors. It's a classic trap.
Speaker 1: Okay, so recursive rendering is out. What's the recommended more
Speaker 1: efficient way?
Speaker 2: Then it's all about moving towards a data oriented design.
Speaker 1: Data oriented meaning.
Speaker 2: Instead of objects owning other objects and calling methods everywhere,
Speaker 2: you focus on the data itself. You store similar data together,
Speaker 2: typically in contiguous arrays, so all the positions might be
Speaker 2: in one array, all their rotations in another, all the
Speaker 2: mesh references in another.
Speaker 1: Right. How does that help?
Speaker 2: It drastically improves CPU cash efficiency. The processor loves working
Speaker 2: on data that's packed together tightly in memory. It spends
Speaker 2: less time waiting for data to be fetched, and operations
Speaker 2: like updating transformation trees become much faster.
Speaker 1: Transformation trees that's figuring out where everything is in the
Speaker 1: world exactly.
Speaker 2: Right, calculating each object's final position, rotation, and scale in
Speaker 2: the world based on its local transformation and its parents transformation.
Speaker 2: Doing this across us thousands of objects is much faster
Speaker 2: when the data is laid out linearly.
Speaker 1: Makes sense. Better data layout equals.
Speaker 2: Better performance fundamentally yes. But even with efficient data layout,
Speaker 2: there's another bottleneck. Just telling the GPU draw this mash,
Speaker 2: now draw this mash, now draw this one. Tens of
Speaker 2: thousands of times per frame.
Speaker 1: That's a lot of chatter between the CPU and GPU
Speaker 1: the draw call overhead.
Speaker 2: Precisely, the CPU gets bogged down just issuing all those
Speaker 2: individual draw commands.
Speaker 1: And I'm guessing this is where a technique like indirect
Speaker 1: rendering comes in, Yeah, to cut down on that chatter exactly.
Speaker 2: Indirect rendering is a massive game changer for handling scenes
Speaker 2: with huge object counts. Instead of the CPU micromanaging every
Speaker 2: single draw call, you know, you basically prepare a list
Speaker 2: of draw commands on the GPU itself in a buffer.
Speaker 2: This list says draw mesh A ten instances starting at
Speaker 2: index X, draw MESHB five instances starting at index Y,
Speaker 2: and so on. Then the CPU issues just one command
Speaker 2: to the GPU like vkcmd draw indirect just one command yep,
Speaker 2: and that one come out and tells the GPU, go
Speaker 2: execute all the draw instructions you find in that buffer.
Speaker 1: Wow, So the GPU pulls the work itself exactly.
Speaker 2: It pulls the draw commands and their parameters directly from
Speaker 2: GPU memory. This dramatically cuts down the CPU overhead. It
Speaker 2: enables rendering scenes with like the book shows, thirty two
Speaker 2: thousand individual meshes or even more, especially when combined with instancing.
Speaker 2: It's key for massive scale rendering.
Speaker 1: That's a huge performance win. But even if you can
Speaker 1: draw thirty two thousand things, most of them aren't actually
Speaker 1: visible to the camera at any given moment, right they're
Speaker 1: off screen or behind other objects.
Speaker 2: Absolutely, and drawing things you can't see is just wasted work,
Speaker 2: which leads to the next crucial optimization frustum culling.
Speaker 1: Right culling only drawing what's inside the camera's view exactly.
Speaker 2: The view frustum is essentially the pyramid shape representing what
Speaker 2: the camera can see. Before drawing, you check if an
Speaker 2: object's bounding box or bounding sphere intersects with this frustum.
Speaker 2: If it's completely outside, you simply skip drawing it, don't
Speaker 2: even send it to the GPU.
Speaker 1: Saving potentially millions of triangles from being processed unnecessarily.
Speaker 2: Potentially, Yes, it's a fundamental optimization. This check can be
Speaker 2: done efficiently on the CPU, or for truly massive scenes
Speaker 2: with maybe hundreds of.
Speaker 1: Thousands of objects, you can do it on the GPU.
Speaker 2: You can offload the culoring logic itself to the GPU
Speaker 2: using compute shaders. The CPU sends rough scene data, the
Speaker 2: GPU compute shader figures out what's visible, and then it
Speaker 2: generates that indirect draw buffal we just talked about, containing
Speaker 2: only the visible.
Speaker 1: Objects, chaining the techniques together exactly.
Speaker 2: The book shows these cool visualizations. The frustum in yellow
Speaker 2: visible meshes green called meshes red really makes it clear
Speaker 2: how effective it is.
Speaker 1: Okay, so building these complex pipelines optimizing them it sounds
Speaker 1: incredibly involved. Debugging must be a nightmare sometimes. How do
Speaker 1: graphics developers actually figure out what's going wrong or why
Speaker 1: performance suddenly drop? What tools do they rely on?
Speaker 2: Good question? Essential activity tools are absolutely key. You can't
Speaker 2: develop this stuff effectively without them. INGUI is a fantastic
Speaker 2: example mention. It stands for Immediate Mode Graphical User Interface.
Speaker 2: It's a bloat free, very easy to use C plus
Speaker 2: plus library for creating debuguiys inside your application.
Speaker 1: Inside the app, not a separate tool.
Speaker 2: Right, you can add sliders, buttons, checkboxes, data plocks directly
Speaker 2: onto your rendering window on the fly. Want to tweak
Speaker 2: a PBR parameter, Add a slider, want to select an
Speaker 2: object in the scene, add.
Speaker 1: A drop down without recompiling exactly.
Speaker 2: It's invaluable for interactive debugging and tweaking, a real life saver.
Speaker 1: Okay, that's for tweaking values. What about when performance tanks?
Speaker 1: How do you find the bottleneck?
Speaker 2: That's where a profiler like Tracy comes in. Tracy is
Speaker 2: mentioned as being used in the book's codebase Lightweight VK.
Speaker 2: It's a powerful frame profiler that integrates directly into your
Speaker 2: C plus plus code. You add simple macros to mark
Speaker 2: sections of your code. Start physics, end physics, start rendering,
Speaker 2: and and then Tracy visualizes exactly how much time is
Speaker 2: spent in each section, both on the CPU and importantly
Speaker 2: on the GPU timeline. You can see which functions are
Speaker 2: taking too long, where the bubbles are in your pipeline.
Speaker 2: You can pinpoint the exact slow spots precisely. It makes
Speaker 2: identifying and optimizing performance bottlenecks much much easier. You're not
Speaker 2: guessing anymore, and beyond UI and profiling, just seeing things
Speaker 2: as crucial. The book also talks about implementing an immediate
Speaker 2: mode three D drawing canvas.
Speaker 1: Like drawing dbug lines and shape yah.
Speaker 2: For rendering auxiliary info directly into the three D scene,
Speaker 2: things like object bounding boxes, collision shapes, light positions, maybe
Speaker 2: the camera frustum itself for debugging culling. It helps visualize
Speaker 2: the invisible structure of the scene. Right makes sense, And
Speaker 2: of course you need good camera controls. A solid first
Speaker 2: person camera implementation, often wrapped in a helper class is
Speaker 2: just fundamental for navigating and inspecting these complex three D
Speaker 2: environments during development.
Speaker 1: Okay, let's pull this all together. We've built the scene,
Speaker 1: optimized it, debugged it. What's that final layer of visual magic,
Speaker 1: the polish that really sells the image on screen and
Speaker 1: makes it look immersive.
Speaker 2: That would be a suite of image based techniques often
Speaker 2: called post processing effects, applied after the main scene is rendered.
Speaker 2: High dynamic range or HDR rendering is absolutely essential here.
Speaker 1: HDR my TV talks about that. How does it apply here?
Speaker 2: Well? Traditional rendering clamps colors between zero point zero black
Speaker 2: and one point zero white but the real world has
Speaker 2: a much wider range of brightness, think the sun versus
Speaker 2: a dimly lit room. HDR rendering uses floating point numbers
Speaker 2: for colors, preserving that huge range of brightness. It captures
Speaker 2: details and super bright highlights and deep shadows simultaneously, just
Speaker 2: like a real camera or your eye would perceive.
Speaker 1: So you get brighter brights and darker darks, more detail exactly.
Speaker 2: It adds incredible depth and realism. But then you have
Speaker 2: a problem. Your monitor can't display that full HDR range.
Speaker 1: Right, So how do you get the HDR image onto
Speaker 1: a standard screen.
Speaker 2: That's where tone mapping comes in. It's a process an
Speaker 2: algorithm that intelligently compresses or maps those HDR values back
Speaker 2: down to the limited range your display can actually show.
Speaker 2: There are different tone mapping operators rein hard Uchimura. The
Speaker 2: book mentions chronos PBR neutral, each giving a slightly different look,
Speaker 2: But the goal is to preserve the intent of the
Speaker 2: HDR image as best as possible. Like compressing audio carefully
Speaker 2: good analogy, you want to reduce the range without crushing
Speaker 2: the details or losing the overall feel. And one more
Speaker 2: related effect is light adaptation or eye adaptation.
Speaker 1: Simulating how our eyes adjust.
Speaker 2: Precisely when you move from a bright area to a
Speaker 2: dark one, or vice versa, your eyes take time to adjust.
Speaker 2: This effect simulates that, gradually changing the overall exposure of
Speaker 2: the scene based on its average brightness. It really enhances immersion,
Speaker 2: making the virtual world feel more reactive and alive.
Speaker 1: Wow, what an incredible deep dive that was. We've really
Speaker 1: gone from the nuts and bolts of setting up a
Speaker 1: Vulcan environment all the way to these sophisticated algorithms creating
Speaker 1: hyperrealistic visuals. You should now have a really solid grasp
Speaker 1: on how modern engines bring these worlds to life with PBR,
Speaker 1: how they manage immense complexity using things like data oriented
Speaker 1: design and indirect rendering, and keep it all running smoothly
Speaker 1: with culling. Plus the tools developers use, and those final
Speaker 1: touches like HDR and tone mapping that add that layer
Speaker 1: of polish. You basically just had a shortcut to being
Speaker 1: seriously well informed about the amazing engineering behind modern three
Speaker 1: D graphics, all thanks to the insights from the Vulcan
Speaker 1: three D Graphics Rendering Cookbook.
Speaker 2: Absolutely, and thinking about all this it makes you wonder
Speaker 2: what's next? What are the big challenges ahead in pushing
Speaker 2: real time rendering even further? You know, consider things like
Speaker 2: real time retracing becoming more mainstream, or maybe even more
Speaker 2: dynamic AI driven ways to generate content on the fly.
Speaker 2: How might these incredibly sophisticated techniques evolve in the next
Speaker 2: few years. What new problems will developers have to solve?
Speaker 2: Thing to think about.