WEBVTT 1 00:00:00.040 --> 00:00:04.639 All right, jumping right into this deep dive Windows kernel programming. 2 00:00:04.759 --> 00:00:09.119 Oh yeah, you handed me some excerpts from Pavel Yosefovic's book, 3 00:00:10.080 --> 00:00:13.679 and I gotta say, some of this stuff intense. Yeah, 4 00:00:13.720 --> 00:00:15.439 it's intense. It's a little over my head. Yeh, that's 5 00:00:15.480 --> 00:00:18.559 why you're here, thankfully. But I'm ready to learn, ready 6 00:00:18.600 --> 00:00:20.960 to uncover the secrets of the kernel some most people 7 00:00:21.000 --> 00:00:22.800 never even you know, think about. 8 00:00:23.120 --> 00:00:28.920 Well, Yusefavich definitely doesn't shy away from the complexity of it. 9 00:00:29.079 --> 00:00:35.159 We're talking processes, threads, virtual memory system calls, object managers, handles, 10 00:00:35.200 --> 00:00:38.759 I mean, the whole nine yards, and even getting into 11 00:00:38.799 --> 00:00:41.679 like setting up a kernel development environment, and you know, 12 00:00:41.719 --> 00:00:43.759 the nuances of programming at the kernel level. 13 00:00:43.840 --> 00:00:46.200 Yeah, I skim those sections, and let's just say, I'm 14 00:00:46.200 --> 00:00:49.079 glad you're the one fluent in kernels. But before we 15 00:00:49.119 --> 00:00:51.000 get too lost in the weeds, here something that caught 16 00:00:51.039 --> 00:00:55.479 my eye. File system minifilters. Yeah, Yusefovich talks about them 17 00:00:55.479 --> 00:00:57.840 like there's some kind of secret weapon for developers. 18 00:00:58.039 --> 00:01:01.200 They kind of are. You know, it's amazing what you 19 00:01:01.200 --> 00:01:02.920 can do with a minifilter. I mean, think about it 20 00:01:02.960 --> 00:01:05.920 this way, like a minifilter can actually step in and 21 00:01:06.040 --> 00:01:09.560 intercept filesystem operations, yeah, before they even get to the 22 00:01:09.560 --> 00:01:10.719 file system itself. 23 00:01:10.799 --> 00:01:13.040 So like, if I'm an anti virus program, I can 24 00:01:13.120 --> 00:01:15.480 use a minifilter to scan a file before it even 25 00:01:15.480 --> 00:01:16.480 gets written to the disc. 26 00:01:16.640 --> 00:01:20.000 That's one example. Yeah, yeah, but it goes way beyond that, right, Like, 27 00:01:20.159 --> 00:01:22.359 you can use it for everything from real time backups 28 00:01:22.680 --> 00:01:26.920 to like really granular access control. Yasufovich even gives an 29 00:01:26.959 --> 00:01:30.079 example of a minifilter that encrypts files on the fly, 30 00:01:30.719 --> 00:01:32.439 completely transparent to the user. 31 00:01:32.799 --> 00:01:34.840 Wow. Yeah, that's pretty mind blowing. 32 00:01:34.920 --> 00:01:35.280 Yeah. 33 00:01:35.280 --> 00:01:37.439 Okay, so before we get too far ahead of ourselves, 34 00:01:37.519 --> 00:01:40.439 let's rewind a bit. Yu Seefavitch spends a good chunk 35 00:01:40.480 --> 00:01:44.079 of the book talking about processes and threads, you know, 36 00:01:44.159 --> 00:01:46.480 the building blocks of everything that happens in the kernel. 37 00:01:46.599 --> 00:01:48.280 It's been a while since I've had to think about 38 00:01:48.280 --> 00:01:51.079 that stuff. Remind me, why are they so important? 39 00:01:51.400 --> 00:01:54.519 Well, think of a process like a container, right, Like 40 00:01:54.719 --> 00:01:57.599 it's a little world into itself. It has its own 41 00:01:57.640 --> 00:02:00.760 memory space, it has its own security set, it has 42 00:02:00.799 --> 00:02:03.599 its own set of resources. And then the threads are 43 00:02:03.640 --> 00:02:06.519 like the workers within that container that actually carry out 44 00:02:06.519 --> 00:02:07.200 the instructions. 45 00:02:07.280 --> 00:02:11.680 Okay, that makes sense. But Yusifavich mentioned something kind of interesting. 46 00:02:11.759 --> 00:02:15.960 He says that a process without any threads is basically useless. 47 00:02:16.960 --> 00:02:18.759 Why is that, Well. 48 00:02:18.560 --> 00:02:20.960 It's like having you know, all the equipment in a factory, 49 00:02:21.000 --> 00:02:24.080 but no workers to run the equipment. Right, Like, without threads, 50 00:02:24.120 --> 00:02:27.280 a process can actually do anything. It's just taking up space. 51 00:02:27.800 --> 00:02:31.159 So the kernel, being you know, the efficient manager it is, 52 00:02:31.439 --> 00:02:32.520 it'll eventually destroy it. 53 00:02:32.919 --> 00:02:35.159 Ah. So it's all about making sure resources are being 54 00:02:35.240 --> 00:02:35.960 used effectively. 55 00:02:36.120 --> 00:02:36.840 Yeah, okay. 56 00:02:37.280 --> 00:02:38.960 Now another thing that always kind of trips me up 57 00:02:39.039 --> 00:02:41.919 is virtual memory. You know, it's like this magical realm 58 00:02:41.960 --> 00:02:44.199 where every process thinks it has all the memory in 59 00:02:44.240 --> 00:02:46.919 the world, even if it's sharing a physical machine with 60 00:02:46.919 --> 00:02:49.240 a bunch of other processes. How does that even work? 61 00:02:49.439 --> 00:02:53.280 Well, Yussefhavich describes it as, you know, like each process 62 00:02:53.319 --> 00:02:56.680 gets its own map to a vast territory, but only 63 00:02:56.719 --> 00:02:59.639 the areas that they actually use get real Okay, Right. 64 00:02:59.840 --> 00:03:03.479 The kernel is like the master cartographers, right, it's managing 65 00:03:03.479 --> 00:03:05.560 all these maps making sure that they don't overlap. 66 00:03:05.719 --> 00:03:08.240 Okay, that's a helpful way to think about it. So 67 00:03:08.319 --> 00:03:12.199 the kernel is constantly juggling all these virtual maps, making 68 00:03:12.199 --> 00:03:15.120 sure that each process has the illusion of its own 69 00:03:15.240 --> 00:03:19.759 private memory space. But what happens when a process needs 70 00:03:19.800 --> 00:03:23.560 to do something that requires like kernel privileges, you know, 71 00:03:23.639 --> 00:03:28.280 like access hardware, modify systems settings. That's where system calls. 72 00:03:27.960 --> 00:03:30.479 Come in, right, Yeah, exactly. So it's like needing to 73 00:03:30.520 --> 00:03:33.199 go through customs, you know, to cross a border. Like 74 00:03:33.280 --> 00:03:36.080 a process can't just waltz into kernel mode whenever it wants. 75 00:03:36.439 --> 00:03:39.439 It needs to present the right credentials, follow the protocol. 76 00:03:39.360 --> 00:03:41.280 And you see, Fitch gets into a really in depth 77 00:03:41.319 --> 00:03:43.280 detail about how this works at like the low level. 78 00:03:43.360 --> 00:03:46.199 He talks about system call numbers and the system service 79 00:03:46.240 --> 00:03:49.400 dispatch table and all sorts of other kind of intricate mechanisms. 80 00:03:50.000 --> 00:03:53.240 Is there anything in particular about system calls that stood 81 00:03:53.240 --> 00:03:55.800 out to you, you know, anything that surprised you or 82 00:03:55.800 --> 00:03:57.479 made you think differently about how they work? 83 00:03:57.840 --> 00:04:00.960 You Know what really struck me is just the level 84 00:04:01.000 --> 00:04:03.759 of indirection involved. Yeah, you know what I mean. Like, 85 00:04:03.759 --> 00:04:07.439 it's not just a simple matter of a user mode 86 00:04:07.439 --> 00:04:11.120 function calling a kernel mode function directly, right. There are 87 00:04:11.319 --> 00:04:14.840 layers upon layers of abstraction, like checks and balances to 88 00:04:15.039 --> 00:04:16.920 ensure security and stability. 89 00:04:17.000 --> 00:04:20.199 So even something as seemingly straightforward as a system call 90 00:04:20.319 --> 00:04:23.839 is actually this really intricate dance between user mode and 91 00:04:23.959 --> 00:04:27.720 kernel mode. It's fascinating but also a little daunting, to 92 00:04:27.759 --> 00:04:28.240 be honest. 93 00:04:28.439 --> 00:04:30.519 Well, yeah, I mean that's the kernel for you, right, 94 00:04:30.560 --> 00:04:33.639 It's a world of complexity and power, and you know 95 00:04:33.720 --> 00:04:38.639 where even small mistakes can have huge consequences. But you know, 96 00:04:38.720 --> 00:04:40.680 that's also what makes it so intriguing. There's always more 97 00:04:40.720 --> 00:04:42.199 to learn, there's always more to discover. 98 00:04:42.439 --> 00:04:44.600 I'm starting to feel like an explorer setting off into 99 00:04:44.680 --> 00:04:49.439 uncharted territory. Speaking of territory, Yosefovich mentions this thing called 100 00:04:49.480 --> 00:04:53.040 the object manager, which he describes as like the kernel's 101 00:04:53.199 --> 00:04:55.720 internal librarian. What's that all about? 102 00:04:56.160 --> 00:04:59.160 Yeah, so the object manager is essentially the kernel's way 103 00:04:59.199 --> 00:05:03.399 of keeping track of air everything, files, processes, threads, devices, 104 00:05:03.439 --> 00:05:07.079 you name it. It's like this vast library, with every 105 00:05:07.120 --> 00:05:09.680 resource neatly cataloged and organized. 106 00:05:09.920 --> 00:05:12.560 So even the kernel needs a good filing system, that's right, 107 00:05:13.079 --> 00:05:15.639 But what's the point of having this object manager? Why 108 00:05:15.680 --> 00:05:18.480 not just let each component manage its own resources. 109 00:05:18.639 --> 00:05:21.759 Well, centralizing resource management this way through the object manager 110 00:05:22.199 --> 00:05:25.920 has a lot of benefits, right. It enforces consistency, it 111 00:05:25.959 --> 00:05:28.839 prevents conflicts, it makes it much easier to track an 112 00:05:28.839 --> 00:05:32.480 audit resource usage, and plus it provides kind of this 113 00:05:32.639 --> 00:05:37.680 uniform interface for accessing and manipulating resources regardless of their type. 114 00:05:37.879 --> 00:05:40.040 That makes sense. It's like having a single point of 115 00:05:40.079 --> 00:05:42.399 contact for all your resource needs rather than having to 116 00:05:42.439 --> 00:05:44.759 deal with a bunch of different departments. Right, But how 117 00:05:44.759 --> 00:05:48.279 do user mode processes actually interact with these objects that 118 00:05:48.319 --> 00:05:49.519 are managed by the kernel. 119 00:05:49.600 --> 00:05:53.120 Ah, that's where handles come in. Okay, so Yosipovic describes 120 00:05:53.160 --> 00:05:57.160 these handles as like tickets that user mode programs use 121 00:05:57.240 --> 00:06:00.879 to access those kernel objects. It's like getting a claim 122 00:06:01.000 --> 00:06:03.480 ticket at a coat check, right, Yeah, you present your 123 00:06:03.519 --> 00:06:05.199 ticket to retrieve your coat. 124 00:06:05.480 --> 00:06:08.399 So a handle is basically a reference to an object 125 00:06:08.480 --> 00:06:11.759 that's being managed by the kernel. Yes, but Yusufhovich also 126 00:06:11.839 --> 00:06:14.759 mentions that each process has its own private handle table. 127 00:06:14.839 --> 00:06:17.879 Why is that, Well, it's all about security and isolation, right. 128 00:06:18.199 --> 00:06:21.240 By giving each process its own handle table, the kernel 129 00:06:21.279 --> 00:06:25.800 can control exactly which processes have access to which objects, 130 00:06:26.680 --> 00:06:28.920 and it can also prevent one process from messing with 131 00:06:28.959 --> 00:06:32.160 another processes objects, even if they happen to have the 132 00:06:32.160 --> 00:06:33.160 same handle value. 133 00:06:33.240 --> 00:06:35.120 Right, So it's like each process gets its own set 134 00:06:35.120 --> 00:06:38.199 of keys to access the kernel's resources, and those keys 135 00:06:38.199 --> 00:06:41.720 don't work for any other process exactly. But Yusufhovich also 136 00:06:41.759 --> 00:06:43.959 points out something kind of interesting. He says that the 137 00:06:44.040 --> 00:06:46.600 name that's displayed for a handle can have different meanings 138 00:06:46.639 --> 00:06:49.040 depending on the object type. What's that all about. 139 00:06:49.519 --> 00:06:51.360 It can be a little confusing at first, but it 140 00:06:51.360 --> 00:06:53.360 actually makes a lot of sense when you think about it. 141 00:06:54.639 --> 00:06:57.639 You know, for some object types like mutex is, the 142 00:06:57.680 --> 00:07:00.800 handle name is simply the name of the objects. Yeah, 143 00:07:00.839 --> 00:07:04.000 But for other object types like files and directories, the 144 00:07:04.040 --> 00:07:07.560 handle name is actually the path to that object in 145 00:07:07.600 --> 00:07:08.319 the file system. 146 00:07:08.639 --> 00:07:11.439 Okay, so the handle name isn't always a literal name. 147 00:07:11.480 --> 00:07:15.399 It can also be like a path or some other 148 00:07:15.480 --> 00:07:19.199 kind of identifier that's meaningful for that specific object type. 149 00:07:19.240 --> 00:07:19.639 That's right. 150 00:07:19.879 --> 00:07:23.120 But all this talk about handles and objects in kernel 151 00:07:23.120 --> 00:07:25.680 mode is making me really curious about what it actually 152 00:07:25.759 --> 00:07:28.519 takes to write code that runs in this kind of 153 00:07:28.600 --> 00:07:32.279 privileged realm. Yusifovich has a whole section on setting up 154 00:07:32.319 --> 00:07:34.879 a kernel development environment, and let's just say it looks 155 00:07:34.879 --> 00:07:37.920 a lot different from my usual user mode coding setup. 156 00:07:37.959 --> 00:07:40.439 Oh yeah, it's a whole different world down there, it is. 157 00:07:40.439 --> 00:07:43.079 You need special tools like the Windows Driver kit. You 158 00:07:43.120 --> 00:07:45.120 need a debugger they can understand what's going on in 159 00:07:45.160 --> 00:07:48.079 the kernel, and you need a healthy dase of patients. 160 00:07:48.399 --> 00:07:52.879 Yusifovich mentions the importance of error handling in Kernal development. 161 00:07:53.240 --> 00:07:57.000 He says, even small mistakes can bring down the entire system. 162 00:07:58.000 --> 00:08:01.519 Why is that what makes kernel development so much more 163 00:08:01.600 --> 00:08:04.079 unforgiving than user mode programming. Yeah. 164 00:08:04.079 --> 00:08:06.319 Well, in user mode, if your code crashes, it usually 165 00:08:06.360 --> 00:08:08.480 only affects the current process, right, the rest of the 166 00:08:08.480 --> 00:08:11.439 system kind of keeps humming along. But in kernel mode, 167 00:08:11.879 --> 00:08:14.720 your code's running in the same address space as the 168 00:08:14.759 --> 00:08:18.079 operating system itself, So a bug in your driver can 169 00:08:18.120 --> 00:08:22.360 actually corrupt critical data structures, you know, leading to system 170 00:08:22.399 --> 00:08:24.639 instability or even a complete crash. 171 00:08:24.720 --> 00:08:27.040 So it's like playing with live wires. 172 00:08:27.399 --> 00:08:28.199 Yeah, pretty much. 173 00:08:28.279 --> 00:08:30.439 One wrong move and the whole house goes dark. 174 00:08:30.519 --> 00:08:34.480 Yeah, exactly. And to make things even more challenging, kernel 175 00:08:34.480 --> 00:08:39.519 development often involves like working with undocumented features reverse engineering 176 00:08:39.600 --> 00:08:40.360 existing drivers. 177 00:08:40.480 --> 00:08:41.000 Oh wow. 178 00:08:41.240 --> 00:08:42.720 Yeah, so it's not for the faint of heart. 179 00:08:42.799 --> 00:08:45.120 Yeah, I'm sarrying to understand why kernel developers are often 180 00:08:45.120 --> 00:08:48.759 seen as like wizards or gurus. But let's dive into 181 00:08:48.799 --> 00:08:52.559 some of the specifics of kernel programming. Here. Yasifavich talks 182 00:08:52.559 --> 00:08:55.960 about different memory pools that drivers can use, paged and 183 00:08:56.039 --> 00:08:58.919 non paged. What's the difference between these pools and why 184 00:08:58.919 --> 00:08:59.519 does it matter? 185 00:09:00.000 --> 00:09:03.159 Okay, so think about it this way. Page pool memory 186 00:09:03.279 --> 00:09:06.759 is kind of like this, you know, flexible workspace that 187 00:09:06.840 --> 00:09:09.799 can be moved around as needed. Okay, right, So it 188 00:09:09.840 --> 00:09:12.120 could be swapped out to disk if the system is 189 00:09:12.200 --> 00:09:14.879 running low on physical memory and brought back in when 190 00:09:14.879 --> 00:09:18.679 it's needed again. But non paged pool memory, on the 191 00:09:18.679 --> 00:09:22.639 other hand, that's like your you know, dedicated office space. 192 00:09:22.679 --> 00:09:23.759 It's always available. 193 00:09:23.919 --> 00:09:27.120 So nontage memory is like the VIP section. Yes, always 194 00:09:27.120 --> 00:09:29.960 guarantee it a spot in RAM exactly. But wouldn't having 195 00:09:30.080 --> 00:09:32.919 everything be non paged be a recipe for disaster? 196 00:09:33.360 --> 00:09:35.480 You're right, You're absolutely right, and that's why drivers have 197 00:09:35.519 --> 00:09:37.600 to be really, really careful about how they use non 198 00:09:37.600 --> 00:09:40.600 paged memory. It's a precious resource that should only be 199 00:09:40.679 --> 00:09:43.440 used for you know, code or data that absolutely cannot 200 00:09:43.480 --> 00:09:46.600 be paged out, things like interrupt handlers or data structures 201 00:09:46.600 --> 00:09:48.519 that need to be accessed very quickly and reliably. 202 00:09:48.759 --> 00:09:54.360 It's all about striking that balance between performance and resource utilization. Yeah, okay, 203 00:09:54.480 --> 00:09:58.399 so how does a driver actually tell the operating system 204 00:09:58.480 --> 00:10:01.559 what it can do? Is there some kind of registration process? 205 00:10:01.840 --> 00:10:05.159 There is? Yeah, So each driver has this special data 206 00:10:05.159 --> 00:10:08.120 structure called a driver object. It's kind of like the 207 00:10:08.200 --> 00:10:11.440 driver's resume right outlining its capabilities, and now it interfaces 208 00:10:11.440 --> 00:10:12.320 with the rest of the system. 209 00:10:12.399 --> 00:10:15.519 Interesting, So what kind of information is stored in this 210 00:10:15.679 --> 00:10:18.919 driver object? What does the kernel need to know about 211 00:10:18.919 --> 00:10:21.360 a driver in order for it to function properly. 212 00:10:21.519 --> 00:10:26.639 Well, it's packed with important details. The driver object includes 213 00:10:26.679 --> 00:10:29.759 things like the driver's name, the device object that it's 214 00:10:29.759 --> 00:10:33.399 associated with, and probably most importantly, a list of function 215 00:10:33.480 --> 00:10:37.039 pointers called the major function array. Okay, and this array 216 00:10:37.240 --> 00:10:41.360 tells the operating system what IO requests the driver can handle. 217 00:10:41.480 --> 00:10:43.480 So it's kind of like a menu of services that 218 00:10:43.519 --> 00:10:46.000 the driver offers, like I can create files, I can 219 00:10:46.039 --> 00:10:47.440 read files, I can delete files. 220 00:10:47.720 --> 00:10:50.720 That's exactly right. Yeah. Each entry in that major function 221 00:10:50.840 --> 00:10:55.799 array points to a specific function within the driver that's 222 00:10:55.879 --> 00:11:00.159 responsible for handling a particular type of IO request. So, 223 00:11:00.200 --> 00:11:02.879 for example, if the operating system needs to create a 224 00:11:02.879 --> 00:11:06.639 new file, It'll check the driver's major function array for 225 00:11:06.720 --> 00:11:09.879 an entry that corresponds to file creation. 226 00:11:09.679 --> 00:11:12.559 Requests, and if the driver has registered a handler for 227 00:11:12.600 --> 00:11:15.919 that request type, then the operating system will call that function, 228 00:11:16.080 --> 00:11:18.799 passing along all the necessary information about the requests. 229 00:11:18.840 --> 00:11:21.480 You got it. And that's where things get really interesting, right, Okay, 230 00:11:21.480 --> 00:11:24.200 because those io requests are packaged up in these things 231 00:11:24.240 --> 00:11:30.240 called IRPs iorequest packets, and Yosefovich really gets into the 232 00:11:30.320 --> 00:11:32.600 nitty gritty of how these IRPs work. 233 00:11:33.080 --> 00:11:36.519 Yeah, I remember him describing IRPs as like the language 234 00:11:36.559 --> 00:11:38.720 of the kernel, the way that all the different components 235 00:11:38.759 --> 00:11:41.600 communicate with each other. Right, But if I'm being honest, 236 00:11:41.639 --> 00:11:44.320 the whole concept of IRP still feels a bit abstract 237 00:11:44.320 --> 00:11:46.879 to me. Can you break it down for me? What 238 00:11:46.919 --> 00:11:50.639 do these IRPs actually look like and how do they 239 00:11:50.720 --> 00:11:51.720 flow through the system? 240 00:11:52.080 --> 00:11:56.720 Okay? So imagine an IRP as an envelope, okay, right, 241 00:11:56.840 --> 00:12:00.360 and inside this envelope is all sorts of information about 242 00:12:00.399 --> 00:12:03.720 the request and what type of operation it is, what 243 00:12:03.799 --> 00:12:06.720 file or device it's targeting, any data that needs to 244 00:12:06.720 --> 00:12:07.799 be transferred, and so on. 245 00:12:07.919 --> 00:12:10.159 So it's like a standardized form that everyone agrees on 246 00:12:10.320 --> 00:12:13.399 just making sure that all the necessary information is included. 247 00:12:12.960 --> 00:12:15.600 Precisely, and these IRPs, they get passed around from one 248 00:12:15.600 --> 00:12:19.559 component to another as the request is processed. So, for example, 249 00:12:19.799 --> 00:12:22.720 if a user mode application wants to write to a file, 250 00:12:23.559 --> 00:12:26.039 it'll make a system call, which will eventually result in 251 00:12:26.240 --> 00:12:29.360 an IRP being created in the kernel, and that IRP 252 00:12:29.559 --> 00:12:32.679 might be passed through several layers of drivers, each one 253 00:12:32.720 --> 00:12:37.120 potentially examining or modifying the request before it finally reaches 254 00:12:37.159 --> 00:12:40.320 the filesystem driver that's responsible for actually writing the data 255 00:12:40.360 --> 00:12:40.679 to disk. 256 00:12:40.799 --> 00:12:42.639 So it's kind of like a relay race, with the 257 00:12:42.720 --> 00:12:45.200 IRP being passed from one runner to the next until 258 00:12:45.200 --> 00:12:47.480 it reaches the finish nine exactly. But with all these 259 00:12:47.480 --> 00:12:51.559 different components potentially handling the IRP, how does the system 260 00:12:51.759 --> 00:12:54.279 make sure that everything happens in the right order and 261 00:12:54.320 --> 00:12:57.039 that nothing gets lost or corrupted along the way. 262 00:12:57.240 --> 00:12:59.320 Well that's where the io manager comes in, okay, right, 263 00:12:59.320 --> 00:13:01.639 It's like the racecord making sure all the runners are 264 00:13:01.639 --> 00:13:04.240 in their lanes and the baton gets passed smoothly. So 265 00:13:04.279 --> 00:13:06.559 the iron manager keeps track of all the IRPs in 266 00:13:06.600 --> 00:13:11.200 the system, it routes them to the appropriate drivers. Yeah, 267 00:13:11.240 --> 00:13:14.519 and it manages things like buffering and completion routines. 268 00:13:14.960 --> 00:13:17.960 It's a complex system, but it's pretty amazing how it 269 00:13:18.000 --> 00:13:21.440 all works together to handle, you know, potentially millions of 270 00:13:21.480 --> 00:13:23.200 these io requests every second. 271 00:13:23.399 --> 00:13:24.159 Yeah. 272 00:13:24.200 --> 00:13:26.279 But let's go back to those mini filters we talked 273 00:13:26.279 --> 00:13:29.440 about earlier. Where do they fit into this whole IRP 274 00:13:29.679 --> 00:13:30.960 processing pipeline. 275 00:13:31.080 --> 00:13:34.360 Ah? Yes, So mini filters are like secret agents that 276 00:13:34.440 --> 00:13:38.840 can intercept those IRPs at various points along their journey. 277 00:13:38.600 --> 00:13:41.039 So they can kind of peek inside the envelope. Yeah, 278 00:13:41.080 --> 00:13:44.159 maybe even slip in a little something extra before passing 279 00:13:44.200 --> 00:13:45.399 it along to the next component. 280 00:13:45.679 --> 00:13:47.600 That's a great way to think about it. And that 281 00:13:47.720 --> 00:13:51.960 ability to intercept and manipulate those IRPs is really what 282 00:13:52.039 --> 00:13:54.720 makes minifilters so incredible powerful. They can, you know, they 283 00:13:54.720 --> 00:13:57.120 can filter, they can modify, redirect, they can even block 284 00:13:57.159 --> 00:14:01.080 IOA requests, all without the user even knowing it's happening. 285 00:14:01.399 --> 00:14:06.519 Right. We talked earlier about antivirus programs using minifilters to 286 00:14:06.600 --> 00:14:10.240 scan files before they're written to the disc, but Yosefavic 287 00:14:10.440 --> 00:14:13.279 mentions a whole range of other applications like back up 288 00:14:13.320 --> 00:14:17.759 and restore data encryption, even performance optimization. It seems like 289 00:14:17.840 --> 00:14:19.519 there's a lot you can do with these things. 290 00:14:19.759 --> 00:14:22.159 The possibilities are really endless, you know. For example, you 291 00:14:22.240 --> 00:14:26.159 could use a mini filter to create a shadow copy 292 00:14:26.200 --> 00:14:28.720 of every file that's written to a specific directory, right, Yeah, 293 00:14:28.840 --> 00:14:30.480 essentially creating a real time backup. 294 00:14:30.600 --> 00:14:35.200 That's impressive. And I remember Yusufovich mentioning something about minifilters 295 00:14:35.240 --> 00:14:39.120 being able to enforce like really granular access control. Yes, 296 00:14:39.240 --> 00:14:40.080 how does that work? 297 00:14:40.279 --> 00:14:43.399 So imagine you want to restrict access to certain files 298 00:14:43.440 --> 00:14:46.480 based on the user's identity, you know, or group membership. 299 00:14:46.879 --> 00:14:50.519 A mini filter can actually examine the IERP for every 300 00:14:50.679 --> 00:14:54.600 file access request, check the user's credentials, and if the 301 00:14:54.679 --> 00:14:58.440 user doesn't have the necessary permissions, the minifilter can simply 302 00:14:58.519 --> 00:14:59.919 block the requests. 303 00:15:00.000 --> 00:15:02.360 So like having a bouncer at the door of every 304 00:15:02.399 --> 00:15:06.080 file checking IDs and making sure that only authorized personnel 305 00:15:06.080 --> 00:15:06.480 get through. 306 00:15:06.559 --> 00:15:07.600 That's a great way to put it. 307 00:15:07.799 --> 00:15:10.480 But with so much power at their disposal, isn't there 308 00:15:10.519 --> 00:15:13.720 a risk that many filters could interfere with each other, right, 309 00:15:13.960 --> 00:15:16.320 or even with the normal operation of the system. 310 00:15:16.480 --> 00:15:19.600 Yeah, that's a valid concern, but the kernel's designed to 311 00:15:19.679 --> 00:15:23.720 manage these interactions very carefully. So MANI filters are assigned 312 00:15:23.720 --> 00:15:27.919 in altitude, which determines their order in the filtering chain altitude. 313 00:15:27.960 --> 00:15:30.200 So it's like stacking them on top of each other, Yes, 314 00:15:30.320 --> 00:15:33.279 with the higher altitude filters getting to see the IERP first. 315 00:15:33.399 --> 00:15:37.720 That's exactly right. And this altitude system is carefully managed 316 00:15:38.000 --> 00:15:42.080 to prevent those kinds of conflicts. In fact, Yosiphovich emphasizes it, 317 00:15:42.080 --> 00:15:45.320 if you want to develop a production grade minifilter, you 318 00:15:45.360 --> 00:15:47.759 actually need to a pain of proper altitude allocation for 319 00:15:47.840 --> 00:15:48.960 Microsoft really, so. 320 00:15:48.919 --> 00:15:51.600 It's not just a free for all where any developer 321 00:15:51.600 --> 00:15:53.879 can choose any altitude they want, not at all. 322 00:15:53.919 --> 00:15:59.399 No, Microsoft has these established ranges of altitudes for different purposes. Right, 323 00:16:00.120 --> 00:16:04.639 Virus filters might occupy one altitude range while backup filters 324 00:16:04.679 --> 00:16:05.440 occupy another. 325 00:16:05.519 --> 00:16:07.639 Okay, that makes sense. It's like air traffic control, right, 326 00:16:07.720 --> 00:16:09.759 making sure that all the planes are flying at the 327 00:16:09.799 --> 00:16:13.919 right altitudes to prevent collisions. But this whole altitude allocation 328 00:16:14.039 --> 00:16:17.000 process sounds pretty involved. Why is it so important? 329 00:16:17.399 --> 00:16:20.759 Well, imagine what could happen if two minifilters with like 330 00:16:20.840 --> 00:16:24.279 incompatible goals were operating at the same altitude. 331 00:16:24.720 --> 00:16:27.879 Let me guess chaos chaos exactly. 332 00:16:28.120 --> 00:16:30.720 You know, one minifilter might be trying to block access 333 00:16:30.720 --> 00:16:33.039 to a file while the other one's trying to allow access. 334 00:16:33.759 --> 00:16:36.320 Or one minifilter might be modifying the data in an 335 00:16:36.360 --> 00:16:39.720 IRP while the other is expecting that data to be unchanged. 336 00:16:40.000 --> 00:16:41.840 It sounds like a recipe for disaster. 337 00:16:42.080 --> 00:16:45.440 Yeah, So this altitude allocation process is really all about 338 00:16:45.519 --> 00:16:48.759 ensuring that mini filters can coexist peacefully and that the 339 00:16:48.799 --> 00:16:50.600 system remains stable and predictable. 340 00:16:50.919 --> 00:16:54.000 So this altitude allocation process is all about ensuring that 341 00:16:54.080 --> 00:16:57.519 mani filters can coexist peacefully and that the system remains 342 00:16:57.519 --> 00:17:00.720 stable and predictable. But let's dive a bit deeper into 343 00:17:00.720 --> 00:17:04.359 the actual structure of a minifilter. How are they implemented 344 00:17:04.799 --> 00:17:08.279 and what are the key components that developers need to understand? Okay, 345 00:17:08.720 --> 00:17:12.200 from what I gathered from Yosifovic, a minifilter is essentially 346 00:17:12.279 --> 00:17:15.799 a driver that registers a set of callback functions with 347 00:17:15.920 --> 00:17:17.279 the filter manager. 348 00:17:17.160 --> 00:17:21.240 Right, and those callback functions are the minifilter's eyes and ears, right, Okay, 349 00:17:21.480 --> 00:17:24.920 allowing you to to observe and interact with IRPs as 350 00:17:24.920 --> 00:17:26.039 they flow through the system. 351 00:17:26.359 --> 00:17:29.759 He mentions two main types of callbacks, pre operation and 352 00:17:29.839 --> 00:17:33.119 post operation. What's the difference between those and why are 353 00:17:33.119 --> 00:17:33.720 they important? 354 00:17:33.839 --> 00:17:36.680 Okay, so imagine you're standing at a checkpoint on a road, 355 00:17:37.079 --> 00:17:40.000 right right, A pre operation callback is like checking the 356 00:17:40.079 --> 00:17:42.720 driver's license and registration before the car is allowed to 357 00:17:42.759 --> 00:17:45.480 pass through, and then a post operation callback is like 358 00:17:45.599 --> 00:17:48.160 checking the trunk after the cars pass through, just to 359 00:17:48.200 --> 00:17:49.480 make sure everything's in order. 360 00:17:49.720 --> 00:17:52.839 So a pre operation callback gives the minifilter a chance 361 00:17:52.880 --> 00:17:57.200 to examine and potentially modify the IRP before the operation 362 00:17:57.319 --> 00:18:00.920 is actually performed, right, while a post operation callback gives 363 00:18:00.920 --> 00:18:03.720 it a chance to inspect the results of the operation 364 00:18:04.240 --> 00:18:06.359 and take any necessary actions based on. 365 00:18:06.359 --> 00:18:10.359 Those results exactly. And those callbacks give mini filters just 366 00:18:10.400 --> 00:18:13.559 this incredible amount of flexibility, right. They can use pre 367 00:18:13.680 --> 00:18:17.680 operation callbacks to block requests, to modify data, to even 368 00:18:17.720 --> 00:18:20.519 redirect the IRP to a different device or file. And 369 00:18:20.599 --> 00:18:23.960 they can use post operation callbacks to log events, to 370 00:18:24.039 --> 00:18:27.880 clean up resources, or even to initiate new io requests. 371 00:18:28.279 --> 00:18:31.079 It's like having checkpoints at every stage of the journey, 372 00:18:31.200 --> 00:18:33.559 allowing the mini filter to kind of control the flow 373 00:18:33.599 --> 00:18:35.960 of traffic and make sure that everything is running smoothly, 374 00:18:36.200 --> 00:18:39.839 that's right. But Yasifovich also mentioned something called contexts in 375 00:18:39.880 --> 00:18:42.079 the context of mini filters. What are those? 376 00:18:42.359 --> 00:18:45.960 Okay? So, contexts are kind of like little sticky notes 377 00:18:46.000 --> 00:18:49.680 that a minifilter can attach to to various objects in 378 00:18:49.720 --> 00:18:54.559 the kernel, you know, yeah, like files, streams, or even volumes. 379 00:18:55.319 --> 00:18:58.400 They're a way for the mini filter to store information 380 00:18:58.519 --> 00:19:01.839 that's specific to that object, you know, like flags or 381 00:19:01.920 --> 00:19:03.960 counters or even cash data. 382 00:19:04.039 --> 00:19:06.559 So it's like adding a little bit of extra information 383 00:19:07.160 --> 00:19:10.680 to the object. Yeah, information that's only visible to the minifilter, 384 00:19:10.839 --> 00:19:11.319 that's right. 385 00:19:11.640 --> 00:19:15.160 And those context can be incredibly useful for managing state 386 00:19:15.480 --> 00:19:19.359 and sharing information between different callback functions within the minifilter. 387 00:19:19.519 --> 00:19:21.240 It sounds like a clever way to kind of keep 388 00:19:21.319 --> 00:19:24.599 track of things right and avoid having to store state 389 00:19:24.640 --> 00:19:28.359 information globally, which could potentially lead to you know, conflicts 390 00:19:28.440 --> 00:19:31.240 or race conditions exactly. But let's talk about something that's 391 00:19:31.400 --> 00:19:33.640 that's always on my mind when it comes to kernel development, 392 00:19:33.920 --> 00:19:38.119 error handling. Yusuphovic stresses the importance of failing fast and 393 00:19:38.160 --> 00:19:41.599 failing safely in kernel mode. Yes, why is that so crucial? 394 00:19:41.920 --> 00:19:45.440 Well, remember, in kernel mode you're operating in the same 395 00:19:45.599 --> 00:19:49.240 address space as the operating system itself. Ah Right, So 396 00:19:49.359 --> 00:19:52.599 even a seemingly minor error in your driver can have 397 00:19:52.720 --> 00:19:54.160 catastrophic consequences. 398 00:19:54.240 --> 00:19:56.880 Right, It's like playing with live wires. One wrong move 399 00:19:57.039 --> 00:19:59.599 and the whole house goes dart exactly. 400 00:19:59.480 --> 00:20:03.359 So if you're driver encounters an unexpected condition, it's essential 401 00:20:03.400 --> 00:20:07.319