WEBVTT 1 00:00:00.120 --> 00:00:03.600 Imagine for a moment, being able to fundamentally alter how 2 00:00:03.640 --> 00:00:07.360 the very core of your operating system functions, not just 3 00:00:07.480 --> 00:00:11.160 patching it, but dynamically changing its behavior, you know, without 4 00:00:11.199 --> 00:00:14.679 rebooting or even recompiling a single line of code. 5 00:00:14.800 --> 00:00:16.079 Yeah, that sounds pretty wild. 6 00:00:16.160 --> 00:00:18.719 That almost sounds like a superpower, right, the ability to 7 00:00:18.960 --> 00:00:22.359 peer into and even reshape your Linux kernel on the fly. 8 00:00:22.600 --> 00:00:23.760 How is that even possible? 9 00:00:23.960 --> 00:00:26.679 It's certainly a profound change in how we interact with 10 00:00:26.719 --> 00:00:32.479 the kernel. And that transformative technology is called eBPF EBPFKA. 11 00:00:32.679 --> 00:00:35.840 This isn't just some minor upgrade. It's really a revolutionary platform. 12 00:00:35.960 --> 00:00:41.079 It's fundamentally changing how we build high performance, security, observability 13 00:00:41.159 --> 00:00:42.159 and networking tools. 14 00:00:42.359 --> 00:00:45.439 And that revolutionary aspect is what we're here to unpack today. 15 00:00:45.560 --> 00:00:48.000 Good plan for this deep dive, our mission is to 16 00:00:48.079 --> 00:00:51.679 explore the power behind eBPF. We'll uncover what it is, 17 00:00:51.920 --> 00:00:54.359 why it's become so incredibly important, especially you know, in 18 00:00:54.399 --> 00:00:58.240 recent years with cloud native stuff, and how it actually 19 00:00:58.280 --> 00:01:01.600 delivers on this promise of un paralleled insight and control. 20 00:01:02.439 --> 00:01:04.400 All our insights today, by the way, are drawn from 21 00:01:04.400 --> 00:01:07.280 the excellent book Learning EVPF by Liz. 22 00:01:07.200 --> 00:01:10.200 Rice A great resource. And you know, even if you 23 00:01:10.280 --> 00:01:13.879 listening aren't planning to write eBPF code yourself, just understanding 24 00:01:13.920 --> 00:01:18.040 the fundamentals that gives you a significant advantage. How so, well, 25 00:01:18.120 --> 00:01:21.359 you're likely to come across eBPF based tooling more and more. 26 00:01:21.560 --> 00:01:25.680 It's rapidly becoming standard in software infrastructure roles, right, and 27 00:01:25.719 --> 00:01:28.359 knowing how it works underneath will help you use those 28 00:01:28.359 --> 00:01:30.519 tools much more effectively. It gives you a real edge. 29 00:01:30.640 --> 00:01:33.760 Okay, so let's unpack this. To really get eBPF, we 30 00:01:33.840 --> 00:01:36.200 probably need to start with the Linux kernel itself. How 31 00:01:36.239 --> 00:01:39.239 would you describe its role for someone maybe not deep 32 00:01:39.319 --> 00:01:41.400 in OS internals. 33 00:01:40.840 --> 00:01:45.560 Sure, think of the kernel as the essential software layer, 34 00:01:45.599 --> 00:01:48.879 sitting directly between your applications and the physical hardware. 35 00:01:49.000 --> 00:01:50.400 The mediator exactly. 36 00:01:50.439 --> 00:01:53.840 It's the traffic cop for all interactions. Immediates every request 37 00:01:53.840 --> 00:01:57.200 from userspace apps through things called system calls or ciscalls, 38 00:01:57.359 --> 00:02:02.480 like reading files, network stuff files, sending network traffic, even 39 00:02:02.560 --> 00:02:06.239 just accessing memory. The kernel handles it all. Everything ultimately 40 00:02:06.319 --> 00:02:07.400 goes through the kernel. 41 00:02:07.079 --> 00:02:10.120 And because applications rely so heavily on the kernel for 42 00:02:10.479 --> 00:02:14.240 well everything, being able to observe those interactions must give 43 00:02:14.280 --> 00:02:17.080 you incredible visibility into what an app is doing. 44 00:02:17.199 --> 00:02:17.800 That's the key. 45 00:02:18.199 --> 00:02:21.560 But where did this idea of hooking into the kernel start. 46 00:02:21.639 --> 00:02:23.680 It had pretty humble origins, didn't they. Oh? 47 00:02:23.680 --> 00:02:28.400 Absolutely. The BPF and EDPF originally stood for Berkeley Packet Filter, 48 00:02:29.080 --> 00:02:30.039 started way back. 49 00:02:29.879 --> 00:02:32.120 In nineteen ninety two, ninety two, Okay. 50 00:02:31.919 --> 00:02:34.639 Yeah, simply as a way to efficiently filter network packets. 51 00:02:34.639 --> 00:02:37.639 We're talking in general purpose thirty two bit instructions looked 52 00:02:37.639 --> 00:02:38.879 a lot like assembly. 53 00:02:38.520 --> 00:02:40.599 Language, so like for TCP dump or something. 54 00:02:40.639 --> 00:02:44.360 Precisely, a tiny piece of BPF code could say filter 55 00:02:44.400 --> 00:02:47.680 out non ippaydance from an Ethernet stream, letting GZP dump 56 00:02:47.759 --> 00:02:51.639 capture only what was relevant. That efficiency was groundbreaking for 57 00:02:51.680 --> 00:02:52.879 network analysis back then. 58 00:02:52.919 --> 00:02:55.319 So the original BPF came into Lenux in ninety seven 59 00:02:55.599 --> 00:02:58.039 kernel two point one one seventy five for that purpose. 60 00:02:58.599 --> 00:03:00.479 But I guess the packet filter part of the name 61 00:03:00.560 --> 00:03:02.360 started to make less sense over time. It did. 62 00:03:02.439 --> 00:03:04.199 It became much more than just packets. 63 00:03:04.439 --> 00:03:07.080 What was the turning point? When did it become more versatile? 64 00:03:07.520 --> 00:03:10.280 Well, the first step beyond just network packets was probably 65 00:03:10.599 --> 00:03:13.919 SUCCUMBPF and twenty twelve Kernel three point. 66 00:03:13.719 --> 00:03:17.080 Five sacked and TTF right for ciscall filter exactly. 67 00:03:17.159 --> 00:03:20.560 It let BPS programs decide whether to allow or deny 68 00:03:20.840 --> 00:03:25.479 user space apps from making specific system calls, so you 69 00:03:25.479 --> 00:03:28.159 could restrict what an application could do at the kernel level. 70 00:03:28.319 --> 00:03:29.319 A security thing. 71 00:03:29.360 --> 00:03:33.039 Yeah, foundational for containers security later on. But the real 72 00:03:33.159 --> 00:03:36.560 extended revolution, the birth of eBPF as we know it, 73 00:03:36.599 --> 00:03:39.560 began in kernel three point one eight. That was twenty fourteen. 74 00:03:39.680 --> 00:03:42.120 Okay, twenty fourteen. And what changed then? It sounds like 75 00:03:42.159 --> 00:03:43.840 it was more than just an incremental update. 76 00:03:44.039 --> 00:03:46.840 Oh, it was a complete overhaul. The instruction set became 77 00:03:46.919 --> 00:03:50.639 sixty four bit, the interpreter was rewritten, Crucial new features 78 00:03:50.639 --> 00:03:52.840 were added. That's just This is where we got eBPF 79 00:03:52.919 --> 00:03:56.080 maps for sharing data between programs and user space. 80 00:03:56.319 --> 00:03:58.240 Ah okay, that sounds. 81 00:03:57.960 --> 00:04:02.400 Important, hugely important. Also the VPF system call for userspace interaction, 82 00:04:02.879 --> 00:04:06.120 and maybe most critically for safety, the eBPF FAIRI fire 83 00:04:06.240 --> 00:04:08.120 was introduced. We'll definitely need to talk more about that. 84 00:04:08.360 --> 00:04:13.159 Absolutely. So if adding new kernel features is traditionally you know, 85 00:04:13.199 --> 00:04:17.519 a massive undertaking months, maybe years to get changes upstream, 86 00:04:17.680 --> 00:04:20.720 how does eBPF change that? What makes it such a 87 00:04:20.720 --> 00:04:22.000 game changer today? 88 00:04:22.240 --> 00:04:25.759 That's the core of its appeal. Really Yeah, eBPF fundamentally 89 00:04:25.839 --> 00:04:28.920 changes how we can extend the kernel. It allows for 90 00:04:29.000 --> 00:04:33.279 dynamic kernel functionality, dynamic meaning meaning eBPF programs can be 91 00:04:33.319 --> 00:04:37.199 loaded and unloaded on demand. This dramatically speeds up adding 92 00:04:37.240 --> 00:04:39.920 new kernel features. We're talking maybe months, not the years 93 00:04:39.920 --> 00:04:41.920 it usually takes for upstream kernel changes. 94 00:04:42.040 --> 00:04:42.439 Wow. 95 00:04:42.560 --> 00:04:45.920 So you can create and deploy custom kernel functionality incredibly 96 00:04:46.000 --> 00:04:49.240 quickly without needing you know, the entire Linux world to 97 00:04:49.360 --> 00:04:51.319 agree and merge it into the main kernel. 98 00:04:51.360 --> 00:04:53.360 And it's not just about speed of deployment, is it. 99 00:04:53.399 --> 00:04:54.560 Performance is key. 100 00:04:54.399 --> 00:04:58.240 Too, absolutely crucial. Once an eBPF program is loaded and 101 00:04:58.360 --> 00:05:02.639 JIT compiled, I'm compiled right, it runs as native machine 102 00:05:02.639 --> 00:05:06.600 instructions directly on the CPU. This avoids those costly back 103 00:05:06.680 --> 00:05:09.720 and forth transitions between kernel space and user space for 104 00:05:09.800 --> 00:05:10.959 every single event. 105 00:05:10.759 --> 00:05:12.360 Which makes it incredibly efficient. 106 00:05:12.439 --> 00:05:15.839 Remarkably efficient. It sounds almost counterintuitive for something running inside 107 00:05:15.879 --> 00:05:19.360 the kernel, but yeah, it allows EVPF tools to operate 108 00:05:19.399 --> 00:05:20.399 with minimal overhead. 109 00:05:20.600 --> 00:05:23.879 That efficiency sounds like a massive win, especially in cloud 110 00:05:23.920 --> 00:05:27.560 native environments. We often see that sidecar model in kubernettes 111 00:05:27.600 --> 00:05:31.120 for injecting things like logging tracing. 112 00:05:30.920 --> 00:05:33.000 Security yep, the sidecar pattern. 113 00:05:33.120 --> 00:05:36.000 How does EVPF compare. Is it a better alternative? 114 00:05:36.360 --> 00:05:40.279 In many ways? Yes, The sidecar approach, while useful, often 115 00:05:40.319 --> 00:05:44.000 means modifying application yamo files, restarting. 116 00:05:43.480 --> 00:05:45.720 Pods, right operational friction. 117 00:05:45.720 --> 00:05:48.839 Exactly, and if something goes wrong, the sidecar might not 118 00:05:48.879 --> 00:05:52.879 even get injected, leaving you blind. Plus, sidecars can potentially 119 00:05:52.879 --> 00:05:56.240 be bypassed by bad actors. How so well, imagine an 120 00:05:56.279 --> 00:06:00.399 attacker deploys an uninstrumented app like a crypto minor sidecar 121 00:06:00.480 --> 00:06:03.839 based security tool which relies on injection, might not even 122 00:06:03.879 --> 00:06:05.639 see it connecting out to its mining pool. 123 00:06:05.759 --> 00:06:08.639 Ah I see, and eBPF avoids that Precisely. 124 00:06:08.959 --> 00:06:12.240 eBPF based tools run on the host and can police 125 00:06:12.319 --> 00:06:16.399 all traffic, all processes on that machine, including everything inside containers. 126 00:06:16.600 --> 00:06:17.399 They're much harder to. 127 00:06:17.439 --> 00:06:20.519 Side steps, so instant complete visibility. 128 00:06:20.160 --> 00:06:24.279 Pretty much without needing app modifications or restarts. That's a 129 00:06:24.319 --> 00:06:27.759 game changer for infrastructure teams wanting comprehensive coverage. 130 00:06:28.040 --> 00:06:30.839 Okay, but these programs are running directly inside the kernel. 131 00:06:31.120 --> 00:06:35.639 That immediately raises safety concerns. What stops a buggy eBPF 132 00:06:35.720 --> 00:06:40.120 program from crashing the whole system or reading memory. It shouldn't. 133 00:06:40.240 --> 00:06:43.680 That's the million dollar question, and the answer is the 134 00:06:43.839 --> 00:06:45.720 eBPF verifier. 135 00:06:45.480 --> 00:06:47.480 The kernel's unseen guardian you called it. 136 00:06:47.680 --> 00:06:50.079 That's a good way to put it. Before an eBPF 137 00:06:50.120 --> 00:06:54.879 program is ever loaded, the verifier performs an exhaustive static analysis. 138 00:06:55.199 --> 00:06:59.720 It checks every single possible execution path, every path, every 139 00:06:59.759 --> 00:07:02.240 path to ensure its safe and won't harm the kernel. 140 00:07:02.360 --> 00:07:05.319 It's like a super strict bouncer at the kernel's door. Honestly, 141 00:07:05.360 --> 00:07:07.319 it's a marvel of engineering that makes this whole thing 142 00:07:07.360 --> 00:07:08.199 possible safely. 143 00:07:08.279 --> 00:07:11.399 So what kinds of checks does this bouncer perform? It 144 00:07:11.480 --> 00:07:13.879 must be incredibly rigorous, it really is. 145 00:07:14.519 --> 00:07:17.480 It tracks the state of each register their types, value 146 00:07:17.600 --> 00:07:22.040 ranges to prevent invalid operations. It enforces trick memory access, 147 00:07:22.040 --> 00:07:24.399 making sure programs only touch memory they're allowed to. 148 00:07:24.600 --> 00:07:26.639 No arbitrary reads are writes none. 149 00:07:27.120 --> 00:07:31.040 It mandates checks for null pointers before they're reference, preventing 150 00:07:31.079 --> 00:07:36.279 common crashes. Very importantly, it guarantees programs run to completion. 151 00:07:36.480 --> 00:07:37.279 How does it do that? 152 00:07:37.519 --> 00:07:41.399 It rejects programs with loops that it can't prove will terminate. 153 00:07:42.000 --> 00:07:45.879 There's also a maximum instruction limit, currently a million instructions 154 00:07:45.879 --> 00:07:47.560 for privileged users a million. 155 00:07:47.600 --> 00:07:49.199 Okay, that's quite a lot. 156 00:07:49.279 --> 00:07:52.759 Actually, it gives you scope for complex logic. The verifier 157 00:07:52.839 --> 00:07:56.199 also checks that helper functions called by the eBPF program 158 00:07:56.399 --> 00:07:59.279 are allowed for that specific program type, and it even 159 00:07:59.399 --> 00:08:03.240 checks for GPL compatible licenses if the program uses certain 160 00:08:03.279 --> 00:08:04.160 restricted helpers. 161 00:08:04.439 --> 00:08:06.519 That's incredibly thorough. It really builds confidence. 162 00:08:06.560 --> 00:08:08.399 It has to be. The kernel stability is. 163 00:08:08.399 --> 00:08:12.399 Paramount, Okay, beyond safety. You mentioned eBPF maps earlier as 164 00:08:12.439 --> 00:08:16.279 a key differentiator from the original BPF. How did maps 165 00:08:16.360 --> 00:08:17.079 change the game? 166 00:08:17.480 --> 00:08:21.279 Maps was a fundamental breakthrough. They are essentially key value stores, 167 00:08:21.399 --> 00:08:24.040 data structures that act as a bridge, a bridge between 168 00:08:24.240 --> 00:08:27.319 between the kernel and user space, or even between different 169 00:08:27.319 --> 00:08:30.920 eBPF programs running in the kernel. They allow communication and state. 170 00:08:30.639 --> 00:08:32.159 Sharing, which wasn't possible before. 171 00:08:32.399 --> 00:08:37.440 Right. Classic BPF was stateless, just processing individual packets. Maps 172 00:08:37.480 --> 00:08:41.720 allow eBPF programs to store state across multiple events or invocations, 173 00:08:41.960 --> 00:08:42.559 so you can use. 174 00:08:42.440 --> 00:08:45.879 Them for things like counters sending metrics from the kernel 175 00:08:46.039 --> 00:08:48.679 up to a userspace monitoring tool exactly. 176 00:08:48.720 --> 00:08:52.279 That's a common use case, or storing configuration data pushed 177 00:08:52.320 --> 00:08:55.960 down from user space, or maintaining state across multiple program 178 00:08:56.000 --> 00:08:58.120 executions like global variables. 179 00:08:58.399 --> 00:09:00.720 And you mentioned tail calls too. What are those? 180 00:09:00.840 --> 00:09:04.360 Tail calls allow one eBPF program to effectively jump to 181 00:09:04.399 --> 00:09:07.600 another eBPF program, replacing itself. It's a way to chain 182 00:09:07.639 --> 00:09:08.399 programs together. 183 00:09:08.519 --> 00:09:11.960 Ah okay, so you can build more complex logic flows. 184 00:09:12.200 --> 00:09:15.120 Precisely, you can chain up to thirty three tail calls currently, 185 00:09:15.720 --> 00:09:18.799 combined with that million instruction limit per program, it gives 186 00:09:18.799 --> 00:09:21.720 you a lot of flexibility for sophisticated in kernel logic. 187 00:09:21.960 --> 00:09:25.159 So maps and tail calls really elevated eBPF from just 188 00:09:25.200 --> 00:09:27.840 a filter to a proper programmable platform. 189 00:09:27.879 --> 00:09:31.360 Absolutely, they are key to building that sophisticated eBPF based 190 00:09:31.360 --> 00:09:32.559 applications we see today. 191 00:09:32.879 --> 00:09:35.440 Now, if you're thinking about building with eBPF, what are 192 00:09:35.440 --> 00:09:38.399 the language choices for the kernel side code itself? I 193 00:09:38.440 --> 00:09:40.000 assume it's pretty low level. 194 00:09:40.399 --> 00:09:43.360 Yes, For the code running in the kernel. Most eBPF 195 00:09:43.399 --> 00:09:46.759 programs are written in C or Rust. Why those Because 196 00:09:46.759 --> 00:09:50.759 they are compilers like Clang LLVM for C can directly 197 00:09:50.799 --> 00:09:54.360 target the EDPF bytecode instruction set. They give you that 198 00:09:54.440 --> 00:09:55.799 necessary low level control. 199 00:09:56.039 --> 00:09:58.519 Makes sense? And what about the user space side? The 200 00:09:58.559 --> 00:10:01.320 applications that load and manage these kernel programs. 201 00:10:01.480 --> 00:10:04.559 There you have much more flexibility. User space applications can 202 00:10:04.600 --> 00:10:08.000 be written in various languages using specific libraries to interact 203 00:10:08.000 --> 00:10:10.000 with the kernel's eBPF capabilities. 204 00:10:10.080 --> 00:10:13.200 When eBPF was first taking off, I remember BCC being 205 00:10:13.360 --> 00:10:14.039 very popular. 206 00:10:14.120 --> 00:10:18.519 Yes, BCC, the BPF compiler collection. It supports Python, Lua, 207 00:10:18.759 --> 00:10:22.440 C plus plus C. It was fantastic for learning and experimentation. 208 00:10:22.519 --> 00:10:23.759 Why was it good for learning? 209 00:10:24.000 --> 00:10:26.320 It abstracted away a lot of the low level complexity, 210 00:10:26.399 --> 00:10:30.559 making EDPF accessible to people who weren't necessarily deep kernel hackers. 211 00:10:30.799 --> 00:10:33.080 You could write Python scripts to load and interact with 212 00:10:33.159 --> 00:10:34.519 CEBPF code. 213 00:10:34.279 --> 00:10:36.519 But it had drawbacks for production it did. 214 00:10:36.840 --> 00:10:39.519 Its main issue was a compile the EDPFC code at 215 00:10:39.600 --> 00:10:41.080 runtime on the target machine. 216 00:10:41.240 --> 00:10:45.120 Oh right, so you needed the compiler toolchain LVM CLIANG 217 00:10:45.519 --> 00:10:48.559 kernel headers on every single server exactly. 218 00:10:48.639 --> 00:10:52.639 That meant bigger dependencies, potential installation headaches, and noticeable startup 219 00:10:52.639 --> 00:10:56.879 delays while the code compiled plus higher resource consumption not 220 00:10:57.000 --> 00:10:58.960 ideal for large scale production deployments. 221 00:10:59.200 --> 00:11:02.240 So there wasn't need something more robust, more portable for 222 00:11:02.399 --> 00:11:06.320 enterprise use. That leads us to KEOII, right, compile once, 223 00:11:06.440 --> 00:11:07.120 run everywhere. 224 00:11:07.279 --> 00:11:11.919 Yes. Keyoor is the modern approach and a huge leap forward. 225 00:11:12.159 --> 00:11:16.200 It's really what made eBPF practical for widespread production use. 226 00:11:16.320 --> 00:11:19.679 How does it achieve that run everywhere magic? It sounds tricky, 227 00:11:19.759 --> 00:11:22.840 with different kernel versions having slightly different data structures, It 228 00:11:22.879 --> 00:11:23.279 is clever. 229 00:11:23.519 --> 00:11:27.320 The key is BTF BPF type format. During compilation, detailed 230 00:11:27.320 --> 00:11:30.360 type information about the kernel structures the eBPF program uses 231 00:11:30.559 --> 00:11:33.519 is embedded into the compiled object file itself. Okay, Then 232 00:11:33.600 --> 00:11:36.080 when you load that program on a target machine, a 233 00:11:36.200 --> 00:11:39.519 userspace library uses that BTF data to understand the specific 234 00:11:39.600 --> 00:11:42.759 kernel version running there. It can then automatically adjust memory 235 00:11:42.799 --> 00:11:45.519 offsets and structure accesses on the fly, so it. 236 00:11:45.440 --> 00:11:47.639 Adapts the pre compiled code at load time. 237 00:11:47.840 --> 00:11:51.720 Essentially, yes, it relocates the memory accesses based on the 238 00:11:51.720 --> 00:11:55.200 actual kernel's layout. This means you can compile your EVPF 239 00:11:55.240 --> 00:11:58.480 program once on your rig machine and then deploy that 240 00:11:58.519 --> 00:12:02.480 same binary across a whole fleet running different kernel versions 241 00:12:02.559 --> 00:12:03.240 within limits. 242 00:12:03.240 --> 00:12:06.279 Of course, that's a massive operational win. No more compiling 243 00:12:06.320 --> 00:12:07.720 on every node huge. 244 00:12:07.799 --> 00:12:10.279 It makes deployment much much simpler. 245 00:12:09.879 --> 00:12:14.000 And the core c library that enables score is LIBPF. 246 00:12:14.240 --> 00:12:17.879 That's right. Lip APF is maintained alongside the kernel source tree, 247 00:12:18.080 --> 00:12:21.559 and it's the reference implementation. It uses auto generated BPF 248 00:12:21.600 --> 00:12:25.559 skeletons c headers that make loading and managing the eBPF 249 00:12:25.639 --> 00:12:28.480 programs and maps much cleaner from user space. 250 00:12:28.279 --> 00:12:30.039 And the benefits over BCC are. 251 00:12:30.240 --> 00:12:34.960 Lower memory footprint, no runtime compilation dependency, no startup delay, 252 00:12:35.200 --> 00:12:38.600 and that crucial keore portability. If you're building a serious 253 00:12:38.600 --> 00:12:42.519 eBPF application today, especially in C, LIBPF is the way 254 00:12:42.559 --> 00:12:42.720 to go. 255 00:12:42.840 --> 00:12:44.960 But it's not just C anymore, is it. The ecosystem 256 00:12:45.000 --> 00:12:46.600 seems to be growing definitely. 257 00:12:46.759 --> 00:12:50.000 The community has built excellent libraries for other popular languages. 258 00:12:50.279 --> 00:12:54.120 For Go developers, there's silly MEPF and also libbed PFG, 259 00:12:54.360 --> 00:12:57.360 which provides Go bindings for lipef. 260 00:12:57.600 --> 00:13:01.399 And for Rust. Rust seems like a natural for systems programming. 261 00:13:01.519 --> 00:13:03.519 It does, and there are several options there too, like 262 00:13:03.600 --> 00:13:08.519 libpfrs red, BPF, and AYA. Each has slightly different approaches 263 00:13:08.559 --> 00:13:11.679 and trade offs, but they all bring eBPF development to 264 00:13:11.799 --> 00:13:14.759 Rust programmers, leveraging Rust safety features. 265 00:13:15.080 --> 00:13:19.000 So lots of options depending on your team's preferred language exactly. 266 00:13:19.039 --> 00:13:20.960 It's much more accessible now than it was just a 267 00:13:21.000 --> 00:13:21.519 few years ago. 268 00:13:21.639 --> 00:13:26.559 Okay, so we have this powerful, safe portable technology, how 269 00:13:26.600 --> 00:13:30.399 is it actually being used? Let's talk real world applications. 270 00:13:30.519 --> 00:13:32.919 Observability seems like a prime candidate. 271 00:13:32.639 --> 00:13:36.240 Absolutely because eBPF programs can attach to so many points ciscolls, 272 00:13:36.320 --> 00:13:39.120 kernel trace points, kernel functions, eight K probes, even user 273 00:13:39.120 --> 00:13:43.759 space functions uprobes. They provide incredibly powerful tracing capabilities. 274 00:13:43.279 --> 00:13:45.519 Deeper than traditional monitoring, much. 275 00:13:45.320 --> 00:13:49.559 Deeper and often with less overhead. You get unparalleled insight 276 00:13:49.639 --> 00:13:55.360 into system behavior, network disc io function calls within applications, 277 00:13:55.399 --> 00:13:58.960 all without modifying or restarting the application itself. It's like 278 00:13:59.000 --> 00:14:01.679 having x ray vision into your running systems. 279 00:14:01.279 --> 00:14:05.080 And in networking. You mentioned BPF's origins there, but eBPF 280 00:14:05.120 --> 00:14:07.240 takes it much further, especially with XDP. 281 00:14:07.799 --> 00:14:12.840 Yes XDP the express data path. This is really powerful. 282 00:14:13.120 --> 00:14:16.600 XDP allows eBPF programs to run at the earliest possible 283 00:14:16.639 --> 00:14:19.559 point in the network stack, right in the driver, even 284 00:14:19.639 --> 00:14:22.960 before the kernel allocates memory for the packet BSKB. 285 00:14:22.799 --> 00:14:25.919 So super fast packet process extremely fast. 286 00:14:26.039 --> 00:14:28.679 This lets you do things like high performance firewalling, dropping 287 00:14:28.720 --> 00:14:32.559 malicious packets for dedas mitigation, or implementing custom load balancing 288 00:14:32.799 --> 00:14:34.720 all at line rate in many cases. 289 00:14:34.399 --> 00:14:37.840 Before the packet even really hits the main kernel networking exactly. 290 00:14:38.080 --> 00:14:41.120 You can make decisions and potentially drop packets with minimal overhead. 291 00:14:41.639 --> 00:14:44.919 Some network cards even support offloading XDP programs directly onto 292 00:14:44.919 --> 00:14:46.120 the NIC hardware itself. 293 00:14:46.159 --> 00:14:49.240 Wow, hardware offload. That's serious performance it is. 294 00:14:49.679 --> 00:14:52.960 It allows you to handle enormous amounts of traffic. Think 295 00:14:53.000 --> 00:14:57.360 about mitigating those packet of death vulnerabilities. XDP can drop 296 00:14:57.440 --> 00:15:00.200 the malicious packet before it can even cause harm in 297 00:15:00.240 --> 00:15:01.080 the network stack. 298 00:15:01.360 --> 00:15:04.639 And this has big implications for Kubernetes networking too, doesn't 299 00:15:04.639 --> 00:15:05.960 it replacing things like. 300 00:15:05.960 --> 00:15:10.919 Iptables fundamentally changing it. Yes. Traditional Kubernetes networking often relies 301 00:15:10.960 --> 00:15:14.440 heavily on iptables for implementing network policies and services and 302 00:15:14.600 --> 00:15:16.240 contract for connection. 303 00:15:15.919 --> 00:15:18.399 Tracking, and those have scaling issues, right they do. 304 00:15:18.799 --> 00:15:22.639 Iptable's rules are processed linearly, so performance degrades is the 305 00:15:22.720 --> 00:15:25.960 number of rules or services grows. That's o n complexity. 306 00:15:26.679 --> 00:15:28.559 Contract can also become a bottleneck. 307 00:15:28.759 --> 00:15:30.159 How does eBPF help. 308 00:15:30.360 --> 00:15:35.039 eBPF based networking solutions like Cilium replace Iptable's rules and 309 00:15:35.120 --> 00:15:38.480 contract with EVPF programs and maps they use efficient hash 310 00:15:38.519 --> 00:15:39.320 maps for lookups. 311 00:15:39.320 --> 00:15:41.080 Hash maps the lookups are faster. 312 00:15:40.919 --> 00:15:45.039 Much faster oh one or constant time lookups for network policies, 313 00:15:45.120 --> 00:15:48.399 service load balancing, and connection tracking regardless of the number 314 00:15:48.399 --> 00:15:51.480 of rules or connections. This is critical for performance and 315 00:15:51.480 --> 00:15:53.639 scalability in large Kubernetes clusters. 316 00:15:53.840 --> 00:15:57.480 That's a significant architectural shift. Now here's a capability that 317 00:15:57.519 --> 00:16:02.919 sounds almost like magic transparent ryption insight. How can eBPF 318 00:16:03.120 --> 00:16:05.879 see inside encrypted traffic without keys? 319 00:16:06.120 --> 00:16:09.919 It's a clever trick. It doesn't actually break the encryption itself. Instead, 320 00:16:10.080 --> 00:16:14.879 eBPF uses uprobes to hook into userspace SSLTLS libraries like 321 00:16:15.000 --> 00:16:17.080 open ssl ortls. 322 00:16:16.559 --> 00:16:19.600 So it attaches to the applications encryption functions precisely. 323 00:16:19.679 --> 00:16:22.559 It attaches to functions like SSL read and SSL write. 324 00:16:23.000 --> 00:16:25.679 It can then intercept the data after the library has 325 00:16:25.720 --> 00:16:29.080 decrypted it for SSL rate or before the library encrypts 326 00:16:29.080 --> 00:16:29.960 it for SSL right. 327 00:16:30.159 --> 00:16:33.480 Ah. So it catches the plaintext data inside the application's 328 00:16:33.559 --> 00:16:37.279 memory just before encryption or just after decryption exactly. 329 00:16:37.320 --> 00:16:40.000 You get visibility into the clear text data flow without 330 00:16:40.039 --> 00:16:43.480 needing access to the private keys or certificates. It's incredibly 331 00:16:43.519 --> 00:16:48.399 powerful for security monitoring, troubleshooting, and even compliance without disrupting 332 00:16:48.399 --> 00:16:49.440 the encryption itself. 333 00:16:49.960 --> 00:16:53.679 That's fascinating. Okay, let's shift fully to security. eBPF seems 334 00:16:53.679 --> 00:16:57.360 poised to move security from just detection to actual prevention. 335 00:16:57.720 --> 00:17:01.200 That's a key trend for years. Second PPF provided a 336 00:17:01.240 --> 00:17:04.559 basic layer by filtering ciscoles, which is used heavily in 337 00:17:04.640 --> 00:17:05.799 Docker and Kubernetes. 338 00:17:06.000 --> 00:17:07.759 Right limiting what a container can do. 339 00:17:07.960 --> 00:17:11.759 Yes, and some eBPF tools like inspector gadget can even 340 00:17:11.759 --> 00:17:15.200 help generate those secom profiles by observing what ciscoles an 341 00:17:15.200 --> 00:17:16.880 application legitimately uses. 342 00:17:17.279 --> 00:17:20.960 But generating perfect profiles is hard right capturing all error 343 00:17:20.960 --> 00:17:22.279 pads and edge cases. 344 00:17:22.400 --> 00:17:26.000 It's very challenging, and traditional detection often suffers from that 345 00:17:26.440 --> 00:17:29.319 Tatau problem time of check to time of use the 346 00:17:29.480 --> 00:17:32.799 race condition. Exactly by the time a separate security agent 347 00:17:32.839 --> 00:17:35.960 detects a malicious ciscle or file access and decides to act, 348 00:17:36.279 --> 00:17:39.160 the damage might already be done. The malicious operation might 349 00:17:39.200 --> 00:17:39.759 have completed. 350 00:17:39.960 --> 00:17:43.720 So how does eBPF enable prevention closing that time gap? 351 00:17:43.839 --> 00:17:46.759 This is where a specific eBPF program type comes in 352 00:17:47.039 --> 00:17:51.480 bpfpot typos. These programs attached to hooks within the Linux 353 00:17:51.480 --> 00:17:53.400 Security Module Framework. 354 00:17:53.200 --> 00:17:55.839 LSM, like a Parmer or Cylenux. 355 00:17:55.440 --> 00:17:59.400 Use exactly the same framework. These eBPF LSM programs can 356 00:17:59.440 --> 00:18:02.759 perform he checks directly inside the kernel before a sensitive 357 00:18:02.759 --> 00:18:05.160 operation happens, and crucially, they can return. 358 00:18:05.039 --> 00:18:07.359 An air code and that error code. 359 00:18:07.039 --> 00:18:09.920 It prevents the operation from proceeding. The kernel respects the 360 00:18:10.000 --> 00:18:12.680 error code returned by the LSM hook, stopping the action 361 00:18:12.720 --> 00:18:16.200 before it even starts. It happens synchronously within the context 362 00:18:16.200 --> 00:18:17.640 of the original operation. 363 00:18:17.480 --> 00:18:20.839 So it's in kernel enforcement, not just after the fact detection. 364 00:18:21.000 --> 00:18:24.720 Precisely, it's a fundamental shift from reactive detection to proactive 365 00:18:24.799 --> 00:18:26.119 synchronous prevention. 366 00:18:25.960 --> 00:18:28.480 And tools like Cilium Tetragon build on this. 367 00:18:28.720 --> 00:18:32.599 They do. Tetragen uses EVPF attaching to k probes and 368 00:18:32.599 --> 00:18:36.200 trace points, not just LSM hooks, to get rich context 369 00:18:36.279 --> 00:18:39.240 about security events. It can filter these events efficiently in 370 00:18:39.279 --> 00:18:41.279 the kernel, reducing overhead, and. 371 00:18:41.240 --> 00:18:43.400 Can it actively stop threats Yes. 372 00:18:44.160 --> 00:18:46.920 Based on his policies, it can use helpers like bpfen 373 00:18:47.039 --> 00:18:50.240 signal to for example, send a sig kill signal to 374 00:18:50.319 --> 00:18:55.440 terminate a malicious process immediately synchronously when a violation is detected. 375 00:18:55.279 --> 00:18:58.519 So detect and terminate right there in the kernel context exactly. 376 00:18:58.559 --> 00:19:02.119 It moves security beyond just observing and alerting to actively 377 00:19:02.200 --> 00:19:04.839 enforcing policy and preventing harm in real time. 378 00:19:05.039 --> 00:19:07.319 So to kind of wrap this deep dive up, eBPF 379 00:19:07.359 --> 00:19:09.680 has really come a long way from just filtering packets. 380 00:19:09.880 --> 00:19:10.960 An incredible journey. 381 00:19:11.119 --> 00:19:17.519 It's evolved into this powerful, versatile platform that's genuinely transforming observability, networking, 382 00:19:17.599 --> 00:19:22.200 and security, giving us unprecedented control and insight right at 383 00:19:22.200 --> 00:19:22.960 the kernel level. 384 00:19:23.279 --> 00:19:25.880 It really feels like we're still just scratching the surface 385 00:19:25.880 --> 00:19:30.319 of what's possible with it. The innovation is happening incredibly fast, and. 386 00:19:30.279 --> 00:19:32.319 The future looks bright. What's next? 387 00:19:32.400 --> 00:19:39.720 Well, The eBPF Foundation, which includes major players like Google, Meta, Microsoft, Netflix, isovalent. 388 00:19:40.599 --> 00:19:42.640 They're working on standardizing. 389 00:19:42.000 --> 00:19:44.799 eBPF standardization is always good, yeah. 390 00:19:44.599 --> 00:19:47.720 And even exploring its adoption for Windows, which could potentially 391 00:19:47.759 --> 00:19:51.359 allow eBPF programs written for Linux to run on Windows too. 392 00:19:51.519 --> 00:19:53.880 Wow. Cross platform eBPF. 393 00:19:53.880 --> 00:19:56.799 That's the long term vision for some. It's a rapidly 394 00:19:56.839 --> 00:19:59.960 evolving field. New kernel features are landing all the time, 395 00:20:00.240 --> 00:20:03.000 though it's worth remembering there's always a lag before the 396 00:20:03.079 --> 00:20:06.119 very latest features make it into stable Linux distributions. 397 00:20:06.160 --> 00:20:08.799 You might be running sure the usual adoption cycle. 398 00:20:08.599 --> 00:20:10.920 Right, but the pace of development is really impressive. 399 00:20:11.160 --> 00:20:13.680 It certainly seems like it's going to shape how many 400 00:20:13.680 --> 00:20:16.519 of our tools work now and in the future. So 401 00:20:16.839 --> 00:20:20.359