WEBVTT 1 00:00:07.879 --> 00:00:10.960 Ah, Warren, you're looking kind of sharp there in that jacket. 2 00:00:11.000 --> 00:00:13.679 You look kind of like a NASCAR driver with all 3 00:00:13.720 --> 00:00:14.960 those corporate logos. 4 00:00:15.119 --> 00:00:15.679 What's going on? 5 00:00:15.720 --> 00:00:18.800 Man's fun It's funny you should mention that, because I 6 00:00:18.879 --> 00:00:21.679 have something interesting for today's episode. One of the recurring 7 00:00:21.719 --> 00:00:24.199 themes on our podcast I feel like is incident management. 8 00:00:24.199 --> 00:00:26.519 It's something like lots of people want to talk about, 9 00:00:26.879 --> 00:00:28.800 and quite a few guests that come on to discuss 10 00:00:28.839 --> 00:00:33.840 their stressful, traumatic experiences with on call and whatnot. One 11 00:00:33.880 --> 00:00:35.960 of these guests has stepped up and wanted to be 12 00:00:36.159 --> 00:00:39.479 today's sponsor for the episode, and that's a pager Duty. 13 00:00:39.560 --> 00:00:41.359 So I've actually been a fan of page Duty in 14 00:00:41.399 --> 00:00:43.759 the past when I've reached for an incident management tool. 15 00:00:43.920 --> 00:00:46.439 And what's nice is, compared to the competitors that we've 16 00:00:46.439 --> 00:00:49.240 heard from, it's clear that they're actually listening to feedback, 17 00:00:49.439 --> 00:00:54.159 unlike you know, other enterprise companies utilizing your internal messaging 18 00:00:54.200 --> 00:00:57.079 platform like Slack to interact with incidents. Especially for US, 19 00:00:57.200 --> 00:01:00.000 I feel like it's like a baseline requirement for communication 20 00:01:00.200 --> 00:01:03.000 and collaboration, and they've actually opened up their Slack integration 21 00:01:03.039 --> 00:01:05.480 to everyone, not just the customers who have shelled out 22 00:01:05.519 --> 00:01:08.280 money for their enterprise plan, so it's really nice to 23 00:01:08.280 --> 00:01:10.439 see compared to competitors where it feels like you need 24 00:01:10.480 --> 00:01:13.079 to pay an enterprise tax even if you aren't an 25 00:01:13.159 --> 00:01:17.280 enterprise company. I'm particularly like their automatic channel creation when 26 00:01:17.319 --> 00:01:18.879 there is an incident. You know, if you have like 27 00:01:18.920 --> 00:01:21.319 lots of incidents, like one every single day, that gets 28 00:01:21.319 --> 00:01:24.920 pretty tedious. So, uh, thank you Page Duty for sponsoring 29 00:01:24.920 --> 00:01:25.519 this episode. 30 00:01:25.680 --> 00:01:28.879 Yeah, that's super cool. Definitely thank you to Page of Duty. 31 00:01:29.120 --> 00:01:31.239 And I'm a big Page of Duty fan. It's one 32 00:01:31.280 --> 00:01:34.280 of those tools which I think there's not a lot 33 00:01:34.319 --> 00:01:36.159 in this category. I could be wrong, but it's just 34 00:01:36.200 --> 00:01:38.120 one of those like when you are trying to do 35 00:01:38.200 --> 00:01:41.000 a certain thing, you know, incident response, like Page of 36 00:01:41.040 --> 00:01:42.680 Duty is just the name that comes to mind. 37 00:01:42.719 --> 00:01:44.799 You a sake of you know, people thinking like it's 38 00:01:44.840 --> 00:01:46.840 really easy to do, and then they go out and 39 00:01:47.079 --> 00:01:48.840 do it themselves. And like I've been in a bunch 40 00:01:48.879 --> 00:01:52.040 of companies where this has been a pattern of trying 41 00:01:52.040 --> 00:01:54.040 to do it yourself. And the best part is when 42 00:01:54.519 --> 00:01:58.560 you hook up your incident management or on call reporting 43 00:01:58.640 --> 00:02:01.959 you know, monitoring solution to your production systems and you're 44 00:02:02.000 --> 00:02:04.159 running on the same infrastructure, and then you have a 45 00:02:04.159 --> 00:02:06.239 production incident. Well you know what else is down at 46 00:02:06.239 --> 00:02:09.719 that exact same moment, So I can I can highly 47 00:02:09.719 --> 00:02:13.000 recommend not building this yourself. That's my own sort of 48 00:02:13.000 --> 00:02:13.919 traumatic experience. 49 00:02:14.080 --> 00:02:16.360 I think that's a rite of passage, building your own 50 00:02:16.400 --> 00:02:19.280 monitoring system. And then you do it and you learn 51 00:02:19.319 --> 00:02:21.280 that lesson that you just brought up and like, wow, 52 00:02:21.400 --> 00:02:25.159 that was stupid, and then you go to patre Duty's 53 00:02:25.159 --> 00:02:26.919 website and move on with your life. 54 00:02:27.360 --> 00:02:30.199 So I've I've shilled out for the episode. So maybe 55 00:02:30.240 --> 00:02:33.560 we'll talk about something interesting now right on. 56 00:02:34.479 --> 00:02:39.120 Well, yeah, I think today's guest might have some interesting topics. 57 00:02:39.840 --> 00:02:41.000 O Mair, how are you, Bud? 58 00:02:41.120 --> 00:02:43.680 Yes? Good, how are you? Thank you for having me here? 59 00:02:44.039 --> 00:02:44.879 Dude, welcome back. 60 00:02:44.919 --> 00:02:47.639 I can't believe we determined it was three years ago, 61 00:02:47.719 --> 00:02:48.840 right that you were last time? 62 00:02:49.400 --> 00:02:51.360 Yeah? Yeah, that's just nuts. 63 00:02:51.800 --> 00:02:54.080 Yeah, Like I don't even want to have this conversation 64 00:02:54.120 --> 00:02:56.360 with myself about what I've done in those last three 65 00:02:56.400 --> 00:02:57.879 years that I also don't remember. 66 00:02:58.240 --> 00:03:00.960 So the listeners are are missing out on that pre 67 00:03:00.960 --> 00:03:05.800 recorded conversations that were having about you know, how long 68 00:03:06.159 --> 00:03:09.080 how long has it been? Actually? Because Will you were 69 00:03:09.080 --> 00:03:11.360 telling us, you know, last three years may or may 70 00:03:11.400 --> 00:03:12.199 not have happened for you. 71 00:03:12.639 --> 00:03:15.479 Yeah, And I did mention before we started recording that 72 00:03:15.560 --> 00:03:20.960 I bucket things into events that happened between before nineteen 73 00:03:21.039 --> 00:03:25.080 ninety and things that happened between nineteen ninety and yesterday. 74 00:03:25.120 --> 00:03:27.319 And I really can't get any more granular than that. 75 00:03:27.639 --> 00:03:31.759 So Omer was just here yesterday, I know exactly. 76 00:03:32.680 --> 00:03:33.560 And you know what's weird. 77 00:03:34.240 --> 00:03:35.919 I noticed that you and I are wearing the same 78 00:03:36.000 --> 00:03:38.879 T shirt today, and I've got to go back and 79 00:03:38.919 --> 00:03:41.159 look further recording because I think we were in the 80 00:03:41.199 --> 00:03:43.719 same T shirt last time you were on as well. 81 00:03:44.039 --> 00:03:46.319 Here's a fun fact. I only have one kind of 82 00:03:46.360 --> 00:03:49.800 T shirt. I have like thirty of these differing colors. 83 00:03:50.400 --> 00:03:54.599 It's the same T shirt. That's very possible. 84 00:03:54.800 --> 00:03:59.560 Yeah, my wardrobe is very much like that as well. 85 00:04:00.039 --> 00:04:01.919 My wife bought me one of these T shirts once 86 00:04:01.960 --> 00:04:03.719 and I was like, oh, that's super cool. 87 00:04:03.759 --> 00:04:06.240 And now that's all I. 88 00:04:06.159 --> 00:04:08.439 Have, just that one T shirt. 89 00:04:09.280 --> 00:04:12.960 Just the one one copy. 90 00:04:13.039 --> 00:04:13.520 Yeah. 91 00:04:13.639 --> 00:04:15.919 I mean Omar's over there flex and saying that he 92 00:04:16.040 --> 00:04:19.000 has like them in different colors and multiple shirts. I 93 00:04:19.040 --> 00:04:20.839 have just this one. 94 00:04:21.759 --> 00:04:23.120 Every day is laundry day for you. 95 00:04:25.079 --> 00:04:30.519 Yeah, sure, yeah, yeah, yeah, yeah, we'll go with cool. 96 00:04:30.600 --> 00:04:35.720 So we were going to talk about Kubernetes and ll ms, right, yeah, 97 00:04:35.920 --> 00:04:37.000 give us a rundown on that. 98 00:04:37.279 --> 00:04:40.360 Over the past years things have happened for me as well. 99 00:04:40.480 --> 00:04:45.680 So I'm an already architect a zest, which means that 100 00:04:45.800 --> 00:04:50.279 I lead our Kubernetes products. So we're building stuff to 101 00:04:50.360 --> 00:04:55.000 help you optimize kubernets clusters. And recently it seems like, 102 00:04:55.120 --> 00:04:59.480 well everybody have noticed the one hype that drags the 103 00:04:59.519 --> 00:05:03.600 world to direction. It's all around AI now and people 104 00:05:03.639 --> 00:05:06.800 are starting to focus more on kubernets and AI at 105 00:05:06.800 --> 00:05:08.879 the same time, which is something I never expected. I 106 00:05:08.879 --> 00:05:12.480 actually thought the word goes specifically for DevOps. I thought 107 00:05:12.519 --> 00:05:15.720 it was always going towards serverless and kind of not 108 00:05:15.879 --> 00:05:19.360 worrying about infrastructure and managing your own stuff or not 109 00:05:19.480 --> 00:05:22.199 caring about resources, which in some way did happen. And 110 00:05:22.279 --> 00:05:27.800 you did see like different platforms like both serverless from Awus, 111 00:05:27.839 --> 00:05:32.519 azure GCPAA, but also platforms like versaill, Heroku, fly Io, 112 00:05:32.680 --> 00:05:34.800 things like that helped you just deploy your app and 113 00:05:34.800 --> 00:05:38.639 move on with your life. But then that's some I 114 00:05:38.680 --> 00:05:40.879 don't know what changed the tide but it seems like 115 00:05:41.839 --> 00:05:46.360 companies are pushing towards Kubernetes. The last time last Cubicon, 116 00:05:46.439 --> 00:05:49.839 I think it was in London, they said seventy something 117 00:05:49.920 --> 00:05:53.160 between seventy and seventy five percent of corporates in the 118 00:05:53.199 --> 00:05:57.639 world are either already using kubernets or migrating. So that 119 00:05:57.839 --> 00:06:00.519 was mind blowing to me. I never saw that coming. 120 00:06:01.519 --> 00:06:06.959 And over time we've started seeing companies naturally go to AI, 121 00:06:07.439 --> 00:06:11.879 either using AI throughout their products or trying to train lms, 122 00:06:12.160 --> 00:06:15.399 or actually adopting them after they've been trained, but trying 123 00:06:15.439 --> 00:06:17.680 to deploy them on their own. This is kind of 124 00:06:18.120 --> 00:06:21.000 things that we can focus on. We can also speak 125 00:06:21.000 --> 00:06:24.279 about the unrecorded part that has to do with AI 126 00:06:24.399 --> 00:06:26.879 hype and how people are vibe going their way to production. 127 00:06:28.000 --> 00:06:29.839 I think we're definitely going to get there. But you 128 00:06:29.879 --> 00:06:31.920 mentioned a lot of interesting things because I was on 129 00:06:31.959 --> 00:06:35.000 the same path for you, like Servilis for me started 130 00:06:35.040 --> 00:06:39.079 like twenty twenty fourteen, even like before Kubernetes was a thing, 131 00:06:39.279 --> 00:06:43.040 and there was ideas of this coming into a lot 132 00:06:43.040 --> 00:06:45.240 of companies, And I feel like there is this question 133 00:06:45.279 --> 00:06:48.279 of like why did not the better technology become more popular, 134 00:06:49.079 --> 00:06:52.199 the more extreme like push things aside, focus only on 135 00:06:52.240 --> 00:06:56.480 the business value. And you basically you just said it 136 00:06:56.519 --> 00:06:59.279 really got me thinking like why did And I'm sure 137 00:06:59.279 --> 00:07:01.439 I'm going to get some angry emails for this. Why 138 00:07:01.439 --> 00:07:04.720 did a worse solution become more popular? Why did you 139 00:07:04.759 --> 00:07:07.279 get better adopted? And I think it's because there's this 140 00:07:07.399 --> 00:07:13.240 natural tendency for humanity to take step changes for things 141 00:07:13.399 --> 00:07:16.759 rather than giant leaps. And there's actually a core concept 142 00:07:16.800 --> 00:07:19.720 for this in mathematics. It's like you found a local 143 00:07:19.759 --> 00:07:21.920 optima and you're just making small little jumps, and in 144 00:07:22.000 --> 00:07:25.639 order to find a larger jump out larger maximal, you 145 00:07:25.680 --> 00:07:27.879 have to make giant leaps. And in Japanese it's called 146 00:07:27.959 --> 00:07:32.279 kaikaku rather than kaisen if you're familiar with manufacturing lean terms. 147 00:07:32.600 --> 00:07:34.560 And I feel like it's really uncomfortable for people to 148 00:07:34.560 --> 00:07:37.120 throw away everything they have and make a huge leap, 149 00:07:37.199 --> 00:07:39.680 and I feel like serverless is a huge leap, and 150 00:07:39.759 --> 00:07:43.000 Kubernetes is people can just keep doing what they're doing 151 00:07:43.040 --> 00:07:46.759 today and delude themselves into thinking they're making a real change. 152 00:07:47.040 --> 00:07:48.759 Do you feel like it's a step back. 153 00:07:49.120 --> 00:07:51.800 I don't think it's better than what the Open container 154 00:07:51.839 --> 00:07:57.040 initiative Docker containers could have been, but given the companies 155 00:07:57.079 --> 00:08:00.120 that were backing it, which was pretty much Docker, I 156 00:08:00.120 --> 00:08:02.600 think that really answered the question of why it didn't 157 00:08:02.600 --> 00:08:03.079 get further. 158 00:08:03.439 --> 00:08:06.120 My opinion of this is that the one great thing 159 00:08:06.160 --> 00:08:10.759 that Kubernetes did is growing insane, like an insane community 160 00:08:10.759 --> 00:08:15.319 around it, which goes to open source projects under C 161 00:08:15.399 --> 00:08:18.639 and CF, but also companies either building on top of 162 00:08:18.639 --> 00:08:21.439 these open source projects or just pushing themselves into C 163 00:08:21.519 --> 00:08:26.279 and CF, which is incredible because from a very raw 164 00:08:26.360 --> 00:08:28.399 product that you had to do so much to just 165 00:08:28.439 --> 00:08:31.040 get to production, you can now get your well you 166 00:08:31.079 --> 00:08:33.120 deploy Kubernetes. You can either do it on your own, 167 00:08:33.120 --> 00:08:35.840 but you probably won't. You use a service, but then 168 00:08:36.000 --> 00:08:39.120 things like helm and Customize and other things around it 169 00:08:39.120 --> 00:08:42.240 can just help you deploy things. And then companies started 170 00:08:42.240 --> 00:08:45.440 building on top of that operators, so you can get 171 00:08:45.639 --> 00:08:50.639 you know, elastic logging, monitoring databases, cash instances, you can 172 00:08:50.960 --> 00:08:54.159 put whatever you want. With an operator, you just help 173 00:08:54.240 --> 00:08:57.080 install something and then you have everything you need. It's 174 00:08:57.080 --> 00:09:00.559 not from the start, but it kind of gives you 175 00:09:00.639 --> 00:09:03.840 control over everything. And then people said, okay, then now 176 00:09:04.000 --> 00:09:06.279 you have to manage infrastructure, have to manage notes and 177 00:09:06.519 --> 00:09:08.919 what's going to be with auto skating and things like that. 178 00:09:09.720 --> 00:09:12.960 But then you have projects like Carpenter, which is AWS 179 00:09:13.039 --> 00:09:16.360 is pushing, but Azure is jumping on that wagon. So 180 00:09:16.720 --> 00:09:19.679 you can kind of have the best of both throws, right, 181 00:09:19.799 --> 00:09:22.759 You keep your control, you don't pay as much, which 182 00:09:22.759 --> 00:09:25.279 is debate of but we can talk about that. It's Kubernetes. 183 00:09:25.679 --> 00:09:28.080 It's something that deploys containers. That's it. 184 00:09:28.200 --> 00:09:30.519 I think there's a nerd aspect of it as well, 185 00:09:30.679 --> 00:09:34.200 because it's something that's just fun to totally nerd out on. 186 00:09:34.399 --> 00:09:37.679 It's so configurable and so flexible. I know quite a 187 00:09:37.720 --> 00:09:40.840 few people who are part of the Kubernetes at Home 188 00:09:41.240 --> 00:09:44.960 project or the Kubernetes Home Lab project, and the amount 189 00:09:45.000 --> 00:09:48.759 of money and level of work these people have put 190 00:09:48.799 --> 00:09:54.279 into their home Kubernetes lab for doing who knows what, 191 00:09:54.440 --> 00:09:57.279 you know, like telling the refrigerator when it's time to 192 00:09:57.360 --> 00:10:03.600 defrost or whatever, and and they just get completely passionate 193 00:10:03.600 --> 00:10:07.240 about it. And I think that's a big allure to 194 00:10:07.360 --> 00:10:10.440 Kubernetes because in our industry were people who like to 195 00:10:10.559 --> 00:10:13.799 just nerd out on stuff and tinker with stuff, and 196 00:10:13.879 --> 00:10:17.200 with servilists, you don't get that option. You can deploy 197 00:10:17.279 --> 00:10:21.000 your container and it works, and if it doesn't work, 198 00:10:21.039 --> 00:10:24.080 you can deploy your container again and then it'll work. 199 00:10:24.519 --> 00:10:25.000 Yeah. 200 00:10:25.039 --> 00:10:26.960 I mean, I think the whole lad thing is interesting. 201 00:10:28.200 --> 00:10:32.039 You got experience doing something that you liked more than 202 00:10:32.080 --> 00:10:35.120 whatever your company was doing because your company was doing 203 00:10:35.200 --> 00:10:39.440 something horrific, and you can go down that route. I mean, 204 00:10:39.440 --> 00:10:42.200 I do see people who are even running stuff at 205 00:10:42.240 --> 00:10:45.159 home that would prefer servile lists, but they feel like 206 00:10:45.200 --> 00:10:47.679 it's too much of a burden to convince their organization 207 00:10:48.240 --> 00:10:51.600 to make the switch. Fundamentally, it does feel like it 208 00:10:51.639 --> 00:10:53.840 has to be a switch, you know, And I think 209 00:10:53.840 --> 00:10:58.360 that cognitive burden or political burden is just too much 210 00:10:58.440 --> 00:11:00.440 for people to deal with a lot of ways. What 211 00:11:00.600 --> 00:11:04.320 I'm actually interested in is something you said earlier, and 212 00:11:04.360 --> 00:11:10.279 that's you're surprised there's a marrying between AI and Kubernetes because, 213 00:11:10.519 --> 00:11:13.200 as we'll pointed out, running locally in a home lab, 214 00:11:13.240 --> 00:11:14.559 like what are you going to run? Like, you're not 215 00:11:14.879 --> 00:11:16.639 going to run dockers Form and you're definitely not going 216 00:11:16.679 --> 00:11:20.600 to run Nomad After everything that Hashi Corpus done, so 217 00:11:20.879 --> 00:11:23.679 like what are you left with? You're you open open 218 00:11:23.679 --> 00:11:26.600 stack or coro os, like you're talking about spinning up 219 00:11:26.679 --> 00:11:29.720 operating systems everywhere. I do feel like Kubernetes is an 220 00:11:29.720 --> 00:11:32.679 answer there. But for AI it also is like one 221 00:11:32.679 --> 00:11:35.559 of the few things that I have actually recommended, especially 222 00:11:35.559 --> 00:11:37.720 if you need to spin up lots of models or 223 00:11:38.000 --> 00:11:41.120 configure the parameters for running those models, for doing infernts 224 00:11:41.159 --> 00:11:44.919 that are different per user or per customer. Having independent 225 00:11:44.960 --> 00:11:48.960 models scales well with different namespaces in Kubernetes, whereas the 226 00:11:49.000 --> 00:11:52.480 other container orchestrators don't really have this concept. You just 227 00:11:52.519 --> 00:11:55.519 spin up the same container over and over again with 228 00:11:55.600 --> 00:11:58.960 the same parameters, and it's not really fundamentally controllable. So 229 00:11:59.039 --> 00:12:00.919 I feel like the models a little bit different there. 230 00:12:01.039 --> 00:12:05.120 The beautiful thing about Kuberneties is that it's very extendable. 231 00:12:05.399 --> 00:12:08.159 You can basically do whatever you want. Right It's a 232 00:12:08.200 --> 00:12:12.159 set of APIs that you have on the basic binary. 233 00:12:12.399 --> 00:12:13.559 But if you want to build on top of that, 234 00:12:13.639 --> 00:12:17.799 we mentioned operators earlier, we're building operators at ST. You 235 00:12:17.879 --> 00:12:20.440 just make up your own APIs, you deploy them to 236 00:12:20.480 --> 00:12:22.559 the cluster. They work, so you can kind of change 237 00:12:22.559 --> 00:12:25.159 whatever you want. You mentioned name spaces, you can if 238 00:12:25.200 --> 00:12:28.120 you want it. You can build your own notion of 239 00:12:28.200 --> 00:12:30.879 namespaces that would fit whatever you're trying to do, and 240 00:12:30.919 --> 00:12:33.000 it would just work. You would have to install it 241 00:12:33.080 --> 00:12:35.919 with other applications as a service provider or as a 242 00:12:35.960 --> 00:12:39.200 stand alone project, but anything is possible. That's how they 243 00:12:39.519 --> 00:12:41.840 This is something I really appreciate about kubernets. First of all, 244 00:12:41.840 --> 00:12:44.320 I totally agree with Will that it's totally something to 245 00:12:44.440 --> 00:12:47.679 nerd on about and just see things moving around and 246 00:12:47.720 --> 00:12:50.879 having your containers shift around different notes. It's cool, it's 247 00:12:50.879 --> 00:12:55.000 fun to play with. Here's a hot take. It created 248 00:12:55.080 --> 00:12:58.840 job security in a sense, so people are kind of motivated. 249 00:12:59.000 --> 00:13:02.600 You mentioned home labs, right, both of you people were deployed. 250 00:13:02.639 --> 00:13:05.039 Who would in their right mind with deploy Kubernets in 251 00:13:05.080 --> 00:13:07.600 their home lab. It's just it's it's so many layers 252 00:13:07.600 --> 00:13:10.480 of complexity. You can solve it with a fifteen other 253 00:13:10.480 --> 00:13:13.600 open source project, but you're going to kubernatis because you 254 00:13:14.000 --> 00:13:16.679 already know the beast and you feel like you've tamed this. 255 00:13:16.759 --> 00:13:19.759 So now you're going to deploy everywhere, including your own home, 256 00:13:20.960 --> 00:13:23.360 and that's how you want to see things progress. Because 257 00:13:23.480 --> 00:13:27.080 you moved people would literally only only answer to recruiting 258 00:13:27.080 --> 00:13:30.399 emails that have Kubernetis in this job description, right, So 259 00:13:30.639 --> 00:13:32.639 that's part of it, and I think it created like 260 00:13:32.720 --> 00:13:36.200 this ecosystem of companies are building for Kubernetes. People only 261 00:13:36.200 --> 00:13:38.879 want to work with Kubernetis. It's be they build a 262 00:13:39.000 --> 00:13:41.240 fomo around it. Everybody wants to be there. It's that 263 00:13:41.679 --> 00:13:45.840 somehow keeps being the next cool technology thing that always 264 00:13:45.879 --> 00:13:49.879 builds up and progresses for partially for a good reason. 265 00:13:50.679 --> 00:13:55.080 You ask what's good to use in your home lab. Well, 266 00:13:55.120 --> 00:13:57.840 you can use Kubernetes. There's k three S, which is 267 00:13:57.879 --> 00:14:03.240 the smaller sibling, the lighter weight thing Kubernetes. There's also 268 00:14:03.399 --> 00:14:07.399 we mentioned Fly Versail, all these commercial companies, there are 269 00:14:07.480 --> 00:14:10.320 open source alternatives to all of these, like Qualify is 270 00:14:10.360 --> 00:14:14.080 one that comes to mind. This is an open source Versail, 271 00:14:14.159 --> 00:14:16.519 slash Heroku, slash Fly that you can deploy in your 272 00:14:16.559 --> 00:14:19.720 home lab, which integrates you with anything. So if you 273 00:14:19.759 --> 00:14:22.519 wanted LMS, for example, you can. It has a list 274 00:14:22.559 --> 00:14:24.720 of you can make up your own plugins. It's just containers. 275 00:14:24.759 --> 00:14:28.080 But you can't choose an LLLM or speak to open 276 00:14:28.120 --> 00:14:32.480 AI from your application, or put your own LM there 277 00:14:32.519 --> 00:14:35.120 to run next to your application. And it works. 278 00:14:35.960 --> 00:14:41.240 So I'm curious about the LLLM aspect of it. You know, 279 00:14:41.720 --> 00:14:45.960 using lllms with Kubernetes, what kind of what do you 280 00:14:45.960 --> 00:14:47.039 actually gain from that? 281 00:14:47.480 --> 00:14:50.600 Right? So, I think there's a lot of use cases. 282 00:14:50.639 --> 00:14:53.360 The two main ones are either I'm an AI company 283 00:14:53.399 --> 00:14:56.159 and there's you know who is not an AI company 284 00:14:56.200 --> 00:14:57.039 these days. 285 00:14:56.799 --> 00:14:58.919 So great anyone who wants funding. 286 00:14:58.679 --> 00:15:03.600 Is every company's name has changed over the past twelve 287 00:15:03.639 --> 00:15:07.919 months to something AI. First of all, there are these 288 00:15:07.919 --> 00:15:11.639 companies that are either they own the LLM or they 289 00:15:12.519 --> 00:15:14.720 have to train it right, so they need a fleet 290 00:15:14.960 --> 00:15:19.600 of large and powerful machines to train their lms. They 291 00:15:19.639 --> 00:15:22.679 need big discs, YadA ya. And then the other part 292 00:15:22.720 --> 00:15:28.240 of it is these services are starting to cost a 293 00:15:28.279 --> 00:15:31.159 ton of money, especially if that's your core business. You're 294 00:15:31.200 --> 00:15:35.159 starting to pay like your cloud bill is now the 295 00:15:35.759 --> 00:15:39.320 second problem in the organization, not the first. So some 296 00:15:39.399 --> 00:15:42.200 companies find it that running your own l M or 297 00:15:42.240 --> 00:15:44.679 you know, there's a ton of open source lms. You 298 00:15:44.720 --> 00:15:46.519 can go to hugging face and get whatever you want, 299 00:15:46.799 --> 00:15:49.879 maybe lighter ones, smaller llms that can work even quicker, 300 00:15:49.919 --> 00:15:53.559 maybe tailor made to whatever you're doing, and then you 301 00:15:53.600 --> 00:15:58.240 can run it alongside your application, which means reduced latency 302 00:15:58.960 --> 00:16:02.879 not as high costs. But then this brings the complexity 303 00:16:02.879 --> 00:16:05.879 complexity story of infrastructure right because you have to right 304 00:16:05.960 --> 00:16:11.639 size things. Speaking about Kubernetes, Kubernets lets you dictate how 305 00:16:11.720 --> 00:16:14.440 much memory and CPU you're going to use. But once 306 00:16:14.519 --> 00:16:17.519 that set, and we'll talk about the last upgrade of kubernets. 307 00:16:17.519 --> 00:16:19.799 But once that's set, it's set. That's it. It's there. 308 00:16:20.919 --> 00:16:23.320 Over time, you probably want to change these requests and 309 00:16:23.360 --> 00:16:26.879 limits to fit whatever the application is doing. Either it's 310 00:16:26.919 --> 00:16:29.679 consuming too or it requested too much and it's not 311 00:16:29.720 --> 00:16:32.879 actually utilizing everything, or it needs more and now there's 312 00:16:33.080 --> 00:16:36.159 no more memory to serve, so you want to change 313 00:16:36.159 --> 00:16:38.320 these things. There's a few ways to do that, but 314 00:16:38.360 --> 00:16:41.799 it's really really hard to do automatically. That's one aspect 315 00:16:41.799 --> 00:16:46.840 of things. The other is these llms are usually just large, 316 00:16:47.240 --> 00:16:50.000 large language models. They also consume They consume a lot 317 00:16:50.039 --> 00:16:54.240 of disk, and with disk space it's exactly the same. 318 00:16:54.240 --> 00:16:59.559 You can create a PVC on kubernets, but once it's there, 319 00:16:59.679 --> 00:17:02.879 it's really hard to change. You can extend it. Sometimes 320 00:17:02.960 --> 00:17:05.079 if you're working with the right cloud provider, the right 321 00:17:05.119 --> 00:17:07.920 c sign. I'm not sure if you're aware. Kubernet Is 322 00:17:07.960 --> 00:17:10.200 one point three to three was just released a week 323 00:17:10.279 --> 00:17:12.119 or two weeks ago, and you can now go to 324 00:17:12.160 --> 00:17:14.640 your requests on a specific pod and change them and 325 00:17:14.720 --> 00:17:16.640 you don't have to restart the pod. They can just 326 00:17:16.759 --> 00:17:20.000 change and it will inflate inside the node to whatever 327 00:17:20.039 --> 00:17:22.039 it needs to consume, which is a really really cool 328 00:17:22.119 --> 00:17:24.319 change and that helps us a lot. You don't have 329 00:17:24.359 --> 00:17:27.039 to restart pods anymore if you want to scale up 330 00:17:27.160 --> 00:17:30.000 the resources that you consume. So that's a really big 331 00:17:30.079 --> 00:17:33.279 release and that's only been out for a week or two. 332 00:17:33.720 --> 00:17:35.599 I mean, I hear things like that and I think, wait, 333 00:17:35.880 --> 00:17:38.240 why wasn't it doing that all along? That seems like 334 00:17:38.279 --> 00:17:41.880 a required fundamental piece of the infrastructure. And you mentioned 335 00:17:41.880 --> 00:17:45.119 earlier the APIs have been figured out. You build a 336 00:17:45.119 --> 00:17:47.359 lot of things around the APIs to make adjustment. So 337 00:17:47.440 --> 00:17:49.640 my question is, like you actually like the api is 338 00:17:49.680 --> 00:17:52.440 that Kubernetes provides, because if you're building operators all the 339 00:17:52.440 --> 00:17:55.440 time to sort of adjust and change the abstraction layer 340 00:17:55.440 --> 00:18:01.200 to interact with your provider containerization level schedule, or your orchestrator, 341 00:18:01.400 --> 00:18:04.200 then I feel like you're getting closer to exactly the 342 00:18:04.279 --> 00:18:07.079 promise that Servilus has been offering all along. 343 00:18:07.960 --> 00:18:12.640 That's a philosophical question right there. Oh, definitely, that's what 344 00:18:12.680 --> 00:18:16.079 you're trying to do, right You're trying to abstract things 345 00:18:16.440 --> 00:18:20.000 by building them yourself because you don't want to mess 346 00:18:20.039 --> 00:18:22.440 it with infra, but you want to control it, which 347 00:18:22.480 --> 00:18:26.400 is kind of the conflict we all have because sometimes 348 00:18:26.519 --> 00:18:28.720 a lot of times we need the control. Right, we 349 00:18:28.799 --> 00:18:31.079 as a company have to have access to the notes, 350 00:18:31.079 --> 00:18:33.880 so we can't work with serverllus. We literally have to 351 00:18:33.880 --> 00:18:37.160 make changes on Linux. By the way, the recent change 352 00:18:37.480 --> 00:18:41.319 why would why can't you change a container? A running container? 353 00:18:41.319 --> 00:18:43.680 Why can't you change? The resource had to be developed 354 00:18:44.359 --> 00:18:47.240 in order for us to get the functionality. So we're 355 00:18:47.279 --> 00:18:50.920 always trying to make changes so that it's easier to 356 00:18:50.960 --> 00:18:54.480 deploy things. Production is more stable, our life are easier. 357 00:18:54.759 --> 00:18:57.599 But we still want the overall control, and there's still 358 00:18:57.599 --> 00:19:01.000 the ability to make changes to APIs and new operators 359 00:19:01.000 --> 00:19:05.039 and you know, change this beast. However we want to 360 00:19:05.079 --> 00:19:05.960 sell new products. 361 00:19:06.200 --> 00:19:08.480 I'm gonna keep going there, Like, how many companies actually 362 00:19:08.559 --> 00:19:09.759 need this level of control? 363 00:19:10.400 --> 00:19:13.240 It depends what you're calling this level. I mean, not 364 00:19:13.359 --> 00:19:18.599 everyone needs access to the noes, but some do in 365 00:19:18.640 --> 00:19:23.200 a way, not every You mentioned the scheduler earlier, By 366 00:19:23.240 --> 00:19:25.480 the way, big thing now in Kubernetis, don't know if 367 00:19:25.519 --> 00:19:28.359 you know, you can't really access the scheduler, which is 368 00:19:28.440 --> 00:19:31.759 just another open source component within kubernets. You cannot use 369 00:19:31.759 --> 00:19:34.759 it in most cloud providers as it was intended to. 370 00:19:35.200 --> 00:19:39.240 Meaning since when you're using Kubernettis through a cloud provider 371 00:19:39.279 --> 00:19:43.759 again Aws, Azure, GCP, or any other flavor, the scheduler 372 00:19:43.799 --> 00:19:46.519 is part of the control plane, you cannot access it. 373 00:19:46.599 --> 00:19:48.799 You can use it if you're instantiated a new pod 374 00:19:48.839 --> 00:19:51.000 the scheduler, we'll take that pod and then we'll schedule 375 00:19:51.039 --> 00:19:54.440 it wherever is available. But if you want to extend 376 00:19:54.480 --> 00:19:56.839 the scheduler, which is something you can do in kubernets, 377 00:19:56.880 --> 00:19:59.480 for example, you can plug in your own extender and 378 00:20:00.039 --> 00:20:05.640 create custom logic scheduling, you can't do it. So that's 379 00:20:05.680 --> 00:20:10.400 another place where Kubernetis was built to serve everyone so 380 00:20:10.440 --> 00:20:12.920 that they can do anything. But then the cloud providers 381 00:20:13.000 --> 00:20:15.680 take bits of it and say, okay, no, no, no, that's serverless. 382 00:20:15.680 --> 00:20:17.400 Now you're not touching that. We take care of that, 383 00:20:17.759 --> 00:20:20.839 which is okay until it's not. I don't know. It's 384 00:20:20.880 --> 00:20:23.720 the ever going conflict of who manages what and who 385 00:20:23.720 --> 00:20:26.720 has access to it. By the way, AWS can run 386 00:20:27.000 --> 00:20:30.640 you know, fargate on AWS. Probably other services other cloud 387 00:20:30.640 --> 00:20:33.839 providers have the same, But fargate you can run Kubernetes 388 00:20:34.119 --> 00:20:38.359 without nodes. I think to some extent it works. But 389 00:20:38.400 --> 00:20:41.119 what happens when you do need the access I don't know. 390 00:20:41.720 --> 00:20:43.720 I think most most companies actually don't use it. 391 00:20:44.119 --> 00:20:46.279 Oh, I mean, so fargate it's sort of a special 392 00:20:46.319 --> 00:20:50.720 part of AWS that gives you the serveralless aspects of 393 00:20:51.000 --> 00:20:54.400 container management without having to go deep into understanding the 394 00:20:54.400 --> 00:20:59.000 complexities of the node management or scheduling. And it was 395 00:20:59.200 --> 00:21:00.559 like I'm going to say it was recent, but it 396 00:21:00.599 --> 00:21:04.119 may have been three years ago still that you couldn't 397 00:21:05.119 --> 00:21:10.000 actually run fargate effectively with an EKS on AWS due 398 00:21:10.000 --> 00:21:12.119 to some of the limitations that came along with it. 399 00:21:12.119 --> 00:21:13.960 Like you could do it, but then for whatever reason, 400 00:21:13.960 --> 00:21:16.599 you wouldn't be able to get Internet access or i 401 00:21:16.680 --> 00:21:19.799