1 00:00:14,279 --> 00:00:18,960 Hey, what's going on everybody? This is another episode of Adventures and DevOps 2 00:00:19,000 --> 00:00:24,199 and I'm your host for today. Will Button joining me today, I have 3 00:00:24,399 --> 00:00:32,320 Corey O'Daniel, CEO and co founder of mass Driver, but also from Let's 4 00:00:32,320 --> 00:00:36,520 See. You're a principal software architect at the Real Real Cloud Solution, architects 5 00:00:36,520 --> 00:00:41,960 at Container Heroes prior to that, staff engineer at Click Trips, and most 6 00:00:42,000 --> 00:00:47,439 importantly, taco afficionado. So two things jumped on my mind right away. 7 00:00:48,000 --> 00:00:51,439 Number one, I need to know what your definition of the perfect taco is. 8 00:00:51,479 --> 00:00:54,280 But also I want to talk about your background because it looks very similar 9 00:00:54,320 --> 00:00:58,520 to mine in terms of the amount of time you spend at each company, 10 00:00:58,560 --> 00:01:02,119 and I think that's an important thing to share with everyone, especially people who 11 00:01:02,159 --> 00:01:06,760 are just starting their careers. But we'll get to that meanwhile, Corey, 12 00:01:06,799 --> 00:01:08,879 thank you for joining me on the show today. Yeah, thanks for having 13 00:01:08,920 --> 00:01:15,439 me and opening with a taco question. Oh okay. I think my favorite 14 00:01:15,599 --> 00:01:19,319 taco, like my go to is just like a classic like Carne Sato Street 15 00:01:19,319 --> 00:01:23,959 taco like that is my happy place and I've found the place in La that's 16 00:01:25,000 --> 00:01:29,040 like one of the better ones I've had in my life. So that's that's 17 00:01:29,159 --> 00:01:32,439 very convenient. It's a place called Cacau, Mexico, Testa, and the 18 00:01:32,439 --> 00:01:34,840 thing is how you pronounced the second word in Eagle Rock. It's amazing, 19 00:01:34,920 --> 00:01:42,359 and their molat is delicious right on. I would I would be I would 20 00:01:42,400 --> 00:01:45,439 be really really sad if you told me that you were living in LA and 21 00:01:45,519 --> 00:01:49,640 couldn't find a good taco spot. I mean the thing that it's like I 22 00:01:49,680 --> 00:01:53,640 always feel I mean, of course La is to have a can taco spot, 23 00:01:53,719 --> 00:01:57,840 right, But like I've I lived in Central America, Mexico under okay, 24 00:01:59,200 --> 00:02:00,799 the place were all So I feel bad saying that. But I mean 25 00:02:00,879 --> 00:02:05,159 like there's like this like interesting thing that happens in La. There's a whole 26 00:02:05,200 --> 00:02:09,080 bunch of people here, like like California is like what thirty plus percent Hispanic, 27 00:02:09,199 --> 00:02:12,639 right, and so like there's a whole bunch of people here, and 28 00:02:12,680 --> 00:02:15,840 then all the rich people are bougie and expect the best, so like all 29 00:02:15,960 --> 00:02:19,919 food has to be fantastic, right, So like I feel like those two 30 00:02:19,919 --> 00:02:23,800 things combined for like just it's like like La is like the best food mall, 31 00:02:24,080 --> 00:02:30,639 like everything's delicious. So I say that and I always feel bad. 32 00:02:30,719 --> 00:02:32,280 So I'm like, I've had some really great tacos in Mexico City, but 33 00:02:32,400 --> 00:02:38,120 like one of the convenience of driving tenants definitely impacts it. Oh for sure, 34 00:02:38,400 --> 00:02:46,080 absolutely, yeah. Yeah. Cool. So let's talk about your your 35 00:02:46,199 --> 00:02:47,919 job history real quick, and I'm going to be real clear, I'm not 36 00:02:47,919 --> 00:02:51,800 trying to call you out on it, because, like I mentioned mine, 37 00:02:51,960 --> 00:02:55,960 it looks very similar where a lot of your your positions you were there for 38 00:02:57,360 --> 00:03:00,400 you know, a year and a half, two years, three years, 39 00:03:00,840 --> 00:03:05,960 and I, you know, like I'm older and like my parents were, 40 00:03:06,000 --> 00:03:08,120 like, you get a job and you work there for forty years and your 41 00:03:08,120 --> 00:03:13,360 golden and I've never held a job for more than three years, and I 42 00:03:13,400 --> 00:03:20,719 think that's just I think part of that is just the new norm because after 43 00:03:20,759 --> 00:03:24,439 three years, Like the way I approach it is that if after three years 44 00:03:24,479 --> 00:03:29,520 of working with you, I haven't accomplished what you hired me to do, 45 00:03:29,840 --> 00:03:32,919 then I have failed. What's your take on that. I've got a couple 46 00:03:32,919 --> 00:03:37,159 of takes on this, but a lot of hot takes, So apologies and 47 00:03:37,199 --> 00:03:42,719 advance to anybody, but uh, I think one we have a lot If 48 00:03:42,719 --> 00:03:46,199 you think about our job is software engineers. Can you just classified kind of 49 00:03:46,240 --> 00:03:50,199 all of us under the software umbrella, whether you don't know. So some 50 00:03:50,199 --> 00:03:53,680 people don't feel like software engineers because they're like DevOps engineers and themal engineers, 51 00:03:53,680 --> 00:03:55,680 but you are, like there's a different abstraction layer, different level, like 52 00:03:55,719 --> 00:03:59,599 different expertise. Like if you think about like us in the grand scheme of 53 00:03:59,599 --> 00:04:03,800 things, we're an interesting group of people where our job is being a really 54 00:04:03,840 --> 00:04:09,000 good tool for the most part, right, Like if you think about where 55 00:04:09,039 --> 00:04:12,560 software started, where it was people that were professionals and they use software to 56 00:04:12,599 --> 00:04:16,079 do their job. Right, slowly software development became its own profession, right, 57 00:04:16,160 --> 00:04:24,680 So I see what we do a lot of time is like like like 58 00:04:24,839 --> 00:04:28,680 what's the best way of phrasing it? Like we are brought in to help 59 00:04:28,720 --> 00:04:32,040 business people code five things, right, Like that's really what we're doing. 60 00:04:32,120 --> 00:04:36,040 There's people that have an idea of how to make money and they don't know 61 00:04:36,040 --> 00:04:40,040 how to do the work that we do, right, and so we're not 62 00:04:40,120 --> 00:04:45,920 always invested in the business idea necessarily that we're writing software for, right, 63 00:04:45,920 --> 00:04:48,959 And like you can see this is plenty of companies, Like I know one 64 00:04:48,959 --> 00:04:51,519 of the companies I work on very early it's not on my resumes because I 65 00:04:51,600 --> 00:04:57,480 was a consultant at the time was honest company for sending baby diapers to your 66 00:04:57,480 --> 00:05:01,160 house. There was a bunch of single men there that built the software. 67 00:05:01,319 --> 00:05:03,680 None of them were like, wow, I can't wait to get some diapers 68 00:05:03,720 --> 00:05:09,160 doing that, Like that's the first thing. Is like like you can love 69 00:05:09,240 --> 00:05:12,959 what you do but not necessarily love the thing that you're building, right, 70 00:05:13,040 --> 00:05:15,360 and that has interesting trade offs, like you might get to the point where 71 00:05:15,360 --> 00:05:20,759 it's like, okay, like all the ideation and collaboration with the business book, 72 00:05:20,759 --> 00:05:24,639 which is kind of like what I really like about software is like talking 73 00:05:24,639 --> 00:05:28,040 through the ideas and figuring out like how to make that code, not necessarily 74 00:05:28,079 --> 00:05:30,160 typing it. You to a point where it's like, Okay, we've done 75 00:05:30,160 --> 00:05:33,439 the ideation, We've done the interesting things. Like I feel like I'm really 76 00:05:33,439 --> 00:05:38,639 in a maintenance mode. I don't love the business, so it's like, 77 00:05:38,680 --> 00:05:44,439 okay, well what is here for me? Right? And a lot of 78 00:05:44,839 --> 00:05:46,519 a lot of careers we see people working ten, twelve, thirteen, fourteen 79 00:05:46,560 --> 00:05:49,839 years, they don't have that choice. I'm a lawyer. If I don't 80 00:05:49,839 --> 00:05:54,279 like being a lawyer anymore, I'm kind of shit out of lot. Right, 81 00:05:54,720 --> 00:05:57,839 and so I think that's one part of it. But we're also we're 82 00:05:57,879 --> 00:06:00,160 all hobbyists, right, Like we all it's oh, it's five o'clock, 83 00:06:00,160 --> 00:06:02,079 it's time to get off work. What am I going to do? I'm 84 00:06:02,079 --> 00:06:04,839 going to continue to sit at my computer and write some open source software. 85 00:06:04,920 --> 00:06:08,800 I'm going to go play with something, right, Like we love to play 86 00:06:08,839 --> 00:06:12,720 and learn new things. I think like those two things, especially for me, 87 00:06:12,839 --> 00:06:15,639 like I have a almost like a wonderlust, like I have to be 88 00:06:15,959 --> 00:06:18,920 doing something more, doing something interesting, engaging myself when I get to a 89 00:06:18,959 --> 00:06:21,279 place where I'm like, okay, like there's just not really anything here that 90 00:06:21,480 --> 00:06:26,600 rewards me. And guess what, the businesses aren't either. The best way 91 00:06:26,879 --> 00:06:30,879 If there's any juniors listening to this, like, sorry, guys, here's 92 00:06:30,959 --> 00:06:33,439 here's the news. The best way to get a raise is to quit your 93 00:06:33,519 --> 00:06:39,759 job and go someplace else. Absolutely a three percent raise, which means you're 94 00:06:39,759 --> 00:06:43,240 losing like eight percent of your salary over the last couple of years with inflation. 95 00:06:43,519 --> 00:06:46,560 The best way to jump twenty percent is get a new job. And 96 00:06:46,040 --> 00:06:49,839 at the end of the day, like I like writing software, but you 97 00:06:49,879 --> 00:06:54,800 know what, I really like getting paid well, right, and so it's 98 00:06:54,800 --> 00:06:57,000 like if I got to the point where it's like there's something that I can 99 00:06:57,360 --> 00:06:59,879 say I did, there's somebody at that company that I can use as a 100 00:07:00,000 --> 00:07:02,279 reference to, like Corey Rocked, and it's like, Okay, I've gotten 101 00:07:02,360 --> 00:07:06,319 like some like interesting raises and maybe I'm like in line with inflation, but 102 00:07:06,360 --> 00:07:09,959 I'm not I'm not making more money. It's like, what is the growth 103 00:07:09,959 --> 00:07:13,480 that I have here if I'm not invested in the business, if I'm not 104 00:07:13,560 --> 00:07:16,639 learning anything new about the software, and I'm not getting paid more money, 105 00:07:16,800 --> 00:07:20,759 Like I'm just degenerating in every way, right, Like, so it's time 106 00:07:20,800 --> 00:07:26,680 to go and what's really interesting? And well this doesn't offend anybody, But 107 00:07:26,720 --> 00:07:30,279 like when I see a resume and somebody's like I worked there for fifteen years, 108 00:07:30,319 --> 00:07:32,920 it's like I don't want to work with you, Like no things is 109 00:07:33,000 --> 00:07:36,519 like you you are so ingrained in like your Google you like I worked at 110 00:07:36,560 --> 00:07:39,879 Google for fifteen years. It's like, well, guess what, Like you 111 00:07:40,439 --> 00:07:45,759 didn't work in the real world, like you worked in like the perfect utopia 112 00:07:45,879 --> 00:07:49,920 of like building software around other software engineers for software engineers like that. Like 113 00:07:50,439 --> 00:07:54,360 so like seeing that person that's like they have enough tenure where you know that 114 00:07:54,360 --> 00:07:57,759 they weren't just getting fired from job to job. They had that experiences, 115 00:07:57,839 --> 00:08:01,639 different teams, different stacks, different products where they can bring a lot more 116 00:08:01,680 --> 00:08:05,839 than just being able to type. Like that's that's kind of the value I'm 117 00:08:05,879 --> 00:08:07,399 looking for. And I'm looking at a resume. I'm terrified if somebody just 118 00:08:07,439 --> 00:08:11,879 sat someplace forever, he just work on like the banner of Amazon for eight 119 00:08:11,959 --> 00:08:15,399 years, Like, what did you do? Right? So I know there's 120 00:08:15,399 --> 00:08:20,879 probably some really great time. Orry. I didn't mean yeah, I hear 121 00:08:20,920 --> 00:08:26,319 you though. That's me in a nutshell. Welcome high everyone right now. 122 00:08:26,480 --> 00:08:31,399 I had a boss that I worked for a long time ago, and I 123 00:08:31,399 --> 00:08:35,960 I interviewed for another job and he found out about it was a smaller town, 124 00:08:37,000 --> 00:08:39,759 and you know, I was terrified that he was just going to fire 125 00:08:39,799 --> 00:08:43,480 me on the spot. And he said, well, did you get the 126 00:08:43,600 --> 00:08:46,720 job? And I said, well, they're waiting, are you? Are 127 00:08:46,720 --> 00:08:50,919 you mad? He said no. He said, if if no one else 128 00:08:50,960 --> 00:08:56,240 wants to hire you, why would I want you working for me? Damn, 129 00:08:56,279 --> 00:09:00,440 that's pretty savage. The funny thing is, I like, I've seen 130 00:09:00,480 --> 00:09:05,000 this. The number of times is like a hiring manager, where like you'll 131 00:09:05,039 --> 00:09:07,559 make an offer to somebody, right they come in the interview, go al, 132 00:09:07,679 --> 00:09:09,200 I really like that person, Like I interviewed one hundred people or whatever, 133 00:09:09,279 --> 00:09:11,840 and that was the one. I'm like, that's my person, and 134 00:09:11,879 --> 00:09:16,600 they're like my company counter and I'm like, yeah, if it took you 135 00:09:16,799 --> 00:09:22,120 leaving for your company to recognize your value, then you accept that, like 136 00:09:22,879 --> 00:09:26,080 I don't, I don't know, Like I mean, you take it, 137 00:09:26,240 --> 00:09:31,440 definitely take it because it's more money, but you should immediately reshot that out 138 00:09:31,480 --> 00:09:35,240 and find something better, Like take the money while you're there, and then 139 00:09:35,399 --> 00:09:39,000 go find the person that appreciates you for who you are and wants to give 140 00:09:39,039 --> 00:09:41,320 you a good salary, not just because you're gonna walk right. Yeah. 141 00:09:41,360 --> 00:09:46,919 And the counterpoint to that is if you do stay with them, Like all 142 00:09:48,000 --> 00:09:52,360 companies go through down cycles, and whenever they hit that next downcycle, they've 143 00:09:52,399 --> 00:09:58,279 already set precedent that they're only going to do the bare minimum to keep you 144 00:09:58,399 --> 00:10:03,080 around, and so whenever they have to lay off people, they're likely to 145 00:10:03,120 --> 00:10:07,519 think, well, hey, this person has already been trying to leave anyway, 146 00:10:07,679 --> 00:10:09,840 so they need to be first on the top and block to go yeah, 147 00:10:09,960 --> 00:10:13,759 especially if like you've like if that salary is no longer convincerate with the 148 00:10:13,840 --> 00:10:18,919 value that you have. You just got a twenty percent raise because you were 149 00:10:18,919 --> 00:10:20,200 going to quit. Like now you're the biggest line item on the team, 150 00:10:20,279 --> 00:10:28,720 right, But now I do think of this analogy just hit me. I 151 00:10:28,759 --> 00:10:33,080 kind of feel like what we do are like the wagon train masters of the 152 00:10:33,159 --> 00:10:39,720 Old West, Like our job is to get the settlers and their wagons out 153 00:10:39,799 --> 00:10:43,480 west, and once we get them out west, we're done. Like you 154 00:10:43,559 --> 00:10:48,639 never hire a wagon master and say, well, we're going to use you 155 00:10:48,840 --> 00:10:54,360 to escort our wagon train for the next fifteen years. No, get us 156 00:10:54,360 --> 00:10:56,639 to this point and then go away. Yeah, it's kind of like it 157 00:10:56,720 --> 00:11:01,360 suffers, like we hump if you wanting eighteen eighty three, is that where 158 00:11:01,360 --> 00:11:05,159 this came from? Where you're just like a fore Gun trail fan. It's 159 00:11:05,240 --> 00:11:09,480 just a side quest finished eighteen eighty three is so good? Yeah, No, 160 00:11:09,600 --> 00:11:11,440 it's funny. It's like, yeah, nobody's ever like, yeah, 161 00:11:11,440 --> 00:11:15,320 we'll get out the Oregon. We'll keep the wagging guy around in case we 162 00:11:15,360 --> 00:11:18,240 want to go to like Montana or Alabama or something. Maybe we'll want to 163 00:11:18,279 --> 00:11:20,039 go back to hudos, right, and like that's what you're just like, 164 00:11:20,120 --> 00:11:22,720 Okay, well I'm here until like we have another direction we want to go, 165 00:11:22,840 --> 00:11:26,240 we to change all this stuff. Yeah, that's an interesting one them. 166 00:11:26,279 --> 00:11:28,840 I was talking to somebody the other day, a buddy of mine that's 167 00:11:28,879 --> 00:11:33,480 working on an AI code product, and I will call them out because I 168 00:11:33,519 --> 00:11:37,440 think the big disagreements on it. But like this whole notion of like we've 169 00:11:37,440 --> 00:11:39,720 had this like for like twenty five years for anybody who's newer to the industry. 170 00:11:39,759 --> 00:11:43,360 But like when Macromedia dream Weaver came out, I remember people being like, 171 00:11:43,720 --> 00:11:46,679 we'll never be like business people are gonna build all the web pages and 172 00:11:46,720 --> 00:11:52,120 it's like, ah no they aren't. But like as like as right, 173 00:11:52,159 --> 00:11:56,480 because at the end of the day, like we are tools for business, 174 00:11:56,600 --> 00:11:58,679 right, Like so this idea of like no code and AI it can help 175 00:11:58,679 --> 00:12:03,399 this is people like build software Like that is a great idea, and like 176 00:12:03,639 --> 00:12:07,440 I'm sure that that will happen eventually, But in the meantime, us being 177 00:12:07,480 --> 00:12:13,360 relegated to being code reviewers for a machine like that feels like the most miserable 178 00:12:13,360 --> 00:12:18,519 intersection of how this did go right. It's like the creativity part, and 179 00:12:18,559 --> 00:12:22,919 like thinking of how to like abstract like business logic like business needs and logic 180 00:12:22,960 --> 00:12:24,679 into code is part of the fun part. And it's like, that's just 181 00:12:24,720 --> 00:12:28,960 a computer and I'm just like, ah, I don't like the way you've 182 00:12:28,000 --> 00:12:33,279 spelled that variable, Like that's a that's a sucky life for software engineers. 183 00:12:33,399 --> 00:12:41,600 So but yeah, wagons AI, it's all the same. That could that 184 00:12:41,639 --> 00:12:50,000 could be our new startup wagon train dot a the bafest path to organ that 185 00:12:50,159 --> 00:12:56,759 ended up. But there is there is like one of the things you mentioned 186 00:12:56,840 --> 00:13:01,480 earlier, you know about the the alignment of you as an employee versus the 187 00:13:01,519 --> 00:13:07,639 company. Like the founders of the company are vested in the product. Me 188 00:13:07,840 --> 00:13:15,120 as an employee, I'm vested in providing a really good living for my family, 189 00:13:15,159 --> 00:13:16,679 putting food on a table, putting a roof over their head, giving 190 00:13:16,720 --> 00:13:20,240 them the things that make them feel like they're having a quality life. And 191 00:13:22,240 --> 00:13:28,519 so we have different goals there. But jumping around like we do also is 192 00:13:28,559 --> 00:13:33,440 a benefit to those companies because those people, like the diaper company, they're 193 00:13:33,600 --> 00:13:41,720 really passionate about how to get diapers to young families. But if I've worked 194 00:13:41,720 --> 00:13:48,519 for a bunch of different industries. I can bring perspective to them and help 195 00:13:48,559 --> 00:13:54,639 them build a product that meets that need better because I know the roadblocks that 196 00:13:54,679 --> 00:14:00,799 they aren't familiar with and can't see because they're focused on that the diapers on 197 00:14:00,799 --> 00:14:03,720 the doorstep thing. Yeah, and it's interesting because, like you know, 198 00:14:03,120 --> 00:14:07,440 in those like as a company grows, especially from startup to so you say 199 00:14:07,519 --> 00:14:11,440 series V and beyond, like the product also changes, right, So, 200 00:14:11,559 --> 00:14:16,960 like one of the things that might bring you into maybe new e commerce companies, 201 00:14:16,000 --> 00:14:18,919 the person had experienced a previously e commerce compani It's like, oh, 202 00:14:18,960 --> 00:14:24,279 we needed somebody who knows inventory systems and you know, billing and all that 203 00:14:24,320 --> 00:14:26,399 stuff. Like, let's bring that person in. But then as like this 204 00:14:26,480 --> 00:14:30,759 company evolved, let's say three years, we're doing like a lot of video 205 00:14:30,799 --> 00:14:33,240 options. It's like, why don't necessarily need like this person's deep knowledge and 206 00:14:33,840 --> 00:14:37,440 e commerce systems anymore? I know they need somebody to news how to do 207 00:14:37,919 --> 00:14:41,480 concurrent video streams to hundreds of thousands of people that might be bidding on this 208 00:14:41,600 --> 00:14:43,600 item, right, and so like our business has changed, the people that 209 00:14:43,639 --> 00:14:46,600 we need involved change, and sure like that person that's been there for three 210 00:14:46,639 --> 00:14:50,639 or four years could go learn how to do like video streaming and you know 211 00:14:50,679 --> 00:14:54,200 all that stuff. But it's like, also you could go Hoch somebody from 212 00:14:54,360 --> 00:14:58,080 you know, Prime or something like that. Right, So I think like 213 00:14:58,720 --> 00:15:03,639 that moving, I think is what makes us really good tools. Again, 214 00:15:03,720 --> 00:15:07,919 like if you sit in this narrow channel for fifteen years doing one thing for 215 00:15:07,000 --> 00:15:11,679 one product with one team, that may have people swapping in and out, 216 00:15:11,759 --> 00:15:15,240 like you were very uniform, right, and so like now when you're coming 217 00:15:15,279 --> 00:15:18,000 into another environment, I feel like it's harder to figure out like how to 218 00:15:18,799 --> 00:15:22,559 how to necessarily fit you in, right. I feel like it was kind 219 00:15:22,559 --> 00:15:24,759 of like limits where you I mean, I guess when you have the prestige 220 00:15:24,759 --> 00:15:26,120 of a big band company can kind of go wherever you want. But like 221 00:15:26,159 --> 00:15:30,559 if you were like, oh, that person was at Macy's for seventeen years, 222 00:15:30,600 --> 00:15:33,519 you know like, well we got to get them in here, right, 223 00:15:33,960 --> 00:15:35,840 But it's someways like that it was a Google for seventeen years, You're 224 00:15:35,840 --> 00:15:37,879 like, oh, how much do how much money we have to live in 225 00:15:37,879 --> 00:15:41,279 a bank. Let's give it to this person, give them the stock. 226 00:15:41,639 --> 00:15:52,039 Right. So when we were chatting over email before the podcast, you mentioned 227 00:15:52,080 --> 00:15:58,279 that DevOps is pure bullshit. So I don't want to care about this. 228 00:15:58,480 --> 00:16:02,679 Yeah, I'm gonna call you out on it. This is fine. I 229 00:16:02,720 --> 00:16:03,919 love talking about this, so, like I have to say that, this 230 00:16:04,000 --> 00:16:07,399 is one of those moments where, gosh, you can't remember who said the 231 00:16:07,480 --> 00:16:10,879 quote, like being able to hold two opposing ideas in your head at the 232 00:16:10,919 --> 00:16:15,600 same time. Who was that? Uh huh. I'm not really good at 233 00:16:15,679 --> 00:16:18,879 quotes, and this is live, so I can't google it. But ultimately 234 00:16:18,639 --> 00:16:22,039 it's like a definission of being crazy. So I was like mesuring Franklin or 235 00:16:22,080 --> 00:16:23,799 something. It's like, oh, that's a sign of intelligence. It's like 236 00:16:23,840 --> 00:16:29,440 also inside of being nuts. But it's really I think it was it. 237 00:16:30,320 --> 00:16:33,240 It was one of these much smarter and successful to me said it, so 238 00:16:33,440 --> 00:16:40,000 you got to but it it is and it isn't, And I think it's 239 00:16:40,000 --> 00:16:44,919 one of those things that's interesting interesting. It's like death ops is what it 240 00:16:44,960 --> 00:16:47,120 is based on, Like how much your zoom dinner is zoomed? Out? 241 00:16:47,200 --> 00:16:52,720 Right? You could watch somebody who's building an application. Let's say it's twenty 242 00:16:52,799 --> 00:16:57,639 thirteen and somebody's building a Ruby full stack app monolith and they deploy it on 243 00:16:57,759 --> 00:17:03,720 Heroku. They are they doing DevOps, I would argue, yeah, they 244 00:17:03,720 --> 00:17:06,240 are. They're they're operating their software happens to be running on a platform. 245 00:17:06,279 --> 00:17:10,200 It's abstracted a lot away from it, but they're operating at the abstraction that 246 00:17:10,200 --> 00:17:14,240 they've been given, right Versus Okay, I'm an engineer. I'm sitting around 247 00:17:14,279 --> 00:17:18,680 three hundred micro services. I'm configuring the Kubernators cluster and all the service meshes. 248 00:17:18,759 --> 00:17:22,200 Like am I doing DevOps? Like yeah, you are too, You're 249 00:17:22,200 --> 00:17:25,440 doing like the dev and the ops like sure, So like it can exist 250 00:17:25,440 --> 00:17:29,160 at different levels and you can create abstractions where people can still do DevOps. 251 00:17:29,160 --> 00:17:33,160 And I think that where we are today and how we like handwave a lot 252 00:17:33,160 --> 00:17:36,839 of the definition of DevOps. Like there's people that are in like it's a 253 00:17:36,880 --> 00:17:40,160 team camp. There's people like it's a collaboration camp. And there's the people 254 00:17:40,160 --> 00:17:42,000 that are like you do everything, like you build it, you run it, 255 00:17:42,039 --> 00:17:45,279 like you have to do it all. Ah, It's like okay, 256 00:17:45,319 --> 00:17:49,319 well, like we kind of like three definitions so like that right, there's 257 00:17:49,319 --> 00:17:53,200 a definite bullshit Like it gave defines, right, but like I have three 258 00:17:53,279 --> 00:18:00,519 of those like don't happen for most people, And I have proof. We 259 00:18:00,640 --> 00:18:04,240 love the Dora report and DevOps, but when you go through it, fifty 260 00:18:04,279 --> 00:18:08,640 percent of people that respond to the Door Reporter like everything's fucked here and it's 261 00:18:08,640 --> 00:18:12,200 just like numbers, like it takes us we really software once a quarter and 262 00:18:12,240 --> 00:18:15,640 it's like okay, and it's like an outage lasts up to two days, 263 00:18:15,640 --> 00:18:21,200 Like fifty percent of the Door report respondents report pure chaos. So like while 264 00:18:21,400 --> 00:18:23,559 walk like we're on Hacker News, We're like, haha, at my company, 265 00:18:23,559 --> 00:18:26,519 we do DevOps great. And it's like, okay, you're a soul 266 00:18:26,640 --> 00:18:30,559 CTO who deploys on verse cell, Like what you're doing it right, like 267 00:18:30,839 --> 00:18:33,160 your hands in fine, And then people will be like, we do devlops 268 00:18:33,160 --> 00:18:37,839 great. We have eighty thousand micro services like you also have ten thousand ops 269 00:18:37,839 --> 00:18:41,920 people who've built an internal platform that makes it where you can do a DevOps 270 00:18:41,920 --> 00:18:44,960 with the abstraction it's given to you, right, And so I feel like 271 00:18:45,839 --> 00:18:49,279 I feel like like it is plausible. It is an idea that makes sense, 272 00:18:49,319 --> 00:18:53,480 and you do want your developers to have a baseline understanding of the abstraction 273 00:18:53,599 --> 00:18:57,519 that they're given. But I think the bullshit is in like like like are 274 00:18:57,519 --> 00:19:00,359 we giving people the right abstraction, And I would argue, like, that's 275 00:19:00,359 --> 00:19:06,319 where it really gets super complicated, And the answer is also like we probably 276 00:19:06,400 --> 00:19:11,920 aren't right. So what's interesting is looking at another report. State of CD 277 00:19:12,000 --> 00:19:15,440 report is another interesting one. The most recent state of CD report found that 278 00:19:15,519 --> 00:19:19,519 only twenty seven percent of companies are using I see infrastructure's code, right, 279 00:19:21,000 --> 00:19:26,039 and so we're all DevOps engineers. We automate everything, do we? If 280 00:19:26,079 --> 00:19:29,440 only twenty seven percent of companies are using infrastructure as code and I ventured, 281 00:19:29,440 --> 00:19:32,160 I bet most of them that marked yes and that aren't using it one hundred 282 00:19:32,160 --> 00:19:36,680 percent throughout the company. How are those other seventy three percent of companies that 283 00:19:36,720 --> 00:19:41,880 are doing DevOps automating stuff? If you weren't using antibal, chef care form 284 00:19:42,319 --> 00:19:48,400 or blooming, are you using Selenium to automate clicking through the AWS dashboard? 285 00:19:48,480 --> 00:19:52,119 Like I'm confused here because we're all doing DevOps, but you're not automating any 286 00:19:52,160 --> 00:19:56,079 of your infrastructure management. That is where a whole other chunk of bullshit comes 287 00:19:56,119 --> 00:20:00,480 in infrastructure management. Like some people consider it a part of DE but it 288 00:20:00,559 --> 00:20:03,960 didn't exist really, I mean it was like CF engine, but like there 289 00:20:04,039 --> 00:20:11,359 wasn't much infrastructure management or even cloud infrastructure when like the DevOps term was pointed, 290 00:20:11,519 --> 00:20:15,359 right, And so this whole idea of cloud infrastructure, which is this 291 00:20:15,440 --> 00:20:18,759 thing that ran our software that over the past ten years has slowly become our 292 00:20:18,799 --> 00:20:22,880 software, and it's going to get worse with AI and LLLM models, Like 293 00:20:23,119 --> 00:20:26,799 a lot of our software is no longer ours. Right in two thousand and 294 00:20:26,839 --> 00:20:30,960 eight, my code was my code. In twenty twelve, it was like, 295 00:20:30,000 --> 00:20:34,039 well, all my cues and background processes are actually done by this sqssn 296 00:20:34,119 --> 00:20:37,920 S thing, Like I've got a lot less code to manage I call people 297 00:20:37,920 --> 00:20:40,880 other people's APIs and now it's like step function in land. That's like my 298 00:20:40,920 --> 00:20:44,880 code's getting smaller and smaller and this cloud service area is growing. And then 299 00:20:44,920 --> 00:20:48,400 you look at llms and it's like I don't have any code. I just 300 00:20:48,440 --> 00:20:52,079 shove hashes in somebody else's database and magic comes out of it. Right, 301 00:20:52,119 --> 00:20:56,960 Like we're getting a lot less code, a lot more consumed cloud services. 302 00:20:56,279 --> 00:21:00,319 And I don't think we ever tradition like went back and like include infrastructure management 303 00:21:00,400 --> 00:21:04,920 like in that loom, right, And then when you do, whose responsibility 304 00:21:04,960 --> 00:21:10,359 is it? Because if you say, all of aws's the responsibility of all 305 00:21:10,400 --> 00:21:14,240 the services that you may or may not use in AWS is your engineer's responsibility. 306 00:21:14,319 --> 00:21:17,559 You're out of your fucking mind. Like their responsibility is to build business 307 00:21:17,680 --> 00:21:22,680 value if they have to go read the twenty seven pages of how to monitor, 308 00:21:23,599 --> 00:21:27,559 configure secure YadA YadA, YadA, YadA, YadA dada postgress. When 309 00:21:27,559 --> 00:21:30,759 it's like locally, I just like Docker run postgress, It's like that's the 310 00:21:30,839 --> 00:21:33,319 abstraction that they should see. They should say, I need postgresses, and 311 00:21:33,319 --> 00:21:37,160 they should be able to assume that whatever's happened on the other side, I've 312 00:21:37,160 --> 00:21:41,000 got secure compliance, scalable, cost effective thing. I shouldn't have to think 313 00:21:41,039 --> 00:21:45,440 about costs, and I shouldn't have to be sidelined on it three months later 314 00:21:45,440 --> 00:21:48,960 when the cfo's pissed off or we're spending thirty grand a month on a misconfigured 315 00:21:48,000 --> 00:21:53,559 post resistance. Oh sorry, So that's why I think it's bullshit. But 316 00:21:53,599 --> 00:21:56,839 here's the catch is, like I'm also one of these people. It's like, 317 00:21:56,880 --> 00:21:59,880 hey, platform engineering, you got to do this right, which I 318 00:22:00,079 --> 00:22:03,039 will admit platform engineering is a subset of DevOps, but the catches, like 319 00:22:03,200 --> 00:22:07,440 most people don't have the budget to actually do it or the talent to actually 320 00:22:07,440 --> 00:22:11,759 do it because their DevOps team or operations team is in drowning and technical debt, 321 00:22:12,160 --> 00:22:15,680 and like trying to ticket ops for their developers and people that just have 322 00:22:15,759 --> 00:22:21,359 developers and don't have an OPS team, they're probably better served by the past 323 00:22:21,480 --> 00:22:23,160 and going directly to the cloud until they get to the point that they need 324 00:22:23,160 --> 00:22:30,519 that scale. So that's me defending the DevOps as bullshit. So it was 325 00:22:30,599 --> 00:22:34,119 like a therapy session. Yeah, I'm gonna get your insurance details after the 326 00:22:34,160 --> 00:22:41,160 podcast here that. Yeah, so I look forward to comments after the show. 327 00:22:45,200 --> 00:22:51,079 No, I agree with you. And the irony here is I actually 328 00:22:51,160 --> 00:22:56,039 have the title at my current employer of DevOps Engineering, and I agree, 329 00:22:56,079 --> 00:23:00,240 like that should not be a thing. And I think part of it just 330 00:23:00,279 --> 00:23:07,200 become to in my perspective is we've we've we're just kind of like that's where 331 00:23:07,200 --> 00:23:11,039 we've drawn the line in the sand, like, oh, we're doing DevOps, 332 00:23:11,119 --> 00:23:12,799 damn it, no matter how hard it is. And it's kind of 333 00:23:12,880 --> 00:23:18,359 like, you know, being a forward guy or a Chevy guy. You 334 00:23:18,400 --> 00:23:21,200 know, if you if you buy a Chevy, you're a Chevy guy. 335 00:23:21,240 --> 00:23:23,359 Now you you know, it's like I can't buy a Ford, or they're 336 00:23:23,400 --> 00:23:26,200 going to kick me out of the line dancing club or whatever, you know, 337 00:23:26,200 --> 00:23:33,559 and we have this failure to adapt with the way that our our industry 338 00:23:33,759 --> 00:23:37,920 has changed. And you know, because you can't, like you mentioned, 339 00:23:37,960 --> 00:23:45,519 you can't ask a software engineer to be proficient in writing code and go, 340 00:23:45,720 --> 00:23:49,160 but you also have to be an expert in Postgress database and uh, an 341 00:23:49,200 --> 00:23:56,720 expert in Docker and kubernetties and security gateways and cost management. There's just so 342 00:23:56,839 --> 00:24:03,839 much context switching there that you'll never get anything better than mediocre across the board. 343 00:24:03,319 --> 00:24:10,559 Yeah, and it's funny because, like I feel like I might be 344 00:24:10,640 --> 00:24:15,279 wrong here, I probably am. Okay, Like I feel like two things 345 00:24:15,640 --> 00:24:18,480 have kind of got us into the mess that we're in with develops, right, 346 00:24:18,480 --> 00:24:21,720 because if you think about just like an engineer, like the amount of 347 00:24:21,759 --> 00:24:26,119 stuff that we will stack as a responsibility on engineers, right, why does 348 00:24:26,160 --> 00:24:27,400 this happen? Right? So? Yeah, I'm a software developer, right, 349 00:24:27,480 --> 00:24:30,400 software? Okay, I use a database. I picked postgrats. Okay, 350 00:24:30,400 --> 00:24:34,559 well this is an Oracle nine I it's Postgress. So I'm assuming that 351 00:24:34,559 --> 00:24:38,440 you're the database administrator. Also software developer, you're going to tune your indexes 352 00:24:38,920 --> 00:24:44,359 right like a decade then fifteen years ago, like we had DBAs, but 353 00:24:44,400 --> 00:24:48,720 like you're seeing them become less and less prevalent in businesses except for like massive, 354 00:24:48,759 --> 00:24:51,240 massive businesses because like, oh, it's like that person will figure it 355 00:24:51,279 --> 00:24:56,000 out, and then so that happens. We stack that responsibility on people, 356 00:24:56,559 --> 00:25:00,720 and I think it's twofold. I think one's business people. I don't understand 357 00:25:00,839 --> 00:25:04,559 like where the lines are. And we're again we're hobbyists. So it's like 358 00:25:04,599 --> 00:25:11,279 okay, like, hey, yay, will do you know how like LLM 359 00:25:11,359 --> 00:25:14,319 building works? And you're like no, And you're like, well, you've 360 00:25:14,319 --> 00:25:15,960 got a better chance of figure it out than me over here. It's working 361 00:25:17,000 --> 00:25:19,599 on the old it's right. And so you're like, if you give a 362 00:25:19,599 --> 00:25:23,160 software engineer like time to go discover something, they will do it. Now, 363 00:25:23,160 --> 00:25:26,720 will they become a master of it? No? Will they get something 364 00:25:26,759 --> 00:25:32,359 that works enough to go into production? Yes? And that's where our product 365 00:25:32,359 --> 00:25:33,960 managers are content, like they're not like, oh, Corey, did you 366 00:25:34,039 --> 00:25:38,519 master building elements before you've lobbed this thing into a kuberinetes pluster. No? 367 00:25:40,359 --> 00:25:42,519 But it works right, And that's all matters. Like I made the button 368 00:25:42,519 --> 00:25:45,920 click faster, I sold more of the things. Like at the end of 369 00:25:45,960 --> 00:25:48,759 the day, that's what matters to the business, not necessarily that it's perfectly 370 00:25:48,799 --> 00:25:53,000 secure. Do you care about that business? Do you care that it's perfectly 371 00:25:53,079 --> 00:25:56,920 secure? Or are you going to buy an insurance plan from Equifax to cover 372 00:25:57,000 --> 00:26:02,480 people's credit when it breaches. It's much much cheaper to buy the insurance plan 373 00:26:02,559 --> 00:26:06,720 than to spend the engineering resources to make it secure. Yeah. Yeah, 374 00:26:06,960 --> 00:26:10,880 as just ask my real estate and investment company that I'm working with, because 375 00:26:10,920 --> 00:26:12,559 I prompted them on that. I'm like, how much time you spill security? 376 00:26:12,680 --> 00:26:15,440 Are things perfectly secure? Well, then we do investing. There was 377 00:26:15,480 --> 00:26:18,640 a breach and like we leaked your social Security number and all your financials. 378 00:26:18,640 --> 00:26:23,000 I was like, I knew my credit's locked already. It's fun. But 379 00:26:23,039 --> 00:26:26,559 so I think, like, you know, that's that's it's really interesting, 380 00:26:26,559 --> 00:26:32,759 Like we'll just stack stuff on people. And I think that's kind of where 381 00:26:33,400 --> 00:26:36,000 it's easy for burnout to happen, right, Like you think you're going to 382 00:26:36,039 --> 00:26:37,759 be focusing on this problem, and you are, You're trying to solve the 383 00:26:37,759 --> 00:26:41,039 problem, but all these other things are kind of tangential to it, and 384 00:26:41,079 --> 00:26:45,079 if you don't have the people that know that thing, your company is looking 385 00:26:45,119 --> 00:26:48,839 around like okay, who has either the bandwidth or who do we think is 386 00:26:48,880 --> 00:26:51,160 like the best engineer that can go figure it out? And like now they 387 00:26:51,200 --> 00:26:53,920 have another technology and other responsibility that kind of lands on their plate. If 388 00:26:53,960 --> 00:26:56,839 you're a small company, you're starting to grow and you're like, okay, 389 00:26:56,960 --> 00:26:59,480 like I'm doing a lot of this cloud stuff, like we should hire our 390 00:26:59,480 --> 00:27:03,359 first DevOps engineer, and so now it's eight people that don't have like this 391 00:27:03,440 --> 00:27:06,759 deep experience in the cloud. They are like, we should hire somebody that 392 00:27:06,799 --> 00:27:11,359 does. Then you start that job, you're the death first DevOps engineering. 393 00:27:11,400 --> 00:27:12,279 You're like, okay, I had a lot of work for like three months 394 00:27:12,319 --> 00:27:15,799 and I was stressful, But now I'm sitting around like five hours of work 395 00:27:15,839 --> 00:27:19,200 to do a month. Right. Then that's where it gets weird, because 396 00:27:19,359 --> 00:27:22,720 like you can do two things at that point time. You can either sit 397 00:27:22,759 --> 00:27:26,079 back and be like I have thirty five hours of drinking coffee, or you 398 00:27:26,119 --> 00:27:29,880 can try to do like real DevOps how do I make this easier for those 399 00:27:29,920 --> 00:27:32,640 developers? How do I make things more efficient? How do I make things 400 00:27:32,680 --> 00:27:36,279 more resilient. How can I start practicing some SRI principles since I have this 401 00:27:36,359 --> 00:27:40,000 bandwidth to help make sure their software is staying up. And that's where I 402 00:27:40,000 --> 00:27:44,799 think I feel like a lot of DevOps people, particularly if you look at 403 00:27:44,839 --> 00:27:47,599 our slash DevOps, so that's a group of people not saying it, but 404 00:27:47,720 --> 00:27:49,680 like the amount of times people that are like I have nothing to do, 405 00:27:49,920 --> 00:27:53,400 Like I've been working here, like I work like ten hours a week and 406 00:27:53,440 --> 00:27:57,079 like everything's like kind of messy still, but like nobody's given me, like 407 00:27:57,599 --> 00:28:00,519 the DNS project to work on. And it's like, well, you're supposed 408 00:28:00,519 --> 00:28:03,920 to be talking to your you're supposed to be doing that collaboration part, talking 409 00:28:03,960 --> 00:28:07,359 to your engineers for how you can make their life better, Like that is 410 00:28:07,440 --> 00:28:11,039 your job, and like that devlops role is like continue to make the software 411 00:28:11,039 --> 00:28:15,640 better, the business better, and the engineering dev experience better. And I 412 00:28:15,680 --> 00:28:19,000 think that like once you start going down that path, like that is a 413 00:28:19,000 --> 00:28:22,960 good place for DevOps to be. But where we're just tossing stuff over a 414 00:28:23,000 --> 00:28:26,920 fence to a person and we're like, oh, well, the devlops team 415 00:28:26,920 --> 00:28:32,359 they configure the cloud things for me, Like that that's that's that's just two 416 00:28:32,440 --> 00:28:38,200 thousand and five. I'll just pay for sure. Yeah, I take the 417 00:28:38,240 --> 00:28:44,880 approach that as a as my role in develops. The software engineers are my 418 00:28:45,000 --> 00:28:48,640 customers. So my job is to make it to like provide the tools that 419 00:28:48,680 --> 00:28:59,359 they can consume that let them do their job more efficiently and putting in putting 420 00:28:59,359 --> 00:29:03,400 in guardrails so that they can do what they need to do. But then 421 00:29:03,440 --> 00:29:07,640 I have guardrails in place so that they don't accidentally do something that they may 422 00:29:07,680 --> 00:29:12,000 not want to do, like you know, expose root access to the public 423 00:29:12,000 --> 00:29:17,319 Internet or something like that. Yeah, I think those guardrails are important. 424 00:29:17,319 --> 00:29:22,240 Like that's actually a term that we use pretty pretty frequently and like our sales 425 00:29:22,279 --> 00:29:27,640 calls and even in our content, Like I feel like again like our software 426 00:29:27,680 --> 00:29:32,440 is becoming more and more cloud services that we're consuming and less of our code 427 00:29:32,559 --> 00:29:36,119 a lot of used to B calls, and I feel like when you're using 428 00:29:36,160 --> 00:29:40,559 the cloud, like there's so many services like how do you do how do 429 00:29:40,599 --> 00:29:44,799 you how do you do event driven systems on AWS? That's an SQUS, 430 00:29:45,000 --> 00:29:49,240 is it event bridge? Is it POFKA, Like like I've had three or 431 00:29:49,279 --> 00:29:52,839 four options just in AWS, right, and then let's say I pick it. 432 00:29:53,559 --> 00:29:56,799 Okay, let's say I pick my solution. Looking through some of the 433 00:29:56,920 --> 00:30:02,960 AWS like models when you like post your thing to make a new thing, 434 00:30:03,079 --> 00:30:07,000 Like, some of them are insane. Like one of the most crazy services 435 00:30:07,000 --> 00:30:11,960 at AWS is that's three. That's three, like one hundred and twenty configuration 436 00:30:11,079 --> 00:30:15,680 options, right, And so there's a lot of stuff that is getting exposed 437 00:30:15,720 --> 00:30:18,799 to your engineers. So if you're like, oh, I did DevOps because 438 00:30:18,839 --> 00:30:22,920 like I installed terra form and I get of action and now they can write 439 00:30:22,960 --> 00:30:26,559 whatever, Like you didn't you just you just gave everyone a massive footgun. 440 00:30:26,599 --> 00:30:30,240 You didn't make their life better. Sure they can do whatever they want, 441 00:30:30,319 --> 00:30:33,079 but like should they be able to do whatever they want? Right? And 442 00:30:33,079 --> 00:30:37,359 I think that what we need to be doing as an industry DevOps, platform, 443 00:30:37,359 --> 00:30:44,400 engineering operations, whatever your company calls it, is like enabling those developers 444 00:30:44,440 --> 00:30:47,119 to do what they need to do, but also shrinking the surface area of 445 00:30:47,119 --> 00:30:49,880 the cloud. Right. So, like if you have an established operation scheme 446 00:30:49,920 --> 00:30:55,119 that understands all the cloud services, being able to have like a catalog of 447 00:30:55,200 --> 00:30:59,000 like here's what we use for a venture and systems, it's COFA. If 448 00:30:59,039 --> 00:31:02,000 you don't want to use COFT, that is your choice, but it's now 449 00:31:02,000 --> 00:31:06,160 your operational burden. Like make an RFCs to why we should also support e 450 00:31:06,200 --> 00:31:08,599 vent bridge and like we will take on owning a vent bridge. But like 451 00:31:08,599 --> 00:31:11,200 if you're just gonna loan range it, like go for it, right. 452 00:31:11,440 --> 00:31:15,920 And also even when you pick a service, like you need to service, 453 00:31:15,960 --> 00:31:18,640 you need to shrink the surface area of that service, right, Like the 454 00:31:18,720 --> 00:31:23,799 cloud's goal is not to be easy. AWS isn't We're going to be easy. 455 00:31:23,880 --> 00:31:27,759 If they're easy, they would only have lights fail. Their job. 456 00:31:27,880 --> 00:31:32,440 Their job is to be capable. Their job is that you can cut this 457 00:31:32,559 --> 00:31:34,440 pizza in any shape. You want to make your party happy. You want 458 00:31:34,480 --> 00:31:37,960 squares customers, you want we cut wedges, You want to do a weird 459 00:31:37,960 --> 00:31:41,039 little zigzag, you want all edge, I don't know whatever you want to 460 00:31:41,079 --> 00:31:44,599 do, Like we should be able to do it with AWS. And here's 461 00:31:44,640 --> 00:31:48,319 all these things, right, But the reality is is like a lot of 462 00:31:48,319 --> 00:31:52,559 those configuration settings don't matter for your use case high Aurora, like you can 463 00:31:52,559 --> 00:31:56,319 configure Aurora. There's so many ways that I'm gonna go provision postcress and I'm 464 00:31:56,359 --> 00:32:00,720 gonna turn on this thing. That is this field that I can turn on. 465 00:32:00,759 --> 00:32:04,319 It's like that's only for my sequel auror it's like, oh, well, 466 00:32:04,359 --> 00:32:07,480 then why was it exposed to me? I only use Postgress. I'm 467 00:32:07,480 --> 00:32:08,759 not going to randomly switch from postgress to my sqel one day, but I 468 00:32:08,839 --> 00:32:12,799 might randomly change a field name right, or like a field value right, 469 00:32:13,480 --> 00:32:16,720 So it shrinking that service area is important? Like why why? So? 470 00:32:16,799 --> 00:32:21,119 I would love somebody in the comments tell me why I'm wrong. Why is 471 00:32:21,160 --> 00:32:28,759 turning encryption off for anything an option anymore? Yeah? I agree with you, 472 00:32:28,920 --> 00:32:32,920 and I think there's like a certain amount of responsibility. I actually just 473 00:32:34,000 --> 00:32:37,880 had this conversation the other day. I think there's a certain amount of responsibility 474 00:32:37,960 --> 00:32:43,480 from the cloud providers to do a better job here. Like if if I'm 475 00:32:44,519 --> 00:32:51,359 as a cloud provider, they advocate infrastructure's code, least privileged security on all 476 00:32:51,440 --> 00:32:54,480 those kinds of things. So why in their documentation does it say, oh, 477 00:32:54,519 --> 00:33:00,359 if you want to push to the artifact repository, you need artifact repository 478 00:33:00,400 --> 00:33:06,039 dot admin privileges, Like do I really need admin privileges or was there a 479 00:33:06,240 --> 00:33:10,240 better way you could have phrased that to help me adhere to the principles that 480 00:33:10,319 --> 00:33:15,440 your marketing team is advocating. Yeah, and again like I don't, I 481 00:33:15,440 --> 00:33:17,160 don't. Yeah, it's like I don't. I don't think if their goal 482 00:33:17,240 --> 00:33:21,319 is to be easy, it's just like it's it's just to have those capabilities 483 00:33:21,319 --> 00:33:25,200 and right like I AM is like a whole other just when you can see 484 00:33:25,240 --> 00:33:28,400 it, like it's funny, like I feel like some of people like what 485 00:33:28,440 --> 00:33:30,359 are you telling me I AM. It's like I AM is cloud blocking. 486 00:33:30,559 --> 00:33:35,599 Like everything we're using Dynamo CloudLock and it's like, no, you can use 487 00:33:35,640 --> 00:33:37,519 Scala. It's like S three. We're locked in. It's like I just 488 00:33:37,559 --> 00:33:42,599 throw Minio in front of GCS or whatever. Like plenty of ways to get 489 00:33:42,640 --> 00:33:45,240 around cloud locking except for I AM. Like if you invest heavily, if 490 00:33:45,240 --> 00:33:50,839 you go beyond the asterisk, Oh my god. That is that is like 491 00:33:51,160 --> 00:33:54,200 everybody's im is just like completely different. You can you can tell how like 492 00:33:54,359 --> 00:33:59,920 hard it is, like there's I AM engineering roles now in large companies. 493 00:34:00,039 --> 00:34:05,200 It's just like, oh, like I can't like it's important. Oh for 494 00:34:05,200 --> 00:34:10,199 sure, I can't imagine. I personally can't imagine what that job. And 495 00:34:12,239 --> 00:34:14,199 hear you saying with like pushing a container. It's like, oh, you 496 00:34:14,239 --> 00:34:16,559 know, I need right. It's like getting to those places where you're like, 497 00:34:17,679 --> 00:34:21,440 my sole role is to figure I am for this place, but the 498 00:34:21,440 --> 00:34:24,119 cloud doesn't give me the ability to make maybe maybe a particular service as secure 499 00:34:24,159 --> 00:34:36,440 as I want, right because of the granularity Google. Yeah, I like 500 00:34:36,559 --> 00:34:37,920 software. It might not sound like it. I actually love it, but 501 00:34:37,960 --> 00:34:42,079 I just yeah, sorry, No, I think that's part of it. 502 00:34:42,119 --> 00:34:47,880 I think that's one of the common characteristics of software engineers is there's a really 503 00:34:47,960 --> 00:34:52,159 really tight balance of which we like more. Do we like writing a code 504 00:34:52,199 --> 00:34:55,559 more, or do we like bitching about writing the code more? And and 505 00:34:55,599 --> 00:35:00,599 it's it's a tight race bitching about it. I coulde. I still do 506 00:35:00,639 --> 00:35:06,079 a lot of development, probably more than I should, but I switched between 507 00:35:06,119 --> 00:35:09,679 Go Go Lang and a lickxer that's where our app's built in pretty frequently. 508 00:35:09,920 --> 00:35:14,199 And you know, whatever language I'm in, I'm just like, oh, 509 00:35:14,199 --> 00:35:22,119 I hate this language. I wish it's if I'm going to rewrite this in 510 00:35:23,079 --> 00:35:27,920 language acts, then you get it done and you're like, that was stupid. 511 00:35:28,239 --> 00:35:30,079 Yeah. I had one of those yesterday. Ask him about four hours 512 00:35:30,079 --> 00:35:32,800 working on something, and I was just like, I finished, I was 513 00:35:32,840 --> 00:35:37,440 like, why did I? Why did I? I'm had a purpose when 514 00:35:37,440 --> 00:35:39,960 I started, it got way out of hand, like I still do it, 515 00:35:39,960 --> 00:35:43,039 and like I over engineered the ship out of it, and I was 516 00:35:43,079 --> 00:35:45,559 like, I don't. I don't know why I spent four hours on that, 517 00:35:45,599 --> 00:35:47,760 But I gotta go do some real work now, right, Yeah, 518 00:35:47,920 --> 00:35:52,239 yeah, I'm gonna do something that I can actually claim because I can't show 519 00:35:52,320 --> 00:35:54,320 tomorrow and say that's all I did. If you feel that way, people, 520 00:35:54,360 --> 00:35:57,920 it's hard to shake. This is this is This is me years into 521 00:35:57,920 --> 00:36:00,719 being a CEO, and I will still go over engine or something if I've 522 00:36:00,719 --> 00:36:05,320 got a spare of time. Oh for sure. Yeah. I actually for 523 00:36:05,480 --> 00:36:07,679 my side projects, I have a group of guys that I hang out with, 524 00:36:08,159 --> 00:36:14,800 and we all know that our sole job as part of that group is 525 00:36:14,840 --> 00:36:19,199 to call each other out on that specific thing, like are you chasing a 526 00:36:19,239 --> 00:36:24,199 shiny object? No? Well okay maybe yes? All right, fine, 527 00:36:24,239 --> 00:36:29,039 I'll go back to what I'm supposed to be doing. I'm gonna start the 528 00:36:29,079 --> 00:36:31,360 antithesis of that group. I'm gonna start like a Rube Goldberg engineering group. 529 00:36:32,599 --> 00:36:37,719 I think you've made that more complicated and like throwing some more technology into It's 530 00:36:37,760 --> 00:36:39,599 like, you know what I could, I could I'm gonna spend a couple 531 00:36:39,639 --> 00:36:47,760 of hours this week and doing that an LLM that generates Bash scripts. Oh 532 00:36:47,800 --> 00:36:53,239 man, that would be just go. I'm afraid of I'm like the next 533 00:36:53,239 --> 00:36:57,000 five years. I'm like, oh, I'm in like like a I think 534 00:36:57,000 --> 00:37:00,239 I'm in the trophic disillusionment around AI. Like people keep asking us, we 535 00:37:00,480 --> 00:37:06,639 did our fundraise this year. We raised eight million dollars and we did it 536 00:37:06,679 --> 00:37:07,960 without putting I mean, it was a hard year to raise. We did 537 00:37:08,000 --> 00:37:13,760 it without putting the words machine learning, data science or AI in our deck. 538 00:37:14,480 --> 00:37:20,000 And we've stayed away from it in like our materials just because like I 539 00:37:20,039 --> 00:37:24,599 don't I don't want to build our product off of like some like trend that 540 00:37:24,760 --> 00:37:30,039 maybe not go where we hope it goes. And seeing some of the other 541 00:37:30,320 --> 00:37:34,320 people in the OPS space like ah AI ops and You're like, Okay, 542 00:37:35,000 --> 00:37:37,440 I'm going to give this a spin, and I'm like, I would like 543 00:37:37,960 --> 00:37:43,800 a Hippa and sock two compliant Ubernetti's cluster. And I will not mention the 544 00:37:43,880 --> 00:37:45,320 name of the other cloud provider, but I shit you not. The code 545 00:37:45,320 --> 00:37:52,679 that it generated was some type scripty codeink wink. It was all the code 546 00:37:52,679 --> 00:37:55,880 to like initialize an KAS cluster, and then the comments I kid you not, 547 00:37:57,119 --> 00:37:59,440 I kid you not. The only code was like some import statements. 548 00:37:59,440 --> 00:38:04,519 It was like stree equals new EKS cluster, and then in the comments it 549 00:38:04,599 --> 00:38:08,320 was like configuration for stock too and nip A compliance here. And I was 550 00:38:08,400 --> 00:38:15,639 like, yeah, the intelligence is pretty fricking artificial if you ask me, 551 00:38:16,840 --> 00:38:20,440 but like I don't know, like the thing that's really freaking about it. 552 00:38:20,480 --> 00:38:22,840 I kind of got it, couldn't even imagine it for bachelors be terrifying, 553 00:38:22,840 --> 00:38:24,400 But like this is this is again a part of our problem, like we 554 00:38:24,440 --> 00:38:30,320 don't have enough opps experience in most of our engineering team, and so the 555 00:38:30,400 --> 00:38:34,159 idea kind of like business people looking at engineers and not being able to tell 556 00:38:34,239 --> 00:38:37,559 like where the lines are the same thing happens when you go and ask something 557 00:38:37,599 --> 00:38:39,360 to write software for something you're not familiar with, if you aren't familiar with 558 00:38:39,639 --> 00:38:44,159 running Gubernetes, or maybe it's even just configuring some serveralist stuff, you know, 559 00:38:44,199 --> 00:38:45,960 step functions or whatever, and you're like, oh, I need some 560 00:38:45,079 --> 00:38:49,199 terrorform. It'll do all the step function stuff, make it for my computer. 561 00:38:49,559 --> 00:38:53,079 It's like okay, like cool, you might run that and it might 562 00:38:53,079 --> 00:38:57,760 apply it may or may not do what you intended it to do. But 563 00:38:57,840 --> 00:39:00,559 like now you have a bag of code in front of you, you don't 564 00:39:00,639 --> 00:39:04,440 understand it's doing something. You don't know if it's doing it right. And 565 00:39:04,559 --> 00:39:09,679 like chat GPT was confidently Hey I did it right. I think like you 566 00:39:09,679 --> 00:39:13,800 have no idea, so like are you going to go Google every single one 567 00:39:13,800 --> 00:39:16,599 of those lines of code that isn't the resource block to figure out like, 568 00:39:16,639 --> 00:39:19,880 oh, is this is this what I need to set for this? Like 569 00:39:19,920 --> 00:39:23,360 no, you're like just like the product manager looking like that works right right? 570 00:39:23,440 --> 00:39:29,480 Yeah, I got step pumption. I'm moving on. It's like that's 571 00:39:29,559 --> 00:39:30,840 that's freaking. It's like I would rather have you just gone on, Like 572 00:39:30,920 --> 00:39:35,679 I would just please go click it in the AWS console, like turn on 573 00:39:35,719 --> 00:39:38,559 a macro reporter, install the Chrome extension that led like records your mouse clicks. 574 00:39:38,800 --> 00:39:42,719 Go do that and just use the Chrome extension as like your I C 575 00:39:42,880 --> 00:39:45,360 tool at that point in time. I'd rather do that than run something through 576 00:39:45,519 --> 00:39:51,199 AI and hope it right. Like that's great. I've been I've been struggling 577 00:39:51,199 --> 00:39:53,559 with this for a while is for people who do click ops, how do 578 00:39:53,639 --> 00:40:00,280 you how do you document and track that? But I never thought of just 579 00:40:00,400 --> 00:40:06,639 using the click recorder and Chrome. That's brilliant. The problem solved. Yeah, 580 00:40:07,519 --> 00:40:09,920 yet we've got these We've got these video files. Just watch them and 581 00:40:09,960 --> 00:40:13,280 you'll see how I was built. Well, if I'm a lot of the 582 00:40:13,320 --> 00:40:15,519 I'm not sure I'm a lot to plug myself. But you're looking for something 583 00:40:15,559 --> 00:40:20,639 that feels like clickops through engineers, but you want all that operational maturity and 584 00:40:20,719 --> 00:40:23,480 security. No, I do want to talk about that, which is one 585 00:40:23,480 --> 00:40:27,079 of the reasons why I was looking forward to having you on the show here, 586 00:40:27,119 --> 00:40:34,199 because I like I'm at the point where I advocate strongly for infrastructure's code 587 00:40:34,800 --> 00:40:42,519 because six weeks from now, whenever something is different, I want to be 588 00:40:42,599 --> 00:40:46,000 able to look at a get history or poor request history and see what has 589 00:40:46,119 --> 00:40:52,000 changed, or run terraform and see that whatever's running out on that provider doesn't 590 00:40:52,000 --> 00:40:55,280 match what terraform says it should be. And that's been one of the big 591 00:40:55,360 --> 00:41:02,639 challenges I've had with the platform engineering tools, that some of them are just 592 00:41:04,280 --> 00:41:07,519 clickops on top of clickops, you know, and it's like if I just 593 00:41:07,559 --> 00:41:10,639 want to click ops, I would just use AWS. So how do you 594 00:41:10,679 --> 00:41:15,719 approach that? Yeah, so I think that. Okay, So I'm gonna 595 00:41:15,719 --> 00:41:17,360 tell you a couple of things about me. I'm a software engineer, so 596 00:41:17,400 --> 00:41:22,079 I understand like I understand vendors, and I have to struggle as like a 597 00:41:22,119 --> 00:41:27,880 CEO and a founder and also a developer where like I don't want to be 598 00:41:27,960 --> 00:41:30,079 the annoying vendor. That's why I was like, I don't know want the 599 00:41:30,079 --> 00:41:31,800 plug like I want to talk about like I want to sell this thing, 600 00:41:31,800 --> 00:41:36,079 but at the same time, like I don't want to annoy people. But 601 00:41:36,280 --> 00:41:42,320 uh but yeah, So what's interesting is so the second thing about me, 602 00:41:42,320 --> 00:41:44,480 I'm going to say something and I'm going to be I'm not going to be 603 00:41:44,480 --> 00:41:50,079 the founder that says this and is disingenuous. I think we're going about enabling 604 00:41:50,119 --> 00:41:53,320 platform engineering much differently than a lot of the other tools out there. Right, 605 00:41:53,400 --> 00:41:58,639 So what's really freaking me out about this is I think platform engineering is 606 00:41:58,679 --> 00:42:01,480 important. Actually, I make a note of like the clickops, how we 607 00:42:01,519 --> 00:42:04,639 do it different. I'm gonna come back to that in a second. Do 608 00:42:04,639 --> 00:42:06,360 I haven't have time to set way? He said, we can go on 609 00:42:06,400 --> 00:42:07,960 forever, right, I can, Yeah, there's there's there's no time on 610 00:42:08,039 --> 00:42:12,840 it. So here here's why this is important. I spend a lot of 611 00:42:12,880 --> 00:42:16,559 time like reading surveys. I like it. It's one of my weird things. 612 00:42:16,639 --> 00:42:19,440 And I feel like a lot of times, like when you get a 613 00:42:19,440 --> 00:42:22,400 survey at the door report, there's like a couple of like pull quotes and 614 00:42:22,440 --> 00:42:23,519 some numbers, and you're like, oh, we're all doing much better. 615 00:42:23,639 --> 00:42:25,639 But then when you like get to page like thirty stuff and you're like, 616 00:42:25,679 --> 00:42:32,079 holy crap, everyone's gonna die. Looking at the stack overflow survey for the 617 00:42:32,119 --> 00:42:37,599 past four or five years is really interesting. If you look at cloud operations 618 00:42:37,599 --> 00:42:44,559 experience, every every administrative role, operations role in the stack Overflow survey has 619 00:42:44,599 --> 00:42:47,800 steadily declined in the amount of people doing it over the past five years. 620 00:42:47,880 --> 00:42:52,480 DBAs, SRES, DevOps, operations, all of them are going down. 621 00:42:52,639 --> 00:42:57,119 Now are we retiring beast? Maybe we have a lot of grave yards that 622 00:42:57,159 --> 00:43:00,840 could be the reason. But also we're just making engineers past. We're making 623 00:43:00,840 --> 00:43:04,400 engineers faster than we've ever made engineers ever. Remember seventy years ago, there 624 00:43:04,480 --> 00:43:07,400 was like one software developer. Now there's more software developers on the planet than 625 00:43:07,440 --> 00:43:12,280 there are people in Australia, Like you met an entire, huge ass country 626 00:43:12,280 --> 00:43:15,119 full of kangaroos and software developers. Like that's something we could do, would 627 00:43:15,119 --> 00:43:19,639 be really weird, but we shouldn't do it. But like right, so, 628 00:43:19,800 --> 00:43:22,519 like where are these people coming from. They're coming from boot camps. 629 00:43:22,559 --> 00:43:24,840 Like we're still on a lot of people coming out of universities. We're printing 630 00:43:24,880 --> 00:43:28,639 people out of boot camps, and that's not a bad thing. Some of 631 00:43:28,639 --> 00:43:30,960 my favorite engineers I've hired are out of boot camps. There's a personality type 632 00:43:30,960 --> 00:43:34,679 that I look for when I see somebody coming from a boot camp, like 633 00:43:34,679 --> 00:43:39,360 a fresh grad no history, that's eighteen. That's a person that was really 634 00:43:39,440 --> 00:43:44,599 those probably doing software, got very excited, went straight to a boot camp 635 00:43:44,599 --> 00:43:45,239 because like I want to get in the job. I don't want to sit 636 00:43:45,239 --> 00:43:47,920 four years learning about algorithms to build a database, like I want to build 637 00:43:47,920 --> 00:43:52,159 a website. That person, right, the person that worked at a power 638 00:43:52,199 --> 00:43:57,119 plant for fifteen years and they're like forty three, that decides I'm going to 639 00:43:57,159 --> 00:44:00,119 give up on this career that I've been doing all this time to go in 640 00:44:00,239 --> 00:44:02,920 software, Like that person has grit. I love that person. I think 641 00:44:02,920 --> 00:44:07,920 that boot camps are an important part of our society. Why how do you 642 00:44:07,960 --> 00:44:14,199 attige boot camps to like civilization? Everything's moving to the cloud. Everybody is 643 00:44:14,199 --> 00:44:16,639 a software company. We're becoming more and more dependent on this stuff. We 644 00:44:16,679 --> 00:44:22,840 need more software developments. But what's happening is boot camps don't teach the cloud. 645 00:44:23,559 --> 00:44:27,800 Boot camps don't teach DevOps. They teach you to build react code. 646 00:44:27,800 --> 00:44:30,000 They teach you to build rails. They teach you to add business value as 647 00:44:30,079 --> 00:44:35,280 quickly as possible, which most businesses don't see. DevOps is business value. 648 00:44:35,360 --> 00:44:37,679 Right. So it's like you come out of this boot camp for twelve weeks, 649 00:44:37,840 --> 00:44:40,079 you got your twenty six thousand dollars tuition with like eight percent or whatever, 650 00:44:40,320 --> 00:44:43,840 and you're like, I'm ready to write some reacts. I'm ready to 651 00:44:44,079 --> 00:44:46,920 rails, generate an active record model, like let's go. And so those 652 00:44:46,920 --> 00:44:51,199 people come in and they don't have any DevOps experience. The don't have any 653 00:44:51,239 --> 00:44:53,039 operations experience, right, And I think that's where we're seeing this, like 654 00:44:53,039 --> 00:44:58,239 like this steep drop is like some people are retiring because like we do have 655 00:44:58,599 --> 00:45:00,920 a lot of gray beards, especially on might be in the data centers and 656 00:45:00,960 --> 00:45:05,599 more on the administrative like Linux to step inside. Those people are going away 657 00:45:06,000 --> 00:45:08,320 for sure. We're also just printing a bunch of people that are very much 658 00:45:09,400 --> 00:45:14,719 closer to the top end of the stack, right. And the thing that 659 00:45:14,719 --> 00:45:16,880 sucks with OPS is we don't teach it in universities either, because if you 660 00:45:16,920 --> 00:45:21,400 talk talk Kubernetes. Actually, if you if you rewind four years and you 661 00:45:21,440 --> 00:45:24,199 taught people how to deploy in the cloud, what would you be teaching them. 662 00:45:24,320 --> 00:45:28,199 Probably not Kubernetes four years ago. Today it'd be Kubernetes, but four 663 00:45:28,239 --> 00:45:34,559 years ago maybe the two instance. Yeah, maybe, Like nothing is really 664 00:45:34,559 --> 00:45:37,880 I'm gonna say something bold here. Nothing's really changed about our software in the 665 00:45:37,920 --> 00:45:44,039 past seventy years. LOOPS variables ifs sure, you've got different syntactical sugar on 666 00:45:44,119 --> 00:45:46,280 it. But in the last twenty years, everything about the way that we 667 00:45:46,440 --> 00:45:52,239 package and ship software to the cloud has changed four times. We had data 668 00:45:52,239 --> 00:45:55,360 centers, then we had vms, then we had everybody started going the servi 669 00:45:55,440 --> 00:45:59,599 list, and then containers came around, and then like big Container orchestration, 670 00:45:59,760 --> 00:46:04,920 then servilest containers, now llms, like we've changed the actually llens might be 671 00:46:04,960 --> 00:46:07,760 the first thing that's happened in like the way we write software that's fundamentally changed. 672 00:46:08,000 --> 00:46:10,320 Besides just like if look, you're not because they're all just jam and 673 00:46:10,320 --> 00:46:15,960 hashes into somebody else's database. Now sorry, So so that is sorry. 674 00:46:16,000 --> 00:46:20,079 So that was me explaining like why we're doing things a little different, Like 675 00:46:20,119 --> 00:46:24,639 the the amount of operational knowledge on this planet is going down in relation to 676 00:46:24,639 --> 00:46:28,880 the amount of software developers we have, and those software developers are only going 677 00:46:28,920 --> 00:46:32,039 to learn OPS on the job. That's a problem, right, Yeah, 678 00:46:32,239 --> 00:46:35,559 are they going to learn ops from? Are they? Are all of your 679 00:46:35,599 --> 00:46:38,400 DevOps engineers going to turn into You're going to make a fifty room campus so 680 00:46:38,440 --> 00:46:42,639 you can teach all your software developers like how to do a DevOps You have 681 00:46:42,679 --> 00:46:45,280 brown bags every single day. Like you're not going to transfer that knowledge from 682 00:46:45,280 --> 00:46:50,280 your DevOps team to your engineers in an efficient way, right, So you 683 00:46:50,360 --> 00:46:52,239 need to build the substraction. You need to lower the surface area so they 684 00:46:52,280 --> 00:46:58,079 can do as much as they have access to without having to like jump over 685 00:46:58,119 --> 00:47:00,320 that wall. Right as soon as you reach around that wall, you're like, 686 00:47:00,360 --> 00:47:04,159 hey, I was able to deploy everything myself, but like it's down 687 00:47:05,400 --> 00:47:07,280 Okay, we they gotta be able to troubleshoot themselves, right if they have 688 00:47:07,320 --> 00:47:09,519 to come around to you and be like, hey, I deployed this thing 689 00:47:09,519 --> 00:47:13,440 that build a container. It went up and that is just fucked Like if 690 00:47:13,480 --> 00:47:17,719 they come around to you like that, like they need to have that knowledge 691 00:47:17,760 --> 00:47:21,920 themselves, like we need to be building those abstractions they can solve these problems 692 00:47:21,920 --> 00:47:27,480 themselves. And also at the same time, I think as a platform and 693 00:47:27,760 --> 00:47:30,760 we do this, and we hope that our users do this for their users 694 00:47:30,960 --> 00:47:34,800 do this, but like as platforms and people this knowledge, we need to 695 00:47:34,800 --> 00:47:38,239 figure out how to get it as efficiently as we can into developers because we 696 00:47:38,239 --> 00:47:42,760 do need more people understanding how this stuff works. And I love when one 697 00:47:42,800 --> 00:47:44,960 of our customers even is like, hey, how does this work? Like 698 00:47:45,119 --> 00:47:46,400 under the hoot, It's like, yeah, you should you should be asking 699 00:47:46,440 --> 00:47:50,079 me that, like you should want to know that, but like if you 700 00:47:50,119 --> 00:47:52,599 don't, you should be able to do your job effect And so what I've 701 00:47:52,920 --> 00:47:57,440 so this is a long lead up to me saying I think we do things 702 00:47:57,440 --> 00:48:00,599 a bit differently, which every startup founder sets but the thing that made it 703 00:48:00,599 --> 00:48:06,159 different for us is everybody's talking about platform engineering, and the first thing they 704 00:48:06,239 --> 00:48:09,599 jump to is who there's like Kubernetes. Oh, guess what, I love 705 00:48:09,679 --> 00:48:14,800 Kubernetes. I develop on Kubernetes locally. Like that's how much of a family 706 00:48:15,840 --> 00:48:19,199 I would be, Like, if you want to use my product, you 707 00:48:19,280 --> 00:48:22,840 have to use Kubernetes. I just made a choice for you, right, 708 00:48:22,880 --> 00:48:27,079 Like you might be perfectly fine running on Lambda, right, but we want 709 00:48:27,079 --> 00:48:30,400 to use that forever. So I think that, and I think one of 710 00:48:30,440 --> 00:48:34,199 the things that's a little scary about this idea of platform engineering is like we're 711 00:48:34,280 --> 00:48:37,679 leaning too heavily towards Kubernetes and not accepting the fact that there are other things. 712 00:48:37,719 --> 00:48:40,719 I mean, not that anybody's going to run out and run masive sphere, 713 00:48:40,760 --> 00:48:44,880 but like there is nomad, there are other run times, there are 714 00:48:44,880 --> 00:48:50,360 things that are outside of Kubernetes. Right. It's very plausible that your software 715 00:48:50,400 --> 00:48:55,360 teams in your company operate their software differently. And so the idea that you're 716 00:48:55,360 --> 00:48:59,880 going to come in and be like we have one platform and everybody, nine 717 00:49:00,079 --> 00:49:01,960 percent of you are going to fit your code into this one platform. It's 718 00:49:02,000 --> 00:49:07,000 like, Okay, that sucks for the jam stacked people. That's suck inc. 719 00:49:07,920 --> 00:49:08,960 So what are we doing? Are we gonna run Are we gonna run 720 00:49:09,159 --> 00:49:13,679 llms on versell? Are you gonna run jam stacked on open Ai? Like? 721 00:49:13,679 --> 00:49:15,280 How are we gonna fit this all on the one platform? The reality 722 00:49:15,400 --> 00:49:20,159 is like much like organizations are different, like your teams are different, and 723 00:49:20,199 --> 00:49:24,360 so you can't have a platform that does everything. Those engineers need to be 724 00:49:24,360 --> 00:49:30,199 able to have the agency to get what they need from the cloud without constantly 725 00:49:30,239 --> 00:49:32,440 bothering another person or reading a white paper, and the business needs to know 726 00:49:32,840 --> 00:49:39,199 that that software is secure and compliant. That's hard to do when most companies 727 00:49:39,239 --> 00:49:44,639 are abysmally failing at DevOps. Fifty one percent of you based on the door 728 00:49:44,719 --> 00:49:46,719 or report, So if that's not you, one of your friends is lying. 729 00:49:50,559 --> 00:49:52,519 And also like a lot of these companies, like you can see a 730 00:49:52,559 --> 00:49:57,960 company with like fifteen engineers and like nobody really has ops experience, right, 731 00:49:58,000 --> 00:50:00,400 So it's like we just don't have enough of this, and so our approach 732 00:50:00,880 --> 00:50:07,800 has been to focus on operations engineers and DevOps engineers that are either coming to 733 00:50:07,840 --> 00:50:13,360 this realization that they cannot scale themselves or their company is kind of mandated, 734 00:50:13,400 --> 00:50:15,280 like hey, Gartner said, eighty percent. Companies are gonna have a platform 735 00:50:15,320 --> 00:50:20,559 team by talking about the wrong side platform team by twenty twenty six, and 736 00:50:20,599 --> 00:50:23,599 it's going to make DevOps more efficient. So DevOps Guide, you're a platform 737 00:50:23,639 --> 00:50:30,199 engineer now and does that mean right? And so our approach is we should 738 00:50:30,199 --> 00:50:35,119 be able to work with So like our product's number one user is operations, 739 00:50:35,159 --> 00:50:37,239 they don't use our site, they use the tools that they know. So 740 00:50:37,360 --> 00:50:43,840 mass driver today works with Terraform, open TOFU and helm and we're currently working 741 00:50:43,880 --> 00:50:49,039 on CloudFormation and Gloomy support. And so the idea is you as an operations 742 00:50:49,039 --> 00:50:52,599 team can decide what cloud services you want to support and then you limit that 743 00:50:52,679 --> 00:50:57,519 API of those cloud services using Terraform, Helm, open Too, Food, 744 00:50:57,519 --> 00:51:00,719 et cetera. And so you just publish okay we or Kafka for events, 745 00:51:00,840 --> 00:51:05,800 we support postgresses or transactional database. We run on Kubernetes. We also support 746 00:51:05,840 --> 00:51:09,880 land n API gateway for the services team, and you essentially write Terraform modules 747 00:51:09,880 --> 00:51:15,400 with your whether it's OPA or bridge crew, whatever inside of it. You 748 00:51:15,480 --> 00:51:21,480 create these modules that are your businesses, security compliance and practices spotified into the 749 00:51:21,599 --> 00:51:24,559 naming conventions, et cetera. You publish them into mass driver instead of like 750 00:51:24,599 --> 00:51:29,320 the Terraform registry or whatever. And now when your developers come in, what 751 00:51:29,400 --> 00:51:31,519 they see to state instead of seeing all the services of a WS, they 752 00:51:31,559 --> 00:51:37,159 see the services that my operations team supports. Now what's really cool is they 753 00:51:37,199 --> 00:51:39,719 just draw infrastructure. So like, okay, like we run on Kubernetes, 754 00:51:39,760 --> 00:51:43,760 Like, throw my Kubernetes cluster up there. I have a whole thought, 755 00:51:43,840 --> 00:51:45,239 I have a whole I have a whole rant. I can go on about 756 00:51:45,239 --> 00:51:49,199 how many Kubernetes plusters you should have. It should be more than one. 757 00:51:49,679 --> 00:51:57,320 Cattle cattle versus pets. People, you got all these all these applications or 758 00:51:57,360 --> 00:52:00,599 cattle cluster is certainly a pet. Is that you're afraid to change it. 759 00:52:04,159 --> 00:52:08,039 But we named it and everything we actually had, sorry i'm gonna say something, 760 00:52:08,440 --> 00:52:13,519 and a little side tangent. We had a cople of company flip trips 761 00:52:13,559 --> 00:52:15,639 that we were at. We had essentially two Kubernetes clusters of van production. 762 00:52:15,719 --> 00:52:19,440 And what we would do is every time that we were doing a Kubernettes upgrade, 763 00:52:19,559 --> 00:52:22,840 we would do it too essentially our standby, and then we'd roll applications, 764 00:52:22,920 --> 00:52:24,599 cut traffic to it. And so like that's how we kind of determined 765 00:52:24,599 --> 00:52:28,039 that, like okay, upgrading at one point two seven or whatever, it 766 00:52:28,159 --> 00:52:31,840 worked. And both of those clusters were pets ish we created them with cub 767 00:52:31,880 --> 00:52:37,599 admin or cub no cops, but they both had names and it was because 768 00:52:37,679 --> 00:52:44,320 Siri could never get Kubernetes right, so we had cuban Eddie and Kubernetti. 769 00:52:46,559 --> 00:52:51,000 But anyways, still like our our approaches, all right, I'm a I'm 770 00:52:51,039 --> 00:52:53,639 a brain map to follow me, but so approaches like make sure the operations 771 00:52:53,639 --> 00:52:57,559 people don't have to learn a new tool chain, right, like use the 772 00:52:57,559 --> 00:53:00,880 stuff that you already know, Like there's not enough of you. I can't 773 00:53:00,239 --> 00:53:05,599 as a business, have you spending more time learning a bunch of new stuff 774 00:53:06,199 --> 00:53:09,000 when engineers are already depending on you for stuff, demanding stuff from you. 775 00:53:09,000 --> 00:53:12,119 Your team is already stressed out, and now we're saying, hey, we 776 00:53:12,159 --> 00:53:15,760 want to make we want to scale you to make it more efficient for them 777 00:53:15,840 --> 00:53:17,119 by adding more work to your plate on top of the other stuff that's going 778 00:53:17,159 --> 00:53:20,639 to happen. It's like okay, well, or was if we're also kind 779 00:53:20,639 --> 00:53:22,679 of like helping with outages and monitoring, et cetera, like Are we just 780 00:53:22,679 --> 00:53:25,760 not doing that anymore? Why we go build a platform, right? Are 781 00:53:27,159 --> 00:53:30,519 we all going to turn into React engineers so we can build backstage plug ins? 782 00:53:30,519 --> 00:53:32,559 And it's like no, Like so our approach has always done an operations 783 00:53:32,599 --> 00:53:36,639 engineer, Like we wanted to build something that worked with the tools that your 784 00:53:36,639 --> 00:53:38,760 team knows. You publish it in there. Your engineers just draw the infrastructure, 785 00:53:38,760 --> 00:53:43,880 and so as they're drawing, what happens is they drag let's say they 786 00:53:43,920 --> 00:53:46,760 drag an API gateway out or a lambda. There's these additions that you can 787 00:53:46,760 --> 00:53:52,719 make to your infrastructure's code we call we internally we call them artifacts. But 788 00:53:52,000 --> 00:53:57,840 it effectively lets you create a diagram of your terraform. So you can say, 789 00:53:57,880 --> 00:54:02,599 like my application run time like terrifle module or healm module depends on Kubernetics 790 00:54:04,480 --> 00:54:07,039 and so now like when you drag that into the campus, that will automatically 791 00:54:07,039 --> 00:54:09,159 connect like the nearescubernets cluster. You can move it around like your engineers don't 792 00:54:09,159 --> 00:54:10,679 have to think about it. So it's like, Okay, we're building this 793 00:54:10,719 --> 00:54:15,079 service for the first time we wrote it. I'm going to use the helm 794 00:54:15,199 --> 00:54:20,199 chart here for running going applications. I drag it on. I set my 795 00:54:21,039 --> 00:54:24,360 Docker repository and it just connects to kubernets cluster, I hit deployed and I 796 00:54:24,360 --> 00:54:28,960 get back to work, right, And so they have this limited view of 797 00:54:28,960 --> 00:54:32,039 the cloud. It's the services that your team supports. But then also your 798 00:54:32,119 --> 00:54:37,199 terraform variables are also when they come into mass Driver, are become heavy documentation. 799 00:54:37,320 --> 00:54:43,000 So it's not just like oar name like. You can actually mass driver 800 00:54:43,000 --> 00:54:45,800 supports like super rich and like cross validation, so you can say, like 801 00:54:46,079 --> 00:54:50,679 if the name is this, then this other field has to have this specific 802 00:54:50,719 --> 00:54:52,960 validation on it as well. So you can start to codify a lot of 803 00:54:53,360 --> 00:54:57,039 constraints into the system. So you can say, okay, we have an 804 00:54:57,079 --> 00:55:04,199 Aurora postgrest bundle and it can run. If anybody's a familiar with Aurora, 805 00:55:04,320 --> 00:55:08,920 you too. There's an instance type called serverless. It's very weird, but 806 00:55:09,719 --> 00:55:14,800 so technically you can switch your Aurora back and forth between servilists and server full. 807 00:55:14,840 --> 00:55:16,559 Like you experience a downtime while it's happening, but you can switch it 808 00:55:16,599 --> 00:55:21,000 back and forth. Right, But the configuration changes wildly depending on if the 809 00:55:21,159 --> 00:55:24,440 instance type it is serverless, not if it's a servilists Aurora the instance type 810 00:55:24,440 --> 00:55:30,119 of servilist, right, and so like internally we have an Aurora Postgres bundle 811 00:55:30,159 --> 00:55:32,039 that we use and we use servilists for all of our not like our preview 812 00:55:32,079 --> 00:55:36,320 environments, and then we use to reful for our production environment. And so 813 00:55:36,679 --> 00:55:39,719 that configuration like it opens on the side, panels your terraform variables and then 814 00:55:39,800 --> 00:55:44,519 as you change values, the configuration changes it, it'll actually expose and hide 815 00:55:44,519 --> 00:55:46,559 fields based on this. So now's the operation scheme. You can say, 816 00:55:46,559 --> 00:55:52,679 you know what, I know that this my sql bin log stuff of Aurora 817 00:55:52,000 --> 00:55:55,559 doesn't apply to Postcrest whatsoever. So those fields that there's no those variables are 818 00:55:55,559 --> 00:56:00,000 even in here, you can and worry about thinking about configuring your resource. 819 00:56:00,159 --> 00:56:04,039 And you can also say it when they you have this men in max scale 820 00:56:04,079 --> 00:56:07,519 for your instances, when somebody selects a servilist instance, those fields are hidden 821 00:56:07,599 --> 00:56:12,400 so they don't accidentally say I want ten serviless and instead they show it shows 822 00:56:12,440 --> 00:56:16,719 you acus right, and so you can start to codify a lot of convention 823 00:56:16,840 --> 00:56:21,599 there for them. And so now your engineers they don't have to learn a 824 00:56:21,639 --> 00:56:24,000 new tool leaver, they don't have to go copy and paste a bunch of 825 00:56:24,039 --> 00:56:29,239 Terraform modules and do a directory, set up some GitHub actions. Literally, 826 00:56:29,320 --> 00:56:30,800 the only thing to do is they put our CLI in there and they says, 827 00:56:30,840 --> 00:56:35,000 Okay, I want to deploy my production database or I wanted to pull 828 00:56:35,000 --> 00:56:38,000 in my production application, and it's pretty much just like the name of the 829 00:56:38,039 --> 00:56:42,679 resource and mass driver and it deploys. You can put adjacent file in there 830 00:56:42,679 --> 00:56:45,559 that it'll get applied to either HELM or Terraform or flu Me to configure it 831 00:56:45,599 --> 00:56:50,639 based on the constraints that you've defined as an operations engineer. And now what's 832 00:56:50,639 --> 00:56:55,199 happened is you're able to scale this operations team. You can do platform engineering 833 00:56:55,840 --> 00:57:00,880 effectively. Mass Driver we're a platform orchestrator if you're familiar with the seven hundred 834 00:57:00,960 --> 00:57:04,679 terms that people have invented in this space, and each of your projects and 835 00:57:04,679 --> 00:57:07,679 mass rivers effectively its own internal developer platform. But the cool thing is it's 836 00:57:07,679 --> 00:57:13,639 not developers having to operations team doesn't have to go and talk to developers and 837 00:57:13,679 --> 00:57:16,719 make an MVP and make a giant form to survey them on what they like 838 00:57:16,800 --> 00:57:20,360 and don't like, and collaboration back and forth. You say, this is 839 00:57:20,400 --> 00:57:22,960 what we support as a business. This is all that we know. This 840 00:57:22,039 --> 00:57:25,519 is what we're using today, and if you need a new service, we 841 00:57:25,599 --> 00:57:29,280 get hub issue and we'll build the module for it. It'll be a mass 842 00:57:29,280 --> 00:57:30,519 reever. You can drag and drop it. Right Like, at the end 843 00:57:30,559 --> 00:57:35,800 of the day, all of your modules and mass rivers get up repositories where 844 00:57:35,800 --> 00:57:37,960 your code is like that's how we kind of do the collaboration. And then 845 00:57:38,719 --> 00:57:43,760 your ops team didn't have to learn anything new. Your engineering team didn't have 846 00:57:43,800 --> 00:57:46,280 to go learn terraform or polloomy or gett actions or anything like that. They 847 00:57:46,320 --> 00:57:50,360 can focus on writing code. They just drag something onto the canvas when they 848 00:57:50,400 --> 00:57:52,840 need it. You as the ops person, get an audit log, get 849 00:57:52,840 --> 00:57:58,599 deployment histories, you can roll back. We have automated costs and metrics built 850 00:57:58,599 --> 00:58:00,360 into it, right, so trying to give that full agency to the developer. 851 00:58:00,360 --> 00:58:02,800 As soon as you can figure something a mass driver, a day later, 852 00:58:02,920 --> 00:58:06,920 costs show up. So if you can figure this postpress citizence, you're 853 00:58:06,920 --> 00:58:09,639 like, holy shit, it's three hundred dollars like the next day right Like, 854 00:58:09,679 --> 00:58:14,920 okay, I'm gonna change my configuration here because I can also see utilizations 855 00:58:14,960 --> 00:58:19,920 three percent, so I don't have a mad engineering manager mad CFO three months 856 00:58:20,000 --> 00:58:22,920 later when they realize that cloud goes high, I, as a software developer, 857 00:58:22,159 --> 00:58:24,920 have this agency to control my costs. Do the same tool that I 858 00:58:25,000 --> 00:58:29,280 provision it in the same place that I can see my metrics. If we 859 00:58:29,320 --> 00:58:34,320 turn your dash your canvae, you're essentially your your network diagram is also your 860 00:58:34,360 --> 00:58:36,480 dashboard. So you can say, okay, I want all these metrics from 861 00:58:36,480 --> 00:58:39,800 cloud watch to show up alongside this instance and set some alarms for them. 862 00:58:40,239 --> 00:58:45,320 So now when you get an alarm goes off at two am, because that's 863 00:58:45,320 --> 00:58:49,440 when grimlins use the internet, right, I think, So it's like two 864 00:58:49,519 --> 00:58:51,840 thirteen, it's like, what in the hell is going on? Who is 865 00:58:51,880 --> 00:58:54,400 a Crown job schedule right now? So now, like at two am, 866 00:58:54,440 --> 00:58:58,760 when you get an alarm in an email, it's like you would like Peter 867 00:58:58,840 --> 00:59:00,599 do right. You think about where we are today, It's like, oh 868 00:59:00,599 --> 00:59:04,360 pat patre alert and you're like, oh fuck, I hate that sounds two 869 00:59:04,360 --> 00:59:07,400 thirteen in the morning. Click on it, uh Ari zero alarm and data 870 00:59:07,440 --> 00:59:10,039 Dog. You go to data Dog and it feels like you just walked into 871 00:59:10,159 --> 00:59:15,639 Vegas at three am, just like crash all over the ground. You're like, 872 00:59:15,679 --> 00:59:17,719 Okay, some something's broke. I can see the things going up this 873 00:59:17,840 --> 00:59:22,440 way. It's red. That's not good. But like, what what's broken? 874 00:59:22,480 --> 00:59:27,280 Okay, it looks like I'm out of database connections on my application? 875 00:59:27,679 --> 00:59:31,280 Well, shit, like our traffic's no different than normally is I din't the 876 00:59:31,320 --> 00:59:36,199 application hasn't been deployed in three weeks because it's pretty stable app. We're not 877 00:59:36,239 --> 00:59:42,199 like actively working on Like what's changed? Like if the traffics the same, 878 00:59:42,639 --> 00:59:46,000 the application hasn't changed in weeks, Why am I out of database connections? 879 00:59:46,000 --> 00:59:50,559 That's super weird. Well, the reason you're out of it is because Tony 880 00:59:51,000 --> 00:59:54,800 on the ops team, he got in trouble because the post residence was too 881 00:59:54,880 --> 00:59:58,800 expensive. CFO was mad. They're like, we got to control our costs. 882 00:59:58,960 --> 01:00:00,880 He scaled down the post for instance, and it can't handle as many 883 01:00:00,920 --> 01:00:05,960 connections now. So now this change that happened that's not in your purview as 884 01:00:05,960 --> 01:00:09,760 a developer, and a different git repo has caused an outage on your application, 885 01:00:10,400 --> 01:00:13,519 and you're sitting there trying to figure out like, why why can't my 886 01:00:13,519 --> 01:00:15,519 application pack the postgress? If you don't have access to that, get repo. 887 01:00:15,599 --> 01:00:20,280 You have no idea if you don't have a dashboard for that Postcriss sentstence, 888 01:00:20,280 --> 01:00:22,360 you have no idea. If you haven't seen the cost to change that 889 01:00:22,360 --> 01:00:23,679 post, for instance, you have no idea. Like we don't have agency 890 01:00:23,719 --> 01:00:28,039 as developers through chaos. Right, So now you're sitting there for hours trying 891 01:00:28,039 --> 01:00:31,159 to figure out what it is. You hop in the slack and you're like 892 01:00:31,320 --> 01:00:35,559 at channel like, has anybody changed anything about the data it's And Tony's like, 893 01:00:35,639 --> 01:00:37,000 yeah, yeah, wait, there's some costs of things. It was 894 01:00:37,000 --> 01:00:39,679 sitting at like ten percent CPU all the time, so we just scaled it 895 01:00:39,719 --> 01:00:44,039 down to some T three micros or whatever. And they're like, oh, 896 01:00:44,039 --> 01:00:45,840 there's your problem. Right, And so with mass drivers with the story's a 897 01:00:45,840 --> 01:00:50,519 little different. Is you get your alert, you click on the link, 898 01:00:51,119 --> 01:00:54,519 we take you to your infrastructure diagram. We highlight the thing that is broken, 899 01:00:55,000 --> 01:00:59,119 and from there you can see the metrics and you can see the change 900 01:00:59,159 --> 01:01:01,079 history of everything. It's the dependency of your applications, is your cluster. 901 01:01:01,519 --> 01:01:06,360 You see your databases, you see your cues, and I can see right 902 01:01:06,400 --> 01:01:09,400 there. I'm like, oh, Chris made a change to the database yesterday. 903 01:01:09,559 --> 01:01:14,280 What did he change, you click on the database you visually diff will 904 01:01:14,320 --> 01:01:17,039 show you the difference to actually keeping track of the changes in your terraform Holme 905 01:01:17,039 --> 01:01:21,599 et cetera. It's like, since yesterday, this is what's changed the instance 906 01:01:21,639 --> 01:01:24,559 type. And you're like, there's your problem, and now I can fix 907 01:01:24,599 --> 01:01:28,199 it. From right there, I can go back to bed and in the 908 01:01:28,199 --> 01:01:30,719 morning, I'll call the CFO and be like, that instance is going to 909 01:01:30,719 --> 01:01:35,679 be expensive again, right to my app to deal with that, maybe if 910 01:01:35,679 --> 01:01:37,440 thro a PG bouncement, then the mix, like the ability to like get 911 01:01:37,480 --> 01:01:42,360 through and resolve an issue like this was never like our intention to the platform. 912 01:01:42,360 --> 01:01:45,239 Our attention was like letting people get infrastructure securely and safely. But what's 913 01:01:45,280 --> 01:01:50,320 happened is like it's just much easier to understand what's going on when you're looking 914 01:01:50,320 --> 01:01:52,039 at a diagram, right, Like we all want diagrams. We draw it 915 01:01:52,079 --> 01:01:55,760 on a whiteboard, we go to our desks and we do some crazy shit 916 01:01:55,840 --> 01:01:59,800 and terraform. And then somebody comes along six months later and they're like, 917 01:02:00,159 --> 01:02:06,159 to diagram this, They terraform, visualize it and get that horrendous resource bag, 918 01:02:06,599 --> 01:02:08,079 and then they try to back that out to like a lucid chart or 919 01:02:08,119 --> 01:02:12,559 like cloud Craft, you're like, hey, we've diagrammed it and nobody updates 920 01:02:12,559 --> 01:02:15,119 it for six months, and then six months later somebody someone breaks and you're 921 01:02:15,159 --> 01:02:16,639 like, this isn't what our system looks like at all, Like we want 922 01:02:16,679 --> 01:02:21,239 it to do. Anybody who says we don't want it to be visual, 923 01:02:21,280 --> 01:02:24,039 do you've ever seen touch a whiteboard or cloud craft or lucid charts, You're 924 01:02:24,039 --> 01:02:28,519 a liar, Like, we want it to be visual, but like there's 925 01:02:28,519 --> 01:02:30,760 not a way of doing it. So I think that's what's fundamentally different about 926 01:02:30,800 --> 01:02:32,199 us, Like we don't want to just throw a portal on top of Kubernetes 927 01:02:32,199 --> 01:02:35,760 and be like you can see all your services. Like it's like we wouldn't 928 01:02:35,840 --> 01:02:37,519 change the way that you think about managing it and change the way to think 929 01:02:37,519 --> 01:02:43,760 about troubleshooting it without making you change all of your tooling. Like developer in 930 01:02:43,760 --> 01:02:47,280 this scenario just writes code and your engineering team or your ops team says, 931 01:02:47,320 --> 01:02:52,360 hey, we support on AWS, we support ekas, and LAMB does great. 932 01:02:53,320 --> 01:02:57,639 We have two separate platforms to land the platform in a Kubernetes platform and 933 01:02:57,639 --> 01:03:00,519 mass driver. I use the one that my team needs if I'm on the 934 01:03:00,519 --> 01:03:01,119 back end team and we're like, actually, you know what, I'm making 935 01:03:01,159 --> 01:03:04,360 the service. I don't know that need a full instance. It is very 936 01:03:04,360 --> 01:03:06,599 small. I could be able to run on LAMB, but I don't have 937 01:03:06,639 --> 01:03:09,800 to go and try to get the ops team or the platform team to like 938 01:03:09,840 --> 01:03:13,400 build out this new functionality for me. I just said, we already have 939 01:03:13,480 --> 01:03:15,800 API gateway and Lambda. The front end team uses it for some jam stack 940 01:03:15,800 --> 01:03:20,960 stuff. I'm going to drag this LAMB over, push a small function up 941 01:03:20,960 --> 01:03:22,400 in there, and call it a day, right, Like, like, 942 01:03:22,519 --> 01:03:28,599 why should I have to engage with a product team and spend time trying to 943 01:03:28,599 --> 01:03:31,039 convince them of what my needs are waiting for them to build an MVP survey. 944 01:03:31,119 --> 01:03:34,840 The rest of the team's like I need to get some work done, 945 01:03:35,239 --> 01:03:40,880 right yeah. So yeah, because then your your ops team becomes the bottleneck 946 01:03:42,480 --> 01:03:45,599 and the limiting factor to your business. And that sucks because like I feel 947 01:03:45,639 --> 01:03:51,280 like we're always the bottleneck. Yeah, they're always the bottleneck. I mean 948 01:03:51,880 --> 01:03:53,719 nobody's ever like, oh my gosh, I was going to go ask the 949 01:03:53,800 --> 01:03:57,719 ops team for something and they already imagined it and did it before I got 950 01:03:58,679 --> 01:04:00,920 No matter what even in the best scenario, you're like you're waiting a bit, 951 01:04:00,079 --> 01:04:02,800 like we're always a little bit of bottle. You bring us in early 952 01:04:03,119 --> 01:04:05,480 on a project and you're like, Okay, we're working on this project. 953 01:04:05,480 --> 01:04:09,400 We're we're trying to figure out what the cloud infrastructure is. It's like you 954 01:04:09,400 --> 01:04:12,119 can do that, But I also feel like when you're very very early in 955 01:04:12,119 --> 01:04:14,679 a project talking about what it could be, like you don't know how the 956 01:04:14,719 --> 01:04:17,440 software and requirements you're going to change, right, and so like the end 957 01:04:17,480 --> 01:04:20,880 of the day, in reality, it's like any given terraform resource or collection 958 01:04:20,920 --> 01:04:24,639 of resources isn't months of work for us, right, It's like trying to 959 01:04:24,639 --> 01:04:28,239 figure out what you need, what abstraction to expose to you, and then 960 01:04:28,239 --> 01:04:30,960 I cobble up some terraform, put in some practices, right my OPA rules, 961 01:04:30,960 --> 01:04:33,719 and like off we go, Like I don't need three months to do 962 01:04:33,760 --> 01:04:36,679 that for a couple of services you're going to use, So like when is 963 01:04:36,719 --> 01:04:40,599 the right time to bring me in? Probably like some percent of the way 964 01:04:40,599 --> 01:04:43,679 through the project, Like once you have something like viable running, maybe you're 965 01:04:43,719 --> 01:04:47,639 using local stack to like figure it out locally, or you're using mass but 966 01:04:49,000 --> 01:04:53,440 as you can, but so like like when do you bring us in, 967 01:04:53,480 --> 01:04:56,039 Like we're always a little bit of a bottleneck, and the reality is is 968 01:04:56,079 --> 01:04:59,920 like maybe Google or not because there's ten thousand OPS engineers, but like the 969 01:05:00,079 --> 01:05:02,800 average company where you're out ratio, I think we say like one to seven 970 01:05:02,920 --> 01:05:08,400 is like the good ratio of OPS to engineers. There's not one to seven 971 01:05:08,440 --> 01:05:12,280 on the planet. Yeah, but like if you look at the if you 972 01:05:12,280 --> 01:05:15,440 look at the stack Overflow survey and you look at the numbers, they're like 973 01:05:15,360 --> 01:05:19,519 it's it's not there. Like if we have twenty seven million software developers, 974 01:05:19,599 --> 01:05:24,039 we do not have two points. Sorry I'm not good enough, but we 975 01:05:24,039 --> 01:05:28,760 don't have seven million operations engineers, right, Like, there's not that many 976 01:05:28,840 --> 01:05:30,960 roles. There is that many roles and they're all sitting open, right Like 977 01:05:31,039 --> 01:05:33,400 we have one of the longest roles that sit open. We also have one 978 01:05:33,400 --> 01:05:35,840 of the shortest tenures. Why is that? Why does it take forever to 979 01:05:35,880 --> 01:05:38,840 place with us? And we quit all the time. We're also one of 980 01:05:38,880 --> 01:05:42,039 the highest paid people on the team. It's because we bounce those jobs constantly, 981 01:05:42,039 --> 01:05:45,559 get those raiss Like we are always going to be a bottleneck until we 982 01:05:45,639 --> 01:05:50,480 start doing something that helps us scale, and even in the platform engineering scenario, 983 01:05:50,599 --> 01:05:54,760 like we still are a bottleneck when somebody starts to move outside of the 984 01:05:54,760 --> 01:05:57,760 platform, right, right, we get that good balance of like the platform, 985 01:05:57,760 --> 01:06:00,599 like I'm never a bottleneck to you because the platform starves what you need 986 01:06:00,719 --> 01:06:03,760 and then when you kind of go outside of like the norm, you go 987 01:06:03,800 --> 01:06:09,320 off that golden path. Now we might be a bottle that now we're talking 988 01:06:09,320 --> 01:06:11,320 about being a product. Okay, great, Like what do you need, 989 01:06:11,400 --> 01:06:14,199 Like, how can we do this for you better? Okay, great? 990 01:06:14,199 --> 01:06:18,719 We need to support ECS because there's some ECS task integration with redshift or something. 991 01:06:18,760 --> 01:06:20,719 I don't know. I don't think that actually tips Like let's say there's 992 01:06:20,719 --> 01:06:24,559 like a very specific use case like okay, cool, let's put together a 993 01:06:24,639 --> 01:06:28,800 real proof of concept terraform module that has ECS. I'll push it up to 994 01:06:28,880 --> 01:06:30,320 you. You can throw it onto your canvas, connect it to the redshift 995 01:06:30,320 --> 01:06:34,559 module we already have, and like, let's let's iterate on this and get 996 01:06:34,559 --> 01:06:38,840 it good enough so that we can have like market like production quality for any 997 01:06:38,840 --> 01:06:44,159 other team that might need to use in ECS, right, And I think 998 01:06:44,159 --> 01:06:48,320 that gets into the social aspect of DevOps too, Like if you can socialize 999 01:06:48,400 --> 01:06:51,519 to the teams that you support, these are the products, like, these 1000 01:06:51,519 --> 01:06:58,760 are the platforms that we've built for you to use, and you can like 1001 01:06:58,920 --> 01:07:03,639 define the rules ving engagement. Then they know when they're venturing outside of those 1002 01:07:03,719 --> 01:07:06,639 rules engagement of engagement, and then they need to come back and talk to 1003 01:07:06,679 --> 01:07:14,840 you about that. But without doing that, the engineering teams see DevOps as 1004 01:07:15,039 --> 01:07:18,880 everything being one off. They're like, oh, I'm going to deploy this 1005 01:07:18,960 --> 01:07:24,239 thing. I gotta go talk to the DevOps guys, and it's such a 1006 01:07:24,239 --> 01:07:27,800 painful process that they wait till the last minute because they just don't want to 1007 01:07:27,840 --> 01:07:32,920 deal with our bullshit. And so I think, just like creating that catalog 1008 01:07:33,039 --> 01:07:36,440 for lack of a better term, like the here's the things you can choose 1009 01:07:36,440 --> 01:07:41,639 from the catalog, and we do custom orders. We just need a little 1010 01:07:41,719 --> 01:07:45,119 lead time. But if you can like get get them to that point where 1011 01:07:45,119 --> 01:07:49,199 they feel like they understand what you do, then you can socialize that conversation. 1012 01:07:50,119 --> 01:07:54,559 And I feel like those conversations are always hard, right because like I 1013 01:07:54,559 --> 01:07:59,880 feel like a lot of people again like maybe not everybody that's listening, like 1014 01:08:00,760 --> 01:08:03,679 has this experience, but like when you go to an ops person, whether 1015 01:08:03,679 --> 01:08:05,840 it's a ticket or you walk over to their desk and like, you need 1016 01:08:05,880 --> 01:08:11,920 something that's like out of the ordinary, right, Like let's let's let's imagine 1017 01:08:11,920 --> 01:08:14,840 that you had this magic tool where you can actually find all your terraform modules 1018 01:08:14,840 --> 01:08:16,239 that your company supports, right, and you go to that, you go 1019 01:08:16,279 --> 01:08:20,119 to that repo and you're like, ah, like Mango at list isn't there. 1020 01:08:20,239 --> 01:08:25,760 We need Mongo outists, We need a document store. Maybe they don't 1021 01:08:25,760 --> 01:08:29,279 have maybe they don't have postcrests and they don't realize that you just use postcrusts. 1022 01:08:29,279 --> 01:08:33,039 Sorry, I need Mongo at lists but it's not here. Now, 1023 01:08:33,079 --> 01:08:35,720 I got to go talk to the ops team. Ticket got to figure it 1024 01:08:35,720 --> 01:08:40,119 out myself. Maybe if that's our if that's our culture, and that's fine. 1025 01:08:40,279 --> 01:08:42,199 But you go to the ops person you say, hey, like we 1026 01:08:42,239 --> 01:08:45,279 need Mongo out lists with a new project, and it's like we do you 1027 01:08:45,319 --> 01:08:47,239 need Mongo at lists because they send you some marketing materials? Or do you 1028 01:08:47,279 --> 01:08:50,039 need mongo because we can throw Mango and a container for you? Like what 1029 01:08:50,039 --> 01:08:54,479 do you like? Why do you need at lists? And also like important 1030 01:08:54,520 --> 01:08:59,520 question like why not why not document dB? Yeah? In aws? Right, 1031 01:08:59,600 --> 01:09:02,840 like and the catches is like it's really funny. Like people anyone out 1032 01:09:02,840 --> 01:09:04,680 there is use document dB. I know, I know, I know, 1033 01:09:04,800 --> 01:09:09,960 chill out. I'm trying to make an example here. I'm I'm not a 1034 01:09:09,960 --> 01:09:14,159 document dB fanboy, but like reality is like there's a like document dB like 1035 01:09:14,239 --> 01:09:18,119 can have a pretty similar experience to Mongo Like there are places that like obviously 1036 01:09:18,159 --> 01:09:21,640 Mango's going to move faster in feature development and like the Mango space, yes, 1037 01:09:21,720 --> 01:09:25,760 but if you're not using you're just using some basic document store, like 1038 01:09:25,800 --> 01:09:30,479 you might not need everything that Mango has. And if it is, I 1039 01:09:30,479 --> 01:09:32,399 think Mango even publishes like a last time I checked, it was like either 1040 01:09:32,479 --> 01:09:36,760 like a list or tool that could show like what document dv didn't support. 1041 01:09:38,520 --> 01:09:41,239 Right, They're like, hey, like use us because this doesn't do this 1042 01:09:41,279 --> 01:09:44,279 one thing. It's like you don't need that operation now and you can't see 1043 01:09:44,319 --> 01:09:48,680 the chance you need that operation or configuration option in the future. Does it 1044 01:09:48,760 --> 01:09:55,640 benefit us to spend the money to have an enterprise account or whatever Mango atlas 1045 01:09:55,720 --> 01:09:58,760 where we'll bind to our vpc. We want to go through the effort of 1046 01:09:58,880 --> 01:10:01,840 peering our VPC with Mongo aut lists, Like do we want to go over 1047 01:10:01,840 --> 01:10:04,520 the public internet to hit them. Like there's a lot of questions in the 1048 01:10:04,520 --> 01:10:08,520 mind, and so like the gut reaction from the person is like why do 1049 01:10:08,560 --> 01:10:12,439 you need why do you need Mongo Outlets, which like it's very hard to 1050 01:10:12,560 --> 01:10:16,159 ask that question without sounding like you're criticizing the person that's made that decision, 1051 01:10:16,239 --> 01:10:20,119 right, And it's like, I'm not saying you're wrong, I'm just trying 1052 01:10:20,119 --> 01:10:23,439 to understand, Like I don't know if you know all the options that are 1053 01:10:23,439 --> 01:10:25,920 available, and I'm trying to figure out, like what is the right service 1054 01:10:25,960 --> 01:10:29,239 for us? Right, so, like can we can we try document dB 1055 01:10:29,399 --> 01:10:30,960 so it doesn't create a ton of work for us because it's if you need 1056 01:10:30,960 --> 01:10:34,760 document dB, I can throw together that terriform module. I'll skip lunch if 1057 01:10:34,760 --> 01:10:39,399 you do this favor. But if you need Mongo Atlas, when we have 1058 01:10:39,439 --> 01:10:43,760 to go sign to enterprise account, talk to procurement and do some VPC bindings, 1059 01:10:44,079 --> 01:10:46,640 maybe we have maybe we've made the dumb choice of having you know, 1060 01:10:46,760 --> 01:10:50,079 that that side or something that that makes it impossible for us to buy into 1061 01:10:50,119 --> 01:10:54,119 other ords it to appear to other people's networks, right, you don't know, 1062 01:10:54,279 --> 01:10:58,720 Like you don't know like how well or chaotic our VPC is, right, 1063 01:10:58,800 --> 01:11:00,279 So like there's there's other things that come to mind when we start talking 1064 01:11:00,319 --> 01:11:04,880 about using these. They says services that sit outside of AWS, and so 1065 01:11:04,960 --> 01:11:11,159 I feel like, like that that's just like a problem that exists like in 1066 01:11:11,239 --> 01:11:14,960 our in our collaboration, and I feel like that's another thing that like we 1067 01:11:15,039 --> 01:11:17,760 just need to be able to simplify, right, like being able to say, 1068 01:11:17,880 --> 01:11:20,760 like, these are the things that we support, you can quickly find 1069 01:11:20,800 --> 01:11:26,840 them, right, and be able to get new services into people's hands quickly 1070 01:11:27,119 --> 01:11:29,840 even to experiment. But like how do we get the time for that? 1071 01:11:30,119 --> 01:11:32,640 Right? Like I've had cases where people have asked or something like I could 1072 01:11:32,640 --> 01:11:38,760 throw together a document dB. Like this is a real scenario. I'm parroting 1073 01:11:38,760 --> 01:11:42,199 from a previous job. It's like I could have put a module together for 1074 01:11:42,239 --> 01:11:45,720 them in maybe an hour or two for them to try out document dB to 1075 01:11:45,720 --> 01:11:50,159 see if it's good enough. But I had too much work to do, 1076 01:11:50,680 --> 01:11:56,159 Like I didn't have enough time to do a little experiment to see if I 1077 01:11:56,159 --> 01:11:59,199 could save myself a bunch of time, a bunch of time, right, 1078 01:11:59,239 --> 01:12:00,079 And so it's just like okky, like they need this thing, and it 1079 01:12:00,119 --> 01:12:03,199 always seems like they needed now right. It's like, oh, like why 1080 01:12:03,199 --> 01:12:04,880 didn't this come up. It's like, well, we didn't know we need 1081 01:12:04,880 --> 01:12:09,399 document databases when we started the project. You were at that meeting. I 1082 01:12:09,520 --> 01:12:12,399 was at that meeting. I wasn't at subsequent ones because like there wasn't anything 1083 01:12:12,439 --> 01:12:15,760 that was needed for me or using postgrass got to the point where like somebody's 1084 01:12:15,760 --> 01:12:18,199 like, oh, we actually need Mango for this, and someone brew installed 1085 01:12:18,239 --> 01:12:21,680 Mango or they added it to their doctor composed file and they didn't think, 1086 01:12:21,760 --> 01:12:25,439 oh shit, somebody else over there is going to have to do work right, 1087 01:12:25,479 --> 01:12:28,119 So now I'm a bottleneck again. It's like I'm not a bottleneck. 1088 01:12:28,119 --> 01:12:30,800 It's that whole like, you know, lack of preparation on your part. 1089 01:12:30,840 --> 01:12:33,800 It's not an emergency on my part. Like an asshole for saying that, 1090 01:12:33,960 --> 01:12:36,880 right and sure, so like that's the thing that just kind of sucks, 1091 01:12:36,920 --> 01:12:45,279 like that collaboration is really really hard to do well for most organizations. Can 1092 01:12:45,319 --> 01:12:48,199 we jump to that thing that we were talking about earlier before the call started? 1093 01:12:48,239 --> 01:12:50,000 I want to share this story. Can we talk about do we have 1094 01:12:50,039 --> 01:12:54,159 time to talk about some IT stuff some some IT help est data stuff. 1095 01:12:54,560 --> 01:12:58,439 Yeah for sure, because we both we both started our careers in the help 1096 01:12:58,520 --> 01:13:01,840 desk. So help desk corse stories. Put take one, go for it 1097 01:13:02,239 --> 01:13:05,399 the one I wanted to share with you earlier. So my career. I 1098 01:13:05,439 --> 01:13:10,479 was originally a physics and an architecture major, but I was working at a 1099 01:13:10,520 --> 01:13:14,520 hospital and I was working in data entry. I'd been doing software development. 1100 01:13:14,640 --> 01:13:16,439 Was like fifteen or sixteen. I wrote a little program that did my job. 1101 01:13:16,840 --> 01:13:19,760 I did my data entry, so I just like read from Excel docs 1102 01:13:19,800 --> 01:13:27,600 and like input them into this like terminal system used for explanation of benefits for 1103 01:13:27,720 --> 01:13:32,920 the hospital I worked at. And when I automated my job, I wasn't 1104 01:13:32,920 --> 01:13:35,520 a software engineer, so I didn't sign like a PIA forum. I didn't. 1105 01:13:35,560 --> 01:13:39,000 They did not have rights to my software. I wrote it at home 1106 01:13:39,119 --> 01:13:41,840 like it was mine and say buy it from me, which is exciting. 1107 01:13:42,439 --> 01:13:46,079 So how I got into help desk was they offered me a thousand dollars for 1108 01:13:46,199 --> 01:13:49,439 my little program and a job on the help desk team. And I was 1109 01:13:49,479 --> 01:13:54,119 like, what, dude, pay me nine dollars an hour and give me 1110 01:13:54,159 --> 01:13:57,920 one thousand bucks. I'm eighteen. I'm like I'm rich. I literally I 1111 01:13:57,920 --> 01:14:00,239 took that thousand dollars, Like did you not I put a down payment on 1112 01:14:00,239 --> 01:14:04,640 a wave Runner like I was imagine I didn't have a place to park it. 1113 01:14:04,720 --> 01:14:08,199 I did not think this through. I lived in a dorm. I'm 1114 01:14:08,199 --> 01:14:11,319 like, what the hell am I gonna do with this wave Runner? But 1115 01:14:11,439 --> 01:14:15,520 I was just so stoked. I was on the help desk team, and 1116 01:14:15,640 --> 01:14:20,239 wow, like help desk is like we give those people so much shit, 1117 01:14:21,199 --> 01:14:25,479 Like I think of every support person at police. It's like it's so hard 1118 01:14:25,520 --> 01:14:30,239 not to be like especially like especially calling help deskers anything technical as a software 1119 01:14:30,239 --> 01:14:33,399 engineer, like yes, I know, I understand how to log and just 1120 01:14:33,520 --> 01:14:36,760 yeah, I know, like right, like and so it's frustrating. But 1121 01:14:36,840 --> 01:14:42,239 like those those like we're frustrated with them, but like their job is really 1122 01:14:42,279 --> 01:14:45,399 hard, like they do run into a bunch of people that are just not 1123 01:14:45,439 --> 01:14:47,600 familiar with computers all day long, and so they have to assume like they're 1124 01:14:47,720 --> 01:14:53,279 meeting more people like that than people that are right. Right, we had 1125 01:14:53,840 --> 01:15:00,239 seventeen offices and my second or seventeen offices across Florida, and so like, 1126 01:15:00,279 --> 01:15:04,119 aren't my my domain was pretty big, like to drive places all the time, 1127 01:15:04,640 --> 01:15:08,039 I had a couple of like data centers, like it was the first 1128 01:15:08,079 --> 01:15:10,239 one was in the building I was in. We have second one up in 1129 01:15:10,319 --> 01:15:15,640 uh, like Zephyr Hills, and so I'm in Fort Myersporters. So anybody 1130 01:15:15,680 --> 01:15:18,119 in Florida, Fort Myers or Zephyr Hills, imagine driving this distance for help 1131 01:15:18,159 --> 01:15:23,199 desk golf. So next time out that person there talking to you like you're 1132 01:15:23,239 --> 01:15:26,560 dumb, like this is this is their life. This is a multi hour 1133 01:15:26,640 --> 01:15:30,479 drive. And effectively what has happened is like there, gosh, I can't 1134 01:15:30,479 --> 01:15:34,359 remember what it was. Something was frozen on their on their computer, and 1135 01:15:34,359 --> 01:15:38,279 every time they rebooted it was frozen, and it was just like okay, 1136 01:15:38,840 --> 01:15:41,439 that's a heart and the troubleshoot, like I can't even like remote desk into 1137 01:15:41,479 --> 01:15:45,479 it, like because they rebooted it. And this is, by the way, 1138 01:15:45,479 --> 01:15:48,439 this is two thousand one or two thousand and two, so we didn't 1139 01:15:48,439 --> 01:15:54,319 have the great tools we have today. So I drive two hours to Zephyr 1140 01:15:54,399 --> 01:15:56,880 Hills. I go into the office and like, show me this computer, 1141 01:15:57,119 --> 01:16:00,039 and sure shit, I sit down at it the screen, I grabbed the 1142 01:16:00,039 --> 01:16:05,560 mouse and go to click to log in. The mouse I hit the tab 1143 01:16:05,600 --> 01:16:09,479 button and it tabs into the username field and I'm like, it's not frozen. 1144 01:16:09,479 --> 01:16:11,680 That's interesting. So that I put in like one of the AVON passers. 1145 01:16:11,680 --> 01:16:14,199 I log in and I'm like, okay, I grab the mouse and 1146 01:16:14,239 --> 01:16:18,359 move it again. Nothing and I'm like. The first thing I do is 1147 01:16:18,399 --> 01:16:24,800 I look under there's this big tower. The mouse is unplugged. My ve 1148 01:16:25,079 --> 01:16:28,279 driven two hours for an unplugged mouse, and I have to drive two hours 1149 01:16:28,319 --> 01:16:32,960 home. And I looked down and I'm just like and so I plug it 1150 01:16:33,000 --> 01:16:35,439 back in. It works, And she was like, how did you fix 1151 01:16:35,479 --> 01:16:40,079 it? I'm like, the mouse, the mouse, and this is this 1152 01:16:40,119 --> 01:16:43,239 isn't like a USB mouse is like one of those old like DIN connector things 1153 01:16:43,239 --> 01:16:46,880 like real hard to yank this thing out. So what I was able to 1154 01:16:46,880 --> 01:16:50,319 tell from looking at it as well was it was pulled out with some force. 1155 01:16:50,399 --> 01:16:56,199 So the assumption is it got wrapped around her rank ankle and pull out 1156 01:16:56,199 --> 01:16:59,399 of the machine some of those sliding things right here, but it slid out 1157 01:16:59,399 --> 01:17:01,800 with like the keyboard right in the mouse right like there was some serious force 1158 01:17:01,880 --> 01:17:04,600 pulled this thing out. Someone must have known, like I feel like a 1159 01:17:04,640 --> 01:17:09,600 trick must have happened. But yeah, that was that was one of my 1160 01:17:09,960 --> 01:17:14,399 that's one of my favorite ones. That was a little fun. My favorite 1161 01:17:14,439 --> 01:17:17,479 one that was not fun. This is second second. I ended up kind 1162 01:17:17,520 --> 01:17:21,960 of focusing in healthcare once I changed careers. So my master's is in healthcare 1163 01:17:23,000 --> 01:17:27,159 information system, so work for another company. And we had these two offices 1164 01:17:29,039 --> 01:17:31,960 that were gosh, I don't want to give away, Well, I guess 1165 01:17:31,920 --> 01:17:35,720 I have to give it away. These two offices that we're connected to the 1166 01:17:35,920 --> 01:17:40,399 data center here and literally across the street was one of our heart surgery centers. 1167 01:17:41,000 --> 01:17:44,760 And this is two thousand and three or so, and like we're pretty 1168 01:17:44,800 --> 01:17:47,119 far along, Like we've got Wi Fi all over the place. We've got 1169 01:17:47,159 --> 01:17:51,439 a like a digital pills system for dispersing like pills in our pharmacy. Like 1170 01:17:51,479 --> 01:17:56,800 we've heavily invested in technology. We had a really great engineering budget. We 1171 01:17:56,840 --> 01:18:00,359 had badges blogging the machines, which is a nightmare for him. We got 1172 01:18:00,359 --> 01:18:04,920 to find so much from doctors just leaving their batch's places by like a hip 1173 01:18:04,920 --> 01:18:10,680 a shopper who came in and like the doctor left the card and they logged 1174 01:18:10,680 --> 01:18:13,520 in and just accessed a bunch of shit. It's just like kill me. 1175 01:18:14,439 --> 01:18:17,319 But my boss was, this is like in the days when you could haze 1176 01:18:17,520 --> 01:18:19,800 new engineers. I don't know if you ever do you ever get hazed? 1177 01:18:20,880 --> 01:18:27,239 Yeah time or two? Yeah, progressing from help desk into like the network 1178 01:18:27,239 --> 01:18:30,279 operations, like working on data center. And so we had these two sites 1179 01:18:30,640 --> 01:18:33,439 and it was going to cost us something insane like one hundred and twenty thousand 1180 01:18:33,520 --> 01:18:41,079 dollars or something wild to run internet like under the road to our other office. 1181 01:18:41,560 --> 01:18:44,239 And we didn't want them like using dial up or anything like that to 1182 01:18:44,319 --> 01:18:46,159 like dial in heart surgery center, Like we need this thing to be online. 1183 01:18:46,640 --> 01:18:50,159 And so we did is we installed microwave dishes on the roofs pointed at 1184 01:18:50,159 --> 01:18:55,199 each other, and we had point to point microwave internet between between these two 1185 01:18:55,520 --> 01:19:00,279 offices. And they started having outages and they were very intermittent. They're very 1186 01:19:00,399 --> 01:19:03,039 random, and they're very brief, but like brief enough that it would like 1187 01:19:03,119 --> 01:19:06,840 stop like an upload or you know, like some sort of scan upload they're 1188 01:19:06,840 --> 01:19:12,279 scanning back to like our medical record system. And so we thought it was 1189 01:19:12,319 --> 01:19:15,359 birds. We thought birds had gotten on the on the things. So my 1190 01:19:15,399 --> 01:19:17,600 boss is like, like, we we like how we can fix this. 1191 01:19:17,600 --> 01:19:19,960 I'm gonna throw some of those little bird things on there, like stop them 1192 01:19:19,960 --> 01:19:25,479 fronts. I don't know what putting those on a microwave dish actually would have 1193 01:19:25,520 --> 01:19:28,039 done. And he's like, what's confirm that birds? First? I'm like, 1194 01:19:29,520 --> 01:19:32,640 what are we to ask them? Like? He's like uh no. 1195 01:19:33,000 --> 01:19:34,399 And I was like, okay, well, what are you gonna do? 1196 01:19:34,600 --> 01:19:38,680 And he's like come here and he walks out to his truck and I should 1197 01:19:38,720 --> 01:19:40,840 you not. He's got one of those umbrellas with a stand on it, 1198 01:19:41,560 --> 01:19:45,600 a lawn chair, and then he had to walking talkies and I'm like, 1199 01:19:45,399 --> 01:19:50,239 he's like, you're gonna this is Florida the summer. He's like, you're 1200 01:19:50,239 --> 01:19:55,000 gonna sit on the roof and when they have an outage, I'm gonna call 1201 01:19:55,000 --> 01:20:00,359 you on the walkie talkie and you're gonna tell me what you see. Plus 1202 01:20:00,680 --> 01:20:02,119 So I'm sitting up there for like six hours and all of a sudden, 1203 01:20:02,680 --> 01:20:08,359 walkie talkie, hey h surgery centers down what's going on? And I'm looking 1204 01:20:08,399 --> 01:20:11,520 at I'm looking at the thing and I'm like, dude, there's no birds 1205 01:20:11,560 --> 01:20:14,119 on this thing. Like I'm looking at like there's there's no birds like I 1206 01:20:14,159 --> 01:20:15,680 don't know why. I still there's no birds, and so like out my 1207 01:20:15,720 --> 01:20:19,039 little like fox hound thing, I can still see that there's like signal going 1208 01:20:19,079 --> 01:20:25,239 through the wire. There's just out and then all of a sudden it's back 1209 01:20:25,319 --> 01:20:30,119 up, and I'm like he's back up. I'm like, I'm stumped, 1210 01:20:30,479 --> 01:20:33,479 Like I'm stump, like this was not in the net plus book. This 1211 01:20:33,640 --> 01:20:36,359 wasn't in the A plus book, Like I got it, I got an 1212 01:20:36,479 --> 01:20:41,239 MC. I see this was not on the test right like, And so 1213 01:20:41,279 --> 01:20:44,039 I'm sitting there for like a while, and maybe like two or three hours 1214 01:20:44,119 --> 01:20:46,439 later, goes out again, and I'm like, maybe it's a burn on 1215 01:20:46,479 --> 01:20:49,560 the other side, Like maybe it's a burn on the receiver. That didn't 1216 01:20:49,600 --> 01:20:51,319 even think about this. I don't know if everybody else is like, it's 1217 01:20:51,319 --> 01:20:56,279 a burn on the other receiver, you idiot. And so I look across 1218 01:20:56,319 --> 01:21:00,159 the street and I don't see the satellite, but I see as a semi 1219 01:21:00,199 --> 01:21:06,800 truck parked on a restante. Like the other building was a house that was 1220 01:21:06,800 --> 01:21:12,279 converted into a surgery center, and we were in this essentially this strip mall 1221 01:21:12,640 --> 01:21:16,239 that had been converted into our data center and like our primary care physician's office. 1222 01:21:16,279 --> 01:21:19,479 So this thing's like thirty forty feet tall, and this house is you 1223 01:21:19,479 --> 01:21:23,600 know however tall houses, and so the beams just hitting the side of this 1224 01:21:23,640 --> 01:21:31,520 truck probably packed it tons of stuff, and so with longer polls. Yeah, 1225 01:21:31,560 --> 01:21:35,039 meanwhile, the driver who had frozen pizzas in his truck shows up with 1226 01:21:35,079 --> 01:21:40,640 fully cooked pizzas at the destination of just like blast and I don't I don't 1227 01:21:40,680 --> 01:21:44,920 even remember how faster that was with was the one hundred megabit I think back 1228 01:21:44,960 --> 01:21:46,840 then for like a network. I don't know what. I don't know what 1229 01:21:46,920 --> 01:21:50,279 the satellite, a little satellite this thing did. But yeah, he's like 1230 01:21:50,319 --> 01:21:56,840 I got fourteen cooked pizzas and a brain tumor or anything else, uh helped 1231 01:21:56,880 --> 01:21:58,920 us. I don't know, Like I kind of want maybe when I retire, 1232 01:21:58,920 --> 01:22:00,640 I'll just go back and work in it for a year where it's like 1233 01:22:00,720 --> 01:22:02,880 it's probably a lot less stressful when you don't need to do it to pay 1234 01:22:02,920 --> 01:22:06,640 your bills, Like I bet it becomes pretty fun. Then it's just like, 1235 01:22:06,680 --> 01:22:11,239 okay, like what kind of weird shit do y' all have for me 1236 01:22:11,359 --> 01:22:17,000 today? Right to see how far you can push the envelope? Oh my 1237 01:22:17,039 --> 01:22:20,319 gosh, but do you have a good help desk color story. Yeah. 1238 01:22:20,399 --> 01:22:30,279 So the first help desk job I had was working for a manufacturing company that 1239 01:22:30,319 --> 01:22:35,359 had a huge sales department selling their product. And you know, this was 1240 01:22:36,279 --> 01:22:49,640 this was like nine two thousand era, and so they the salespeople were just 1241 01:22:49,800 --> 01:22:55,000 coming out of the stage of having their roll of decks where they would pick 1242 01:22:55,039 --> 01:22:58,479 up the phone and dial and then write down in a notebook what they were 1243 01:22:58,479 --> 01:23:01,479 doing. So they were trying to get them to you CRM software, and 1244 01:23:01,760 --> 01:23:05,520 at the same time they had to build computer skills, you know. But 1245 01:23:05,640 --> 01:23:09,800 I get this call from this sales guy. He's like, hey, I'm 1246 01:23:09,800 --> 01:23:14,439 out of disk base. I can't save anything on my computer. And I 1247 01:23:14,520 --> 01:23:16,640 had the ability to check their disk base and I looked and I'm like, 1248 01:23:16,720 --> 01:23:19,800 now you've got You've got plenty of disk space. He's like, look, 1249 01:23:19,800 --> 01:23:24,560 I'm telling you I cannot save anything on my computer. I don't know what 1250 01:23:24,600 --> 01:23:28,760 your little, your little magic thing down there does, but it's wrong because 1251 01:23:28,760 --> 01:23:32,319 I'm telling you I cannot save anything on my computer. Like, all right, 1252 01:23:32,680 --> 01:23:34,920 I'll come up. So I'll go up there, up on the third 1253 01:23:34,920 --> 01:23:40,239 floor. I go up there, find him in this sea of cubicles, 1254 01:23:40,560 --> 01:23:44,680 like, Okay, what's going on? And he's like look, and he 1255 01:23:44,760 --> 01:23:50,079 shows me his desktop, and on his desktop he had icons for every saved 1256 01:23:50,520 --> 01:23:55,319 document he had sent to a customer, lined up from left to right, 1257 01:23:57,239 --> 01:24:00,359 and there was not room to put another icon there. He's like, I 1258 01:24:00,399 --> 01:24:02,359 can't save it. There's no place to put it. Yeah, he was, 1259 01:24:02,479 --> 01:24:05,279 he was. He was geographically out of space, That's what he was 1260 01:24:05,279 --> 01:24:09,399 trying to say. Right, I was writing down what I thought it was, 1261 01:24:09,439 --> 01:24:14,479 and that was not it. You guess. I was guessing he had 1262 01:24:14,479 --> 01:24:15,840 the hard drive you're seeing that, but he was trying to save something to 1263 01:24:15,880 --> 01:24:24,119 a floppy diss Yeah, it was too big, but it definitely wasn't the 1264 01:24:24,159 --> 01:24:31,920 grid on Windows ninety eight. Oh my gosh. It is fun. It 1265 01:24:32,039 --> 01:24:35,039 is fun, but it is also frustrating. So be nice to your support 1266 01:24:35,039 --> 01:24:42,279 people. Yeah. Absolutely, Like with a grain of salt, understand like 1267 01:24:43,159 --> 01:24:45,319 who they might have been talking to just before they got on the phone with 1268 01:24:45,359 --> 01:24:48,319 you. It's so hard. It's hard, like too when it's just like, 1269 01:24:48,399 --> 01:24:53,600 oh man, like like making it. I always felt bad when somebody's 1270 01:24:53,640 --> 01:24:55,960 like I feel dumb, and it's like, I mean, like I don't 1271 01:24:56,000 --> 01:24:59,239 know I'd feel pretty dumb if I was doing open heart surgery, like I'd 1272 01:24:59,279 --> 01:25:04,640 fucked out, like like you shouldn't have to know. But uh, oh 1273 01:25:04,720 --> 01:25:11,680 gosh, that's funny. Cool. Well, Corey, it's been great having 1274 01:25:11,720 --> 01:25:14,680 you on the show Man. This has been a blast. I have had 1275 01:25:14,680 --> 01:25:18,039 a great time. This is super fun cool. Well, I'm happy to 1276 01:25:18,039 --> 01:25:23,640 have you back on anytime, because I'm sure we've got more stories, especially 1277 01:25:23,760 --> 01:25:28,760 healthcare stories. We could go down the healthcare path. I worked as as 1278 01:25:28,800 --> 01:25:34,680 a support operations engineer for a teleradiology company, and there's just so many stories 1279 01:25:34,720 --> 01:25:39,359 coming out of that that are just too funny. And I think it's been 1280 01:25:39,439 --> 01:25:45,479 long enough now where like any like anything I could get sued for saying has 1281 01:25:45,560 --> 01:25:50,119 past, so I could tell those stories now. Oh my gosh, I 1282 01:25:50,159 --> 01:25:56,720 have I have one that can't be told on camera. That is right mind 1283 01:25:56,800 --> 01:25:59,239 boggling. I don't know if, like if you had this experience. But 1284 01:25:59,359 --> 01:26:02,399 doctors, doctors all think they're God, or at least most of the ones 1285 01:26:02,399 --> 01:26:06,039 that I met, Yeah, when they're doctors that like own like that. 1286 01:26:06,159 --> 01:26:10,359 I work with a big practice and the doctors were all like board members and 1287 01:26:10,399 --> 01:26:14,680 shareholders of the practice, like ooh, that ego is gets big, but 1288 01:26:14,760 --> 01:26:18,159 we had one where his ego was big and his sense of humor was brutally 1289 01:26:18,239 --> 01:26:23,800 messed up. Yeah I'll hear that story once, but not streaming live anymore 1290 01:26:23,840 --> 01:26:29,880 because it is not it is beyond rated. Are right on, And that's 1291 01:26:29,920 --> 01:26:32,680 a segue for our new podcast, Things we can't say on the air, 1292 01:26:32,840 --> 01:26:40,319 So, oh my gosh, listen. No, sincerely, thank you. 1293 01:26:40,479 --> 01:26:43,039 I've had a blast. I appreciate you coming on the show, and thanks 1294 01:26:43,039 --> 01:26:45,439 to everyone who hung out for us this entire time. I hope it was 1295 01:26:45,560 --> 01:26:49,079 helpful, entertaining, and I hope to see you all next week. Yeah, 1296 01:26:49,119 --> 01:26:51,600 if it wasn't helpful, I really hope it was entertaining. That's like 1297 01:26:51,600 --> 01:26:58,239 it was over in life and life. But they're always competing, and yeah, 1298 01:26:58,279 --> 01:26:59,960 if you could just hit one of them every once in a while, 1299 01:27:00,039 --> 01:27:06,359 all that's that's winning. If I give somebody a ball, alright, alright, 1300 01:27:06,399 --> 01:27:11,479 see everyone, Hm