WEBVTT 1 00:00:02.000 --> 00:00:04.679 For real this time, Warren, sound check. 2 00:00:05.639 --> 00:00:06.519 I think I'm still here. 3 00:00:07.120 --> 00:00:12.359 All right, you are welcome, glad to have you. Jillian, Hello, Hello, 4 00:00:12.960 --> 00:00:19.039 welcome back after your time jet setting around the world 5 00:00:19.079 --> 00:00:19.960 or whatever you were doing. 6 00:00:20.679 --> 00:00:22.440 I was helping somebody move last week. 7 00:00:22.519 --> 00:00:25.120 So now that I like live back home my family, 8 00:00:25.199 --> 00:00:27.160 I keep on being expected to be like a real 9 00:00:27.199 --> 00:00:28.320 adult who shows. 10 00:00:28.120 --> 00:00:28.559 Up for them. 11 00:00:31.399 --> 00:00:34.759 It's been quite the transition for me. So there we go. 12 00:00:36.280 --> 00:00:39.759 And then joining us today, Adriana. I forgot to ask 13 00:00:39.799 --> 00:00:44.200 how to pronounce your last name Valleyla Valela Valela. Oh 14 00:00:44.280 --> 00:00:47.320 you were close everybody that's an e in there. Yeah, yeah, 15 00:00:47.359 --> 00:00:50.840 I should probably have my glasses checked. But hey, welcome. 16 00:00:50.880 --> 00:00:55.560 I'm glad to have you here, happy to here right on. 17 00:00:55.960 --> 00:00:58.399 So the big thing I want to shout out right 18 00:00:58.439 --> 00:01:01.560 away is you are the host of the Geeking Out podcast. 19 00:01:02.359 --> 00:01:05.280 I am right on, so tell us how that's going. 20 00:01:06.280 --> 00:01:13.159 So I started the podcast, say twenty twenty three. I 21 00:01:13.280 --> 00:01:16.239 was in the fall of twenty twenty three. It came 22 00:01:16.400 --> 00:01:18.319 on the heels of a previous podcast that I was 23 00:01:18.359 --> 00:01:22.959 doing through work with former coworker of mine on the 24 00:01:23.040 --> 00:01:26.239 Margarita Medina and Our podcast. And this was a work 25 00:01:26.280 --> 00:01:29.480 related podcast, and you had the best name. It was 26 00:01:29.519 --> 00:01:36.120 called on Call Me Maybe. Damn it. It was like 27 00:01:36.159 --> 00:01:39.359 so much fun. We had about two seasons of it. 28 00:01:39.400 --> 00:01:42.159 I want to say it was like about twenty six episodes, 29 00:01:42.680 --> 00:01:46.640 and then we were no longer able to continue it. 30 00:01:46.680 --> 00:01:49.200 So then I thought, well, I want to still keep podcasting, 31 00:01:49.239 --> 00:01:52.920 so I started my own podcast, Geeking Out. And then 32 00:01:52.959 --> 00:01:55.319 I was because we used to have like an editor 33 00:01:55.439 --> 00:01:57.079 for on Call Me Maybe, and I'm like, damn it, 34 00:01:57.120 --> 00:01:59.079 I don't know how to do any of this podcast 35 00:01:59.200 --> 00:02:02.920 editing stuff. And my daughter, who I guess was fourteen 36 00:02:02.959 --> 00:02:04.879 at the time, she's like, I'll help you out, mom. 37 00:02:05.280 --> 00:02:08.479 So I'm like, okay. So she helps me out with 38 00:02:08.680 --> 00:02:12.319 the editing. I've up to my editing game as well. 39 00:02:12.319 --> 00:02:16.199 But she's also designed the logo for it, which has 40 00:02:16.280 --> 00:02:19.599 copy batas, which I love. I don't know. They're just 41 00:02:20.000 --> 00:02:23.840 fun animals, and I got introduced to them on Instagram. 42 00:02:24.120 --> 00:02:26.280 I started like getting all these videos. I'm like, oh 43 00:02:26.280 --> 00:02:28.439 my god, where have you been all my life? So anyway, 44 00:02:28.360 --> 00:02:33.639 they're like our mascot, so cute. They're doable because I 45 00:02:34.439 --> 00:02:39.479 just scored a couple at Miniso. I don't know if 46 00:02:39.479 --> 00:02:42.719 you guys have that in the States. But yeah, it 47 00:02:42.759 --> 00:02:45.560 was Yeah. I was like, oh my god, this is 48 00:02:45.599 --> 00:02:50.639 the best. And yeah, and the podcast itself I have. 49 00:02:51.080 --> 00:02:54.400 It's a tech podcast. I interview a lot of folks 50 00:02:54.479 --> 00:02:58.360 in tech. I especially like to give voices to women 51 00:02:58.599 --> 00:03:03.039 and other under ruppers groups, and of had a I 52 00:03:03.039 --> 00:03:06.560 don't know a combination of people who are both super 53 00:03:06.560 --> 00:03:09.759 well known in the industry and not so well known. 54 00:03:10.680 --> 00:03:14.120 So I guess my highest profile guest was Kelsey high Tower. 55 00:03:15.360 --> 00:03:20.000 I've had Charity Majors, Liz Fung, Jones, Hazel Weekly, so 56 00:03:20.240 --> 00:03:22.960 lots of lots of fun guests and then other people 57 00:03:23.000 --> 00:03:25.520 that I've met along the way where I'm like, you 58 00:03:25.560 --> 00:03:27.639 have such a cool story, you should be on my podcast. 59 00:03:28.919 --> 00:03:31.759 Right, Like, once you start a podcast, like you're always 60 00:03:32.080 --> 00:03:35.439 it feels like a sales role, like always be closing. 61 00:03:35.840 --> 00:03:38.240 You know, you're always like trying to pull people into 62 00:03:38.280 --> 00:03:38.719 the show. 63 00:03:39.560 --> 00:03:40.240 So true. 64 00:03:41.439 --> 00:03:47.159 True, So you're also a CNCF ambassador and principal developer 65 00:03:47.400 --> 00:03:52.879 advocate at Dina Trace. That's correct, right on, Dina Trace 66 00:03:52.960 --> 00:03:55.560 is cool. Like some of the stuff that they expose 67 00:03:55.599 --> 00:03:58.520 and dig into, it's like, wow, you you went way 68 00:03:58.520 --> 00:04:00.240 deeper into this than I wanted to go. 69 00:04:01.879 --> 00:04:04.680 It's so true and it's funny. So I just I 70 00:04:04.800 --> 00:04:08.199 just started my job at Dina Trace in November of 71 00:04:08.280 --> 00:04:12.840 last year, so I'm pretty fresh, and you know, I 72 00:04:12.840 --> 00:04:17.560 I I came in because of my connection to the 73 00:04:17.639 --> 00:04:20.519 open telemetry community and my previous job. I was a 74 00:04:20.560 --> 00:04:23.959 developer advocate at light Step, which had gotten acquired by 75 00:04:24.000 --> 00:04:27.199 a service now, so I'd gotten into the open telemetry 76 00:04:27.240 --> 00:04:30.120 and observability community and and so when I joined Dina Trace, 77 00:04:30.160 --> 00:04:33.480 I'm like, well, you know, it's like also an observability vendor, 78 00:04:33.519 --> 00:04:35.360 but it's like so much more. And so one of 79 00:04:35.399 --> 00:04:39.199 my one of my co workers, h Andy Grabner, he's 80 00:04:39.240 --> 00:04:41.399 been with Dina Trace forever. I call him mister Dina 81 00:04:41.480 --> 00:04:46.040 Trace because he's like he's so he's so passionate about 82 00:04:46.120 --> 00:04:49.360 the platform and what we do and and everything. And 83 00:04:49.399 --> 00:04:51.839 he's really helped me. Uh He's given me a tour 84 00:04:51.920 --> 00:04:56.079 of the platform, and we have this video series that started. 85 00:04:56.120 --> 00:04:58.800 It started because he was like, hey, you know, it 86 00:04:58.839 --> 00:05:00.319 would be good for you to like get to know 87 00:05:00.399 --> 00:05:03.480 the platform. And every time he showed me new stuff, 88 00:05:03.480 --> 00:05:06.279 I'm like, dude, this stuff is so cool. You know 89 00:05:06.319 --> 00:05:07.959 it would be really fun if we did like a 90 00:05:07.959 --> 00:05:11.120 YouTube reaction video series kind of thing where you just 91 00:05:11.399 --> 00:05:13.240 show me stuff for the first time and I react 92 00:05:13.279 --> 00:05:16.639 to it. So then we started this video series called 93 00:05:16.720 --> 00:05:19.199 Dina Trace. Can do that with open telemetry, and my 94 00:05:19.639 --> 00:05:23.279 reactions are are like O natural because I can't act. 95 00:05:23.360 --> 00:05:26.000 So it's been it's been a fun way to learn 96 00:05:26.120 --> 00:05:29.959 about what the product does and also like share share 97 00:05:30.000 --> 00:05:33.279 that same wonder with the rest of the world. 98 00:05:33.439 --> 00:05:37.600 So yeah, yeah, Dina Trace is like a gateway drug, 99 00:05:37.800 --> 00:05:40.240 Like you come for the observability and then you're like, 100 00:05:40.360 --> 00:05:44.000 oh wow, and then just go deep down the rabbit hole. 101 00:05:44.360 --> 00:05:47.480 That name sounds familiar. It was was Grabner on our 102 00:05:47.480 --> 00:05:48.240 show already. 103 00:05:48.759 --> 00:05:53.879 He was, yeah, I think it's been Actually, I don't 104 00:05:53.879 --> 00:05:56.160 know how long ago. I had this conversation with my 105 00:05:56.439 --> 00:05:58.480 I have this conversation with my wife the other day. 106 00:05:58.519 --> 00:06:02.040 Like I have three time periods in my life. I 107 00:06:02.079 --> 00:06:07.600 can group things like prior to nineteen ninety, between nineteen 108 00:06:07.680 --> 00:06:11.199 ninety and yesterday and today, and I can't get any 109 00:06:11.199 --> 00:06:12.240 more granular than that. 110 00:06:13.600 --> 00:06:16.199 I think that's one up on most of the population, though, 111 00:06:18.040 --> 00:06:22.680 like what happened five minutes ago and that's it, right, So. 112 00:06:22.720 --> 00:06:26.920 True Also, I thought nineteen ninety was like yesterday, Like 113 00:06:27.040 --> 00:06:28.279 where did this time go? 114 00:06:28.519 --> 00:06:30.279 Don't do the math, don't do the math. It's just 115 00:06:30.360 --> 00:06:31.319 gonna do no. 116 00:06:31.319 --> 00:06:31.480 No. 117 00:06:32.000 --> 00:06:34.600 It really depresses me when people are like so I 118 00:06:34.639 --> 00:06:36.800 was like born in nineteen ninety three. I'm like, oh, 119 00:06:36.879 --> 00:06:39.360 I was in my first year of high school. Cool. 120 00:06:44.120 --> 00:06:49.079 Cool. So we were going to talk today about observability 121 00:06:49.120 --> 00:06:52.839 in the CICD platform, So tell me a little bit 122 00:06:52.839 --> 00:06:53.839 about what that means to you. 123 00:06:55.160 --> 00:06:58.480 So, I guess the main thing is, I think when 124 00:06:58.519 --> 00:07:03.560 we talk about observability, there's thillse this uh stigma. I 125 00:07:03.560 --> 00:07:05.199 guess I don't know if stigma is the right word, 126 00:07:05.199 --> 00:07:07.920 but we have this association that observability is like an 127 00:07:08.000 --> 00:07:12.240 essry concern, right, because that's that's where when when things 128 00:07:12.279 --> 00:07:16.120 go kaka in prad, you know, you you turn to 129 00:07:16.199 --> 00:07:19.839 your observability solution and look through you know, the traces, 130 00:07:19.839 --> 00:07:21.639 the logs and metrics to figure out what's going on 131 00:07:22.120 --> 00:07:26.600 and all that, which is awesome, but you know, it's 132 00:07:26.360 --> 00:07:28.680 it's so much more than that, because, first of all, 133 00:07:28.759 --> 00:07:33.360 like it, observability is a team sport, right in order 134 00:07:33.439 --> 00:07:37.040 for that telemetry to even get emitted in the first 135 00:07:37.040 --> 00:07:42.720 place and it Yes, there's some telemetry that you automatically 136 00:07:42.920 --> 00:07:46.319 pick up from from your infrastructure and whatnot, but there's 137 00:07:46.360 --> 00:07:51.120 also the telemetry that has to be written by somebody, right, 138 00:07:51.480 --> 00:07:56.600 so you're you're developers and and so the developers have 139 00:07:56.680 --> 00:07:59.560 to care about observability, right, So now we're we're not 140 00:07:59.600 --> 00:08:03.040 seeing observability is just like, oh, it's not just a 141 00:08:03.120 --> 00:08:06.600 necessary thing. Someone had to put in that telemetry in 142 00:08:06.720 --> 00:08:10.279 order for us to be able to even have this 143 00:08:10.360 --> 00:08:17.160 conversation of understanding what our system's doing. And we can 144 00:08:17.199 --> 00:08:20.920 take it a step further and say, well, you know 145 00:08:22.160 --> 00:08:27.279 what if during the software development life cycle, like you know, 146 00:08:27.759 --> 00:08:31.800 developers instrument their code. This gets handed off to QA, 147 00:08:32.159 --> 00:08:36.679 and QA can use the observability data and say, hey, 148 00:08:37.720 --> 00:08:40.759 I can use this to troubleshoot my code or to 149 00:08:40.799 --> 00:08:44.480 troubleshoot the code that I'm testing, and I can provide 150 00:08:44.519 --> 00:08:48.320 that feedback then to developers and say hey, I found 151 00:08:48.360 --> 00:08:50.399 a bug and this is what's happening. Or they can 152 00:08:50.440 --> 00:08:53.039 say hey, I found a bug. I don't know why 153 00:08:53.080 --> 00:08:56.360 this is happening. You need to instrument better. So now 154 00:08:56.600 --> 00:09:00.919 we're like we're shifting, We're shifting left right on observability. 155 00:09:01.039 --> 00:09:04.120 So we've got so we've got the development side, we've 156 00:09:04.120 --> 00:09:07.399 got the qaside, we've got the production side with with 157 00:09:07.480 --> 00:09:12.279 our sries and whatnot. But then there's that also piece 158 00:09:12.320 --> 00:09:15.399 in the middle, which is, you know, our CICD pipelines 159 00:09:15.480 --> 00:09:20.000 that we've come to rely heavily on our CICD pipelines 160 00:09:20.639 --> 00:09:23.679 to ensure that our code gets built and our automatic 161 00:09:24.000 --> 00:09:28.720 automated tests get run and gets deployed to production. Awesome, 162 00:09:29.200 --> 00:09:33.039 but like what happens when that pipeline goes KACA Because 163 00:09:33.159 --> 00:09:37.799 that pipeline, even though it's internal in itself, it is 164 00:09:37.879 --> 00:09:43.720 a production system. So how frustrating is it when you've 165 00:09:43.720 --> 00:09:46.200 got you know, your CICD pipeline is working like a 166 00:09:46.200 --> 00:09:49.080 well oiled machine, and you're like, this is amazing, And 167 00:09:49.120 --> 00:09:54.080 then you come to realize that suddenly one day something 168 00:09:54.120 --> 00:09:57.000 goes weird, some change was made, and you have no 169 00:09:57.120 --> 00:09:59.879 idea why it's failing, And wouldn't it be nice if 170 00:09:59.919 --> 00:10:05.200 we could also have observability into our CIICD pipelines. And 171 00:10:05.279 --> 00:10:08.639 so we are starting to see a movement in that direction, 172 00:10:09.240 --> 00:10:13.320 which is amazing because now we're no longer in the 173 00:10:13.399 --> 00:10:18.639 dark around around our CICD pipeline. So this is for me. 174 00:10:18.720 --> 00:10:21.759 I think this is a really fascinating topic. I dug 175 00:10:21.799 --> 00:10:24.799 into it a little bit a couple of years back. 176 00:10:26.000 --> 00:10:29.879 So I have I have this video course that I 177 00:10:29.919 --> 00:10:33.519 did with O'Reilly that it came out last year early 178 00:10:33.600 --> 00:10:36.360 last year, and as part of it, I'm like, hey, 179 00:10:36.399 --> 00:10:39.440 I want to do something on observability CICD pipelines. Is 180 00:10:39.480 --> 00:10:43.039 like just a short chapter on that. And then I'm 181 00:10:43.080 --> 00:10:45.519 like starting to do some research. I'm like, crap, there's 182 00:10:45.600 --> 00:10:48.840 like nothing on this, what the hell? And then I 183 00:10:48.919 --> 00:10:51.440 was I was messaging one of my friends in open Telemetry. 184 00:10:51.440 --> 00:10:55.080 We we're both maintainers of the Open Telemetry and User 185 00:10:55.120 --> 00:10:58.399 SIG and we've done a bunch of talks together. She's 186 00:10:58.720 --> 00:11:02.279 she's my hotel writer. Recently, we we we talked at 187 00:11:02.360 --> 00:11:04.799 keep Con together all the time. It's like it's a 188 00:11:04.840 --> 00:11:07.879 great partnership. And I'm messengering her. I'm like, you know, 189 00:11:08.799 --> 00:11:10.840 there's really not a lot of material out there on 190 00:11:11.200 --> 00:11:14.799 the observability of c ICD pipelines. She's like, that could 191 00:11:14.799 --> 00:11:18.440 be a really good talk topic. I'm like, that's so awesome. 192 00:11:18.519 --> 00:11:21.360 So we we put together we put together a talk 193 00:11:22.120 --> 00:11:24.279 I think it was I want to say it was 194 00:11:26.039 --> 00:11:31.000 I want to say it was cubecan Chicago in twenty 195 00:11:31.039 --> 00:11:33.919 twenty three that we we talked. We did that talk 196 00:11:33.960 --> 00:11:40.039 for Observability Day, and it's been nice to see that 197 00:11:40.159 --> 00:11:45.720 space evolve over time. I think there's now an official 198 00:11:46.120 --> 00:11:51.480 like CICD SIG within open Telemetry I want to say, 199 00:11:51.840 --> 00:11:56.519 so then there's actual like movement towards standardizing the observability 200 00:11:56.559 --> 00:11:59.519 around CICD, because at the time when we were we 201 00:11:59.519 --> 00:12:03.159 were doing the stuff, like when we were investigating, it 202 00:12:03.200 --> 00:12:06.200 was this mish mash of tools that were available. So like, 203 00:12:06.240 --> 00:12:10.799 for example, depending on what tool used for CICD. So 204 00:12:10.960 --> 00:12:14.799 Jenkins for example, had some observability capabilities built in, so 205 00:12:14.840 --> 00:12:19.600 it did emit some like hotel some Hotel signals. But 206 00:12:19.679 --> 00:12:24.039 then get Hub if you wanted to have like observable 207 00:12:24.080 --> 00:12:27.960 get hub actions, there's some like homegrown options. But now 208 00:12:28.000 --> 00:12:31.360 that means you're having to rely on someone else maintaining 209 00:12:31.399 --> 00:12:34.600 that if you want to have observable CICD pipelines, And 210 00:12:34.639 --> 00:12:37.039 then what if they stopped maintaining those get hub acts, 211 00:12:37.159 --> 00:12:39.120 are they discontinue? 212 00:12:39.159 --> 00:12:39.320 Yeah? 213 00:12:39.440 --> 00:12:41.840 Right, So like that that's a little bit scary if 214 00:12:41.840 --> 00:12:44.320 you want to rely on that get lab. At the 215 00:12:44.360 --> 00:12:48.720 time when we were investigating they they were starting they 216 00:12:48.759 --> 00:12:54.480 were having conversations around standardizing around that answable. At the time, 217 00:12:54.720 --> 00:12:58.120 I think they had like an hotel plug in so 218 00:12:58.159 --> 00:13:01.039 you can have like uh, when you're when you're doing 219 00:13:01.039 --> 00:13:03.759 your your ANTSPEL playbooks, you have some observability around that. 220 00:13:04.240 --> 00:13:10.919 And then there's there is I want to say, there 221 00:13:11.000 --> 00:13:14.039 was like an hotel. I forget now what it's called, 222 00:13:14.039 --> 00:13:18.679 but basically you could have hotel for Bash. And it 223 00:13:18.759 --> 00:13:21.080 was funny. We were we were talking about this at 224 00:13:21.080 --> 00:13:25.039 our talk and then later at that cupcon recent I 225 00:13:25.080 --> 00:13:29.639 met the person who who created that, Amy Toby from 226 00:13:29.919 --> 00:13:33.159 She used to work at Equinox, but she created that tool. 227 00:13:33.200 --> 00:13:36.879 I'm like, oh my god, like mega fangirling. I'm like, 228 00:13:36.960 --> 00:13:38.200 we talked about your. 229 00:13:38.039 --> 00:13:39.360 Tool and you're here. 230 00:13:42.200 --> 00:13:45.240 So when we talk about observability in the C I 231 00:13:45.279 --> 00:13:48.279 C D pipeline, what kind of metrics or insights are 232 00:13:48.320 --> 00:13:49.759 you looking to expose there? 233 00:13:51.320 --> 00:13:54.120 You're wanting to look at things like how long you're 234 00:13:54.159 --> 00:14:02.200 spending on on your builds, how long you're spending like 235 00:14:02.440 --> 00:14:07.240 at each stage of your pipeline, for example, identifying pipeline failures. 236 00:14:07.840 --> 00:14:11.919 That's another thing, because and and and being able to 237 00:14:11.919 --> 00:14:16.000 standardize it with open telemetry in particular, because I think 238 00:14:16.039 --> 00:14:18.600 that's the main thing too, is because like a lot 239 00:14:18.679 --> 00:14:20.559 of I would say, a lot of the industry has 240 00:14:20.600 --> 00:14:25.080 moved towards standardizing on open telemetry, making sure that then 241 00:14:25.120 --> 00:14:28.039 we're still continuing to speak that same language. 242 00:14:27.720 --> 00:14:29.279 Right, gotcha. 243 00:14:29.399 --> 00:14:31.320 Yeah, So then you still have access to all of 244 00:14:31.320 --> 00:14:34.679 those same insights, but you get it from your same 245 00:14:34.919 --> 00:14:38.320 observability tool that you get all of your other metrics. 246 00:14:37.919 --> 00:14:41.399 From exactly exactly. And and also like you know, you 247 00:14:41.440 --> 00:14:44.240 add distributed tracing into the into the mix, and all 248 00:14:44.240 --> 00:14:46.759 of a sudden, now you're also able to have this 249 00:14:46.919 --> 00:14:50.480 nice visualization of like your your build pipeline, right you 250 00:14:50.519 --> 00:14:54.000 can you can see like all of the stages nicely 251 00:14:55.039 --> 00:14:58.120 as well through through your observability vendor, which I think 252 00:14:58.200 --> 00:14:58.879 is really cool. 253 00:14:59.559 --> 00:15:01.799 I think I think every company I worked for had 254 00:15:01.840 --> 00:15:05.600 a pretty good hands on strategy for managing observability of 255 00:15:05.639 --> 00:15:08.919 their pipelines. Whenever it failed, someone got an email and 256 00:15:08.960 --> 00:15:12.759 then they went to the product and they clicked retry. 257 00:15:14.759 --> 00:15:16.960 So my favorite one is when you're in a feting 258 00:15:17.120 --> 00:15:19.519 and somebody's like, shouldn't this have been done already? 259 00:15:19.600 --> 00:15:21.320 And then I go and I check the pipeline and 260 00:15:21.320 --> 00:15:23.480 I'm like, well it should have, but it failed and 261 00:15:23.519 --> 00:15:24.240 so it didn't. 262 00:15:24.480 --> 00:15:25.120 And then. 263 00:15:27.559 --> 00:15:30.720 You didn't even have the emails. Yeah, that's step one. 264 00:15:31.240 --> 00:15:33.360 No, it's just it's too many emails and then I 265 00:15:33.399 --> 00:15:33.960 turn them off. 266 00:15:34.000 --> 00:15:36.639 It's like how every like messaging platform is really great 267 00:15:36.720 --> 00:15:38.679 until everybody's on there and then I get too many 268 00:15:38.679 --> 00:15:40.440 messages and then I and then I turn it off. 269 00:15:40.639 --> 00:15:43.200 It's like, oh, I mean, let's let's go the horror 270 00:15:43.279 --> 00:15:47.200 story route. So the one I know is we were 271 00:15:47.279 --> 00:15:50.279 using SPN at the time, so already a great start. 272 00:15:50.879 --> 00:15:51.399 Oh damn. 273 00:15:52.120 --> 00:15:54.840 And this was an upgrade from what my previous company 274 00:15:54.840 --> 00:15:58.480 had been doing, which was no source control for their 275 00:15:58.600 --> 00:16:02.080 source code. So this was yeah, this was like, wow, 276 00:16:02.759 --> 00:16:04.799 actually there is a company that knows how to do 277 00:16:04.879 --> 00:16:08.639 source control. Because I had been using get for a 278 00:16:08.679 --> 00:16:11.080 lot of years before that, and so I was shocked 279 00:16:11.200 --> 00:16:12.879 that this is what the state of the industry was. 280 00:16:14.080 --> 00:16:16.840 But the genius thing was that you couldn't figure out, 281 00:16:17.320 --> 00:16:19.600 like you knew who the committer was for each each failure, 282 00:16:19.679 --> 00:16:22.480 but you had no idea if it was like their fault. 283 00:16:22.679 --> 00:16:25.759 So the genius thing was that they converted it so 284 00:16:25.799 --> 00:16:28.080 that when an email went out, they tried to dynamically 285 00:16:28.120 --> 00:16:31.519 figure out who made the change that actually caused the problem, 286 00:16:31.600 --> 00:16:33.519 so that they could actually put the right people in 287 00:16:33.559 --> 00:16:36.000 the email. Like this was. This is not a trivial thing, 288 00:16:36.120 --> 00:16:38.519 especially if there's like you know, multiple things going on. 289 00:16:38.600 --> 00:16:44.000 You have two thousand engineers committing to the same mono repo. Yeah, 290 00:16:44.039 --> 00:16:45.080 I don't work there anymore. 291 00:16:46.559 --> 00:16:49.039 Did you take two thousand, two thousand engineers? 292 00:16:49.120 --> 00:16:49.879 Yeah? Yeah? 293 00:16:50.000 --> 00:16:53.440 Is that like real or is that hyper because that's 294 00:16:53.799 --> 00:16:54.720 a lot of engineers too. 295 00:16:54.799 --> 00:16:55.759 That's a lot of engineers. 296 00:16:55.919 --> 00:16:56.120 Yeah. 297 00:16:56.840 --> 00:16:59.960 I mean there's like I mean monolists and mono repos 298 00:17:00.120 --> 00:17:02.320 like this is the this is the Google way right there. 299 00:17:02.320 --> 00:17:05.279 I think that's the two trains of companies here, the 300 00:17:05.279 --> 00:17:08.440 ones that go well oiled monolith on one side and I 301 00:17:08.440 --> 00:17:10.359 mean right before well the oil model that there's like 302 00:17:10.359 --> 00:17:13.240 distributed monolith, and then on the other side is micro 303 00:17:13.279 --> 00:17:16.799 services everywhere and individual repository. So the closer you get 304 00:17:16.839 --> 00:17:19.759 to the monolith side, the more you have just engineers 305 00:17:19.759 --> 00:17:21.519 thrown at the problem. I think Google last check was 306 00:17:21.519 --> 00:17:24.400 like over one hundred thousand or something like. It's it's 307 00:17:24.440 --> 00:17:28.119 a massive number that these companies try to make happen, 308 00:17:28.160 --> 00:17:31.079 So two thousand is not that big in my in 309 00:17:31.119 --> 00:17:35.400 my experience, you know. 310 00:17:35.480 --> 00:17:39.119 You mentioning SVN. I worked at a place that used 311 00:17:39.119 --> 00:17:42.119 this version control tool called Harvest. 312 00:17:42.400 --> 00:17:45.359 I don't know if you beat me, because I don't 313 00:17:45.359 --> 00:17:47.079 know that one. I like, I have a long list 314 00:17:47.119 --> 00:17:49.119 of ones that I've seen and Harvest is not on 315 00:17:49.160 --> 00:17:49.640 that list. 316 00:17:50.240 --> 00:17:52.759 And it was like such a piece of crap. And 317 00:17:53.039 --> 00:17:55.799 it was one of those ones to where you had 318 00:17:55.839 --> 00:17:58.240 to like check in check out the file. So like 319 00:17:58.319 --> 00:18:01.000 while that file was checked check doubt, you know, no one, 320 00:18:01.480 --> 00:18:04.519 no one could touch it. And I mean it was 321 00:18:04.559 --> 00:18:09.480 better than I've worked at places where we had a 322 00:18:09.599 --> 00:18:15.119 network drive with the source code, pray pray that someone 323 00:18:15.119 --> 00:18:16.680 didn't overwrite your work. 324 00:18:17.200 --> 00:18:19.839 So at least by default, even on Windows, there's a 325 00:18:19.880 --> 00:18:22.519 little bit of conflict resolution. But before I convinced my 326 00:18:22.559 --> 00:18:26.079 company to move to GET, at the time they were 327 00:18:26.160 --> 00:18:31.920 using before SPN, they were actually using uh Perforce by Microsoft, 328 00:18:32.160 --> 00:18:35.240 and that didn't have file conflict resolution. So if two 329 00:18:35.279 --> 00:18:37.640 people committed a file at the same time, it would 330 00:18:37.680 --> 00:18:41.480 literally crash the entire database and you were not restoring 331 00:18:41.480 --> 00:18:42.359 your source code. 332 00:18:45.720 --> 00:18:48.799 Now, has anyone ever used ClearCase. 333 00:18:50.519 --> 00:18:52.000 You go to tell us that you have an hotel 334 00:18:52.240 --> 00:18:56.880 uh for for every single source codel? 335 00:18:57.599 --> 00:19:01.039 Oh my god, wouldn't that be something? But ClearCase I 336 00:19:01.079 --> 00:19:04.240 think they were bought by IBM at some point and 337 00:19:04.839 --> 00:19:09.519 it was the most ridiculous source control system ever because 338 00:19:09.559 --> 00:19:14.720 you had to write like configurations for being able to 339 00:19:15.200 --> 00:19:18.960 do the source control. So it's like akin to like 340 00:19:19.400 --> 00:19:21.839 you know on mainframes where you have to write the 341 00:19:21.960 --> 00:19:26.880 JCL to like run your code. It was kind of 342 00:19:26.920 --> 00:19:30.839 like that. It was like so archaic and like it 343 00:19:30.960 --> 00:19:33.279 hurt my brain and I'm like, I don't want to 344 00:19:33.279 --> 00:19:37.799 touch version control ever again after this experience. And then 345 00:19:37.839 --> 00:19:40.480 we moved to GET and I'm like, thank god, someone 346 00:19:40.559 --> 00:19:41.319 understands me. 347 00:19:42.319 --> 00:19:47.200 So, whenever we're talking about putting observability in a CICD platform, 348 00:19:47.640 --> 00:19:52.440 are there specific are there specific plug ins or specific 349 00:19:53.880 --> 00:19:56.519 tools that help you instrument this, or are we talking 350 00:19:56.519 --> 00:20:01.160 about just like using Bash and netcat to fire off 351 00:20:01.240 --> 00:20:02.359 data to an endpoint? 352 00:20:03.200 --> 00:20:07.720 So you if you're if you're c i CD tool 353 00:20:08.160 --> 00:20:13.000 supports it, then I would definitely say use whatever plugins 354 00:20:13.039 --> 00:20:17.160 are available the official plug ins. Use that because I 355 00:20:17.200 --> 00:20:20.440 think that'll give you some good insights. But barring that, 356 00:20:21.839 --> 00:20:27.000 there is, for example, there is a plug in for 357 00:20:27.400 --> 00:20:35.039 Java around uh, like specific for for Java builds. I 358 00:20:35.119 --> 00:20:38.359 believe that gives you some additional c I c D 359 00:20:38.759 --> 00:20:44.079 insights or stuff like around Maven and gradle uh. And 360 00:20:44.119 --> 00:20:46.920 then there's this, uh I'm trying to remember now what 361 00:20:47.000 --> 00:20:50.519 the hotel bash thing is called. I'm going to just 362 00:20:50.559 --> 00:20:52.680 google this quickly dash. 363 00:20:54.519 --> 00:20:58.799 Uh. 364 00:20:58.920 --> 00:21:01.799 I can't find the one I'm looking for. Oh yeah, 365 00:21:01.839 --> 00:21:04.000 open telementary cli hotel cli. 366 00:21:05.400 --> 00:21:11.240 That's I could imagine that that one is probably pretty 367 00:21:11.279 --> 00:21:14.039 popular because I mean, let's just be honest, most of 368 00:21:14.079 --> 00:21:16.480 your CICD stuff is just a Bash command. 369 00:21:16.559 --> 00:21:22.599 No, no, don't say that. 370 00:21:22.799 --> 00:21:26.039 I mean, barring that, you can definitely you can definitely 371 00:21:26.160 --> 00:21:28.680 use that. It's it'll at least give you something. 372 00:21:29.799 --> 00:21:32.720 What do you have to instrument like within the pipeline. 373 00:21:32.759 --> 00:21:35.359 So I think we talked a little bit about the 374 00:21:35.480 --> 00:21:38.160 let's say, the steps or the jobs or the workflows 375 00:21:38.160 --> 00:21:40.480 that you have and the amount of time that they're 376 00:21:40.480 --> 00:21:42.559 spending in each one of them, or like where the 377 00:21:42.559 --> 00:21:46.160 failures end up being, especially for I mean c ID 378 00:21:46.279 --> 00:21:49.480 pipelines are notoriously flaky in some way, like maybe the 379 00:21:49.759 --> 00:21:52.519 package repository is down, or the machine runs out of 380 00:21:52.519 --> 00:21:55.680 memory or gets gets killed because gethub thinks that you're 381 00:21:55.759 --> 00:21:59.519 running a crypto minor on there, or any number of 382 00:21:59.519 --> 00:22:01.960 other things. But like you know, one one step above 383 00:22:02.000 --> 00:22:04.920 that though, Uh, there's a lot that goes on during 384 00:22:04.920 --> 00:22:07.480 the CICD pipelines. And I can imagine there's tools like 385 00:22:07.640 --> 00:22:10.240 you have you mentioned a bunch like you know, say antsable, 386 00:22:10.279 --> 00:22:13.079 but obviously there's I A C stuff. Maybe you know 387 00:22:13.079 --> 00:22:17.279 you're building artifacts and whatnot. How hard is it to 388 00:22:17.319 --> 00:22:19.839 get into all those tools and get all that telementary 389 00:22:19.880 --> 00:22:22.240 data to actually you know, export it out somewhere, even 390 00:22:22.279 --> 00:22:23.480 if the platform supports them. 391 00:22:24.440 --> 00:22:29.119 So one of the things that we explored was there's 392 00:22:29.200 --> 00:22:34.640 this component of so for I don't know how familiar 393 00:22:34.680 --> 00:22:36.960 you are with with open telemetry. 394 00:22:36.839 --> 00:22:38.839 You're going to get a range of in the audience here, 395 00:22:38.880 --> 00:22:41.279 like there are some that are going to be experienced 396 00:22:41.279 --> 00:22:43.319 and on the other side may have no idea what 397 00:22:43.359 --> 00:22:45.440 you mean when you even say hotel all right. 398 00:22:45.680 --> 00:22:51.160 So, so basically open telemetry there's like a couple of parts. 399 00:22:51.200 --> 00:22:52.599 There's like the A P I and s d K, 400 00:22:53.839 --> 00:22:58.960 so use that to instrument use that to instrument your code. 401 00:22:59.079 --> 00:23:02.079 So you go into your Java and Python code or whatnot, 402 00:23:02.119 --> 00:23:04.920 and there's like a bunch of hotel languages that are supported, 403 00:23:05.440 --> 00:23:07.640 and you go in and you write the instrumentation. So 404 00:23:07.680 --> 00:23:10.599 that means like you're manually typing in like this is 405 00:23:10.720 --> 00:23:13.960 where I want to insert my traces, my logs, my metrics. 406 00:23:14.720 --> 00:23:23.359 There's also automatic instrumentation and where basically there's like essentially 407 00:23:23.400 --> 00:23:26.119 like a wrapper around your code that as long as 408 00:23:26.160 --> 00:23:30.079 you're using like certain like popular libraries like Python flasks 409 00:23:30.079 --> 00:23:35.880 for example, it'll it'll automatically generate some telemetry like some 410 00:23:35.960 --> 00:23:40.240 traces for for for your code there for any code 411 00:23:40.359 --> 00:23:44.279 like using using that library. But then there's another component 412 00:23:44.279 --> 00:23:46.960 in open telemetry, which is the Hotel collector. So that's 413 00:23:47.079 --> 00:23:50.559 a vendor neutral agent basically, and you can think of 414 00:23:50.599 --> 00:23:53.680 it as like an ETL tool, So it'll extract the 415 00:23:53.720 --> 00:23:58.519 telemetry from multiple sources. So the sources can be like 416 00:23:58.599 --> 00:24:04.119 your applicationmetry can be your infrastructure telemetry from various various 417 00:24:04.160 --> 00:24:07.920 things at the same time. And then there's processors which 418 00:24:07.960 --> 00:24:11.640 basically can you know, massage your data. You can add attributes, 419 00:24:11.680 --> 00:24:17.240 remove attributes. You can do some some transformations. So for example, 420 00:24:17.640 --> 00:24:22.319 you know you're using like underscores and you're naming conventions 421 00:24:22.319 --> 00:24:25.960 for your attributes, but you're you want to switch to 422 00:24:26.119 --> 00:24:30.200 like dot notation, use that, and you do. You do 423 00:24:30.279 --> 00:24:33.440 that in the collector and then it gets sent somewhere 424 00:24:33.680 --> 00:24:35.880 and you can send it to multiple somewheres at the 425 00:24:35.880 --> 00:24:39.759 same time with the collector. So basically this enables you 426 00:24:39.839 --> 00:24:43.720 to for example, if you don't have like an all 427 00:24:43.720 --> 00:24:46.720 in one tool that can ingest all all of your 428 00:24:46.759 --> 00:24:49.039 telemetry that you're collecting, then you can send it like 429 00:24:49.440 --> 00:24:52.359 to one tool for traces, one for logs, one for metrics. 430 00:24:52.720 --> 00:24:55.279 I don't recommend that ideally, you want everything sitting in 431 00:24:55.319 --> 00:24:57.599 the same in the same back end, because then you 432 00:24:57.640 --> 00:24:59.599 know you have like your single source of truth and 433 00:24:59.640 --> 00:25:02.920 all the related data. But if that's your if that's 434 00:25:02.960 --> 00:25:06.079 your setup, the open Plumetry Collector allows you to do that. 435 00:25:07.240 --> 00:25:09.079 Now I'm trying to think of where I was going 436 00:25:09.160 --> 00:25:11.599 with that when we were talking about the collector. 437 00:25:12.119 --> 00:25:13.519 You know, you can think about this for a moment. 438 00:25:13.759 --> 00:25:17.160 You know, just to sell hotel, you have three components here. 439 00:25:17.200 --> 00:25:20.720 You have your code that's emitting traces or data or 440 00:25:20.759 --> 00:25:22.960 logged and you have someplace where you want to see them, 441 00:25:22.960 --> 00:25:25.640 you know, maybe it's your grafhaunas, your elastic searches of 442 00:25:25.680 --> 00:25:28.519 the world, and you could have a custom protocol and 443 00:25:28.640 --> 00:25:30.920 for a long time you had a custom protocol, custom 444 00:25:31.039 --> 00:25:34.279 libraries on the software development side to get that log 445 00:25:34.359 --> 00:25:37.519 information from the source to the saying to the target 446 00:25:37.559 --> 00:25:40.000 where you wanted to go. And that means that every 447 00:25:40.000 --> 00:25:43.160 single time you want to change which provider you were utilizing, 448 00:25:43.279 --> 00:25:45.920 or which language or which team you're working on, you 449 00:25:45.960 --> 00:25:48.279 needed to find a new library for then the new 450 00:25:48.279 --> 00:25:49.920 thing that you were utilizing. Wouldn't it be great if 451 00:25:49.920 --> 00:25:52.400 it was some sort of standard that made this all easy. 452 00:25:52.640 --> 00:25:52.880 You know. 453 00:25:53.200 --> 00:25:54.240 That's hotel for me. 454 00:25:55.440 --> 00:25:57.599 Yeah, essentially, essentially. 455 00:25:57.680 --> 00:25:59.920 Yeah, well, you know, I can see one problem here 456 00:26:00.160 --> 00:26:03.319 is making sure that every single thing that you were 457 00:26:03.400 --> 00:26:07.039 utilizing could be supported. For instance, you mentioned Python flask. 458 00:26:07.279 --> 00:26:09.240 You know, it's great that there's a library out there 459 00:26:09.240 --> 00:26:10.839 that you can throw into Python flask or if it's 460 00:26:10.839 --> 00:26:13.559 supported by flask by default, and it just it just 461 00:26:13.599 --> 00:26:16.599 works because the output of those logs matches the standard. 462 00:26:16.759 --> 00:26:18.240 But I can imagine there's lots of tools that you 463 00:26:18.240 --> 00:26:20.240 could be using you brought up you know, bashed cli 464 00:26:20.319 --> 00:26:22.519 as one of them, which you know don't have these 465 00:26:22.559 --> 00:26:24.519 things by default. My example would be, you know, let's 466 00:26:24.559 --> 00:26:30.319 assume everyone's using open tofood today. What is what does 467 00:26:30.359 --> 00:26:32.680 that look like? You know it? Like? Does do these 468 00:26:32.720 --> 00:26:35.319 tools I see offer configuration to make it easy to 469 00:26:35.359 --> 00:26:37.559 do that or is it a matter of like having 470 00:26:37.640 --> 00:26:42.599 to parse uh you know output you know, raw text 471 00:26:42.599 --> 00:26:44.000 out and get it converted. 472 00:26:44.720 --> 00:26:48.240 Yes, now I remember I was going with that. Yeah, So, like, 473 00:26:48.319 --> 00:26:50.039 for example, one one thing that a lot of these 474 00:26:50.079 --> 00:26:52.960 tools have in common is that they met logs right, 475 00:26:53.640 --> 00:26:57.720 and so you can so the hotel collector has this 476 00:26:57.799 --> 00:27:01.119 component called the foul Logs receiver where it can basically 477 00:27:01.359 --> 00:27:05.079 just logs and it'll parse out your logs given like 478 00:27:05.119 --> 00:27:09.039 some rejects expressions, so that you can do something useful 479 00:27:09.119 --> 00:27:10.920 with the data and send that to you to your 480 00:27:10.920 --> 00:27:13.599 hotel backend. So that's where you, you know, things that 481 00:27:13.640 --> 00:27:17.920 aren't necessarily like have hotel baked in, you can kind 482 00:27:17.960 --> 00:27:21.359 of turn it otel esque and get it to send 483 00:27:21.400 --> 00:27:24.920 you send the hotel data that you need to hopefully 484 00:27:25.240 --> 00:27:26.519 troubleshoot a little bit better. 485 00:27:27.519 --> 00:27:29.559 So well, well, was about to make this joke, but 486 00:27:29.559 --> 00:27:33.160 I'm going to steal it from him because you I 487 00:27:33.200 --> 00:27:36.519 think fundamentally, it's like, oh, I have a problem. The 488 00:27:36.559 --> 00:27:39.240 solution I know it's to use rejects. And when you 489 00:27:39.279 --> 00:27:41.000 think that the solution is red jocks, and now you 490 00:27:41.039 --> 00:27:42.039 have two problems. 491 00:27:42.799 --> 00:27:46.200 I know, I always have to look up rejects anything. 492 00:27:48.200 --> 00:27:49.680 I mean, I think this is where log for JA 493 00:27:50.720 --> 00:27:54.319 the exploit ended up coming from, is having to parse 494 00:27:54.359 --> 00:27:58.519 literally log messages that we're coming from everywhere, and you 495 00:27:58.559 --> 00:28:02.039 could get it too, execute arbitrary code because of it, 496 00:28:02.079 --> 00:28:05.079 and you know there's a number of huge gotcha's there 497 00:28:05.160 --> 00:28:08.559 that if you'd not really experienced, especially parsing logs, that 498 00:28:09.000 --> 00:28:11.640 you'll end up in situations with your reject will like 499 00:28:11.680 --> 00:28:14.279 literally crash due to catastrophic backtracking. 500 00:28:14.960 --> 00:28:18.000 So that's that's something I hadn't thought of, like when 501 00:28:18.039 --> 00:28:20.119 we first started talking about this, Like I was thinking 502 00:28:20.160 --> 00:28:23.000 about like during the CICD process, you know, there's the 503 00:28:23.039 --> 00:28:28.319 build time, but then there's also like time associated with 504 00:28:29.400 --> 00:28:32.720 you know, with Terraform going off and doing terraform things, 505 00:28:32.759 --> 00:28:37.599 which is can be really significant at times because sometimes 506 00:28:37.599 --> 00:28:43.160 you know, Terraform decides to just completely delete and rebuild 507 00:28:43.200 --> 00:28:45.720 this thing from scratch. You're like, dude, I just I 508 00:28:45.759 --> 00:28:48.599 just wanted you to change this parameter, and so this 509 00:28:49.279 --> 00:28:51.279 could be a really good way of exposing that. 510 00:28:52.200 --> 00:28:55.200 Yeah, yeah, exactly. It's it's all of a sudden, like 511 00:28:56.279 --> 00:29:01.200 you're able to see the things that you weren't seeing 512 00:29:01.240 --> 00:29:06.880 before necessarily, right, So it's not just the troubleshooting troubleshooting 513 00:29:06.880 --> 00:29:10.400 when things go bad, but also like can I use 514 00:29:10.480 --> 00:29:16.799 this information now to further optimize? And even the other 515 00:29:16.880 --> 00:29:19.480 part too, which is like you think it's going okay 516 00:29:20.720 --> 00:29:23.400 and you find out otherwise, right, like behind the scenes, 517 00:29:23.440 --> 00:29:27.119 something very bad is happening that wouldn't have necessarily been 518 00:29:27.160 --> 00:29:30.759 exposed because there's like no catastrophic failure of your pipeline 519 00:29:30.759 --> 00:29:34.119 as far as you're concerned. It's completing and things are 520 00:29:34.559 --> 00:29:36.880 you know, getting delivered, built and whatever. 521 00:29:39.039 --> 00:29:44.440 That's one of my favorite phrases to hear is oh, 522 00:29:44.519 --> 00:29:46.079 that error message is okay. 523 00:29:47.640 --> 00:29:48.240 No. 524 00:29:48.240 --> 00:29:50.799 No, just the fact that you called it an error 525 00:29:50.880 --> 00:29:54.319 message makes it not okay. Can we agree on that? 526 00:29:55.319 --> 00:29:56.319 Yeah, exactly. 527 00:29:57.039 --> 00:29:57.160 So. 528 00:29:57.400 --> 00:29:59.720 I think because we're in the engineering discipline, you know, 529 00:29:59.759 --> 00:30:01.680 we have to be cognizant of this. And if we 530 00:30:01.759 --> 00:30:04.559 have new data available, new metrics that we're able to track. 531 00:30:04.720 --> 00:30:06.799 It's going to create a signal that caused us to 532 00:30:06.839 --> 00:30:08.680 make a change. And so one of the questions I 533 00:30:08.680 --> 00:30:11.160 want to ask you is do you feel like that 534 00:30:11.279 --> 00:30:16.279 Hotel has caused this shift in the mindset or focused 535 00:30:16.319 --> 00:30:19.640 areas that we've been dealing with in the last let's say, 536 00:30:19.880 --> 00:30:21.680 before up and to this point. So I'm not sure 537 00:30:21.720 --> 00:30:23.480 exactly how old it is. I want to say it's 538 00:30:23.480 --> 00:30:26.400 like five years now, although that maybe that it's a 539 00:30:26.440 --> 00:30:27.640 little bit short, not right. 540 00:30:27.680 --> 00:30:29.400 It started in twenty nineteen. 541 00:30:29.799 --> 00:30:32.359 Okay, great, So before that, you know, we didn't have it. 542 00:30:32.440 --> 00:30:35.279 It's like, has there been some fundamental shift with how 543 00:30:35.319 --> 00:30:40.200 we're tackling problems in the observability space? And you said 544 00:30:40.200 --> 00:30:43.200 shift left? So really development teams, engineering teams in general, 545 00:30:43.720 --> 00:30:45.279 compared to what we were doing beforehand. 546 00:30:46.240 --> 00:30:52.519 I mean, I think with Hotel, because so pre Hotel, 547 00:30:52.559 --> 00:30:54.519 like it was kind of a free for all, and 548 00:30:54.599 --> 00:30:57.960 there had been attempts at standardization, right because there was 549 00:30:58.359 --> 00:31:01.960 open census on the one side from Google and then 550 00:31:02.160 --> 00:31:07.279 open tracing from CNCF, and then like each vendor also 551 00:31:07.480 --> 00:31:12.160 had their own thing. And so I think there's a 552 00:31:12.200 --> 00:31:17.960 lot of time and effort expended into you know, maintaining 553 00:31:18.000 --> 00:31:22.319 these like instrumentation libraries. And now with Hotel, I think 554 00:31:22.400 --> 00:31:25.160 the conversation has shifted because we're like, Okay, this is 555 00:31:25.440 --> 00:31:28.599 the single standard. We all agree that it's going to 556 00:31:28.680 --> 00:31:32.319 work this way, and now we're not using like you know, 557 00:31:32.640 --> 00:31:36.720 now it's not a single organization or individual organizations using 558 00:31:36.720 --> 00:31:40.200 their brain power like I'm gonna say in air, it's 559 00:31:40.240 --> 00:31:43.240 wasting their time on instrumentation libraries, because we're all like 560 00:31:43.319 --> 00:31:46.119 as competitors working together towards a common goal. So now 561 00:31:46.160 --> 00:31:50.880 we're we're you know, combining brains towards a singular purpose, 562 00:31:51.000 --> 00:31:55.880 which means then we're we've essentially democratized data in the 563 00:31:55.920 --> 00:32:00.960 sense that now all of these observability tools are ingesting 564 00:32:01.000 --> 00:32:05.400 the same data. And so the differentiator is what do 565 00:32:05.480 --> 00:32:08.160 they do with your data in a way that is 566 00:32:08.240 --> 00:32:09.279 helpful to you? 567 00:32:09.400 --> 00:32:09.640 Right? 568 00:32:11.440 --> 00:32:15.079 And the answer to that question varies, right because what's 569 00:32:15.160 --> 00:32:19.279 meaningful to me might not be the same, it might 570 00:32:19.319 --> 00:32:23.000 not be as important to you. And I think the 571 00:32:23.079 --> 00:32:26.519 other thing that Hotel provides that open sends us and 572 00:32:26.599 --> 00:32:29.440 open Tracing didn't provide at the time was this unified 573 00:32:29.559 --> 00:32:33.480 view of TRACE's logs and metrics where now we have 574 00:32:33.599 --> 00:32:37.119 this ability. First of all, we have a standard for 575 00:32:37.319 --> 00:32:40.920 these three signals, but also we have correlation of the 576 00:32:40.960 --> 00:32:44.319 three signals. And I think the correlation is really important 577 00:32:44.319 --> 00:32:50.079 because I I for me, at the backbone of observability 578 00:32:50.200 --> 00:32:52.680 is the distributed trace, because it tells the story right 579 00:32:52.799 --> 00:32:56.200 end to end. And then you've got the supporting actors. 580 00:32:56.240 --> 00:32:59.279 We've got the metrics that give us an idea of 581 00:32:59.720 --> 00:33:03.559 thing like our CPU usage and are our RAM usage 582 00:33:03.640 --> 00:33:07.759 or how long we've spent on a particular process, or 583 00:33:07.799 --> 00:33:10.559 even having an idea of like hey, I sold like 584 00:33:10.599 --> 00:33:15.799 fifty telescopes last month compared to this month. And then 585 00:33:15.839 --> 00:33:20.119 our logs that are like our point in time indicators, right, 586 00:33:20.279 --> 00:33:25.799 and all these things separate are like yeah, that's that's cool, 587 00:33:25.920 --> 00:33:30.160 that's useful, but together like they paint that full picture. Right, 588 00:33:30.200 --> 00:33:34.359 we have this very rich understanding of what's happening with 589 00:33:34.480 --> 00:33:38.200 our systems. And I think that's that's the thing that 590 00:33:38.319 --> 00:33:42.559 observability brings us. And I'm going to borrow a definition 591 00:33:42.640 --> 00:33:45.519 of observability that I really like from Hazel Weekly, which 592 00:33:45.599 --> 00:33:52.119 is observability allows us to ask meaningful questions, get useful 593 00:33:52.160 --> 00:33:55.640 answers and act effectively on the information that we get. 594 00:33:56.559 --> 00:34:01.759 And I think we are we are getting to that point. 595 00:34:02.200 --> 00:34:04.839 I'm not. I don't think we're fully there in the 596 00:34:04.880 --> 00:34:08.719 same way that like you know, so many organizations back 597 00:34:08.760 --> 00:34:11.480 in the day and maybe even to a certain extent now, 598 00:34:11.639 --> 00:34:15.519 or like we're doing DevOps because we have a CICD pipeline. 599 00:34:15.559 --> 00:34:18.960 It's like we have observability because we are collecting metrics 600 00:34:18.960 --> 00:34:23.480 and sending them to Blah back in. It's like you're 601 00:34:24.079 --> 00:34:25.079 you're on your way. 602 00:34:26.559 --> 00:34:26.719 There. 603 00:34:26.800 --> 00:34:30.440 Yet I think the real thing there now is can 604 00:34:30.480 --> 00:34:32.559 we have an AI agent that will look through your 605 00:34:32.599 --> 00:34:35.000 logs and tell you what the problem is, because that's 606 00:34:35.039 --> 00:34:35.880 what I think I need. 607 00:34:36.639 --> 00:34:39.239 Yeah, And I think a lot of vendors and that 608 00:34:39.400 --> 00:34:41.000 is a fair ask, and I think a lot of 609 00:34:41.079 --> 00:34:44.840 vendors are moving in that direction. I mean, including Dino Trace. 610 00:34:44.920 --> 00:34:48.079 Dina Trace has an AI assistant named Davis AI. 611 00:34:49.039 --> 00:34:51.679 And well, I guess that's the success of hotel is 612 00:34:51.719 --> 00:34:54.119 because now you can shop around for the provider that 613 00:34:54.199 --> 00:34:57.639 offers you that exact benefit without feeling like you you 614 00:34:57.760 --> 00:35:01.320 lose your data or have to spend engineering time to 615 00:35:01.639 --> 00:35:06.519 actually adhere to whatever backwards protocol that that provider is offering. 616 00:35:07.159 --> 00:35:09.519 Yeah, but like biologists don't like paying for stuff. 617 00:35:09.960 --> 00:35:12.159 I know, I know what it what it means. Like 618 00:35:12.480 --> 00:35:14.320 when you say they don't like paying for stuff, what 619 00:35:14.360 --> 00:35:16.320 they mean is they don't like paying for stuff when 620 00:35:16.360 --> 00:35:18.719 that money has like clear price tags on it. But 621 00:35:18.760 --> 00:35:22.679 when there's suspect value associated with running it yourself on 622 00:35:22.679 --> 00:35:26.119 on prem hardware, then they have no problem shelling out 623 00:35:26.960 --> 00:35:28.800 you know, hundreds of thousands of dollars for it. 624 00:35:28.840 --> 00:35:29.400 Well, that's true. 625 00:35:29.440 --> 00:35:31.559 I mean we can still have on site HPCs like 626 00:35:31.639 --> 00:35:33.800 that's that's fine, That's totally fine. 627 00:35:34.119 --> 00:35:39.800 Right, just throw Grafana in your in your cluster and 628 00:35:39.880 --> 00:35:41.639 you know, you can use your hotel collector and point 629 00:35:41.639 --> 00:35:43.679 it at that and you get all your data into 630 00:35:43.800 --> 00:35:47.000 whatever you're saying. You know that Graffana is pointed at 631 00:35:47.280 --> 00:35:48.000 and you're going to go. 632 00:35:49.599 --> 00:35:52.039 I mean sometimes, but not all the platforms that I 633 00:35:52.119 --> 00:35:54.800 use have like all these nifty tools you know built 634 00:35:54.800 --> 00:35:58.800 into them, like like the AWS, like Healthcare Pipeline platform 635 00:35:59.039 --> 00:36:00.440 does not have this stuff built in. 636 00:36:00.440 --> 00:36:05.000 It just sends everything a cloud watch and then it's like, well, 637 00:36:05.039 --> 00:36:05.679 good luck with that. 638 00:36:06.960 --> 00:36:09.800 I think you, I think you started that problem. Definition 639 00:36:09.880 --> 00:36:12.039 with the problem a ws. 640 00:36:14.039 --> 00:36:16.599 WS is listen, I'm still I'm still on my like 641 00:36:16.719 --> 00:36:19.880 drifting like a WS pay my bills please, So we 642 00:36:19.920 --> 00:36:22.079 can't we can't bad talk them because every once in 643 00:36:22.119 --> 00:36:24.559 a while they do throw some credits my way. 644 00:36:24.800 --> 00:36:27.119 So I love you as. 645 00:36:26.920 --> 00:36:28.639 And if you stick around at the end of the episode, 646 00:36:28.719 --> 00:36:31.320 I actually have something to say about a WS credits. 647 00:36:31.400 --> 00:36:36.239 So you have a coupon to send me. I love coupons. 648 00:36:36.239 --> 00:36:38.719 Well, Jillian, unfortunately you're going to be excluded from this. 649 00:36:41.960 --> 00:36:44.760 I wonder why a host of this podcast is going 650 00:36:44.800 --> 00:36:47.280 to be excluded from a giveaway we're having. 651 00:36:47.679 --> 00:36:48.840 Oh okay, that's. 652 00:36:50.360 --> 00:36:52.119 Damn it. 653 00:36:52.320 --> 00:36:55.800 Fine. So one of the things I want to dig into, 654 00:36:56.119 --> 00:37:03.000 like I see the benefits of US, where I struggle 655 00:37:03.039 --> 00:37:07.719 with implementation. No, you know, like the developers I support, 656 00:37:07.920 --> 00:37:11.559 they see the benefits of this, but only after I 657 00:37:11.679 --> 00:37:14.800 build the whole thing for them and show it to them. 658 00:37:15.000 --> 00:37:20.199 So what are some ideas you have to like get 659 00:37:20.360 --> 00:37:25.159 them excited to kind of like throw a few hours 660 00:37:25.199 --> 00:37:27.719 of their own time towards getting this implemented. 661 00:37:30.039 --> 00:37:32.800 I think I think one way to be effective with 662 00:37:32.840 --> 00:37:37.199 this is making developers responsible for their code and prod 663 00:37:37.559 --> 00:37:40.320 like right after, right after it's deployed. 664 00:37:42.480 --> 00:37:43.679 Align the ascentives. 665 00:37:43.880 --> 00:37:46.599 Nothing tells a story like staring at a screen at 666 00:37:46.599 --> 00:37:49.159 two am, knowing other reason you're there. 667 00:37:49.880 --> 00:37:57.199 Yeah, exactly exactly. I think another thing on exactly right 668 00:38:00.079 --> 00:38:01.800 other thing I was going to say, you know, back 669 00:38:01.800 --> 00:38:04.920 to what I was saying in the beginning about qa's 670 00:38:05.679 --> 00:38:09.800 using that telemetry also during the testing phase to be 671 00:38:09.880 --> 00:38:17.920 able to identify bugs in testing making basically making telemetry 672 00:38:18.079 --> 00:38:24.840 quality gait. So before going into QA, you it's a 673 00:38:25.239 --> 00:38:30.000 it's basically mandatory to have instrumented your code. Otherwise you don't, 674 00:38:30.639 --> 00:38:34.800 you don't pass go. Basically that's another that's another way 675 00:38:34.840 --> 00:38:37.320 to incentivize and and I think one way to think 676 00:38:37.360 --> 00:38:40.719 about it, and I think this is where people have 677 00:38:40.719 --> 00:38:42.719 have a bit of a hard time like wrapping their 678 00:38:42.760 --> 00:38:43.559 brains around it. 679 00:38:44.360 --> 00:38:44.480 Uh. 680 00:38:44.920 --> 00:38:48.559 I see instrumenting code as no different than like, you know, 681 00:38:48.599 --> 00:38:52.320 we're writing print statements all the time, we write log statements. 682 00:38:52.599 --> 00:38:57.000 So like you're already writing logs, So do it the 683 00:38:57.000 --> 00:39:02.639 open telemetry way. You're already like that's part of your mindset. 684 00:39:02.719 --> 00:39:06.840 So adding traces here and there isn't a terrible idea, 685 00:39:07.519 --> 00:39:10.199 especially if it can help you as well as a 686 00:39:10.239 --> 00:39:13.840 developer debug your own code, and I think that's another 687 00:39:13.920 --> 00:39:16.559 value add It's like, oh my god, I have more 688 00:39:16.559 --> 00:39:19.679 insight when I'm writing my code to understand why this 689 00:39:19.920 --> 00:39:23.239 like weird era keeps happening like every fifty runs of 690 00:39:23.280 --> 00:39:27.000 the program, Like wouldn't that be nice? 691 00:39:27.440 --> 00:39:29.599 I think you could put your perspective though, on the 692 00:39:29.760 --> 00:39:32.159 controversial side, right, Like, I mean, maybe there are the 693 00:39:32.199 --> 00:39:34.719 engineers out there that they don't write any bugs, there's 694 00:39:34.800 --> 00:39:38.000 no production, never goes down, there's no incidence whatever, and 695 00:39:38.039 --> 00:39:40.920 then and then you know, this isn't a value added activity. 696 00:39:41.000 --> 00:39:43.119 So you know, if they're out there and they're thinking 697 00:39:43.119 --> 00:39:46.079 to themselves, all my code is absolutely perfect and I'm 698 00:39:46.079 --> 00:39:48.760 not accountable for it. I don't know if there can 699 00:39:48.800 --> 00:39:49.679 be another argument. 700 00:39:50.480 --> 00:39:53.679 I know, well, what can you do to question perfection 701 00:39:53.920 --> 00:39:54.159 like that? 702 00:39:55.559 --> 00:39:59.199 You did say something interesting though, which is basically what 703 00:39:59.239 --> 00:40:02.440 you're advocating for here, which is shift left on telemetry. 704 00:40:02.599 --> 00:40:06.079 And one of the complaints that I've heard from uh 705 00:40:06.199 --> 00:40:08.840 my collities across the industry have been, well, there's like 706 00:40:08.880 --> 00:40:13.440 shift left testing and shift left telemetry and shift left security, 707 00:40:13.800 --> 00:40:16.440 and now we're doing infrastructure as code instead of release 708 00:40:16.519 --> 00:40:19.840 engineering I mean at some point, like if everything is 709 00:40:19.880 --> 00:40:22.920 shift left, there's there's nothing left, Like everything is now 710 00:40:22.960 --> 00:40:24.159 on the right, right. 711 00:40:25.760 --> 00:40:32.199 Right. I mean, yeah, you're right, but like I guess 712 00:40:32.239 --> 00:40:35.840 the way I'll look at it is, but if you're like, 713 00:40:36.400 --> 00:40:38.159 if you don't shift left, then it's going to be 714 00:40:38.199 --> 00:40:39.960 so much more painful after. 715 00:40:40.199 --> 00:40:41.960 Oh, for sure, there's no question. 716 00:40:43.000 --> 00:40:45.239 And I don't I don't know. Maybe maybe it's me, 717 00:40:45.639 --> 00:40:48.519 like my personality. I like learning new tech. So it's 718 00:40:48.559 --> 00:40:51.599 like you got to learn you know, terror form, Yay, 719 00:40:52.000 --> 00:40:54.039 that's awesome, you get to learn doctor, yay. 720 00:40:54.079 --> 00:40:54.400 Cool. 721 00:40:56.039 --> 00:41:00.320 I think I think honestly, for this to succeed, you 722 00:41:00.360 --> 00:41:02.639 almost you have to come in with the right mindset, 723 00:41:02.760 --> 00:41:06.719 and and there's no better way to do that than 724 00:41:06.760 --> 00:41:10.000 with like fresh hung blood, right. The new folks coming 725 00:41:10.000 --> 00:41:13.440 into the industry like that, that is the way. You 726 00:41:13.480 --> 00:41:15.679 know that, that is the way that it works. But 727 00:41:15.719 --> 00:41:17.719 for the older folks you're like, oh, I'm telling I 728 00:41:17.719 --> 00:41:19.559 mean they I'll have to do all this extra crap. 729 00:41:19.679 --> 00:41:20.519 Oh damn it. 730 00:41:21.119 --> 00:41:23.480 I like how you stopped after a terraform and doctor 731 00:41:23.480 --> 00:41:25.599 and you didn't say, oh, we have to use Kubernetes. 732 00:41:25.679 --> 00:41:31.840 Yeah, like Kubernettes and I have a love hate relationship. 733 00:41:31.400 --> 00:41:33.960 That was a given, like who doesn't love Kubernetes. 734 00:41:36.960 --> 00:41:40.440 So I mean that's an interesting perspective that those that 735 00:41:40.480 --> 00:41:43.039 are more experienced are likely I don't I want to 736 00:41:43.079 --> 00:41:45.079 say that they're you know, thinking the wrong direction or 737 00:41:45.159 --> 00:41:48.039 have the not the right mindset, but they're focused in 738 00:41:48.079 --> 00:41:50.400 the areas where they think it has the most value. 739 00:41:50.519 --> 00:41:53.159 Like maybe the experience helps tell them what they need 740 00:41:53.239 --> 00:41:55.760 to do more effectively. And if you are looking at 741 00:41:56.079 --> 00:41:58.119 something and say, oh, well, I don't know how to 742 00:41:58.159 --> 00:41:59.760 test this effectively, and I don't know how to make 743 00:41:59.760 --> 00:42:02.119 this is super secure, and I don't know where my 744 00:42:02.159 --> 00:42:04.440 bugs are going to be, then you do want to 745 00:42:04.440 --> 00:42:06.360 take all of these steps, at least a little bit 746 00:42:06.360 --> 00:42:08.320 in each one of these directions. And that isn't to 747 00:42:08.360 --> 00:42:12.280 say you need to design the perfect platform, but having 748 00:42:12.360 --> 00:42:15.079 the logs end up on an ephoremal like on hard 749 00:42:15.159 --> 00:42:18.639 disk of an ephemeral compute environment, like isn't going to 750 00:42:18.639 --> 00:42:20.880 help you when there's a problem in that thing crashes. 751 00:42:21.519 --> 00:42:25.800 Yeah, very true, Very true. Yeah. I you know, I 752 00:42:26.960 --> 00:42:32.800 think with this kind of shift left it like for 753 00:42:32.840 --> 00:42:35.199 those of us who've been in the industry long enough, 754 00:42:35.280 --> 00:42:39.039 I think embracing that is born out of one of 755 00:42:39.119 --> 00:42:43.320 two things, extreme trauma for your like oh my god, 756 00:42:43.360 --> 00:42:46.639 I can't take this anymore, and and the other is 757 00:42:46.760 --> 00:42:50.960 just like pure curiosity like ooh this looks really cool, 758 00:42:51.760 --> 00:42:54.639 and like having an open mind. And I honestly, I 759 00:42:54.679 --> 00:42:58.360 think the most successful techies are the ones who are 760 00:42:58.840 --> 00:43:02.480 open to change, open to like what what the new 761 00:43:02.519 --> 00:43:06.679 tooling brings. And maybe maybe it's like, oh, this thing 762 00:43:06.679 --> 00:43:08.679 doesn't do exactly what I wanted to do. And then 763 00:43:08.719 --> 00:43:11.480 this was like where startups are born, right. I mean, 764 00:43:11.760 --> 00:43:15.639 Krubernetes has got a whole ecosystem around it because of 765 00:43:16.400 --> 00:43:20.400 I guess it does stuff, but like some stuff is 766 00:43:20.679 --> 00:43:24.960 kind of gnarly to do, so let's make things easier. 767 00:43:26.280 --> 00:43:29.159 And I think that sort of thing drives innovation. At 768 00:43:29.199 --> 00:43:31.840 the end of the day, are you are. 769 00:43:31.760 --> 00:43:33.599 You happy where we're at, Like do you feel like 770 00:43:33.679 --> 00:43:35.800 that there's like just one more thing, like get the 771 00:43:35.840 --> 00:43:39.679 observability done in CICD pipelines and then everything will be great? 772 00:43:39.920 --> 00:43:43.599 Or do you see like there's a concrete objective next 773 00:43:43.599 --> 00:43:46.920 step to get to the you know, the pinnacle of 774 00:43:47.920 --> 00:43:50.000 perfection and observability. 775 00:43:50.280 --> 00:43:53.199 Ooh, that's a great question. Actually it's a topic for 776 00:43:53.360 --> 00:43:56.000 a talk that I'm giving next week. Out of observability, 777 00:43:56.039 --> 00:43:58.440 Dan like, leg it, what is it? All right, I'm 778 00:43:58.440 --> 00:44:02.800 gonna plug it. So basically, the ideas, why does observability 779 00:44:02.800 --> 00:44:06.440 have to exist within the traditional confines of tech? Why 780 00:44:06.480 --> 00:44:11.000 don't we bring observability and open telemetry to things like 781 00:44:11.280 --> 00:44:14.960 the recruitment process, Like you know, it's it's a yucky 782 00:44:15.039 --> 00:44:18.400 jobs market out there, and the recruiting process has always 783 00:44:18.400 --> 00:44:22.840 been painful. You never know, you know, you send out 784 00:44:22.840 --> 00:44:24.559 a resume, you don't know if you're going to get 785 00:44:24.559 --> 00:44:28.480 a response, and when you do it might be a while. 786 00:44:28.599 --> 00:44:30.840 And then when you finally get that interview, it's it 787 00:44:30.920 --> 00:44:33.760 might be like a chunk of time between interviews. Wouldn't 788 00:44:33.760 --> 00:44:38.639 it be nice if organizations put observability in their recruitment process, 789 00:44:38.639 --> 00:44:41.840 for example, so that they have an understanding of like 790 00:44:42.760 --> 00:44:46.039 how long it takes in each stage of the interview process. 791 00:44:46.880 --> 00:44:49.400 Wouldn't it be cool to have a distributed trace that 792 00:44:49.480 --> 00:44:54.000 represents the end to end recruitment process. And we can 793 00:44:54.039 --> 00:44:57.599 take this to like several under several other industries, like 794 00:44:57.679 --> 00:45:03.039 even the healthcare industry. Look at hospital r waiting times. 795 00:45:03.800 --> 00:45:08.400 Understanding you know what if what if you do open telemetry, 796 00:45:08.440 --> 00:45:13.079 apply open telemetry and observe ability to uh to ers 797 00:45:13.119 --> 00:45:15.280 where you're like, okay, now you have an idea of, 798 00:45:15.320 --> 00:45:17.639 like what the workflow is from from intake all the 799 00:45:17.679 --> 00:45:21.320 way to getting treatment. You have an idea of like 800 00:45:21.400 --> 00:45:23.840 the types of cases that get seen faster versus the 801 00:45:23.840 --> 00:45:27.480 ones that don't. You can have a better understanding. You 802 00:45:27.480 --> 00:45:31.920 have better data on racial profiling. For example, wait times 803 00:45:32.719 --> 00:45:35.480 the amount of time you know first to get seen, 804 00:45:35.559 --> 00:45:37.880 to get like imaging done, to get the results of 805 00:45:37.920 --> 00:45:40.639 the imaging. So I feel like the sky is the limit, 806 00:45:41.079 --> 00:45:44.400 and I think it's I would say easier. I'm going 807 00:45:44.440 --> 00:45:46.920 to say in air quotes and in larger organizations that 808 00:45:46.960 --> 00:45:52.840 have access to to be like monetary access to to 809 00:45:52.960 --> 00:45:56.360 be able to like you know, purchase a subscription from 810 00:45:56.400 --> 00:46:00.800 a SaaS vendor or or like run a homegrown solution 811 00:46:00.960 --> 00:46:04.519 for observability. But I think I think it can open 812 00:46:04.599 --> 00:46:06.519 up some really cool possibilities around that. 813 00:46:07.199 --> 00:46:08.920 See, I had a secret fear You're going to say, 814 00:46:08.960 --> 00:46:09.960 government oversight. 815 00:46:13.920 --> 00:46:15.920 Why would that be relevant at all? 816 00:46:16.400 --> 00:46:21.639 Yeah? So I know, but I think you touched on something. Yeah, 817 00:46:21.719 --> 00:46:24.920 well okay, yeah, so we won't we won't go there, 818 00:46:25.119 --> 00:46:28.320 uh I do. I do want to touch on something. 819 00:46:28.360 --> 00:46:31.519 I think you you added a nuance too, which is 820 00:46:31.559 --> 00:46:35.960 that the value that could be captured here is actually 821 00:46:35.960 --> 00:46:39.679 directly related to the business and not arbitrary tech metrics 822 00:46:39.719 --> 00:46:42.159 about your running service like how many two hundreds and 823 00:46:42.159 --> 00:46:43.880 three hundred do you have or how long it's running, 824 00:46:44.199 --> 00:46:47.320 but maybe the value to the actual customer and or 825 00:46:47.360 --> 00:46:50.119 the pain they're suffering through the you know, user experience 826 00:46:50.119 --> 00:46:53.239 of your UI because you you automatically coded it using 827 00:46:53.239 --> 00:46:58.480 one of these new vibe coding auto creation UIs, and 828 00:46:58.519 --> 00:47:02.280 so you know, yeah, I think it's a really good point, 829 00:47:02.280 --> 00:47:04.480 and like this could be applied to to every industry, 830 00:47:04.519 --> 00:47:06.280 not just ones that are hard in tech. You know, 831 00:47:06.360 --> 00:47:08.719 how are you collecting metrics today? How are you actually 832 00:47:08.719 --> 00:47:12.639 evaluating these things? Don't you want data? And now I'm 833 00:47:12.679 --> 00:47:15.280 starting to wonder all of those smaller companies that are 834 00:47:15.519 --> 00:47:19.760 between zero and let's say two or five employees, you know, 835 00:47:19.760 --> 00:47:22.559 what are they doing. I know that like maybe the 836 00:47:22.599 --> 00:47:25.360 last thirty years was all about digital transformation, and I 837 00:47:25.400 --> 00:47:28.760 still know companies are hiring digital transformation consultants to make 838 00:47:28.800 --> 00:47:32.480 this happen. And I think you've hit on like, really 839 00:47:32.719 --> 00:47:34.880 the why which is you're missing the data? You're like, 840 00:47:34.880 --> 00:47:38.199 you're not collecting it, and this, this process, this standard 841 00:47:38.320 --> 00:47:39.960 is what's going to help you achieve that. 842 00:47:40.719 --> 00:47:43.440 Yeah? Yeah, and I mean unfortunately, like we speak in 843 00:47:43.559 --> 00:47:46.199 data right, like show me, show me the metrics, and 844 00:47:46.239 --> 00:47:49.920 then tie the metrics to the money, right. 845 00:47:50.920 --> 00:47:54.760 Well, Jerry Maguire reference there, show me the money. 846 00:47:54.960 --> 00:47:58.679 Yeah, yeah, I do. 847 00:47:58.760 --> 00:48:00.400 I do want to share this because I I do 848 00:48:00.440 --> 00:48:04.599 think this list is quite ridiculous. So you're a CNCF ambassador, 849 00:48:04.719 --> 00:48:08.360 you have a podcast, you're an author, I wrote a 850 00:48:08.400 --> 00:48:10.800 bunch of other things down here. But I see, like, 851 00:48:11.239 --> 00:48:16.960 you know what conference speaker, right, what is there still 852 00:48:17.000 --> 00:48:20.320 some milestone you're hoping to achieve next after this that 853 00:48:20.360 --> 00:48:22.480 you're you know, currently focused on. You're like, no, I've 854 00:48:22.480 --> 00:48:25.000 done enough things, you know, I feel accomplished enough. 855 00:48:26.880 --> 00:48:29.360 Oh well, I would love to, like someday, keynote at 856 00:48:29.400 --> 00:48:32.800 a cupe gun. I got my first keynote last year 857 00:48:33.360 --> 00:48:37.679 at KCD in Potho, Portugal, and that was like, that 858 00:48:37.800 --> 00:48:41.719 was super exciting. I've never been asked a keynote. I 859 00:48:41.719 --> 00:48:44.920 think I think the ultimate experience would be a keynote 860 00:48:45.000 --> 00:48:47.559 on a at a really large conference that would be 861 00:48:47.639 --> 00:48:51.880 super fun. I did also find out this one's up 862 00:48:51.920 --> 00:48:54.000 there on the list. I found out last week that 863 00:48:54.079 --> 00:48:57.800 I'll be going to keep Con Japan, which is super 864 00:48:57.880 --> 00:49:00.480 exciting because it's the first ever keep conjup and I've 865 00:49:00.519 --> 00:49:04.440 never been to Japan. And I'll be giving a talk 866 00:49:04.519 --> 00:49:11.239 on basically what we can do to make our observability greener. 867 00:49:12.039 --> 00:49:15.719 And it's, uh, it's it's a follow on, if you 868 00:49:15.800 --> 00:49:17.360 will to a talk that I'm giving next week in 869 00:49:17.400 --> 00:49:21.079 London with my same co speaker, Nancy Chohan, where we 870 00:49:21.400 --> 00:49:24.280 it's the talk is next week is called how Green 871 00:49:24.360 --> 00:49:26.719 is my Open Telemetry Collector? And it talks about like 872 00:49:27.119 --> 00:49:30.480 what you can do to start looking at at optimizing 873 00:49:30.480 --> 00:49:32.880 your hotel collector to make it, you know, more more 874 00:49:32.960 --> 00:49:33.960 environmentally friendly. 875 00:49:34.639 --> 00:49:37.840 Do do you feel lucky now that there is another 876 00:49:37.880 --> 00:49:40.840 piece of technology out there that is just so much 877 00:49:40.880 --> 00:49:43.719 worse for the environment that no one's paying attention to 878 00:49:44.679 --> 00:49:47.239 any sort of problem with story extra data I mean 879 00:49:47.280 --> 00:49:49.280 storing the data. I mean now that that's a trivial 880 00:49:49.320 --> 00:49:51.039 matter as far as impact goes. 881 00:49:52.119 --> 00:49:56.679 It's funny that, like, you know, I almost feel I 882 00:49:56.719 --> 00:49:59.280 feel guilty, like working in this industry to be honest, 883 00:49:59.360 --> 00:50:03.159 because like I've always, like since I was a kid, 884 00:50:03.199 --> 00:50:07.199 I was like really into like environmental stuff, and you know, 885 00:50:07.320 --> 00:50:12.039 like I bring a reusable cup to like Starbucks, or 886 00:50:12.239 --> 00:50:14.719 like I love bubble Tea, so my local bubble tea place, 887 00:50:14.840 --> 00:50:18.360 I'll bring a reusable cup. And you know, I've done 888 00:50:18.440 --> 00:50:21.400 the reasonable shopping bags for like twenty years, and yet 889 00:50:21.480 --> 00:50:26.360 I'm in an industry that is inherently terrible for the environment. 890 00:50:27.039 --> 00:50:29.199 You know, data centers I think contribute to like one 891 00:50:29.280 --> 00:50:32.039 or one to two percent of of like the world's 892 00:50:32.079 --> 00:50:36.320 greenhouse gas emissions. And then you add AI into the mix, 893 00:50:36.440 --> 00:50:39.880 and it's like ouch observability, I mean the fact that 894 00:50:39.920 --> 00:50:44.559 we're trying to understand our systems better through observability. Well, 895 00:50:44.599 --> 00:50:48.320 guess what, You're emitting a crap ton of data, So 896 00:50:48.559 --> 00:50:51.960 your systems are are expending more energy in doing so, 897 00:50:52.199 --> 00:50:56.280 and then your your observability tooling and ingesting the data 898 00:50:56.639 --> 00:51:01.519 are also emitting a crop ton of energy to do that. 899 00:51:02.800 --> 00:51:07.320 So it's like we're you know, we're adding to the problem. 900 00:51:07.320 --> 00:51:10.559 But then I also feel like technology can solve the problem, 901 00:51:10.719 --> 00:51:14.800 Like you know, those same AI agents that that do 902 00:51:15.000 --> 00:51:18.880 expend a lot of energy can also helpless further optimize 903 00:51:19.599 --> 00:51:24.199 our you know, our energy usage to lessen our carbon footprint. 904 00:51:24.199 --> 00:51:26.760 I think it's all that will be a balance. 905 00:51:27.239 --> 00:51:31.440 Well, there's the paradox there, and I don't remember the 906 00:51:32.440 --> 00:51:36.079 philosopher's name. Hopefully someone else does. If you increase the 907 00:51:36.239 --> 00:51:39.840 efficiency or you optimize it, you end up with more 908 00:51:39.920 --> 00:51:43.960 usage because it becomes cheaper and more so in the end. 909 00:51:44.039 --> 00:51:48.400 So that's not Unfortunately, that's not a path forward that 910 00:51:48.559 --> 00:51:50.599 I'm willing to I'm willing to bet on. But like 911 00:51:50.639 --> 00:51:54.719 I'm still bringing reusable like paper bags to the grocery 912 00:51:54.719 --> 00:51:57.480 store for my bread and vegetables, like I am, I 913 00:51:57.880 --> 00:52:00.880 am just as bad, uh know, put it in a 914 00:52:00.920 --> 00:52:03.599 backpack and no plastic bags or anything. And I've still 915 00:52:03.639 --> 00:52:05.440 got the same paper bag that my wife is like, 916 00:52:05.440 --> 00:52:08.639 why are you still reusing that? Carry stuff in? 917 00:52:09.920 --> 00:52:11.880 I always forget mine. So I ended up making it 918 00:52:11.920 --> 00:52:14.079 a trip out of the store with thirty seven things 919 00:52:14.079 --> 00:52:14.880 in my arms. 920 00:52:15.280 --> 00:52:17.000 Oh yeah, backpack. 921 00:52:17.719 --> 00:52:19.280 I used to buy the bag. 922 00:52:19.599 --> 00:52:21.920 It's ridiculous. At this point, I have like a closet 923 00:52:21.920 --> 00:52:22.480 full of them. 924 00:52:22.599 --> 00:52:27.280 I'm like, yeah, just with you should donate them to 925 00:52:27.880 --> 00:52:30.320 some other people, like just or sell them right outside 926 00:52:30.360 --> 00:52:32.639 the store. Like just see like when Will comes out 927 00:52:32.639 --> 00:52:33.679 of the store and he's carrying a. 928 00:52:33.679 --> 00:52:36.079 Lot of things, be like, hey, ooh, clever. 929 00:52:37.880 --> 00:52:41.000 I don't know if that's legal not for resale? 930 00:52:41.400 --> 00:52:46.039 Yeah, probably not, damn it. 931 00:52:46.559 --> 00:52:47.280 I think you just have. 932 00:52:47.280 --> 00:52:49.360 To be fifty feet away from the door, and by 933 00:52:49.360 --> 00:52:53.320 that time I'm a very motivated customer of your product. Anyway, 934 00:52:54.480 --> 00:52:57.239 if you haven't, if you haven't gotten to your motive 935 00:52:57.239 --> 00:53:02.199 transportation after fifty feet, I mean, I I. 936 00:53:01.159 --> 00:53:04.760 Worry what's going on there? I will ask maybe you 937 00:53:04.760 --> 00:53:06.800 can spoil it a little bit for us? What as 938 00:53:06.840 --> 00:53:09.880 you mentioned data centers as not being that environmentally friendly? 939 00:53:10.440 --> 00:53:13.559 Is it? Is it the data storage? Is it? The compute? 940 00:53:13.760 --> 00:53:15.239 Is it you know, memory usage? 941 00:53:15.280 --> 00:53:15.440 Is it? 942 00:53:15.480 --> 00:53:17.920 The menu? The hardware manufacturing that's doing it? So the 943 00:53:17.960 --> 00:53:20.159 building new data center, like do you know like which 944 00:53:20.239 --> 00:53:25.119 area is contributing or is the most problematic for us? 945 00:53:25.880 --> 00:53:28.840 I don't know specifically, but I would gather like the 946 00:53:28.880 --> 00:53:33.079 power consumption alone of data centers is huge and puts 947 00:53:33.119 --> 00:53:36.800 like a massive strain on on power grids. So there's 948 00:53:36.920 --> 00:53:41.280 definitely I would say, I would guess. Now, don't don't 949 00:53:41.360 --> 00:53:43.320 quote me on that, but I would guess that that's 950 00:53:43.519 --> 00:53:45.599 that would definitely eat into things a. 951 00:53:45.639 --> 00:53:48.880 Fair bit from some of the stuff I've heard. It's 952 00:53:48.920 --> 00:53:51.039 the cooling mhm. 953 00:53:51.400 --> 00:53:54.159 I could I could believe that. Yeah, dealing with extra 954 00:53:54.280 --> 00:53:57.400 heat is a huge challenge. But if it's the energy, 955 00:53:57.440 --> 00:53:59.519 then what we have to make sure realistically, is it 956 00:53:59.559 --> 00:54:01.360 the energy we're creating is green? 957 00:54:02.360 --> 00:54:07.239 Just build more nuclear power plants always the solution it is. 958 00:54:09.199 --> 00:54:10.920 I can tell Warren Lust to disagree. 959 00:54:11.119 --> 00:54:14.360 Oh no, no, no, no, I no, no, I absolutely 960 00:54:14.400 --> 00:54:18.119 agree there is no better form of energy, even though 961 00:54:18.159 --> 00:54:21.000 there's all these problems with I say nuclear, but you 962 00:54:21.039 --> 00:54:23.599 know we're saying fission right because we're not at the 963 00:54:23.599 --> 00:54:27.119 fusion stage yet. And there's just a lot of arguments 964 00:54:27.159 --> 00:54:29.119 where like what do you do with the wastewater? I'm like, 965 00:54:29.440 --> 00:54:33.280 compare that to the mining of the raw materials and 966 00:54:33.280 --> 00:54:36.559 the manufacturing of solar panels, or the actual damage to 967 00:54:37.599 --> 00:54:41.880 like my migratory birds for flight pass for wind turbines, 968 00:54:42.599 --> 00:54:45.800 and not to mention the non renewable ones, like you know, 969 00:54:45.840 --> 00:54:49.480 it's just so absurd to me. Sorry, that's that's my 970 00:54:49.519 --> 00:54:55.920 own personal rant. Just cancel, just cancel all non commercial aircraft. 971 00:54:55.920 --> 00:54:57.679 There should be no private jets. You know that will 972 00:54:57.679 --> 00:54:59.880 solve a majority of the world's problems from there. 973 00:55:00.079 --> 00:55:08.639 True, I can't disagree. It's not going to impact my life. 974 00:55:08.679 --> 00:55:09.760 I'll tell you that for sure. 975 00:55:13.039 --> 00:55:15.360 Damn it, I'm gonna cancel my Gulf Stream. 976 00:55:15.159 --> 00:55:15.880 Order right. 977 00:55:18.480 --> 00:55:19.360 Hold on b RB. 978 00:55:20.599 --> 00:55:20.800 Yeah. 979 00:55:24.800 --> 00:55:27.079 But now that's like a few minutes ago, you brought 980 00:55:27.119 --> 00:55:30.800 up a really interesting point about using hotel metrics in 981 00:55:30.920 --> 00:55:33.360 other parts of the business, and like immediately my mind 982 00:55:33.440 --> 00:55:38.239 exploded with like ten different things in the company I 983 00:55:38.280 --> 00:55:43.599 work for right now, I'm like, holy shit, Like they 984 00:55:43.719 --> 00:55:47.199 they totally need to see this on a on a 985 00:55:47.239 --> 00:55:51.079 metrics dashboard. And it's like, you know, you mentioned recruiting process, 986 00:55:51.119 --> 00:55:56.199 but I'm thinking like the sales pipeline or the implementation 987 00:55:56.400 --> 00:56:02.880 pipeline whenever we implement someone onto our product, the employee 988 00:56:02.880 --> 00:56:06.159 review process, what stage that's at, Like there's just so 989 00:56:06.199 --> 00:56:10.519 many different things where like oh wow, even Jira, like 990 00:56:11.519 --> 00:56:18.599 where are things stalling at moving tickets from from new 991 00:56:18.719 --> 00:56:20.599 to done? 992 00:56:20.880 --> 00:56:26.000 Yeah, yeah exactly, or like onboarding new employees. It's such 993 00:56:26.199 --> 00:56:28.920 like such a pain no matter where you go, there 994 00:56:28.960 --> 00:56:33.159 is not a streamlined onboarding process right. 995 00:56:33.239 --> 00:56:35.880 Time to first commit. That's a big metric for me, 996 00:56:36.079 --> 00:56:38.880 when I bring somebody on, like how long before their 997 00:56:38.920 --> 00:56:41.840 first commit goes to production? How are you measuring that 998 00:56:43.280 --> 00:56:44.360 manually at this point? 999 00:56:45.760 --> 00:56:48.360 Is it? Because there's just not a scale that you 1000 00:56:48.400 --> 00:56:51.280 would need to make it like you're not hiring that 1001 00:56:51.360 --> 00:56:53.800 many people like your turn rates you know low, and 1002 00:56:54.679 --> 00:56:57.000 so you know, I guess that maybe the counter argument, 1003 00:56:57.559 --> 00:57:01.679 why collect the data when the manual process is still sufficient? 1004 00:57:02.559 --> 00:57:06.440 Yeah? For that for that specific example, time to first commit, 1005 00:57:06.559 --> 00:57:10.280 you know, it would be hard to justify automating it 1006 00:57:11.079 --> 00:57:13.639 unless you already had everything in place and it was 1007 00:57:13.679 --> 00:57:18.119 just building the dashboard that shows it. So that one 1008 00:57:19.840 --> 00:57:21.960 hopefully it's a little value. Like if you're putting on 1009 00:57:22.039 --> 00:57:26.000 that many new employees where you have to build a 1010 00:57:26.079 --> 00:57:29.280 dashboard for that, maybe you should be looking at metrics 1011 00:57:29.280 --> 00:57:32.119 of like what am I doing to piss my employees 1012 00:57:32.159 --> 00:57:34.639 off and make them leave? Maybe that's a better metric 1013 00:57:34.760 --> 00:57:35.840 for that scenario. 1014 00:57:36.280 --> 00:57:37.840 I mean, I think you're onto something there, because if 1015 00:57:37.880 --> 00:57:40.440 you're if you're pushing the data towards to them and 1016 00:57:40.480 --> 00:57:43.199 they have to now consume technical dashboards. I think what 1017 00:57:43.199 --> 00:57:46.800 we're saying is we're hoping that by doing this, we're 1018 00:57:46.920 --> 00:57:50.920 changing the role from directly hands on to someone that's 1019 00:57:51.000 --> 00:57:54.400 more understanding of what like of a knowledge management process 1020 00:57:54.559 --> 00:57:57.079 is in that area. So you know you're talking about HR, 1021 00:57:57.159 --> 00:58:00.320 but not so it's not HR anymore. It's a new 1022 00:58:00.400 --> 00:58:03.280 kind of human resources where it's already being managed. Now 1023 00:58:03.280 --> 00:58:06.079 it's about improving the process and that's a whole other 1024 00:58:06.119 --> 00:58:06.840 step above it. 1025 00:58:08.920 --> 00:58:12.079 Yeah, we don't have hright now, we have people business partners. 1026 00:58:12.760 --> 00:58:18.079 M Well yeah, you laugh. But I do think that 1027 00:58:18.119 --> 00:58:20.559 there is something like all labels are wrong, some of 1028 00:58:20.559 --> 00:58:22.840 them are useful. And I think if you call it people, 1029 00:58:22.960 --> 00:58:26.079 they are more two things happen. They do think about 1030 00:58:26.119 --> 00:58:29.360 their leaders, like how to build leaders and whatnot, and 1031 00:58:29.400 --> 00:58:32.639 then more importantly about the careers of these people rather 1032 00:58:32.719 --> 00:58:36.559 than as you know, fundamental resources like your turn rate 1033 00:58:36.679 --> 00:58:39.599 is important and whatnot. And I think that's that's something 1034 00:58:39.639 --> 00:58:40.760 that's only happened recently. 1035 00:58:43.360 --> 00:58:45.960 See, I don't know about calling HR people. I mean, 1036 00:58:46.000 --> 00:58:47.920 clearly they're people, right, but like they're not. 1037 00:58:51.840 --> 00:58:55.639 They're not robots yet, Like they're not there for the employees. 1038 00:58:55.639 --> 00:58:58.480 They're there to protect like the company. 1039 00:58:58.679 --> 00:59:00.920 So this idea that they're sure for you in your 1040 00:59:00.960 --> 00:59:04.000 career is maybe. 1041 00:59:02.880 --> 00:59:05.239 We don't Maybe I'm going to disagree with that. 1042 00:59:05.360 --> 00:59:08.719 Well, that's the point. So if this organization is there 1043 00:59:08.760 --> 00:59:11.360 to protect the company, then why of course the company 1044 00:59:11.360 --> 00:59:14.239 would want to be making decisions based off of a 1045 00:59:14.320 --> 00:59:17.440 metrics and a framework that is collecting actual data about 1046 00:59:17.480 --> 00:59:21.039 the organization before making those decisions. And there was something 1047 00:59:21.360 --> 00:59:23.360 there was a research study done like ten or twenty 1048 00:59:23.400 --> 00:59:26.880 years ago that was a consult came in and had asked, 1049 00:59:26.920 --> 00:59:30.480 like all the executives of an organization to make a 1050 00:59:30.519 --> 00:59:32.920 guess about how successful their sales will be over the 1051 00:59:32.920 --> 00:59:36.039 next couple of quarters. And they were all like, of course, 1052 00:59:36.079 --> 00:59:39.119 super confident about whatever it is that they were doing, 1053 00:59:39.280 --> 00:59:41.760 and just absolutely wrong in a lot of ways. And 1054 00:59:41.840 --> 00:59:43.280 I think you see the same thing over and over 1055 00:59:43.320 --> 00:59:46.039 again across the field. So like a majority of people 1056 00:59:46.199 --> 00:59:50.119 think they're better than average, which is statistically not possible. 1057 00:59:50.760 --> 00:59:52.960 And I think this is where, you know, having the 1058 00:59:52.960 --> 00:59:56.760 additional data just goes to show you that you're making 1059 00:59:56.800 --> 01:00:01.760 more accurate decisions no matter what they are. True story, 1060 01:00:05.199 --> 01:00:07.440 Will just can't wait to get back to to or 1061 01:00:07.480 --> 01:00:09.480 get to work so he can start implementing these in 1062 01:00:09.519 --> 01:00:10.760 his least favorite department. 1063 01:00:12.280 --> 01:00:13.159 I am Hotel. 1064 01:00:13.239 --> 01:00:16.079 Everything that's right Hotel everything. 1065 01:00:16.719 --> 01:00:19.239 So, speaking of which, how did you end up as 1066 01:00:20.639 --> 01:00:23.039 part of the Hotel sid group? 1067 01:00:24.360 --> 01:00:29.199 Oh? Well, so, first of all, I got into Hotel 1068 01:00:29.519 --> 01:00:35.360 because of my previous job at Lightstep, which was my 1069 01:00:35.400 --> 01:00:38.000 first job as a developer advocate. So before that, I'd 1070 01:00:38.039 --> 01:00:41.000 been doing a mix of like individual contributor work and 1071 01:00:41.039 --> 01:00:45.280 management work. I decided at that point at my previous 1072 01:00:45.360 --> 01:00:50.519 job that I'm done with management. Thanks, but no thanks, 1073 01:00:50.639 --> 01:00:55.599 had had a good run, but we dine and But 1074 01:00:57.039 --> 01:01:00.239 so the job before light Step, when I was a manager, right, 1075 01:01:00.320 --> 01:01:03.760 i'd been I was managing two teams thirteen people total, 1076 01:01:04.679 --> 01:01:08.559 and I was managing a platform engineering team and an 1077 01:01:08.599 --> 01:01:13.119 observability team. Platform engineering team was a Hashi Corp stack 1078 01:01:13.199 --> 01:01:15.960 and I knew Kubernetes and they were using Nomad. So 1079 01:01:16.000 --> 01:01:18.360 it was like, great, now I have to learn a 1080 01:01:18.440 --> 01:01:21.480 thing that I don't know, which my brain was like yay, 1081 01:01:21.599 --> 01:01:27.079 fun stuff. And also this observability team, which I was 1082 01:01:27.119 --> 01:01:31.840 new to observability. I've been dabbling. Like my understanding of 1083 01:01:31.960 --> 01:01:39.239 observability came from like reading Charity Majors tweets, and you know, 1084 01:01:39.920 --> 01:01:42.519 my thought was, well, I have to do right by 1085 01:01:42.519 --> 01:01:45.440 my team, by my organization and if I'm going to 1086 01:01:45.519 --> 01:01:48.400 lead an observability team at the organization, and we were 1087 01:01:48.440 --> 01:01:53.639 an observability practices team, So defining the observability strategy at 1088 01:01:53.639 --> 01:01:56.280 the company two cows, which is that two Cows if 1089 01:01:56.280 --> 01:02:00.719 you remember the yeah, yes, yes, but not doing that. 1090 01:02:00.840 --> 01:02:03.440 It was not the download of free Windows software anymore. 1091 01:02:03.440 --> 01:02:10.400 They're a domain wholesaler when I joined, and so as 1092 01:02:10.440 --> 01:02:12.880 part of it, I'm like at the time, I already 1093 01:02:12.920 --> 01:02:16.760 had like this blog on Medium where I've been using 1094 01:02:16.800 --> 01:02:19.760 the blog to like basically learn in public, right document 1095 01:02:20.320 --> 01:02:22.599 document cool things that I've discovered and share them with 1096 01:02:22.679 --> 01:02:26.039 the world, because my personal pet peeve is a lot 1097 01:02:26.079 --> 01:02:29.840 of stuff is documented very poorly in tech. People assume 1098 01:02:29.960 --> 01:02:33.440 that you know what they're talking about, and it makes 1099 01:02:33.440 --> 01:02:35.840 me think of those math textbooks where they're like, we'll 1100 01:02:35.960 --> 01:02:40.480 leave the proof to the reader, and it's like, no, 1101 01:02:40.679 --> 01:02:42.599 show me how the proof works, because I have no 1102 01:02:42.679 --> 01:02:45.079 freaking clue. This is you know how I feel with 1103 01:02:45.159 --> 01:02:48.960 regards to most technical blogs, so you know mine are 1104 01:02:49.000 --> 01:02:53.719 in excruciating detail. So I basically thought, well, I'm gonna 1105 01:02:53.920 --> 01:02:56.199 I'm going to learn nomad in public. I'm going to 1106 01:02:56.280 --> 01:03:00.159 learn observability in public. Blog blog blog as I'm as 1107 01:03:00.159 --> 01:03:02.000 I'm doing my job, and then one of my blog 1108 01:03:02.079 --> 01:03:05.960 posts got the attention of my former manager at Lightstep 1109 01:03:06.559 --> 01:03:08.880 and they reached out to me and said, hey, how 1110 01:03:08.920 --> 01:03:10.679 would you like to do this for a living? I'm like, 1111 01:03:10.840 --> 01:03:15.400 what you could do this for a living? And when 1112 01:03:15.400 --> 01:03:18.880 I started on the job, they said, you know, it 1113 01:03:18.880 --> 01:03:21.880 would be cool to like contribute to open telemetry. And 1114 01:03:22.480 --> 01:03:25.559 I had been in like, you know, super enterprise corporate 1115 01:03:25.599 --> 01:03:29.119 life pretty much up until that point, where like closed 1116 01:03:29.119 --> 01:03:32.320 source all the things, like the most open source stuff 1117 01:03:32.320 --> 01:03:36.079 we did was like Java and Maven and everything else 1118 01:03:36.199 --> 01:03:40.079 was like there better be, like, you know, a support 1119 01:03:40.119 --> 01:03:42.840 plan for this open source software, otherwise we're not going 1120 01:03:42.920 --> 01:03:45.480 to buy it, which otherwise we're not going to use it, 1121 01:03:45.480 --> 01:03:48.800 which I understand, like large enterprise, they got to cover 1122 01:03:48.840 --> 01:03:51.159 their asses. But so that was my first forayan to 1123 01:03:51.280 --> 01:03:55.360 open source. So I first started just contributing to the 1124 01:03:55.360 --> 01:03:58.400 hotel docs and then there was an opening in the 1125 01:03:58.440 --> 01:04:01.800 Hotel End User SIG and the end Users SIG basically 1126 01:04:01.800 --> 01:04:05.039 connects end users with each other with an open telemetry, 1127 01:04:05.039 --> 01:04:07.599 but we also relay feedback from the end users to 1128 01:04:07.760 --> 01:04:13.000 the open tele telemetry maintainers. So there was there was 1129 01:04:13.960 --> 01:04:16.719 a gap in leadership because one of the one of 1130 01:04:16.719 --> 01:04:20.119 the original founders of the SIG had changed jobs moved 1131 01:04:20.119 --> 01:04:23.639 away from Open Telemetry. So, you know, my my manager 1132 01:04:24.239 --> 01:04:26.719 at the time, asked if I wanted to step in 1133 01:04:26.760 --> 01:04:30.119 and help out, and that's that's how I got involved. 1134 01:04:30.119 --> 01:04:31.920 And at the time it was a working group and 1135 01:04:31.960 --> 01:04:36.199 then it was converted into into a SIG, And we've 1136 01:04:36.599 --> 01:04:40.760 done a bunch of things to like just really elevate 1137 01:04:41.480 --> 01:04:44.760 elevate the hotel community as part of the SIG. So 1138 01:04:44.800 --> 01:04:47.440 we do a bunch of regular things, like we have 1139 01:04:47.519 --> 01:04:51.679 this series called Hotel Me where we interview one of 1140 01:04:51.679 --> 01:04:54.280 our end users and they share how they use open 1141 01:04:54.320 --> 01:04:57.760 Telemetry in their own organizations. We have Hotel and Practice, 1142 01:04:57.760 --> 01:05:02.639 which is basically it's a meetup style thing where you know, 1143 01:05:02.719 --> 01:05:06.119 you have a cool presentation on something hotel, like, come 1144 01:05:06.199 --> 01:05:09.559 on and and and present to us, like you know, 1145 01:05:09.639 --> 01:05:11.280 you want to test out a talk that you want 1146 01:05:11.280 --> 01:05:13.920 to give, like, use us as your guinea pig, and 1147 01:05:13.920 --> 01:05:15.920 we do them as live streams and then the recordings 1148 01:05:15.920 --> 01:05:19.480 are available afterwards for folks to consume on the on 1149 01:05:19.519 --> 01:05:23.760 the Hotel YouTube channel we've run. We've partnered with the 1150 01:05:23.800 --> 01:05:28.079 other SIGs to run end user surveys to and understand like, 1151 01:05:28.159 --> 01:05:30.599 for example, the first one we did was on the 1152 01:05:30.599 --> 01:05:34.079 Hotel Collector. The collector folks wanted to partner with us 1153 01:05:34.079 --> 01:05:38.760 to understand how how end users use the collector to 1154 01:05:39.000 --> 01:05:43.559 help inform the direction of the collector, like what what 1155 01:05:43.639 --> 01:05:46.000 features are most important to the user so that they 1156 01:05:46.039 --> 01:05:49.079 can you know, push forward with those as part of 1157 01:05:49.119 --> 01:05:53.880 the Collector's roadmap. So that's effectively how I got involved 1158 01:05:53.880 --> 01:05:55.599 with Hotel. So most of my work is in the 1159 01:05:55.639 --> 01:06:00.079 sig I do. I'll pop in every so often and 1160 01:06:00.159 --> 01:06:03.400 update docs and read mes, especially like when I'm doing 1161 01:06:03.400 --> 01:06:07.159 research for my my technical talks and I'm digging into 1162 01:06:07.159 --> 01:06:09.639 a topic and I'll notice, oh there's a gap here. 1163 01:06:10.480 --> 01:06:12.599 I do two things. One is like I'll write a 1164 01:06:12.639 --> 01:06:15.480 blog post on it because I love to do that. 1165 01:06:16.199 --> 01:06:17.800 But then the other thing is like I want to 1166 01:06:17.800 --> 01:06:21.599 be a good open source citizen and also like I 1167 01:06:21.679 --> 01:06:24.119 want the docs, i e. The source of truth to 1168 01:06:24.320 --> 01:06:27.519 have the information that I also make available in my 1169 01:06:27.599 --> 01:06:31.000 blog so so that you know, we have that completeness. 1170 01:06:31.039 --> 01:06:33.440 And I encourage everyone in open source to do that 1171 01:06:33.519 --> 01:06:36.639 as well. You know, like so many vendors have wonderful 1172 01:06:36.639 --> 01:06:40.599 blog posts out there on observability, like on on Open Telemeture, 1173 01:06:40.599 --> 01:06:42.480 and I think it's great. But if you're noticing a 1174 01:06:42.519 --> 01:06:45.880 gap in the docks, like take take the time to 1175 01:06:45.880 --> 01:06:49.719 to update update those docks, update those read mes, because 1176 01:06:49.760 --> 01:06:51.639 it'll just save a lot of people a lot of 1177 01:06:51.880 --> 01:06:54.719 a lot of effort. Because the docs are the place 1178 01:06:54.880 --> 01:06:58.400 I think where most people start start their journey, and 1179 01:06:58.440 --> 01:07:00.639 then they'll you know, move to the blogs and the 1180 01:07:00.719 --> 01:07:03.920 YouTube videos and whatnot for added information. 1181 01:07:04.239 --> 01:07:09.280 Yeah, for sure. And it's like writing docks is really hard, 1182 01:07:09.719 --> 01:07:13.840 so there's there's always room for improvement, and especially for 1183 01:07:13.880 --> 01:07:17.639 people who are just starting their career, like that's such 1184 01:07:17.679 --> 01:07:23.719 a great way to just start getting getting some experience, 1185 01:07:23.920 --> 01:07:27.119 you know, because you read the docs, you try it, 1186 01:07:27.119 --> 01:07:31.159 it doesn't work, you go off, you cuss and rant 1187 01:07:31.280 --> 01:07:33.039 for a while, and then you come back and then 1188 01:07:33.079 --> 01:07:35.079 you try it again, and eventually it kind of like 1189 01:07:35.199 --> 01:07:39.400 clicks and like making that poor request back to the 1190 01:07:39.400 --> 01:07:42.480 docks is a great way to start building a portfolio 1191 01:07:42.639 --> 01:07:46.599 of expertise that will ultimately help you move on to 1192 01:07:46.679 --> 01:07:50.400 bigger and higher paying roles big time. 1193 01:07:50.480 --> 01:07:53.000 I think writing is probably one of the best things 1194 01:07:53.000 --> 01:07:55.119 you can do for your career or like whatever, whatever 1195 01:07:55.159 --> 01:07:57.440 the thing is that you do better with video or 1196 01:07:57.480 --> 01:07:59.960 audio or like whatever. Just but just start to get 1197 01:08:00.039 --> 01:08:01.599 your own voice and perspective out there. 1198 01:08:02.000 --> 01:08:04.320 I don't think we should be discouraging people from entering 1199 01:08:04.360 --> 01:08:06.320 the industry. 1200 01:08:09.280 --> 01:08:10.360 I'm saying they have to write it. 1201 01:08:11.639 --> 01:08:13.800 I mean, I don't know about other people, but I 1202 01:08:13.840 --> 01:08:17.760 became an engineer because I wanted to just solve equations 1203 01:08:17.800 --> 01:08:20.800 all the time. That that was my goal. And now 1204 01:08:20.960 --> 01:08:24.039 I don't do anything with numbers or math in any way, 1205 01:08:24.279 --> 01:08:27.119 and I spend it all the time going to conferences 1206 01:08:27.319 --> 01:08:31.520 at writing writing books like Adriana. Yeah that's my life. 1207 01:08:31.920 --> 01:08:35.239 Yeah. Well, communication is such a key part of being 1208 01:08:35.239 --> 01:08:40.640 a good engineer, and I think it's it's underplayed a lot. 1209 01:08:40.439 --> 01:08:42.600 But when it comes to writing, I know a lot 1210 01:08:42.640 --> 01:08:48.840 of people with engineering oriented minds either aren't good writers 1211 01:08:48.960 --> 01:08:52.960 or don't consider themselves good writers. And lately I've been 1212 01:08:53.159 --> 01:08:57.119 using AI so I'll write something up and then copy 1213 01:08:57.119 --> 01:09:00.359 paste it into AI and just have it just give 1214 01:09:00.399 --> 01:09:02.319 me some feedback on it. And I found that to 1215 01:09:02.399 --> 01:09:03.239 be really helpful. 1216 01:09:03.680 --> 01:09:05.560 Yeah, I think I think it's a good starting point. 1217 01:09:05.600 --> 01:09:11.079 My caution on that personally is I find AI. You know, 1218 01:09:11.119 --> 01:09:15.159 it's like you use with hair, use with care. AI 1219 01:09:15.319 --> 01:09:20.479 can take away your own like personal writing voice. So 1220 01:09:20.640 --> 01:09:24.079 that that's just my my personal take on it. 1221 01:09:24.680 --> 01:09:25.239 True story. 1222 01:09:26.079 --> 01:09:27.600 Yeah, I don't think you can have the AI just 1223 01:09:27.640 --> 01:09:29.680 like straight up right stuff for you or it is 1224 01:09:30.560 --> 01:09:33.079 like very bland, like very very very bland. But I 1225 01:09:33.079 --> 01:09:36.319 think you can use something like most of the grammar 1226 01:09:36.399 --> 01:09:37.960 check tools all incorporate AI. 1227 01:09:38.039 --> 01:09:39.840 Now, So if you're using like Grammarly or pro. 1228 01:09:39.760 --> 01:09:42.960 Writing Aid or something and you're worried about punctuation or spelling, 1229 01:09:43.479 --> 01:09:44.439 that that would be me. 1230 01:09:44.680 --> 01:09:46.960 That would be me worried about the punctuation and the spelling. 1231 01:09:47.039 --> 01:09:48.960 It's not like listen, it's just not going to happen 1232 01:09:49.119 --> 01:09:52.479 if there's not some type of tool out there, or 1233 01:09:52.560 --> 01:09:54.880 unless I hired an editor, which like I'm not going 1234 01:09:54.960 --> 01:09:55.920 to do for a blog. 1235 01:09:55.720 --> 01:09:57.760 Post, like nobody's going to do that. But I do 1236 01:09:57.840 --> 01:10:00.000 think these you know, these tools do catch quite a bit. 1237 01:10:00.159 --> 01:10:04.800 But I agree, and you know, like I'll whenever I'm questioning, 1238 01:10:04.840 --> 01:10:08.039 you know, the grammar on something, I'll throw that into 1239 01:10:08.039 --> 01:10:12.439 AI too, you know, to verify either I'm heinously wrong 1240 01:10:12.560 --> 01:10:16.520 or like hey, I got it right, Yay. 1241 01:10:16.840 --> 01:10:18.760 Do you have a tool of choice for your work? 1242 01:10:20.319 --> 01:10:25.399 For me, I'll use a Microsoft co Pilot every so often. 1243 01:10:25.600 --> 01:10:29.079 So like for my talks, for example, I have like 1244 01:10:29.319 --> 01:10:32.920 talk mascots so on. On my slides, I'll have like 1245 01:10:33.479 --> 01:10:37.439 a theme a theme animal. So for example, for my 1246 01:10:38.239 --> 01:10:41.359 one of my talks next week, the the Green Collector Talk, 1247 01:10:41.840 --> 01:10:44.920 we have a polar bear wearing a green recycling t 1248 01:10:45.039 --> 01:10:52.239 shirt throughout. So Copilot generates some some fun, some fun animations, 1249 01:10:52.399 --> 01:10:55.279 and you know, sometimes I'll ask it to like I 1250 01:10:55.279 --> 01:10:57.520 did ask it when I was researching a talk to 1251 01:10:57.640 --> 01:11:01.239 like write me some terraform code to do X, which 1252 01:11:01.279 --> 01:11:04.199 was helpful, but then it hallucinated and generated me a 1253 01:11:05.359 --> 01:11:08.079 actually it was in terraformos pullumi. It generated me a 1254 01:11:08.079 --> 01:11:10.840 Pulumi function that didn't exist, and that kind of pissed 1255 01:11:10.880 --> 01:11:13.720 me off for an hour. I'm like, where is this 1256 01:11:14.079 --> 01:11:18.760 bloody function? Did not exist? But yeah, yeah, that's that's 1257 01:11:18.840 --> 01:11:21.560 that's that's the main one that I use. My dad 1258 01:11:21.600 --> 01:11:25.039 swears by perplexity. He says it's quite good. I've never 1259 01:11:25.079 --> 01:11:27.319 tried it, but he swears by it. 1260 01:11:27.760 --> 01:11:30.680 At least one of your parents using a modern tools. 1261 01:11:30.920 --> 01:11:33.600 Oh my God. My dad is a retired software architect 1262 01:11:33.680 --> 01:11:36.840 and he like learned Rust for fun two years ago. 1263 01:11:36.880 --> 01:11:41.359 He's seventy two, and he's like, yes, I'm writing my 1264 01:11:41.439 --> 01:11:44.560 own rust crates to do like some performance testing on 1265 01:11:44.600 --> 01:11:47.640 some code that I wrote, and I'm using statistical analysis 1266 01:11:47.680 --> 01:11:50.640 methods blah blah blah. I'm like, dude, I learned this 1267 01:11:50.680 --> 01:11:54.039 stuff in university and I don't remember a bit of it. 1268 01:11:54.680 --> 01:11:58.600 And he's like, you know, refreshing his knowledge on this stuff. 1269 01:11:58.600 --> 01:12:00.479 I'm like, do you. 1270 01:12:02.399 --> 01:12:04.119 Right on? Well, this feels like a good time to 1271 01:12:04.159 --> 01:12:05.520 move over to Picks. What do you think? 1272 01:12:06.000 --> 01:12:07.319 Yes, let's do it. 1273 01:12:07.439 --> 01:12:08.479 Warren over to you. 1274 01:12:09.079 --> 01:12:10.880 Well, I knew it was going to meet me first, 1275 01:12:11.239 --> 01:12:13.399 so I was actually surprised that you weren't just going 1276 01:12:13.439 --> 01:12:14.359 to immediately go to me. 1277 01:12:14.680 --> 01:12:16.439 So yeah, I tried to change it up, Like man, 1278 01:12:16.479 --> 01:12:17.640 I always pick on Warren. 1279 01:12:17.760 --> 01:12:22.119 So well, it's okay. I'm always prepared, so it works out. 1280 01:12:23.640 --> 01:12:27.760 So today I'm going to be super lame, but we're 1281 01:12:27.760 --> 01:12:29.760 going to have a survey that's going to be posted 1282 01:12:29.800 --> 01:12:33.520 on a venture in DevOps dot com slash survey at 1283 01:12:34.119 --> 01:12:36.800 because I have them. I'm going to give away five 1284 01:12:38.199 --> 01:12:41.600 awards of aws credits based on the responses. I don't 1285 01:12:41.600 --> 01:12:42.800 know how many of them out yet. We're going to 1286 01:12:42.800 --> 01:12:45.039 see based on the responses, and I'm not sure where 1287 01:12:45.039 --> 01:12:47.000 the questions are yet, but the survey is going to 1288 01:12:47.039 --> 01:12:49.079 be there. I assure everyone. 1289 01:12:49.800 --> 01:12:52.520 Right on, that'll be cool. I'm looking forward to hearing 1290 01:12:52.960 --> 01:12:55.600 from folks on that. And I mean, you get in 1291 01:12:55.680 --> 01:12:58.880 AWS credits for it, so it's going to be a 1292 01:12:58.880 --> 01:13:01.119 little incentive to go into it. 1293 01:13:02.399 --> 01:13:06.239 Yeah, right, who doesn't love cloud credits? 1294 01:13:06.319 --> 01:13:07.399 Right for sure? 1295 01:13:08.560 --> 01:13:10.439 Andriana should bring us a pick today. 1296 01:13:10.960 --> 01:13:15.960 A thing that I really like. It's an activity. I'm 1297 01:13:15.960 --> 01:13:22.239 a rock climber, so I love bouldering. I think you know, 1298 01:13:22.439 --> 01:13:25.800 if if if you have kids that like to scramble 1299 01:13:25.880 --> 01:13:30.079 up things, highly recommend taking your kids bouldering, and also 1300 01:13:30.159 --> 01:13:33.279 as an adult try it out. That is my pick 1301 01:13:33.319 --> 01:13:36.359 for you, and as a personal thing. Every every city 1302 01:13:36.359 --> 01:13:39.479 that I visit, you know when when, whether it's on 1303 01:13:39.600 --> 01:13:42.239 vacation or at a conference, I always make it a 1304 01:13:42.239 --> 01:13:46.119 point of checking out the local bouldering gym. Bouldering is 1305 01:13:46.119 --> 01:13:48.119 a little bit scary for those who aren't familiar with 1306 01:13:48.199 --> 01:13:51.920 rock climbing because there's like the rope climbing and then 1307 01:13:51.960 --> 01:13:55.800 there's bouldering, which is like your up I want to say, 1308 01:13:55.960 --> 01:14:01.479 like ten feet up, no rope, big fluffy mat at 1309 01:14:01.479 --> 01:14:06.000 the bottom. You can still get injured. I sprained my 1310 01:14:06.039 --> 01:14:11.359 ankle twice, the same ankle from just a bad fall. 1311 01:14:11.439 --> 01:14:14.760 But it is great fun, especially if you're looking to 1312 01:14:14.960 --> 01:14:19.199 just like step outside of you know, whatever it is 1313 01:14:19.199 --> 01:14:21.000 that you do is your your day job. It's a 1314 01:14:21.039 --> 01:14:25.079 great way to just decompress because you've got nothing better 1315 01:14:25.119 --> 01:14:27.720 to do. But you know, focus on getting to the 1316 01:14:27.760 --> 01:14:30.560 next move and going up the wall, and you can't 1317 01:14:31.159 --> 01:14:34.760 your your mind can't flinch, can't get distracted because otherwise 1318 01:14:34.800 --> 01:14:35.239 you fall. 1319 01:14:37.439 --> 01:14:41.920 It's very therapeutic, Jillian, are you back with us? 1320 01:14:42.159 --> 01:14:47.119 Okay, So I'm gonna pick the newest Expeditionary Force book. 1321 01:14:47.199 --> 01:14:49.039 It's a science fiction series. 1322 01:14:48.720 --> 01:14:51.479 That's like, unlike most science fiction, it's kind of campy 1323 01:14:51.880 --> 01:14:54.279 and it kind of like goofy and silly and doesn't 1324 01:14:54.279 --> 01:14:56.359 have a lot of horror or gore or things that 1325 01:14:56.399 --> 01:14:58.720 I don't like reading, which is very hard to find 1326 01:14:58.720 --> 01:15:00.840 in science fiction I found because a lot of them 1327 01:15:01.479 --> 01:15:04.199 have just stuff that I don't want to read about. 1328 01:15:05.000 --> 01:15:08.399 So that's an Expeditionary Force by Craig Allenson. It's probably 1329 01:15:08.399 --> 01:15:11.640 one of my favorite science fiction series right on. 1330 01:15:13.279 --> 01:15:17.640 Cool. I'm going to go with two picks this week. One, 1331 01:15:17.920 --> 01:15:21.159 there's a guy, he's a kid, Let's be honest, he's 1332 01:15:21.159 --> 01:15:24.920 a kid. I think he's probably mid twenties. His name 1333 01:15:24.960 --> 01:15:30.199 is Dan Coe, and he's really fascinating just his take 1334 01:15:30.359 --> 01:15:33.199 and like the amount of work and effort he's put 1335 01:15:33.279 --> 01:15:41.359 into studying like philosophy and like the meaning of life 1336 01:15:41.399 --> 01:15:44.560 and your your purpose, and you're calling like really extraordinary 1337 01:15:44.560 --> 01:15:47.039 stuff for someone who's so young. He just released a 1338 01:15:47.039 --> 01:15:51.880 new book yesterday called Purpose Profit two that he's giving 1339 01:15:51.880 --> 01:15:55.880 it away for free on his website, the danco dot 1340 01:15:55.920 --> 01:16:01.359 com d A. N. Koe and started reading it. But 1341 01:16:01.479 --> 01:16:04.279 he's written so much other great stuff following him on 1342 01:16:04.560 --> 01:16:06.600 x His stuff there has been so cool. I'm just 1343 01:16:06.600 --> 01:16:09.560 gonna go right ahead and recommend his book before I've 1344 01:16:09.600 --> 01:16:12.560 even finished reading it. I just feel like the quality 1345 01:16:12.640 --> 01:16:16.640 is going to be there for anyone who's interested in 1346 01:16:16.680 --> 01:16:21.479 reading that. And then the second pick I have so 1347 01:16:21.920 --> 01:16:25.239 there's a really like humiliating story I'll go with first. 1348 01:16:25.319 --> 01:16:28.479 To set the stage for this, Like I spent my 1349 01:16:28.800 --> 01:16:34.439 youth going to a lot of heavy metal concerts and 1350 01:16:34.520 --> 01:16:38.479 playing guitar in heavy metal bands, and so it turns 1351 01:16:38.479 --> 01:16:42.199 out that because of all the headbanging involved, you can 1352 01:16:42.279 --> 01:16:45.319 actually get whiplash. And so I have like the long 1353 01:16:45.399 --> 01:16:52.880 term effects of whiplash from my taste in music, and 1354 01:16:52.920 --> 01:16:56.319 it's I've had lots of had some different problems with 1355 01:16:56.359 --> 01:16:58.920 it over the years. But recently I got this thing 1356 01:16:58.960 --> 01:17:02.560 called an iron neck to like strengthen my neck muscles. 1357 01:17:02.560 --> 01:17:05.399 And so that's my second pick. It's like this really 1358 01:17:05.439 --> 01:17:08.720 super cool looking head gadget that you put on your head. 1359 01:17:10.039 --> 01:17:12.119 Definitely want to you're the kid with the helmet, that's 1360 01:17:12.159 --> 01:17:16.800 what you write, right, Yeah, It's a total fashion statement, 1361 01:17:16.920 --> 01:17:19.760 absolute fashion statement. So if you're just looking to like 1362 01:17:20.319 --> 01:17:24.720 have you painted it? Like, if you want to like 1363 01:17:24.880 --> 01:17:28.640 either improve your neck muscles or just improve like your 1364 01:17:28.680 --> 01:17:31.720 social credibility in life, you want to go streatting around 1365 01:17:31.760 --> 01:17:36.039 town with the iron neck on. But yeah, right, we're. 1366 01:17:35.920 --> 01:17:40.399 Gonna have to see this the whole next episode. I 1367 01:17:40.880 --> 01:17:43.520 think it's just gonna have to be a deal. 1368 01:17:43.560 --> 01:17:45.359 Deal the next fashion statement. 1369 01:17:45.880 --> 01:17:48.439 Right, So yeah, those are my two picks. Dan Coe's 1370 01:17:48.479 --> 01:17:51.439 new book at the danco dot com and iron Neck 1371 01:17:51.760 --> 01:17:53.520 and and like The reason I bring up the iron 1372 01:17:53.560 --> 01:17:56.319 neck is even if you don't have whiplash from spinning 1373 01:17:56.399 --> 01:17:59.760 your youth headbanging I'm sitting at a desk all day 1374 01:17:59.800 --> 01:18:04.800 long hunched over your keyboard also has negative impact on 1375 01:18:04.840 --> 01:18:07.399 your posture and your next strength. So this is a 1376 01:18:07.439 --> 01:18:10.880 way to help counteract that so that whenever you do 1377 01:18:11.000 --> 01:18:12.920 make it to old age, that you still have the 1378 01:18:12.960 --> 01:18:16.600 ability to you know, stand upright or even perhaps look 1379 01:18:16.600 --> 01:18:17.039 at sky. 1380 01:18:17.520 --> 01:18:19.439 I think I'm gonna need to see some research on that, 1381 01:18:20.359 --> 01:18:23.560 all right, man, I don't know. 1382 01:18:23.600 --> 01:18:26.680 I think it's a pretty bold claim to old age. 1383 01:18:26.880 --> 01:18:27.880 I don't know about that. 1384 01:18:30.319 --> 01:18:32.760 I want like a standing ovation from the universe if 1385 01:18:32.760 --> 01:18:34.520 I make it past sixty. 1386 01:18:36.640 --> 01:18:39.439 So there's a podcast that I've been listening to called 1387 01:18:39.439 --> 01:18:43.840 Wiser than Me. It's with Seinfeld actress Julia Louis Dreyfus, 1388 01:18:44.079 --> 01:18:49.880 where she basically interviews all these like older ladies like 1389 01:18:51.279 --> 01:18:54.479 you know, high profile older ladies like Jane Fonda, Amy 1390 01:18:54.560 --> 01:18:58.560 Pan and it's been it was recommended to me by 1391 01:18:58.640 --> 01:19:02.079 actually one of my podcast guests. It is great fun 1392 01:19:02.920 --> 01:19:04.319 right on cool. 1393 01:19:04.399 --> 01:19:07.479 And then there's your podcast as well, right, the Geeking 1394 01:19:07.520 --> 01:19:09.640 Out podcast that is correct. 1395 01:19:09.760 --> 01:19:13.439 Yes, just look up geeking Out with Adriana Villela, because 1396 01:19:13.479 --> 01:19:16.279 otherwise if you just look up geeking out, it's gonna 1397 01:19:16.359 --> 01:19:19.479 like give you like so many different listings on on 1398 01:19:19.560 --> 01:19:23.600 the various podcasting apps. So geeking Out with Adriana Villela 1399 01:19:24.119 --> 01:19:26.199 do that search term. You should be able to find 1400 01:19:26.239 --> 01:19:28.840 the correct one. And there's a copy Vada on the 1401 01:19:29.960 --> 01:19:31.319 on on the cover art. 1402 01:19:31.479 --> 01:19:36.800 So awesome. Adriana, thank you so much for being on 1403 01:19:36.840 --> 01:19:41.920 the show. This has been fun. Thanks for having me anytime, 1404 01:19:42.239 --> 01:19:45.560 come on back anytime that you want. Warren Chilean as always, 1405 01:19:45.600 --> 01:19:46.720 thank you both for being here. 1406 01:19:48.000 --> 01:19:49.000 Thank you was fun. 1407 01:19:50.520 --> 01:19:53.560 And to all of our listeners, thank you for listening 1408 01:19:53.640 --> 01:19:57.039 to the show, and be sure and check out the 1409 01:19:57.199 --> 01:20:02.760 website for the survey to get your elf some AWS credits. Alright, cool, 1410 01:20:02.800 --> 01:20:03.720 we'll see you guys next week. 1411 01:20:07.279 --> 01:20:07.920 M hmm.