WEBVTT 1 00:00:02.000 --> 00:00:05.480 Go wild. So that's how the episode starts with Jillian Wild. 2 00:00:07.000 --> 00:00:11.080 Speaking of which, staying on the internet, Yeah too late, 3 00:00:11.480 --> 00:00:16.280 it's too late. Yeah, okay, Speaking of which, you are 4 00:00:16.399 --> 00:00:19.280 back on the show with us. You're back in the US, 5 00:00:20.359 --> 00:00:21.800 So it's been going on. 6 00:00:23.760 --> 00:00:26.399 Well, I mean I moved continent, so that was that 7 00:00:26.519 --> 00:00:27.399 was a pretty big thing. 8 00:00:27.600 --> 00:00:29.359 Now I'm in the US full time, which. 9 00:00:29.160 --> 00:00:32.159 Is exciting, Like just two hours north of Boston and 10 00:00:32.320 --> 00:00:34.560 a little town that probably nobody has ever heard of 11 00:00:34.600 --> 00:00:36.640 in New Hampshire. But it's been fun. 12 00:00:36.759 --> 00:00:38.600 It's been it's been a big adjustment, but it's been 13 00:00:38.600 --> 00:00:39.240 a really good time. 14 00:00:40.320 --> 00:00:44.039 Yeah. From you were in Doka. 15 00:00:43.799 --> 00:00:45.880 Right Doha? 16 00:00:46.399 --> 00:00:50.600 Oh yeah, yeah, in the Middle East, So yeah, Doha 17 00:00:50.759 --> 00:00:54.520 to New Hampshire. There's some adjustment there. 18 00:00:54.840 --> 00:00:57.359 There, there is. There's a lot more color here, like 19 00:00:57.479 --> 00:00:58.799 just in general the story. 20 00:01:00.640 --> 00:01:03.320 I'm sure. Yeah, I've not been to Dolla, but I 21 00:01:03.320 --> 00:01:05.840 have been to Dubai several times, and I think they're 22 00:01:05.879 --> 00:01:06.719 fairly similar. 23 00:01:06.879 --> 00:01:09.040 Yeah. Yeah, it's like less than an hour's plane ride. 24 00:01:09.079 --> 00:01:10.040 They're about the same thing. 25 00:01:10.640 --> 00:01:15.599 Yeah. Cool. So back in the US permanently or is 26 00:01:15.599 --> 00:01:17.599 this a temporary thing for you? 27 00:01:17.680 --> 00:01:19.359 Permanently? No living here now? 28 00:01:19.879 --> 00:01:22.640 Right on. Well cool, Well, welcome back home. 29 00:01:23.239 --> 00:01:24.640 Thank you. It's been nice. 30 00:01:24.640 --> 00:01:28.079 I'm very happy to be back, and it's nice that professionally, 31 00:01:28.120 --> 00:01:29.760 it's so much better for me too. I don't have 32 00:01:29.799 --> 00:01:32.120 to work split shifts and that's great, and I get 33 00:01:32.159 --> 00:01:34.359 to be back on the podcast and talk to you guys, 34 00:01:34.400 --> 00:01:35.599 which is obviously. 35 00:01:35.200 --> 00:01:35.879 The biggest win. 36 00:01:36.719 --> 00:01:37.280 Of course. 37 00:01:37.400 --> 00:01:40.359 Yeah, of course, go harass my clients in Boston. Now 38 00:01:40.400 --> 00:01:42.599 that's where most of them are at and here we are. 39 00:01:42.879 --> 00:01:45.040 Oh but two hours like and you know, so you 40 00:01:45.079 --> 00:01:47.599 said maybe no one knows about that, but I actually 41 00:01:47.599 --> 00:01:50.879 made that drive fair number of times up to Lake Ossipe, 42 00:01:51.000 --> 00:01:55.200 So there's there's a whole bunch of town there that 43 00:01:55.239 --> 00:01:58.719 I could probably rattle off. And maybe maybe you're living 44 00:01:58.719 --> 00:02:01.319 in one of those. That's a long I get from 45 00:02:02.680 --> 00:02:03.640 New Hampshire downtown. 46 00:02:04.000 --> 00:02:06.400 I'm not driving, though, so it doesn't bother me. I 47 00:02:06.480 --> 00:02:08.360 drive to the bus station and then one of those 48 00:02:08.479 --> 00:02:11.560 very nice buses takes me into Boston. I'm not trying 49 00:02:11.560 --> 00:02:13.319 to drive in Boston, Like, that's not a thing that 50 00:02:13.319 --> 00:02:13.800 we're doing. 51 00:02:15.039 --> 00:02:17.360 It doesn't really matter, right because you're on the East 52 00:02:17.439 --> 00:02:19.560 coast where they actually have public transit. 53 00:02:20.840 --> 00:02:25.199 No, the bus is actually nice, Like I don't actually 54 00:02:25.240 --> 00:02:26.960 mind taking the bus. It has like the really nice 55 00:02:26.960 --> 00:02:29.840 big seats and it goes right into South Station and 56 00:02:29.879 --> 00:02:31.759 then I don't know, most of my clients are like 57 00:02:31.759 --> 00:02:33.120 ten fifteen minutes from there. 58 00:02:34.680 --> 00:02:36.400 That's cool. I do kind of miss that. I miss 59 00:02:36.479 --> 00:02:38.479 that every once in a while of just seeing and 60 00:02:38.520 --> 00:02:39.919 working with people face to face. 61 00:02:41.479 --> 00:02:43.400 I do. I actually do like that, Like I wouldn't 62 00:02:43.479 --> 00:02:45.599 even really want to have a hybrid job, but showing 63 00:02:45.680 --> 00:02:48.120 up like I don't know quarterly and just going to 64 00:02:48.120 --> 00:02:50.560 go see everybody. Usually it's for some type of meetup. 65 00:02:50.599 --> 00:02:53.400 There's like, you know, a big meeting of the mind's 66 00:02:53.400 --> 00:02:55.759 happening or a conference or a talk or something like that, 67 00:02:55.840 --> 00:02:58.159 and then I'll go for sure. 68 00:02:59.439 --> 00:03:02.840 So what was it like moving across continents? Did you 69 00:03:02.879 --> 00:03:08.240 like bring like couches and mattresses and furniture and did no? 70 00:03:09.159 --> 00:03:12.800 No, no, it's all I don't know. 71 00:03:12.879 --> 00:03:15.159 I guess like living overseas, all the furniture and stuff 72 00:03:15.199 --> 00:03:17.439 got treated as kind of disposable because there was enough 73 00:03:17.479 --> 00:03:20.319 of those moves. And the last place that we were 74 00:03:20.360 --> 00:03:22.360 at was the last place that we were at was 75 00:03:22.360 --> 00:03:25.080 mostly furnished anyway, So I didn't have a lot of furniture. 76 00:03:25.599 --> 00:03:28.800 So I bought like furniture when I bought the house 77 00:03:28.800 --> 00:03:30.800 that I'm living in now right on. 78 00:03:31.520 --> 00:03:35.840 Yeah, cool, We did that whenever we moved to Oregon 79 00:03:35.879 --> 00:03:38.800 from Arizona. The people that bought our house in Arizona 80 00:03:39.479 --> 00:03:42.000 wanted to buy most of the furniture too, So when 81 00:03:42.000 --> 00:03:46.120 we got up here, we had almost no furniture. And 82 00:03:46.240 --> 00:03:47.639 so the first thing we had to do was, you know, 83 00:03:47.680 --> 00:03:53.479 go drop thousands of dollars on furniture. And I can't 84 00:03:53.520 --> 00:03:56.159 say we made the best decisions, you know, because like 85 00:03:56.199 --> 00:04:00.680 you're like desperate for a couch and a mass so 86 00:04:01.479 --> 00:04:04.199 you don't really research it or take time to say 87 00:04:05.560 --> 00:04:09.319 do I really like this? You just like say I 88 00:04:09.319 --> 00:04:12.520 think I'll like this, and then three months later you're like, 89 00:04:14.280 --> 00:04:16.160 I probably should have gone a different direction. 90 00:04:16.519 --> 00:04:17.680 You're an amateur mover. 91 00:04:17.839 --> 00:04:18.160 That's fun. 92 00:04:18.360 --> 00:04:20.279 Yeah, I mean I do, because I'm insane. 93 00:04:20.399 --> 00:04:22.800 Like I would just buy some of the Amazon like 94 00:04:22.839 --> 00:04:25.360 Futon mattresses, like the kind that I take when I 95 00:04:25.360 --> 00:04:27.399 go camping, and I would throw those on the floor 96 00:04:27.439 --> 00:04:29.439 with some pillows, and that's what we would have until 97 00:04:29.480 --> 00:04:32.519 I found a couch that I obsessed research and I 98 00:04:32.560 --> 00:04:35.920 have like a whole decision matrix of couch versus size 99 00:04:35.959 --> 00:04:37.600 in the living room versus how I'm gonna like this 100 00:04:37.759 --> 00:04:40.079 versus different seasons, and how I'm gonna like it? How 101 00:04:40.160 --> 00:04:43.959 much can the destroy it? Like, there's a lot. 102 00:04:43.800 --> 00:04:45.600 That goes into these decisions for. 103 00:04:45.519 --> 00:04:47.279 Me, So you figured it out. 104 00:04:47.680 --> 00:04:50.040 That's not how I do anything, ever. 105 00:04:50.519 --> 00:04:53.600 The futon sofa and I also used to buy a 106 00:04:53.680 --> 00:04:57.439 queen size blow up mattress because you know that's fairly 107 00:04:57.519 --> 00:05:00.360 cheap and can help you a lot there with even 108 00:05:00.360 --> 00:05:02.240 knowing what your situation is going to be. If you 109 00:05:02.319 --> 00:05:05.879 can't find a quick futon sofa and then yeah, then 110 00:05:05.879 --> 00:05:08.839 you've got as much time as you're willing to sacrifice 111 00:05:08.920 --> 00:05:13.000 into living into this terrible situation until you actually define 112 00:05:13.000 --> 00:05:15.360 the mattress you're looking for, the furniture you're willing to 113 00:05:16.040 --> 00:05:19.040 I got a lot of black previously because I used 114 00:05:19.040 --> 00:05:26.199 to use my moving boxes as partial furniture. This box 115 00:05:26.279 --> 00:05:28.800 turned sideways is like a shelf. You stack some boxes 116 00:05:28.839 --> 00:05:30.720 on top, and then you have a bureau made out 117 00:05:30.759 --> 00:05:31.319 of cardboard. 118 00:05:31.639 --> 00:05:32.319 There, he does. 119 00:05:35.519 --> 00:05:37.360 Although I am willing to fight you on the air 120 00:05:37.360 --> 00:05:40.519 mattresses because air mattresses are like one hundred bucks now, 121 00:05:40.600 --> 00:05:43.600 and my kids break them fairly regularly, so they don't 122 00:05:43.639 --> 00:05:45.600 last long enough for us, which is why I have 123 00:05:45.720 --> 00:05:47.360 moved on, which is why I moved on to the 124 00:05:47.399 --> 00:05:51.959 futon mattress like lifestyle, and I'm willing to commit to that. 125 00:05:52.120 --> 00:05:53.519 Yeah, for life. 126 00:05:53.680 --> 00:05:54.079 That's it. 127 00:05:54.560 --> 00:05:56.560 Nothing bad is going to happen to host mattresses ever, 128 00:05:57.120 --> 00:05:57.399 right on. 129 00:05:57.680 --> 00:06:02.160 Yeah, cool, Well, welcome back. Excited to have you back 130 00:06:02.199 --> 00:06:02.959 on the show. 131 00:06:03.959 --> 00:06:06.279 And thank you for letting me back. 132 00:06:07.199 --> 00:06:12.439 Yeah, of course, anytime, especially for today's episode because we're 133 00:06:12.480 --> 00:06:18.800 talking about the twenty twenty four Door Report. Because I mean, 134 00:06:18.920 --> 00:06:21.879 by the time this episode releases, I'm sure everyone will 135 00:06:21.879 --> 00:06:25.160 have read all one hundred and twenty pages and this 136 00:06:25.199 --> 00:06:27.199 is going to be a pretty boring episode because you 137 00:06:27.240 --> 00:06:31.240 will have digested all the information already. But for the 138 00:06:31.560 --> 00:06:33.800 two or three people who didn't read it, we're here 139 00:06:33.879 --> 00:06:35.759 to get you at the speed so that you can 140 00:06:36.680 --> 00:06:39.600 converse with everyone else as if you spent just as 141 00:06:39.680 --> 00:06:41.399 much time reading it as they did. 142 00:06:42.560 --> 00:06:44.720 So I think instead of people reading it, we're all 143 00:06:44.720 --> 00:06:49.000 going to have to brains and then the report will 144 00:06:49.040 --> 00:06:51.839 automatically be wrong because it says there ai ho's and 145 00:06:51.920 --> 00:06:56.439 how as much have been impact as was expected, except 146 00:06:56.439 --> 00:07:00.519 in Drug Discovery, where it's been insane. But what could happen? 147 00:07:01.720 --> 00:07:03.920 I honestly, while I was reading it, I had this 148 00:07:04.040 --> 00:07:06.680 fear like, oh, you like, our whole episode could have 149 00:07:06.839 --> 00:07:11.079 just been feeding the report into notebook LM and having 150 00:07:11.160 --> 00:07:15.079 that running instead of the legitimate adventures and DevOps for 151 00:07:15.279 --> 00:07:18.680 our you know, faithful viewers here. 152 00:07:20.800 --> 00:07:27.319 Well, I haven't actually read, so. 153 00:07:27.360 --> 00:07:29.759 I did go through. I did go through almost all 154 00:07:29.800 --> 00:07:30.040 of it. 155 00:07:30.439 --> 00:07:32.839 There were some pages at the end that I skipped 156 00:07:32.839 --> 00:07:36.160 that was just repeating a little bit about what Dora 157 00:07:36.319 --> 00:07:39.560 does and for those that actually don't know, Dora was 158 00:07:39.560 --> 00:07:43.879 a separate company that Google bought in twenty eighteen that 159 00:07:44.000 --> 00:07:48.439 was focused on the delivery metrics that we all know 160 00:07:48.720 --> 00:07:51.600 and now love in and around engineering to have. 161 00:07:52.480 --> 00:07:53.399 I mean, there's nothing else. 162 00:07:53.560 --> 00:07:55.199 I say, there's nothing else, although there are a bunch 163 00:07:55.199 --> 00:07:57.839 of companies that have started to experiment with a sort 164 00:07:57.879 --> 00:08:00.800 of revision of them called Space, which I don't have 165 00:08:00.839 --> 00:08:06.639 a lot of experience with, but does handle additional aspects 166 00:08:06.680 --> 00:08:08.879 that some of the Dora metrics are missing. 167 00:08:09.800 --> 00:08:11.839 Is space another acronym for something? 168 00:08:12.319 --> 00:08:15.079 Yeah, you know, because people love acronyms and they don't 169 00:08:15.079 --> 00:08:17.680 mean anything, and I'm sure it stands for something, but 170 00:08:19.000 --> 00:08:22.240 I think realistically most organizations can do a better job 171 00:08:22.319 --> 00:08:25.800 by just understanding where their metrics are currently at and 172 00:08:25.839 --> 00:08:28.439 focusing on improving them irrelevant of what they are, because 173 00:08:28.480 --> 00:08:32.799 no metric is entirely perfect. Because of the companies that 174 00:08:32.879 --> 00:08:36.600 I've worked with in my history, often what happens is 175 00:08:36.840 --> 00:08:39.360 they say, oh, we're we have great Dora metrics, or 176 00:08:39.399 --> 00:08:41.399 we even have bad door metrics, and we need to 177 00:08:41.399 --> 00:08:44.000 do better improving them. And then I look at how 178 00:08:44.000 --> 00:08:47.399 they're capturing those metrics and they couldn't be more wrong. 179 00:08:47.639 --> 00:08:50.200 Like you look at them and they're say, let's say, 180 00:08:50.320 --> 00:08:53.559 using feature flags and whatnot, and so you're looking at 181 00:08:53.559 --> 00:08:55.919 your time to roll out changes in production. But those 182 00:08:56.200 --> 00:08:59.200 changes in production have never been tested, they're never being used, 183 00:08:59.240 --> 00:09:02.360 and they're behind flags that aren't even being that don't 184 00:09:02.399 --> 00:09:05.080 make that functionality available to your customers. And so it's like, 185 00:09:05.080 --> 00:09:07.039 can you really even say it's in the production. And 186 00:09:07.159 --> 00:09:09.279 yet these companies are saying, oh, yeah, we roll out 187 00:09:09.320 --> 00:09:11.679 our code immediately. It's all in craud, it's all being used, 188 00:09:11.919 --> 00:09:15.399 and so our values for that are really high. So 189 00:09:15.519 --> 00:09:17.480 I mean, I think in those cases that it doesn't 190 00:09:17.480 --> 00:09:20.120 matter what the metric is if you're recording it in 191 00:09:20.120 --> 00:09:22.919 a way which doesn't encourage good engineering practices. 192 00:09:24.399 --> 00:09:26.480 So I think maybe we should backtrack that a little bit. 193 00:09:26.519 --> 00:09:27.600 What is a Dora metric? 194 00:09:30.080 --> 00:09:34.399 Dora is this little girl in a cartoon with a 195 00:09:34.440 --> 00:09:40.360 monkey that gets chased by a fox, and evidently she 196 00:09:40.399 --> 00:09:44.240 does reporting in between episodes of making cartoons. 197 00:09:46.080 --> 00:09:48.440 That's fair. I'll bet she'd be a better reporter than most. 198 00:09:49.519 --> 00:09:55.480 Yeah, that was a reference to Dora the Explorer. For 199 00:09:55.519 --> 00:09:58.960 anyone who didn't get that joke, or. 200 00:09:59.080 --> 00:10:02.440 Maybe before my time, it was gone. 201 00:10:03.200 --> 00:10:04.879 Yeah, it was when my kids were young, so it's 202 00:10:04.879 --> 00:10:06.759 probably pretty old at this point. 203 00:10:07.159 --> 00:10:09.919 My oldest really went through like a phase where she 204 00:10:09.960 --> 00:10:10.720 really loved Dora. 205 00:10:10.960 --> 00:10:12.000 She was like three or something. 206 00:10:12.000 --> 00:10:14.000 That was very cute. It was one of the shows 207 00:10:14.000 --> 00:10:16.200 that I actually liked too. Not all of them, but 208 00:10:16.279 --> 00:10:17.360 some of them are adorable. 209 00:10:18.080 --> 00:10:22.120 I'm sorry, I totally derailed the topic. What was your question, Jillian? 210 00:10:22.480 --> 00:10:24.879 Yeah, what is it? Door? We're talking about Dora metrics 211 00:10:24.879 --> 00:10:26.759 and it's like, well, what is a Dora metric? First 212 00:10:26.759 --> 00:10:28.039 of all, I mean, I suppose we can all kind 213 00:10:28.039 --> 00:10:30.360 of infer it, right, It's some type of metric that 214 00:10:30.399 --> 00:10:33.320 we're using to and for how devopsy we really all are, 215 00:10:33.559 --> 00:10:36.559 Like who's got the highest score? Is there a scoreboard 216 00:10:36.639 --> 00:10:38.879 that we can we can enter ourselves into, Like what 217 00:10:38.919 --> 00:10:39.600 are we doing here? 218 00:10:40.240 --> 00:10:42.799 So there was actually like four official ones that everyone 219 00:10:43.080 --> 00:10:46.720 could could be using. And this is where the test 220 00:10:46.759 --> 00:10:48.720 really comes because I don't know if I remember all 221 00:10:48.720 --> 00:10:49.159 of them off. 222 00:10:49.159 --> 00:10:49.679 On my head. 223 00:10:50.159 --> 00:10:53.320 But there is a mean time to resolution when a 224 00:10:53.360 --> 00:10:56.720 failure happens and you need to go and resolve it 225 00:10:56.799 --> 00:11:00.360 and get your production state back to working according to 226 00:11:00.440 --> 00:11:05.200 your SLAS. Then there's cycle time to deploy changes from 227 00:11:05.240 --> 00:11:09.480 the time you start working on them. There's change failure rate, 228 00:11:09.639 --> 00:11:14.159 how often failures actually happen because of the changes you're making. 229 00:11:14.200 --> 00:11:16.960 And I missed one and I'm hoping Will knows which 230 00:11:16.960 --> 00:11:17.600 one I missed. 231 00:11:18.200 --> 00:11:23.840 Yeah, we have change lead time, deployment frequency with frequency yeah, 232 00:11:24.440 --> 00:11:28.200 then the change fail rate and deployment recovery time. 233 00:11:28.759 --> 00:11:30.320 Yeah. 234 00:11:31.000 --> 00:11:33.279 Yeah. So it's interesting. It's one hundred and twenty pages 235 00:11:33.600 --> 00:11:39.799 of texts that just boils down to those four metrics. 236 00:11:40.120 --> 00:11:42.159 So normally, normally that's what they talk about. 237 00:11:42.799 --> 00:11:44.759 And so the question is that since they've been doing 238 00:11:44.799 --> 00:11:48.600 this what's new every year that is relevant because once 239 00:11:48.639 --> 00:11:50.840 you sort of share that these are the metrics that 240 00:11:50.879 --> 00:11:53.759 every engineering team could be utilizing to decide like how 241 00:11:53.799 --> 00:11:57.080 great they are. And they talk about benchmarks and where 242 00:11:57.519 --> 00:12:02.679 elite teams are versus maybe behind the curve teams that 243 00:12:02.799 --> 00:12:05.519 have a long way to go, what changes from year 244 00:12:05.519 --> 00:12:10.039 to year. And so this year they dove a lot 245 00:12:10.080 --> 00:12:13.600 more into three areas. The first one was they started 246 00:12:13.600 --> 00:12:17.240 talking about a new metric the amount of rework you have, 247 00:12:17.679 --> 00:12:19.879 so when you thought something was completed and you have 248 00:12:19.960 --> 00:12:22.679 to go back. And this is really interesting because it 249 00:12:22.679 --> 00:12:24.679 was always something that made a lot of sense to 250 00:12:24.720 --> 00:12:27.399 me and wasn't included something else that they don't really 251 00:12:27.399 --> 00:12:29.519 talk about. It's like the number of support tickets you get, 252 00:12:29.559 --> 00:12:30.960 like whether or not, Like there may not be a 253 00:12:30.960 --> 00:12:33.440 production incident in any way, but your customers may be 254 00:12:33.480 --> 00:12:35.559 concerned and confused, your users. 255 00:12:35.240 --> 00:12:37.440 May not know what's going on, and that's not really included. 256 00:12:37.799 --> 00:12:40.960 So they talk about expanding the metrics and maybe breaking 257 00:12:40.960 --> 00:12:43.679 them down into different areas that are all independent of 258 00:12:43.720 --> 00:12:46.279 each other. And that's sort of what's important, right, Metrics 259 00:12:46.320 --> 00:12:49.720 that depend on each other or have high correlation or 260 00:12:49.799 --> 00:12:52.919 high cohesion tend not to be as effective because they measure. 261 00:12:52.720 --> 00:12:53.200 The same thing. 262 00:12:54.039 --> 00:12:57.879 Then the report moves on to in depth review of 263 00:12:58.320 --> 00:13:02.120 the impact AI in our industry and the world, really 264 00:13:02.159 --> 00:13:04.840 about engineering teams and software, and the last part talks 265 00:13:04.879 --> 00:13:07.679 a lot about platform engineering and its impact. 266 00:13:09.120 --> 00:13:11.600 Yeah, before we dig into those, I think the thing 267 00:13:11.639 --> 00:13:16.120 that stood out to me most in here is the 268 00:13:16.159 --> 00:13:20.720 section applying insights from Dora, and it says the recommended 269 00:13:20.759 --> 00:13:24.120 approach is to identify the outcome you would like to improve, 270 00:13:24.919 --> 00:13:29.080 measure your baseline or current state, and then I feel 271 00:13:29.080 --> 00:13:34.960 like this is where so many places start falling down. 272 00:13:35.399 --> 00:13:38.120 So the next step is to develop a hypothesis about 273 00:13:38.120 --> 00:13:40.759 what might get you to your closer state, and then 274 00:13:41.039 --> 00:13:45.000 agree to a plan for improvement, do the work, and 275 00:13:45.039 --> 00:13:47.399 then the last one measure the progress that you've made. 276 00:13:47.600 --> 00:13:50.480 I feel like that's where we fail a lot of times. 277 00:13:50.559 --> 00:13:52.639 You know, we like say, hey, we're going to make 278 00:13:52.639 --> 00:13:56.279 this better, and then we start working on it and 279 00:13:56.360 --> 00:13:59.960 never take a step back to say did we actually 280 00:14:00.080 --> 00:14:03.519 make it better? And if not, let's rip that out, 281 00:14:03.600 --> 00:14:07.639 And so over time we end up dragging all of 282 00:14:07.639 --> 00:14:12.799 these failed attempts with us adding you know, layers of 283 00:14:13.759 --> 00:14:19.919 complexity or bureaucracy or whatever the addition was, and we 284 00:14:19.960 --> 00:14:22.000 just keep dragging it along with us, even though it 285 00:14:22.039 --> 00:14:27.000 doesn't work for us or improve the situation for us. 286 00:14:27.559 --> 00:14:27.759 Yeah. 287 00:14:27.799 --> 00:14:29.440 No, I mean I think we bring that up a 288 00:14:29.480 --> 00:14:32.759 lot in this podcast, and I think the report really 289 00:14:32.799 --> 00:14:36.000 dives into it in the executive summary section. Which is 290 00:14:37.000 --> 00:14:41.159 an experiment is a hypothesis that is time bound with 291 00:14:42.039 --> 00:14:46.200 retrospective at some point. And often I think we do 292 00:14:46.279 --> 00:14:48.759 see a lot of organizations say, oh, we could try 293 00:14:48.799 --> 00:14:52.080 this thing, which is great, like you should absolutely experiment, 294 00:14:52.639 --> 00:14:55.320 but then there is no review, like was it even 295 00:14:55.360 --> 00:14:56.039 a good thing. 296 00:14:55.879 --> 00:14:56.480 For us to do? 297 00:14:56.960 --> 00:15:00.840 And that's like there is no like any experiment is 298 00:15:00.879 --> 00:15:03.480 a success, even if it rejects your hypothesis, if you 299 00:15:03.639 --> 00:15:06.320 come out with some learning. The only failed experiment is 300 00:15:06.320 --> 00:15:09.519 when you start it and never stop. And I think 301 00:15:09.519 --> 00:15:12.080 that's what a lot of organizations end up doing. They 302 00:15:12.159 --> 00:15:15.000 try something out, they experiment with a different team structure, 303 00:15:15.799 --> 00:15:18.399 even something that we believe is good. You know. I 304 00:15:18.399 --> 00:15:20.679 think it was a few years ago that the concept 305 00:15:20.840 --> 00:15:24.519 of team toypapologies came out with different types of teams 306 00:15:24.600 --> 00:15:28.679 and how they will work, stream aligned teams, platform teams, 307 00:15:29.759 --> 00:15:33.279 complex subsystem teams, etc. And it's a great idea to 308 00:15:33.320 --> 00:15:37.159 shift an organizational structure to focus on these, But even 309 00:15:37.200 --> 00:15:39.360 if it's a thing that everyone agrees on, you should 310 00:15:39.399 --> 00:15:42.080 still be like, we believe that a different org structure 311 00:15:42.080 --> 00:15:45.519 would make sense for us, and then measure why that 312 00:15:45.679 --> 00:15:47.720 is the case at some point later, like did it 313 00:15:47.759 --> 00:15:51.039 actually have an impact? Because if you don't do that, 314 00:15:51.120 --> 00:15:52.720 someone else is just going to say later, oh, that 315 00:15:52.879 --> 00:15:54.360 was bad for us, and you're not going to have 316 00:15:54.399 --> 00:15:57.679 any data to really back that up. A common argument 317 00:15:57.720 --> 00:16:03.519 against that has been the notion like rapid reteaming or 318 00:16:03.600 --> 00:16:06.200 dynamic reteaming. I think it's a book on the topic, actually, 319 00:16:06.360 --> 00:16:09.720 which is this concept ofok, like you swarm on an idea, 320 00:16:09.799 --> 00:16:11.759 you just get it to completion, and the you, you know, 321 00:16:11.879 --> 00:16:13.919 di spand the team and create a new team to 322 00:16:13.960 --> 00:16:16.600 focus on it, and without really understanding when your core 323 00:16:16.679 --> 00:16:19.759 businesses or why you're making the change, just arbitrarily making 324 00:16:19.840 --> 00:16:22.639 changes your organization, you don't know if it's going to 325 00:16:22.720 --> 00:16:23.639 be successful in the end. 326 00:16:24.120 --> 00:16:27.000 So my favorite way of thinking about applying the scientific 327 00:16:27.039 --> 00:16:29.519 method is when it involves people, because the only way 328 00:16:29.519 --> 00:16:31.519 that you can actually do that is to spin off 329 00:16:31.559 --> 00:16:35.879 parallel universes and have this whole double lunch where you 330 00:16:36.000 --> 00:16:38.679 have all these simulations of different people, and then you 331 00:16:38.720 --> 00:16:40.480 have your crystal ball where you're like, well, how did 332 00:16:40.480 --> 00:16:43.440 this actually work and how many times? And then you 333 00:16:43.440 --> 00:16:45.399 have the different iterations, and then you would have different 334 00:16:45.399 --> 00:16:48.440 like branching models of the different iterations too, and since 335 00:16:48.480 --> 00:16:50.679 you can't actually do that, you just get to guess. 336 00:16:50.679 --> 00:16:54.320 But at least with technology, we could have isolated environments 337 00:16:54.360 --> 00:16:58.440 where we really applied the scientific method. But you can't 338 00:16:58.440 --> 00:17:01.559 so much with people because people are chaos. 339 00:17:02.039 --> 00:17:03.440 I mean, you're absolutely right, Jillian. 340 00:17:03.480 --> 00:17:05.680 I feel like most companies don't have the sort of 341 00:17:05.720 --> 00:17:07.799 scale to be able to do this, and it's a 342 00:17:07.839 --> 00:17:10.599 mistake to believe, especially if you're like selling to businesses, 343 00:17:10.720 --> 00:17:14.680 that you have the ability to segregate a clear control 344 00:17:14.759 --> 00:17:17.359 group and a test and multiple test groups to validate 345 00:17:17.400 --> 00:17:19.559 what's going on. I do remember there was a not 346 00:17:19.720 --> 00:17:23.680 great book that referenced one of the major tech companies 347 00:17:25.039 --> 00:17:28.279 in the world today, and they had talked extensively about 348 00:17:28.319 --> 00:17:32.920 horse trading, individual countries, populations, as far as users go 349 00:17:33.240 --> 00:17:38.920 to perform human psychological experiments on the information that the 350 00:17:39.079 --> 00:17:40.160 company was providing them. 351 00:17:40.160 --> 00:17:42.720 To see how humans would deal with that sort of. 352 00:17:42.759 --> 00:17:46.559 Information, and so like at scale, you know, there is something, 353 00:17:46.599 --> 00:17:49.039 but that's not like most of the companies, Like it's 354 00:17:49.039 --> 00:17:50.839 not my company. I don't think it's well as company. 355 00:17:50.880 --> 00:17:53.000 I'm sure it's not yours, Agilian, Like most people don't 356 00:17:53.000 --> 00:17:54.839 work at a company where this is actually a real 357 00:17:54.839 --> 00:17:57.160 thing that could be done at a human level scale. 358 00:17:58.279 --> 00:18:02.359 No, I'm also uncomfortable with idea of like the human experimentation, 359 00:18:02.599 --> 00:18:08.160 Like my scenario was a lot more fun because I 360 00:18:08.160 --> 00:18:09.680 would just like to point that out there. 361 00:18:11.319 --> 00:18:14.079 So what you're saying is it's okay to experiment with 362 00:18:14.200 --> 00:18:17.039 humans as long as they're in parallel universes, just not 363 00:18:17.160 --> 00:18:17.799 in this one. 364 00:18:18.599 --> 00:18:23.440 Yeah, fine, I mean we agreed, that's all fine, Like 365 00:18:23.440 --> 00:18:25.759 we all like the crossover events and the Arrowverse, like 366 00:18:25.799 --> 00:18:26.559 this is what we do. 367 00:18:27.200 --> 00:18:27.960 Okay, I wouldn't. 368 00:18:27.960 --> 00:18:30.559 I wouldn't have mentioned the arrow Verse, but you know, 369 00:18:30.680 --> 00:18:33.920 I uh yeah, I mean I I'll just have done 370 00:18:34.119 --> 00:18:37.079 quite heavily in science fiction and fantasy, for sure. I 371 00:18:37.079 --> 00:18:39.559 Mean I always liked the question like did you did 372 00:18:39.640 --> 00:18:40.359 you enjoy. 373 00:18:40.160 --> 00:18:41.319 Being an older sibling? 374 00:18:41.920 --> 00:18:43.640 Because I only have younger siblings and I'm like I 375 00:18:43.640 --> 00:18:45.680 honestly don't know how to answer that question. I've never 376 00:18:45.720 --> 00:18:47.000 been a younger sibling, so I have. 377 00:18:46.960 --> 00:18:48.880 To see this is what I mean. We can't spin 378 00:18:48.920 --> 00:18:51.359 you off like we can't. We can't create like another 379 00:18:51.440 --> 00:18:53.480 experiment with you as a person where then you have 380 00:18:53.640 --> 00:18:57.119 the perception of having older siblings, but it doesn't work. 381 00:18:57.960 --> 00:18:59.839 I could spend it, Listen, I have a neuroscience to 382 00:19:00.200 --> 00:19:02.240 could spend like a lot of time mark doing about this, 383 00:19:02.359 --> 00:19:02.960 but we may. 384 00:19:04.599 --> 00:19:07.440 Want the actual devox portion of the show. 385 00:19:07.920 --> 00:19:10.279 I mean, I'm just wondering if there's, like you're secretly 386 00:19:10.319 --> 00:19:14.640 developing some sort of medical treatment here that like would 387 00:19:14.640 --> 00:19:16.960 cause you to forget your memories and still be able 388 00:19:16.960 --> 00:19:21.079 to persist in a new hypothetical reality where you could actually. 389 00:19:22.960 --> 00:19:28.000 My second time, oh that exists, Go smoke some weed, 390 00:19:34.039 --> 00:19:36.359 wake up in a room with a bunch of empty 391 00:19:36.400 --> 00:19:39.799 cereal bowls and boxes of fruity pebbles, going Wait, right. 392 00:19:43.440 --> 00:19:46.640 Is memory loss associated with potsmoking? 393 00:19:47.759 --> 00:19:49.400 I've heard that it is. I don't know that for 394 00:19:49.640 --> 00:19:50.079 a fact. 395 00:19:50.720 --> 00:19:52.920 I mean, maybe maybe we're missing out on this prime 396 00:19:53.519 --> 00:19:59.160 experimental tool that we could be using in legitimate professional. 397 00:19:58.839 --> 00:20:02.680 Applications, absolutely legitimate, I'm sure we could get lots of 398 00:20:02.720 --> 00:20:07.680 sign up for that and it'll be fine. 399 00:20:09.960 --> 00:20:11.519 Well, ethical dilemmas there. 400 00:20:11.960 --> 00:20:13.920 Well, what they did in my university is they used 401 00:20:13.920 --> 00:20:15.440