WEBVTT 1 00:00:04.639 --> 00:00:08.199 With Laurent Segoland from London and Gerard Reed from Berlin. 2 00:00:08.720 --> 00:00:10.839 This is Redefining Energy. 3 00:00:11.039 --> 00:00:15.119 Today on Redefine Energy, part our summer series. We re 4 00:00:15.320 --> 00:00:20.120 air all the episodes, but this one is fascinating because 5 00:00:20.160 --> 00:00:23.440 it's an interview by Michael Barnard on our sister show, 6 00:00:23.600 --> 00:00:28.440 Redefining Energy Tech with the great Professor Ben Fleiberg, the 7 00:00:28.559 --> 00:00:31.480 author of How Big Things Get Done. 8 00:00:32.079 --> 00:00:35.799 And Lauren was read that book last year or there before. 9 00:00:35.920 --> 00:00:42.399 Yeah, amazing, absolutely amazing. So listen to Michael Bernard interviewing 10 00:00:42.640 --> 00:00:44.119 Professor Ben Fleidberg. 11 00:00:44.840 --> 00:00:51.079 Welcome back to Redefining Energy Tech with your host Michael Barnard. Hi, 12 00:00:51.880 --> 00:00:55.399 welcome back to Redefining Energy Tech. I'm your host, Michael Barnard. 13 00:00:56.399 --> 00:00:58.600 This is a repeat of a discussion I had with 14 00:00:58.679 --> 00:01:03.079 Professor Ben Flobier, author of twenty twenty three's top selling 15 00:01:03.119 --> 00:01:07.040 business book, How Big Things Get Done. Listen in for 16 00:01:07.079 --> 00:01:13.159 their discussion of nuclear modularity, scaling and mega projects. 17 00:01:14.239 --> 00:01:16.000 Thank you very much, Thank you for having me. 18 00:01:16.760 --> 00:01:20.400 Now the excuse for having you because I gladly spend 19 00:01:20.640 --> 00:01:22.560 hours and hours and hours talking to the about a 20 00:01:22.560 --> 00:01:25.840 bunch of topics that sadly we don't have time for 21 00:01:26.519 --> 00:01:29.760 is the publication of your upcoming book with Dan Gardner, 22 00:01:29.959 --> 00:01:32.840 and so you know that book is How Big Things 23 00:01:32.879 --> 00:01:36.760 Get Done, and it explores your decades of research and 24 00:01:36.879 --> 00:01:41.640 experience in dealing with megaprojects, their failures, what works, and 25 00:01:41.680 --> 00:01:44.040 what doesn't. And today we're going to focus on some 26 00:01:44.120 --> 00:01:47.120 clean tech aspects out of that. But before we get 27 00:01:47.159 --> 00:01:50.000 into that, how did you end up being the person 28 00:01:50.359 --> 00:01:53.560 that is the go to person for governments globally when 29 00:01:53.560 --> 00:01:57.640 megaprojects need to be started effectively, when portfolios and megaprojects 30 00:01:57.680 --> 00:01:59.719 need to get moving, or when they just go off 31 00:01:59.760 --> 00:02:01.879 the literally in some cases. 32 00:02:02.640 --> 00:02:06.000 So, like you mentioned, I'm an economic geographer, and economic 33 00:02:06.040 --> 00:02:09.639 geography is a special branch of the economics that focus 34 00:02:09.680 --> 00:02:12.879 on what we call spacial economics. So you look at 35 00:02:12.919 --> 00:02:15.919 how how do how do economics play out in space? 36 00:02:15.960 --> 00:02:18.280 And that could be urban space or regional space, on 37 00:02:18.439 --> 00:02:20.680 national space, so you basically can look at what's going 38 00:02:20.719 --> 00:02:23.960 on on the surface of the planets in economic terms. 39 00:02:24.960 --> 00:02:28.000 And that's what I was raised on, you know, as 40 00:02:28.039 --> 00:02:33.400 a student, undergraduate, graduate PhD post of everything, And at 41 00:02:33.400 --> 00:02:37.039 one stage I started noticing that the citis and regions 42 00:02:37.039 --> 00:02:40.199 were being built in larger and larger chunks. So the 43 00:02:40.280 --> 00:02:43.000 projects that make up a city, to take that city 44 00:02:43.039 --> 00:02:46.280 as an example, just got larger and larger. And I 45 00:02:46.360 --> 00:02:49.560 noticed this early on and I identified it as a 46 00:02:49.599 --> 00:02:51.800 mega trend. You know, I said, okay, this is going 47 00:02:51.879 --> 00:02:54.360 to be the future. It's actually not planning that is 48 00:02:54.400 --> 00:02:57.080 so important as such, you know, which you talk about 49 00:02:57.080 --> 00:03:00.039 planning as lendings planning. It's much more what are the 50 00:03:00.080 --> 00:03:03.680 specific projects that a city decides to do and how 51 00:03:03.719 --> 00:03:06.479 they're delivered. That's what's going to be defining cities and 52 00:03:06.639 --> 00:03:11.759 actually regions and ultimately nations in the future. This is 53 00:03:11.800 --> 00:03:14.319 a long time ago. I saw this just by observing, 54 00:03:14.360 --> 00:03:16.560 you know, what was happening on the ground, including in 55 00:03:16.599 --> 00:03:19.400 my home country, which is Denmark, but also other countries. 56 00:03:20.159 --> 00:03:22.800 And I started looking around and found I couldn't find 57 00:03:22.840 --> 00:03:25.639 any data on this, even though already at that stage 58 00:03:26.120 --> 00:03:29.800 trillions and trillions of dollars were spent on these big projects, 59 00:03:30.120 --> 00:03:32.800 and nobody who was collecting data. You couldn't get data 60 00:03:32.840 --> 00:03:35.080 on them. So as a scholar, you know, this is 61 00:03:35.120 --> 00:03:37.599 an ideal of situation. That's There's that why the area 62 00:03:37.639 --> 00:03:39.960 on the map nobody has startied it before, let me 63 00:03:40.000 --> 00:03:42.199 get in there and study it. So that's how it started. 64 00:03:43.120 --> 00:03:45.759 Why was there in irritated? It strikes me as you know, 65 00:03:45.800 --> 00:03:49.319 it's an obvious insight, but why do you think that was? 66 00:03:49.319 --> 00:03:53.439 We've been building big stuff for thousands of years. Why 67 00:03:53.479 --> 00:03:58.919 haven't people been collecting even good anecdotal data. 68 00:03:58.479 --> 00:04:01.439 For For several reasons. One is that it's very hard 69 00:04:01.479 --> 00:04:04.159 to connect this data. I didn't know this at the beginning, 70 00:04:04.199 --> 00:04:07.879 but I got educated quick about how difficult it is 71 00:04:08.080 --> 00:04:10.560 to get high quality data. There's so much crap data 72 00:04:10.560 --> 00:04:12.719 out there, and in a big project, there are so 73 00:04:12.759 --> 00:04:15.199 many versions of data. So just to choose the right 74 00:04:15.280 --> 00:04:17.639 version of the data, which would be the version that 75 00:04:17.720 --> 00:04:21.000 has validity and reliability from a scholarly point of view, 76 00:04:21.480 --> 00:04:23.639 is a major job in its own right. So that's 77 00:04:23.680 --> 00:04:26.279 one thing. And that meant that they were only like, 78 00:04:26.360 --> 00:04:29.079 there might be some data on individual projects or a 79 00:04:29.120 --> 00:04:32.040 handful of projects, but there was nothing that would allow 80 00:04:32.120 --> 00:04:35.000 statistical analyscence, you know, for which you need a large example. 81 00:04:36.000 --> 00:04:39.079 And so it wasn't really on the ratar of scholars either. 82 00:04:39.160 --> 00:04:41.920 And practitioners are so in the here and now, they're 83 00:04:42.040 --> 00:04:45.079 very huge oriented. They're always thinking about the next project. 84 00:04:45.199 --> 00:04:48.360 Nobody thinks once you deliver a project. Actually, it's very 85 00:04:48.480 --> 00:04:52.560 very rare to have anybody stop and think, Okay, what 86 00:04:52.639 --> 00:04:55.040 can we learn from what we are what we just did? 87 00:04:55.120 --> 00:04:57.319 You know, why don't we connect some data on the 88 00:04:57.360 --> 00:05:00.480 outcomes and then see, you know, how the outcomes compared 89 00:05:00.519 --> 00:05:03.560 to the plans. This is almost never done. So this 90 00:05:03.879 --> 00:05:07.279 this is a combination of reasons, you know why, data 91 00:05:07.319 --> 00:05:10.839 that just were not available until we started collecting data. 92 00:05:10.879 --> 00:05:13.560 And it actually took us five years to collect the 93 00:05:13.600 --> 00:05:17.279 first data set of just two hundred and fifty eight projects, 94 00:05:17.279 --> 00:05:19.680 which was the largest data set in the world, you know, 95 00:05:20.199 --> 00:05:22.759 when we polish this in two thousand and two, you know, 96 00:05:22.839 --> 00:05:25.680 so this is just over twenty years ago now and 97 00:05:26.199 --> 00:05:28.680 today it's ludicrous to think that two hundred and fifty 98 00:05:28.720 --> 00:05:31.399 eight projects could be the largest data set the world. Now, 99 00:05:31.920 --> 00:05:33.879 just my group, you know, might see we have a 100 00:05:33.959 --> 00:05:37.439 data set that has now grown to over sixteen thousand projects, 101 00:05:37.439 --> 00:05:39.639 you know, which still is the largest database in the 102 00:05:39.800 --> 00:05:42.519 in the world. But that's how we started, and two 103 00:05:42.639 --> 00:05:45.680 hundred and fifty eight was actually enough to start doing 104 00:05:45.720 --> 00:05:49.000 statistical analysis and come up with some valid and reliable 105 00:05:49.079 --> 00:05:52.319 answers to questions like how much does it actually cost 106 00:05:52.399 --> 00:05:55.399 to deliver a big project, what's the likely cost over 107 00:05:55.519 --> 00:05:59.240 on the big project, how long does it take, what's 108 00:05:59.279 --> 00:06:02.240 the likely schedule over on? What are the benefits generated? 109 00:06:02.800 --> 00:06:07.079 And is it the benefits that were plans or do 110 00:06:07.199 --> 00:06:10.360 we get more benefits or less benefits than we're playing. 111 00:06:10.399 --> 00:06:12.480 So these were the kind of questions we could start 112 00:06:12.600 --> 00:06:15.600 rigorously analyzing after we got those first two hundred and 113 00:06:15.639 --> 00:06:16.920 fifty eight projects. 114 00:06:17.439 --> 00:06:20.160 Let's just take an example, like we connected over nuclear energy, 115 00:06:20.160 --> 00:06:22.279 and we'll get into that in a bit, but let's 116 00:06:22.279 --> 00:06:26.279 just take the history of nuclear generation buildouts. You know, 117 00:06:26.360 --> 00:06:29.160 I've looked at that, You've looked at that. Take Ontario, 118 00:06:29.279 --> 00:06:33.319 where I've lived many times. That place they had a 119 00:06:34.120 --> 00:06:38.279 provincial strategy and the richest largest province in Canada equivalent 120 00:06:38.319 --> 00:06:43.800 to Baden. It's not Rickenstein, but that big the biggest 121 00:06:43.800 --> 00:06:48.800 province in Germany. And they took twenty billion dollars off 122 00:06:48.839 --> 00:06:51.839 the books and shoved into general accounting at one point, 123 00:06:52.360 --> 00:06:54.959 and you know, fifteen years ago or so, they pushed 124 00:06:55.000 --> 00:06:59.560 it back into the utilities books. But it's now visible 125 00:06:59.560 --> 00:07:01.519 and they're trying to pay it down. But I look 126 00:07:01.560 --> 00:07:06.360 at France, and you know, if we take Macron, when 127 00:07:06.399 --> 00:07:08.600 he was in charge of the organization that should know this, 128 00:07:09.279 --> 00:07:11.480 has admitted that he could never figure out how much 129 00:07:11.480 --> 00:07:16.680 they actually spend because as national strategies or major political strategies, 130 00:07:17.120 --> 00:07:20.240 money kind of flows in in odd ways in different 131 00:07:20.240 --> 00:07:23.360 accounting systems. So how do you reconcile some of those 132 00:07:23.480 --> 00:07:28.199 challenges in a way that was academically sufficiently of quality. 133 00:07:29.000 --> 00:07:31.759 So we just look at that, and you're right, there 134 00:07:31.759 --> 00:07:35.360 are myriad ways that you can hide costs, take them 135 00:07:35.360 --> 00:07:37.600 off the books, or you can include them in the 136 00:07:37.600 --> 00:07:42.319 books obbitrarity, and that's a huge problem. So basically, we 137 00:07:42.399 --> 00:07:45.120 started this in detail for each and every project and 138 00:07:45.160 --> 00:07:47.480 see if we can make sense of me. Sometimes we can't, 139 00:07:47.560 --> 00:07:51.680 then we don't include the project. We have clear criteria 140 00:07:51.759 --> 00:07:55.040 that that data have to be valid and reliable, So 141 00:07:55.079 --> 00:07:58.040 if there's something dodgy that we can't explain, we will 142 00:07:58.079 --> 00:08:01.720 simply leave out that project because we're very, very teenly 143 00:08:01.720 --> 00:08:04.680 aware that we don't want to produde our database. Which 144 00:08:04.720 --> 00:08:08.000 is the problem with most data that consultants are using 145 00:08:08.160 --> 00:08:10.199 is that it's garbage in, garbage out that they are 146 00:08:10.240 --> 00:08:12.879 so they're so hungry for just having data that they 147 00:08:12.879 --> 00:08:15.839 will accept data even if the quality of the data 148 00:08:15.959 --> 00:08:19.839 might be a dubious quality of dubious status, and they 149 00:08:19.839 --> 00:08:23.160 include it anyway, And in that way you get a 150 00:08:23.240 --> 00:08:26.199 database that is not you know, really worth it. And 151 00:08:26.680 --> 00:08:28.959 I decided from the very start that we were not 152 00:08:29.000 --> 00:08:30.920 going to do that because I felt, you know, we're 153 00:08:30.920 --> 00:08:33.480 going to be spending a lot of time on this 154 00:08:33.759 --> 00:08:36.799 and that would be completely meaningless if we didn't know 155 00:08:37.000 --> 00:08:39.720 the validity and the liability about data, So we would 156 00:08:39.720 --> 00:08:44.519 not include anything that had unacceptable validity and reliability. That 157 00:08:44.559 --> 00:08:47.240 doesn't mean that you know, the data are perfect. There 158 00:08:47.360 --> 00:08:49.960 there's no such thing as perfect data. On big projects 159 00:08:50.000 --> 00:08:52.799 like like we're talking about here, there would always be 160 00:08:53.600 --> 00:08:56.759 things that you know could be better and that are 161 00:08:56.799 --> 00:09:00.039 not entirely clear. But still you know the day it 162 00:09:00.159 --> 00:09:03.039 would have to be above as thrashold where you could say, hey, 163 00:09:03.120 --> 00:09:08.159 this this data actually contains a really worthwhile information. It 164 00:09:08.840 --> 00:09:11.919 tells you know, the truth about what the costs were here, 165 00:09:12.000 --> 00:09:14.120 and it couldn't if it couldn't claim that, it wouldn't 166 00:09:14.159 --> 00:09:18.279 include it all the schedule, all the bit of its. 167 00:09:18.360 --> 00:09:20.600 Yeah, and it's interesting what you get up with because, 168 00:09:20.720 --> 00:09:22.879 as you say, sixteen thousand data points and now you 169 00:09:22.919 --> 00:09:25.639 can do statistical analysis. One of the things I thought 170 00:09:25.639 --> 00:09:27.840 was most fascinating in the book was, you know, you 171 00:09:27.919 --> 00:09:30.600 not only looked at costs and bud and schedule overruns, 172 00:09:30.799 --> 00:09:33.000 but you also looked at benefit of cruels and you 173 00:09:33.080 --> 00:09:37.960 actually did a fundamental piece of analysis to disprove a 174 00:09:38.639 --> 00:09:42.440 theory about unlocking creativity and doing stuff, even though it 175 00:09:42.480 --> 00:09:43.840 appears to be a bad idea. 176 00:09:44.720 --> 00:09:45.679 That insight was. 177 00:09:45.919 --> 00:09:49.240 I mean, zero point five percent of projects, if memory serves, 178 00:09:49.919 --> 00:09:52.320 actually come in on time, on a budget or close 179 00:09:52.399 --> 00:09:55.679 to it, and actually deliver benefits point five percent. Can 180 00:09:55.679 --> 00:09:58.559 you unpack that a bit and unpack that argument that 181 00:09:58.639 --> 00:09:59.440 you countered. 182 00:10:00.440 --> 00:10:04.600 Yeah, So the point five percent we call the iron 183 00:10:04.679 --> 00:10:09.039 law projects, and the iron law goes like this, over budgets, 184 00:10:09.559 --> 00:10:13.639 over time, on the benefits, over and over again. That's 185 00:10:13.679 --> 00:10:18.039 the Iron law. And about forty eight percent of projects 186 00:10:18.240 --> 00:10:23.039 projects come in on budget are better. About eight percent 187 00:10:23.120 --> 00:10:26.919 of projects come in on budgets and on time or better. 188 00:10:27.960 --> 00:10:31.399 And then you know the famous share point five percent. 189 00:10:31.600 --> 00:10:35.120 Those are the projects that come in on budget, on 190 00:10:35.320 --> 00:10:39.440 time and deliver the promised benefits or better. So that's 191 00:10:39.480 --> 00:10:41.919 how little it is. Half a percent. There's a there's 192 00:10:41.960 --> 00:10:44.639 a diagram in the book. And the interesting thing is 193 00:10:44.639 --> 00:10:46.519 that that half percent, you know, when we had to 194 00:10:46.559 --> 00:10:49.480 show that in the diagram, it's invisible because you need 195 00:10:49.559 --> 00:10:51.480 to show the one hundred percent, you need to show 196 00:10:51.519 --> 00:10:54.919 the forty eight percent, you need to show the eight percent, 197 00:10:55.080 --> 00:10:58.360 and then the half percent just becomes invisible. And that's 198 00:10:58.399 --> 00:11:01.080 pretty much a good metaphor, you know, for how things 199 00:11:01.120 --> 00:11:04.759 are in real life, that the products that tick all 200 00:11:04.759 --> 00:11:09.120 the boxes regarding success, which is on budget, on time, 201 00:11:09.399 --> 00:11:13.600 and on benefits, are better. I'visible. That's how few they are. 202 00:11:14.159 --> 00:11:16.039 I want to emphasize that it doesn't mean that they 203 00:11:16.080 --> 00:11:18.879 don't exist, especially when you started thousands of projects. Even 204 00:11:18.919 --> 00:11:21.399 half a percent will give you like a couple of handfuls. 205 00:11:21.840 --> 00:11:24.240 And that's what's really interesting to me is how on 206 00:11:24.320 --> 00:11:27.200 earth to the people who are doing these projects, how 207 00:11:27.240 --> 00:11:29.279 do they beat the odds? You know, because the share 208 00:11:29.320 --> 00:11:32.039 point five percent, that's your base rate, that's your objective 209 00:11:32.039 --> 00:11:34.039 base rate. When you look at these products like going 210 00:11:34.080 --> 00:11:36.480 to the casino, if you play the if you play 211 00:11:36.519 --> 00:11:39.440 the game of building a big project, your odds are 212 00:11:39.879 --> 00:11:42.480 shirru point five percent that you will win it. By winning, 213 00:11:42.559 --> 00:11:45.399 you define it as you know, being on budget, on time, 214 00:11:45.480 --> 00:11:49.240 and on benefits. So those are pretty steep odds, you know, 215 00:11:49.399 --> 00:11:51.519 and if you are going to go in and claim 216 00:11:51.600 --> 00:11:54.440 I can beat those odds, you better have something and 217 00:11:54.480 --> 00:11:56.240 I want to know what that's something is. And we 218 00:11:56.279 --> 00:11:58.480 spend a lot of time that in our research, and 219 00:11:58.759 --> 00:12:00.879 when we wrote the book, you know, picked the brains 220 00:12:01.240 --> 00:12:05.120 and started the data of people who actually beat the odds. 221 00:12:05.879 --> 00:12:07.519 Just as you would if you were in a casino, 222 00:12:07.720 --> 00:12:10.279 as you were interested in winning a casino. There's somebody 223 00:12:10.360 --> 00:12:12.799 walking around the casino and they beat the roulette, they 224 00:12:12.799 --> 00:12:15.840 beat blackjack, and so on consistently. You want to know 225 00:12:15.879 --> 00:12:19.279 what's going on there. And believe me, the casinos have 226 00:12:19.399 --> 00:12:22.960 detectives that are on this immediately and cameras that are 227 00:12:23.000 --> 00:12:25.919 on people. They'll be on this immediately and study what's 228 00:12:25.960 --> 00:12:27.919 going on here in a can see in a casino, 229 00:12:28.200 --> 00:12:31.360 it's usually fraud, you know, somebody's gaming the system of 230 00:12:31.480 --> 00:12:34.200 counting cards or whatever, and they will be expelled from 231 00:12:34.200 --> 00:12:37.440 the casino very quickly. So we were like those detectives 232 00:12:37.480 --> 00:12:39.840 and those cameras, we were watching all the people doing 233 00:12:39.879 --> 00:12:42.159 projects and finding who are the ones that actually are 234 00:12:42.159 --> 00:12:45.320 winning here, And we didn't kick them out. We actually 235 00:12:45.559 --> 00:12:49.279 sought them out and said, can we please talk, can 236 00:12:49.320 --> 00:12:51.519 we please look at your data, can we please look 237 00:12:51.559 --> 00:12:55.679 at your documentation, and so on, and then we figured 238 00:12:55.679 --> 00:12:57.519 out what is it that they're doing that they are 239 00:12:57.600 --> 00:12:58.320 successful at this. 240 00:12:59.039 --> 00:13:03.159 It's an interesting question speak institutional investors. Occasionally they reach 241 00:13:03.200 --> 00:13:06.559 out for, like you know, somewhat often contrarian perspectives on 242 00:13:07.000 --> 00:13:10.440 the decarbonization future. But one of the things that occurs 243 00:13:10.440 --> 00:13:13.000 to me is you've got a set of seventeen or 244 00:13:13.039 --> 00:13:17.240 twenty five categories of projects, and only five of those 245 00:13:18.200 --> 00:13:21.080 are in the sweet spot of usually delivering on time, 246 00:13:21.120 --> 00:13:24.799 on budget, and on you know, in delivering benefits. And 247 00:13:24.879 --> 00:13:27.759 you know, that's an easy list. I mean, transmission is 248 00:13:27.799 --> 00:13:30.159 in there, for example, and but you know then we 249 00:13:30.279 --> 00:13:32.799 kind of going together stuff Nuclear is way down the 250 00:13:32.840 --> 00:13:35.480 bottom of the list. You know, what are some interesting 251 00:13:35.519 --> 00:13:38.519 examples of things that you'll usually go wrong or usually 252 00:13:38.519 --> 00:13:39.879 go right in your perspective. 253 00:13:40.679 --> 00:13:43.320 This is for me, this is the most interesting result 254 00:13:43.360 --> 00:13:44.879 in the book, you know, And it's the first time 255 00:13:44.879 --> 00:13:48.039 we publish these, and we look at twenty five different projects. 256 00:13:48.039 --> 00:13:50.639 We present the data for those twenty five different projects, 257 00:13:51.000 --> 00:13:53.200 and we find, like you say, four or five projects 258 00:13:53.200 --> 00:13:56.840 type do very well and all the risks are doing 259 00:13:57.039 --> 00:13:59.600 rather poor to different degrees to be shure and be 260 00:13:59.679 --> 00:14:04.120 show different degrees. And so some of the successes are 261 00:14:04.159 --> 00:14:07.720 as you mentioned, transmission, and the most successful is actually 262 00:14:07.720 --> 00:14:10.919 solar power, you know, solar energy, that's the most successful. 263 00:14:11.600 --> 00:14:14.759 After that wind wind energy. So now we have solar 264 00:14:14.799 --> 00:14:17.919 wind transmissions. Wow, that's interesting because that's exactly what we 265 00:14:17.960 --> 00:14:20.600 need in order to solve the climate crisis. Right, So 266 00:14:20.879 --> 00:14:24.600 I just I went through but I saw this, I said, WHOA, 267 00:14:25.480 --> 00:14:28.360 what a result? How lucky? Can I mean? It might 268 00:14:28.360 --> 00:14:30.519 have been the opposite, you know, and at the opposite 269 00:14:30.679 --> 00:14:32.919 and which you also ask for what are the examples 270 00:14:32.960 --> 00:14:37.320 are really underperforming projects? You know, one of the worst 271 00:14:37.399 --> 00:14:44.159 is nuclear power, only only you know, surpassed in worseness 272 00:14:44.279 --> 00:14:48.559 by nuclear storage, you know, so nuclear is way at 273 00:14:48.559 --> 00:14:51.279 the other end. So basically in nuclear and solar at 274 00:14:51.320 --> 00:14:53.600 opposite ends of the scale. Yeah, but lots of other 275 00:14:53.639 --> 00:14:55.879 stuff is at the bad end of the scales. Who 276 00:14:56.360 --> 00:14:58.919 like the Olympic Games is one project type we study 277 00:14:59.080 --> 00:15:03.440 that's that's has terrible performance consistently it's not so hard 278 00:15:03.440 --> 00:15:05.159 to understand. I mean you always do it. In a 279 00:15:05.159 --> 00:15:09.320 new country. We talk about something called the eternal beginner's syndrome. 280 00:15:09.480 --> 00:15:12.399 You know that that it's always beginners who are doing this, 281 00:15:12.480 --> 00:15:16.159 so you don't get accumulated experience for the Olympic Games. 282 00:15:16.840 --> 00:15:19.679 Hydroelectric power same things. So big dams, whether they are 283 00:15:19.720 --> 00:15:23.039 hydroelectric or not. You know, big dams are really difficult 284 00:15:23.080 --> 00:15:28.080 to build and they also have bad performance airports, high 285 00:15:28.120 --> 00:15:31.000 speed rail, lots of other project types. So there it's 286 00:15:31.039 --> 00:15:33.279 a bad and at the other end we have these 287 00:15:33.399 --> 00:15:38.000 good performing, well performing types. And the secret to the 288 00:15:38.039 --> 00:15:42.679 difference is we also uncover that is actually fat jales, 289 00:15:44.000 --> 00:15:47.799 so thin tailed projects. Projects that have thin tail distributions 290 00:15:47.840 --> 00:15:51.519 for their performance are doing well. So they have more 291 00:15:51.559 --> 00:15:54.480 or less like a normal distribution of course and coursed 292 00:15:54.559 --> 00:15:59.240 overrun and scheduled, scheduled oberon and benefits, and that means 293 00:15:59.240 --> 00:16:02.039 that they're managing. So they are normal in that sings. 294 00:16:02.080 --> 00:16:05.840 They it is statistically right. The other projects that are 295 00:16:05.840 --> 00:16:09.279 going bad have fat chains. So what Ne's in talent 296 00:16:09.360 --> 00:16:13.279 calls next ones, and they are some by some people, 297 00:16:13.279 --> 00:16:16.759 they are considered outliers. We don't consider them outliers because 298 00:16:16.759 --> 00:16:19.879 they're actually part of the distribution. It's not like some 299 00:16:19.879 --> 00:16:23.639 some error, statistical error. It's actually real performance, real projects 300 00:16:23.679 --> 00:16:26.919 who performed like that, So they are part of the distribution, 301 00:16:27.120 --> 00:16:30.720 and they are extreme like you could have on IT projects, 302 00:16:30.720 --> 00:16:34.360 which is the worst project type. Regarding outliers. You know, 303 00:16:34.840 --> 00:16:36.960 if you look at the fat tail, the average in 304 00:16:37.000 --> 00:16:39.440 the fat tail is more than four hundred percent cost overron. 305 00:16:39.480 --> 00:16:41.600 That means that there are products with much higher cost 306 00:16:41.639 --> 00:16:44.120 over runs than the four hundred because that's an average, right, 307 00:16:44.720 --> 00:16:48.159 So in IT to find cost over runs of four 308 00:16:48.240 --> 00:16:52.240 or five six hundred percent is not unusual, and that's 309 00:16:52.240 --> 00:16:55.039 a completely different board game should be in, you know, 310 00:16:55.600 --> 00:16:57.960 compared to the Living Solar, where you never find that 311 00:16:58.080 --> 00:16:59.919 kind of cost of run, at least not so far 312 00:17:00.039 --> 00:17:01.240 in the data that we have seen. 313 00:17:02.080 --> 00:17:05.559 Yeah, it's interesting because I prefer the Michelle Wooker's gray 314 00:17:05.640 --> 00:17:09.200 Rhino metaphor completely understand why, you know, for a more 315 00:17:09.240 --> 00:17:12.000 Western origed book, which I think your book is. You know, 316 00:17:12.519 --> 00:17:16.079 black Swan has become the dominant metaphor for risk since 317 00:17:16.119 --> 00:17:18.319 Talent published, you know, around the same time as the 318 00:17:18.440 --> 00:17:21.960 subprime mortgage crisis. My experience with the book is that 319 00:17:22.680 --> 00:17:25.079 people have used it as an excuse to say we 320 00:17:25.160 --> 00:17:29.839 couldn't have possibly known about these things, as opposed to say, 321 00:17:30.200 --> 00:17:33.359 we ignored these things that were pretty obvious if we 322 00:17:33.480 --> 00:17:36.839 thought about them, and we didn't build a resilient system 323 00:17:37.480 --> 00:17:41.039 that would accommodate fat tail risks knowing that they're present. 324 00:17:41.680 --> 00:17:45.119 And so, you know, Wooker's metaphor, you you redefine, you 325 00:17:45.200 --> 00:17:48.119 define the way people should think about black swans in 326 00:17:48.160 --> 00:17:51.319 your book, and I recommend people read it obviously, but 327 00:17:51.880 --> 00:17:53.839 I would say that you're actually redefine it much more 328 00:17:53.880 --> 00:17:57.440 like Walker's gray rhino metaphor, which oddly it's much more 329 00:17:57.519 --> 00:18:00.279 dominant in Asia than it is in the way She's 330 00:18:00.599 --> 00:18:03.359 She's had massively more penetration there. It's been discussed at 331 00:18:03.359 --> 00:18:06.960 the highest levels of the Chinese pollup Borrow, for example, 332 00:18:07.000 --> 00:18:09.519 in terms of guiding their strategies. You can kind of 333 00:18:09.519 --> 00:18:12.799 see some of those long termism behaviors and the way 334 00:18:12.799 --> 00:18:15.279 they're structuring out stuff versus you know, some of the 335 00:18:15.279 --> 00:18:20.319 short termism that often democracies indulging around electoral cycles. We're 336 00:18:20.359 --> 00:18:24.000 seeing those kind of implications. So have you actually read 337 00:18:24.240 --> 00:18:27.160 a worker's book and have many examples or contrasts there 338 00:18:27.200 --> 00:18:29.119 that you you know, to to. 339 00:18:28.920 --> 00:18:32.039 Pull that apart. So the way I understand it is 340 00:18:32.079 --> 00:18:35.880 that the gray rhinos are less extreme than the lex ones, 341 00:18:36.000 --> 00:18:38.640 so that they are they are extreme, but they are 342 00:18:38.680 --> 00:18:42.359 not as extreme. Actually, our definition, we don't have to 343 00:18:42.440 --> 00:18:45.519 call it lex ones or gray rhinos or anything. Our 344 00:18:45.519 --> 00:18:48.519 definition is actually statistically. You know, statisticians have a very 345 00:18:48.559 --> 00:18:51.400 clear definition of what extreme values are. So there's something 346 00:18:51.440 --> 00:18:56.400 called extreme valued theory and statistics. And you define an outlier. 347 00:18:56.400 --> 00:18:58.599 So if it's tossed over run, for instance, would you 348 00:18:58.640 --> 00:19:02.160 would say it's a The technical definition is that it's 349 00:19:02.400 --> 00:19:07.680 one point five interquartile ranges about the third quartile. And 350 00:19:07.720 --> 00:19:10.359 that's you know, that's objective. That's just the ways that 351 00:19:10.359 --> 00:19:12.240 it is. This and they don't care what you call it, 352 00:19:12.559 --> 00:19:14.799 whether you call it gray or black or whatever it is. 353 00:19:14.960 --> 00:19:17.759 That's what's called an extreme value, an extreme value theory. 354 00:19:18.279 --> 00:19:21.799 As I understand, gray rhinos they come in, they're not 355 00:19:21.880 --> 00:19:24.960 necessarily that extreme, but they're still extreme enough to create 356 00:19:25.119 --> 00:19:27.920 lots of problems, you know, in practical policy and planning. 357 00:19:28.079 --> 00:19:29.480 And I would say the way we do it, we 358 00:19:29.519 --> 00:19:32.960 consider the whole range you're familiar with reference stars forecasting. 359 00:19:33.000 --> 00:19:35.839 I've seen in what you're writing, and if you read 360 00:19:35.880 --> 00:19:37.880 about the way we do that, you will know that 361 00:19:37.960 --> 00:19:41.200 we look at the total distribution, the total probability distribution, 362 00:19:41.319 --> 00:19:44.599 or whatever variable it is that we are forecasting. And 363 00:19:44.759 --> 00:19:47.440 I find that gives me really peace of mind because 364 00:19:47.480 --> 00:19:49.279 I know there's nothing I don't look at, you know, 365 00:19:49.319 --> 00:19:51.359 So whether it's gray or black or what it is, 366 00:19:51.519 --> 00:19:55.599 it's being considered. And we do pay special attention to 367 00:19:56.480 --> 00:19:59.079 the most extreme values because we know that these are 368 00:19:59.079 --> 00:20:01.319 the most damaging. If you end up in the tail, 369 00:20:01.480 --> 00:20:03.640 you are in trouble, and you are not only in 370 00:20:03.680 --> 00:20:06.759 a little trouble, you're in big trouble. And it's very 371 00:20:06.799 --> 00:20:10.640 hard to, you know, protect yourself against through contingencies because 372 00:20:10.680 --> 00:20:13.000 the contingencies just have to be too high. You will 373 00:20:13.039 --> 00:20:15.519 never get those kinds of contingencies. It's not even rational 374 00:20:15.559 --> 00:20:19.279 within an organization to allocate that much contingency to anyone 375 00:20:19.319 --> 00:20:21.799 project because it means you can't use the resources for 376 00:20:21.839 --> 00:20:25.200 other projects. Right, So what we recommend is that when 377 00:20:25.799 --> 00:20:29.240 that like eighty percent of the probability distribution, you can 378 00:20:29.240 --> 00:20:32.920 protect yourself with contingencies, and we calculate the exact size 379 00:20:32.920 --> 00:20:36.160 of those contingencies, and then we say the remaining part 380 00:20:36.200 --> 00:20:38.680 of the probability distribution, you really go you got to 381 00:20:38.759 --> 00:20:41.440 go in and mitigate so that doesn't happen. We call 382 00:20:41.480 --> 00:20:45.640 that black sworn management. And then then there's slogan of 383 00:20:45.960 --> 00:20:49.519 blacksworan management is cut the tail. You need to cut 384 00:20:49.519 --> 00:20:51.759 the tail. And how do you cut the tail? You 385 00:20:51.799 --> 00:20:54.559 cut the tail through mitigation. So let's take a really 386 00:20:54.599 --> 00:20:57.920 simple example like Fukushima, you know, and you need to 387 00:20:57.920 --> 00:21:02.440 protect the nuclear power plant against a tsunami wave. Simple 388 00:21:02.519 --> 00:21:05.079 you build a wall that's tall enough and then that's it. 389 00:21:05.200 --> 00:21:08.000 You know, that's cutting the tails. Or move it uphill 390 00:21:08.039 --> 00:21:11.200 because that it uphill, you know, that's even easier, even easy. 391 00:21:11.240 --> 00:21:14.680 They build the plant downhill from a tsunami high level 392 00:21:14.680 --> 00:21:16.160 marker from the seventeenth century. 393 00:21:16.759 --> 00:21:19.039 So we go through an exercise like this, and of 394 00:21:19.480 --> 00:21:23.039 course most of the time it's not a tsunami wall 395 00:21:23.079 --> 00:21:25.799 that is the solution. I'm just mentioning that because it's 396 00:21:25.839 --> 00:21:29.680 so easy to understand, or building in a further in 397 00:21:29.799 --> 00:21:32.319 country or up you know, at a at a higher 398 00:21:32.400 --> 00:21:36.279 level and with different projects and project types. You know, 399 00:21:36.640 --> 00:21:40.559 mitigation means different things, but you can always sit down 400 00:21:40.640 --> 00:21:42.400 and think, how do I cut the chain. It's the 401 00:21:42.519 --> 00:21:45.680