WEBVTT

1
00:00:00.880 --> 00:00:03.240
<v Speaker 1>Okay, I want you to imagine a regional power grid

2
00:00:03.240 --> 00:00:07.759
<v Speaker 1>for a second. It's, you know, autonomously adjusting its voltage

3
00:00:07.799 --> 00:00:12.240
<v Speaker 1>to match this incoming totally unprecedented winter storm.

4
00:00:12.400 --> 00:00:14.320
<v Speaker 2>Right, highly critical situation exactly.

5
00:00:14.720 --> 00:00:18.719
<v Speaker 1>So thousands of sensors are feeding data into an artificial intelligence,

6
00:00:19.079 --> 00:00:22.160
<v Speaker 1>and this AI is actively balancing the load, so hospitals

7
00:00:22.160 --> 00:00:23.920
<v Speaker 1>stay online, home stay warm.

8
00:00:24.440 --> 00:00:25.679
<v Speaker 3>But then imagine that.

9
00:00:25.760 --> 00:00:29.320
<v Speaker 1>AI hallucinates just a single data point.

10
00:00:29.800 --> 00:00:30.960
<v Speaker 2>Oh wow, yeah right.

11
00:00:31.079 --> 00:00:34.079
<v Speaker 1>It makes this probabilistic guess that is mathematically wrong, and

12
00:00:34.119 --> 00:00:38.280
<v Speaker 1>before a human operator can even blink, three substations catch fire.

13
00:00:38.719 --> 00:00:41.520
<v Speaker 1>I mean, we aren't talking about a bad chatbot generating

14
00:00:41.520 --> 00:00:44.159
<v Speaker 1>a picture of a guy with seven fingers anymore at all.

15
00:00:44.200 --> 00:00:46.719
<v Speaker 1>The era where you know an AI making a mistake

16
00:00:46.880 --> 00:00:49.039
<v Speaker 1>is just this funny screenshot you share with your friends

17
00:00:49.039 --> 00:00:50.200
<v Speaker 1>that is entirely.

18
00:00:49.880 --> 00:00:53.679
<v Speaker 2>Over it really is. The stakes have shifted completely, mostly

19
00:00:53.679 --> 00:00:56.759
<v Speaker 2>because the environment is no longer contained to a browser

20
00:00:56.759 --> 00:01:00.159
<v Speaker 2>window on your laptop. Right, We're in the middle of

21
00:01:00.200 --> 00:01:04.120
<v Speaker 2>this massive technological leap. I mean, autonomous AI agents are

22
00:01:04.159 --> 00:01:10.120
<v Speaker 2>being plugged directly into physical critical infrastructure. We're talking power grids,

23
00:01:10.239 --> 00:01:13.959
<v Speaker 2>industrial control systems, aviation, financial market.

24
00:01:14.000 --> 00:01:16.760
<v Speaker 3>It's stuff that runs the real world exactly.

25
00:01:17.319 --> 00:01:19.400
<v Speaker 2>And when you hand the keys of the physical world

26
00:01:19.480 --> 00:01:23.799
<v Speaker 2>over to a probabilistic reasoning engine, well, an unconstrained error

27
00:01:23.879 --> 00:01:28.400
<v Speaker 2>doesn't just cause software friction, it causes actual service blackouts,

28
00:01:28.719 --> 00:01:33.239
<v Speaker 2>cascading financial contagion, and, like you said, catastrophic physical damage.

29
00:01:32.920 --> 00:01:35.560
<v Speaker 1>Which means that old Silicon Valley mantra, you know, the

30
00:01:35.560 --> 00:01:38.200
<v Speaker 1>whole move fast and break things idea.

31
00:01:38.040 --> 00:01:39.079
<v Speaker 2>Yeah, that doesn't fly here.

32
00:01:39.120 --> 00:01:41.959
<v Speaker 3>It's essentially a death sentence in this context. You cannot

33
00:01:42.000 --> 00:01:44.000
<v Speaker 3>move fast and break a regional power grid.

34
00:01:44.040 --> 00:01:46.719
<v Speaker 1>And honestly, that brings us to the core mission of

35
00:01:46.760 --> 00:01:47.760
<v Speaker 1>our deep dive today.

36
00:01:47.879 --> 00:01:49.480
<v Speaker 2>It's a fascinating ship to look into.

37
00:01:49.480 --> 00:01:50.439
<v Speaker 3>It really is.

38
00:01:50.519 --> 00:01:54.760
<v Speaker 1>We are unpacking the terrifying but also thrilling reality of

39
00:01:54.760 --> 00:01:57.319
<v Speaker 1>how engineers are actually attempting to put a fence around

40
00:01:57.359 --> 00:01:59.959
<v Speaker 1>an AI. Because you can't just let a large language

41
00:02:00.000 --> 00:02:02.840
<v Speaker 1>model directly pull the levers of a drone, swarm or

42
00:02:02.879 --> 00:02:03.879
<v Speaker 1>a power plant.

43
00:02:04.120 --> 00:02:07.200
<v Speaker 2>You'd be asking for a disaster. So the industry is

44
00:02:07.239 --> 00:02:12.039
<v Speaker 2>pivoting to this very specific, highly rigid architecture designed to

45
00:02:12.120 --> 00:02:16.159
<v Speaker 2>guarantee safety. It's a complete paradigm shift away from open

46
00:02:16.280 --> 00:02:16.960
<v Speaker 2>ended execution.

47
00:02:17.159 --> 00:02:20.800
<v Speaker 1>Okay, let's unpack this because the architecture here is just wild.

48
00:02:20.879 --> 00:02:23.680
<v Speaker 3>We are trying to marry two very different concepts.

49
00:02:23.800 --> 00:02:26.919
<v Speaker 2>Right. You have a probabilistic AI that guess is based

50
00:02:26.960 --> 00:02:30.800
<v Speaker 2>on statistical weights, and a physical system that requires absolute

51
00:02:30.879 --> 00:02:32.639
<v Speaker 2>mathematical certainty exactly.

52
00:02:33.280 --> 00:02:36.879
<v Speaker 1>So, to explain how this failsafe framework operates without getting

53
00:02:36.919 --> 00:02:40.000
<v Speaker 1>super bogged down in the jargon, think about like a

54
00:02:40.000 --> 00:02:42.840
<v Speaker 1>bowling alley, but a bowling alley functioning at light speed.

55
00:02:42.919 --> 00:02:44.719
<v Speaker 2>Okay, I like this a bowling alley.

56
00:02:44.840 --> 00:02:45.280
<v Speaker 3>Yeah.

57
00:02:45.439 --> 00:02:48.879
<v Speaker 1>So in this setup, the artificial intelligence, the reasoning engine

58
00:02:48.919 --> 00:02:51.439
<v Speaker 1>is the bowling ball. It's just trying to find the

59
00:02:51.479 --> 00:02:54.520
<v Speaker 1>best most efficient path down the lane to knock over

60
00:02:54.560 --> 00:02:55.039
<v Speaker 1>the pins.

61
00:02:55.280 --> 00:02:58.080
<v Speaker 3>That's the AI trying to generate an action, right, it's.

62
00:02:57.960 --> 00:03:00.319
<v Speaker 2>Just doing what it's trained to do, finding pattern to

63
00:03:00.400 --> 00:03:00.960
<v Speaker 2>reach your goal.

64
00:03:01.280 --> 00:03:03.919
<v Speaker 3>But you don't just let the ball roll wherever it wants.

65
00:03:03.960 --> 00:03:05.520
<v Speaker 3>You have a state verifier.

66
00:03:06.080 --> 00:03:08.840
<v Speaker 1>Think of the state verifier as a sensor that checks

67
00:03:08.879 --> 00:03:10.919
<v Speaker 1>the bowler's stance before the ball.

68
00:03:10.800 --> 00:03:11.560
<v Speaker 3>Is even released.

69
00:03:11.639 --> 00:03:13.080
<v Speaker 2>Oh, that's a great way to put it.

70
00:03:13.159 --> 00:03:17.840
<v Speaker 1>Yeah, it's mathematically checking if the operational state, like say

71
00:03:17.879 --> 00:03:21.199
<v Speaker 1>the thermal threshold of a power transformer, is actually within

72
00:03:21.360 --> 00:03:24.280
<v Speaker 1>valid bounds before anything else happens.

73
00:03:23.919 --> 00:03:27.120
<v Speaker 2>Right, establishing the absolute reality of the environment at that

74
00:03:27.319 --> 00:03:30.479
<v Speaker 2>exact millisecond. And then once the ball is rolling, you

75
00:03:30.520 --> 00:03:31.879
<v Speaker 2>have the deterministic guardrail.

76
00:03:31.960 --> 00:03:34.240
<v Speaker 3>The heavy iron bumpers exactly the bumper lanes.

77
00:03:34.280 --> 00:03:38.840
<v Speaker 2>These are the physical, uncompromising boundaries. If the AI proposes

78
00:03:38.879 --> 00:03:41.919
<v Speaker 2>an action that violates safety protocols, like sending too much

79
00:03:42.000 --> 00:03:45.240
<v Speaker 2>voltage down a line, these guardrails don't ask the AI

80
00:03:45.319 --> 00:03:46.479
<v Speaker 2>to reconsider, right.

81
00:03:46.520 --> 00:03:48.840
<v Speaker 3>They don't crop the LLM and say are you sure?

82
00:03:49.080 --> 00:03:51.919
<v Speaker 2>No, not at all. They act like gravity. The guardrails

83
00:03:51.960 --> 00:03:56.000
<v Speaker 2>are hard coded, inflexible policy layers. They physically bounce the

84
00:03:56.039 --> 00:03:59.759
<v Speaker 2>decision back into a safe zone, overriding the neural network entirely.

85
00:04:00.120 --> 00:04:03.000
<v Speaker 3>The AI just cannot bypass the physics of the bumper.

86
00:04:03.280 --> 00:04:06.840
<v Speaker 2>What's fascinating here is how these systems hander a total

87
00:04:06.840 --> 00:04:10.240
<v Speaker 2>loss of control. I mean, what happens when the ball

88
00:04:10.360 --> 00:04:12.199
<v Speaker 2>keeps hitting the bumpers over and over.

89
00:04:12.240 --> 00:04:15.039
<v Speaker 1>Right, If the AI just keeps hallucinating bad ideas.

90
00:04:15.199 --> 00:04:19.360
<v Speaker 2>Yeah, if the reasoning engine keeps proposing dangerous actions, or

91
00:04:19.399 --> 00:04:22.120
<v Speaker 2>if the telemetry data shows the system's state is drifting

92
00:04:22.160 --> 00:04:25.759
<v Speaker 2>into an anomaly, the system has this built in circuit breaker.

93
00:04:26.040 --> 00:04:28.720
<v Speaker 2>It initiates a protocol called graceful degradation.

94
00:04:29.120 --> 00:04:32.199
<v Speaker 3>Graceful degradation, I love that term. So it doesn't just

95
00:04:32.240 --> 00:04:33.240
<v Speaker 3>blue screen and crash.

96
00:04:33.439 --> 00:04:36.759
<v Speaker 2>No, it automatically steps down the level of autonomy. If

97
00:04:36.800 --> 00:04:41.000
<v Speaker 2>the AI models internal confidence drops, the circuit breaker instantly

98
00:04:41.040 --> 00:04:42.720
<v Speaker 2>strips the neural network of its control.

99
00:04:42.959 --> 00:04:44.920
<v Speaker 3>It just kicks the AI out of a driver's.

100
00:04:44.639 --> 00:04:48.399
<v Speaker 2>Seat automatically, and it falls back to classical algorithmic control.

101
00:04:48.959 --> 00:04:51.560
<v Speaker 2>Or if it's really bad, it triggers an immediate human

102
00:04:51.600 --> 00:04:55.000
<v Speaker 2>in the loop escalation. It locks the system down until

103
00:04:55.040 --> 00:04:56.680
<v Speaker 2>a human operator takes the stick.

104
00:04:56.839 --> 00:04:57.240
<v Speaker 1>Wow.

105
00:04:57.560 --> 00:05:01.800
<v Speaker 2>And a crucial detail here the system immutable audit.

106
00:05:01.480 --> 00:05:04.040
<v Speaker 3>Trails, meaning everything is recorded.

107
00:05:03.839 --> 00:05:07.800
<v Speaker 2>Everything, every raw observation, the intermediate chain of thought from

108
00:05:07.800 --> 00:05:11.199
<v Speaker 2>the AI, the verifier checks, it's all stored in Tampa

109
00:05:11.240 --> 00:05:13.439
<v Speaker 2>evidence storage for regulatory auditing.

110
00:05:13.480 --> 00:05:17.560
<v Speaker 1>Okay, so having this incredibly secure bowling alley structure sounds

111
00:05:17.600 --> 00:05:21.560
<v Speaker 1>great in theory, but where exactly is this highly guarded

112
00:05:21.560 --> 00:05:25.480
<v Speaker 1>AI actually driving the ball? Which brings us to the

113
00:05:25.519 --> 00:05:28.519
<v Speaker 1>specific domains adopting.

114
00:05:27.920 --> 00:05:30.160
<v Speaker 2>This, And this is where the stakes get very real.

115
00:05:30.120 --> 00:05:32.560
<v Speaker 1>Right, and I have to play Devil's advocate for the

116
00:05:32.720 --> 00:05:36.040
<v Speaker 1>listener here. If these guardrails and circuit breakers are so

117
00:05:36.199 --> 00:05:38.839
<v Speaker 1>air tight, why are we still so nervously? What is

118
00:05:38.839 --> 00:05:42.040
<v Speaker 1>the actual worst case scenario if this control loop fails.

119
00:05:42.199 --> 00:05:44.519
<v Speaker 2>Well, let's walk through the specific domains from the research,

120
00:05:44.560 --> 00:05:47.480
<v Speaker 2>because the failure impacts are drastically different depending on where

121
00:05:47.480 --> 00:05:50.079
<v Speaker 2>the AI is deployed. Let's start with cybersecurity, like a

122
00:05:50.079 --> 00:05:54.199
<v Speaker 2>security operation center or SC Okay, the intended goal there

123
00:05:54.439 --> 00:05:58.399
<v Speaker 2>is the autonomous isolation of compromise endpoints, stopping hackers at

124
00:05:58.399 --> 00:05:59.040
<v Speaker 2>machine speed.

125
00:05:59.439 --> 00:06:03.240
<v Speaker 1>Right, if every single proposed action has to be routed

126
00:06:03.240 --> 00:06:07.439
<v Speaker 1>through a state verifier and evaluated against these heavy guardrails,

127
00:06:07.720 --> 00:06:11.240
<v Speaker 1>aren't we adding massive latency? Doesn't that slow the AI

128
00:06:11.360 --> 00:06:13.279
<v Speaker 1>down too much to fight a cyber attack?

129
00:06:13.439 --> 00:06:16.879
<v Speaker 2>That's a common concern, But actually the latency of a

130
00:06:16.920 --> 00:06:21.000
<v Speaker 2>deterministic check is magnetudes lower than the neural reasoning itself.

131
00:06:21.079 --> 00:06:24.120
<v Speaker 2>Oh really yeah, evaluating a hard coded math limit like

132
00:06:24.519 --> 00:06:27.399
<v Speaker 2>if x is greater than y block the action that

133
00:06:27.439 --> 00:06:30.720
<v Speaker 2>takes microseconds. The neural network is what takes time. So

134
00:06:30.759 --> 00:06:33.079
<v Speaker 2>the guardrails are just a lightweight, high speed filter.

135
00:06:33.279 --> 00:06:35.639
<v Speaker 3>Okay, makes sense. So what's the worst case scenario there?

136
00:06:36.040 --> 00:06:38.480
<v Speaker 2>Well, if the loop fails, or if you don't have

137
00:06:38.560 --> 00:06:42.040
<v Speaker 2>good guardrails, the AI might get over zealous. It sees

138
00:06:42.079 --> 00:06:44.879
<v Speaker 2>a false positive threat and decides the most efficient way

139
00:06:44.920 --> 00:06:47.079
<v Speaker 2>to protect the network is to accidentally shut down your

140
00:06:47.079 --> 00:06:48.240
<v Speaker 2>core enterprise.

141
00:06:47.879 --> 00:06:51.399
<v Speaker 1>Services, so it paralyzes your own infrastructure to save it exactly,

142
00:06:51.480 --> 00:06:54.639
<v Speaker 1>But that scales up terrifyingly when we move to energy.

143
00:06:54.319 --> 00:06:56.920
<v Speaker 3>Empower grids like the winter storm scenario from earlier.

144
00:06:57.160 --> 00:07:01.079
<v Speaker 2>Right there, the AI handles real time parameter tuning and

145
00:07:01.120 --> 00:07:04.439
<v Speaker 2>load balancings. A failure doesn't just mean the lights flicker.

146
00:07:05.000 --> 00:07:07.639
<v Speaker 2>It means you alter the phase synchronization of the grid,

147
00:07:07.959 --> 00:07:13.000
<v Speaker 2>which means physical transformer overloads, the copper inside literally melts,

148
00:07:13.680 --> 00:07:16.120
<v Speaker 2>and when one transformer trips, the power surges to the

149
00:07:16.160 --> 00:07:19.839
<v Speaker 2>next one overloading now and too, a domino effect, exactly,

150
00:07:20.319 --> 00:07:24.560
<v Speaker 2>a cascading frequency failure resulting in a regional blackout, all

151
00:07:24.560 --> 00:07:27.240
<v Speaker 2>because a neural network made a bad statistical guess.

152
00:07:28.000 --> 00:07:30.319
<v Speaker 1>And if an AI can physically melt a grid, it

153
00:07:30.319 --> 00:07:32.639
<v Speaker 1>can definitely crash the economy.

154
00:07:32.399 --> 00:07:35.040
<v Speaker 2>Which brings up financial markets. I mean, we've had algorithmic

155
00:07:35.079 --> 00:07:36.040
<v Speaker 2>trading for a long time run.

156
00:07:36.160 --> 00:07:40.040
<v Speaker 3>Yeah, but those traditional algos are deterministic. They follow strict

157
00:07:40.079 --> 00:07:40.639
<v Speaker 3>if then.

158
00:07:40.600 --> 00:07:43.319
<v Speaker 4>Rules they only do what they're explicitly told, right, But

159
00:07:43.480 --> 00:07:46.720
<v Speaker 4>these new autonomous agents are managing risk based on open

160
00:07:46.839 --> 00:07:51.240
<v Speaker 4>ended reasoning, sentimentoprosis, real time geopolitical data.

161
00:07:51.360 --> 00:07:53.959
<v Speaker 3>If the loop fails here, a flash crash isn't just

162
00:07:54.000 --> 00:07:54.399
<v Speaker 3>a dip.

163
00:07:54.800 --> 00:07:58.920
<v Speaker 1>It could trigger systemic market contagion across multiple institutions before

164
00:07:58.959 --> 00:08:00.319
<v Speaker 1>a human even realize.

165
00:08:00.000 --> 00:08:00.720
<v Speaker 3>This is what happened.

166
00:08:00.759 --> 00:08:04.240
<v Speaker 2>Okay, that's terrifying. But then we have aerospace and defense. Yes,

167
00:08:04.360 --> 00:08:06.879
<v Speaker 2>and this is where the system faces the most adversarial

168
00:08:06.959 --> 00:08:11.759
<v Speaker 2>environments possible. They focus on swarm coordination and jam resistant

169
00:08:11.879 --> 00:08:12.639
<v Speaker 2>edge execution.

170
00:08:12.800 --> 00:08:17.079
<v Speaker 1>Jam resistant meaning like a fleet of drones operating where

171
00:08:17.160 --> 00:08:19.879
<v Speaker 1>their signal to home base is completely cut off.

172
00:08:20.120 --> 00:08:22.959
<v Speaker 2>Yes, they have to make tactical decisions on their own.

173
00:08:23.240 --> 00:08:26.759
<v Speaker 2>And if a state verifier misinterprets a sensor reading there,

174
00:08:27.519 --> 00:08:31.879
<v Speaker 2>the failure impact isn't just losing a vehicle, it's unintended escalation.

175
00:08:31.759 --> 00:08:34.240
<v Speaker 3>And AI hallucinating a hostile target.

176
00:08:34.120 --> 00:08:38.559
<v Speaker 2>Exactly, engaging based on a probabilistic error in a contested area.

177
00:08:38.679 --> 00:08:43.399
<v Speaker 1>Okay, hearing about unintended escalation and transformer overloads it naturally

178
00:08:43.440 --> 00:08:44.480
<v Speaker 1>prompts this realization.

179
00:08:44.600 --> 00:08:48.200
<v Speaker 3>Standard software engineering rules just aren't enough to prevent these disasters.

180
00:08:48.279 --> 00:08:51.600
<v Speaker 1>Right, If the stakes are this apocalyptic, the traditional ways

181
00:08:51.600 --> 00:08:54.399
<v Speaker 1>we test software must be fundamentally broken when it comes

182
00:08:54.399 --> 00:08:54.759
<v Speaker 1>to AI.

183
00:08:54.879 --> 00:08:57.000
<v Speaker 2>They absolutely are. And if we connect this to the

184
00:08:57.000 --> 00:08:59.759
<v Speaker 2>bigger picture, this is the core of the technical dilemma

185
00:08:59.759 --> 00:09:01.240
<v Speaker 2>with certification, right.

186
00:09:01.360 --> 00:09:04.480
<v Speaker 1>This is what the source is called the nondeterminism dilemma.

187
00:09:04.679 --> 00:09:06.200
<v Speaker 3>Let's break this down for the listener.

188
00:09:06.960 --> 00:09:11.200
<v Speaker 1>In traditional safety engineering, like aviation software, everything is deterministic

189
00:09:11.919 --> 00:09:13.440
<v Speaker 1>input X get y.

190
00:09:13.600 --> 00:09:16.559
<v Speaker 2>Yes, the DO one seventy eight C standard in aviation,

191
00:09:16.639 --> 00:09:19.440
<v Speaker 2>for example, Yeah, you can test it exhaustively until it's

192
00:09:19.440 --> 00:09:21.399
<v Speaker 2>mathematically proven to never deviate.

193
00:09:21.799 --> 00:09:25.919
<v Speaker 3>But neural networks are probabilistic. So how on.

194
00:09:25.799 --> 00:09:30.120
<v Speaker 1>Earth do you formally certify a system when identical environmental

195
00:09:30.200 --> 00:09:32.840
<v Speaker 1>inputs might yield a completely different action.

196
00:09:32.799 --> 00:09:34.960
<v Speaker 3>On a Tuesday than they did on a Monday.

197
00:09:35.200 --> 00:09:37.919
<v Speaker 2>You've hit on the biggest open research challenge today. You

198
00:09:37.919 --> 00:09:41.279
<v Speaker 2>can't just trace a billion parameter neural network back to

199
00:09:41.320 --> 00:09:43.960
<v Speaker 2>one line of code. It's impossible, and there were two

200
00:09:44.039 --> 00:09:46.799
<v Speaker 2>massive hurdles here. First is distribution shift.

201
00:09:46.759 --> 00:09:49.720
<v Speaker 1>And edge cases, which is when the real world throws

202
00:09:49.720 --> 00:09:52.039
<v Speaker 1>something at the AI that wasn't in its training data.

203
00:09:52.200 --> 00:09:55.679
<v Speaker 2>Exactly what happens when the AI faces an unprecedented extreme

204
00:09:55.720 --> 00:09:58.279
<v Speaker 2>weather event on that power grid. It might not just crash,

205
00:09:58.600 --> 00:10:02.639
<v Speaker 2>It might produce an uncalibrat a high confidence hallucination constantly long,

206
00:10:03.000 --> 00:10:05.840
<v Speaker 2>dangerously long, but it thinks it's ninety nine percent correct.

207
00:10:05.919 --> 00:10:06.759
<v Speaker 3>And the second.

208
00:10:06.559 --> 00:10:09.320
<v Speaker 2>Hurdle, This is the part that is just fascinating from

209
00:10:09.360 --> 00:10:12.440
<v Speaker 2>a system's perspective. Emergence swarm dynamics.

210
00:10:12.519 --> 00:10:14.559
<v Speaker 3>Oh, this is the multiple agents thing, right.

211
00:10:14.600 --> 00:10:17.960
<v Speaker 2>When you have multiple autonomous agents interacting simultaneously, like in

212
00:10:17.960 --> 00:10:22.440
<v Speaker 2>a multi agent's cyber defense, they create these unpredictable system

213
00:10:22.519 --> 00:10:28.000
<v Speaker 2>level feedback loops. Agent A reacts to agent b's probabilistic guess,

214
00:10:28.360 --> 00:10:30.320
<v Speaker 2>which changes agency's state.

215
00:10:30.639 --> 00:10:33.320
<v Speaker 3>It's just a compounding chaotic mess.

216
00:10:33.360 --> 00:10:37.840
<v Speaker 2>It creates a state space explosion. Proving mathematically that a

217
00:10:37.919 --> 00:10:43.000
<v Speaker 2>dynamic swarm won't trigger unpredictable emergence is borderline impossible right now.

218
00:10:42.960 --> 00:10:45.600
<v Speaker 1>Which forces us to look at the physical reality of

219
00:10:45.600 --> 00:10:48.120
<v Speaker 1>the hardware too, because to ground this back in reality,

220
00:10:48.600 --> 00:10:51.080
<v Speaker 1>these systems have massive physical limits.

221
00:10:51.080 --> 00:10:54.039
<v Speaker 3>We call them swap constraints right, size, weight, and.

222
00:10:54.000 --> 00:10:56.120
<v Speaker 2>Power absolutely crucial point because.

223
00:10:55.919 --> 00:10:58.600
<v Speaker 1>An air gap defense system or a drone sworm doesn't

224
00:10:58.639 --> 00:11:01.840
<v Speaker 1>have the luxury of pinging a massive power hungry server.

225
00:11:01.759 --> 00:11:04.360
<v Speaker 3>Farm in the cloud. They have strict power budgets.

226
00:11:04.399 --> 00:11:06.759
<v Speaker 2>You can't put a liquid cooled server rack inside a.

227
00:11:06.720 --> 00:11:09.759
<v Speaker 1>Drone exactly, so they have to use highly optimized small

228
00:11:09.840 --> 00:11:12.720
<v Speaker 1>language models or SLMs, running locally.

229
00:11:12.360 --> 00:11:13.039
<v Speaker 3>On the edge.

230
00:11:13.200 --> 00:11:16.080
<v Speaker 2>And when you shrink a model to fit on edge hardware,

231
00:11:16.399 --> 00:11:20.399
<v Speaker 2>you inherently reduce its reasoning capacity. It's faster, but it's

232
00:11:20.440 --> 00:11:24.759
<v Speaker 2>statistically more prone to hallucinate or miss a complex pattern.

233
00:11:24.639 --> 00:11:28.279
<v Speaker 1>Which makes those heavy iron bumper lanes. The deterministic guardrail

234
00:11:28.360 --> 00:11:29.960
<v Speaker 1>is even more desperately needed.

235
00:11:30.159 --> 00:11:35.080
<v Speaker 2>Precisely, we have probabilistic models constrained by power, prone to

236
00:11:35.200 --> 00:11:39.360
<v Speaker 2>unpredictable swarm behavior, and yet we desperately need them to

237
00:11:39.399 --> 00:11:42.919
<v Speaker 2>manage our physical infrastructure. A human just can't process a

238
00:11:42.960 --> 00:11:44.639
<v Speaker 2>million grid sensors fast enough.

239
00:11:44.799 --> 00:11:48.919
<v Speaker 1>So this intense tension needing the AI but fearing the AI,

240
00:11:49.240 --> 00:11:53.159
<v Speaker 1>forces the industry toward a brand new hybrid solution.

241
00:11:53.039 --> 00:11:55.440
<v Speaker 2>And that ultimate goal is what the industry is calling

242
00:11:55.480 --> 00:11:56.440
<v Speaker 2>certified autonomy.

243
00:11:56.559 --> 00:11:59.600
<v Speaker 1>Right, it's not about letting machines run entirely independent and

244
00:11:59.639 --> 00:12:00.000
<v Speaker 1>on mode.

245
00:12:00.639 --> 00:12:04.240
<v Speaker 2>No, it's about marrying the brilliant open ended adaptability of

246
00:12:04.279 --> 00:12:07.840
<v Speaker 2>deep learning with a guaranteed predictability of classical control theory

247
00:12:07.840 --> 00:12:09.360
<v Speaker 2>and formal logic, which.

248
00:12:09.159 --> 00:12:12.519
<v Speaker 1>Is huge for you, the listener, to understand because this

249
00:12:12.600 --> 00:12:15.480
<v Speaker 1>transition from AI being a fun novelty that writes your

250
00:12:15.519 --> 00:12:19.639
<v Speaker 1>emails to the invisible manager of the power grid, financial markets,

251
00:12:19.679 --> 00:12:23.440
<v Speaker 1>and aviation. It means the era of move fast and

252
00:12:23.480 --> 00:12:26.679
<v Speaker 1>break things is well and truly over. We are entering

253
00:12:26.720 --> 00:12:30.080
<v Speaker 1>an era of mathematically proven operational boundaries.

254
00:12:30.320 --> 00:12:33.440
<v Speaker 2>It's a fundamental shift in how civilization operates. But I

255
00:12:33.480 --> 00:12:35.000
<v Speaker 2>want to leave you with a thought to moll over.

256
00:12:35.480 --> 00:12:38.919
<v Speaker 2>Building on those immutable audit trails we talked about earlier, Okay,

257
00:12:38.960 --> 00:12:42.000
<v Speaker 2>late on this. Consider this scenario. What happens when the

258
00:12:42.000 --> 00:12:46.279
<v Speaker 2>AI operates perfectly within its mathematically proven guardrails, but a

259
00:12:46.320 --> 00:12:47.639
<v Speaker 2>catastrophe still occurs?

260
00:12:47.799 --> 00:12:49.240
<v Speaker 3>Wait? How would that happen?

261
00:12:49.440 --> 00:12:51.840
<v Speaker 2>Because the human engineers designed the wrong guardrails in the

262
00:12:51.840 --> 00:12:55.039
<v Speaker 2>first place? Oh man, Right, When the unalterable audit trail

263
00:12:55.200 --> 00:12:58.600
<v Speaker 2>proves definitively that the AI did exactly what we constrained

264
00:12:58.639 --> 00:13:01.120
<v Speaker 2>to do, who takes the legal fall for the black

265
00:13:01.200 --> 00:13:03.720
<v Speaker 2>app Is it the AI developer who made the reasoning

266
00:13:03.759 --> 00:13:08.039
<v Speaker 2>engine the safety engineer who hardcoded the salty guardrails or

267
00:13:08.080 --> 00:13:09.600
<v Speaker 2>the auditor who signed off on it.

268
00:13:09.960 --> 00:13:13.480
<v Speaker 1>Wow, we built the perfect bowling alley, but we pointed

269
00:13:13.480 --> 00:13:14.679
<v Speaker 1>the lane at a crowd of.

270
00:13:14.639 --> 00:13:19.120
<v Speaker 2>People exactly when. The math is flawless, but the human

271
00:13:19.159 --> 00:13:22.679
<v Speaker 2>intent was flawed. Where does the liability land that.

272
00:13:22.720 --> 00:13:24.600
<v Speaker 3>Is, Yeah, that's a chilling realization.

273
00:13:25.480 --> 00:13:28.519
<v Speaker 1>The most dangerous part of this incredibly advanced AI loop

274
00:13:28.840 --> 00:13:30.879
<v Speaker 1>might still be the humans riding the rules.

275
00:13:31.320 --> 00:13:33.240
<v Speaker 3>Well. That gives you plenty to explore on your own

276
00:13:33.320 --> 00:13:35.240
<v Speaker 3>the next time you flip a light switch or bord

277
00:13:35.240 --> 00:13:35.600
<v Speaker 3>a flight.

278
00:13:35.960 --> 00:13:38.919
<v Speaker 1>The invisible architecture taking over our physical world is an

279
00:13:38.960 --> 00:13:42.039
<v Speaker 1>absolute marvel, but it is walking a razor thin line.

280
00:13:42.240 --> 00:13:43.399
<v Speaker 3>Keep questioning the systems

281
00:13:43.440 --> 00:13:45.759
<v Speaker 1>Around you, stay curious, and we will catch you on

282
00:13:45.840 --> 00:13:46.840
<v Speaker 1>the next beef dive.