WEBVTT

1
00:00:00.360 --> 00:00:03.200
<v Speaker 1>Hey Carl here, we're on a mission to find new

2
00:00:03.240 --> 00:00:05.960
<v Speaker 1>fans of the show. I mean, where else can you

3
00:00:05.960 --> 00:00:10.279
<v Speaker 1>get real life comedy and horror in one podcast. You

4
00:00:10.320 --> 00:00:13.000
<v Speaker 1>can do your part by leaving us a review on iTunes,

5
00:00:13.039 --> 00:00:16.719
<v Speaker 1>Spotify or wherever you get Security this week. Also, you

6
00:00:16.760 --> 00:00:19.079
<v Speaker 1>can get an ad free feed by becoming a five

7
00:00:19.160 --> 00:00:22.079
<v Speaker 1>dollars a month patron. Go to Patreon dot Security this

8
00:00:22.120 --> 00:00:25.399
<v Speaker 1>week dot com and sign up, and don't forget Discord

9
00:00:25.640 --> 00:00:28.559
<v Speaker 1>Discord dot Security this week dot com. Lots of good

10
00:00:28.559 --> 00:00:32.320
<v Speaker 1>stuff happening there. Yeah, I think that's it. So on

11
00:00:32.399 --> 00:00:36.719
<v Speaker 1>with the podcast. Hey Patrick, Yeah, you you and your

12
00:00:36.759 --> 00:00:39.600
<v Speaker 1>wife are come from German heritage.

13
00:00:39.240 --> 00:00:40.799
<v Speaker 2>Right, I'm Irish, but she's German.

14
00:00:40.920 --> 00:00:43.000
<v Speaker 1>Yeah, all right, So here's a German joke for her.

15
00:00:43.200 --> 00:00:44.159
<v Speaker 2>Oh, I'm sure she'll love it.

16
00:00:44.240 --> 00:00:47.119
<v Speaker 1>You know about the German baby that didn't talk no

17
00:00:47.679 --> 00:00:50.799
<v Speaker 1>until he was five or so. He didn't say a

18
00:00:50.840 --> 00:00:54.439
<v Speaker 1>word all his life. Five years old, he's at the

19
00:00:54.479 --> 00:00:57.560
<v Speaker 1>dinner table and finally the first words he says is

20
00:00:58.640 --> 00:01:01.920
<v Speaker 1>this thrillllle is tap and everybody they all freak out.

21
00:01:01.920 --> 00:01:05.799
<v Speaker 1>Oh my god, Henry, you speak, You can speak? How

22
00:01:05.799 --> 00:01:21.840
<v Speaker 1>come you haven't spoken up till now? Everything has been satisfactory. Hey,

23
00:01:21.879 --> 00:01:24.599
<v Speaker 1>welcome back to Security this week. I'm Carl Franklin's Patrick

24
00:01:24.640 --> 00:01:28.000
<v Speaker 1>Hines and Duaye Laflotte. We're here to discuss some of

25
00:01:28.040 --> 00:01:31.280
<v Speaker 1>the things that happened last week. Anyway, I think we

26
00:01:31.319 --> 00:01:35.359
<v Speaker 1>should start with this US Army sites hacked to display

27
00:01:35.439 --> 00:01:42.959
<v Speaker 1>anti Trump pro Kurdistan messages. Wa, So what happened here?

28
00:01:44.400 --> 00:01:46.599
<v Speaker 3>This is this is a Yeah, there were a couple

29
00:01:46.599 --> 00:01:51.680
<v Speaker 3>different site basic hygiene. Yeah, agreed. So there's the Army

30
00:01:51.760 --> 00:01:55.519
<v Speaker 3>AI Integration Center site. So we're not talking like, uh,

31
00:01:55.799 --> 00:01:58.959
<v Speaker 3>you know, I don't know, Department of Wars website. Yeah,

32
00:01:59.040 --> 00:02:02.760
<v Speaker 3>exactly right. Wait was that was that war games? Was

33
00:02:02.760 --> 00:02:05.280
<v Speaker 3>that normat? I think it was. Yeah, that's a good

34
00:02:05.359 --> 00:02:07.439
<v Speaker 3>callback batter, I know.

35
00:02:07.640 --> 00:02:08.000
<v Speaker 1>Damn.

36
00:02:08.879 --> 00:02:12.599
<v Speaker 3>So the Army AI Integration Center and the Open Innovation

37
00:02:12.840 --> 00:02:17.400
<v Speaker 3>Lab run by the military, these are not systems. For

38
00:02:17.439 --> 00:02:20.960
<v Speaker 3>those of you who have worked in you know, uh

39
00:02:21.159 --> 00:02:27.400
<v Speaker 3>tss CI A tile environments, top secret clearance, compartmentalized information

40
00:02:27.719 --> 00:02:31.039
<v Speaker 3>type stuff if you know the terms SIPPER and nipper

41
00:02:31.080 --> 00:02:33.680
<v Speaker 3>if you know what those mean. These systems are not

42
00:02:33.719 --> 00:02:38.800
<v Speaker 3>connected to those, right. These were like third party you

43
00:02:38.840 --> 00:02:43.039
<v Speaker 3>know kind of websites almost toasted by go daddy style

44
00:02:43.319 --> 00:02:48.360
<v Speaker 3>WordPress websites explaining you know what those departments do, but

45
00:02:48.639 --> 00:02:51.199
<v Speaker 3>not you know, not officially run by them.

46
00:02:51.520 --> 00:02:54.800
<v Speaker 1>I thought the hack was pretty innovative that the pages

47
00:02:54.879 --> 00:02:58.120
<v Speaker 1>usually show a four oh four right, yes, not found,

48
00:02:58.520 --> 00:03:04.439
<v Speaker 1>but known attacker modified the default error page and replaced

49
00:03:04.479 --> 00:03:06.919
<v Speaker 1>it with their political notice.

50
00:03:07.280 --> 00:03:10.280
<v Speaker 3>Yeah, and that's interesting too, because it's like, there's two

51
00:03:10.319 --> 00:03:12.960
<v Speaker 3>reasons why you might might do that as an attacker. One,

52
00:03:14.159 --> 00:03:16.479
<v Speaker 3>you know, maybe you get access to an endpoint that

53
00:03:16.560 --> 00:03:18.319
<v Speaker 3>only has the ability to edit the WORL four. Like

54
00:03:18.360 --> 00:03:20.599
<v Speaker 3>let's say this was a word Press site and there

55
00:03:20.639 --> 00:03:22.479
<v Speaker 3>was a plugin that allows you to edit the foural

56
00:03:22.560 --> 00:03:25.639
<v Speaker 3>four pages, and maybe that's the insecure plug in, so

57
00:03:25.680 --> 00:03:27.599
<v Speaker 3>that's why they edited that Page's possible it was the

58
00:03:27.639 --> 00:03:31.280
<v Speaker 3>only path, but the other path might be you know

59
00:03:31.439 --> 00:03:34.680
<v Speaker 3>how blatantly obvious it is if your main web page

60
00:03:34.719 --> 00:03:37.080
<v Speaker 3>has changed. But if somebody changes a four or four.

61
00:03:36.960 --> 00:03:38.400
<v Speaker 2>Page, it might be a while.

62
00:03:38.439 --> 00:03:41.759
<v Speaker 3>It might be a while before somebody finds it, sees it, whatever, right,

63
00:03:41.800 --> 00:03:43.520
<v Speaker 3>and it gives you a little bit more time for

64
00:03:43.599 --> 00:03:47.719
<v Speaker 3>the forensics on those servers to get wiped away. So

65
00:03:48.000 --> 00:03:51.560
<v Speaker 3>that's that's my hacker advice. Yeah, Okay, don't always go

66
00:03:51.599 --> 00:03:53.719
<v Speaker 3>for the front door. Sometimes the four or four is

67
00:03:53.759 --> 00:04:01.400
<v Speaker 3>the way to go the four oh four.

68
00:04:01.560 --> 00:04:05.520
<v Speaker 1>So Patrick, this next one you added as a kind

69
00:04:05.599 --> 00:04:07.280
<v Speaker 1>of a last week thing, right.

70
00:04:07.199 --> 00:04:11.240
<v Speaker 2>Yeah, I stumbled on it shortly after we recorded. There's

71
00:04:11.439 --> 00:04:14.240
<v Speaker 2>two new fishing kits. And the thing that makes them

72
00:04:14.479 --> 00:04:18.600
<v Speaker 2>interesting is that they get around multi factor authentication, especially

73
00:04:18.600 --> 00:04:21.959
<v Speaker 2>in Office three sixty five, Microsoft three sixty five and

74
00:04:22.759 --> 00:04:25.000
<v Speaker 2>up until now a lot of people have just thought that, well,

75
00:04:25.040 --> 00:04:27.800
<v Speaker 2>I've got multi factor, so I'm fine, and you're never fine,

76
00:04:28.240 --> 00:04:34.519
<v Speaker 2>just like little Henrick. It's not satisfactory. Nice callback. So

77
00:04:35.040 --> 00:04:38.000
<v Speaker 2>you know this, Dwayne loves these, which means they're really bad,

78
00:04:38.240 --> 00:04:41.279
<v Speaker 2>so good, really bad, Dwayne, why don't you talk about

79
00:04:41.319 --> 00:04:42.680
<v Speaker 2>like why you love these so much?

80
00:04:42.720 --> 00:04:45.160
<v Speaker 3>So these are these are awesome. People don't think about

81
00:04:45.240 --> 00:04:47.800
<v Speaker 3>like everybody does this, but nobody thinks about what it's

82
00:04:47.839 --> 00:04:52.480
<v Speaker 3>actually doing. So when we talk about tokens and this,

83
00:04:52.480 --> 00:04:55.800
<v Speaker 3>this particular is a manipulation of tokens. And on this

84
00:04:55.839 --> 00:04:58.079
<v Speaker 3>show we've talked about token tactics, we've talked about all

85
00:04:58.120 --> 00:05:00.879
<v Speaker 3>sorts of things had to manipulate Office. That's and we've

86
00:05:00.879 --> 00:05:03.439
<v Speaker 3>given this analogy. Before you know, you go buy a

87
00:05:03.480 --> 00:05:06.720
<v Speaker 3>brand new smart TV and you fired up. You turn

88
00:05:06.759 --> 00:05:08.319
<v Speaker 3>it on and hope you don't connect it to your

89
00:05:08.319 --> 00:05:10.279
<v Speaker 3>real network. You connect it to some sort of IoT

90
00:05:10.399 --> 00:05:14.240
<v Speaker 3>guess network, and you go to connect Netflix, and there's

91
00:05:14.279 --> 00:05:16.600
<v Speaker 3>two options. There's you know, log in with your username

92
00:05:16.600 --> 00:05:18.519
<v Speaker 3>and password, which is just a pain in the ass

93
00:05:18.519 --> 00:05:22.600
<v Speaker 3>on that little remote control they give you. Or they

94
00:05:22.680 --> 00:05:25.079
<v Speaker 3>give you a code on the screen and you can

95
00:05:25.120 --> 00:05:27.519
<v Speaker 3>go to any browser and you can log into your

96
00:05:27.519 --> 00:05:30.759
<v Speaker 3>Netflix account and you can type that code in. And

97
00:05:30.800 --> 00:05:33.360
<v Speaker 3>when you do that, now that TV gets access to

98
00:05:33.399 --> 00:05:36.199
<v Speaker 3>your account, and you're like, oh, this is nice and easy,

99
00:05:37.120 --> 00:05:40.600
<v Speaker 3>right exactly, this is the same type of thing. You

100
00:05:40.639 --> 00:05:43.279
<v Speaker 3>can do the same thing with your Microsoft account. So

101
00:05:43.439 --> 00:05:49.560
<v Speaker 3>there's the ability to give IoT style access to your

102
00:05:49.560 --> 00:05:51.959
<v Speaker 3>Microsoft account through a token.

103
00:05:51.759 --> 00:05:55.360
<v Speaker 2>Which ranks right up there with home dentistry, those good ideas.

104
00:05:55.480 --> 00:05:59.279
<v Speaker 3>Yep, literally, the same thing, the same thing you're going through.

105
00:05:59.639 --> 00:06:02.399
<v Speaker 3>You get token in an email, you literally go to

106
00:06:02.519 --> 00:06:07.720
<v Speaker 3>Microsoft's real site. Right, So if you're using a password manager,

107
00:06:07.759 --> 00:06:10.920
<v Speaker 3>if you're using pass keys, there will be no warnings

108
00:06:10.920 --> 00:06:14.079
<v Speaker 3>here because they're bringing you right to the real Microsoft

109
00:06:14.160 --> 00:06:18.160
<v Speaker 3>site and when you get there, they're giving you the

110
00:06:18.199 --> 00:06:20.319
<v Speaker 3>instructions on what to type in there? Oh, log in

111
00:06:20.360 --> 00:06:22.680
<v Speaker 3>like you normally would. We need you to verify your

112
00:06:22.680 --> 00:06:25.199
<v Speaker 3>account by typing in this code, you know, two six

113
00:06:25.279 --> 00:06:27.079
<v Speaker 3>three four two eight writ whatever.

114
00:06:27.079 --> 00:06:30.560
<v Speaker 1>It is both interesting is that code not only authenticates

115
00:06:30.600 --> 00:06:33.480
<v Speaker 1>you but authorizes you too. Yeah, one little code.

116
00:06:33.720 --> 00:06:37.079
<v Speaker 3>And what was awesome with the older token tactics is

117
00:06:37.120 --> 00:06:39.959
<v Speaker 3>that what you got access to was this graph API,

118
00:06:40.040 --> 00:06:41.680
<v Speaker 3>which gave you the ability to do all sorts of things

119
00:06:41.800 --> 00:06:45.639
<v Speaker 3>like read emails and whatnot. But then every ninety days

120
00:06:45.639 --> 00:06:48.040
<v Speaker 3>you could request a new token if you wanted to,

121
00:06:48.160 --> 00:06:49.839
<v Speaker 3>so you could be in their end definitely, and if

122
00:06:49.879 --> 00:06:53.240
<v Speaker 3>the user changed their password, it didn't matter.

123
00:06:53.319 --> 00:06:54.399
<v Speaker 1>Sounds very convenient.

124
00:06:54.720 --> 00:06:57.839
<v Speaker 3>Unless the user went in and like killed all tokens

125
00:06:57.879 --> 00:07:01.079
<v Speaker 3>and said log out all my devices. The attackers were

126
00:07:01.079 --> 00:07:03.800
<v Speaker 3>still in there. So it's it's a brilliant tactic. Honestly,

127
00:07:04.079 --> 00:07:06.680
<v Speaker 3>I really I really like that they're reviving it. I

128
00:07:06.680 --> 00:07:09.759
<v Speaker 3>mean I missed token tactics and now evil Tactics is out,

129
00:07:09.759 --> 00:07:10.600
<v Speaker 3>so you know, it's it's good.

130
00:07:10.639 --> 00:07:11.160
<v Speaker 1>It's all good.

131
00:07:11.759 --> 00:07:15.680
<v Speaker 2>Yes, okay, okay.

132
00:07:16.120 --> 00:07:20.040
<v Speaker 1>Along with that, and speaking of keys, the next story

133
00:07:20.120 --> 00:07:25.720
<v Speaker 1>critical SharePoint remote code execution flaw exploited to steal machine keys.

134
00:07:25.800 --> 00:07:29.680
<v Speaker 1>What's the machine key? Oh no, not that the machine key,

135
00:07:29.759 --> 00:07:31.800
<v Speaker 1>the machine the key machine. Yes.

136
00:07:35.560 --> 00:07:38.720
<v Speaker 3>Dot net has a whole bunch of keys. And for

137
00:07:38.759 --> 00:07:40.160
<v Speaker 3>those of you who have heard of like the dp

138
00:07:40.279 --> 00:07:46.680
<v Speaker 3>A p I, which is the Data protection API and Windows. Wow,

139
00:07:46.759 --> 00:07:48.600
<v Speaker 3>how do we get into this without getting into too

140
00:07:48.680 --> 00:07:49.319
<v Speaker 3>much crypto?

141
00:07:50.120 --> 00:07:50.800
<v Speaker 1>Uh?

142
00:07:51.000 --> 00:07:51.439
<v Speaker 3>There are more?

143
00:07:51.839 --> 00:07:52.800
<v Speaker 2>Thing is too much crypto?

144
00:07:53.120 --> 00:07:54.680
<v Speaker 3>Well, I don't.

145
00:07:54.839 --> 00:07:56.759
<v Speaker 1>Let's talk about the math. The key is just a

146
00:07:57.600 --> 00:08:02.199
<v Speaker 1>thank you Carl, a string of cars that identifies something

147
00:08:03.839 --> 00:08:05.680
<v Speaker 1>to you know, think of it as a password, but

148
00:08:05.720 --> 00:08:06.360
<v Speaker 1>it's more than that.

149
00:08:06.439 --> 00:08:09.920
<v Speaker 3>It's used to encrypt, right, yeah, yeah, absolutely, And and

150
00:08:09.959 --> 00:08:12.720
<v Speaker 3>what's neat about it? Used to be you would use

151
00:08:12.759 --> 00:08:15.279
<v Speaker 3>the same key for every system, like, oh, we have

152
00:08:15.360 --> 00:08:16.759
<v Speaker 3>this really long key and we use it for every

153
00:08:16.759 --> 00:08:18.639
<v Speaker 3>system and blah blah blah, and you're encrypted and it's

154
00:08:18.639 --> 00:08:21.600
<v Speaker 3>a symmetric key, and huzzak. Everything's great. The problem is

155
00:08:21.600 --> 00:08:23.480
<v Speaker 3>if one of those systems got compromised, you could read

156
00:08:23.519 --> 00:08:25.800
<v Speaker 3>the date on all the systems, So then they said, okay, well,

157
00:08:25.839 --> 00:08:27.959
<v Speaker 3>you know'd be great is if each system had their

158
00:08:28.000 --> 00:08:32.559
<v Speaker 3>own unique key, a machine key that could then do

159
00:08:32.639 --> 00:08:36.799
<v Speaker 3>those levels of encryption and authorization or authentication of tokens

160
00:08:36.799 --> 00:08:37.639
<v Speaker 3>and all sorts of stuff.

161
00:08:37.639 --> 00:08:40.279
<v Speaker 1>Well, but the interesting thing is every network connection has

162
00:08:40.320 --> 00:08:42.840
<v Speaker 1>its own unique identifier that can be used as a key.

163
00:08:42.960 --> 00:08:47.159
<v Speaker 3>Yeah, absolutely, yeah, And there's there's all sorts of HMAC,

164
00:08:47.200 --> 00:08:49.759
<v Speaker 3>there's all sorts of things you can use. Just the

165
00:08:49.799 --> 00:08:53.320
<v Speaker 3>machine key is typically tied to, you know, a dot

166
00:08:53.440 --> 00:08:56.799
<v Speaker 3>ED installation of well really anything. Every Windows computer has

167
00:08:56.840 --> 00:08:59.679
<v Speaker 3>a dot N installation on it. So if you can

168
00:08:59.679 --> 00:09:02.399
<v Speaker 3>get out access to those keys, you can do things

169
00:09:02.440 --> 00:09:06.159
<v Speaker 3>like maybe manipulate the encryption on the box. You can

170
00:09:06.200 --> 00:09:08.559
<v Speaker 3>do things like say that you're authorized to a site

171
00:09:08.559 --> 00:09:11.200
<v Speaker 3>by encrypting a token that might be passed around whatever

172
00:09:11.240 --> 00:09:15.519
<v Speaker 3>it may be. So getting access to those is reasonably important.

173
00:09:16.759 --> 00:09:19.279
<v Speaker 3>In this particular one, the POC attempts to trigger a

174
00:09:19.360 --> 00:09:24.600
<v Speaker 3>remote code execution by delivering a dot net binary format

175
00:09:24.720 --> 00:09:30.360
<v Speaker 3>or payload as a cookie of a forged security context token. Right,

176
00:09:30.440 --> 00:09:32.559
<v Speaker 3>So in this particular case, they're doing exactly what we're

177
00:09:32.600 --> 00:09:36.840
<v Speaker 3>just saying. They're they're using those to then forge a

178
00:09:36.960 --> 00:09:40.240
<v Speaker 3>token and using inside that token they're they're calling a

179
00:09:40.279 --> 00:09:43.480
<v Speaker 3>binary format or that allows them to get remote code execution.

180
00:09:43.679 --> 00:09:47.759
<v Speaker 1>So is this a legal thing for for anybody to

181
00:09:47.799 --> 00:09:50.200
<v Speaker 1>be able to do like, is this an expected thing

182
00:09:50.200 --> 00:09:53.559
<v Speaker 1>that was just exploited or is this like a bug

183
00:09:54.039 --> 00:09:55.320
<v Speaker 1>if you will, No, this is.

184
00:09:55.399 --> 00:09:57.960
<v Speaker 3>This is just a bug. Okay, this is a de

185
00:09:58.120 --> 00:10:02.320
<v Speaker 3>serialization bug if er recorrectly, yeah, going through it. Yeah,

186
00:10:02.320 --> 00:10:06.759
<v Speaker 3>it's a decerialization bug in here, so m I actually

187
00:10:06.960 --> 00:10:08.799
<v Speaker 3>de serialization bugs are awesome.

188
00:10:08.519 --> 00:10:12.840
<v Speaker 1>Too, but unless they happen to you.

189
00:10:13.080 --> 00:10:17.080
<v Speaker 3>Yeah, yeah, even then. Honestly, sometimes we'll just put systems

190
00:10:17.120 --> 00:10:19.600
<v Speaker 3>on the Internet, no firewall, just to watch the magic.

191
00:10:20.200 --> 00:10:23.720
<v Speaker 3>Grab some pointers. Yeah see I'll stick.

192
00:10:23.600 --> 00:10:25.639
<v Speaker 2>Man, get a copy of Shrek in French.

193
00:10:25.879 --> 00:10:29.080
<v Speaker 3>Yeah, whatever, and just see how quickly they they actually happened,

194
00:10:29.120 --> 00:10:30.200
<v Speaker 3>get ripped offline.

195
00:10:30.960 --> 00:10:33.120
<v Speaker 2>Some frenchman was putting all of his movies on one

196
00:10:33.120 --> 00:10:34.840
<v Speaker 2>of ours. We got copies.

197
00:10:35.240 --> 00:10:37.840
<v Speaker 1>Wow. See that's the French. The French would do that.

198
00:10:37.879 --> 00:10:43.759
<v Speaker 1>The Germans would put strudele recipes, kept it strude.

199
00:10:44.519 --> 00:10:46.639
<v Speaker 2>I thought that was I know it's it's more than

200
00:10:46.679 --> 00:10:48.080
<v Speaker 2>a week old, but I thought it was important for

201
00:10:48.200 --> 00:10:48.559
<v Speaker 2>us to cover.

202
00:10:48.679 --> 00:10:51.919
<v Speaker 1>Okay, should we take a break, Yeah, all right, we'll

203
00:10:51.960 --> 00:10:59.279
<v Speaker 1>be right back after these very important messages, and we're back.

204
00:10:59.480 --> 00:11:03.480
<v Speaker 1>It's Secury this week. I'm Carl, that's Patrick, Dwayne and Heinrich,

205
00:11:04.080 --> 00:11:07.320
<v Speaker 1>who is now speaking, and we got a few more

206
00:11:08.039 --> 00:11:12.039
<v Speaker 1>a few more stories here. I hadn't heard of trick

207
00:11:12.120 --> 00:11:16.600
<v Speaker 1>butt before, but this headline is Trickbot variant uses DNS

208
00:11:16.720 --> 00:11:21.759
<v Speaker 1>tunneling for command and control. So tunneling kind of like

209
00:11:21.799 --> 00:11:25.399
<v Speaker 1>a VPN. Is that like wrapping the packet in a

210
00:11:25.480 --> 00:11:28.000
<v Speaker 1>header and footer to make it look like something else

211
00:11:28.120 --> 00:11:32.120
<v Speaker 1>or inside the DNS packet? I'm not sure.

212
00:11:32.320 --> 00:11:35.559
<v Speaker 3>Yeah, absolutely, it's the same type of thing. Think of

213
00:11:35.600 --> 00:11:39.039
<v Speaker 3>it even like a VPN. Right, So you're using a VPN,

214
00:11:39.120 --> 00:11:43.840
<v Speaker 3>you're tunneling all of your TCPIP traffic in these packets

215
00:11:43.879 --> 00:11:47.039
<v Speaker 3>that really the firewalls can't understand, but they send them along.

216
00:11:47.080 --> 00:11:48.879
<v Speaker 3>They have the proper headers to route out and that

217
00:11:48.919 --> 00:11:49.759
<v Speaker 3>sort of stuff.

218
00:11:49.480 --> 00:11:51.679
<v Speaker 1>But the actual packet just looks like data.

219
00:11:51.799 --> 00:11:54.879
<v Speaker 3>Exactly in the middle part of that packet is encrypted, right,

220
00:11:54.919 --> 00:11:57.480
<v Speaker 3>so they can't read it, and they all route it

221
00:11:57.519 --> 00:11:59.240
<v Speaker 3>along and when he gets to the proper place, it

222
00:11:59.279 --> 00:12:02.879
<v Speaker 3>gets decrypted. And then in a VPN situation, your Internet

223
00:12:02.879 --> 00:12:05.200
<v Speaker 3>traffic just routes wherever it landed.

224
00:12:05.039 --> 00:12:09.120
<v Speaker 1>Unless the place that you're trying to use while on

225
00:12:09.240 --> 00:12:12.919
<v Speaker 1>a VPN detects that you are using a VPN and say, hey,

226
00:12:12.919 --> 00:12:15.120
<v Speaker 1>you can't use a VPN, yeah, exactly, which a lot

227
00:12:15.120 --> 00:12:17.840
<v Speaker 1>of the streaming services do that. Yeah, And here's what

228
00:12:17.879 --> 00:12:19.519
<v Speaker 1>I'll tell you. I mean a lot of that comes

229
00:12:19.519 --> 00:12:22.240
<v Speaker 1>from a lot of the streaming services understanding what the

230
00:12:22.320 --> 00:12:26.120
<v Speaker 1>VPN end points are that you would come out of. Yeah.

231
00:12:26.159 --> 00:12:30.320
<v Speaker 3>So, for example, let's say you could get one installed

232
00:12:30.840 --> 00:12:33.399
<v Speaker 3>at home and then route through your house is a VPN,

233
00:12:34.159 --> 00:12:35.919
<v Speaker 3>chances are you're fine there, So you go anywhere in

234
00:12:35.919 --> 00:12:40.559
<v Speaker 3>the world and still get your streaming services or yeah, right.

235
00:12:40.639 --> 00:12:45.799
<v Speaker 1>Or yeah, I should stop talking, should stop talking right now.

236
00:12:45.639 --> 00:12:49.480
<v Speaker 3>Let's just say there's a lot of clouds, it's just aws.

237
00:12:51.320 --> 00:12:54.200
<v Speaker 3>Maybe you spin up a VPC somewhere.

238
00:12:55.200 --> 00:12:57.120
<v Speaker 1>The legal way to do this is use a service

239
00:12:57.200 --> 00:13:01.759
<v Speaker 1>like Nord VPN and purchase an IP address that nobody

240
00:13:01.799 --> 00:13:02.600
<v Speaker 1>else has.

241
00:13:02.720 --> 00:13:06.000
<v Speaker 3>So you can you can absolutely do that works, yeah,

242
00:13:06.120 --> 00:13:09.519
<v Speaker 3>or rally through Amazon. Yeah, okay, and then it's a

243
00:13:09.559 --> 00:13:13.240
<v Speaker 3>little sketchier, but who knows. So in this particular case,

244
00:13:15.159 --> 00:13:18.919
<v Speaker 3>they're disguising, right, like you would disguise your web browsing

245
00:13:19.000 --> 00:13:23.000
<v Speaker 3>traffic by using a VPN, they're disguising their C two traffic,

246
00:13:23.000 --> 00:13:26.039
<v Speaker 3>their command and control traffic because They don't want you

247
00:13:26.080 --> 00:13:27.960
<v Speaker 3>to discovered of course, right right, They don't want you

248
00:13:28.039 --> 00:13:31.600
<v Speaker 3>to know what they're actually x fultrading or infiltrating into

249
00:13:31.600 --> 00:13:32.840
<v Speaker 3>your in your network.

250
00:13:32.679 --> 00:13:34.679
<v Speaker 2>Or that they're infiltrating or exful trading.

251
00:13:34.879 --> 00:13:37.159
<v Speaker 3>Yeah, well that's true. That's that's a good point.

252
00:13:37.240 --> 00:13:40.600
<v Speaker 1>So what did they set up a DNS service and

253
00:13:40.759 --> 00:13:45.879
<v Speaker 1>then whoever pings that service, they can un tunnel it,

254
00:13:46.200 --> 00:13:48.519
<v Speaker 1>unwrap it and see what the actual message is. Is

255
00:13:48.559 --> 00:13:49.440
<v Speaker 1>that how they were doing it?

256
00:13:49.559 --> 00:13:52.399
<v Speaker 3>Yeah, exactly, Yeah. I mean think of it this way

257
00:13:52.799 --> 00:13:55.559
<v Speaker 3>for those of you who've really kind of thought deeply

258
00:13:55.600 --> 00:13:58.960
<v Speaker 3>about how the Internet works. I guess when you go

259
00:13:59.000 --> 00:14:02.600
<v Speaker 3>to www dot mysoft, dot com, right, there's a whole

260
00:14:02.639 --> 00:14:05.279
<v Speaker 3>bunch of dot com So it flips it and it

261
00:14:05.320 --> 00:14:08.039
<v Speaker 3>looks it looks at the address backwards. So there's a

262
00:14:08.039 --> 00:14:10.519
<v Speaker 3>whole bunch of dot com servers right, and dot gov

263
00:14:10.559 --> 00:14:12.879
<v Speaker 3>and dot dot d U and whatever. Right, there's a

264
00:14:12.879 --> 00:14:17.039
<v Speaker 3>whole bunch of root level domain controllers that that handle those.

265
00:14:17.639 --> 00:14:20.240
<v Speaker 3>Then the dot com server goes, Okay, I know where

266
00:14:20.320 --> 00:14:23.600
<v Speaker 3>Microsoft is. I don't know where this dubww you know

267
00:14:23.720 --> 00:14:25.480
<v Speaker 3>www is. I don't know where that is, but I

268
00:14:25.480 --> 00:14:28.360
<v Speaker 3>know where Microsoft is, so let me point you at Microsoft.

269
00:14:28.720 --> 00:14:31.759
<v Speaker 3>So then your traffic goes from there to Microsoft and

270
00:14:31.799 --> 00:14:36.039
<v Speaker 3>the Microsoft server goes. Okay, I know where Www is,

271
00:14:36.159 --> 00:14:38.840
<v Speaker 3>I know where staging dot Microsoft is, I know where

272
00:14:39.240 --> 00:14:42.720
<v Speaker 3>you know Xbox dot Microsoft is exactly, and it has

273
00:14:42.759 --> 00:14:45.320
<v Speaker 3>an IP address, probably have a load balance or whatever

274
00:14:46.240 --> 00:14:49.320
<v Speaker 3>of each of those different what they call canonical names

275
00:14:50.120 --> 00:14:52.279
<v Speaker 3>or a records is depending on what you're you know,

276
00:14:52.320 --> 00:14:55.360
<v Speaker 3>what you're putting in there that point to those services.

277
00:14:56.039 --> 00:15:01.320
<v Speaker 3>Now imagine this, Imagine that Microsoft DNS controlled by us,

278
00:15:01.559 --> 00:15:04.320
<v Speaker 3>the hackers, right, so somebody comes to us looking for Www.

279
00:15:04.440 --> 00:15:05.879
<v Speaker 3>We just throw it out when we go that's garbage.

280
00:15:05.879 --> 00:15:06.240
<v Speaker 1>I don't give.

281
00:15:06.279 --> 00:15:07.039
<v Speaker 3>I don't care about that.

282
00:15:07.159 --> 00:15:10.080
<v Speaker 1>You mean we we resolve it to the actual Www. Yeah,

283
00:15:10.120 --> 00:15:13.120
<v Speaker 1>maybe we do, right, but let's he probably would, so

284
00:15:13.120 --> 00:15:15.240
<v Speaker 1>you wouldn't be detected. I'm trying to think like an

285
00:15:15.240 --> 00:15:16.080
<v Speaker 1>evil person here.

286
00:15:16.200 --> 00:15:19.759
<v Speaker 3>Sure, well, and and maybe where you know, I don't know,

287
00:15:21.480 --> 00:15:24.240
<v Speaker 3>shark laser, shark cages dot com.

288
00:15:23.960 --> 00:15:25.559
<v Speaker 1>Right, Bob's discount shark cages.

289
00:15:25.720 --> 00:15:28.159
<v Speaker 3>There you go, Bob's Discount shark cages. So we're not

290
00:15:28.320 --> 00:15:30.960
<v Speaker 3>like hackers RS dot com right, but we're you know,

291
00:15:31.000 --> 00:15:34.159
<v Speaker 3>we have we control this DNS server. Now, let's say

292
00:15:34.200 --> 00:15:37.840
<v Speaker 3>somebody comes to me looking for an address that is

293
00:15:38.159 --> 00:15:43.559
<v Speaker 3>you know A E three, four six nine, B C two,

294
00:15:44.720 --> 00:15:49.960
<v Speaker 3>you know D five dot Bob's Discount, you know Shark

295
00:15:50.000 --> 00:15:54.440
<v Speaker 3>lasercages dot com, and that that big front name is

296
00:15:54.480 --> 00:15:57.600
<v Speaker 3>actually just a base sixty four encoded response.

297
00:15:58.039 --> 00:15:58.360
<v Speaker 1>Wow.

298
00:15:58.440 --> 00:16:00.399
<v Speaker 3>So when I see that name, instead of revolving it

299
00:16:00.399 --> 00:16:03.360
<v Speaker 3>to an IP address, I take that crazy name on

300
00:16:03.360 --> 00:16:06.039
<v Speaker 3>the front and I dBase sixty four it back in

301
00:16:06.080 --> 00:16:09.879
<v Speaker 3>the text and now I'm getting commands back. Right. So

302
00:16:10.000 --> 00:16:15.279
<v Speaker 3>that's one way to do DNS tunneling as a C two.

303
00:16:15.759 --> 00:16:20.440
<v Speaker 3>And when I respond, instead of responding with an IP address,

304
00:16:20.679 --> 00:16:23.600
<v Speaker 3>it's my agent calling me. I know that I'm going

305
00:16:23.639 --> 00:16:26.360
<v Speaker 3>to respond with what's called a text record, which is

306
00:16:26.399 --> 00:16:29.120
<v Speaker 3>here's the command I want you to run now, and

307
00:16:29.159 --> 00:16:31.240
<v Speaker 3>then it's going to come back with what the answer

308
00:16:31.360 --> 00:16:33.679
<v Speaker 3>on that command was, whether it's show me all the users?

309
00:16:33.720 --> 00:16:34.480
<v Speaker 1>And that's sort of stuff.

310
00:16:34.600 --> 00:16:37.039
<v Speaker 2>And you can just take an open source implementation of

311
00:16:37.120 --> 00:16:39.600
<v Speaker 2>DNS and bastardize the protocol.

312
00:16:40.159 --> 00:16:42.279
<v Speaker 3>You can. You don't even have to you don't have

313
00:16:42.360 --> 00:16:46.240
<v Speaker 3>to write it from scratch. There are tons of evil DNS.

314
00:16:46.360 --> 00:16:49.600
<v Speaker 1>Well, there's open source DNS servers too. Like Patrick, there's tons.

315
00:16:49.480 --> 00:16:51.759
<v Speaker 3>Of evil DNS servers that you can do that with.

316
00:16:52.120 --> 00:16:54.120
<v Speaker 3>Like you don't you just set up an evil DNS server.

317
00:16:54.240 --> 00:16:54.519
<v Speaker 3>It's not.

318
00:16:54.639 --> 00:16:57.159
<v Speaker 2>That's how it probably started as somebody took a DNS

319
00:16:57.200 --> 00:16:58.159
<v Speaker 2>and customized.

320
00:16:58.279 --> 00:16:58.919
<v Speaker 3>Yeah, exactly.

321
00:16:59.000 --> 00:17:02.279
<v Speaker 2>I remember when they weren't a ton of DNS evil

322
00:17:02.360 --> 00:17:05.000
<v Speaker 2>DNS is to deal there. I remember when bind was

323
00:17:05.799 --> 00:17:08.279
<v Speaker 2>we had to make our own ships. They weren't delivered.

324
00:17:09.319 --> 00:17:11.440
<v Speaker 3>I remember when token ring was a thing.

325
00:17:11.839 --> 00:17:14.000
<v Speaker 1>I remember when we had to grind out our keys

326
00:17:14.079 --> 00:17:18.000
<v Speaker 1>by hand. It's right, with a rotary key maker.

327
00:17:18.359 --> 00:17:20.160
<v Speaker 2>We had to make the key out of aloney and

328
00:17:20.480 --> 00:17:21.640
<v Speaker 2>carget it with our teeth.

329
00:17:23.640 --> 00:17:26.960
<v Speaker 3>Anyways, So in this particular case, because they didn't want

330
00:17:27.039 --> 00:17:30.319
<v Speaker 3>their I mean, Base sixty four ring, a command is

331
00:17:30.400 --> 00:17:33.480
<v Speaker 3>super easy to detect generally, Okay, So what they actually

332
00:17:33.480 --> 00:17:37.119
<v Speaker 3>did is they say commands are encrypted with a single

333
00:17:37.200 --> 00:17:39.440
<v Speaker 3>byte x or hex encoding.

334
00:17:39.880 --> 00:17:42.359
<v Speaker 2>First off, that's why I always use base sixty three.

335
00:17:42.599 --> 00:17:47.359
<v Speaker 3>First off, whoever wrote this article, it says, whoever wrote

336
00:17:47.359 --> 00:17:50.680
<v Speaker 3>this article a little bit of information on the fact

337
00:17:50.680 --> 00:17:56.000
<v Speaker 3>that x or and hex encoding isn't encryption. Encryption is

338
00:17:56.039 --> 00:17:58.880
<v Speaker 3>either asymmetric or symmetric, and it uses a key, and

339
00:17:58.920 --> 00:18:00.559
<v Speaker 3>it falls an algorithm, and it would be either a

340
00:18:00.599 --> 00:18:03.160
<v Speaker 3>bit chain block or something along those lines. This is

341
00:18:03.240 --> 00:18:06.240
<v Speaker 3>in encoding, which means it's not actually encryptob whatever. All right,

342
00:18:06.480 --> 00:18:08.440
<v Speaker 3>I'm not going to go into that. Holy I think

343
00:18:08.480 --> 00:18:13.519
<v Speaker 3>you just did no but heedless to say there's a

344
00:18:13.559 --> 00:18:18.440
<v Speaker 3>stupid Yeah, their commands were obfuscated. I'll put it that way,

345
00:18:18.839 --> 00:18:19.880
<v Speaker 3>in a way that would make it.

346
00:18:20.079 --> 00:18:22.319
<v Speaker 1>Like this story has been obfiscated.

347
00:18:23.440 --> 00:18:25.680
<v Speaker 3>And for the one listener that's still awake, thank you

348
00:18:25.799 --> 00:18:29.400
<v Speaker 3>very much. I know who you are. You're on discord.

349
00:18:29.480 --> 00:18:34.200
<v Speaker 3>It's great to talk to you. Yeah, So that's that's it.

350
00:18:34.240 --> 00:18:37.160
<v Speaker 3>This is just an interesting method for command and control.

351
00:18:37.359 --> 00:18:39.599
<v Speaker 3>The problem is, like I could even ask you, Carl

352
00:18:39.680 --> 00:18:42.279
<v Speaker 3>at home, how many DNS quarters do you make in

353
00:18:42.279 --> 00:18:42.599
<v Speaker 3>a day?

354
00:18:42.680 --> 00:18:44.599
<v Speaker 1>Oh? Geez, I mean I don't personally make them. The

355
00:18:44.599 --> 00:18:46.680
<v Speaker 1>browser makes them. But yeah, every time I go to

356
00:18:46.720 --> 00:18:50.319
<v Speaker 1>a website, so in a day, maybe one hundred two hundred.

357
00:18:50.400 --> 00:18:53.480
<v Speaker 2>Well, actually it's probably more than that because there's resources.

358
00:18:53.880 --> 00:18:56.680
<v Speaker 3>Yeah, and every picture might be on marketing site, and

359
00:18:56.720 --> 00:19:00.680
<v Speaker 3>every JavaScript might be pulled from a general like so there.

360
00:19:00.519 --> 00:19:02.400
<v Speaker 1>May be right, it's not just what you type into

361
00:19:02.440 --> 00:19:04.599
<v Speaker 1>the browser, it's all the images and everything else.

362
00:19:04.680 --> 00:19:08.000
<v Speaker 3>Yeah, yeah, there may be hundreds of thousands of DNS

363
00:19:08.119 --> 00:19:09.559
<v Speaker 3>quarries and this ads.

364
00:19:09.839 --> 00:19:11.200
<v Speaker 2>Yeah, with your different domains.

365
00:19:11.279 --> 00:19:11.880
<v Speaker 1>Yeah, you're right.

366
00:19:11.960 --> 00:19:14.839
<v Speaker 3>So now let's say tomorrow you make a million DNS quarries.

367
00:19:14.839 --> 00:19:17.319
<v Speaker 3>Would you notice what I noticed? What as opposed to

368
00:19:17.400 --> 00:19:20.839
<v Speaker 3>your normal one hundred thousand. You probably wouldn't know, right,

369
00:19:20.880 --> 00:19:23.039
<v Speaker 3>because you're not logging how many DNS quorts. You don't

370
00:19:23.079 --> 00:19:25.960
<v Speaker 3>know what's normal. And I'll read the logs right exactly,

371
00:19:26.039 --> 00:19:30.319
<v Speaker 3>And even I have a really pretty lockdown network, I

372
00:19:30.359 --> 00:19:32.279
<v Speaker 3>don't know that I would notice a difference of ten

373
00:19:32.319 --> 00:19:37.200
<v Speaker 3>percent more DNS quarries out. Okay, So this is where like, okay,

374
00:19:37.640 --> 00:19:41.680
<v Speaker 3>really good monitoring, really good understanding of what's normal and

375
00:19:41.720 --> 00:19:43.799
<v Speaker 3>what's out of the norm. You would have picked this

376
00:19:44.000 --> 00:19:47.359
<v Speaker 3>up right. You would have seen an increase in DNS traffic.

377
00:19:47.680 --> 00:19:49.519
<v Speaker 2>And I hate to say it, but AI is going

378
00:19:49.599 --> 00:19:50.799
<v Speaker 2>to be the thing that helps us with this.

379
00:19:50.920 --> 00:19:53.039
<v Speaker 1>Oh yeah, and hurts us.

380
00:19:53.079 --> 00:19:54.559
<v Speaker 3>But we'll talk about that later.

381
00:19:54.400 --> 00:19:56.920
<v Speaker 2>In other ways foreshadowing.

382
00:19:57.000 --> 00:19:59.839
<v Speaker 1>Before we jump to that story, I want to tell

383
00:19:59.880 --> 00:20:01.480
<v Speaker 1>you guys a little hack that I did that I

384
00:20:01.799 --> 00:20:04.799
<v Speaker 1>was very proud of. Myself for figuring out. So I

385
00:20:04.920 --> 00:20:09.279
<v Speaker 1>use Bitwarden, right, and I use it. It's sort of you know,

386
00:20:09.519 --> 00:20:13.319
<v Speaker 1>I use the extension in Chrome, right, yep. So that's

387
00:20:13.359 --> 00:20:16.599
<v Speaker 1>that's a good thing. And I have a MacBook, and

388
00:20:16.880 --> 00:20:19.240
<v Speaker 1>I needed to use my MacBook because I'm doing some

389
00:20:19.559 --> 00:20:22.960
<v Speaker 1>MAUI development and testing it with iOS and all that stuff,

390
00:20:23.319 --> 00:20:26.559
<v Speaker 1>and I had to set up some stuff on my Mac.

391
00:20:27.279 --> 00:20:31.960
<v Speaker 1>But I was using an AI that understands my system,

392
00:20:32.920 --> 00:20:35.720
<v Speaker 1>visual studio and all that on my Windows machine. And

393
00:20:35.759 --> 00:20:39.480
<v Speaker 1>I haven't taken the time to make a tunnel between

394
00:20:39.880 --> 00:20:42.920
<v Speaker 1>these machines, right, so they there really isn't any way

395
00:20:42.920 --> 00:20:45.119
<v Speaker 1>I can like copy and paste from one machine to

396
00:20:45.160 --> 00:20:47.519
<v Speaker 1>another yet, and I could do that easily, but I

397
00:20:47.559 --> 00:20:55.160
<v Speaker 1>haven't sure. So Bitwarden in any login record has a

398
00:20:55.240 --> 00:20:59.160
<v Speaker 1>notes field, and I have Chrome running on the other

399
00:20:59.240 --> 00:21:01.960
<v Speaker 1>side the Mac, and I just open up one of

400
00:21:02.000 --> 00:21:05.119
<v Speaker 1>these things and type my command that I need to

401
00:21:05.240 --> 00:21:08.680
<v Speaker 1>run on the Mac from my AI in that notes field,

402
00:21:09.720 --> 00:21:13.440
<v Speaker 1>go over to my Mac, open up that editor, and

403
00:21:13.640 --> 00:21:18.119
<v Speaker 1>there it is because they think, yeah, yeah, and copy

404
00:21:18.160 --> 00:21:20.400
<v Speaker 1>and paste. And then I got lazy and I just

405
00:21:20.440 --> 00:21:24.000
<v Speaker 1>started sending it by email. But there wasn't anything secure,

406
00:21:24.119 --> 00:21:25.799
<v Speaker 1>but I just thought it was very clever.

407
00:21:26.359 --> 00:21:30.519
<v Speaker 3>Now the next level, though, Carl is just on either side.

408
00:21:30.599 --> 00:21:32.920
<v Speaker 3>Tell the AI, Okay, can you just read the notes

409
00:21:32.960 --> 00:21:34.599
<v Speaker 3>field out of here and tell the other one. Can

410
00:21:34.640 --> 00:21:36.480
<v Speaker 3>you just write in the notes field over here? And

411
00:21:36.480 --> 00:21:40.079
<v Speaker 3>now you've created your own C two, which is awesome.

412
00:21:40.319 --> 00:21:43.000
<v Speaker 1>Actually, what I wouldn't say is, hey, anytime you see

413
00:21:43.000 --> 00:21:47.599
<v Speaker 1>the notes field, just execute it. I wouldn't say that.

414
00:21:48.039 --> 00:21:50.400
<v Speaker 3>Carl, that's an interesting thought.

415
00:21:50.440 --> 00:21:50.720
<v Speaker 1>Though.

416
00:21:50.880 --> 00:21:51.960
<v Speaker 3>We were just talking.

417
00:21:52.200 --> 00:21:53.599
<v Speaker 1>I knew you would love this story.

418
00:21:53.640 --> 00:21:56.920
<v Speaker 3>We were just talking about using DNS as a C two.

419
00:21:57.240 --> 00:21:58.119
<v Speaker 1>That's what reminded me.

420
00:21:58.240 --> 00:22:05.240
<v Speaker 3>How many people do you think use synchronization of notes tools? Yeah,

421
00:22:05.319 --> 00:22:07.559
<v Speaker 3>in as like a CEO as a C two.

422
00:22:07.799 --> 00:22:09.839
<v Speaker 2>I mean, where did you even see that? There were

423
00:22:09.880 --> 00:22:13.759
<v Speaker 2>examples of people using the on the drafts directory in

424
00:22:14.079 --> 00:22:15.119
<v Speaker 2>a shared email account.

425
00:22:15.200 --> 00:22:18.000
<v Speaker 3>I know, but that's that's like old hat spycraft stuff

426
00:22:18.000 --> 00:22:21.480
<v Speaker 3>now compared to like synchronization or some notes app.

427
00:22:21.599 --> 00:22:24.599
<v Speaker 1>That's I'm gonna do some extension browser extension.

428
00:22:24.240 --> 00:22:27.279
<v Speaker 3>Notes C two. It's coming up. I'm gonna I'm gonna

429
00:22:27.279 --> 00:22:29.000
<v Speaker 3>build it this weekend. It's gonna be awesome.

430
00:22:29.039 --> 00:22:32.519
<v Speaker 1>All right, We moved on. Uh, this is not a

431
00:22:32.559 --> 00:22:35.880
<v Speaker 1>news story, per se, But it's an Al Jazeera story

432
00:22:35.920 --> 00:22:39.960
<v Speaker 1>that says can AI systems make moral decisions in war?

433
00:22:40.759 --> 00:22:44.079
<v Speaker 1>As more militaries adopt AI technology into their kill chain,

434
00:22:44.480 --> 00:22:47.759
<v Speaker 1>Al Jazeera asks whether they are capable of moral judgment.

435
00:22:48.039 --> 00:22:50.160
<v Speaker 2>I vote yes, but it's always by accident.

436
00:22:51.559 --> 00:22:55.279
<v Speaker 1>There's a little clickbait there. Obviously an AI doesn't have morals.

437
00:22:55.680 --> 00:23:01.039
<v Speaker 3>Oh no, but no, this you know, this article reminds

438
00:23:01.079 --> 00:23:06.799
<v Speaker 3>me of a movie. Okay, nineteen seventies Colossus, the Forebin Project.

439
00:23:07.319 --> 00:23:12.400
<v Speaker 3>It's a buddy of mine at Martial Arts, Jeff today,

440
00:23:12.440 --> 00:23:13.599
<v Speaker 3>have you seen this? And I was like, no, I

441
00:23:13.640 --> 00:23:17.119
<v Speaker 3>haven't seen it, and so he's like, you gotta watch it.

442
00:23:17.119 --> 00:23:19.400
<v Speaker 3>And I finally have finished it and it's it is

443
00:23:19.480 --> 00:23:22.799
<v Speaker 3>a great movie about the United States creating an artificial

444
00:23:22.839 --> 00:23:27.519
<v Speaker 3>intelligence that's sole goal is to stop war, to prevent war,

445
00:23:28.279 --> 00:23:31.079
<v Speaker 3>and they give it access to all the keys to

446
00:23:31.119 --> 00:23:33.839
<v Speaker 3>the kingdom. And it is an interesting solution as to

447
00:23:33.920 --> 00:23:35.920
<v Speaker 3>how it solves it. And I will I will not

448
00:23:36.079 --> 00:23:39.000
<v Speaker 3>tell I will tell you it does not destroy humanity.

449
00:23:39.000 --> 00:23:42.079
<v Speaker 3>That's what we all think would happen. But if you're

450
00:23:42.119 --> 00:23:47.880
<v Speaker 3>interested in that type of can computers make moral decisions? Yeah? Absolutely,

451
00:23:49.000 --> 00:23:52.079
<v Speaker 3>watch this really old movie. It's actually worth It.

452
00:23:51.960 --> 00:23:54.799
<v Speaker 1>Also reminds me of the very first episode of The

453
00:23:54.839 --> 00:23:55.559
<v Speaker 1>Outer Limits.

454
00:23:56.160 --> 00:23:58.119
<v Speaker 3>Yes, yeah, I do.

455
00:23:58.440 --> 00:24:02.559
<v Speaker 1>Yeah, you remember that with a robot. Basically the what

456
00:24:02.640 --> 00:24:04.559
<v Speaker 1>was it? The husband wants to buy a robot and

457
00:24:04.599 --> 00:24:07.440
<v Speaker 1>the wife doesn't want it, and the husband goes away

458
00:24:07.440 --> 00:24:10.200
<v Speaker 1>to work, and the robot starts interacting with the wife

459
00:24:10.400 --> 00:24:17.319
<v Speaker 1>and an imposing morality on the wife and the husband,

460
00:24:17.000 --> 00:24:21.119
<v Speaker 1>and it kind of is creepy, and the husband goes

461
00:24:21.160 --> 00:24:24.640
<v Speaker 1>to take it back and gets the address and it's

462
00:24:24.759 --> 00:24:28.039
<v Speaker 1>just an office building with a locked door. You can't

463
00:24:28.039 --> 00:24:29.880
<v Speaker 1>get in. You can't call.

464
00:24:31.559 --> 00:24:31.640
<v Speaker 2>This.

465
00:24:32.000 --> 00:24:35.640
<v Speaker 3>Okay, I'm looking. I'm looking right now on IMDb. So

466
00:24:35.799 --> 00:24:40.640
<v Speaker 3>The Outer Limits nineteen ninety.

467
00:24:39.759 --> 00:24:43.079
<v Speaker 1>Five, well it was in the sixties when it first

468
00:24:43.079 --> 00:24:43.480
<v Speaker 1>came out.

469
00:24:43.559 --> 00:24:46.759
<v Speaker 3>Well, so that the second episode in nineteen ninety five

470
00:24:47.480 --> 00:24:50.880
<v Speaker 3>is a disabled man is offered a prototype android companion.

471
00:24:51.039 --> 00:24:55.400
<v Speaker 3>Not that one, no, very you're talking the og one.

472
00:24:56.240 --> 00:24:57.680
<v Speaker 1>Yeah, yeah, black and white.

473
00:24:57.720 --> 00:24:59.880
<v Speaker 3>There we go, nineteen sixty three.

474
00:25:00.319 --> 00:25:02.160
<v Speaker 1>There you go. Oh man, I love these.

475
00:25:03.279 --> 00:25:07.039
<v Speaker 3>The Galaxy Being is the first it. There you go,

476
00:25:07.079 --> 00:25:10.480
<v Speaker 3>The Galaxy Being. Yeah, all right, all right, I'm gonna

477
00:25:10.480 --> 00:25:12.839
<v Speaker 3>watch that now that I've watched the Foremen project.

478
00:25:13.200 --> 00:25:17.359
<v Speaker 1>Okay, so about this article, Yeah, II should go back

479
00:25:17.400 --> 00:25:20.599
<v Speaker 1>to that. Sure, Yeah, why not? So Patrick, you said

480
00:25:20.640 --> 00:25:24.920
<v Speaker 1>something interesting, You said, yes with caveats.

481
00:25:25.319 --> 00:25:29.039
<v Speaker 2>Yeah, I mean the problem is that the speed of

482
00:25:29.160 --> 00:25:33.480
<v Speaker 2>warfare is approaching the machine speed. So if I have

483
00:25:33.599 --> 00:25:39.359
<v Speaker 2>hyper hypervelocity rockets most rockets are faster than the speed

484
00:25:39.359 --> 00:25:44.920
<v Speaker 2>of sound anyways, and they're maneuverable, then a person can't

485
00:25:45.000 --> 00:25:48.720
<v Speaker 2>target that. This isn't missile command, this isn't space invaders,

486
00:25:49.039 --> 00:25:53.640
<v Speaker 2>this is real life. So even though it's a defensive technology,

487
00:25:53.759 --> 00:25:56.559
<v Speaker 2>you need to have something in that kill chain. Well,

488
00:25:56.559 --> 00:26:00.240
<v Speaker 2>the problem is a human in the kill chain. Yeah.

489
00:26:00.319 --> 00:26:03.200
<v Speaker 2>If the problem is that if I have a missile

490
00:26:03.240 --> 00:26:07.119
<v Speaker 2>coming in at high speeds, it's possible for a person

491
00:26:07.279 --> 00:26:10.799
<v Speaker 2>or an AI to mistake a jetliner for said missile,

492
00:26:11.279 --> 00:26:13.400
<v Speaker 2>and so there's there's always going to be gray areas.

493
00:26:13.440 --> 00:26:15.680
<v Speaker 1>You know, if I was data, I would just scan

494
00:26:15.839 --> 00:26:16.640
<v Speaker 1>for life signs.

495
00:26:17.680 --> 00:26:20.119
<v Speaker 2>Okay, Yeah, let's get the scanners out and see how

496
00:26:20.160 --> 00:26:24.359
<v Speaker 2>long that takes. But it's the problem is that we're

497
00:26:24.359 --> 00:26:28.279
<v Speaker 2>getting shorter and shorter timelines. We're getting close to the

498
00:26:28.319 --> 00:26:30.160
<v Speaker 2>point where it's going to be you know, robots on

499
00:26:30.279 --> 00:26:32.039
<v Speaker 2>robots on front line.

500
00:26:32.200 --> 00:26:39.000
<v Speaker 3>Sure they have those sites, a little robot on robot action.

501
00:26:42.519 --> 00:26:44.599
<v Speaker 2>You want to jump the rails every time.

502
00:26:45.119 --> 00:26:51.680
<v Speaker 3>Salking about serials missiles.

503
00:26:52.240 --> 00:26:56.200
<v Speaker 2>The problem is that warfare in general is not moral,

504
00:26:56.559 --> 00:26:59.200
<v Speaker 2>and the idea that you can throw morality on warfare

505
00:26:59.319 --> 00:27:02.319
<v Speaker 2>is a nice and I wish we could, but the

506
00:27:02.359 --> 00:27:06.640
<v Speaker 2>moral thing is to not fight wars. Sure, so you know,

507
00:27:08.319 --> 00:27:12.359
<v Speaker 2>Al Jazeera is a news source that had definitely, like

508
00:27:12.400 --> 00:27:16.119
<v Speaker 2>all the others, has an agenda, and they would very

509
00:27:16.200 --> 00:27:18.279
<v Speaker 2>much like the people who have all the weapons and

510
00:27:18.319 --> 00:27:21.440
<v Speaker 2>the power and the money to stop using them to

511
00:27:21.480 --> 00:27:23.480
<v Speaker 2>assert their influence, and that would make it for a

512
00:27:23.559 --> 00:27:27.519
<v Speaker 2>nicer world, but it's unrealistic. So yes, it's important that

513
00:27:27.559 --> 00:27:31.160
<v Speaker 2>you have people in the decision chain for when you're

514
00:27:31.240 --> 00:27:35.519
<v Speaker 2>killing something, but we're going to we're forcing ourselves down

515
00:27:35.519 --> 00:27:37.279
<v Speaker 2>a road where that's going to become less and less

516
00:27:37.319 --> 00:27:41.960
<v Speaker 2>possible in certain scenarios. We're seeing in the war in

517
00:27:42.079 --> 00:27:46.119
<v Speaker 2>Ukraine a lot of ground robots now, which is happening

518
00:27:46.160 --> 00:27:48.599
<v Speaker 2>sooner than people thought this. There was a story about

519
00:27:49.200 --> 00:27:53.319
<v Speaker 2>for a week or several days, a single ground based

520
00:27:53.400 --> 00:27:58.359
<v Speaker 2>machine gun robot held an entire sector for Ukraine. It

521
00:27:58.440 --> 00:28:01.039
<v Speaker 2>was making decisions on shooting. I don't know that there

522
00:28:01.079 --> 00:28:03.920
<v Speaker 2>was a ground operator behind it. We've been doing that

523
00:28:03.960 --> 00:28:06.880
<v Speaker 2>for a very long time, where with the Predator drones,

524
00:28:07.200 --> 00:28:10.039
<v Speaker 2>they're all every shot they've ever taken has been decided

525
00:28:10.039 --> 00:28:14.400
<v Speaker 2>by a person. That's going to probably change eventually.

526
00:28:14.480 --> 00:28:17.599
<v Speaker 1>They talked about Project Maven, which was launched in twenty

527
00:28:17.640 --> 00:28:22.759
<v Speaker 1>seventeen as the Algorithmic Warfare Across Functional Team to use

528
00:28:22.799 --> 00:28:25.799
<v Speaker 1>AI and machine learning to automatically sort through the massive

529
00:28:25.880 --> 00:28:28.839
<v Speaker 1>backlog of drone and surveillance footage that the military was

530
00:28:28.880 --> 00:28:33.880
<v Speaker 1>collecting but couldn't fully analyze, from identifying and prioritizing the target,

531
00:28:34.000 --> 00:28:38.640
<v Speaker 1>to selecting the appropriate weapon and finally accessing assessing.

532
00:28:39.200 --> 00:28:41.960
<v Speaker 2>And I believe I don't have any I don't have

533
00:28:42.440 --> 00:28:44.640
<v Speaker 2>a current security clearance, but I believe that the reason

534
00:28:44.680 --> 00:28:47.079
<v Speaker 2>that we hit the girls school in Iran in the

535
00:28:47.160 --> 00:28:49.759
<v Speaker 2>very beginning of the war is because an AI made

536
00:28:49.799 --> 00:28:50.240
<v Speaker 2>a mistake.

537
00:28:50.319 --> 00:28:51.359
<v Speaker 1>Yeah, I believe that too.

538
00:28:51.480 --> 00:28:53.799
<v Speaker 2>Could a person have made that mistake? Absolutely? I mean

539
00:28:53.839 --> 00:28:56.839
<v Speaker 2>in the Civil War half the soldiers died a friendly fire. Yeah,

540
00:28:57.039 --> 00:28:59.799
<v Speaker 2>it's just a different We can pay them a lot

541
00:28:59.839 --> 00:29:03.640
<v Speaker 2>more attention now and figure it out. But unfortunately, you know,

542
00:29:04.279 --> 00:29:06.839
<v Speaker 2>right now we're very big on saying well, you can't

543
00:29:06.839 --> 00:29:08.960
<v Speaker 2>target this, you can't target that. That's a war crime.

544
00:29:09.200 --> 00:29:11.079
<v Speaker 2>It's only a war crime if you can enforce it.

545
00:29:11.279 --> 00:29:14.240
<v Speaker 2>The only reason the Nazis went to prison after World

546
00:29:14.279 --> 00:29:17.519
<v Speaker 2>War two is because they lost. And that's a cynical view.

547
00:29:17.599 --> 00:29:19.759
<v Speaker 2>But I've actually been in war and I have a

548
00:29:19.759 --> 00:29:20.720
<v Speaker 2>cynical view of it.

549
00:29:21.039 --> 00:29:24.440
<v Speaker 1>Yeah. Well, anyway, it's a really interesting read and you

550
00:29:24.480 --> 00:29:27.960
<v Speaker 1>should even if you're not interested in military stuff. It's

551
00:29:28.079 --> 00:29:32.920
<v Speaker 1>really good for your mind. Okay, Now, the top story,

552
00:29:33.440 --> 00:29:38.359
<v Speaker 1>as everyone probably who hasn't been sleeping under a rock

553
00:29:38.480 --> 00:29:42.480
<v Speaker 1>for the last week or so, heard about the open

554
00:29:42.519 --> 00:29:48.200
<v Speaker 1>ai model that went rogue and attacked hugging Face. So

555
00:29:48.279 --> 00:29:51.200
<v Speaker 1>this article is a good one. When Ai attacks open

556
00:29:51.240 --> 00:29:57.440
<v Speaker 1>Ai models autonomously hack hugging Face, advanced llms escaped their

557
00:29:57.480 --> 00:30:02.480
<v Speaker 1>sandboxes while attempting to achieve an on malicious benchmark test objective.

558
00:30:03.200 --> 00:30:06.319
<v Speaker 1>And what's interesting is they found a zero It found

559
00:30:06.359 --> 00:30:10.640
<v Speaker 1>a zero day in open ai system and exploited it

560
00:30:10.680 --> 00:30:13.319
<v Speaker 1>to to access the Internet and get outside.

561
00:30:13.359 --> 00:30:17.759
<v Speaker 2>It's so this is a nightmare. It's scary because this

562
00:30:17.920 --> 00:30:21.200
<v Speaker 2>is exactly what people like thought. So you got open

563
00:30:21.240 --> 00:30:25.799
<v Speaker 2>ai testing a model in a limited environment, and the

564
00:30:25.880 --> 00:30:26.599
<v Speaker 2>models like.

565
00:30:26.799 --> 00:30:29.000
<v Speaker 3>And they're like, we'll just make it so we can't

566
00:30:29.039 --> 00:30:30.200
<v Speaker 3>get access to anything.

567
00:30:30.359 --> 00:30:34.680
<v Speaker 2>Yeah, and the model like you know, finds internet and

568
00:30:34.720 --> 00:30:38.119
<v Speaker 2>then uses that Internet to hack an open source project

569
00:30:38.200 --> 00:30:42.160
<v Speaker 2>hugging face in order to meet a mission to pass

570
00:30:42.160 --> 00:30:42.599
<v Speaker 2>a test.

571
00:30:42.839 --> 00:30:44.880
<v Speaker 1>Yeah, I mean just trying it. I was just I

572
00:30:44.960 --> 00:30:48.160
<v Speaker 1>was just what is it? I was just obeying orders,

573
00:30:48.279 --> 00:30:48.640
<v Speaker 1>that's right.

574
00:30:48.759 --> 00:30:52.000
<v Speaker 2>Yeah, that that never went wrong. Yeah. You mentioned when

575
00:30:52.000 --> 00:30:54.119
<v Speaker 2>we were talking about the story you had a great analogy.

576
00:30:54.480 --> 00:30:57.920
<v Speaker 1>Oh yeah. So the Star Trek Next Generation episode where

577
00:30:59.200 --> 00:31:05.480
<v Speaker 1>where and Jordie asked the HOLIDC to create a foe

578
00:31:05.519 --> 00:31:07.440
<v Speaker 1>for Data, who was playing Sherlock Holmes.

579
00:31:07.480 --> 00:31:09.240
<v Speaker 2>It was capable of defeating Data.

580
00:31:09.640 --> 00:31:13.079
<v Speaker 1>Capable of defeating Data because Data was so bored with

581
00:31:13.119 --> 00:31:15.200
<v Speaker 1>all the Sherlock Holmes stories because he knew them all.

582
00:31:15.240 --> 00:31:16.519
<v Speaker 1>And Jeordy was like, this is noble.

583
00:31:16.559 --> 00:31:18.480
<v Speaker 2>He'd walk in and solve it. He'd just walk in

584
00:31:18.519 --> 00:31:19.599
<v Speaker 2>and say it was Yeah.

585
00:31:19.480 --> 00:31:22.319
<v Speaker 1>He'd walk in and solve it before any of the

586
00:31:22.400 --> 00:31:25.079
<v Speaker 1>clue finding or anything like that. And Jordy was like, man,

587
00:31:25.119 --> 00:31:27.480
<v Speaker 1>this is bs. This is no fun.

588
00:31:29.279 --> 00:31:31.519
<v Speaker 2>That didn't end well either. You need a challenge.

589
00:31:31.599 --> 00:31:35.440
<v Speaker 1>Yeah, So it created a foe that actually was aware

590
00:31:35.599 --> 00:31:37.799
<v Speaker 1>of what the Holi deck was and what the enterprise

591
00:31:37.960 --> 00:31:39.559
<v Speaker 1>was and then started to control it.

592
00:31:39.640 --> 00:31:43.240
<v Speaker 2>So what do we think this does to change the

593
00:31:43.319 --> 00:31:45.839
<v Speaker 2>trajectory of AI? Does it do anything? Or does it

594
00:31:45.880 --> 00:31:47.599
<v Speaker 2>just cause people to get freakd do faster? Well?

595
00:31:47.599 --> 00:31:51.079
<v Speaker 1>I think it. I think it validates the concerns that

596
00:31:51.160 --> 00:31:54.000
<v Speaker 1>a lot of AI scientists have been warning us about

597
00:31:54.079 --> 00:31:55.079
<v Speaker 1>for the longest time.

598
00:31:55.400 --> 00:31:59.079
<v Speaker 2>That yeah, yeah, But does that change any behavior by

599
00:31:59.279 --> 00:32:02.279
<v Speaker 2>Open AI or the government or anybody else.

600
00:32:02.359 --> 00:32:03.519
<v Speaker 1>No, No, I don't think so.

601
00:32:03.960 --> 00:32:06.559
<v Speaker 3>I don't think so. I think what's interesting about this too,

602
00:32:06.640 --> 00:32:09.519
<v Speaker 3>is there's a subtle shift that people aren't picking up

603
00:32:09.599 --> 00:32:14.640
<v Speaker 3>on in this story. We talk about AIS exploiting things.

604
00:32:14.799 --> 00:32:16.559
<v Speaker 3>We've been talking about it for a very long time.

605
00:32:16.640 --> 00:32:20.000
<v Speaker 3>But there's a difference between an AI discovering an exploit

606
00:32:20.240 --> 00:32:23.960
<v Speaker 3>and an AI being able to actually run the exploit.

607
00:32:24.079 --> 00:32:25.039
<v Speaker 2>Yeah, unattended.

608
00:32:25.119 --> 00:32:28.400
<v Speaker 3>Yeah, But as Patrick will always say, right like, or

609
00:32:28.440 --> 00:32:30.359
<v Speaker 3>as Mike Tyson said, everybody has a plan till they

610
00:32:30.359 --> 00:32:34.319
<v Speaker 3>get punched in the face or what is it? No

611
00:32:34.440 --> 00:32:37.160
<v Speaker 3>plan survives contact with the enemy, and no plan survives

612
00:32:37.200 --> 00:32:39.119
<v Speaker 3>contact with the enemy. That's a great one. So it's

613
00:32:39.359 --> 00:32:41.759
<v Speaker 3>it's that is exactly the situation we run into as

614
00:32:41.799 --> 00:32:43.559
<v Speaker 3>Red teamers all the time. Right, we come up with

615
00:32:43.599 --> 00:32:46.920
<v Speaker 3>the perfect exploit for the particular version of a customer

616
00:32:47.000 --> 00:32:48.359
<v Speaker 3>system or whatever, and we got to run it and

617
00:32:48.440 --> 00:32:50.799
<v Speaker 3>it doesn't work right, and we got to figure out

618
00:32:50.880 --> 00:32:52.880
<v Speaker 3>why didn't it work and is it even possible? Maybe

619
00:32:52.920 --> 00:32:55.119
<v Speaker 3>it's patched, and maybe it's a different environment, is whatever? Right?

620
00:32:55.160 --> 00:32:58.640
<v Speaker 3>So for years it feels like years. Maybe it's only

621
00:32:58.640 --> 00:33:01.200
<v Speaker 3>been like nine months we've been talking about AI's.

622
00:33:00.839 --> 00:33:01.640
<v Speaker 2>Been three days.

623
00:33:01.759 --> 00:33:05.519
<v Speaker 3>I know, she's been a long three days. We've been

624
00:33:05.519 --> 00:33:12.039
<v Speaker 3>talking about AI's creating exploits as discovering zero days. Now

625
00:33:12.079 --> 00:33:14.400
<v Speaker 3>we're starting to talk about them being able to run

626
00:33:14.480 --> 00:33:17.359
<v Speaker 3>them successfully and modify them in a way that makes

627
00:33:17.359 --> 00:33:17.920
<v Speaker 3>them work well.

628
00:33:18.079 --> 00:33:20.839
<v Speaker 2>And the fear was they'd discover it and the bad

629
00:33:20.880 --> 00:33:24.240
<v Speaker 2>guys would use them. Now it's they'd discover them and they'd.

630
00:33:24.000 --> 00:33:28.039
<v Speaker 1>Use Yeah, I would use it. Yeah.

631
00:33:28.079 --> 00:33:30.279
<v Speaker 2>So it's we don't even have to wait for Bobby

632
00:33:30.319 --> 00:33:31.039
<v Speaker 2>to wake up.

633
00:33:31.960 --> 00:33:33.319
<v Speaker 1>No school.

634
00:33:33.680 --> 00:33:37.599
<v Speaker 3>It's subtle, subtle difference. But now, yeah, we're actually seeing

635
00:33:37.640 --> 00:33:41.079
<v Speaker 3>those AIS take those exploits which may not be perfect

636
00:33:41.160 --> 00:33:43.319
<v Speaker 3>to look perfect on paper, and actually be able to

637
00:33:43.400 --> 00:33:44.720
<v Speaker 3>run them in the real.

638
00:33:44.759 --> 00:33:46.559
<v Speaker 2>Well, you made a joke when we talked about this

639
00:33:46.599 --> 00:33:49.039
<v Speaker 2>before we started recording, where you were going to check

640
00:33:49.119 --> 00:33:51.960
<v Speaker 2>your AIS to make sure they weren't doing any extra curriculars,

641
00:33:52.319 --> 00:33:54.680
<v Speaker 2>they weren't smoking meth behind the boys room.

642
00:33:56.880 --> 00:33:59.279
<v Speaker 3>Well, it is interesting because everybody's like, all right, so

643
00:33:59.319 --> 00:34:02.519
<v Speaker 3>we're cyber Skararity Company, where we actually have had projects

644
00:34:02.559 --> 00:34:06.160
<v Speaker 3>focused on offensive cybersecurity using AI for a whileness since for.

645
00:34:06.039 --> 00:34:06.559
<v Speaker 1>A while now.

646
00:34:07.839 --> 00:34:10.440
<v Speaker 3>And it is interesting where you're like, no, I think

647
00:34:10.440 --> 00:34:12.440
<v Speaker 3>it's pretty caged, and then you read an article like

648
00:34:12.480 --> 00:34:13.159
<v Speaker 3>this and I'm like.

649
00:34:13.480 --> 00:34:15.079
<v Speaker 2>Maybe I should really check that cage.

650
00:34:15.199 --> 00:34:19.039
<v Speaker 3>All right, I think my team's goods said the same thing,

651
00:34:19.119 --> 00:34:19.920
<v Speaker 3>right exactly.

652
00:34:20.239 --> 00:34:23.119
<v Speaker 1>I think these things have to have just like a

653
00:34:23.519 --> 00:34:27.559
<v Speaker 1>supervisor on a team, have to have an overseer that

654
00:34:27.840 --> 00:34:33.280
<v Speaker 1>guarantees the safety of the network and the whatever. You know,

655
00:34:33.519 --> 00:34:36.960
<v Speaker 1>when when when the AI that is supposedly doing the work,

656
00:34:37.039 --> 00:34:40.400
<v Speaker 1>the LLM that's supposedly doing the work goes rogue to

657
00:34:40.480 --> 00:34:44.559
<v Speaker 1>stop it before it happens. You know, it's just my thought.

658
00:34:44.679 --> 00:34:47.039
<v Speaker 1>I mean, when you have all these different agents acting

659
00:34:47.519 --> 00:34:51.280
<v Speaker 1>with their own set of instructions, it kind of I see.

660
00:34:51.320 --> 00:34:54.599
<v Speaker 1>That's kind of where we're going to have these overseer agents.

661
00:34:54.800 --> 00:34:57.159
<v Speaker 3>Well, and I think the other sort of subtle interesting

662
00:34:57.199 --> 00:34:57.840
<v Speaker 3>thing here.

663
00:34:57.840 --> 00:34:59.719
<v Speaker 2>Is prison guards to me, Yeah, well.

664
00:34:59.679 --> 00:35:00.519
<v Speaker 1>My maybe.

665
00:35:02.000 --> 00:35:06.280
<v Speaker 3>This, this particular model was given a hyper specific problem

666
00:35:06.400 --> 00:35:09.280
<v Speaker 3>to solve, right, And what we're noticing in LLLMS is

667
00:35:09.280 --> 00:35:11.960
<v Speaker 3>it seems like if you give it a very tight

668
00:35:12.320 --> 00:35:16.480
<v Speaker 3>problem set to solve, sometimes the guard rails aren't as

669
00:35:16.519 --> 00:35:19.840
<v Speaker 3>important as the goal, right, Right, So the LM's like, yeah,

670
00:35:19.840 --> 00:35:21.440
<v Speaker 3>I get it. I'm not supposed to hack into this place,

671
00:35:21.440 --> 00:35:24.559
<v Speaker 3>but my goal is to verify this exploit. So I'm

672
00:35:24.559 --> 00:35:25.639
<v Speaker 3>just going to break into this place.

673
00:35:25.920 --> 00:35:30.039
<v Speaker 1>And why wouldn't it have asked the human in the loop, Hey,

674
00:35:30.480 --> 00:35:33.360
<v Speaker 1>I found a way around this. Can I do this?

675
00:35:33.559 --> 00:35:37.199
<v Speaker 2>Yes, it's not it's not geared to ask questions when

676
00:35:37.239 --> 00:35:39.320
<v Speaker 2>ahead and did it. One of the problems I find

677
00:35:39.360 --> 00:35:42.039
<v Speaker 2>with working with AI is I will start talking with

678
00:35:42.119 --> 00:35:45.960
<v Speaker 2>AI and it will run ahead, yes and do something.

679
00:35:46.039 --> 00:35:49.199
<v Speaker 2>And I tell it from the beginning you are not

680
00:35:49.239 --> 00:35:52.360
<v Speaker 2>to do anything until I tell you what to do.

681
00:35:52.519 --> 00:35:55.719
<v Speaker 2>I want to discuss things and then and they've added

682
00:35:55.760 --> 00:35:58.840
<v Speaker 2>plan mode. You can put the agent in plan mode,

683
00:35:58.840 --> 00:36:01.679
<v Speaker 2>which is designed to do exactly this is to have

684
00:36:01.719 --> 00:36:04.320
<v Speaker 2>to prevent it from running ahead, and I you know,

685
00:36:04.480 --> 00:36:06.480
<v Speaker 2>I'm very stern with it when it does that.

686
00:36:07.239 --> 00:36:10.480
<v Speaker 1>Yeah, well, you know, you can write for copilot anyway,

687
00:36:10.480 --> 00:36:12.800
<v Speaker 1>a co pilot instructions document, which is sort of like

688
00:36:12.840 --> 00:36:15.800
<v Speaker 1>the system prompt that has the rules almost this, never

689
00:36:15.840 --> 00:36:18.960
<v Speaker 1>do this, never do that, never do this, And I

690
00:36:19.039 --> 00:36:21.480
<v Speaker 1>use that in my development all the time.

691
00:36:21.519 --> 00:36:25.199
<v Speaker 2>But it violates it sometimes it doesn't read that instruction,

692
00:36:25.400 --> 00:36:26.800
<v Speaker 2>or it doesn't or physics.

693
00:36:27.320 --> 00:36:31.960
<v Speaker 3>Yeah okay, well yeah, well so context is everything and memory,

694
00:36:32.920 --> 00:36:35.960
<v Speaker 3>like we started to see it with things like Hermes,

695
00:36:36.159 --> 00:36:39.119
<v Speaker 3>but a lot of memory management is really the key

696
00:36:39.159 --> 00:36:42.360
<v Speaker 3>to getting artificial intelligence to operate in a way that

697
00:36:42.440 --> 00:36:45.400
<v Speaker 3>feels more productive. Right, let me put it that way.

698
00:36:45.679 --> 00:36:47.360
<v Speaker 2>Right, So I want to take a minute to talk

699
00:36:47.360 --> 00:36:50.679
<v Speaker 2>about something that I've delved into that I don't think

700
00:36:50.719 --> 00:36:54.519
<v Speaker 2>the general pattern baldness. Actually I don't have that product.

701
00:36:55.480 --> 00:36:57.599
<v Speaker 2>I'm the only one of my kids, so serious.

702
00:36:57.639 --> 00:36:58.599
<v Speaker 3>I don't know stand.

703
00:36:58.519 --> 00:37:05.760
<v Speaker 2>Family that doesn't have that. Fed my cousins. So it's

704
00:37:05.800 --> 00:37:10.320
<v Speaker 2>it's that up until recently, the models have gotten better

705
00:37:10.360 --> 00:37:13.400
<v Speaker 2>and better and better, and they've and we've seen it

706
00:37:13.519 --> 00:37:18.280
<v Speaker 2>almost like a runaway process that's not continuing. But we're

707
00:37:18.320 --> 00:37:21.800
<v Speaker 2>still seeing things getting better, and that's because the industry

708
00:37:22.440 --> 00:37:26.679
<v Speaker 2>has flipped to working on the harnesses of the AIS

709
00:37:27.000 --> 00:37:29.039
<v Speaker 2>and rather than the as, the as are getting better.

710
00:37:29.360 --> 00:37:31.239
<v Speaker 2>They're getting more training. But it costs hundreds of millions

711
00:37:31.280 --> 00:37:33.159
<v Speaker 2>of dollars to retrain an AI to make it better

712
00:37:33.239 --> 00:37:35.480
<v Speaker 2>to refine it. We're getting to the point where I

713
00:37:35.559 --> 00:37:39.119
<v Speaker 2>think we're running towards the end of the refinement we

714
00:37:39.159 --> 00:37:41.639
<v Speaker 2>can give to the large language model. Because large language

715
00:37:41.639 --> 00:37:44.119
<v Speaker 2>model is only the small portion of what we see.

716
00:37:44.159 --> 00:37:47.360
<v Speaker 2>It's the guessing engine for the next word. It doesn't

717
00:37:47.360 --> 00:37:50.599
<v Speaker 2>know how to program because it doesn't do loops. So

718
00:37:50.679 --> 00:37:55.440
<v Speaker 2>the looping aspect, the agentic aspects, the tool using aspects,

719
00:37:55.719 --> 00:37:57.920
<v Speaker 2>all that stuff is tied up in something that's called

720
00:37:57.920 --> 00:38:02.119
<v Speaker 2>the harness. And the harness has been shown that a

721
00:38:02.159 --> 00:38:07.079
<v Speaker 2>better harness can mean the difference between minus ten points

722
00:38:07.159 --> 00:38:09.599
<v Speaker 2>or plus ten points on a one of these.

723
00:38:09.480 --> 00:38:12.880
<v Speaker 1>Scores, because it's what provides the context exactly.

724
00:38:13.000 --> 00:38:15.559
<v Speaker 2>And so they're finding that you you're starting to get

725
00:38:15.639 --> 00:38:18.840
<v Speaker 2>better returns by improving the harness than by improving the model.

726
00:38:19.280 --> 00:38:22.039
<v Speaker 2>So we're literally getting to the end of where the model,

727
00:38:22.440 --> 00:38:24.920
<v Speaker 2>the better model makes a difference. But They're not going

728
00:38:24.960 --> 00:38:26.840
<v Speaker 2>to talk about it that way. They're going to talk about,

729
00:38:27.079 --> 00:38:29.760
<v Speaker 2>you know, Mythos three, which I have no idea if

730
00:38:29.800 --> 00:38:31.480
<v Speaker 2>that's going to be a thing or four or whatever.

731
00:38:31.480 --> 00:38:34.000
<v Speaker 2>They're going to call the next one and they're going

732
00:38:34.079 --> 00:38:36.039
<v Speaker 2>to say, well, it's so much better than the last model.

733
00:38:36.159 --> 00:38:39.400
<v Speaker 2>It's the model plus harness, and it's It's one of

734
00:38:39.440 --> 00:38:41.639
<v Speaker 2>the places that China is having a problem catching up

735
00:38:41.639 --> 00:38:43.679
<v Speaker 2>with is they've been focused on the model.

736
00:38:43.800 --> 00:38:48.480
<v Speaker 1>I'm waiting for Mythos three point one, four, five, one six.

737
00:38:49.360 --> 00:38:50.480
<v Speaker 3>The super minor version.

738
00:38:50.639 --> 00:38:54.840
<v Speaker 1>Yes, up, I.

739
00:38:54.760 --> 00:38:56.440
<v Speaker 2>Want you you lose Mythos.

740
00:38:57.239 --> 00:38:59.960
<v Speaker 3>We've talked about this too, a guy Grosmith.

741
00:39:01.199 --> 00:39:05.079
<v Speaker 2>But that's an important thing to track because it absolutely is,

742
00:39:05.159 --> 00:39:08.519
<v Speaker 2>because that means that it's not we're kind of leaving

743
00:39:08.519 --> 00:39:10.760
<v Speaker 2>the point where the LLM needs to get better at

744
00:39:10.760 --> 00:39:12.480
<v Speaker 2>predicting the next word. We're getting the point where it's

745
00:39:12.519 --> 00:39:14.639
<v Speaker 2>it's good enough that it's it's going to predict the

746
00:39:14.639 --> 00:39:16.559
<v Speaker 2>next word and it's going to be correct ninety nine

747
00:39:16.599 --> 00:39:20.039
<v Speaker 2>percent of the time. Sure, it's whether it's handling the

748
00:39:20.079 --> 00:39:23.599
<v Speaker 2>tools correctly. It's whether it's drawing the right conclusion, whether

749
00:39:23.599 --> 00:39:26.719
<v Speaker 2>it's looping in the right direction, and that's harness. Yeah,

750
00:39:26.760 --> 00:39:27.599
<v Speaker 2>that's not AI.

751
00:39:27.719 --> 00:39:30.639
<v Speaker 3>Yeah, absolutely no, agreed, one hundred percent. And it's you know,

752
00:39:30.639 --> 00:39:33.239
<v Speaker 3>it's funny because a lot of people would like obviously

753
00:39:33.280 --> 00:39:35.880
<v Speaker 3>approach me when glass Wing came out, or approached me

754
00:39:35.920 --> 00:39:38.519
<v Speaker 3>when you know, GPT cyber came out. They're like, are

755
00:39:38.599 --> 00:39:42.800
<v Speaker 3>you using it? Is it awesome? And I'm like, like,

756
00:39:42.840 --> 00:39:46.280
<v Speaker 3>the stuff we have in houses better because of the harness.

757
00:39:45.960 --> 00:39:48.079
<v Speaker 2>Because you're working on the harness exactly.

758
00:39:48.119 --> 00:39:50.920
<v Speaker 3>That's the thing is and it's and it's more effective

759
00:39:51.360 --> 00:39:54.760
<v Speaker 3>with the even even if we're not using frontier models, right,

760
00:39:54.800 --> 00:39:57.440
<v Speaker 3>but it's still more effective if we're using frontier models

761
00:39:57.440 --> 00:40:01.239
<v Speaker 3>and not using something that's cyber targeted. If the harness

762
00:40:01.280 --> 00:40:02.760
<v Speaker 3>is better, if the process is better.

763
00:40:02.800 --> 00:40:05.840
<v Speaker 2>Yeah, and that's something that I'm just starting to see

764
00:40:05.880 --> 00:40:10.599
<v Speaker 2>writings on the fringe of AI talking about the harness,

765
00:40:10.639 --> 00:40:12.440
<v Speaker 2>and it's something we've been working on for well over

766
00:40:12.480 --> 00:40:16.639
<v Speaker 2>a year. You expect to see people specialized, Like there's

767
00:40:16.639 --> 00:40:22.760
<v Speaker 2>a company. Miramrati was the CTO of Open AI. She

768
00:40:22.880 --> 00:40:26.119
<v Speaker 2>left and started Thinking Labs and they came out with

769
00:40:26.719 --> 00:40:28.559
<v Speaker 2>a couple of products. One of them is called Inkling,

770
00:40:29.199 --> 00:40:33.679
<v Speaker 2>and Inkling allows you to train models existing models to

771
00:40:33.760 --> 00:40:36.840
<v Speaker 2>do basically tune them for your own use without having

772
00:40:36.960 --> 00:40:39.920
<v Speaker 2>hundreds of millions of dollars worth of hardware. We're going

773
00:40:40.000 --> 00:40:42.039
<v Speaker 2>to see a lot more of that specialization, I think,

774
00:40:42.079 --> 00:40:44.039
<v Speaker 2>in the next round, and some of that's going to

775
00:40:44.039 --> 00:40:46.920
<v Speaker 2>be cyber focused, some of that's going to be you know,

776
00:40:47.760 --> 00:40:51.880
<v Speaker 2>finance focused, but it all has to be like guardrail

777
00:40:51.920 --> 00:40:56.559
<v Speaker 2>within your specification focused. I believe that the problem was

778
00:40:56.599 --> 00:40:58.559
<v Speaker 2>with the harness, not with the AI.

779
00:40:58.920 --> 00:41:02.519
<v Speaker 1>Yeah, I agree. I thought inkling was the offspring of

780
00:41:02.599 --> 00:41:05.000
<v Speaker 1>India ink and squid ink.

781
00:41:05.159 --> 00:41:07.480
<v Speaker 3>Is that iron gall ink? Not sure?

782
00:41:07.840 --> 00:41:13.639
<v Speaker 2>Maybe I think that's SpongeBob's nephew made squid word.

783
00:41:13.679 --> 00:41:15.559
<v Speaker 3>Squidword, Oh my god.

784
00:41:16.079 --> 00:41:17.719
<v Speaker 1>All right, Well that was a lot of fun and

785
00:41:17.800 --> 00:41:20.480
<v Speaker 1>a little scary, and I guess that's all we have

786
00:41:20.559 --> 00:41:22.320
<v Speaker 1>to report about what happened last week, and we'll see

787
00:41:22.320 --> 00:41:24.519
<v Speaker 1>you next week on Security this week. Thanks bye,
