WEBVTT

1
00:00:00.960 --> 00:00:02.879
<v Speaker 1>Hey Richard, Hey Carl, what do you know?

2
00:00:03.120 --> 00:00:06.679
<v Speaker 2>Well, I know that our friend Michelle Rubusta Monte is

3
00:00:06.719 --> 00:00:08.880
<v Speaker 2>with us to tell us about something that's going on

4
00:00:09.519 --> 00:00:11.199
<v Speaker 2>adjacent to DEV Intersection.

5
00:00:11.240 --> 00:00:15.560
<v Speaker 1>What is it? It's cybersecurity Intersection. Let's let Michelle tell

6
00:00:15.599 --> 00:00:16.199
<v Speaker 1>that story.

7
00:00:16.679 --> 00:00:21.320
<v Speaker 3>Hey Michelle, Hey Carl, Hey Richard, how are you.

8
00:00:21.000 --> 00:00:22.839
<v Speaker 2>Tell us about cybersecurity Intersection?

9
00:00:23.480 --> 00:00:26.920
<v Speaker 3>Well, so, Richard and I are partnering with the group

10
00:00:27.000 --> 00:00:30.320
<v Speaker 3>that does DEV Intersection and next Gen AI, and we

11
00:00:30.600 --> 00:00:34.200
<v Speaker 3>are putting on a new conference dedicated to one hundred

12
00:00:34.240 --> 00:00:40.000
<v Speaker 3>percent security focused topics. And I mean, honestly, the lineup

13
00:00:40.039 --> 00:00:43.679
<v Speaker 3>of speakers is incredible. We have Paula A. Jenis, who's

14
00:00:43.719 --> 00:00:47.600
<v Speaker 3>here from Poland and does keynotes all over the world

15
00:00:47.719 --> 00:00:50.600
<v Speaker 3>and is one of the top rated RSA speakers and

16
00:00:50.640 --> 00:00:53.159
<v Speaker 3>black hat speaker. We're so lucky to have her. But

17
00:00:53.240 --> 00:00:56.000
<v Speaker 3>she's not only keynoting, she's got a workshop teaches you

18
00:00:56.039 --> 00:01:01.039
<v Speaker 3>about protecting your environments against hackers and shows you about

19
00:01:01.280 --> 00:01:03.719
<v Speaker 3>how to you know, do attacks so that you can

20
00:01:03.759 --> 00:01:07.840
<v Speaker 3>prevent them. It's pretty cool and sessions like that as well.

21
00:01:07.840 --> 00:01:10.799
<v Speaker 3>But we also have speakers from Microsoft. We have we

22
00:01:10.840 --> 00:01:15.120
<v Speaker 3>have speakers that specialize in you know secure coding practices,

23
00:01:15.640 --> 00:01:20.439
<v Speaker 3>Azure security, Zuero, trust architectures on Azure UH and people

24
00:01:20.439 --> 00:01:23.519
<v Speaker 3>who do decision maker tracks, so things around governance policy

25
00:01:23.560 --> 00:01:26.200
<v Speaker 3>and you know how to how to manage and your

26
00:01:26.200 --> 00:01:29.480
<v Speaker 3>production operations keep them secure. So it's an amazing group

27
00:01:29.519 --> 00:01:31.040
<v Speaker 3>of speakers, really excited about it.

28
00:01:31.120 --> 00:01:33.560
<v Speaker 2>And I think I can count myself among the group

29
00:01:33.599 --> 00:01:34.599
<v Speaker 2>of speakers there.

30
00:01:35.040 --> 00:01:37.120
<v Speaker 3>Well, yes you can. That is great.

31
00:01:37.480 --> 00:01:42.159
<v Speaker 2>Yeah, I'm doing a securing Blazer Server applications talk and

32
00:01:42.239 --> 00:01:45.959
<v Speaker 2>also I think we're doing a Security this Week live

33
00:01:46.040 --> 00:01:48.480
<v Speaker 2>show there somewhere that is correct.

34
00:01:48.719 --> 00:01:51.200
<v Speaker 3>Yeah, we'll be recording Security this Week Live. We're going

35
00:01:51.239 --> 00:01:54.719
<v Speaker 3>to have a great panel with some folks. The interesting

36
00:01:54.719 --> 00:01:57.920
<v Speaker 3>thing here is we don't really have a Microsoft and

37
00:01:58.000 --> 00:02:02.120
<v Speaker 3>dot net and Azure focused toecurity conference yet, so that's

38
00:02:02.159 --> 00:02:05.159
<v Speaker 3>the reason we're putting this on as well. You know

39
00:02:05.200 --> 00:02:08.280
<v Speaker 3>there are other security conferences, but they have a spread

40
00:02:08.280 --> 00:02:10.439
<v Speaker 3>of topics that maybe don't focus on the things you

41
00:02:10.520 --> 00:02:13.080
<v Speaker 3>do day to day. And you know this overlaps with

42
00:02:13.199 --> 00:02:17.560
<v Speaker 3>again our community of folks that specialize in again dot net,

43
00:02:17.680 --> 00:02:21.039
<v Speaker 3>Azure and yeah, they need to keep it secure too,

44
00:02:21.280 --> 00:02:22.840
<v Speaker 3>So with tons of talks.

45
00:02:23.719 --> 00:02:27.240
<v Speaker 1>Cyber Intersection is part of a trio of conferences we're doing.

46
00:02:27.280 --> 00:02:30.360
<v Speaker 1>They have Intersection alongside the next Gen AI conference all

47
00:02:30.439 --> 00:02:34.680
<v Speaker 1>in Orlando the week of October fifth through tenth. That's

48
00:02:34.759 --> 00:02:38.080
<v Speaker 1>workshops and the main conference. And you can get a

49
00:02:38.120 --> 00:02:41.400
<v Speaker 1>special registration code if you sign up through Cybersecurity Intersection

50
00:02:41.639 --> 00:02:42.360
<v Speaker 1>dot com.

51
00:02:42.680 --> 00:02:47.400
<v Speaker 3>Yeah, so if you sign up at Cybersecurity Intersection dot com,

52
00:02:47.599 --> 00:02:52.520
<v Speaker 3>then you put in this code so Alliance cyber three

53
00:02:52.599 --> 00:02:56.280
<v Speaker 3>hundred and you'll get three hundred off the entry price.

54
00:02:56.560 --> 00:02:59.680
<v Speaker 3>So that's a special code that only works at cybersecurity

55
00:03:00.599 --> 00:03:04.439
<v Speaker 3>dot com. And then you have access to all the conferences.

56
00:03:04.479 --> 00:03:09.400
<v Speaker 2>Like Richard said, Wow, that's cool. Thanks Michelle. I'm looking

57
00:03:09.439 --> 00:03:24.120
<v Speaker 2>forward to it and I'll see you there. Hey, get

58
00:03:24.159 --> 00:03:26.800
<v Speaker 2>down rock and Roll. It's Carl Franklin and Richard Campbell

59
00:03:26.800 --> 00:03:27.759
<v Speaker 2>for dot net Rocks.

60
00:03:28.000 --> 00:03:29.400
<v Speaker 1>Hey, Richard, how you do it, Bud?

61
00:03:29.479 --> 00:03:33.400
<v Speaker 2>I'm good, getting psyched up to go down to Orlando.

62
00:03:33.759 --> 00:03:37.240
<v Speaker 1>Yeah, it's almost time back to a new dev Intersection

63
00:03:37.400 --> 00:03:41.319
<v Speaker 1>and next jen AI and the New Cybersecurity Conference side

64
00:03:41.360 --> 00:03:42.680
<v Speaker 1>by side. Yep, yep.

65
00:03:42.879 --> 00:03:45.919
<v Speaker 2>Looking forward to doing a live security this week's show

66
00:03:46.120 --> 00:03:46.599
<v Speaker 2>down there.

67
00:03:46.719 --> 00:03:50.479
<v Speaker 1>That should be fun, fun and you're crazy thing with Maddie.

68
00:03:50.560 --> 00:03:54.560
<v Speaker 1>Oh god, you're going to aspireify dot net rocks here.

69
00:03:54.599 --> 00:03:56.639
<v Speaker 2>I have no idea what to expect. That could be

70
00:03:56.680 --> 00:03:57.439
<v Speaker 2>a horror show.

71
00:03:57.879 --> 00:04:00.319
<v Speaker 1>This is you know, you love a good you know,

72
00:04:00.439 --> 00:04:02.960
<v Speaker 1>trapease act, just going without a net.

73
00:04:03.080 --> 00:04:07.919
<v Speaker 2>Absolutely, as long as I don't you know, screw up

74
00:04:07.919 --> 00:04:10.000
<v Speaker 2>too badly, it should it should work out fun.

75
00:04:10.039 --> 00:04:13.240
<v Speaker 1>You know, a good crash it burns fun too, but it.

76
00:04:13.199 --> 00:04:16.800
<v Speaker 2>Could be fun. Yeah, yeah, yeah, Okay, let's start with

77
00:04:16.879 --> 00:04:20.720
<v Speaker 2>nineteen seventy. That's the episode number. Oh yeah, and a

78
00:04:21.000 --> 00:04:23.199
<v Speaker 2>bunch of things happened in nineteen seventy.

79
00:04:23.480 --> 00:04:24.519
<v Speaker 1>Where do you want to start?

80
00:04:24.639 --> 00:04:27.279
<v Speaker 2>Well, the unhappy things, the Kent State shootings.

81
00:04:27.399 --> 00:04:28.399
<v Speaker 1>Yeah, it's terrifying.

82
00:04:28.639 --> 00:04:32.000
<v Speaker 2>On May fourth, National Guard troops killed four students during

83
00:04:32.000 --> 00:04:35.639
<v Speaker 2>protest against the Vietnam War at Kent State University in Ohio,

84
00:04:37.519 --> 00:04:42.360
<v Speaker 2>leading to nationwide outrage and the song what is it

85
00:04:42.399 --> 00:04:43.600
<v Speaker 2>for Dead in Ohio?

86
00:04:43.800 --> 00:04:44.160
<v Speaker 1>Who's that?

87
00:04:44.319 --> 00:04:47.759
<v Speaker 2>Neil Young or Crosby Sills, Nash and Young? I'm not sure.

88
00:04:49.600 --> 00:04:55.399
<v Speaker 2>Nigerian Civil War. The conflict ended in January when Biaffron

89
00:04:55.560 --> 00:04:59.160
<v Speaker 2>forces the Affron forces surrendered after a thirty two month

90
00:04:59.199 --> 00:05:03.439
<v Speaker 2>struggle for independ and it's the first Earth Day was observed.

91
00:05:03.480 --> 00:05:07.279
<v Speaker 2>On April twenty second, the Beatles broke up and let

92
00:05:07.319 --> 00:05:10.439
<v Speaker 2>it be. McCartney said he was leaving the band on

93
00:05:10.480 --> 00:05:13.199
<v Speaker 2>April tenth. That was the end of that. But John

94
00:05:13.279 --> 00:05:17.040
<v Speaker 2>Lennon instant karma. He wrote and recorded this hit song

95
00:05:17.079 --> 00:05:22.519
<v Speaker 2>in a single day, showcasing his prolific creativity. Diana Ross

96
00:05:22.519 --> 00:05:25.920
<v Speaker 2>and the Supremes gave their final concert in Las Vegas

97
00:05:25.920 --> 00:05:31.600
<v Speaker 2>in January fourteenth. Back to the bad stuff the Tonguhai earthquake.

98
00:05:32.439 --> 00:05:38.120
<v Speaker 2>Devastating earthquakes struck Tongue High County, China on January fifth,

99
00:05:38.160 --> 00:05:42.720
<v Speaker 2>resulting in significant casualties, with estimates of up to fourteen thousand,

100
00:05:43.079 --> 00:05:47.480
<v Speaker 2>six hundred and twenty one deaths. Yeah, and an avalanche

101
00:05:47.560 --> 00:05:51.000
<v Speaker 2>in someplace that in France that I can't pronounce zi

102
00:05:52.680 --> 00:05:56.480
<v Speaker 2>fool full, Sorry about that killed forty two people, making

103
00:05:56.480 --> 00:06:00.519
<v Speaker 2>one of the worst disasters in French skiing history. You

104
00:06:00.560 --> 00:06:04.920
<v Speaker 2>can talk about the science, yea science. Some things happened,

105
00:06:05.079 --> 00:06:05.959
<v Speaker 2>well I was.

106
00:06:06.480 --> 00:06:09.360
<v Speaker 1>I mean the space one's the obvious one. After having

107
00:06:09.519 --> 00:06:12.759
<v Speaker 1>both Apollo nine, Pollo ten, and Paula eleven and Apollo

108
00:06:12.839 --> 00:06:15.360
<v Speaker 1>twelve all in nineteen sixty nine, there was only one

109
00:06:15.360 --> 00:06:19.319
<v Speaker 1>Apollo mission in nineteen seventeen. That was a Poulla thirteen.

110
00:06:19.399 --> 00:06:23.120
<v Speaker 1>It launched on April eleventh, and on April thirteenth they said,

111
00:06:23.720 --> 00:06:25.000
<v Speaker 1>we've had a problem.

112
00:06:24.600 --> 00:06:28.279
<v Speaker 2>Here, Houston. We've got a problem. And we're a great

113
00:06:28.319 --> 00:06:28.879
<v Speaker 2>movie too.

114
00:06:29.040 --> 00:06:31.680
<v Speaker 1>Yeah, and you've seen the movie, a beautiful rendering of

115
00:06:32.319 --> 00:06:35.720
<v Speaker 1>more or less what happened. The HBO Earth of the

116
00:06:35.720 --> 00:06:38.839
<v Speaker 1>Moon series, if you ever get a chance to watch,

117
00:06:38.959 --> 00:06:42.519
<v Speaker 1>that does a version of Apollo thirteen, but from the

118
00:06:42.639 --> 00:06:45.480
<v Speaker 1>view of the people on the ground, so you only

119
00:06:45.519 --> 00:06:48.720
<v Speaker 1>ever hear the astronauts over the radio, which is how

120
00:06:48.800 --> 00:06:51.839
<v Speaker 1>it was. Right. Sure, here's the crazy thing to realize.

121
00:06:51.920 --> 00:06:54.600
<v Speaker 1>So the explosion in the tank happens on April on

122
00:06:54.680 --> 00:06:59.920
<v Speaker 1>April thirteenth, the splash downy April seventeenth. It was four days. Wow,

123
00:07:00.160 --> 00:07:02.560
<v Speaker 1>the whole thing's four days. I know, it feels like forever.

124
00:07:03.120 --> 00:07:03.800
<v Speaker 1>It's four days.

125
00:07:03.800 --> 00:07:04.079
<v Speaker 2>Wow.

126
00:07:04.079 --> 00:07:05.959
<v Speaker 1>But it was four days of are these guys going

127
00:07:06.040 --> 00:07:08.800
<v Speaker 1>to make it? You know, like four days of sheer terror. Yeah,

128
00:07:08.800 --> 00:07:12.759
<v Speaker 1>it was. And of course they the lunar module Aquarius

129
00:07:12.879 --> 00:07:16.079
<v Speaker 1>was turned into a lifeboat because the power systems, a

130
00:07:16.079 --> 00:07:17.639
<v Speaker 1>little bit of battery that was left in the command

131
00:07:17.680 --> 00:07:19.199
<v Speaker 1>module is going to need for re entry, so they

132
00:07:19.319 --> 00:07:22.279
<v Speaker 1>basically powered down the command module and then use the

133
00:07:22.480 --> 00:07:26.600
<v Speaker 1>Life Sports system for two four to three and just

134
00:07:26.720 --> 00:07:28.800
<v Speaker 1>four days and they were able to get home amazing

135
00:07:28.839 --> 00:07:31.560
<v Speaker 1>and survive. It's a great story. And of course the

136
00:07:31.600 --> 00:07:33.720
<v Speaker 1>next Apollo mission would be delayed while they dealt with

137
00:07:33.720 --> 00:07:36.040
<v Speaker 1>some of those issues, and in nineteen seventy one, you'll

138
00:07:36.040 --> 00:07:39.279
<v Speaker 1>get Apollo fourteen. Talk about that next week apparently. Yeah.

139
00:07:39.319 --> 00:07:44.160
<v Speaker 1>On the computer side of things, Nicholas Worth releases Pascal Woll.

140
00:07:44.560 --> 00:07:47.160
<v Speaker 1>He worked previously on the language I'll Go sixty and

141
00:07:47.199 --> 00:07:49.680
<v Speaker 1>there's some derivations therein he was trying to do a

142
00:07:49.680 --> 00:07:56.600
<v Speaker 1>combination of sort of procedural and algorithmic programming. So Popular

143
00:07:56.680 --> 00:08:00.560
<v Speaker 1>Language did some great things. But on the heart work side,

144
00:08:00.600 --> 00:08:04.040
<v Speaker 1>for me, the show stealer is my you know, iba

145
00:08:04.439 --> 00:08:10.399
<v Speaker 1>Intel's most important product, the eleven O three, the d ram. Okay,

146
00:08:10.519 --> 00:08:14.000
<v Speaker 1>this is what Moore's law actually was about, was making

147
00:08:14.720 --> 00:08:18.120
<v Speaker 1>RAM right based on a bunch of other developments to

148
00:08:18.160 --> 00:08:22.680
<v Speaker 1>make a transistor based memory. They were able to make

149
00:08:22.720 --> 00:08:27.480
<v Speaker 1>a silicon substrate for an eighteen dip pin dip can

150
00:08:27.920 --> 00:08:31.759
<v Speaker 1>with one K of RAM in it for sixty bucks.

151
00:08:31.800 --> 00:08:34.399
<v Speaker 2>Wow, that seems cheap back then, and.

152
00:08:34.480 --> 00:08:37.480
<v Speaker 1>One cent per bit, and it was small because then

153
00:08:37.519 --> 00:08:40.759
<v Speaker 1>they were largely using core magnet Ferris cores for memory.

154
00:08:40.840 --> 00:08:45.080
<v Speaker 1>So this was very compact and it was adopted immediately everywhere.

155
00:08:45.120 --> 00:08:48.440
<v Speaker 1>It's it's uptake. That's also the same year that the

156
00:08:48.480 --> 00:08:51.559
<v Speaker 1>first version of the IBM system three seventy comes out

157
00:08:51.559 --> 00:08:54.519
<v Speaker 1>with all semi conductor RAM, but that was not Intel's RAM.

158
00:08:54.559 --> 00:08:58.480
<v Speaker 1>But shortly after that, Intel's RAM just dominates the market

159
00:08:58.519 --> 00:09:03.360
<v Speaker 1>and sends Intel on its trajectory. Although nineteen seventy one

160
00:09:03.360 --> 00:09:06.039
<v Speaker 1>they'll make arguably and even more in product important product.

161
00:09:06.440 --> 00:09:09.960
<v Speaker 1>Tune in next week for nineteen seventy one, nineteen seventy one.

162
00:09:10.000 --> 00:09:13.600
<v Speaker 1>But yes, the eleven O three was there, you know,

163
00:09:13.759 --> 00:09:17.320
<v Speaker 1>definitive product. They were rammed digit you know, semiconductor ramming.

164
00:09:18.039 --> 00:09:18.799
<v Speaker 1>And that's what I got.

165
00:09:18.879 --> 00:09:22.360
<v Speaker 2>All right, well, I guess we should carry on with

166
00:09:22.480 --> 00:09:23.960
<v Speaker 2>better no framework.

167
00:09:23.600 --> 00:09:24.919
<v Speaker 1>Roll the crazy music possible.

168
00:09:32.360 --> 00:09:34.399
<v Speaker 2>All right, man, what do you got again? I looked

169
00:09:34.399 --> 00:09:37.360
<v Speaker 2>for a trending repost on GitHub and I found MCP

170
00:09:37.720 --> 00:09:42.120
<v Speaker 2>for Unity. Oh my, yeah, you know Unity Create create

171
00:09:42.200 --> 00:09:47.559
<v Speaker 2>games with the Unity. It's a graphical tool that uses

172
00:09:47.639 --> 00:09:50.879
<v Speaker 2>c sharp and JavaScript for scripting, but it also does

173
00:09:50.919 --> 00:09:54.039
<v Speaker 2>all of the three D stuff. So here's what it is,

174
00:09:54.559 --> 00:10:00.399
<v Speaker 2>proudly sponsored and maintained by Coplay, the best AI systant

175
00:10:00.440 --> 00:10:03.440
<v Speaker 2>for Unity. There you go create your Unity apps with

176
00:10:03.639 --> 00:10:06.840
<v Speaker 2>l l MS. M CP for Unity acts as a bridge,

177
00:10:06.879 --> 00:10:11.159
<v Speaker 2>allowing a assistance like Claude Cursor to interact directly with

178
00:10:11.240 --> 00:10:16.080
<v Speaker 2>your Unity editor via a local MCP model context protocol.

179
00:10:16.120 --> 00:10:20.320
<v Speaker 2>We've been talking about those. A local MCP client use

180
00:10:20.360 --> 00:10:25.000
<v Speaker 2>your lll M tools to manage assets, control scenes, edit scripts,

181
00:10:25.039 --> 00:10:27.320
<v Speaker 2>and automate tasks within Unity.

182
00:10:27.639 --> 00:10:32.000
<v Speaker 1>Pretty cool. Interesting, Yeah, a good show to actually walk

183
00:10:32.080 --> 00:10:36.080
<v Speaker 1>through the process of, you know, including making a game

184
00:10:36.080 --> 00:10:39.279
<v Speaker 1>in Unity with with the MCPM, with l MS in

185
00:10:39.279 --> 00:10:40.000
<v Speaker 1>the role. Yep.

186
00:10:40.759 --> 00:10:43.919
<v Speaker 2>Also code it with the AI. Dot com is up

187
00:10:44.240 --> 00:10:50.480
<v Speaker 2>and the first episode is there and we're basically using

188
00:10:50.600 --> 00:10:56.679
<v Speaker 2>playwright to with the code agent in the visual studio

189
00:10:56.720 --> 00:11:00.679
<v Speaker 2>code nice and using clauds on it. And we basically

190
00:11:01.039 --> 00:11:06.720
<v Speaker 2>one prompt told it to create a user documentation of

191
00:11:08.279 --> 00:11:15.120
<v Speaker 2>Jeff Fritz's copilot do John dot com website, and it

192
00:11:15.159 --> 00:11:18.240
<v Speaker 2>did a pretty good job. What we didn't show was

193
00:11:18.559 --> 00:11:22.879
<v Speaker 2>what's involved in setting up the playwright MCP so that

194
00:11:22.960 --> 00:11:25.679
<v Speaker 2>the agent can use it. Oh yeah, and it turns

195
00:11:25.720 --> 00:11:29.840
<v Speaker 2>out that's pretty complex. You need node JS and NPM

196
00:11:30.120 --> 00:11:34.080
<v Speaker 2>and all that stuff, and we're looking for a video

197
00:11:34.919 --> 00:11:37.159
<v Speaker 2>on how to do that, so look in the show

198
00:11:37.200 --> 00:11:40.080
<v Speaker 2>notes for that. Cool, but that's it for a better

199
00:11:40.120 --> 00:11:42.200
<v Speaker 2>no framework. Who's talking to us? I have a common

200
00:11:42.200 --> 00:11:45.120
<v Speaker 2>of a show nineteen sixty nine. Yes, that's last week's

201
00:11:45.159 --> 00:11:48.879
<v Speaker 2>show with our friend James monte Magno.

202
00:11:48.639 --> 00:11:51.440
<v Speaker 1>And we talked a little bit about the AI tooling

203
00:11:51.480 --> 00:11:54.960
<v Speaker 1>inside of Visual Studio code and its relationship with Visual

204
00:11:55.000 --> 00:11:57.399
<v Speaker 1>Studio and so on. And our friend Richard Rukima, also

205
00:11:57.480 --> 00:12:00.440
<v Speaker 1>known as Coputer, has this common but he says, I

206
00:12:00.440 --> 00:12:02.879
<v Speaker 1>think Richard nailed it. Do you like the code or

207
00:12:02.919 --> 00:12:06.320
<v Speaker 1>do you like a solution? I consider my expertise working

208
00:12:06.320 --> 00:12:09.120
<v Speaker 1>with AI as a beginner, especially after listening to James,

209
00:12:09.159 --> 00:12:12.039
<v Speaker 1>but I felt that vibe of joy in getting things

210
00:12:12.120 --> 00:12:15.960
<v Speaker 1>done so fast. So do I like the if then else?

211
00:12:16.080 --> 00:12:18.080
<v Speaker 1>Or do I like ask you for a future reviewing

212
00:12:18.120 --> 00:12:21.120
<v Speaker 1>the result? I'm long past the joy of knowing how

213
00:12:21.159 --> 00:12:25.759
<v Speaker 1>to write procedural code. Yeah. An interesting aspect of this is, like,

214
00:12:26.240 --> 00:12:28.159
<v Speaker 1>is it the more experienced folks that are going to

215
00:12:28.159 --> 00:12:31.559
<v Speaker 1>embrace these tools faster? Because it's typically the more junior

216
00:12:31.600 --> 00:12:33.799
<v Speaker 1>people that tend to jump on the bandwagona new things,

217
00:12:33.840 --> 00:12:36.840
<v Speaker 1>but I hear the same tone over and over again. Yep.

218
00:12:37.000 --> 00:12:40.279
<v Speaker 1>Certainly in terms of respectful interaction with AI, I don't

219
00:12:40.360 --> 00:12:44.120
<v Speaker 1>prescribe to the harsh language, as I feel it reveals character.

220
00:12:44.320 --> 00:12:48.039
<v Speaker 1>It's an interesting statement rights in my character not to

221
00:12:48.159 --> 00:12:51.480
<v Speaker 1>be harsh fully or and to focus on being respectful communication.

222
00:12:51.679 --> 00:12:54.000
<v Speaker 1>I don't think AI should be treated any different, not

223
00:12:54.039 --> 00:12:57.120
<v Speaker 1>for the benefit of AI or the benefit of myself.

224
00:12:57.200 --> 00:13:00.480
<v Speaker 2>Yeah, exactly. You're not going to feel good, you know,

225
00:13:00.759 --> 00:13:01.759
<v Speaker 2>using harsh language.

226
00:13:01.799 --> 00:13:04.320
<v Speaker 1>Putting those mean words out there is as much impact

227
00:13:04.360 --> 00:13:06.240
<v Speaker 1>on you as it is on anything else. And leave me.

228
00:13:06.320 --> 00:13:10.720
<v Speaker 1>The software is not affected, that's thing, right.

229
00:13:11.000 --> 00:13:13.240
<v Speaker 2>The only thing left to be affected is you.

230
00:13:13.720 --> 00:13:18.320
<v Speaker 1>Yeah, so be kind to yourself. It's not necessary, right, Hey, Richard,

231
00:13:18.399 --> 00:13:20.759
<v Speaker 1>I'm pretty sure you've got a copy of Music code

232
00:13:20.759 --> 00:13:23.200
<v Speaker 1>By already, but thanks so much for your comment. But

233
00:13:23.240 --> 00:13:24.759
<v Speaker 1>if you'd like a copy of music, Cobey, I write

234
00:13:24.759 --> 00:13:26.639
<v Speaker 1>a comment on the website at dot net rocks dot

235
00:13:26.679 --> 00:13:28.399
<v Speaker 1>com or on the facebooks to publish every show there

236
00:13:28.399 --> 00:13:30.200
<v Speaker 1>and every comment there, and never reading the show, will

237
00:13:30.240 --> 00:13:31.159
<v Speaker 1>send your copy of music Go.

238
00:13:31.399 --> 00:13:33.519
<v Speaker 2>Music to code By is still going strong after all

239
00:13:33.559 --> 00:13:36.399
<v Speaker 2>these years twenty two tracks. You can get him in

240
00:13:37.279 --> 00:13:41.440
<v Speaker 2>uh wave, flack or MP three and that's at Music

241
00:13:41.480 --> 00:13:46.080
<v Speaker 2>to Code by dot Net. Okay, let's bring back our

242
00:13:46.120 --> 00:13:50.360
<v Speaker 2>friend Joseph Finney. Joseph is a mobile product owner in

243
00:13:50.559 --> 00:13:54.000
<v Speaker 2>MVP by day and he builds productivity apps for Windows

244
00:13:54.039 --> 00:13:58.399
<v Speaker 2>by night. When he's not programming, he's burning running and

245
00:13:58.480 --> 00:14:00.919
<v Speaker 2>enjoying tasty coffee and beer in Milwaukee.

246
00:14:00.960 --> 00:14:03.279
<v Speaker 1>Hey Joe, Hello, welcome back to having that.

247
00:14:03.440 --> 00:14:07.320
<v Speaker 4>Good to be back talking more about the hot topic

248
00:14:07.360 --> 00:14:09.039
<v Speaker 4>of the day, AI.

249
00:14:08.840 --> 00:14:11.320
<v Speaker 1>With a Century. Yeah, but you've got you've got a

250
00:14:11.360 --> 00:14:13.240
<v Speaker 1>cool angle of this. That's why I asked you that

251
00:14:13.399 --> 00:14:15.679
<v Speaker 1>to come on. So what are you working on?

252
00:14:15.840 --> 00:14:18.679
<v Speaker 4>Well, one of my most popular apps that I make

253
00:14:18.799 --> 00:14:22.360
<v Speaker 4>is text grab, which is pretty basic. It's also the

254
00:14:22.399 --> 00:14:26.840
<v Speaker 4>basis for the Power Toys Text Extractor, which is basically

255
00:14:26.879 --> 00:14:29.720
<v Speaker 4>select a region on your screen of somebody who sent

256
00:14:29.759 --> 00:14:32.919
<v Speaker 4>you text that you can't actually select and put somewhere

257
00:14:32.960 --> 00:14:35.639
<v Speaker 4>where you want it. And it does some on device

258
00:14:35.840 --> 00:14:41.960
<v Speaker 4>local OCR. Pretty simple, and now with these new models,

259
00:14:42.960 --> 00:14:47.720
<v Speaker 4>the OCR is getting better. But it does change compatibility

260
00:14:47.799 --> 00:14:51.279
<v Speaker 4>and devices, but it's it's pretty interesting what we can

261
00:14:51.320 --> 00:14:54.000
<v Speaker 4>do here now with these local models Microsoft's making it

262
00:14:54.039 --> 00:14:58.399
<v Speaker 4>easier with some of their Windows AI APIs, and then

263
00:14:58.399 --> 00:15:00.919
<v Speaker 4>there's it just gets more and more complicated from there.

264
00:15:01.320 --> 00:15:03.440
<v Speaker 2>Mm hmm. So I have an app that I'm running

265
00:15:03.519 --> 00:15:08.000
<v Speaker 2>right here that does little OCR and I'm using Tesseract

266
00:15:08.399 --> 00:15:11.120
<v Speaker 2>to read the text in a bitmap at a certain coordinate.

267
00:15:12.799 --> 00:15:16.000
<v Speaker 2>That is that the sort of representing the state of

268
00:15:16.000 --> 00:15:17.759
<v Speaker 2>the art before AI got into the mix.

269
00:15:17.840 --> 00:15:20.000
<v Speaker 4>Yeah, I would say it's it's similar. Tests React was

270
00:15:20.039 --> 00:15:23.159
<v Speaker 4>the open source project that Google took over I think.

271
00:15:23.240 --> 00:15:26.200
<v Speaker 4>I think actually HP started it way back there and

272
00:15:26.240 --> 00:15:29.440
<v Speaker 4>then kind of Google took it over. Yeah, it's on GitHub.

273
00:15:29.480 --> 00:15:33.200
<v Speaker 4>There's a lot of models. It's very widely used and loved.

274
00:15:35.200 --> 00:15:39.080
<v Speaker 4>Text grab does enable you to download tests earect and

275
00:15:39.120 --> 00:15:41.480
<v Speaker 4>then you can interact with it through the CLI. Well

276
00:15:41.679 --> 00:15:44.720
<v Speaker 4>text grab will just interact with it directly, but there's

277
00:15:44.759 --> 00:15:46.200
<v Speaker 4>a little bit of setting up. You do have to

278
00:15:46.200 --> 00:15:50.000
<v Speaker 4>download it. It's a it's another installer. It's through ub

279
00:15:50.159 --> 00:15:52.639
<v Speaker 4>Mannheim I think who does the installation. So there's definitely

280
00:15:52.720 --> 00:15:54.679
<v Speaker 4>some hoops you have to jump through to get it working.

281
00:15:54.440 --> 00:15:57.200
<v Speaker 2>And there's a data set that goes along with it, right.

282
00:15:57.159 --> 00:15:59.519
<v Speaker 4>Yeah, Yeah, so yeah, you have to download the languages.

283
00:15:59.559 --> 00:16:02.200
<v Speaker 4>There's a lot. One of the benefits there of Tessaact

284
00:16:02.320 --> 00:16:04.399
<v Speaker 4>is that there's a lot of languages, and they have

285
00:16:05.480 --> 00:16:10.799
<v Speaker 4>packages for scripts, and they have packages for like handwritten

286
00:16:11.120 --> 00:16:15.480
<v Speaker 4>and so it's really high quality. Originally, Textcrab was built

287
00:16:15.559 --> 00:16:19.799
<v Speaker 4>using the Windows ten ocr APIs, which are definitely older,

288
00:16:19.960 --> 00:16:22.480
<v Speaker 4>not as good, but they're very fast. So that was

289
00:16:22.519 --> 00:16:24.759
<v Speaker 4>kind of the nice thing there. They're built in, they're fast,

290
00:16:24.799 --> 00:16:26.919
<v Speaker 4>they're quick for most stuff. It worked pretty well cool

291
00:16:27.039 --> 00:16:28.960
<v Speaker 4>test erect was a bump up, but again you have

292
00:16:29.000 --> 00:16:31.480
<v Speaker 4>that complexity where you have to download the models locally.

293
00:16:31.639 --> 00:16:35.440
<v Speaker 4>But it's open source, it's available, it's free. And now

294
00:16:35.840 --> 00:16:39.960
<v Speaker 4>there's these Windows AI APIs that Microsoft has released. I

295
00:16:40.000 --> 00:16:42.480
<v Speaker 4>don't think we know exactly what those models are. I

296
00:16:42.480 --> 00:16:46.600
<v Speaker 4>don't think they've shared. I haven't learned what they are exactly.

297
00:16:46.960 --> 00:16:49.600
<v Speaker 2>But what was the acronym that you used before we

298
00:16:49.679 --> 00:16:52.240
<v Speaker 2>started recording for this new.

299
00:16:52.080 --> 00:16:56.360
<v Speaker 4>Wind WINML, Windows and machine learning.

300
00:16:56.720 --> 00:16:59.519
<v Speaker 2>Okay, and this is new, yeah, literally days old than

301
00:16:59.559 --> 00:17:00.799
<v Speaker 2>we don't know anything about it.

302
00:17:00.840 --> 00:17:03.480
<v Speaker 4>Well, the win mL stuff is kind of a middle

303
00:17:03.559 --> 00:17:05.839
<v Speaker 4>layer here, Okay. So I would say there's like three

304
00:17:06.400 --> 00:17:12.200
<v Speaker 4>general levels of intensity. If you are a local Windows

305
00:17:12.240 --> 00:17:17.920
<v Speaker 4>app developer and you want to get ocr image language

306
00:17:18.400 --> 00:17:20.039
<v Speaker 4>models like all of that stuff. If you want to

307
00:17:20.079 --> 00:17:22.240
<v Speaker 4>do that in your app. I would say there's like

308
00:17:22.240 --> 00:17:25.559
<v Speaker 4>three different tiers of complexity that you can engage in,

309
00:17:26.000 --> 00:17:30.119
<v Speaker 4>and the first one is the new Windows aiapis. And

310
00:17:30.160 --> 00:17:32.480
<v Speaker 4>these were released kind of around the time the Copilot

311
00:17:32.480 --> 00:17:36.839
<v Speaker 4>plus PCs were released, Okayne, and they've been rolling out. Yeah,

312
00:17:36.839 --> 00:17:39.880
<v Speaker 4>they've been rolling out slowly. They were in experimental. You

313
00:17:39.920 --> 00:17:42.400
<v Speaker 4>had to be on the insider preview to build them.

314
00:17:42.400 --> 00:17:44.839
<v Speaker 4>To use them, you have to have a co Pilot

315
00:17:44.839 --> 00:17:48.720
<v Speaker 4>plus PC. But you know, there's a higher bar kind

316
00:17:48.720 --> 00:17:51.839
<v Speaker 4>of on the consumer side, but that means it's easier

317
00:17:51.880 --> 00:17:54.920
<v Speaker 4>on the developer side. So they basically in the code

318
00:17:55.039 --> 00:17:57.680
<v Speaker 4>when you're building, you just have to check does this

319
00:17:57.799 --> 00:18:02.559
<v Speaker 4>device support these APIs? If so, do it very simple

320
00:18:02.640 --> 00:18:04.480
<v Speaker 4>and like that's it. You don't have to manage models,

321
00:18:04.519 --> 00:18:07.319
<v Speaker 4>you don't have to manage memory or downloading, and you

322
00:18:07.319 --> 00:18:10.200
<v Speaker 4>don't have to worry about shipping. You know, a five

323
00:18:10.319 --> 00:18:14.000
<v Speaker 4>gig model with your app. They're already on the device.

324
00:18:14.119 --> 00:18:16.920
<v Speaker 4>If the device supports it, then you can kind of

325
00:18:17.000 --> 00:18:20.640
<v Speaker 4>light up those features, turn on those buttons, show that capability,

326
00:18:20.720 --> 00:18:21.480
<v Speaker 4>and boom, it's there.

327
00:18:21.559 --> 00:18:21.839
<v Speaker 1>Kelly.

328
00:18:21.920 --> 00:18:25.799
<v Speaker 2>My wife bought a new Copilot plus PC. She didn't,

329
00:18:25.839 --> 00:18:27.880
<v Speaker 2>of course know it. We went to best Buy together,

330
00:18:27.960 --> 00:18:30.960
<v Speaker 2>you know, and she picked it out. But the first

331
00:18:30.960 --> 00:18:33.839
<v Speaker 2>thing I did is immediately turned off all this stuff.

332
00:18:34.519 --> 00:18:36.920
<v Speaker 2>It's going to get in the way. The thing that

333
00:18:36.960 --> 00:18:41.200
<v Speaker 2>takes screenshots all the time. I can't remember the name

334
00:18:41.240 --> 00:18:44.559
<v Speaker 2>of it now, recall, recall, that's it. It was turned

335
00:18:44.559 --> 00:18:48.079
<v Speaker 2>off by default. So that's good. That's good. I did

336
00:18:48.079 --> 00:18:48.799
<v Speaker 2>not want that on.

337
00:18:49.279 --> 00:18:51.480
<v Speaker 1>It's a really powerful tool. People love it, you know,

338
00:18:51.720 --> 00:18:53.920
<v Speaker 1>like because the bottom line is you can you can

339
00:18:53.960 --> 00:18:57.119
<v Speaker 1>ask the machine, he where did I see such and such,

340
00:18:57.160 --> 00:18:58.480
<v Speaker 1>and it'll find it for you. Yeah.

341
00:18:58.480 --> 00:19:00.680
<v Speaker 2>I just don't have that kind of problem, like I

342
00:19:00.720 --> 00:19:02.720
<v Speaker 2>know where I saw stuff, and I keep good notes

343
00:19:02.759 --> 00:19:05.400
<v Speaker 2>and dot your machine. Yeah, she didn't want.

344
00:19:05.279 --> 00:19:09.400
<v Speaker 4>It, so yeah, I also don't use it like I

345
00:19:09.559 --> 00:19:12.559
<v Speaker 4>have AI features in well, AI, I should say, I

346
00:19:12.599 --> 00:19:14.839
<v Speaker 4>know this show Richard has talked a lot about how

347
00:19:15.119 --> 00:19:18.000
<v Speaker 4>you have these big amorphous buckets of AI, and then

348
00:19:18.039 --> 00:19:20.799
<v Speaker 4>as soon as you start explaining it and giving a

349
00:19:20.839 --> 00:19:24.039
<v Speaker 4>more clear, straightforward name to it, it stops really being AI.

350
00:19:24.640 --> 00:19:28.400
<v Speaker 4>And that's kind of where the OCR and LLM and

351
00:19:28.799 --> 00:19:32.920
<v Speaker 4>image segmentation and image detection. So those are all under

352
00:19:32.960 --> 00:19:36.960
<v Speaker 4>this umbrella of AI, and it can be a little

353
00:19:37.519 --> 00:19:38.000
<v Speaker 4>I don't know.

354
00:19:38.079 --> 00:19:40.680
<v Speaker 1>You left the impolite part, Joe, which is like, so

355
00:19:40.759 --> 00:19:44.799
<v Speaker 1>for me, the term artificial intelligence means something that doesn't work. Yeah,

356
00:19:44.839 --> 00:19:47.599
<v Speaker 1>there you go, because as soon as it does work,

357
00:19:47.839 --> 00:19:49.160
<v Speaker 1>it gets a new name.

358
00:19:49.200 --> 00:19:52.519
<v Speaker 4>Software, right right, that's it's a module. Yeah, well, I

359
00:19:52.519 --> 00:19:56.160
<v Speaker 4>should say, then the using name space in dot net

360
00:19:56.359 --> 00:19:59.920
<v Speaker 4>is AI. But then after that there's always dot tech

361
00:20:00.279 --> 00:20:04.200
<v Speaker 4>that imaging that image recognition. So there's a bunch of

362
00:20:04.920 --> 00:20:07.880
<v Speaker 4>there's a bunch of APIs after the namespace that actually

363
00:20:08.039 --> 00:20:10.279
<v Speaker 4>point to the real APIs, the real functionality of what

364
00:20:10.319 --> 00:20:14.799
<v Speaker 4>you're actually trying to do. And I don't think you

365
00:20:14.799 --> 00:20:18.079
<v Speaker 4>can easily turn all of that off. I would say,

366
00:20:18.200 --> 00:20:20.720
<v Speaker 4>so there's a lot of experiences that are built on

367
00:20:20.799 --> 00:20:23.880
<v Speaker 4>top of this technology that's already in these Copilot plus PCs,

368
00:20:24.440 --> 00:20:26.759
<v Speaker 4>and you could turn those experiences off. You know, they're

369
00:20:26.799 --> 00:20:29.559
<v Speaker 4>not going to run by default. But Microsoft does a

370
00:20:29.599 --> 00:20:33.319
<v Speaker 4>pretty good job of managing bringing down the model, keeping

371
00:20:33.319 --> 00:20:35.440
<v Speaker 4>it up to date, and making it really easy for

372
00:20:35.599 --> 00:20:38.480
<v Speaker 4>developers to interact with, which is kind of what you want, right,

373
00:20:38.559 --> 00:20:42.000
<v Speaker 4>You want something really simple easy. It's a super complex problem,

374
00:20:42.200 --> 00:20:44.440
<v Speaker 4>but you could just say, you know, send this block

375
00:20:44.480 --> 00:20:46.400
<v Speaker 4>of text, summarize it, and then get it back.

376
00:20:46.759 --> 00:20:48.960
<v Speaker 2>So in case anyone hasn't figured it out by now,

377
00:20:49.000 --> 00:20:53.200
<v Speaker 2>the Copilot plus PC has a local LLM built into it.

378
00:20:53.319 --> 00:20:53.559
<v Speaker 1>Yep.

379
00:20:53.640 --> 00:20:55.759
<v Speaker 2>And you know, this is the kind of thing that

380
00:20:55.839 --> 00:20:57.720
<v Speaker 2>you might think of if you were going to use

381
00:20:57.759 --> 00:21:04.319
<v Speaker 2>OLAMA right and download models and you know, train it,

382
00:21:04.680 --> 00:21:07.519
<v Speaker 2>run it on a laptop or something like that, the

383
00:21:07.559 --> 00:21:08.880
<v Speaker 2>gaming PC or something.

384
00:21:09.079 --> 00:21:10.839
<v Speaker 4>Yeah, there's that's just kind of where I said, there's

385
00:21:10.839 --> 00:21:15.680
<v Speaker 4>like these different layers of the complexity and the easiest, simplest,

386
00:21:15.720 --> 00:21:18.480
<v Speaker 4>like lowest level, easiest for any developer out there to

387
00:21:18.519 --> 00:21:21.200
<v Speaker 4>integrate into their Windows app. Any Windows app by the way,

388
00:21:21.519 --> 00:21:26.559
<v Speaker 4>so WPF or when UI or wind forms you can

389
00:21:26.680 --> 00:21:29.160
<v Speaker 4>or MAUI, you can do them all. It does have

390
00:21:29.200 --> 00:21:33.200
<v Speaker 4>to have identity, some sort of identity because SB there's

391
00:21:33.400 --> 00:21:35.839
<v Speaker 4>Microsoft doesn't want to just open up these APIs to

392
00:21:36.519 --> 00:21:39.960
<v Speaker 4>any random raw ex But if you want to do

393
00:21:40.160 --> 00:21:44.279
<v Speaker 4>some more maybe more niche stuff, maybe a little bit

394
00:21:44.279 --> 00:21:47.559
<v Speaker 4>more complicated stuff, or you want to use this specific model,

395
00:21:47.720 --> 00:21:49.039
<v Speaker 4>you can kind of use what I would call like

396
00:21:49.079 --> 00:21:52.960
<v Speaker 4>the next step of complexity here, and that's win mL

397
00:21:53.000 --> 00:21:54.799
<v Speaker 4>and that's there's a little bit of a middle layer

398
00:21:54.839 --> 00:21:56.799
<v Speaker 4>there where you can go download your own on X

399
00:21:56.880 --> 00:22:00.400
<v Speaker 4>models and run those and it makes it easy. There's

400
00:22:00.440 --> 00:22:04.240
<v Speaker 4>like a basically a standardized interface and you say, run

401
00:22:04.279 --> 00:22:06.920
<v Speaker 4>this model. You don't have to necessarily optimize it for

402
00:22:06.960 --> 00:22:11.279
<v Speaker 4>the specific hardware and it can run CPU, GPU and

403
00:22:11.359 --> 00:22:15.079
<v Speaker 4>PU and it's an easy way. But again, there you

404
00:22:15.200 --> 00:22:18.799
<v Speaker 4>have to manage the model. So if you want that,

405
00:22:18.880 --> 00:22:20.559
<v Speaker 4>if you need that in your application, maybe you have

406
00:22:20.599 --> 00:22:24.759
<v Speaker 4>it specifically fine tuned for your application, or you have

407
00:22:24.799 --> 00:22:29.359
<v Speaker 4>a model that isn't in the box, or I don't

408
00:22:29.359 --> 00:22:31.359
<v Speaker 4>know if there are other legal or.

409
00:22:31.599 --> 00:22:33.759
<v Speaker 1>Hey, I'm just appreciating you're talking about something other than

410
00:22:33.759 --> 00:22:37.960
<v Speaker 1>in the LLM, because it's just it's just overwhelming right now.

411
00:22:38.039 --> 00:22:40.480
<v Speaker 1>So you know, clearly there's a bunch of other models

412
00:22:40.480 --> 00:22:42.519
<v Speaker 1>out there and all of those infrastructure, and I'm including

413
00:22:42.559 --> 00:22:44.759
<v Speaker 1>links to onyx and things like if you haven't looked here,

414
00:22:45.039 --> 00:22:47.920
<v Speaker 1>there's lots of good work being done for specific tasks.

415
00:22:48.319 --> 00:22:51.960
<v Speaker 4>Yeah, and I think immediately people can kind of get

416
00:22:51.960 --> 00:22:55.240
<v Speaker 4>annoyed by, oh, LM, why do I need an LLM

417
00:22:55.240 --> 00:22:58.160
<v Speaker 4>in my model? I'll need AI and it definitely has

418
00:22:58.200 --> 00:23:01.720
<v Speaker 4>become synonymous like AI and LLM. Yeah, but there are

419
00:23:01.799 --> 00:23:03.559
<v Speaker 4>so many If you go to hugging face and you

420
00:23:03.599 --> 00:23:06.799
<v Speaker 4>look at all the different categories, I mean OCR, image segmentation,

421
00:23:06.920 --> 00:23:11.920
<v Speaker 4>image detection, object detection, huggy face, oh yeah, hugging face,

422
00:23:12.200 --> 00:23:15.640
<v Speaker 4>hugging face, hugging face. Yeah, this is a I think

423
00:23:15.759 --> 00:23:19.920
<v Speaker 4>Facebook is kind of backing it. And it's a big

424
00:23:20.440 --> 00:23:25.119
<v Speaker 4>repository for models, so you can access models, you get

425
00:23:25.119 --> 00:23:27.319
<v Speaker 4>download models, and if you're thinking.

426
00:23:27.839 --> 00:23:32.000
<v Speaker 1>Before the insanity of lllms, we had we had good

427
00:23:32.039 --> 00:23:36.279
<v Speaker 1>tooling around just building machine models for object detection and

428
00:23:36.880 --> 00:23:39.960
<v Speaker 1>recognizers and OCR all these good things, right, Like, it's

429
00:23:40.000 --> 00:23:43.160
<v Speaker 1>just there was so much going on before chat GPT

430
00:23:43.279 --> 00:23:45.440
<v Speaker 1>showed up and just overwhelm the message.

431
00:23:45.519 --> 00:23:47.599
<v Speaker 2>Wow, hugging face looks awesome.

432
00:23:47.720 --> 00:23:52.920
<v Speaker 4>Yeah, it's it is a huge, kind of big repository

433
00:23:53.000 --> 00:23:56.440
<v Speaker 4>of models online where you can go download them. But

434
00:23:56.640 --> 00:23:59.319
<v Speaker 4>if you're a normal person who's just curious and says

435
00:23:59.480 --> 00:24:01.319
<v Speaker 4>I want to kind of to try some of these out,

436
00:24:01.680 --> 00:24:03.960
<v Speaker 4>it's not as easy. You can't just download them and

437
00:24:04.000 --> 00:24:07.640
<v Speaker 4>then run them. They are not programs, their models, so

438
00:24:08.000 --> 00:24:11.440
<v Speaker 4>you need to interface with them somehow, and there is

439
00:24:11.680 --> 00:24:15.400
<v Speaker 4>actually a way if you are inclined. You can download

440
00:24:15.440 --> 00:24:19.519
<v Speaker 4>an app from Microsoft called the AI Dev Gallery app.

441
00:24:20.119 --> 00:24:22.880
<v Speaker 4>And what this is it's kind of a playground for

442
00:24:23.039 --> 00:24:25.680
<v Speaker 4>people who are curious about models and different models and

443
00:24:25.720 --> 00:24:28.480
<v Speaker 4>how this all works. It's open source on GitHub, it's

444
00:24:28.519 --> 00:24:31.559
<v Speaker 4>in the Microsoft Store and it is a really low

445
00:24:31.640 --> 00:24:34.400
<v Speaker 4>barrier to entry if you are interested in trying some

446
00:24:34.440 --> 00:24:36.119
<v Speaker 4>of these models out on your own device.

447
00:24:36.240 --> 00:24:36.519
<v Speaker 2>Wow.

448
00:24:36.599 --> 00:24:39.319
<v Speaker 4>So you can download models from hugging Face. You can

449
00:24:39.440 --> 00:24:42.400
<v Speaker 4>run them. They're very limited, basic samples, so don't expect

450
00:24:42.400 --> 00:24:46.279
<v Speaker 4>anything grandiose or chaining them together. But it's a great

451
00:24:46.279 --> 00:24:48.119
<v Speaker 4>way to play with those Hugging Face models.

452
00:24:48.279 --> 00:24:48.599
<v Speaker 2>Very cool.

453
00:24:48.640 --> 00:24:51.920
<v Speaker 1>Did you ever play with Cagle, because we've talked about

454
00:24:51.920 --> 00:24:54.920
<v Speaker 1>this on the show Ages Ago. Just like there is

455
00:24:54.960 --> 00:24:58.759
<v Speaker 1>another playground for practicing your mL skills.

456
00:24:58.680 --> 00:25:00.680
<v Speaker 4>I've never tried. It is in a a website or

457
00:25:00.720 --> 00:25:02.400
<v Speaker 4>a technology.

458
00:25:01.920 --> 00:25:06.599
<v Speaker 1>They actually run competitions for you know. The sort of

459
00:25:06.599 --> 00:25:09.640
<v Speaker 1>famous one for them was the predict how many people

460
00:25:09.680 --> 00:25:13.240
<v Speaker 1>survive the Titanic sinking. There was a bunch of different

461
00:25:14.920 --> 00:25:17.839
<v Speaker 1>models or different competitions, and some of them have a

462
00:25:17.839 --> 00:25:20.440
<v Speaker 1>lot of money in them because they're actually you know,

463
00:25:20.720 --> 00:25:25.119
<v Speaker 1>organizations encourage folks to mature a model particular problem space

464
00:25:25.160 --> 00:25:27.640
<v Speaker 1>that they can then use elsewhere. There was things like

465
00:25:27.759 --> 00:25:35.000
<v Speaker 1>aneurysm detection and even sports predicting. So just again a

466
00:25:35.039 --> 00:25:39.720
<v Speaker 1>reminder that there's things other than llms.

467
00:25:39.839 --> 00:25:43.279
<v Speaker 4>Right, And I would say that is the like the farthest,

468
00:25:43.319 --> 00:25:46.759
<v Speaker 4>the highest tier of integrating AI models into your app,

469
00:25:46.960 --> 00:25:50.799
<v Speaker 4>your local Windows app is making your own models, training

470
00:25:50.880 --> 00:25:53.960
<v Speaker 4>your own models from scratch, So you can do that.

471
00:25:54.039 --> 00:25:57.799
<v Speaker 4>I mean, you can ship models and integrate them directly in.

472
00:25:58.079 --> 00:26:01.920
<v Speaker 4>It's again way more integration work, but it's way more

473
00:26:01.960 --> 00:26:05.000
<v Speaker 4>fine tuned. So if you have a specific application where

474
00:26:05.039 --> 00:26:07.480
<v Speaker 4>you need a model that can do very niche things

475
00:26:07.640 --> 00:26:11.839
<v Speaker 4>or very specific data sets, it's possible. It's doable, and

476
00:26:12.319 --> 00:26:14.000
<v Speaker 4>there's ways to do it. You should check it out.

477
00:26:14.519 --> 00:26:17.759
<v Speaker 4>One of the nice things about this current age of

478
00:26:17.920 --> 00:26:20.480
<v Speaker 4>programming is a lot of these big popular apps are

479
00:26:20.519 --> 00:26:23.160
<v Speaker 4>open source, so you can just see how it's done,

480
00:26:23.440 --> 00:26:25.920
<v Speaker 4>and you obviously read the license, but a lot of

481
00:26:25.920 --> 00:26:29.039
<v Speaker 4>this stuff is available to see how other people are

482
00:26:29.039 --> 00:26:30.880
<v Speaker 4>integrating these AI models.

483
00:26:31.039 --> 00:26:31.319
<v Speaker 1>Guys.

484
00:26:31.559 --> 00:26:33.799
<v Speaker 2>I know we've talked about deep seek a bit on

485
00:26:33.880 --> 00:26:37.759
<v Speaker 2>this show, and Joe's nodding his head, so he knows

486
00:26:37.799 --> 00:26:40.759
<v Speaker 2>about it, and this was the model that came out

487
00:26:40.759 --> 00:26:46.039
<v Speaker 2>of China that uses a lot less resources and is

488
00:26:46.079 --> 00:26:50.319
<v Speaker 2>therefore cheaper to run than you know, chat GPT was,

489
00:26:50.400 --> 00:26:52.839
<v Speaker 2>and everybody was like, oh my god, open ai is

490
00:26:52.880 --> 00:26:56.359
<v Speaker 2>going down, and it didn't. And then there were concerns

491
00:26:56.359 --> 00:27:02.279
<v Speaker 2>about you know, if I use deep Seek, am I

492
00:27:02.319 --> 00:27:07.160
<v Speaker 2>sharing data with you know, the country of China and

493
00:27:07.960 --> 00:27:10.039
<v Speaker 2>is it safe in all of these things. But you

494
00:27:10.119 --> 00:27:13.160
<v Speaker 2>can also I think, correct me if I'm wrong, but

495
00:27:13.799 --> 00:27:17.799
<v Speaker 2>download it the app and run it locally like olama.

496
00:27:18.240 --> 00:27:18.759
<v Speaker 2>Is that true?

497
00:27:18.920 --> 00:27:19.240
<v Speaker 1>Yeah?

498
00:27:19.400 --> 00:27:23.279
<v Speaker 4>That So one of the nice things about deepseek is

499
00:27:23.279 --> 00:27:27.359
<v Speaker 4>how small it is. But they also have NPU optimized

500
00:27:27.599 --> 00:27:31.640
<v Speaker 4>models which you can go download and there's also an

501
00:27:31.640 --> 00:27:33.279
<v Speaker 4>extension for vs code.

502
00:27:33.440 --> 00:27:36.519
<v Speaker 2>Wait wait, go back to the is M or NPU

503
00:27:36.880 --> 00:27:38.160
<v Speaker 2>and what is that?

504
00:27:38.160 --> 00:27:41.200
<v Speaker 4>That's the neural processing unit. So you kind of have

505
00:27:41.279 --> 00:27:44.400
<v Speaker 4>your CPU, your GPU, and your NPU.

506
00:27:44.839 --> 00:27:45.519
<v Speaker 1>And this was.

507
00:27:45.480 --> 00:27:50.559
<v Speaker 4>The core the chip, the part of the CPU in

508
00:27:50.599 --> 00:27:54.000
<v Speaker 4>these ARM devices that really made it easy to run

509
00:27:54.039 --> 00:27:56.200
<v Speaker 4>these models locally and efficiently.

510
00:27:56.200 --> 00:27:59.000
<v Speaker 1>Okay, part of the requirement for a copilot plus PCs

511
00:27:59.000 --> 00:28:01.160
<v Speaker 1>that it has an MPU of at least what is it,

512
00:28:01.240 --> 00:28:04.839
<v Speaker 1>forty tops or trillion operations per second.

513
00:28:04.920 --> 00:28:07.440
<v Speaker 2>So if you have a copile plus PC, you can

514
00:28:07.720 --> 00:28:10.920
<v Speaker 2>download deep Seek and use it even if you don't,

515
00:28:10.960 --> 00:28:13.039
<v Speaker 2>and you're probably going to get good results.

516
00:28:13.519 --> 00:28:16.839
<v Speaker 4>Yeah, you don't have to have a NPU, but a

517
00:28:16.920 --> 00:28:20.759
<v Speaker 4>lot of these models. So Microsoft makes a LM called

518
00:28:20.920 --> 00:28:26.480
<v Speaker 4>five Silica, and this model they have they've been releasing three,

519
00:28:26.680 --> 00:28:29.920
<v Speaker 4>three point five, they just released four. It's optimized for

520
00:28:29.960 --> 00:28:33.720
<v Speaker 4>the CPU and the GPU and not the NPU right now,

521
00:28:34.000 --> 00:28:36.519
<v Speaker 4>at least the models that they've released, and there are

522
00:28:36.599 --> 00:28:39.119
<v Speaker 4>models out there that you can get that are optimized

523
00:28:39.119 --> 00:28:41.000
<v Speaker 4>for the NPU. So if you do have a device

524
00:28:41.400 --> 00:28:44.400
<v Speaker 4>that is OM device or low power device and you

525
00:28:44.440 --> 00:28:46.720
<v Speaker 4>want more of an optimized model, you can find them

526
00:28:47.319 --> 00:28:50.039
<v Speaker 4>and run them. And you can also do that in

527
00:28:50.160 --> 00:28:54.079
<v Speaker 4>VS code. There's an extension called AI Toolkit for Visual

528
00:28:54.079 --> 00:28:58.920
<v Speaker 4>Studio Code, and that's another kind of playground esque place,

529
00:28:59.039 --> 00:29:02.480
<v Speaker 4>but you can also do the model refinement and fine

530
00:29:02.519 --> 00:29:06.119
<v Speaker 4>tuning in there. So there's a lot of ways that

531
00:29:06.160 --> 00:29:09.440
<v Speaker 4>you can experiment with these models without really being a pro.

532
00:29:09.920 --> 00:29:12.680
<v Speaker 4>So if you're just curious and you have a lot

533
00:29:12.720 --> 00:29:14.359
<v Speaker 4>of hard drive space, that is the one thing that

534
00:29:14.400 --> 00:29:18.119
<v Speaker 4>I'll say, I recently upgraded my surface hard drive from

535
00:29:18.119 --> 00:29:21.160
<v Speaker 4>a five to twelve to a two terabyte because these

536
00:29:21.240 --> 00:29:25.400
<v Speaker 4>models are big and if you want accurate ones, they're

537
00:29:25.759 --> 00:29:26.359
<v Speaker 4>very large.

538
00:29:26.440 --> 00:29:29.240
<v Speaker 2>I just saw Richard probably knows about this, but there

539
00:29:29.240 --> 00:29:34.319
<v Speaker 2>are now twenty two terabyte SSD drives. Yeah, for like

540
00:29:34.359 --> 00:29:38.039
<v Speaker 2>around five hundred bucks. Can you wrap your mind around that.

541
00:29:38.119 --> 00:29:39.279
<v Speaker 1>It's a lot of storage.

542
00:29:39.359 --> 00:29:43.240
<v Speaker 2>Oh my goodness, Like me know, Joe's like is shaking

543
00:29:43.279 --> 00:29:44.359
<v Speaker 2>his head, like what.

544
00:29:44.759 --> 00:29:47.000
<v Speaker 4>One drive twenty two terramytes.

545
00:29:46.559 --> 00:29:49.279
<v Speaker 2>Twenty two terabyte SSD five hundred bucks?

546
00:29:49.359 --> 00:29:50.599
<v Speaker 4>You should that's not a typeout.

547
00:29:51.119 --> 00:29:53.359
<v Speaker 2>No, there's a couple of different brands.

548
00:29:53.440 --> 00:29:54.039
<v Speaker 4>That's amazing.

549
00:29:54.119 --> 00:29:56.880
<v Speaker 1>Yeah, ridiculous, Yeah, that really is. I think I should.

550
00:29:56.920 --> 00:29:59.200
<v Speaker 1>I don't think they are sists. I think they're spinning drives.

551
00:29:59.279 --> 00:30:04.680
<v Speaker 1>Oh really two terabytes? Yeah SSDs the solid state ones

552
00:30:04.720 --> 00:30:05.680
<v Speaker 1>and aren't that big yet?

553
00:30:05.759 --> 00:30:06.119
<v Speaker 2>Okay?

554
00:30:06.240 --> 00:30:10.359
<v Speaker 1>The still twenty two terabytes is madness? Like that's just

555
00:30:10.400 --> 00:30:11.240
<v Speaker 1>a lot of storage.

556
00:30:11.400 --> 00:30:15.960
<v Speaker 4>Yeah, it really is. And the AI Toolkit and vs

557
00:30:15.960 --> 00:30:20.559
<v Speaker 4>code does allow you to interact with these llms through

558
00:30:20.559 --> 00:30:23.240
<v Speaker 4>the web, and so GitHub will host some of these models,

559
00:30:23.279 --> 00:30:25.839
<v Speaker 4>other providers will host them, and so you can kind

560
00:30:25.839 --> 00:30:31.480
<v Speaker 4>of do comparisons. So there's the local foundry, and that's

561
00:30:31.759 --> 00:30:34.759
<v Speaker 4>what Microsoft has branded there. You know, I've called it,

562
00:30:34.759 --> 00:30:36.880
<v Speaker 4>I think the second tier kind of where you have

563
00:30:37.880 --> 00:30:40.240
<v Speaker 4>win mL and you have your local models and you're

564
00:30:40.400 --> 00:30:43.799
<v Speaker 4>doing that work. So you have your local models and

565
00:30:43.880 --> 00:30:46.839
<v Speaker 4>you can compare those two cloud hosted models and test

566
00:30:46.880 --> 00:30:48.960
<v Speaker 4>them because again, you know software, you have to be

567
00:30:49.000 --> 00:30:52.000
<v Speaker 4>able to test it. So it is hard too with

568
00:30:52.119 --> 00:30:54.519
<v Speaker 4>these how do you compare them? Like, which one's good,

569
00:30:54.519 --> 00:30:56.519
<v Speaker 4>which one's bad? Is it good enough? Is it good

570
00:30:56.599 --> 00:30:58.480
<v Speaker 4>enough in our use cases? And it can be tedious

571
00:30:58.519 --> 00:31:01.480
<v Speaker 4>to test manually. But there are a lot of tools

572
00:31:01.480 --> 00:31:06.079
<v Speaker 4>out there to experiment, get started, and if anybody's curious,

573
00:31:06.079 --> 00:31:09.799
<v Speaker 4>I definitely you should check out the aidev gallery for sure.

574
00:31:10.200 --> 00:31:11.759
<v Speaker 4>That is a lot of fun to play around with

575
00:31:11.799 --> 00:31:16.200
<v Speaker 4>those different models and for a little bit more advanced scenarios,

576
00:31:16.480 --> 00:31:21.279
<v Speaker 4>what more language focused. The AI toolkit in vs code

577
00:31:21.319 --> 00:31:24.160
<v Speaker 4>is another really fun I'm looking at deep seak here

578
00:31:24.279 --> 00:31:27.279
<v Speaker 4>right now. You can download it on your device and

579
00:31:27.359 --> 00:31:27.720
<v Speaker 4>run it.

580
00:31:27.839 --> 00:31:30.039
<v Speaker 2>Wow, it seems like a pretty good place to take

581
00:31:30.079 --> 00:31:32.480
<v Speaker 2>a break. So we'll be right back after these very

582
00:31:32.839 --> 00:31:33.839
<v Speaker 2>important messages.

583
00:31:34.000 --> 00:31:34.559
<v Speaker 1>Stay tuned.

584
00:31:36.839 --> 00:31:39.559
<v Speaker 2>You know, dot net six has officially reached the end

585
00:31:39.599 --> 00:31:42.839
<v Speaker 2>of support and now is the time to upgrade. Dot

586
00:31:42.880 --> 00:31:46.519
<v Speaker 2>Net eight is well supported on AWS. Learn more at

587
00:31:46.559 --> 00:31:50.480
<v Speaker 2>aws dot Amazon dot com, slash dot net.

588
00:31:53.440 --> 00:31:55.559
<v Speaker 1>And we're back. It's don that Rocks emergor Campbell. Let's

589
00:31:55.559 --> 00:31:58.359
<v Speaker 1>call Franklin. You talking a bit to our friend Joe

590
00:31:58.599 --> 00:32:03.720
<v Speaker 1>about work with local models and also and the non

591
00:32:04.039 --> 00:32:06.119
<v Speaker 1>LLM stuff just sort of a good reminder there's been

592
00:32:06.160 --> 00:32:08.519
<v Speaker 1>all kinds of cool stuff going on in the mL

593
00:32:08.599 --> 00:32:11.720
<v Speaker 1>space that didn't necessarily have to do with language per se.

594
00:32:12.279 --> 00:32:15.160
<v Speaker 1>But you know, you've you've hinted this a couple of

595
00:32:15.160 --> 00:32:17.640
<v Speaker 1>times in the first half. It's like, if you want

596
00:32:17.680 --> 00:32:21.119
<v Speaker 1>to own the model, you know, there's a lot of

597
00:32:21.200 --> 00:32:24.440
<v Speaker 1>models available to download from hugey face and all these

598
00:32:24.480 --> 00:32:27.519
<v Speaker 1>other places. Why would you want to own a model

599
00:32:27.559 --> 00:32:30.000
<v Speaker 1>because it sounds like a lot of work. It's like

600
00:32:30.119 --> 00:32:31.039
<v Speaker 1>owning a framework.

601
00:32:31.119 --> 00:32:34.119
<v Speaker 4>Yeah, yeah, it is like, don't trust somebody who says

602
00:32:34.160 --> 00:32:36.400
<v Speaker 4>they can write their own language and write their own

603
00:32:36.440 --> 00:32:38.119
<v Speaker 4>ide You're like, oh.

604
00:32:38.240 --> 00:32:41.880
<v Speaker 1>Their own garbage collector, you know, their own crypto library.

605
00:32:42.359 --> 00:32:44.480
<v Speaker 1>Like these are all scary things to me. So when

606
00:32:44.480 --> 00:32:46.359
<v Speaker 1>someone says I'll just make our own model, I'm like,

607
00:32:46.640 --> 00:32:47.799
<v Speaker 1>why do we need to do that?

608
00:32:48.400 --> 00:32:52.160
<v Speaker 4>Well, if you're in the industry. If you have insane

609
00:32:52.200 --> 00:32:56.920
<v Speaker 4>amounts of data and a niche in a specific industry,

610
00:32:57.960 --> 00:33:00.000
<v Speaker 4>it might be worth it for you to look into

611
00:33:00.160 --> 00:33:03.240
<v Speaker 4>doing this. And if you have a hard time processing

612
00:33:03.319 --> 00:33:05.960
<v Speaker 4>large amounts of data to get insights and actions out

613
00:33:06.000 --> 00:33:09.039
<v Speaker 4>of it, which is kind of the idea here, right,

614
00:33:09.039 --> 00:33:11.960
<v Speaker 4>what you have an entire language that you have to

615
00:33:11.960 --> 00:33:14.319
<v Speaker 4>train these models on, or you have an entire data

616
00:33:14.319 --> 00:33:18.079
<v Speaker 4>set of images with boxes drawn around the dogs or

617
00:33:18.200 --> 00:33:22.599
<v Speaker 4>dog breeds or very specific things like that. If that's

618
00:33:22.680 --> 00:33:24.799
<v Speaker 4>what you need to do, is something where it's not

619
00:33:24.839 --> 00:33:29.519
<v Speaker 4>available or it's not good enough, there's really no other

620
00:33:29.519 --> 00:33:31.680
<v Speaker 4>way around it than to build your own model today.

621
00:33:32.319 --> 00:33:33.519
<v Speaker 4>But it really is that data.

622
00:33:33.559 --> 00:33:37.119
<v Speaker 1>It's I mean that being said, this is all sort

623
00:33:37.160 --> 00:33:39.440
<v Speaker 1>of non terministic thing, like you're never going to get

624
00:33:39.440 --> 00:33:41.799
<v Speaker 1>one hundred percent out of a machine learning model.

625
00:33:41.839 --> 00:33:45.839
<v Speaker 4>It's probabilistic, right, absolutely, even maybe especially so some of

626
00:33:45.880 --> 00:33:48.519
<v Speaker 4>the image detection ones, and a lot of times they'll

627
00:33:48.519 --> 00:33:53.000
<v Speaker 4>give you back a number a fraction of confidence, and

628
00:33:53.039 --> 00:33:54.839
<v Speaker 4>I think maybe this is why they don't get as

629
00:33:54.960 --> 00:33:58.000
<v Speaker 4>much play as they're not as exciting for individuals to use.

630
00:33:58.519 --> 00:34:01.200
<v Speaker 4>It's like the could take a picture of your cat

631
00:34:01.359 --> 00:34:03.759
<v Speaker 4>and then your phone will draw a box around it

632
00:34:03.759 --> 00:34:06.279
<v Speaker 4>and say that's a cat. Yep, that's a cat. So

633
00:34:06.640 --> 00:34:09.119
<v Speaker 4>I think it's a lot less interesting. The language ones

634
00:34:09.199 --> 00:34:11.639
<v Speaker 4>just kind of capture people's imagination and there's a lot

635
00:34:11.679 --> 00:34:14.079
<v Speaker 4>more back and forth. But when you really think about

636
00:34:14.079 --> 00:34:16.920
<v Speaker 4>building an application, like what are you doing? Maybe you

637
00:34:17.000 --> 00:34:20.760
<v Speaker 4>have a you're playing around with your Raspberry Pie as

638
00:34:20.760 --> 00:34:22.519
<v Speaker 4>a security system for your house, and you want to

639
00:34:22.519 --> 00:34:25.159
<v Speaker 4>add a vision system and you want to do box

640
00:34:25.199 --> 00:34:28.239
<v Speaker 4>detection and you have hours and hours and hours and

641
00:34:28.280 --> 00:34:31.320
<v Speaker 4>hours of security footage. Or maybe you have a specific

642
00:34:31.440 --> 00:34:34.199
<v Speaker 4>niche application where you're trying to, you know, detect a

643
00:34:34.239 --> 00:34:37.519
<v Speaker 4>particular squirrel who's given you trouble. It's a fun you know,

644
00:34:37.559 --> 00:34:38.760
<v Speaker 4>it's a fun experiment and you.

645
00:34:38.719 --> 00:34:40.599
<v Speaker 1>Can do a bear or a bear.

646
00:34:40.880 --> 00:34:43.480
<v Speaker 2>Joe, do you have a toi less squirrel bird feeder?

647
00:34:44.000 --> 00:34:44.239
<v Speaker 1>No?

648
00:34:44.320 --> 00:34:47.800
<v Speaker 2>I do not seeing this YouTube? Check YouTube for toil

649
00:34:47.840 --> 00:34:51.880
<v Speaker 2>less squirrel terrible right. It's basically it goes between you

650
00:34:51.960 --> 00:34:54.119
<v Speaker 2>know what you hang the bird feeder on and the

651
00:34:54.119 --> 00:34:56.320
<v Speaker 2>bird feeder, so it's got a hook on either side.

652
00:34:56.840 --> 00:35:01.000
<v Speaker 2>It detects weight and so when there's a squirrel on it,

653
00:35:01.000 --> 00:35:04.679
<v Speaker 2>it just starts spinning and the squirrels go flying. It's

654
00:35:04.760 --> 00:35:06.280
<v Speaker 2>hilarious to whirl the squirrel.

655
00:35:06.400 --> 00:35:10.000
<v Speaker 4>Yeah, that you could build an AI powered twirl a squirrel.

656
00:35:10.159 --> 00:35:12.199
<v Speaker 1>There you go, There you go. I don't think that's necessary.

657
00:35:12.239 --> 00:35:15.960
<v Speaker 1>I am thinking about animal recognition this particular part of

658
00:35:15.960 --> 00:35:18.079
<v Speaker 1>the world where you know. The one that would be

659
00:35:18.119 --> 00:35:20.960
<v Speaker 1>tricky that I would really challenge myself would be whale

660
00:35:20.960 --> 00:35:23.440
<v Speaker 1>detection because we've had you know, you don't have a

661
00:35:23.440 --> 00:35:25.679
<v Speaker 1>lot of time to pick up on the fact that

662
00:35:25.760 --> 00:35:28.599
<v Speaker 1>there's whale blow, like they're going by, and it could

663
00:35:28.599 --> 00:35:30.440
<v Speaker 1>be orcers and it could be humpbacks, and it could

664
00:35:30.480 --> 00:35:32.559
<v Speaker 1>be grays, and it could be porpoises, and it could

665
00:35:32.559 --> 00:35:34.960
<v Speaker 1>be dolphins. Like you have to be a lot of

666
00:35:34.960 --> 00:35:37.599
<v Speaker 1>stuff going on. You have to be on the surface.

667
00:35:38.159 --> 00:35:41.519
<v Speaker 1>We hear no, no, we hear them like we hear

668
00:35:41.599 --> 00:35:44.519
<v Speaker 1>whale blow before we see the whale because it travels

669
00:35:44.679 --> 00:35:46.599
<v Speaker 1>like when they when they exhale its loud.

670
00:35:46.599 --> 00:35:48.960
<v Speaker 2>Well, you could identify a whale by the sounds it's

671
00:35:49.000 --> 00:35:49.480
<v Speaker 2>making too.

672
00:35:49.559 --> 00:35:53.280
<v Speaker 1>Yeah, I wonder. Yeah, speaking of it still seems nuts

673
00:35:53.320 --> 00:35:55.199
<v Speaker 1>to build your own model like that just seems like

674
00:35:55.239 --> 00:35:56.159
<v Speaker 1>a thing I don't want to own.

675
00:35:56.239 --> 00:35:59.280
<v Speaker 4>Yeah, it's it's definitely the research side of things. And

676
00:35:59.480 --> 00:36:01.719
<v Speaker 4>I know people have been saying for a long time

677
00:36:01.960 --> 00:36:05.199
<v Speaker 4>that data is the new oil, right, this is the

678
00:36:05.199 --> 00:36:08.199
<v Speaker 4>new black gold of do you have the data? Do

679
00:36:08.280 --> 00:36:12.920
<v Speaker 4>you have the databases? Is it structured, is it consistent,

680
00:36:13.079 --> 00:36:16.000
<v Speaker 4>is it clean? Is it real? Is it good? And

681
00:36:16.039 --> 00:36:19.119
<v Speaker 4>if you have all that, I think we have a very

682
00:36:19.159 --> 00:36:21.119
<v Speaker 4>small number of people who can say yes, we have

683
00:36:21.199 --> 00:36:22.840
<v Speaker 4>that right and you don't have to spend all that

684
00:36:22.880 --> 00:36:25.760
<v Speaker 4>time cleaning the data, which is such a challenge where

685
00:36:25.880 --> 00:36:29.280
<v Speaker 4>you have so much noise in the data today that

686
00:36:29.320 --> 00:36:30.920
<v Speaker 4>if you're trying to train a model, Yeah.

687
00:36:31.000 --> 00:36:32.760
<v Speaker 2>If how I was going to use a local LM,

688
00:36:33.039 --> 00:36:37.320
<v Speaker 2>I would want it to understand C sharp, JavaScript, Blazer,

689
00:36:37.880 --> 00:36:42.320
<v Speaker 2>you know, and CSS. That's and I don't know how

690
00:36:42.679 --> 00:36:46.599
<v Speaker 2>realistic that is. Like I know that the current models

691
00:36:46.639 --> 00:36:49.519
<v Speaker 2>like Claude's on it, and you know even chat GPT

692
00:36:49.800 --> 00:36:53.559
<v Speaker 2>understand it. But for lack of a better word, sorry, Richard,

693
00:36:53.559 --> 00:36:57.719
<v Speaker 2>didn't mean to offend you. There. They're programmed, you know,

694
00:36:58.000 --> 00:37:01.239
<v Speaker 2>they're they're trained against it. But what does it take

695
00:37:01.679 --> 00:37:04.719
<v Speaker 2>to do that locally, to train the models to train well,

696
00:37:04.880 --> 00:37:08.079
<v Speaker 2>or to get a model that understands you know, programmers

697
00:37:08.079 --> 00:37:10.119
<v Speaker 2>speak languages and stuff they do.

698
00:37:10.239 --> 00:37:13.360
<v Speaker 4>Yeah, local models will and they can write code. I

699
00:37:13.360 --> 00:37:17.079
<v Speaker 4>think part of the challenge that you'll see if you

700
00:37:17.079 --> 00:37:20.000
<v Speaker 4>start using them is speed. So the response speed of

701
00:37:20.039 --> 00:37:22.639
<v Speaker 4>a local model is going to be much slower actually

702
00:37:22.679 --> 00:37:26.519
<v Speaker 4>than a cloud hosted one because your computer cannot compete

703
00:37:26.519 --> 00:37:29.800
<v Speaker 4>with a server with a rack of GPUs. Yeah, well

704
00:37:29.960 --> 00:37:31.559
<v Speaker 4>maybe yours, Carl, not mine.

705
00:37:31.599 --> 00:37:34.039
<v Speaker 2>Oh, I don't know. I don't think so. But you know,

706
00:37:34.239 --> 00:37:37.239
<v Speaker 2>I think if I had a great Copilot plus PC,

707
00:37:38.280 --> 00:37:41.679
<v Speaker 2>you know, with a lot of RAM and a lot

708
00:37:41.679 --> 00:37:45.280
<v Speaker 2>of storage, and I just set it over in a

709
00:37:45.280 --> 00:37:47.039
<v Speaker 2>closet somewhere, I could probably use that.

710
00:37:47.199 --> 00:37:48.079
<v Speaker 4>Yeah, you should try it.

711
00:37:48.280 --> 00:37:48.639
<v Speaker 1>Yeah.

712
00:37:48.920 --> 00:37:51.800
<v Speaker 4>Another challenge is going to be context, which is how

713
00:37:51.840 --> 00:37:54.920
<v Speaker 4>big of a context window can the model actually hold

714
00:37:54.960 --> 00:37:57.119
<v Speaker 4>in the provider there's all of that, there's a lot

715
00:37:57.159 --> 00:38:01.280
<v Speaker 4>of infrastructure in between the model and actually getting stuff out.

716
00:38:01.320 --> 00:38:03.239
<v Speaker 4>So speeding context, I would say, are going to be

717
00:38:03.280 --> 00:38:06.039
<v Speaker 4>your biggest risks where you don't necessarily just want it

718
00:38:06.039 --> 00:38:09.800
<v Speaker 4>to give you new greenfield CSS. You want it to

719
00:38:09.800 --> 00:38:13.760
<v Speaker 4>give you new CSS in the right spot for your codings.

720
00:38:14.000 --> 00:38:14.480
<v Speaker 4>Which is that?

721
00:38:14.519 --> 00:38:15.840
<v Speaker 1>And I want a much harder question.

722
00:38:15.920 --> 00:38:18.239
<v Speaker 2>I wanted to remember everything we've said, Like I want

723
00:38:18.280 --> 00:38:21.079
<v Speaker 2>as big a context as I can possibly get. So

724
00:38:21.320 --> 00:38:24.599
<v Speaker 2>is that just a measure of more RAM or is

725
00:38:24.599 --> 00:38:27.880
<v Speaker 2>it the more that context you have, the slower it's

726
00:38:27.920 --> 00:38:31.239
<v Speaker 2>going to be to come up with a new answer.

727
00:38:31.320 --> 00:38:33.519
<v Speaker 4>Yeah, that's a good question. I would love to hear

728
00:38:33.679 --> 00:38:37.599
<v Speaker 4>an expert who actually knows more about context and how

729
00:38:37.679 --> 00:38:39.840
<v Speaker 4>that differs from the training data and how it differs

730
00:38:39.840 --> 00:38:45.159
<v Speaker 4>from fine tuning, because in my experiences with local AI,

731
00:38:45.280 --> 00:38:47.800
<v Speaker 4>I have a pretty narrow context window that you could

732
00:38:47.840 --> 00:38:50.480
<v Speaker 4>basically feed it, Hey, here's everything I know, and you

733
00:38:50.519 --> 00:38:53.599
<v Speaker 4>feed it with the prompt yeah, and you say okay,

734
00:38:53.719 --> 00:38:55.440
<v Speaker 4>now do this and then give it back to me.

735
00:38:55.760 --> 00:38:57.880
<v Speaker 4>But you're not feeding it documents.

736
00:38:57.960 --> 00:38:59.760
<v Speaker 1>The thing that's made a difference for me has been

737
00:38:59.800 --> 00:39:02.119
<v Speaker 1>the video card and the amount of memory in the

738
00:39:02.159 --> 00:39:04.280
<v Speaker 1>video card, Like playing with frame Pack and a couple

739
00:39:04.320 --> 00:39:07.559
<v Speaker 1>of other models, and so I'm running a fifty eighty

740
00:39:07.559 --> 00:39:11.239
<v Speaker 1>with sixteen gigs of v RAM, and that has made

741
00:39:11.239 --> 00:39:14.239
<v Speaker 1>a huge difference for running bigger models. No, I'm not

742
00:39:14.239 --> 00:39:17.480
<v Speaker 1>talking about building models, but actually executing a more complex workload.

743
00:39:18.239 --> 00:39:20.440
<v Speaker 1>And if you have got the money to spend, because

744
00:39:20.480 --> 00:39:24.239
<v Speaker 1>they're thousands of dollars like those top in RTX cards.

745
00:39:24.239 --> 00:39:26.559
<v Speaker 1>Now you can get ninety six gigs in them. Jeez,

746
00:39:26.599 --> 00:39:29.559
<v Speaker 1>it's a ten thousand dollars card. But you know that

747
00:39:29.719 --> 00:39:32.000
<v Speaker 1>seems to be the thing that makes the most difference

748
00:39:32.320 --> 00:39:34.000
<v Speaker 1>for a lot of these kinds of tools when you

749
00:39:34.039 --> 00:39:35.320
<v Speaker 1>want to kindle a lot of contact.

750
00:39:35.400 --> 00:39:38.079
<v Speaker 2>What about an NPU? Is that gonna do it less

751
00:39:38.079 --> 00:39:40.199
<v Speaker 2>than more than a ten thousand dollars video card.

752
00:39:40.360 --> 00:39:43.239
<v Speaker 1>No, because there's just no You know they talk about

753
00:39:43.320 --> 00:39:46.239
<v Speaker 1>that Copilot plus PC has forty tops. I don't know

754
00:39:46.239 --> 00:39:49.199
<v Speaker 1>what that means. Yeah, that's the trend trilling operation per second.

755
00:39:49.199 --> 00:39:52.920
<v Speaker 1>It's the measure of its compute power for neural nets. Okay,

756
00:39:53.000 --> 00:39:55.760
<v Speaker 1>my fifty eighty has thirteen hundred TOP. I see so.

757
00:39:56.320 --> 00:39:58.320
<v Speaker 1>And when you look at what Nvidious selling the data

758
00:39:58.320 --> 00:40:01.360
<v Speaker 1>centers and things, is their giant GPU like that with

759
00:40:01.519 --> 00:40:03.920
<v Speaker 1>huge amounts of memory, this super fast memory and them

760
00:40:03.920 --> 00:40:04.960
<v Speaker 1>for scale processing.

761
00:40:05.079 --> 00:40:07.119
<v Speaker 4>Yeah, the NPU, I think was more of a play

762
00:40:07.360 --> 00:40:12.320
<v Speaker 4>for a continuous operation or in the background and on

763
00:40:12.599 --> 00:40:16.159
<v Speaker 4>mobile devices where battery and power consumption is a much

764
00:40:16.199 --> 00:40:19.480
<v Speaker 4>bigger concern for individuals, where they're thinking, well, I don't

765
00:40:19.519 --> 00:40:22.840
<v Speaker 4>want this GPU chugging away in the background. Can I

766
00:40:22.880 --> 00:40:25.079
<v Speaker 4>get something? Can I get something good enough, and that's

767
00:40:25.159 --> 00:40:28.400
<v Speaker 4>kind of where that minimum bar is that doesn't absolutely

768
00:40:28.440 --> 00:40:30.840
<v Speaker 4>consume my battery life. You know, you open your computer

769
00:40:30.920 --> 00:40:32.760
<v Speaker 4>up and it's like, hey, I was working in the

770
00:40:32.760 --> 00:40:34.639
<v Speaker 4>background seeing if anything was happening.

771
00:40:35.119 --> 00:40:37.880
<v Speaker 1>No, thank you. Yeah. Yeah. And it's been an argument

772
00:40:37.960 --> 00:40:42.280
<v Speaker 1>now that you can jack up a PC enough with

773
00:40:42.400 --> 00:40:44.639
<v Speaker 1>those with a couple of those big GPUs and run

774
00:40:44.679 --> 00:40:48.280
<v Speaker 1>a mid size LLM on it. So you know, certainly,

775
00:40:48.320 --> 00:40:50.360
<v Speaker 1>I've had conversations with folks where it's like, I am

776
00:40:50.400 --> 00:40:52.840
<v Speaker 1>not prepared to send any of this data to the cloud.

777
00:40:53.280 --> 00:40:56.199
<v Speaker 1>What can I do one hundred percent local? Yeah.

778
00:40:56.199 --> 00:40:58.400
<v Speaker 4>Another thing that you do have to consider if you're

779
00:40:58.400 --> 00:41:01.000
<v Speaker 4>going to get into building and those apps are especially

780
00:41:01.039 --> 00:41:05.679
<v Speaker 4>local apps, is the idea of multi modal. Yeah, these models,

781
00:41:06.000 --> 00:41:10.199
<v Speaker 4>these local models, at least the Windows aiapis are not multimodel,

782
00:41:10.480 --> 00:41:11.519
<v Speaker 4>so you will have to.

783
00:41:11.519 --> 00:41:14.079
<v Speaker 2>In other words, you can't talk to them and write

784
00:41:14.159 --> 00:41:16.039
<v Speaker 2>to them exactly. Is that what you mean?

785
00:41:16.320 --> 00:41:16.559
<v Speaker 1>Right?

786
00:41:16.599 --> 00:41:18.360
<v Speaker 4>So you're going to have to build that. I mean

787
00:41:18.360 --> 00:41:20.400
<v Speaker 4>you could, but you're going to have to put a

788
00:41:20.599 --> 00:41:24.519
<v Speaker 4>speech recognition model in front of the LM or a

789
00:41:24.599 --> 00:41:28.559
<v Speaker 4>object detection model plus an OCR model plus that you know,

790
00:41:28.599 --> 00:41:30.880
<v Speaker 4>you have to maybe chain these models together and then

791
00:41:30.960 --> 00:41:35.000
<v Speaker 4>you can get that multimodal experience where you can drop images,

792
00:41:35.039 --> 00:41:36.719
<v Speaker 4>you can put PDFs in, but you have to be

793
00:41:36.760 --> 00:41:39.840
<v Speaker 4>able to read the PDF. So these lllms don't read

794
00:41:39.880 --> 00:41:43.760
<v Speaker 4>PDFs by default locally. You do have to get them

795
00:41:43.760 --> 00:41:46.800
<v Speaker 4>into a text format. So if you're thinking about how

796
00:41:46.960 --> 00:41:48.639
<v Speaker 4>you can apply this into your work, and I know

797
00:41:48.679 --> 00:41:51.440
<v Speaker 4>a lot of enterprises, a lot of companies, a lot

798
00:41:51.440 --> 00:41:55.119
<v Speaker 4>of their data is not in raw text format, so

799
00:41:55.239 --> 00:41:56.239
<v Speaker 4>you do have to get it there.

800
00:41:56.320 --> 00:41:59.880
<v Speaker 1>Yeah, but there's an MCP for PDFs. So you know,

801
00:42:00.880 --> 00:42:02.480
<v Speaker 1>glue these bits together.

802
00:42:02.440 --> 00:42:04.199
<v Speaker 4>Right, yeap, but you will have to do the gluing.

803
00:42:04.719 --> 00:42:05.800
<v Speaker 4>Some assembly required.

804
00:42:05.880 --> 00:42:08.599
<v Speaker 1>This is the job, right, Like, this is not just

805
00:42:08.639 --> 00:42:12.480
<v Speaker 1>an app you run, but we are assembling parts to

806
00:42:12.559 --> 00:42:15.159
<v Speaker 1>try and get to a place where a model could

807
00:42:15.159 --> 00:42:15.559
<v Speaker 1>be built.

808
00:42:15.639 --> 00:42:19.039
<v Speaker 2>So if you were going to build a local LLM

809
00:42:19.880 --> 00:42:25.679
<v Speaker 2>Joe yourself using some existing technology, would you first reach

810
00:42:25.719 --> 00:42:28.320
<v Speaker 2>for deep seek or would you go for just the

811
00:42:28.360 --> 00:42:30.760
<v Speaker 2>stuff that Microsoft is exposing in Windows.

812
00:42:31.039 --> 00:42:33.760
<v Speaker 4>Yeah, I just reach for this stuff or a Microsoft

813
00:42:33.800 --> 00:42:37.800
<v Speaker 4>is exposing in Windows and their five model. It's pretty good,

814
00:42:38.360 --> 00:42:42.679
<v Speaker 4>it's pretty robust, and I would say it's a nice

815
00:42:42.880 --> 00:42:46.960
<v Speaker 4>middle middle ground there for building on top of and

816
00:42:47.000 --> 00:42:51.239
<v Speaker 4>fine tuning. I don't have enough time to be building

817
00:42:51.239 --> 00:42:54.079
<v Speaker 4>all these applications and learn the APIs and learning the

818
00:42:54.719 --> 00:42:57.760
<v Speaker 4>political history of where all these models come from. So

819
00:42:58.920 --> 00:43:02.920
<v Speaker 4>it is a The benefit of Microsoft as a software

820
00:43:03.760 --> 00:43:06.599
<v Speaker 4>provider is it's the one throat to choke, right, this

821
00:43:06.760 --> 00:43:10.239
<v Speaker 4>is the one person you go to. They provide a

822
00:43:10.239 --> 00:43:12.199
<v Speaker 4>lot of the tooling, they provide a lot of the models.

823
00:43:12.480 --> 00:43:14.159
<v Speaker 4>Is it the best of any of the world's the

824
00:43:14.199 --> 00:43:17.079
<v Speaker 4>absolute best. No, But when you're doing a lot of

825
00:43:17.079 --> 00:43:20.360
<v Speaker 4>different stuff, sometimes you just have to have some heuristics

826
00:43:20.360 --> 00:43:23.360
<v Speaker 4>here and just make the decision making. There's an infinite

827
00:43:23.440 --> 00:43:25.519
<v Speaker 4>number of decisions that you have to make when you're

828
00:43:25.639 --> 00:43:29.719
<v Speaker 4>picking all of these. So starting just with the built

829
00:43:29.719 --> 00:43:32.559
<v Speaker 4>in tools, the built in APIs, it's a great easy

830
00:43:32.599 --> 00:43:36.039
<v Speaker 4>way to get started. And if they don't work for you,

831
00:43:36.800 --> 00:43:40.760
<v Speaker 4>then you can start making other questions and decisions. And yeah,

832
00:43:40.800 --> 00:43:43.119
<v Speaker 4>but I would say start with the built in stuff

833
00:43:43.199 --> 00:43:44.199
<v Speaker 4>definitely at first.

834
00:43:44.320 --> 00:43:46.760
<v Speaker 1>Okay, yeah, here I knew Ivious read I'd read this.

835
00:43:46.800 --> 00:43:51.639
<v Speaker 1>I just looked at up again. Gptoss is a version

836
00:43:51.920 --> 00:43:56.440
<v Speaker 1>of GPT three that can be run locally on a

837
00:43:56.480 --> 00:43:59.119
<v Speaker 1>machine with sixty four gigs around and a fifty to

838
00:43:59.199 --> 00:44:02.280
<v Speaker 1>ninety with twenty five gigs of v RAM. So that's

839
00:44:02.360 --> 00:44:07.559
<v Speaker 1>roughly six or seven thousand dollars PC somewhere in that neighborhood,

840
00:44:07.559 --> 00:44:08.800
<v Speaker 1>depending on how much you pay for the video car.

841
00:44:08.840 --> 00:44:10.480
<v Speaker 1>The video cards can be driving around it. But that's

842
00:44:10.559 --> 00:44:14.519
<v Speaker 1>running you know, GPT three, which is what the original

843
00:44:14.559 --> 00:44:17.280
<v Speaker 1>GitHub copilot was built. Again, Like, that's a pretty torquy,

844
00:44:18.639 --> 00:44:21.400
<v Speaker 1>pretty good little LM one hundred and twenty billion parameters.

845
00:44:21.800 --> 00:44:23.400
<v Speaker 1>Like it's not GPT.

846
00:44:23.000 --> 00:44:23.639
<v Speaker 2>Four, but.

847
00:44:25.199 --> 00:44:28.639
<v Speaker 1>Especially in a narrow scope application like a NOME set

848
00:44:28.679 --> 00:44:32.239
<v Speaker 1>of code, that's pretty robust. Man, you could do a

849
00:44:32.239 --> 00:44:32.880
<v Speaker 1>lot with that.

850
00:44:33.280 --> 00:44:35.199
<v Speaker 4>Yeah, you could do a lot with that. And also

851
00:44:35.360 --> 00:44:38.440
<v Speaker 4>you have to consider the big question of why would

852
00:44:38.480 --> 00:44:40.519
<v Speaker 4>you build local ever, you know, why do it at all?

853
00:44:40.639 --> 00:44:43.719
<v Speaker 4>Obviously privacy is a concern for a lot of people

854
00:44:43.760 --> 00:44:45.559
<v Speaker 4>of why would you do this stuff locally on your

855
00:44:45.559 --> 00:44:49.039
<v Speaker 4>own computer? If you have network concerns, if you don't

856
00:44:49.039 --> 00:44:52.639
<v Speaker 4>have reliable or high quality or high speed internet, then

857
00:44:52.800 --> 00:44:56.239
<v Speaker 4>obviously this is the only solution for you. But then

858
00:44:56.320 --> 00:45:00.280
<v Speaker 4>also there's the cost concern and the cost question of yeah,

859
00:45:00.480 --> 00:45:03.320
<v Speaker 4>you don't necessarily want to make some code that runs

860
00:45:03.320 --> 00:45:05.880
<v Speaker 4>out and is running all these llms, and then you

861
00:45:05.960 --> 00:45:08.440
<v Speaker 4>come back with a bill for you know, thousands of

862
00:45:08.519 --> 00:45:12.000
<v Speaker 4>tens of thousands of dollars because your credits went crazy. Right,

863
00:45:12.159 --> 00:45:15.360
<v Speaker 4>But when you have it local again, try There's so

864
00:45:15.519 --> 00:45:19.840
<v Speaker 4>many cool tools, the AIDEV gallery, the AI toolkit, and

865
00:45:20.800 --> 00:45:24.039
<v Speaker 4>then there's the APIs available already today. There's so many

866
00:45:24.039 --> 00:45:27.239
<v Speaker 4>ways to get started and try and see. I you know,

867
00:45:27.280 --> 00:45:29.159
<v Speaker 4>what is your application, what could it be? Try it

868
00:45:29.199 --> 00:45:31.199
<v Speaker 4>out because you might not have to sign up get

869
00:45:31.199 --> 00:45:33.239
<v Speaker 4>an API key at all. You could do all this

870
00:45:33.239 --> 00:45:36.000
<v Speaker 4>stuff locally. And then if you want to do batch

871
00:45:36.119 --> 00:45:39.199
<v Speaker 4>processing of again your own data, maybe you want to

872
00:45:39.480 --> 00:45:42.599
<v Speaker 4>kind of use these models to put the data into

873
00:45:42.599 --> 00:45:45.719
<v Speaker 4>a particular shape or clean it or work through it.

874
00:45:46.400 --> 00:45:49.360
<v Speaker 4>But you don't want to pay tokens to do all

875
00:45:49.400 --> 00:45:52.079
<v Speaker 4>that work. Well, do it locally, do it overnight. Build

876
00:45:52.119 --> 00:45:55.280
<v Speaker 4>an app, your own app, not something you ship necessarily,

877
00:45:55.320 --> 00:45:57.840
<v Speaker 4>but do it locally, you know, process that data locally,

878
00:45:57.880 --> 00:45:59.840
<v Speaker 4>and then go from there. Maybe you're going to build

879
00:45:59.840 --> 00:46:01.440
<v Speaker 4>your model, but first you have to get all the

880
00:46:01.480 --> 00:46:02.599
<v Speaker 4>data in the right shape.

881
00:46:02.559 --> 00:46:05.920
<v Speaker 1>Right, and and you're trading time for money right right.

882
00:46:06.000 --> 00:46:08.559
<v Speaker 1>Essentially the game you're playing here. It's like, Okay, if

883
00:46:08.559 --> 00:46:10.880
<v Speaker 1>I run it on the cloud, it's going to cost

884
00:46:10.960 --> 00:46:13.280
<v Speaker 1>me more, but I get it done less time, or

885
00:46:13.320 --> 00:46:15.920
<v Speaker 1>I'm restricted to my own hardware so it may take longer.

886
00:46:16.679 --> 00:46:18.320
<v Speaker 1>And then you start, you know, doing the economics. So

887
00:46:18.519 --> 00:46:21.280
<v Speaker 1>just looking up the high end. Yeah, the ninety six

888
00:46:21.440 --> 00:46:26.400
<v Speaker 1>gig in Nvidia RTX pro six thousand Blackwell, that's the

889
00:46:26.440 --> 00:46:28.400
<v Speaker 1>big Box twelve thousands.

890
00:46:28.440 --> 00:46:30.039
<v Speaker 2>Well, you know, it's not only the money, but as

891
00:46:30.119 --> 00:46:33.880
<v Speaker 2>Joe said, the security and the privacy that may trump

892
00:46:34.360 --> 00:46:37.639
<v Speaker 2>any kind of money, and you know, and that may

893
00:46:37.639 --> 00:46:39.440
<v Speaker 2>be the requirement you know.

894
00:46:39.719 --> 00:46:42.159
<v Speaker 1>Sorry that was Canadian dollars, just nine thousand Americans.

895
00:46:42.280 --> 00:46:44.760
<v Speaker 4>Ah, well that totally changes.

896
00:46:45.360 --> 00:46:51.320
<v Speaker 1>Yeah, everything's different. Now he just saved me two thousand

897
00:46:52.360 --> 00:46:55.039
<v Speaker 1>three grand grand. But again, if I'm playing that game

898
00:46:55.079 --> 00:46:57.639
<v Speaker 1>of the cost benefit, like what am I spending on

899
00:46:57.719 --> 00:47:02.039
<v Speaker 1>tokens at that scale? True? And I really get the

900
00:47:02.119 --> 00:47:05.679
<v Speaker 1>sense that as this sort of bubble starts to burst

901
00:47:05.679 --> 00:47:08.400
<v Speaker 1>and people need to make money, like tokens ain't getting

902
00:47:08.480 --> 00:47:09.760
<v Speaker 1>cheaper nowp.

903
00:47:09.760 --> 00:47:16.000
<v Speaker 4>Yeah, I have been using Claude and Codex and Copilot.

904
00:47:16.159 --> 00:47:21.119
<v Speaker 4>There's definitely times where I have three computers running and

905
00:47:21.159 --> 00:47:23.639
<v Speaker 4>they're I'm just kind of like telling them to keep

906
00:47:23.639 --> 00:47:27.119
<v Speaker 4>going over. They're checking and building, but it's never going

907
00:47:27.199 --> 00:47:30.159
<v Speaker 4>to be cheaper than it is now, Like this is

908
00:47:30.199 --> 00:47:31.679
<v Speaker 4>the cheapest is going to be. They're trying to get

909
00:47:31.679 --> 00:47:35.039
<v Speaker 4>as many users as possible, but that floor has to rise.

910
00:47:35.119 --> 00:47:38.519
<v Speaker 4>I mean, I know Anthropic was having some issues a

911
00:47:38.559 --> 00:47:43.000
<v Speaker 4>couple of weeks ago with limits and quality, and Codex

912
00:47:43.039 --> 00:47:45.400
<v Speaker 4>I think had something a month or so ago where

913
00:47:45.599 --> 00:47:48.679
<v Speaker 4>the limits. And again, if you're relying on these cloud services,

914
00:47:49.119 --> 00:47:51.440
<v Speaker 4>not only are you relying on them to stay up

915
00:47:51.639 --> 00:47:54.400
<v Speaker 4>and your connection to them to say live, but you're

916
00:47:54.400 --> 00:47:57.840
<v Speaker 4>also relying on the model and the pricing and the

917
00:47:57.880 --> 00:48:00.679
<v Speaker 4>availability at all from a business to point for them

918
00:48:00.719 --> 00:48:03.559
<v Speaker 4>to stay up. Because it might make sense today, I was.

919
00:48:03.599 --> 00:48:07.000
<v Speaker 1>Talking to some folks abroad that are big, like running

920
00:48:07.039 --> 00:48:10.519
<v Speaker 1>five sixty seven simultaneous instances because they're working that fast

921
00:48:10.639 --> 00:48:14.119
<v Speaker 1>right tuned models, reaching these things, and they said that

922
00:48:14.199 --> 00:48:18.079
<v Speaker 1>over July fourth everything got dramatically faster, like they got

923
00:48:18.079 --> 00:48:21.079
<v Speaker 1>a ton of work going July fourth because Americans weren't

924
00:48:21.079 --> 00:48:24.440
<v Speaker 1>working like these, like these cloud infrastructures are stressed to

925
00:48:24.519 --> 00:48:28.239
<v Speaker 1>the limit and slowing performance as it is right and

926
00:48:28.280 --> 00:48:30.239
<v Speaker 1>say and the only and the proof we've had is

927
00:48:30.280 --> 00:48:33.440
<v Speaker 1>like when the stress isn't a high, things are better.

928
00:48:33.559 --> 00:48:36.519
<v Speaker 1>So there is this interesting argument about at what point

929
00:48:36.559 --> 00:48:39.079
<v Speaker 1>does this make more sense to be local versus remote?

930
00:48:39.320 --> 00:48:41.199
<v Speaker 1>And this is going to be a shared resource too,

931
00:48:41.239 --> 00:48:44.000
<v Speaker 1>like these big boxes don't have to be per dev

932
00:48:44.159 --> 00:48:47.239
<v Speaker 1>They could be shared out again with potential performance issues

933
00:48:47.280 --> 00:48:50.280
<v Speaker 1>like well, of course, I'm such a hardware geek, like

934
00:48:50.320 --> 00:48:52.440
<v Speaker 1>I'd love to build out a rack of this stuff.

935
00:48:52.480 --> 00:48:53.679
<v Speaker 2>It would be fun, wouldn't it.

936
00:48:53.679 --> 00:48:55.960
<v Speaker 1>It would be and you know, and then now I've

937
00:48:56.000 --> 00:48:59.960
<v Speaker 1>got the heat and power problems right.

938
00:49:00.039 --> 00:49:03.320
<v Speaker 4>To live it firsthand. Well to your point about shared resources,

939
00:49:03.559 --> 00:49:06.280
<v Speaker 4>that is one of the nice things about win mL

940
00:49:06.519 --> 00:49:12.840
<v Speaker 4>that just released Execution Provider that Microsoft announced making it

941
00:49:12.880 --> 00:49:17.000
<v Speaker 4>easier for local devs to integrate models is if you

942
00:49:17.159 --> 00:49:20.599
<v Speaker 4>have an application and you need a model, do you

943
00:49:20.719 --> 00:49:24.239
<v Speaker 4>download it? And then every single one of your applications

944
00:49:24.320 --> 00:49:28.119
<v Speaker 4>is downloading a five gig LM. Yeah, obviously that becomes

945
00:49:28.239 --> 00:49:31.280
<v Speaker 4>untenable very quickly unless you have that twenty two terabyte

946
00:49:31.360 --> 00:49:35.719
<v Speaker 4>drive in your computer. Solow you yeah, yeah, more than one.

947
00:49:35.880 --> 00:49:40.119
<v Speaker 4>It does allow you to share models across application rich

948
00:49:40.159 --> 00:49:41.920
<v Speaker 4>so you can have one machine install.

949
00:49:41.679 --> 00:49:44.039
<v Speaker 2>Richard, you were right. I thought they were SSDs.

950
00:49:44.119 --> 00:49:48.360
<v Speaker 1>They're not. They're HDDs ds. There are a few SSDs

951
00:49:48.400 --> 00:49:51.119
<v Speaker 1>over eight terabytes, but most of them the line seems

952
00:49:51.119 --> 00:49:53.360
<v Speaker 1>to be eight. By the way, the RTX six thousand,

953
00:49:53.679 --> 00:49:55.559
<v Speaker 1>six hundred watts each.

954
00:49:56.400 --> 00:49:58.199
<v Speaker 2>That's why I have solar panels.

955
00:49:58.280 --> 00:50:00.960
<v Speaker 1>Yeah, that's it, you know, like oil BOYD. I'm just

956
00:50:01.000 --> 00:50:03.960
<v Speaker 1>thinking about how much you remember in the end, this

957
00:50:04.039 --> 00:50:06.840
<v Speaker 1>is moving electrons around and generating heat like you just

958
00:50:06.880 --> 00:50:09.599
<v Speaker 1>made rocks make heat. Like that's saying time watts. You're

959
00:50:09.599 --> 00:50:10.880
<v Speaker 1>gonna feel it. You don't want to sit in the

960
00:50:10.920 --> 00:50:13.159
<v Speaker 1>room with that thing only man, No, it's going to

961
00:50:13.239 --> 00:50:16.880
<v Speaker 1>be crazy. But it is an interesting point of view

962
00:50:17.760 --> 00:50:20.400
<v Speaker 1>as we're still going through this to say, what are

963
00:50:20.440 --> 00:50:21.880
<v Speaker 1>we going to shift local? What are we going to

964
00:50:21.960 --> 00:50:25.360
<v Speaker 1>run remote? Like, what's feasible at what makes sense for

965
00:50:26.000 --> 00:50:28.880
<v Speaker 1>folks here? And I think, you know, not everything has

966
00:50:28.920 --> 00:50:30.800
<v Speaker 1>to be cloud and not everybody wants it there.

967
00:50:30.800 --> 00:50:35.119
<v Speaker 4>Right, And I think you just have to be you

968
00:50:35.159 --> 00:50:37.159
<v Speaker 4>know wide. I'm not saying to get super deep on

969
00:50:37.239 --> 00:50:39.000
<v Speaker 4>all of this stuff, but the tools for you to

970
00:50:39.039 --> 00:50:43.039
<v Speaker 4>get your feet wet are available, and when you're CTO

971
00:50:43.320 --> 00:50:46.079
<v Speaker 4>or more probably more likely, your CFO comes to you

972
00:50:46.159 --> 00:50:50.360
<v Speaker 4>and says, hey, we can't afford this bill anymore. Your

973
00:50:50.519 --> 00:50:53.079
<v Speaker 4>critical application can't use this LLM. You have to stop,

974
00:50:53.199 --> 00:50:56.440
<v Speaker 4>or you have to change something because either somebody's prices

975
00:50:56.440 --> 00:50:58.000
<v Speaker 4>went up or the business model changed.

976
00:50:58.159 --> 00:50:58.960
<v Speaker 1>Yeah, what are you going to do?

977
00:50:58.960 --> 00:51:00.639
<v Speaker 4>What are you going to reach for? And getting your

978
00:51:00.639 --> 00:51:03.440
<v Speaker 4>feet wet in some of these local models, it's a

979
00:51:03.440 --> 00:51:05.880
<v Speaker 4>great way to have an answer or have some sort

980
00:51:05.880 --> 00:51:08.039
<v Speaker 4>of solution or see if that solution will work.

981
00:51:08.079 --> 00:51:11.320
<v Speaker 1>Now you're swapping op X for CAPEX and then, you know,

982
00:51:11.559 --> 00:51:14.800
<v Speaker 1>using CFO speak like, we have two ways to solve

983
00:51:14.840 --> 00:51:17.199
<v Speaker 1>this problem. We spend month over month on it, or

984
00:51:17.199 --> 00:51:20.159
<v Speaker 1>we made a capital investment and spend less. You know,

985
00:51:20.760 --> 00:51:22.800
<v Speaker 1>let's do the math. You know, if you want to

986
00:51:22.800 --> 00:51:24.480
<v Speaker 1>talk to CFO, bring a spreadsheet.

987
00:51:24.599 --> 00:51:29.199
<v Speaker 4>Yeah, exactly. And it's as as we've said, as you've said,

988
00:51:29.480 --> 00:51:33.599
<v Speaker 4>stuff is changing so fast. So if you get super deep,

989
00:51:33.800 --> 00:51:36.079
<v Speaker 4>if you start training your own model, and then tomorrow

990
00:51:36.199 --> 00:51:38.880
<v Speaker 4>somebody comes out with a model that just makes all

991
00:51:38.920 --> 00:51:42.119
<v Speaker 4>that effort useless. This is again, this is like the

992
00:51:42.599 --> 00:51:44.920
<v Speaker 4>sweet spot, right, Isn't this where the Windows developer has

993
00:51:45.000 --> 00:51:48.000
<v Speaker 4>kind of always loved to live where they're like, yeah, yeah. Yeah,

994
00:51:48.000 --> 00:51:51.079
<v Speaker 4>we're not like hardware level, we're not doing machine code.

995
00:51:51.079 --> 00:51:53.880
<v Speaker 4>But then we're also not just like bleeding like the

996
00:51:53.920 --> 00:51:55.599
<v Speaker 4>best of the best. It's like, okay, we're in the

997
00:51:55.599 --> 00:51:57.880
<v Speaker 4>middle here where we got models, we got a local

998
00:51:58.239 --> 00:52:01.199
<v Speaker 4>It's it's efficient, it's it's a good balance.

999
00:52:01.320 --> 00:52:03.119
<v Speaker 1>Yeah. Well, and I'm going to call back to Cagle

1000
00:52:03.159 --> 00:52:05.559
<v Speaker 1>again because one of the other ways you can get

1001
00:52:05.559 --> 00:52:08.320
<v Speaker 1>a model built is to put out a bounty on

1002
00:52:08.480 --> 00:52:12.559
<v Speaker 1>Cagle in a competition to have someone build it for you. Effectively,

1003
00:52:13.159 --> 00:52:15.639
<v Speaker 1>there you go. So you've got the data set, but

1004
00:52:15.719 --> 00:52:17.719
<v Speaker 1>you don't want to actually do the construction. You can

1005
00:52:18.199 --> 00:52:22.880
<v Speaker 1>host a competition and define your problem space and provide

1006
00:52:22.880 --> 00:52:24.920
<v Speaker 1>the sample data, and a bunch of people compete for

1007
00:52:25.199 --> 00:52:28.320
<v Speaker 1>to deliver you the best model. It's a weird world, man,

1008
00:52:28.519 --> 00:52:30.280
<v Speaker 1>is like, if you want to go deep into mL,

1009
00:52:30.320 --> 00:52:32.880
<v Speaker 1>there's so many interesting things to be done here. M hmmm.

1010
00:52:33.480 --> 00:52:36.119
<v Speaker 2>I had the weird meta thought that you could get

1011
00:52:36.159 --> 00:52:40.760
<v Speaker 2>a model to build your model instead of you know,

1012
00:52:40.880 --> 00:52:42.239
<v Speaker 2>farming it out for a bounty.

1013
00:52:42.320 --> 00:52:45.400
<v Speaker 1>Well, you're not wrong to interact with an LM to

1014
00:52:45.480 --> 00:52:49.039
<v Speaker 1>start constructing a plan around how a model would get built,

1015
00:52:49.039 --> 00:52:51.119
<v Speaker 1>because that you know, in the end, they are a

1016
00:52:51.159 --> 00:52:53.239
<v Speaker 1>pretty clever search tool for best practices.

1017
00:52:53.639 --> 00:52:57.880
<v Speaker 4>Yeah, search and tokenization is a really nice thing that

1018
00:52:57.920 --> 00:53:00.639
<v Speaker 4>you can do with your local LM of crunching some

1019
00:53:00.719 --> 00:53:03.800
<v Speaker 4>of your data, your text, tokenize it make it easier

1020
00:53:03.800 --> 00:53:08.519
<v Speaker 4>to search, have that more natural language available for your users.

1021
00:53:08.719 --> 00:53:10.880
<v Speaker 4>It's a really hard thing to code, but if you

1022
00:53:10.880 --> 00:53:13.039
<v Speaker 4>have local l MS, I can help you build that.

1023
00:53:13.199 --> 00:53:14.480
<v Speaker 1>Why not. Yeah, that's cool.

1024
00:53:15.280 --> 00:53:17.000
<v Speaker 2>Anything else on your mind that you want to touch

1025
00:53:17.039 --> 00:53:19.360
<v Speaker 2>on before we call it a show?

1026
00:53:20.039 --> 00:53:22.280
<v Speaker 4>Not really, I mean we touched on a lot here. Yeah,

1027
00:53:22.280 --> 00:53:23.400
<v Speaker 4>we just try it.

1028
00:53:23.679 --> 00:53:27.039
<v Speaker 1>We went we went on a ride today friend again. Yeah,

1029
00:53:27.079 --> 00:53:28.360
<v Speaker 1>but this is the kind of deep.

1030
00:53:28.159 --> 00:53:31.639
<v Speaker 2>Dive into local lms and local AI that I really

1031
00:53:31.679 --> 00:53:35.400
<v Speaker 2>wanted to get to. So I'm very very happy we talked.

1032
00:53:35.519 --> 00:53:36.199
<v Speaker 2>Thank you, Joe.

1033
00:53:36.440 --> 00:53:37.559
<v Speaker 4>Yeah, happy to be here.

1034
00:53:37.840 --> 00:53:38.960
<v Speaker 2>I'm right and we'll.

1035
00:53:38.800 --> 00:53:58.880
<v Speaker 5>Talk to you next time on dot net rocks.

1036
00:54:03.079 --> 00:54:05.800
<v Speaker 2>Dot net rocks is brought to you by Franklin's Net

1037
00:54:05.880 --> 00:54:09.840
<v Speaker 2>and produced by Pop Studios, a full service audio, video

1038
00:54:09.920 --> 00:54:14.000
<v Speaker 2>and post production facility located physically in New London, Connecticut,

1039
00:54:14.239 --> 00:54:19.039
<v Speaker 2>and of course in the cloud online at pwop dot com.

1040
00:54:19.239 --> 00:54:21.360
<v Speaker 2>Visit our website at d O T N E t

1041
00:54:21.599 --> 00:54:25.639
<v Speaker 2>r o c k S dot com for RSS feeds, downloads,

1042
00:54:25.760 --> 00:54:29.480
<v Speaker 2>mobile apps, comments, and access to the full archives going

1043
00:54:29.480 --> 00:54:32.880
<v Speaker 2>back to show number one, recorded in September two thousand

1044
00:54:32.920 --> 00:54:35.559
<v Speaker 2>and two. And make sure you check out our sponsors.

1045
00:54:35.719 --> 00:54:38.760
<v Speaker 2>They keep us in business. Now go write some code,

1046
00:54:39.079 --> 00:54:39.840
<v Speaker 2>See you next time.

1047
00:54:40.760 --> 00:54:42.559
<v Speaker 4>You got jas.