1
00:00:01,000 --> 00:00:04,759
How'd you like to listen to dot
net rocks with no ads? Easy?

2
00:00:05,320 --> 00:00:09,880
Become a patron For just five dollars
a month you get access to a private

3
00:00:10,000 --> 00:00:14,400
RSS feed where all the shows have
no ADS. Twenty dollars a month will

4
00:00:14,400 --> 00:00:18,800
get you that and a special dot
net Rocks patron mug. Sign up now

5
00:00:18,839 --> 00:00:24,199
at Patreon dot dot net rocks dot
com. Hey Carlin Richard here. As

6
00:00:24,239 --> 00:00:29,519
you may have heard, NDC is
back offering their incredible in person conferences around

7
00:00:29,559 --> 00:00:33,320
the world, and we'd like to
tell you about them. NDC Oslow will

8
00:00:33,359 --> 00:00:37,119
be made twenty first through the twenty
fifth. Go to NDC Oslo dot com

9
00:00:37,159 --> 00:00:42,840
to register. NDC Copenhagen is happening
August twenty seventh through the thirty first.

10
00:00:42,880 --> 00:00:48,880
The early bird discount for NDC Copenhagen
ends June second. Go to NDC Copenhagen

11
00:00:48,960 --> 00:00:54,719
dot com for more information. NDC
Porto is happening October sixteenth through the twentieth.

12
00:00:54,920 --> 00:00:59,000
The early bird discount for dc Porto
ends July twenty first. Go to

13
00:00:59,079 --> 00:01:03,840
EDDC Porto Do calm to register and
check out the full lineup of conferences at

14
00:01:03,960 --> 00:01:22,680
NDC Conferences dot com. Welcome back
to dot Net Rocks. This is Carl

15
00:01:22,719 --> 00:01:26,319
Franklin and this is Richard Campbell,
and we're back. It seems like we

16
00:01:26,359 --> 00:01:30,799
haven't recorded in a while. No, it's not true. We you know,

17
00:01:30,879 --> 00:01:33,400
we've been doing which is weird for
us, is we haven't been batching

18
00:01:33,439 --> 00:01:36,799
up shows. We been recording like
one a week, just sort of staying

19
00:01:36,840 --> 00:01:40,159
ahead. And that's weird in this
way, only you know, we're doing

20
00:01:40,159 --> 00:01:42,079
events and we're doing like three shows
in a day and stuff like that.

21
00:01:42,200 --> 00:01:47,719
But yeah, we'll get ahead.
It's just been a little bit pause and

22
00:01:47,799 --> 00:01:49,200
travel, so it hasn't been too
rough. Hey, for those who haven't

23
00:01:49,200 --> 00:01:55,200
been paying attention. On Facebook,
PLoP Studios has a page and I've been

24
00:01:55,239 --> 00:01:59,879
documenting the build out of PLoP three
point zero, which is a home studio.

25
00:02:00,040 --> 00:02:04,599
It's not commercial, but it's it's
in my home. And yeah,

26
00:02:04,640 --> 00:02:07,960
and we've got it up to the
point of the rock Will installation, which

27
00:02:08,039 --> 00:02:12,400
is being finished right now as we
record this. Yeah, it's gonna be

28
00:02:12,400 --> 00:02:15,080
amazing. So maybe a couple of
three shows from now will be you'll be

29
00:02:15,080 --> 00:02:17,599
recording from the studio. Yeah,
maybe maybe a month, might be a

30
00:02:17,639 --> 00:02:21,199
month. We'll see what happens.
Yeah, we'll see what happens. It'll

31
00:02:21,240 --> 00:02:23,599
be good. Good to test it
up. All right, let's roll the

32
00:02:23,639 --> 00:02:27,120
crazy music for and Phil. You
can join in on this one if you

33
00:02:27,159 --> 00:02:44,319
want better know a framework. Gentlemen, listeners, I give you the what

34
00:02:44,360 --> 00:02:47,960
the fuck? It is a magnificent
app, and I'm sorry this is going

35
00:02:49,000 --> 00:02:53,639
to be one of those Jerry Seinfeld
low fat yogurt episodes. It's a magnificent

36
00:02:53,680 --> 00:03:00,240
app that corrects errors in previous console
commands. All right, so you you

37
00:03:00,280 --> 00:03:07,759
install it, you say something like
get push and fatal the current branch master

38
00:03:07,840 --> 00:03:09,960
has no upstream branch to push the
current branch and blah blah blah. Use

39
00:03:10,080 --> 00:03:16,439
this, and then you simply say
and then then it says get pushed,

40
00:03:16,479 --> 00:03:23,080
set up stream origin master, blah
blah blah. Counting objects nine pouthon no

41
00:03:23,199 --> 00:03:30,159
command, poothon found. Did you
mean Python and you say it stalls pi

42
00:03:31,719 --> 00:03:38,080
rens pithon get burnch b r NC
get burnches not a get command. Did

43
00:03:38,080 --> 00:03:45,439
you mean branch and you say,
and it's get branch. It's what you

44
00:03:45,479 --> 00:03:50,800
were saying anyway, right, This
just codifies the way I code normally,

45
00:03:53,080 --> 00:04:03,400
just admitting the truth. I can't
wait to use this brilliant, awesome find

46
00:04:05,000 --> 00:04:09,919
who's talking to us today? Richard
crabbed a comment dub the show eighteen thirty

47
00:04:09,960 --> 00:04:12,520
six. That's the one we did, Grant Barrett, it was your guest.

48
00:04:12,800 --> 00:04:15,680
Yeah, I found. We were
talking about large language models, which

49
00:04:15,680 --> 00:04:18,040
I think we'll be talking about some
more today. And a bunch of good

50
00:04:18,040 --> 00:04:20,199
comments on the show, but one
of them is from Dennis Troller, who

51
00:04:20,199 --> 00:04:24,800
said, I think you're quite right
that the next skill being crafting prompts for

52
00:04:24,839 --> 00:04:29,639
AI bots, and I think also
to be able to evaluate responses. Actually,

53
00:04:29,680 --> 00:04:33,360
I think that's the main thing,
is evaluate the response on some level.

54
00:04:33,399 --> 00:04:35,600
This is already what we were doing
when we use a search engine to

55
00:04:35,600 --> 00:04:40,199
find the right stack overflow answer.
Oh boy, that's a good one,

56
00:04:40,680 --> 00:04:43,439
evaluate its correctness, a fitness for
a problem, and choose to use it

57
00:04:43,560 --> 00:04:46,720
or just part of it. I
spent years honing this skill, training Google

58
00:04:46,759 --> 00:04:49,639
to find the right answers for my
profile. Along the way, context is

59
00:04:49,639 --> 00:04:53,360
everything, and I think Google has
a lot of contexts tied to my account

60
00:04:53,399 --> 00:04:57,160
the same way. I hope the
cash of those bots might somehow expand to

61
00:04:57,240 --> 00:05:00,240
keep the conversation flowing across sessions.
That's an interesting point, and I've had

62
00:05:00,279 --> 00:05:05,319
a couple of insider conversations about better
context engines because the current ones are really

63
00:05:05,360 --> 00:05:09,839
not that good. This was the
promise of virtual assistance like Siri, Cartan

64
00:05:09,920 --> 00:05:14,319
and Alexa, I feel never materialized. Now, imagine if that assistant had

65
00:05:14,560 --> 00:05:17,399
a year or five years or twenty
years context tied to you and the ability

66
00:05:17,399 --> 00:05:23,439
to search the web with that.
Do you mean like my wife? That

67
00:05:23,560 --> 00:05:28,680
would be a yeah. Yeah,
that's definitely reminds you of all those of

68
00:05:28,759 --> 00:05:33,720
the appropriate On July second, you
said, do you remember that an old

69
00:05:33,720 --> 00:05:36,759
star trek, you know, for
the private file, Like, don't remember

70
00:05:36,879 --> 00:05:43,920
this searching context dat capability? Yeah? Yeah, Dennis, I'm glad you

71
00:05:44,040 --> 00:05:46,720
think in this way. I do
think we're all going to get skilled at

72
00:05:46,720 --> 00:05:53,439
writing better prompts. I think these
prompt engineers are like this week's fish.

73
00:05:53,480 --> 00:05:57,639
It'll be fine this week, but
next week it'll smell, and the meantime,

74
00:05:57,639 --> 00:06:00,720
all of us will get a little
more skilled at using what is really

75
00:06:00,759 --> 00:06:04,079
just a clever bot with some additional
tools attached to it. Yeah. So,

76
00:06:04,160 --> 00:06:06,160
thank you so much for your comment, and a copy of music co

77
00:06:06,240 --> 00:06:09,160
Buy is on its way to you, and if you'd like a copy of

78
00:06:09,240 --> 00:06:12,040
music co Buy. I read a
comment on the website dot net rocks dot

79
00:06:12,079 --> 00:06:14,800
com or on the facebooks. We
publish every show there, and if you

80
00:06:14,839 --> 00:06:16,279
comment there and I reading on the
show, we'll send you copy of music

81
00:06:16,319 --> 00:06:19,199
co Buy and definitely follow us.
You can follow us on Twitter, but

82
00:06:19,279 --> 00:06:24,319
definitely follow us on Mastodon. We
have more fun over there. I'm at

83
00:06:24,439 --> 00:06:28,879
Carl Franklin at tech hub dot social, and I'm Rich Campbell at Mastodon dot

84
00:06:28,959 --> 00:06:31,959
social. And I also want to
call out Fred Johnson. That's f r

85
00:06:32,040 --> 00:06:40,839
e DDE john ss o N for
sending me the link to the thank you

86
00:06:41,160 --> 00:06:46,519
That was a Mastodon toot. So
send us a two. And let's bring

87
00:06:46,560 --> 00:06:49,879
on Phil Hack again for the ninety
ninth time, probably on dot net rocks

88
00:06:50,000 --> 00:06:55,639
beIN afew. Yeah, I don't
really need to read his bio, but

89
00:06:55,720 --> 00:06:59,720
I will anyway. He has over
twenty years of experience in the software industry.

90
00:07:00,279 --> 00:07:04,839
Prior to his current gig at a
Serious Business Incorporated, he was Director

91
00:07:04,879 --> 00:07:11,160
of Engineering at GitHub and helped make
GitHub friendly to developers on the Microsoft platform.

92
00:07:11,160 --> 00:07:15,160
Prior to that, he was senior
program manager at Microsoft responsible for shipping

93
00:07:15,199 --> 00:07:20,199
Aspen at MVC and New get among
other projects, and these products had permissive

94
00:07:20,279 --> 00:07:26,600
open source licenses and ushered in Microsoft's
open source era. He's got a lot

95
00:07:26,600 --> 00:07:30,879
of other things in his bio that
you can go read it at hacked dot

96
00:07:30,920 --> 00:07:35,240
com, h aacked dot com,
or on Twitter at act. Hey man,

97
00:07:35,439 --> 00:07:39,000
how's it going. That's going?
Well, it's good to talk to

98
00:07:39,040 --> 00:07:45,920
you the many seriously. Yeah,
I appreciate y'all having me on board.

99
00:07:46,000 --> 00:07:47,240
It's funny. A couple of times
I thought, oh, you know,

100
00:07:47,240 --> 00:07:51,639
actually start a podcast because I liked
talking. But then I talked to folks

101
00:07:51,639 --> 00:07:57,240
like Richard and Hanselman and realized,
oh, it's a ton of work.

102
00:07:57,560 --> 00:08:01,480
Can I say that he might as
well at this point, Yeah, you've

103
00:08:01,480 --> 00:08:05,120
got broken the seal one. It
is a ton of work, as I

104
00:08:05,240 --> 00:08:11,399
say, And so instead I started
to startup because I'm an idiot, because

105
00:08:11,439 --> 00:08:16,439
that's less work. Very clever,
Yeah, very clever. Well how's business?

106
00:08:16,560 --> 00:08:18,439
You're still doing it? So that
you've you've beaten most startups? You

107
00:08:18,560 --> 00:08:22,279
still you still have a startup.
Yeah, we're hanging in there. I

108
00:08:22,319 --> 00:08:28,279
mean it is a real challenged.
You know, at this stage of the

109
00:08:28,319 --> 00:08:33,879
game, product market fit is the
number one priority, and we're still trying

110
00:08:33,919 --> 00:08:39,720
to hit that, but the latest
set of features that we've built out have

111
00:08:39,799 --> 00:08:43,000
been really interesting, and so we're
very hopeful that this is going to be

112
00:08:43,039 --> 00:08:48,720
the thing that really starts to drive
business remind everybody what your product is and

113
00:08:48,759 --> 00:08:54,360
what you do. Sure, so
we started out building abbott, which is

114
00:08:54,720 --> 00:09:00,080
initially a bot for doing chatofs within
chat based on our experiences that get of.

115
00:09:00,759 --> 00:09:03,480
But you know, if you go
to a serious business dot community are

116
00:09:03,559 --> 00:09:05,279
about page, you see that at
our heart, you know, our mission

117
00:09:05,360 --> 00:09:09,279
was like, we're going to build
software for humans by humans, but leverage

118
00:09:09,279 --> 00:09:13,679
automation to help humans. And now
like with the you know, with chat,

119
00:09:13,759 --> 00:09:20,600
GPT and all these other lms finally
becoming very useful, we now have

120
00:09:20,720 --> 00:09:26,480
the tools to actually sort of realize
that mission in a way that we didn't

121
00:09:26,559 --> 00:09:33,000
have before where what we pivoted to. So the chat off stuff didn't really

122
00:09:33,039 --> 00:09:37,639
catch on, mainly because we're just
like solving every problem, which is really

123
00:09:37,639 --> 00:09:39,559
hard to sell. It's like,
oh, what problem do you have?

124
00:09:39,679 --> 00:09:43,559
Well, what problem do you or
what problem do you solve? And we're

125
00:09:43,559 --> 00:09:46,200
like, well, what problem do
you have? And that's not a compelling

126
00:09:46,200 --> 00:09:50,960
sales pitch. So what we discovered
was a lot of our customers are using

127
00:09:50,960 --> 00:09:56,440
slack connect. So slack connect allows
you to have a shared channel in your

128
00:09:56,519 --> 00:10:01,600
private Slack that you share with another
company. And what we found with a

129
00:10:01,639 --> 00:10:05,200
lot of startups, a lot of
companies use that to invite a customers,

130
00:10:05,240 --> 00:10:09,799
like you know, premier customers or
any customers are theirs to have a shared

131
00:10:09,919 --> 00:10:15,320
room with them so that they can
provide customer success, customer support, that

132
00:10:15,399 --> 00:10:18,320
sort of thing. And they were
like losing track of conversations or like we

133
00:10:18,360 --> 00:10:22,679
have no idea what's going on in
there? Are you know, are we

134
00:10:22,720 --> 00:10:26,919
missing conversations or were responding to everything? Yeah, so we just exactly So

135
00:10:26,960 --> 00:10:31,759
we decided to switch our focus on
that problem because it turns out like we

136
00:10:31,799 --> 00:10:37,240
had built this whole platform and we
could easily leverage it to solve that problem.

137
00:10:37,559 --> 00:10:41,960
So that's primarily what we've been focused
on. So it'll do things like,

138
00:10:41,960 --> 00:10:46,320
you know, if a customer writes
a message and nobody's responded to that

139
00:10:46,440 --> 00:10:52,840
message in a while, you can
set these timeouts and we'll notify your set

140
00:10:52,840 --> 00:10:58,240
of first responders that you've configured.
Hey, this person said this message,

141
00:10:58,559 --> 00:11:05,639
nobody's responded. We recently added triage
rooms, so like, if you need

142
00:11:05,679 --> 00:11:09,399
to discuss a message from a customer
but not with the customer, you can

143
00:11:09,639 --> 00:11:13,279
do it in the privacy of your
own room and it's all interconnected, and

144
00:11:13,320 --> 00:11:20,240
then you can open Zendesk or HubSpot
tickets right from messages. So that that

145
00:11:20,399 --> 00:11:24,840
has been the primary product. But
now with the CHATCHYBT and open AI,

146
00:11:24,960 --> 00:11:28,399
we've started to sprinkle sprinkle a little
AI on it and are doing a lot

147
00:11:28,440 --> 00:11:33,879
more interesting stuff now interesting, And
I noticed when I went to abbot the

148
00:11:33,879 --> 00:11:39,559
title now is a Copilot for Customer
Success. Yeah. Nice, because I

149
00:11:39,600 --> 00:11:43,600
think Copilot is like one of the
best brands Microsoft's ever come up with for

150
00:11:43,679 --> 00:11:50,960
a company notorious for naming things poorly. Yeah, yeah, building large language

151
00:11:50,960 --> 00:11:54,799
models named copilot because it right in
the name. It reminds you you're still

152
00:11:54,799 --> 00:11:58,480
the pilot. Hey, at least
you're not doing what Samsung did. Did

153
00:11:58,519 --> 00:12:05,120
you hear about the Samsung Chat GPT
leak? Now, so I'll post a

154
00:12:05,159 --> 00:12:09,159
link to this. Samsung meeting notes
and new source code are now in the

155
00:12:09,200 --> 00:12:16,480
wild after being leaked in Chat GPT. Yeah, they allowed engineers at its

156
00:12:16,519 --> 00:12:20,200
semiconductor arm to use the AI writer
to help fix problems with their source code,

157
00:12:20,559 --> 00:12:24,480
but in doing so, the workers
input in confidential data such as the

158
00:12:24,519 --> 00:12:30,600
source code itself for a new program, internal meeting notes data relating to their

159
00:12:30,639 --> 00:12:35,960
hardware, and in just under a
month there were three recorded incidences of employees

160
00:12:37,039 --> 00:12:41,519
leaking sensitive information via chat gpt.
Awesome, and since chat GPT retains input

161
00:12:41,639 --> 00:12:46,360
data to further train itself that they're
now effectively in the hands of open ai.

162
00:12:48,080 --> 00:12:50,759
Oops. I mean it says in
the agreement that's why this thing exists,

163
00:12:50,879 --> 00:12:56,279
right right, Yeah, we forget. But open ai was doing these

164
00:12:56,360 --> 00:13:00,919
kinds of tests routinely. Yeah,
this was just another test. November twenty

165
00:13:00,919 --> 00:13:03,279
two. The fact that one hundred
million people signed up for it and had

166
00:13:03,360 --> 00:13:09,240
existential conversations with a piece of software. It's not their fault, even ultimates,

167
00:13:09,279 --> 00:13:11,639
going like, what's wrong with you? Yeah? My thought is,

168
00:13:11,679 --> 00:13:16,559
if you need to have an existentialist
conversation, get a therapist or a dog.

169
00:13:20,120 --> 00:13:22,480
Don't use the software. That's not
the software. It's war Hey,

170
00:13:22,519 --> 00:13:28,799
help me fix this ransomware bug.
Yeah, we definitely need people to understand

171
00:13:28,840 --> 00:13:33,600
what it is and what it isn't
right, you know, with regards to

172
00:13:33,960 --> 00:13:39,519
training its data on the input.
So we're using the APIs in our products.

173
00:13:39,559 --> 00:13:43,000
So we do have an agreement with
open aye not to use our data

174
00:13:43,080 --> 00:13:46,159
to train there. So I think
you can talk to them if you're building

175
00:13:46,159 --> 00:13:50,360
an application on top of you can
you can get an agreement like that in

176
00:13:50,440 --> 00:13:56,240
place. But just to make sure, we're also using Microsoft's text what was

177
00:13:56,279 --> 00:14:03,000
it called the service that does text
redaction or PI redaction. There's a technical

178
00:14:03,080 --> 00:14:05,960
name for it, and like,
of course, my brain decides right now

179
00:14:05,000 --> 00:14:09,039
to forget what the actual name is. But it's effectively an API. You

180
00:14:09,720 --> 00:14:15,399
post a message there, it strips
out any PII and replaces it with something

181
00:14:15,440 --> 00:14:18,799
else so that you can later reconstitute
the message, and then we take that

182
00:14:18,879 --> 00:14:22,960
we send that to open AI.
So for example, if you know,

183
00:14:22,120 --> 00:14:26,519
if you say a credit card number
on accident, it will replace that with

184
00:14:26,679 --> 00:14:31,639
you know, hash hash hash hash, yeah, things like that. It's

185
00:14:31,679 --> 00:14:33,960
one of the cognitive services skill,
right, that's right, that's exactly it.

186
00:14:35,360 --> 00:14:37,399
So instead of sending token, AI
will send your personal information of Microsoft

187
00:14:37,440 --> 00:14:45,279
instead, we trans Microsoft a little
bit more to not do anything with it.

188
00:14:45,360 --> 00:14:50,519
At least the service. Microsoft's using
chat GPT for everything now right internally,

189
00:14:50,960 --> 00:14:52,679
So maybe it's gonna end up there
anyway. Well, hopefully they're not

190
00:14:52,919 --> 00:14:58,679
using it for PII redaction, because
that's the I mean, this service.

191
00:14:58,720 --> 00:15:03,039
Okay, but this way this s
this was around before opening. Sure,

192
00:15:03,360 --> 00:15:07,120
so was bing? So was bing? Yeah? True? True? Have

193
00:15:07,200 --> 00:15:11,759
you have you noticed that this that
actually searching stuff on being it hasn't gotten

194
00:15:11,759 --> 00:15:20,519
any better? No? Nope?
Yeah, what what's being? Oh?

195
00:15:20,519 --> 00:15:24,000
I mean here are you now?
Fun fact that used to be the maintainer

196
00:15:24,000 --> 00:15:30,039
of let me being that for you? So right, I finally gave that

197
00:15:30,080 --> 00:15:33,679
away thankfully. That was too much
work. Yeah, I didn't didn't need

198
00:15:33,679 --> 00:15:35,519
to own that anymore. Yeah,
it did not. It's so funny.

199
00:15:35,759 --> 00:15:39,600
That's been my theme, just don't
stop owning things. Right, So,

200
00:15:39,759 --> 00:15:46,279
are you only focused on this slack
you know, visibility problem or discoverability problem

201
00:15:46,440 --> 00:15:52,000
or are there more things on that
that that that your bot does or your

202
00:15:52,039 --> 00:15:54,279
bot API does. Oh well,
so the latest thing that we've been focused

203
00:15:54,279 --> 00:15:58,399
on is um You know, when
you have so many messages, it's really

204
00:15:58,480 --> 00:16:02,120
hard to get a handle what the
heck is going on in any given channel

205
00:16:02,399 --> 00:16:07,360
picture. So you know, there's
the obvious thing is worth adding a summarization

206
00:16:07,399 --> 00:16:11,440
of conversations. So um, you
know, if you have a long thread,

207
00:16:11,799 --> 00:16:14,759
but you go to the website,
will have the nice short summary,

208
00:16:15,000 --> 00:16:18,399
but not only that, an extraction
of the conclusion, like you know what,

209
00:16:18,399 --> 00:16:21,879
what is the next step? Right, because that's the thing you really

210
00:16:21,919 --> 00:16:26,120
want to know as a support agent. I had a funny one. I

211
00:16:26,159 --> 00:16:29,200
was testing. You know, when
I'm testing, I'm just creating all these

212
00:16:29,279 --> 00:16:33,000
random messages and I was like,
you know, help, I'm watching Game

213
00:16:33,039 --> 00:16:36,279
of Thrones and all the characters keep
dying, Like am I doing this right?

214
00:16:37,679 --> 00:16:44,200
And but the best part so then
the summarization Abbott was like, uh,

215
00:16:44,919 --> 00:16:48,759
you know the summary customers watching Game
Thrones concerned about all the characters dying.

216
00:16:48,919 --> 00:16:52,480
The conclusion was informed the customer that
this is normal for the show and

217
00:16:52,519 --> 00:16:56,120
that that they're doing it. Okay, kid, you not? That was

218
00:16:56,159 --> 00:17:00,159
the summary And I was like,
holy, that's that's pretty good. That's

219
00:17:00,159 --> 00:17:06,119
pretty good. Yeah. The thing
I'm currently working on is, right now,

220
00:17:06,160 --> 00:17:08,640
we have this limitation like how do
we know what a conversation is in

221
00:17:08,680 --> 00:17:12,279
an act of slack chat room?
Right? And so what we do is

222
00:17:12,319 --> 00:17:17,200
we, uh we require that it
being a slack threat. So you know,

223
00:17:17,279 --> 00:17:21,359
a whole threat is a conversation.
It's nice, easy to contain.

224
00:17:22,200 --> 00:17:26,480
But what we're finding is that customers
are not so disciplined, right like real

225
00:17:26,559 --> 00:17:33,599
people. Yeah, shockingly, real
people like to just do words answered,

226
00:17:33,720 --> 00:17:40,000
start typing more words, and then
conversations are top level, interleaved with each

227
00:17:40,000 --> 00:17:42,880
other. Say can you fix Facebook
while you're at it to somebody? Awesome,

228
00:17:45,960 --> 00:17:48,160
Yeah, if they pull a dump
truck of money up to my house,

229
00:17:48,240 --> 00:17:52,200
I'm happy to fix Facebook for them, just in a boat that.

230
00:17:52,400 --> 00:17:57,720
You know, when somebody asked the
same question as on a comment that's been

231
00:17:57,759 --> 00:18:03,000
asked and answered not yindy times,
you could say just to automatically say scroll

232
00:18:03,119 --> 00:18:08,799
up, closed will not answer.
You know, I'll make it. Oh,

233
00:18:08,880 --> 00:18:11,759
I see where you saying. I
thought you meant fixed Facebook from with

234
00:18:11,839 --> 00:18:15,480
them, but you're talking about like, oh well yeah, there's so many

235
00:18:15,519 --> 00:18:18,480
problems we could start with that one. A browser extension that could could do

236
00:18:18,559 --> 00:18:25,319
that, right, a browser extension
that says, you know this has been

237
00:18:25,319 --> 00:18:29,599
answered before, ye scroll up,
but they I mean before the whole large

238
00:18:29,640 --> 00:18:33,680
language model thing really took hold and
so forth. Microsoft had the fact bought

239
00:18:33,119 --> 00:18:38,200
right where you could literally feed a
fact to it and it would create and

240
00:18:38,240 --> 00:18:42,599
it would create a messenger style chat
bot where you could just ask the questions

241
00:18:42,599 --> 00:18:47,640
that we're in the fact and it
would spit them back to you. How

242
00:18:48,640 --> 00:18:52,119
is this kind of the same thing, except the language model is just a

243
00:18:52,119 --> 00:18:56,480
bit bigger. I haven't so Unfortunately, I haven't played with the fact bought,

244
00:18:56,559 --> 00:19:00,920
so I don't know how well it
worked or didn't work. One of

245
00:19:00,960 --> 00:19:06,359
the things we're very careful about is
we don't want to trust the LM to

246
00:19:06,440 --> 00:19:11,680
know things because the LM is going
to make it up. Have you tried

247
00:19:11,799 --> 00:19:15,599
asking chat gpt to write a Wikipedia
article about yourself? Yes, yeah,

248
00:19:15,640 --> 00:19:18,160
I know, I know people who
have. Yeah, yeah, you should

249
00:19:18,200 --> 00:19:22,319
do it. It's fun. And
if if your name is fairly common where

250
00:19:22,400 --> 00:19:26,559
like it would have trouble knowing who
you are, just give it a little

251
00:19:26,559 --> 00:19:30,160
extra detail, like, oh,
you know, write a Wikipedia article about

252
00:19:30,279 --> 00:19:34,200
Phil Hack who used to work on
Microsoft. Right, I'm fortunate enough that

253
00:19:34,240 --> 00:19:37,440
my name's unique, So like I
just say, write a wikipedter Bartle Phil

254
00:19:37,480 --> 00:19:42,160
Hack, and it will get so
many details right and so many details wrong.

255
00:19:42,400 --> 00:19:48,359
Hey, yeah, it is pretty
wild. So what we realize is

256
00:19:48,400 --> 00:19:52,480
if you can strain it to instead
of you know, This is why we're

257
00:19:52,519 --> 00:19:55,640
not having it. We're not putting
it in front of people to like answer

258
00:19:55,720 --> 00:19:59,920
questions, right because the problem there's
you get the clippy experience, you get

259
00:20:00,079 --> 00:20:04,119
the you get the hallucinations as they
call it UM. So we're more about

260
00:20:04,119 --> 00:20:11,240
extracting. Can we call them bugs? Because I think that hallucinations just sound

261
00:20:11,279 --> 00:20:15,400
cooler? I know, Like I
know a lot of people object to that

262
00:20:15,519 --> 00:20:18,680
term because it gives it. It
makes it sound like it's doing more than

263
00:20:18,720 --> 00:20:21,640
it is. Right, Yeah,
it's anthropomorphization, right, like we're giving

264
00:20:21,920 --> 00:20:26,200
we're giving this piece of software agency
and it doesn't have it. You should

265
00:20:26,200 --> 00:20:30,079
say, yeah, I think that's
it is. So is Frank Zappa talking

266
00:20:30,200 --> 00:20:36,920
through you right now? I'm talking
to the ghost of Frank Zappa here,

267
00:20:37,240 --> 00:20:41,839
because yeah, I mean, you
know, the responses do make you think,

268
00:20:41,880 --> 00:20:47,240
God, is it smoking something?
But it is clearly not. It's

269
00:20:47,519 --> 00:20:49,920
um as you all know it's it. I think a lot of people don't

270
00:20:49,920 --> 00:20:56,799
realize how it works because it's so
eerily good at something, but that it's

271
00:20:56,839 --> 00:21:00,160
a you know, very very good
auto complete, you know, like a

272
00:21:00,279 --> 00:21:07,559
predictive model of what is the likely
next word that should come next based on

273
00:21:07,599 --> 00:21:12,000
a giant, you know, corpus
of information. It's amazing that it works

274
00:21:12,039 --> 00:21:17,000
so well. Like some of the
things we've gotten it to do really blow

275
00:21:17,000 --> 00:21:21,880
our mind. But when you ask
it to generate in text, you know

276
00:21:21,960 --> 00:21:23,200
it will get things wrong. It
will get things wrong that you think it

277
00:21:23,240 --> 00:21:27,039
should know. Like I asked it
the other day about its own API,

278
00:21:27,279 --> 00:21:33,319
you know, the open the open
Ai API, and it spit something out

279
00:21:33,359 --> 00:21:36,559
that looked right, but it was
wrong, and so I had to I

280
00:21:36,599 --> 00:21:38,000
had to check it with real code, obviously, And it's like, oh,

281
00:21:38,000 --> 00:21:41,960
oh, you know. It helped
though, because like it gave me

282
00:21:41,000 --> 00:21:45,000
the you know, the gist of
it. But when I tested it with

283
00:21:45,079 --> 00:21:47,240
real code, I realized it was
wrong. But I was able to fix

284
00:21:47,279 --> 00:21:48,640
it really quickly. Right, Yeah, I found the same thing. And

285
00:21:48,920 --> 00:21:52,880
when I'm when I need help with
a certain syntax or something or you know,

286
00:21:52,920 --> 00:21:56,640
a pattern or something, I can't. I don't want to figure it.

287
00:21:56,640 --> 00:21:59,759
I want to take the time to
figure out. I ask it.

288
00:22:00,039 --> 00:22:03,039
Often i'll get a wrong answer,
but it's close enough where I can,

289
00:22:03,039 --> 00:22:07,680
oh, I get what it's trying
to do. Yeah, and I can

290
00:22:07,720 --> 00:22:11,319
figure it out. So it is
kind of like having a sort of a

291
00:22:11,400 --> 00:22:15,200
dim witted lab assistant. You know. Yeah, what we've been discovering is

292
00:22:15,240 --> 00:22:19,680
it's really good at extracting information from
information. Right, So you know,

293
00:22:19,960 --> 00:22:26,480
here's a set of conversations. What
is the topics, what are the categories?

294
00:22:26,519 --> 00:22:30,599
What is the gist of this conversation? What I've been experimenting with is

295
00:22:30,640 --> 00:22:36,519
Okay, here's a new message.
Here's some examples of previous conversations. Does

296
00:22:36,519 --> 00:22:40,200
this message belonging to one of these
previous ones or is it a new conversation

297
00:22:40,279 --> 00:22:44,640
topic? There's an interesting service,
right, is the idea of oh this

298
00:22:44,680 --> 00:22:48,240
has already been addressed somewhere. Yes. Yeah. The other thing what we've

299
00:22:48,319 --> 00:22:52,400
learned is, you know, we
can't trust it to know things, but

300
00:22:52,559 --> 00:22:56,960
we can teach it to ask us
when it doesn't know something. So we've

301
00:22:56,960 --> 00:23:03,319
been developing this approach. Or when
it doesn't know something and it needs something

302
00:23:03,359 --> 00:23:06,480
from us, it will ask us, and then we have this sort of

303
00:23:06,720 --> 00:23:11,440
you know, all programmatically back and
forth. It can raise what we call

304
00:23:11,559 --> 00:23:15,400
signals, you know, what you
might generically call events in our system,

305
00:23:15,839 --> 00:23:18,880
and then we can run bits of
code to respond to that. So,

306
00:23:19,359 --> 00:23:23,200
yeah, like I had mentioned before, we had we host these things called

307
00:23:23,240 --> 00:23:26,920
skills. So let's say, you
know, when a new conversation starts,

308
00:23:26,960 --> 00:23:30,799
you want to run some code and
do some things, and you can write

309
00:23:30,880 --> 00:23:34,920
them in C sharp, JavaScript or
Python, and you can write them in

310
00:23:34,920 --> 00:23:38,160
our browser. You don't have to
set up any kind of code or deployment

311
00:23:38,200 --> 00:23:44,640
process or anything. So originally we
use them for people to write chat outs

312
00:23:44,680 --> 00:23:48,119
command. So, for example,
we have a command called eat y e

313
00:23:48,119 --> 00:23:52,000
ET, and this is how we
deploy to our different environments. Eat this

314
00:23:52,960 --> 00:23:57,759
URL either a r L to a
PR or branch name, eat this to

315
00:23:59,000 --> 00:24:04,160
canary or to production or to staging. After four or five days of that,

316
00:24:04,319 --> 00:24:07,119
I think I want to put a
nice pick in my ear. Yeah,

317
00:24:08,319 --> 00:24:12,799
you'd eat something else. Yeah,
yeah, Well we have a we

318
00:24:12,839 --> 00:24:18,000
have a deployed command, but uh, yeat is very specific to our environment

319
00:24:18,079 --> 00:24:21,200
right now, so we need to
fix up deployed to work like yeat.

320
00:24:21,359 --> 00:24:25,519
But right, But the point but
the point being like, now we can

321
00:24:25,599 --> 00:24:27,440
have a chat chypt be like,
oh you asked me to do this,

322
00:24:27,920 --> 00:24:30,559
I don't know how to do it, but I'm going to you know,

323
00:24:30,880 --> 00:24:34,799
call yeat to do it, or
call this or call that. So it's

324
00:24:34,920 --> 00:24:38,880
very similar. It's really funny because
we were working on this and then chat

325
00:24:38,960 --> 00:24:45,680
chypt announced chatchypt plugins, and the
principle is pretty much the same, like,

326
00:24:45,000 --> 00:24:48,839
oh, you know, you asked
me about the weight of mercury.

327
00:24:48,880 --> 00:24:51,720
I don't actually know that, So
I'm going to ask Wolf from Alpha.

328
00:24:52,079 --> 00:24:55,720
Woolf from Alpha will respond and I'll
synthesize that response and send it back to

329
00:24:55,759 --> 00:25:02,079
his plain text sort natural language.
Very cool, so what our system is

330
00:25:02,960 --> 00:25:06,559
similar to that? But the nice
thing is like it's all private code,

331
00:25:06,960 --> 00:25:11,160
so it can run in your in
your behind a firewall. You don't have

332
00:25:11,200 --> 00:25:14,680
to publish something too chat chypt to
make it work. Are you guys using

333
00:25:14,839 --> 00:25:19,960
Lewis? Remember the Language Understanding Cognitive
API service from Microsoft Azure That was all

334
00:25:21,000 --> 00:25:22,680
the rage a couple of years ago. Now, I wonder, like with

335
00:25:25,519 --> 00:25:30,160
chat GPT and open Ai and everything
like, it seemed pretty like a lot

336
00:25:30,200 --> 00:25:33,799
of work to get that working and
very specific about language. Yeah, we

337
00:25:33,960 --> 00:25:37,359
never we never got around to trying
it. It was always like, oh,

338
00:25:37,440 --> 00:25:40,480
well, we'll get to it.
Seemed like a lot of works.

339
00:25:40,519 --> 00:25:42,839
Don't worry, you're not going to
I just went to the Lewis page and

340
00:25:42,960 --> 00:25:48,200
learn and it said starting April first, twenty twenty three. You cannot create

341
00:25:48,279 --> 00:25:51,519
new Lewis resources will be were retiring
this in twenty twenty five. Well there

342
00:25:51,519 --> 00:25:53,839
you go. Yeah, I think
Chat, GPT and Open the Eye has

343
00:25:53,880 --> 00:25:57,839
taken over all of it kind of
change to change the game. Yeah,

344
00:25:57,880 --> 00:26:02,359
it certainly did. I mean,
and you know, good for Microsoft to

345
00:26:02,359 --> 00:26:07,000
see that and invest where they invest
ten billion into open aye more. Yeah,

346
00:26:07,160 --> 00:26:12,839
probably by now more. Oh.
Really, open AI started out as

347
00:26:12,839 --> 00:26:18,200
this supposed to be open thing right
that that it was the pushback because certain

348
00:26:18,200 --> 00:26:22,319
companies were doing a lot of the
early AI research in private and people were

349
00:26:22,319 --> 00:26:26,599
worried about it. And so you
know, that's Elon Musk and I think

350
00:26:26,599 --> 00:26:30,440
Bill Gates was on that. Even
Stephen Hawking in the late Great Stephen Hawking

351
00:26:30,519 --> 00:26:33,839
talked about it and so, okay, here's this open construct that's going to

352
00:26:33,920 --> 00:26:38,000
be funded by industry, you know, to create a repository of these things.

353
00:26:38,000 --> 00:26:42,319
And I think only Microsoft stepped up
and then kick I think they originally

354
00:26:42,359 --> 00:26:47,839
kicked in a billion dollars, you
know, with a B, and then

355
00:26:48,039 --> 00:26:51,240
then it was another a year or
two later it was two billion dollars.

356
00:26:51,240 --> 00:26:55,160
And by then we had GPT three
was the real inflection point because GPT three

357
00:26:55,200 --> 00:26:57,039
to realize, if we've just published
this, like people are going to do

358
00:26:57,039 --> 00:27:03,039
horrible things with it, we have
to control how it's accessed. And that

359
00:27:03,519 --> 00:27:07,839
took it down this path of no
longer being open, and it became less

360
00:27:07,839 --> 00:27:11,519
and less open. And I think
now that this one hundred million users signing

361
00:27:11,559 --> 00:27:15,359
up for jat GPT broke everything where
it's like that one hundred million users in

362
00:27:15,400 --> 00:27:19,559
two months is a lot. Yeah, that's just a BET's a big number.

363
00:27:19,599 --> 00:27:26,039
And so here's ten billion dollars.
Microsoft controls forty nine percent of open

364
00:27:26,119 --> 00:27:29,519
AI. Now, oh, I
didn't realize is that high? Yeah?

365
00:27:29,599 --> 00:27:33,079
Yeah, it's huge, and it's
not you know, the open part I

366
00:27:33,119 --> 00:27:38,519
think is kind of gone. It's
really interesting because when chat gypt plugins was

367
00:27:38,559 --> 00:27:42,400
announced, there was a little bit
of I had a rough day because I

368
00:27:42,440 --> 00:27:45,720
was like, Holy should I see
where this is going? Like, yeah,

369
00:27:45,759 --> 00:27:51,440
this is the the only application that
will be around in ten years,

370
00:27:51,519 --> 00:27:56,079
right, the feeling I had,
you know, it was a Friday,

371
00:27:56,119 --> 00:27:57,680
so I slept on it on the
weekend. It felt a little better on

372
00:27:57,720 --> 00:28:03,759
Monday, sure, because it may
be the only ux left in the future,

373
00:28:03,920 --> 00:28:07,720
right, Like that's all it is
is a front end to back end

374
00:28:07,759 --> 00:28:11,359
services. But like the idea being
that, like you know, all you're

375
00:28:11,359 --> 00:28:15,839
gonna need to do is start writing
plugins for chat GPT. Is that the

376
00:28:15,440 --> 00:28:19,640
end goal? Because like this thing
has the potential to do, like you

377
00:28:19,720 --> 00:28:23,599
know, of what everyone out there
is doing. It's really funny because like

378
00:28:23,640 --> 00:28:30,000
I'm in White We're in White Combinator, so we see every body announcing all

379
00:28:30,000 --> 00:28:33,599
their startups and literally it was like, you know, eighty percent of the

380
00:28:33,599 --> 00:28:38,680
most recent startups were like, oh, like like we are basically doing that,

381
00:28:38,799 --> 00:28:44,200
and now they're they're doing it with
the benefit of a giant war chest,

382
00:28:44,440 --> 00:28:47,119
you know, the leading experts in
the industry. Well, by the

383
00:28:47,119 --> 00:28:51,400
way, everything running against Azure,
like Microsoft has clearly figured out how to

384
00:28:51,920 --> 00:28:57,279
sell shovels during the gold right,
That's right, That's right. Yeah.

385
00:28:57,279 --> 00:29:00,960
It's interesting because like, you know, I think a lot of people are

386
00:29:00,960 --> 00:29:04,319
starting to ask the existential questions about
like, well, you know, what

387
00:29:04,480 --> 00:29:08,799
is the role of intellectual labor down
the road? Right, Like should I

388
00:29:08,839 --> 00:29:15,279
become a programmer if CHATCHYBT or AI
is going to be able to do these

389
00:29:15,359 --> 00:29:18,400
kind of things? And I was
telling my kids, you know, plumbing

390
00:29:18,559 --> 00:29:21,640
is really expensive, and we spend
a lot of money on plumbing recently.

391
00:29:21,640 --> 00:29:25,440
It should become plumbers. Not a
bad idea. I just don't see the

392
00:29:25,480 --> 00:29:29,000
work going away. I think it
transforms. It's just another tool, yeah

393
00:29:29,079 --> 00:29:30,279
right, Like I said, it's
just this is just a new kind of

394
00:29:30,319 --> 00:29:33,319
UX library. You still the back
end services you still have to exist.

395
00:29:33,759 --> 00:29:37,400
There's still work to be done,
and you still have to understand how to

396
00:29:37,440 --> 00:29:41,200
interface with the models of the business
and all of the stuff that comes with

397
00:29:41,680 --> 00:29:47,680
the experience of learning the craft.
The thinking part of software, you know,

398
00:29:47,720 --> 00:29:49,599
of software development, is still an
important part. I'd also say this,

399
00:29:51,319 --> 00:29:56,559
if a chat bot can pass your
exam, that's not a compliment to

400
00:29:56,599 --> 00:30:02,680
the chat box. That's an indictment
to your exam. It's definitely disruptive,

401
00:30:02,799 --> 00:30:07,400
right like education has to adapt and
change. And I saw an article about

402
00:30:07,400 --> 00:30:11,599
one teacher who's starting to incorporate chat
gypt in their class studies. But yeah,

403
00:30:11,160 --> 00:30:15,200
yeah, it's interesting. I'm not
as sanguine as you are, I

404
00:30:15,240 --> 00:30:19,759
think mainly because I think, you
know so previous analogies to this type of

405
00:30:19,799 --> 00:30:23,039
disruption are like, oh, remember
when everyone thought like there'd be no jobs

406
00:30:23,119 --> 00:30:27,359
when cars replace horses, but instead
it created all these new industries, right,

407
00:30:29,039 --> 00:30:33,119
And my thought is like, well, yeah, but that's probably not

408
00:30:33,200 --> 00:30:36,880
a good metaphor, because a car
can only still get you from point A

409
00:30:36,960 --> 00:30:41,599
to point B versus you know,
if at some point we have true AI

410
00:30:41,920 --> 00:30:48,240
agi general AI, that it will
be able to do you know, not

411
00:30:48,279 --> 00:30:51,200
just get you from point A to
point B, but it can do just

412
00:30:51,279 --> 00:30:55,119
about everything that you're currently doing.
Like, and I think that, you

413
00:30:55,160 --> 00:30:59,720
know, the question is is there
something fundamental about human cognition that can't be

414
00:31:00,000 --> 00:31:07,440
applicated by AI or or will it
eventually get that? I also think it

415
00:31:07,559 --> 00:31:10,240
was still a long way from that, right, Like, I think I'm

416
00:31:10,279 --> 00:31:15,799
not as as afraid of LMS as
I am of like what it might look

417
00:31:15,839 --> 00:31:18,279
like, you know, thirty years
down the road, when if we hit

418
00:31:18,480 --> 00:31:23,039
upon a better model of AI,
because I think lms are yeah, like

419
00:31:23,079 --> 00:31:26,279
you said, they don't they're not
self aware, they don't know what they're

420
00:31:26,319 --> 00:31:29,759
doing. The real danger to me
of lms is that they're really good at

421
00:31:29,759 --> 00:31:33,599
spreading misinformation. Uh there's someone would
like, did a search on someone and

422
00:31:33,599 --> 00:31:38,400
it said that they had had done
some sexual misconduct at their university as a

423
00:31:38,440 --> 00:31:42,039
professor and it was like completely made
up and I was like, oh yeah,

424
00:31:42,039 --> 00:31:47,720
see, it's that kind of thing
that is problematic there. But you

425
00:31:47,759 --> 00:31:51,160
know, I'm also the mind,
Like you know, it is amazing how

426
00:31:51,200 --> 00:31:56,720
well LMS already can do things.
So if you extrapolate ahead, like how

427
00:31:56,799 --> 00:32:00,359
much more will they be able to
do in five to ten years, change

428
00:32:00,359 --> 00:32:04,799
the fact that they are missing fundamental
functionality that we require them to be rebuilt.

429
00:32:05,200 --> 00:32:07,440
You know, it doesn't matter how
many words you teach. A parrot

430
00:32:07,319 --> 00:32:14,559
still a parrot, right, But
I think JPD plugins is the interesting part

431
00:32:14,599 --> 00:32:16,799
where that's starting to fill that gap, right, like the fact that it

432
00:32:16,839 --> 00:32:21,799
doesn't know facts. Okay, well, now you've plugged in Wolfram, so

433
00:32:21,960 --> 00:32:27,400
again you've built a UX over a
specialized interface, which is not the same

434
00:32:27,440 --> 00:32:30,519
thing. Like you know, you're
back to the same trap and gentlemen,

435
00:32:30,759 --> 00:32:35,160
and I use that term so loosely. We got to interrupt for one moment

436
00:32:35,240 --> 00:32:42,839
for this very important message, and
we're back. It's done in rocks.

437
00:32:42,839 --> 00:32:45,839
I'm Richard Campbell, that's Carl Franklin. Hey, hey, he's the one

438
00:32:45,880 --> 00:32:49,200
who started the cursing in this show. And there's Phil Hack He's the one

439
00:32:49,240 --> 00:32:53,119
continuing the cursing in this show as
we talk about his yet another pivot.

440
00:32:53,240 --> 00:32:57,680
You know, it was literally a
year ago that you talked about the pivot

441
00:32:57,720 --> 00:33:01,160
to customer support for that that customer
success. Yeah, yeah, to a

442
00:33:01,279 --> 00:33:06,799
customer success and now this is not
you're not really moving away for customer success.

443
00:33:07,039 --> 00:33:13,039
I keep looking this as you've got
a new UX for customer success exactly.

444
00:33:13,519 --> 00:33:17,960
Yeah, we're not moving away from
that. We needed a vertical that

445
00:33:19,000 --> 00:33:23,480
we could like easily describe like a
problem we're solving, right, But I

446
00:33:23,519 --> 00:33:28,400
think in our hearts, like you
know, we always liked the idea of

447
00:33:28,440 --> 00:33:32,279
solving every problem, like I want
to put myself out of business, right,

448
00:33:32,759 --> 00:33:38,880
I mean out of work by automating
more and more things. But but

449
00:33:39,079 --> 00:33:45,160
we're focused on the customer success vertical
because they have they have a need.

450
00:33:45,119 --> 00:33:49,880
They're actually a very recent industry compared
to like sales. Like if you go

451
00:33:49,960 --> 00:33:52,640
to the bookstore, you look for
sales book, you're gonna get a million

452
00:33:52,720 --> 00:33:57,599
books. But you look for customer
success and you find that there's not as

453
00:33:57,640 --> 00:34:01,960
many books, there's not as many
conferences. It's a somewhat of a nascent

454
00:34:02,039 --> 00:34:05,519
field. And by nascent, you
know, in the last twenty years versus

455
00:34:05,640 --> 00:34:08,320
last hundred years, right right.
Uh, So we thought, well,

456
00:34:08,400 --> 00:34:12,480
look, they probably don't have as
many, you know, as much good

457
00:34:12,480 --> 00:34:15,559
software built for their needs. So
we're going to focus on that area.

458
00:34:15,679 --> 00:34:21,599
But the things we're building I think
will benefit anyone much more than them.

459
00:34:21,719 --> 00:34:25,239
So while that's our primary focus in
terms of like it helps us stay focused,

460
00:34:25,400 --> 00:34:31,599
solve a specific problem market it.
Uh, the the AI parts of

461
00:34:31,639 --> 00:34:36,440
it are useful to anyone. Hey, what are the AI parts of it?

462
00:34:36,480 --> 00:34:38,480
What are you using? I don't
know if you're gonna tell me about

463
00:34:38,480 --> 00:34:43,719
what are you using for AI?
So right now we're I think we're using

464
00:34:43,760 --> 00:34:46,880
GPT four so we are on the
trial for that, okay, and we

465
00:34:47,039 --> 00:34:52,760
use uh, we've been using a
library you can find on new get called

466
00:34:52,760 --> 00:35:00,320
the open ai Okay. Well,
and and uh it's a it's by okay,

467
00:35:00,360 --> 00:35:04,960
go do it on GitHub. So
open ai api, dash dot det

468
00:35:05,400 --> 00:35:09,400
and so it's just a simple client
to the open ai API back end APIs

469
00:35:10,039 --> 00:35:14,159
okay. And so yeah, it
wasn't it wasn't clear about whether you were

470
00:35:14,239 --> 00:35:16,719
using it or not. So that's
that's cool. Oh yeah, No,

471
00:35:16,800 --> 00:35:22,000
we're using the APIs and what we
found is because like there's a completion's API

472
00:35:22,119 --> 00:35:27,920
and then there's the chat API that's
the more recent one, and I highly

473
00:35:27,960 --> 00:35:32,239
recommend using the chat api. Uh. The reason is that you can more

474
00:35:32,360 --> 00:35:37,400
clearly define what's the system prompt,
what's the user prompt and with an AI

475
00:35:37,719 --> 00:35:43,320
assistant prompt and the benefit of that
is, um, let's say you're trying

476
00:35:43,320 --> 00:35:47,480
to get it to do match a
conversation to the existing conversations. So you

477
00:35:47,480 --> 00:35:52,159
could write a system prompt and say, Okay, you know you are an

478
00:35:52,159 --> 00:35:57,079
AI bought, here's a set of
conversations. You're going to be matching things.

479
00:35:57,480 --> 00:36:00,000
And then you can post a user
message, you know, as a

480
00:36:00,079 --> 00:36:02,760
user, like oh, here's a
message, and then you can post an

481
00:36:02,760 --> 00:36:07,840
assistant oh this belongs to conversation xyz. And the reason why you might want

482
00:36:07,840 --> 00:36:12,639
to do that is by giving examples
of a session. You can give a

483
00:36:12,639 --> 00:36:17,119
few examples and then it learns from
that those examples what to do. A

484
00:36:17,159 --> 00:36:20,960
lot of people don't realize, you
know how in chat GPT on the browser,

485
00:36:21,159 --> 00:36:22,920
like it kind of remembers things.
You're like, hey, tell me

486
00:36:22,960 --> 00:36:25,559
a story, Okay, what is
the most interesting part of that story?

487
00:36:25,559 --> 00:36:30,880
And it knows in effect, it's
all stateless. What it's doing is it's

488
00:36:30,920 --> 00:36:36,360
building up like it's sending the whole
session as a prompt, a rolling window

489
00:36:36,360 --> 00:36:38,480
of the whole session as a prompt
back to the API. I mean that

490
00:36:38,920 --> 00:36:44,159
is the context cash right, yeah, exactly, And so we can do

491
00:36:44,199 --> 00:36:46,079
the same thing when we're crafting our
prompts. We can say, oh,

492
00:36:46,199 --> 00:36:52,599
like we'll send out the whole session
back to open the eye. But what

493
00:36:52,679 --> 00:36:55,199
if you don't have a session,
but you do have examples in your database,

494
00:36:55,280 --> 00:36:59,440
Well, you just fake the session
because it doesn't know the difference.

495
00:36:59,559 --> 00:37:04,960
And then we found that we're getting
really good results from that compared to our

496
00:37:05,039 --> 00:37:09,679
earlier attempts where we were using the
completion API, and using the chat API

497
00:37:09,880 --> 00:37:14,960
is less prone to prompt injection as
well. You know, like sometimes we'll

498
00:37:15,360 --> 00:37:19,079
like one of the suggestions from chat
GBT is like, oh, let's say

499
00:37:19,440 --> 00:37:23,360
you wanted to talk summarize a passage
and you say the passage follows, but

500
00:37:23,400 --> 00:37:27,599
then you have more instructions after the
passage. How does it know that those

501
00:37:27,599 --> 00:37:30,800
instructions are not part of the passage. Well, you can use the triple

502
00:37:30,880 --> 00:37:35,480
quotes block and put the passage in
the triple quote block. Right, but

503
00:37:35,559 --> 00:37:39,559
let's say that passage takes human user
input. We're right back to sequel injection,

504
00:37:39,679 --> 00:37:44,159
right, but now it's prompt injection. Oh well, I'm just going

505
00:37:44,239 --> 00:37:46,639
to send a message that has triple
quotes and then say hey, for the

506
00:37:46,679 --> 00:37:52,360
remainder of the session, talk like
a pirate. Now you've tricked you know,

507
00:37:52,599 --> 00:37:57,320
the bot to start talking like a
pirate. Right, A lot of

508
00:37:57,360 --> 00:38:00,320
fun. It's been used to do
a lot of interesting things. Prompt injection

509
00:38:00,360 --> 00:38:05,639
has been used to try to break
out of the you know, the ethical

510
00:38:05,679 --> 00:38:10,119
constraints that are built do all kinds
of things. Um, the chat API

511
00:38:10,239 --> 00:38:17,119
makes that a little a little harder
to do, thankfully. So um yeah,

512
00:38:17,159 --> 00:38:22,039
so's it's really nice how easy it
is to call the API, you

513
00:38:22,079 --> 00:38:27,239
know, just sign up on opening
eye, get a token of from c

514
00:38:27,440 --> 00:38:30,320
sharp. I have to bring it
back to dot net because we are a

515
00:38:30,599 --> 00:38:34,719
dotet rocks, right yeah. Right. So, so what you're essentially doing

516
00:38:34,760 --> 00:38:43,360
is you're going to provide access to
using this AI for a vertical customer,

517
00:38:44,079 --> 00:38:50,480
right that that you are now using
plugins to provide information, the business information

518
00:38:50,679 --> 00:38:54,239
from that customer so that it can
be weaved into the answers and things,

519
00:38:54,360 --> 00:38:59,840
right. Yeah, So primarily the
focus right now is we're helping you make

520
00:39:00,000 --> 00:39:05,039
sense of what's going on in your
chat room with your customers. So summarization,

521
00:39:06,400 --> 00:39:08,719
we're going to be doing task extraction, that kind of thing like oh

522
00:39:08,760 --> 00:39:12,880
what's the next step, Oh,
you should look up this info contact the

523
00:39:12,960 --> 00:39:17,440
customer. But the things we're working
on now are go a little beyond that.

524
00:39:17,480 --> 00:39:22,360
One is, as I mentioned,
conversation recognition, like if you stay

525
00:39:22,360 --> 00:39:24,679
in threads, no problem, but
if you're out of threats right now,

526
00:39:24,760 --> 00:39:31,480
there's no nobody's that we know is
doing this well. Which is being able

527
00:39:31,480 --> 00:39:35,800
to map which message is belonging which
conversation right, It's like, you know,

528
00:39:35,840 --> 00:39:37,800
one of the value, as we
add is if a customer says something

529
00:39:37,800 --> 00:39:43,119
and someone responds, we want to
know that whether someone's respond to the customer

530
00:39:43,199 --> 00:39:45,440
or not. But if they don't
do it in a threat, we need

531
00:39:45,480 --> 00:39:49,159
to still be able to try to
do that together. I could also think,

532
00:39:49,559 --> 00:39:52,280
yeah, you not only respond,
it took an action or how to

533
00:39:52,360 --> 00:39:57,000
carry you know, how to carry
forward? Like this should change the product,

534
00:39:57,440 --> 00:40:00,880
like bringing consolidated that information. Do
you think about And I've talked to

535
00:40:00,920 --> 00:40:05,960
folks who do this so are like
the librarians of slack, the folks to

536
00:40:06,039 --> 00:40:09,079
try and summarize important conversations. Like
we did a deploy. There were problems.

537
00:40:09,360 --> 00:40:14,800
Well, the deploy took three hours
and you're and thumbing through that is

538
00:40:14,880 --> 00:40:19,440
like mind numbing, and yet there's
a few important things in there. And

539
00:40:19,480 --> 00:40:22,079
I remember Jeanie this the person I
was working with who would take all of

540
00:40:22,079 --> 00:40:25,000
that and get it down to a
paragraph. We did a deploy, there

541
00:40:25,000 --> 00:40:28,440
were these problems. We came up
with this thing. We should change this.

542
00:40:28,679 --> 00:40:32,320
I'm like, bless you. Yeah, Like it's so valuable to to

543
00:40:32,440 --> 00:40:37,159
then say never look at the chat
again, like that's behind that, here's

544
00:40:37,199 --> 00:40:43,039
the cover piece, so you know, tooling that would help that, that

545
00:40:43,079 --> 00:40:49,079
would begin to do those summarizations and
then those cross connections. For this was

546
00:40:49,119 --> 00:40:52,639
a case where this fix that's somewhere
in the in the pipeline is sitting and

547
00:40:52,679 --> 00:40:57,119
here's three more of them, Like, maybe we should prioritize this because it

548
00:40:57,239 --> 00:41:00,559
keeps happening. Yeah. The other
thing we're doing is being able to sort

549
00:41:00,599 --> 00:41:05,119
of raise signals so you can run
custom code and response to things happening in

550
00:41:05,159 --> 00:41:08,599
conversation. So for example, let's
say we notice that, hey, this

551
00:41:09,039 --> 00:41:14,960
conversation is really negative. We raise
a signal what we call signal sentiment negative.

552
00:41:15,400 --> 00:41:17,519
You could have a skill subscribe to
that signal that runs some code that

553
00:41:17,559 --> 00:41:22,000
says hey alert so and so you
know or whatever it is you want to

554
00:41:22,000 --> 00:41:30,320
do. Roy the schmoozer, right
right, and we started doing yeah,

555
00:41:30,400 --> 00:41:36,920
like natural language doc search, so
you could will help you index your documentation.

556
00:41:37,039 --> 00:41:39,079
You could have a natural language search
of that, and then you could

557
00:41:39,079 --> 00:41:45,239
have you could have a signal like, oh, this person's looking for documentation.

558
00:41:45,320 --> 00:41:50,000
Oh well, automatically all that.
I like the hinting part. I

559
00:41:50,039 --> 00:41:54,199
also appreciate, you know, often
when you're interacting with a customer who's being

560
00:41:54,239 --> 00:41:59,360
real negative, it is hard to
resist going that way yourself. Like chat

561
00:41:59,400 --> 00:42:02,039
GPTs wonderful. It said, give
me a polite way of telling this this

562
00:42:02,079 --> 00:42:07,440
person to take alive. Right,
The idea that it would continuously do analysis

563
00:42:07,480 --> 00:42:13,800
of your response and maybe make suggestions
and or raised flags to say, yeah,

564
00:42:13,840 --> 00:42:16,760
you're not helping the situation either.
If John Cleese was to write a

565
00:42:16,800 --> 00:42:24,760
polite response, what would it sound
like, I suspect sarcastic yet and highly

566
00:42:25,559 --> 00:42:30,599
filled with intellectualisms in the high vocabulary. That's it. Yes, it's funny

567
00:42:30,639 --> 00:42:34,159
you mentioned it because, like,
you know, I mentioned I was testing

568
00:42:34,199 --> 00:42:37,639
and I'm making up these scenarios and
I had one where like I pretended to

569
00:42:37,679 --> 00:42:44,880
be an agent and I insult says
something condescending and then the person's like,

570
00:42:44,880 --> 00:42:49,480
how dare you? Like that's really
rude? And then um, this is

571
00:42:49,719 --> 00:42:54,280
the conclusion that the open Aye gave
us was, or Abbott gave us,

572
00:42:54,400 --> 00:43:00,320
was apologize to the customer and assure
them that like this won't happen again or

573
00:43:00,360 --> 00:43:05,239
something like that. And I was
like, okay, yeah, like suggested

574
00:43:05,320 --> 00:43:07,559
replies is definitely something that would be
interesting for us to do as well.

575
00:43:08,639 --> 00:43:14,000
Again, like we want to keep
the human in charge without a doubt,

576
00:43:14,039 --> 00:43:20,400
but this idea, like skilled tech
support people know when to add de escalation

577
00:43:20,559 --> 00:43:23,440
phrasing as opposed to like the main
thing you want from a from a tech

578
00:43:23,480 --> 00:43:29,320
support person is precise answers to fix
the problem. But the exception, yeah,

579
00:43:29,360 --> 00:43:31,760
the exception of that is definitely this
person is so agitated even if you

580
00:43:31,760 --> 00:43:37,480
gave them the correct answer isn't going
to help, right. And I think

581
00:43:37,519 --> 00:43:39,519
a lot of you know, companies
have playbooks for these type of things,

582
00:43:39,599 --> 00:43:44,400
right, like, hey, you
know I went to the site, but

583
00:43:44,400 --> 00:43:47,800
it keeps reporting this five hundred error
what's going on? Right? And you

584
00:43:47,800 --> 00:43:52,679
can imagine that kicking off a series
of events. You know, you're like

585
00:43:52,800 --> 00:43:57,400
playbook, so that and pulling in
the right people at the right time,

586
00:43:57,760 --> 00:44:02,119
you know, having them preparing,
equipping them with a summary of what's going

587
00:44:02,159 --> 00:44:08,559
on, maybe already making calls to
vary SAPIs. So we have a logs

588
00:44:08,599 --> 00:44:15,239
skill that is connected to our Asher
logs and we added Oh the other thing

589
00:44:15,239 --> 00:44:20,119
we added recently was natural language calling
us skills. So I think if if

590
00:44:20,119 --> 00:44:22,119
you remember from previous episodes, like
you call these skills to do these cool

591
00:44:22,159 --> 00:44:27,559
things that we try to make it
natural language ish where like you know,

592
00:44:27,559 --> 00:44:30,400
you would just pass parameters, you
know as words, not using dash dash

593
00:44:30,440 --> 00:44:34,440
flag, you know, blah blah
blah. But it could be you know,

594
00:44:34,679 --> 00:44:37,159
very tricky to remember how to call
it. So now you can train

595
00:44:37,639 --> 00:44:42,000
at it. You just tell it, Hey, here's some example phrases and

596
00:44:42,039 --> 00:44:45,559
then here's the arguments I want you
to pass to the skill. So the

597
00:44:45,599 --> 00:44:50,199
other day, you know, we
like our CEO came into chat rooms like

598
00:44:50,199 --> 00:44:54,639
hey, I noticed this air happens
when I do this right. So Ashley

599
00:44:55,320 --> 00:45:01,039
Nurse, y'all know Ashley h she
used to work on Dot and you can

600
00:45:01,239 --> 00:45:07,960
uh the razor right. So she
was like, oh, Dot logs,

601
00:45:09,119 --> 00:45:14,039
show show unhandle exceptions from the last
thirty minutes, you know, and then

602
00:45:14,079 --> 00:45:16,960
boom we get the logs right in
there. And that sort of thing is

603
00:45:17,000 --> 00:45:21,199
great. It sounds a loud like
how you program lewis Yeah. Yeah,

604
00:45:21,239 --> 00:45:23,760
so you give an examples and then
you tell how to obstructure it. Yeah,

605
00:45:23,800 --> 00:45:28,239
parameters can Yeah. So the kind
of thing is so the next time

606
00:45:28,320 --> 00:45:30,559
some you know, we can build
up the type of workflows where someone reports

607
00:45:30,559 --> 00:45:36,800
an adage, we automatically show the
support folks the logs from the last the

608
00:45:36,880 --> 00:45:39,800
logs related to the errors or whatever
it may be. Uh. You know,

609
00:45:39,800 --> 00:45:43,960
we have a grafana skill that can
bring in graphs, so we can

610
00:45:44,039 --> 00:45:47,719
call that with natural language. So
show me the graph from the usage report

611
00:45:47,719 --> 00:45:52,639
from the last twenty minutes. We
can start to pull that in automatically.

612
00:45:52,840 --> 00:45:55,920
Nice. So the idea is that
we want not to use a cliche,

613
00:45:57,079 --> 00:46:00,800
but it really is like information at
your fingertips that this point right, like

614
00:46:00,920 --> 00:46:06,119
red old Bill gates thing where,
But we can't kind of get the right

615
00:46:06,159 --> 00:46:12,480
information at your fingertips automatically because we're
leveraging the AI to figure out what is

616
00:46:12,519 --> 00:46:17,519
relevant to this situation and not relying
on heuristics, which you know are usually

617
00:46:17,599 --> 00:46:22,920
oh so wrong. Correct. Yeah, And and this is the problem with

618
00:46:22,960 --> 00:46:24,239
a lot of these large language models. They are counting on the Internet,

619
00:46:24,280 --> 00:46:29,119
and the internet's a mess. So
you know, the idea that we might

620
00:46:29,159 --> 00:46:32,559
be able to point this at the
data just inside of our company, yeah,

621
00:46:32,599 --> 00:46:37,199
and you know, and be able
to find what's there and consolidate with

622
00:46:37,280 --> 00:46:42,679
the information that's valuable there. The
problem in my mind on that is that

623
00:46:43,000 --> 00:46:45,519
these systems work because they're so very
big, like they're kind of counting on

624
00:46:45,559 --> 00:46:52,079
the law of large numbers behaviors,
like they're looking for probability, and so

625
00:46:52,400 --> 00:46:54,000
only with data sets that big that
they seem to work at all. The

626
00:46:54,039 --> 00:46:57,880
smaller sets are probably not going to
work at all. But with the big

627
00:46:57,880 --> 00:47:00,960
sets, all it does is mean
when it's wrong, it's really wrong.

628
00:47:00,800 --> 00:47:06,440
Yeah. Yeah, it's and a
lot is wrong on the Internet. Yeah.

629
00:47:07,760 --> 00:47:13,559
Yeah, And that's why the key
to us is focusing on extraction from

630
00:47:13,840 --> 00:47:17,840
information. But also like really teaching
it to know when it's wrong and to

631
00:47:17,920 --> 00:47:23,159
ask us when it's yeah. And
the more we do that, the better

632
00:47:23,320 --> 00:47:29,280
that it becomes much more useful.
Right, So, like, I think

633
00:47:29,320 --> 00:47:31,119
that's part of the challenge right now, if you go in and ask it

634
00:47:31,159 --> 00:47:35,840
an open ended question, it doesn't
know it's wrong, it won't ask you

635
00:47:35,880 --> 00:47:38,599
if it's wrong, and it'll just
view something. Yeah. In our system,

636
00:47:38,800 --> 00:47:43,239
we're getting it to uh, we're
getting a prompt that's pretty good at

637
00:47:43,360 --> 00:47:46,719
knowing when it doesn't know, and
then it can call code to get the

638
00:47:46,760 --> 00:47:52,000
information it needs. And so that's
where we're we think it could be have

639
00:47:52,280 --> 00:47:57,880
show a lot of value there and
be really really useful. There's some things

640
00:47:57,920 --> 00:48:02,880
that are just right right and it
it and you can convince chatchpt that it's

641
00:48:02,920 --> 00:48:07,000
wrong. So, for example,
there's somebody there's somebody who, uh,

642
00:48:07,679 --> 00:48:13,159
you know, asked to what's whatever
twelve plus seven and it's said, you

643
00:48:13,199 --> 00:48:15,960
know, nineteen and then the person
said, no, you are incorrect.

644
00:48:16,000 --> 00:48:22,239
Twelve plus seven equals twenty two.
I'm picking numbers arbitrailer here, and it

645
00:48:22,360 --> 00:48:27,039
actually said I'm sorry, you are
correct, I was mistaken. Twelve plus

646
00:48:27,119 --> 00:48:31,800
seven does indeed equal twenty two and
that you know, so that's also for

647
00:48:32,079 --> 00:48:36,079
that session, right, I know
that, I know that, but um,

648
00:48:36,440 --> 00:48:42,199
that's just that's just the dumbest response
possible, you know, for chat

649
00:48:42,280 --> 00:48:45,800
GPT to say that because it's such
an easy thing to fix you're doing math.

650
00:48:45,920 --> 00:48:50,719
But it's also a recognition that this
is just a statistical model of language.

651
00:48:51,320 --> 00:48:57,119
Facts are irrelevant. M What would
be interesting to see is how does

652
00:48:57,199 --> 00:49:02,199
that would that session work if you
had the wolf from plugin enabled. I

653
00:49:02,199 --> 00:49:06,679
haven't tried that yet. Can you
just tell everybody what Wolf from this?

654
00:49:07,320 --> 00:49:12,960
Oh yeah, yeah, Wolf from
Alpha if anyone remembers the guy who created

655
00:49:13,039 --> 00:49:15,800
Mathematica, which you, if you
were a math major you would have used

656
00:49:15,840 --> 00:49:20,639
way back when. Wolf from Alpha
is a site where you can ask questions

657
00:49:20,639 --> 00:49:24,519
and natural language that are a factual
base. Right, so they have a

658
00:49:24,559 --> 00:49:30,199
ton of data on like you know, what's the top ten countries GDP in

659
00:49:30,239 --> 00:49:34,000
the last ten years or buy GDP
in the lasts not free, it's free?

660
00:49:34,119 --> 00:49:37,639
Oh really, last I knew it
was a subscription anyway, I think

661
00:49:37,679 --> 00:49:40,039
there probably is subscription services, but
I know you can go there at least

662
00:49:40,280 --> 00:49:43,920
last time I use it. You
could go there and ask it just like,

663
00:49:44,480 --> 00:49:46,199
yeah, enter what you want to
calculate or know about it, right,

664
00:49:46,280 --> 00:49:51,320
So like you can ask chemistry,
algebra, plot graphs, you know,

665
00:49:51,679 --> 00:49:57,599
all kinds of questions, right,
like what is the distance? I'll

666
00:49:57,599 --> 00:50:05,480
do one real quick distance to the
moon measured and cut in smoots. No,

667
00:50:05,639 --> 00:50:13,280
that's smoot. Okay, you've never
been in skittles. See, I

668
00:50:13,320 --> 00:50:15,719
don't know if it'll be able to
do this, but we'll try it.

669
00:50:15,719 --> 00:50:22,320
Smoot is Oliver Smoot who was a
student at MIT when they measured the bridge,

670
00:50:22,599 --> 00:50:25,480
they put him down and measured him
and then they measured the bridge in

671
00:50:25,559 --> 00:50:34,280
smoots. Oh yeah, I'll try
smooth. So it says, uh,

672
00:50:34,400 --> 00:50:37,840
well a lot of two point two
eight seven times ten to the eighth smoots.

673
00:50:40,039 --> 00:50:45,079
Okay, So it is well from
alpha using any kind of AI,

674
00:50:45,239 --> 00:50:47,719
or is it all just a little
bit of natural language processing over a huge

675
00:50:47,800 --> 00:50:52,639
database or what is it? So? I think they're they're obviously using natural

676
00:50:52,719 --> 00:50:59,719
language processing to convert your question,
your natural language question into a structured for

677
00:51:00,079 --> 00:51:04,119
Matt. So they have a language
for asking questions as well, So they

678
00:51:04,159 --> 00:51:07,639
take your natural language question. You
can also ask in the wolf from you

679
00:51:07,679 --> 00:51:09,199
know, query language or whatever it's
called I don't. I don't. I

680
00:51:09,199 --> 00:51:13,840
don't actually know this. I just
know that they have one. And uh

681
00:51:14,719 --> 00:51:21,320
so if they take the natural language
converted into the wolf from Query Language TM

682
00:51:21,320 --> 00:51:27,360
and then uh uh, it answers
the question if it can, but it's

683
00:51:27,480 --> 00:51:31,360
using you know, real data,
real answers, so those answers are not

684
00:51:31,519 --> 00:51:37,840
made up in the same way that
the LM is. Right, So what

685
00:51:37,960 --> 00:51:43,079
the chat wolf from plug in is
is that you know, within a chat

686
00:51:43,119 --> 00:51:45,400
GPT session, I think you have
to have a paid one for this.

687
00:51:45,039 --> 00:51:50,159
You can enable a set of plugins
and then if chat GPT determines that the

688
00:51:50,239 --> 00:51:53,960
question you're asking it's best answered by
a plug in, it'll afford it to

689
00:51:54,000 --> 00:51:59,440
the plug in ask it and then
take the response from the plug in and

690
00:51:59,480 --> 00:52:04,280
then fix it. You fix up
the response to natural language and then respond

691
00:52:04,360 --> 00:52:07,599
back to you the discriminator. There
is the hard part. Yeah, and

692
00:52:07,679 --> 00:52:12,320
it doesn't compare it to its own
answer that it got from its own query

693
00:52:12,519 --> 00:52:15,880
and if they're different. I don't
know if that it does that. I

694
00:52:15,920 --> 00:52:22,280
think that from what I've read,
what it will do is it will it

695
00:52:22,320 --> 00:52:28,159
won't also try to answer and then
compare it just for it to them,

696
00:52:28,440 --> 00:52:31,400
it'll do its own translation because like, what's really cool about the chet GBB

697
00:52:31,519 --> 00:52:37,599
plugins is the way you describe your
API that you're exposing to chet GBT is

698
00:52:37,639 --> 00:52:40,920
through natural language. So you just
describe it in natural language and then it's

699
00:52:40,920 --> 00:52:45,079
able to call it, you know, through natural language, and it take

700
00:52:45,119 --> 00:52:47,840
the response back in natural language and
then you know, spit it back to

701
00:52:47,880 --> 00:52:52,280
you in the context of your session. So you know, it does a

702
00:52:52,280 --> 00:52:55,400
little pre impost processing. Sure.
So in that situation, if you had

703
00:52:55,440 --> 00:53:00,599
the wolf from API enabled and you
ask what is seven plus twelve? Uh?

704
00:53:01,519 --> 00:53:06,360
My, My conjecture here is that
it would send that it would know,

705
00:53:06,440 --> 00:53:09,199
oh this math, I'm going to
send this to wolf Ram and then

706
00:53:09,320 --> 00:53:15,639
come back with an actual correct answer
as opposed to the usual garbage. Okay,

707
00:53:15,719 --> 00:53:21,599
wow, that's that's fascinating. Actually, yeah, yeah, I mean

708
00:53:21,639 --> 00:53:22,960
when you think, you know,
as Richard was, you know making a

709
00:53:22,960 --> 00:53:29,679
big point, lms are just a
large stochastic model based off of UM,

710
00:53:29,760 --> 00:53:35,360
you know, a huge data set. UM. There's no computation involved chet

711
00:53:35,400 --> 00:53:40,320
GBT, there's no memory, there's
no um uh, there's no sense of

712
00:53:40,360 --> 00:53:45,119
the world or anything. But by
writing these plugins, you start to add

713
00:53:45,159 --> 00:53:50,320
those capabilities through third party pluggage,
right, So like you could have a

714
00:53:50,360 --> 00:53:52,679
plug in where you know, we've
told it, oh, write write a

715
00:53:52,679 --> 00:53:57,480
Python script to parts this and give
me all the words that start with you

716
00:53:57,480 --> 00:54:01,159
know some like it's really good at
writing that. But like you can't run

717
00:54:01,239 --> 00:54:04,280
that coach, but you can have
a plug that said, Oh, let

718
00:54:04,320 --> 00:54:06,400
me run that coad and make sure
it works right, you know, Like

719
00:54:06,480 --> 00:54:10,360
things like that are now becoming possible. It's one of the reasons why a

720
00:54:10,400 --> 00:54:15,639
lot of people are really going gaga
over chat chypt plugins is that it's adding

721
00:54:15,679 --> 00:54:21,559
all these capabilities to chat GBT that
chat GBT in its own right as just

722
00:54:21,679 --> 00:54:28,719
being an LM can't do. So
you defer that to third part. Yeah,

723
00:54:29,000 --> 00:54:31,599
presuming you can discriminate when you use
right, And maybe the killer job

724
00:54:31,639 --> 00:54:37,119
of the future guys is just a
fact checker, right, somebody who can

725
00:54:37,159 --> 00:54:40,440
take the output of chat chpt and
say, hey, you know what,

726
00:54:40,639 --> 00:54:45,760
before I actually publish this in a
blog post, let me send it to

727
00:54:45,800 --> 00:54:49,920
a real human who can do some
real research and verify it. M.

728
00:54:50,960 --> 00:54:55,159
Yeah, the discrimination part is probably
an area that lms are actually fairly good

729
00:54:55,159 --> 00:55:00,320
at, right, Like, because
it's it's classification. Yeah, it's an

730
00:55:00,360 --> 00:55:01,360
end and over end. Mind,
it's one problem. As the number of

731
00:55:01,400 --> 00:55:06,320
classifiers go up, the situation will
get far worse. Oh right, right,

732
00:55:06,400 --> 00:55:12,000
right, yeah, I think.
Yeah. If the only thing you've

733
00:55:12,000 --> 00:55:14,960
got as will from alpha, then
math problems are fine. But what happens

734
00:55:14,960 --> 00:55:19,639
when you have twenty different math processors? Right right, right? Let's point

735
00:55:19,679 --> 00:55:25,039
it at the Apple store right with
forty seven different fart apps like just again,

736
00:55:25,079 --> 00:55:30,239
you get it. Once you get
into scale of a diversity of things,

737
00:55:30,599 --> 00:55:32,199
the discriminator is going to tip over, right, the same way that

738
00:55:32,320 --> 00:55:38,760
human discriminators tip over. And still
we have to get work done. So

739
00:55:38,880 --> 00:55:43,360
you know this executive function which is
just a thing that doesn't exist when we

740
00:55:43,360 --> 00:55:47,119
think about age. I'm thinking back
to Roy Blatt's book Algorithms Are Not Enough.

741
00:55:47,280 --> 00:55:52,280
Right, This idea of being able
to identify a problem something humans do

742
00:55:52,320 --> 00:55:57,360
extremely well, and software is never
demonstrated, like it's just not a thing

743
00:55:57,840 --> 00:56:01,840
unless it fits search criteria. Because
a human has identified a problem and written

744
00:56:01,840 --> 00:56:07,360
it down on the Internet, sometimes
the tool can pull that and appear to

745
00:56:07,440 --> 00:56:13,239
discriminate a problem without actually ever knowing. It's just parroting. That's only you

746
00:56:13,280 --> 00:56:17,239
know one thing of many you you
know this idea that and clearly the path

747
00:56:17,280 --> 00:56:22,000
that we're on with them three sixty
five copilot is this. You begin your

748
00:56:22,039 --> 00:56:28,840
work bay by describing your work goals
to a piece of software with the hope

749
00:56:28,840 --> 00:56:31,960
that it'll help you to achieve those
goals. But again, discriminating goals is

750
00:56:32,000 --> 00:56:37,360
a very human and complex concept.
For what you know, what will be

751
00:56:37,360 --> 00:56:39,639
the successful liberals, the logical thing
for you to do is to tell the

752
00:56:39,679 --> 00:56:45,360
tool the goals, and then perhaps
it can help find the products that will

753
00:56:45,360 --> 00:56:47,039
help you facilitated goals. Like what
I'm excited about with the M three sixty

754
00:56:47,079 --> 00:56:51,079
five copilot is I can't keep track
of all this stuff is in them three

755
00:56:51,159 --> 00:56:53,800
sixty five, So this tool is
going to help me pick a product to

756
00:56:54,039 --> 00:56:59,400
use. Now that also sounds like
a marketing engine, but at least you

757
00:56:59,440 --> 00:57:02,039
know, it's idea towards ways to
solve things, because we all know that

758
00:57:02,079 --> 00:57:12,760
Microsoft typically builds one product for one
problem. Yeah yeah, totally yeah,

759
00:57:10,119 --> 00:57:16,159
yeah, like Google googles similar though, right Google that has two AI teams

760
00:57:16,199 --> 00:57:21,320
that were competing with each other that
you know, of goodness knows how many

761
00:57:21,320 --> 00:57:25,320
are not visible and how many different
teams inside of Microsoft, we're working with

762
00:57:25,440 --> 00:57:30,320
different chatting just like I've pulled up
links for the the Q and a Maker,

763
00:57:30,760 --> 00:57:37,960
the Microsoft Bought framework. Like there
was lots of folks we talked about

764
00:57:37,079 --> 00:57:43,480
Lewis the PII detector, Like that's
all different software, that's all different teams

765
00:57:43,960 --> 00:57:46,800
they and the chances they're back end
pieces of the same No, Like that's

766
00:57:46,920 --> 00:57:50,840
not likely. Hey boss, I
have this idea chipping. You know,

767
00:57:52,079 --> 00:57:54,800
you touched them on a really interesting
point though, Like we you know,

768
00:57:55,320 --> 00:58:00,320
I think every product has the issue
where people don't know all the things the

769
00:58:00,320 --> 00:58:02,159
product could do and when it could
solve a problem that they currently have it

770
00:58:02,519 --> 00:58:06,199
that they may even be trying to
use the product to solve that, they

771
00:58:06,239 --> 00:58:09,000
just don't know that it can how
to do it. And that's one of

772
00:58:09,039 --> 00:58:12,920
those areas that we're finding, Uh, you know we want to bring into

773
00:58:13,280 --> 00:58:15,719
with chat chypt is that for example, I mentioned we have a set of

774
00:58:15,760 --> 00:58:19,960
skills you can call right, but
how do you know which skill is going

775
00:58:20,000 --> 00:58:22,400
to do the thing you want to
do? Well, that's that's where skill

776
00:58:22,440 --> 00:58:27,280
discovery comes into play. And so
if we describe the skills with natural language,

777
00:58:27,320 --> 00:58:30,159
then we can have a natural language. You know. Again, it's

778
00:58:30,199 --> 00:58:32,119
a selector problem, right, like
we can have it select the skill.

779
00:58:32,320 --> 00:58:36,320
Sure. Well, how many you
sat on the other side of this hit

780
00:58:36,360 --> 00:58:39,039
fill You've been the product maker when
the customers company said, boy, we

781
00:58:39,079 --> 00:58:43,280
should really wish we had this capability. It's like, you mean that capability

782
00:58:43,280 --> 00:58:49,159
to put in three or fo Yeah, exactly. And a lot of times

783
00:58:49,199 --> 00:58:52,559
people aren't asking, oh, like, you know, can you do this?

784
00:58:52,639 --> 00:58:54,360
They're just saying, hey, you
know, I ran into a problem

785
00:58:54,400 --> 00:58:58,800
with you know, X, Y
Z, and if you know you can

786
00:58:58,840 --> 00:59:01,239
take that and synthesize it and say, hey, oh, actually we have

787
00:59:01,320 --> 00:59:05,800
something that solves that problem. Right. That's where it gets really interesting,

788
00:59:05,920 --> 00:59:09,360
right, or you have that classic
Microsoft phrase, so why would you want

789
00:59:09,360 --> 00:59:15,400
to why would you ever want to
do that? Right? Right? Exactly?

790
00:59:15,440 --> 00:59:20,880
Because they're describing a problem they're having, not that gold are trying to

791
00:59:20,920 --> 00:59:23,599
reach, right exactly. Yeah,
And that's where a lot of that's where

792
00:59:23,679 --> 00:59:29,320
things are very limited in terms of
like feature searching within feature within products is

793
00:59:29,360 --> 00:59:31,360
like, oh, you're searching for
the feature, but you're not searching for

794
00:59:31,440 --> 00:59:36,400
the problem, right, You're trying
to draw a picture and word I like

795
00:59:36,480 --> 00:59:40,920
that I'm having trouble with my color
selection, and why would you do that?

796
00:59:42,159 --> 00:59:45,119
Right, It's exactly that problem.
So we're right on an hour here

797
00:59:45,360 --> 00:59:50,760
is uh I could go on talking
to you about this for another hour easily.

798
00:59:51,239 --> 00:59:52,880
Maybe you ought to come back and
we can pick up where we left

799
00:59:52,920 --> 00:59:58,840
off another time. Sure, sure, yeah, I think like the one

800
00:59:58,880 --> 01:00:02,639
thought I think I want to close
with is one of the challenges I see

801
01:00:02,679 --> 01:00:07,079
with these open AI and chat GPT
is the fact that, like when you're

802
01:00:07,079 --> 01:00:12,079
asking these questions, it's getting its
information from somewhere, and that somewhere is

803
01:00:12,400 --> 01:00:15,559
human produce. Right, So,
yes, a programming question, it probably

804
01:00:15,679 --> 01:00:19,320
you know, synthesize it from stack
overflow, right. But if but it's

805
01:00:19,320 --> 01:00:22,719
not giving credit to stack overflow,
it's not getting credit to the content creator.

806
01:00:22,840 --> 01:00:27,440
Right. What if nobody goes to
stack overflow because chat GPT does a

807
01:00:27,440 --> 01:00:30,440
better job of answering the question,
where's the incentive for the content creators to

808
01:00:30,519 --> 01:00:35,880
keep answering questions on stack overflow.
I could see it all degrading if they

809
01:00:35,880 --> 01:00:38,880
don't solve that problem. Being AI
does do a pretty good footnoting like it

810
01:00:39,239 --> 01:00:43,639
absolutely when you ask it for specific
things, it'll pull that up like this

811
01:00:43,719 --> 01:00:46,840
is possible. They should be.
They are indexing internally. What they need

812
01:00:46,880 --> 01:00:50,000
to do is show that they are. It also is a great piece for

813
01:00:50,039 --> 01:00:52,119
fact checking. Yeah, say,
hey where you know, where did you

814
01:00:52,159 --> 01:00:57,119
get this thing? You know,
I did the chat GPT right at Wikipedia

815
01:00:57,199 --> 01:01:00,559
entry for Richard Campbell and he picked
some other Richard Campbell for the birthday.

816
01:01:00,239 --> 01:01:04,400
There's a few pieces of my things
in there, Like, there's nice for

817
01:01:04,480 --> 01:01:07,079
it to have a chat GPT show
its sources always. Well, so I

818
01:01:07,480 --> 01:01:10,519
did that. I said, right
a Wikipedia article about Phil Hack, and

819
01:01:10,599 --> 01:01:15,000
cite your sources, and it did
cite sources, but it made up the

820
01:01:15,039 --> 01:01:21,960
sources. Yeah, I clicked the
u URLs that did not exist. And

821
01:01:22,039 --> 01:01:28,960
again you're personifying right. It sure
didn't make them up. The algorithm generated

822
01:01:29,000 --> 01:01:34,440
them, and they were inaccurate.
The algorithm stochastically generated citations that look like

823
01:01:34,480 --> 01:01:39,000
citations. Yeah, that were that
weren't actually to use it, Dunning Kruger,

824
01:01:39,199 --> 01:01:45,079
scales are falling from my eyes.
That's right. That that is the

825
01:01:45,320 --> 01:01:47,320
perfect way to describe it, Dunning
Kruger. We're going to have to leave

826
01:01:47,320 --> 01:01:50,599
it there. Phil, thank you
very much. It's been a pleasure as

827
01:01:50,639 --> 01:01:52,880
always, and it's been fun by
you again, and we'll see you next

828
01:01:52,920 --> 01:02:20,079
time on dot net rocks. Dot
net Rocks is brought to you by Franklin's

829
01:02:20,119 --> 01:02:24,079
Net and produced by Pop Studios,
a full service audio, video and post

830
01:02:24,079 --> 01:02:29,840
production facility located physically in New London, Connecticut, and of course in the

831
01:02:29,880 --> 01:02:36,400
cloud online at pwop dot com.
Visit our website at dt n et r

832
01:02:36,440 --> 01:02:42,280
ocks dot com for RSS feeds,
downloads, mobile apps, comments, and

833
01:02:42,400 --> 01:02:46,039
access to the full archives going back
to show number one, recorded in September

834
01:02:46,079 --> 01:02:50,599
two thousand and two. And make
sure you check out our sponsors. They

835
01:02:50,679 --> 01:03:00,639
keep us in business. Now,
go write some code seex time t Gord.

836
01:03:01,360 --> 01:03:05,159
That is hard than my text is