1 00:00:01,000 --> 00:00:04,759 How'd you like to listen to dot net Rocks with no ads? Easy? 2 00:00:05,320 --> 00:00:09,400 Become a patron for just five dollars a month. You get access to a 3 00:00:09,480 --> 00:00:14,240 private RSS feed where all the shows have no ads. Twenty dollars a month, 4 00:00:14,240 --> 00:00:18,399 we'll get you that and a special dot net Rocks patron mug. Sign 5 00:00:18,440 --> 00:00:23,440 up now at Patreon dot dot net rocks dot com. Hey Carlin Richard. 6 00:00:23,480 --> 00:00:28,000 Here. As you may have heard, NDC is back offering their incredible in 7 00:00:28,120 --> 00:00:33,960 person conferences around the world. DC Porto is happening October sixteenth through the twentieth. 8 00:00:34,240 --> 00:00:38,280 Go to Eddcporto dot com to register and check out the full lineup of 9 00:00:38,320 --> 00:00:43,600 conferences at NDC Conferences dot com. Hey there, this is Jeff Fritz, 10 00:00:43,840 --> 00:00:48,159 the Purple Blazer guy from Microsoft, letting you in on a little secret about 11 00:00:48,240 --> 00:00:52,920 my friend Carl Franklin. You know the guy who started dot net Rocks, 12 00:00:52,960 --> 00:00:57,719 the first podcast about dot net in two thousand and two, The guy who's 13 00:00:57,719 --> 00:01:03,040 been teaching Blazer on YouTube since twenty Yeah, that Carl Franklin. Well, 14 00:01:03,400 --> 00:01:07,079 Carl's joined up with the folks from Code in a Castle to teach a week 15 00:01:07,159 --> 00:01:12,159 long hands on Blazer class at Are you ready to get this at a castle 16 00:01:12,439 --> 00:01:19,040 slash villa in Tuscany. It's sort of a luxury vacation with Blazer learning built 17 00:01:19,040 --> 00:01:26,879 in. Carl's calling it the Blazer master Class. You'll learn Blazer from the 18 00:01:26,920 --> 00:01:30,920 ground up, finishing the week with the ability to build and deploy Blazer applications. 19 00:01:33,120 --> 00:01:36,920 Since the training happens for only four hours in the morning over six days, 20 00:01:37,280 --> 00:01:41,920 you can bring your significant other your partner with you and you should right 21 00:01:42,400 --> 00:01:46,959 that this part of Italy is absolutely beautiful. There's so much to see and 22 00:01:47,159 --> 00:01:52,560 do and and Larry and Marco from Code in a Castle are organizing daily activities 23 00:01:52,640 --> 00:01:56,840 both at the castle and in the area. The castle is in the Marema, 24 00:01:57,280 --> 00:02:01,760 a less touristed region of Tuscany, offering both classic Tuscan hill country as 25 00:02:01,760 --> 00:02:07,479 well as easy access to the Etruscan Riviera, with sublime local food, wine 26 00:02:07,479 --> 00:02:13,560 and olive oil around every corner. Breakfast is included every day. There will 27 00:02:13,560 --> 00:02:16,919 be two communal dinners at the castle book ending the experience, and most other 28 00:02:17,000 --> 00:02:23,360 meals and all activities are included. And did I mention you'll learn Blazer in 29 00:02:23,479 --> 00:02:28,599 person from Carl Franklin Listen. Space is limited and for very good reason. 30 00:02:28,879 --> 00:02:34,439 This is quality training in a beautiful setting. Go to code in Acastle dot 31 00:02:34,479 --> 00:02:40,680 com slash Blazer twenty twenty three. That's bla z o R two zero two 32 00:02:40,840 --> 00:02:46,199 three to take advantage of this amazing opportunity to join Carl in Tuscany for an 33 00:02:46,280 --> 00:02:53,520 unforgettable week of La dolce vita while advancing your programming skills in this important new 34 00:02:53,599 --> 00:03:09,800 technology. Hey guess what it's time for dot net rocks. I'm Carl Franklin 35 00:03:09,879 --> 00:03:13,840 and I'm Richard Campbell, Brian McKay, my friend, our friend is here 36 00:03:13,879 --> 00:03:17,400 with us today talking about some AI stuff. But first, how you doing? 37 00:03:17,479 --> 00:03:21,000 Man? I am not that well. I mean this stuff you should 38 00:03:21,000 --> 00:03:23,280 talk about on Donna Rocks and stuff you shouldn't. But let's face it, 39 00:03:23,319 --> 00:03:25,240 after twenty something years, you guys know my life. Yeah, my father 40 00:03:25,319 --> 00:03:30,879 passed away this week. Yeah it sucks and it sucks, and but he 41 00:03:30,039 --> 00:03:35,639 was his lungs had failed him. There wasn't anything to be done. He 42 00:03:36,280 --> 00:03:38,479 passed with some dignity. All of us were they able to be there at 43 00:03:38,560 --> 00:03:44,599 least via zoom, including family from New Zealand. So yeah, I'm not 44 00:03:44,599 --> 00:03:46,479 gonna say it was really great because it was really awful, but at least 45 00:03:46,479 --> 00:03:50,199 we could all be together for that moment. Yeah, and it sounds like 46 00:03:50,240 --> 00:03:53,080 he was suffering a bit, so a little bit of relief. He was, 47 00:03:53,240 --> 00:03:57,560 and it was it was well, he died with some dignity and we 48 00:03:57,599 --> 00:03:59,879 could all hope to be so lucky. Well, sorry to hear that. 49 00:04:00,080 --> 00:04:01,879 He was a good man. Sorry to hear that. Buddy, we sounded 50 00:04:01,919 --> 00:04:04,080 so much alike. Let me tell you how much I like we sounded. 51 00:04:04,479 --> 00:04:09,039 I answered his phone more or more than one occasion at his house, and 52 00:04:09,080 --> 00:04:12,039 they would just and whoever was called was just talking to me like I was 53 00:04:12,120 --> 00:04:14,840 him and say no, no, I'm the Sun, and they would literally 54 00:04:14,879 --> 00:04:17,199 not believe me. Wow, It's like yeah, sure, Doug, and 55 00:04:17,240 --> 00:04:25,759 they would continue. He taught you how to do electronics and stuff. Yeah, 56 00:04:25,800 --> 00:04:29,079 yeah, no, he was an electrical engineer. He built electronic cashrowdisterers 57 00:04:29,199 --> 00:04:31,720 and you I think it was one time you had that experience where I took 58 00:04:31,759 --> 00:04:34,199 your seat pat machine and Bart went, oh that's what's wrong with that? 59 00:04:34,240 --> 00:04:39,240 Resolders of parts and here you go. Yeah, well he was having you 60 00:04:39,360 --> 00:04:46,079 resolder solder, see what I'm yeah, yeah, man, resoldering and unsoldering 61 00:04:46,240 --> 00:04:48,800 chips from boards that he was fixing or something he had. You're doing that 62 00:04:48,800 --> 00:04:51,079 when you were like seven or something, right, Yeah, yeah, you're 63 00:04:51,079 --> 00:04:54,959 exactly right. Yeah, had the soldering airand in my hand, my whole 64 00:04:54,959 --> 00:04:58,720 life. Wow. Well, we'll raise a glass to him. Indeed, 65 00:04:58,879 --> 00:05:02,879 cheers, and let's move on now with something a little more cheerful, Better 66 00:05:02,920 --> 00:05:15,079 No Framework, all right, man? When he got well, our friend 67 00:05:15,120 --> 00:05:17,759 Brian McKay gave me this one. He's got so many links to so many 68 00:05:17,839 --> 00:05:23,639 cool projects. This would be not his first contribution to Better No Framework, 69 00:05:23,720 --> 00:05:27,879 I'm correct, No, no, no, he has. I would say 70 00:05:27,920 --> 00:05:32,519 probably seven or eight or nine, ten or eleven or twelve stories thirteen stories 71 00:05:33,879 --> 00:05:41,079 provided by provided by Brian. But anyway, this is Smallville and it's generative 72 00:05:41,240 --> 00:05:45,720 agents for video games, and we just did an AI bought show on agents. 73 00:05:46,000 --> 00:05:53,879 And agents are these things that use gpt APIs and things like that LM 74 00:05:53,959 --> 00:05:58,439 models to do things and then refine them so you can shame them together. 75 00:05:58,600 --> 00:06:00,959 So the first thing might do when you give it a problem is break it 76 00:06:00,959 --> 00:06:05,360 down into parts and then ask itself how to define those parts, and keep 77 00:06:05,360 --> 00:06:10,600 going down until it gets very detailed. For just for one example. But 78 00:06:10,680 --> 00:06:16,600 this is generative agents that are virtual characters that can store memories and dynamically react 79 00:06:16,680 --> 00:06:21,720 to their environment. And so they're able to observe their surrounding, store memories, 80 00:06:21,720 --> 00:06:25,839 and react to state changes in the world. So you basically give them 81 00:06:25,839 --> 00:06:29,399 a personality in your program and you let them free and there's a virtual world 82 00:06:30,000 --> 00:06:32,360 and they go around and they do life. Now I know you're not a 83 00:06:32,480 --> 00:06:36,839 video game guy. Yeah it used to be. Yeah, spend more time 84 00:06:36,920 --> 00:06:43,759 programming, I guess. I mean we talk about like really contemporary games and 85 00:06:44,279 --> 00:06:46,959 the and the animated characters and so forth. There's a game called Assassin's Creed 86 00:06:47,040 --> 00:06:50,279 and there was there was an ancient Greek version of the game, and it's 87 00:06:50,319 --> 00:06:54,879 an interesting there's a whole larger subtext to all of this and so forth. 88 00:06:55,319 --> 00:06:57,959 But I stopped playing the game. I just started hanging out in the world 89 00:06:58,040 --> 00:07:00,920 because the world was that cool, right, So there'd be things like I 90 00:07:00,920 --> 00:07:04,160 remember one time just following around an elderly woman in one of these villages and 91 00:07:04,199 --> 00:07:09,040 she didn't get up in the morning, go down to the market, buy 92 00:07:09,439 --> 00:07:14,079 flower, take it home, make it into bread. Wow, right, 93 00:07:14,120 --> 00:07:16,639 Like that's how the game had such a state when you think about a generative 94 00:07:16,639 --> 00:07:20,879 agent in this equation, the idea that she would remember interacting with me, 95 00:07:21,399 --> 00:07:25,360 that perhaps if I had been aggressive to her in anyway, she'd be afraid 96 00:07:25,360 --> 00:07:27,399 of me, like she'd see me and move away from me. Like that, 97 00:07:27,439 --> 00:07:32,199 You could permanently change these NPC characters, or affect these NBC characters without 98 00:07:32,199 --> 00:07:34,360 them having to read a law of software for it. That that's just be 99 00:07:34,439 --> 00:07:39,560 an intrinsic part of the game, right, that you just let them develop 100 00:07:39,600 --> 00:07:43,800 how they how they will normally exactly right. That that's just it's fascinating to 101 00:07:43,800 --> 00:07:47,480 me in the play experience that you would have an impact in the game. 102 00:07:47,560 --> 00:07:50,199 Like man, you know, often in those kinds of games, you have 103 00:07:50,240 --> 00:07:54,759 an event where like everybody's there, there's a big crowd, and you do 104 00:07:54,839 --> 00:07:58,439 something extraordinary, you know, you behead the king whatever. That may be 105 00:07:58,720 --> 00:08:01,600 the idea that you'd never directly interrupted with that character, but that character had 106 00:08:01,639 --> 00:08:07,800 been there and had seen that thing, and you had affected their behavior towards 107 00:08:07,800 --> 00:08:09,959 you. Yeah, Like, I don't know if games they can do that 108 00:08:11,120 --> 00:08:16,079 right now, Well, Smallville's looks like it's going to be in that camp. 109 00:08:16,439 --> 00:08:18,879 We'll let Brian talk a little bit more about that. Well, I'm 110 00:08:18,879 --> 00:08:22,480 sure, yes, yeah, But first I guess people are talking to us 111 00:08:22,480 --> 00:08:26,279 today, Richard. Sorry, people talk to us most days. Friend. 112 00:08:26,439 --> 00:08:30,519 And this is from show eighteen forty eight, fairly recent. That's the one 113 00:08:30,560 --> 00:08:33,480 we did at Techaram and Antwerp with Jody Birchill and we talked about the no 114 00:08:33,600 --> 00:08:39,000 free lunch and machine waitning and I was really great to talk to her because 115 00:08:39,000 --> 00:08:43,200 she literally is a professional in this space and I think helped us ground a 116 00:08:43,240 --> 00:08:48,759 bit more what's happening with the generative machine learning models in This comment comes from 117 00:08:48,879 --> 00:08:52,080 Lucas, who says, very interesting episode. Maybe you missed an opportunity to 118 00:08:52,120 --> 00:08:58,559 talk about what probably most interesting listeners, which is CHATTYBT and code. Based 119 00:08:58,600 --> 00:09:01,639 on how you described it it works, I still don't get how it's able 120 00:09:01,679 --> 00:09:05,279 to produce reasonable unit tests from a random block and code I've paste into it, 121 00:09:05,639 --> 00:09:09,000 Explain how a chunk of code works, how it could translate code from 122 00:09:09,000 --> 00:09:11,679 one language to another. It seems to be much more than just glowing together 123 00:09:13,159 --> 00:09:16,679 related sentences words of gods from the Internet. And this is my favorite sentence 124 00:09:16,720 --> 00:09:20,679 of the whole thing. The Lucas said, the write a review of prompts 125 00:09:20,720 --> 00:09:24,759 and pseudos philosophical conversation you could have with it, make fun anecdotes, but 126 00:09:24,879 --> 00:09:28,679 I don't think it's the most interesting part for developers. Yeah, and I 127 00:09:28,720 --> 00:09:31,440 really appreciate that, Lucas. I mean, one of the reasons we didn't 128 00:09:31,440 --> 00:09:35,279 focus on code with Jody is that she was a machine learning professional, and 129 00:09:35,279 --> 00:09:39,480 I really wanted to talk more broadly about what was going on with these technologies 130 00:09:39,759 --> 00:09:45,440 and her concerns around it because she was a professional. It is interesting obviously 131 00:09:45,519 --> 00:09:48,679 you're referencing get a co Pilot more than anything, but chat GYBT it applies 132 00:09:48,679 --> 00:09:52,600 with us as well, which is that And I think Jody talked a bit 133 00:09:52,639 --> 00:09:56,799 about this. The tokenization of language is an important part of the power of 134 00:09:56,919 --> 00:10:00,960 large language models because it also creates a sense of bide to actionality that it's 135 00:10:01,720 --> 00:10:05,799 not only does it know from words what code you might want because of those 136 00:10:05,840 --> 00:10:09,279 sentence relationships, but it can work the other way that when presented code, 137 00:10:09,360 --> 00:10:13,279 it knows what language to produce for you to describe that code. It's not 138 00:10:13,399 --> 00:10:16,519 always right, according to you know, get hub compile itself. They're still 139 00:10:16,559 --> 00:10:22,720 batting less than fifty percent of compilable code on the initial prompt. But it 140 00:10:22,799 --> 00:10:26,360 is interesting to see that consistently that number goes up, although not by very 141 00:10:26,480 --> 00:10:33,759 much with multiple refined prompts, so it still has a way to go. 142 00:10:33,879 --> 00:10:35,919 But there are plenty of shows, believe me, where we're going to talk 143 00:10:35,960 --> 00:10:39,919 about large language models and code. So yeah, this is only the beginning 144 00:10:41,600 --> 00:10:46,080 and wherever it goes from there, I mean large language plus plus I imagine. 145 00:10:46,159 --> 00:10:50,679 And they now have plugins for accessing the Internet, as I learned from 146 00:10:50,679 --> 00:10:52,159 Brian. Well you know what, I'm just gonna we'll get there, introduced 147 00:10:52,159 --> 00:10:54,919 Brian and let him talk to about all that stuff. Okay, yeah, 148 00:10:56,000 --> 00:10:58,120 hey, Lucas, thank you so much for your comment. And a copy 149 00:10:58,159 --> 00:11:00,039 of us to cod Buy is on its way to un If you'd like a 150 00:11:00,120 --> 00:11:01,799 copy of Music Code By, I write a comment on the website at Dona 151 00:11:01,960 --> 00:11:05,320 Rocks dot com or on the facebooks. We publish every show there, and 152 00:11:05,360 --> 00:11:07,000 if you comment there when you're reading the show, we'll send your copy mused 153 00:11:07,000 --> 00:11:09,720 to go by and you can follow us on Twitter or x or whatever the 154 00:11:09,720 --> 00:11:13,960 hell they're calling it these days. But the real fun is over on Mastodon. 155 00:11:13,480 --> 00:11:18,720 I'm at Carl Franklin at tech Hub dot Social, and I'm Rich Campbell 156 00:11:18,799 --> 00:11:22,840 at Masadondo Social Sensitude. We'd like to hear from you, of course over 157 00:11:22,879 --> 00:11:26,000 there and share our stories and all that stuff. Awesome. So let's introduce 158 00:11:26,039 --> 00:11:31,480 Brian. Brian McKay is the co host of The AI Bought Show alongside myself, 159 00:11:31,879 --> 00:11:35,639 and also serves as the CTO of Roster, a company dedicated to transforming 160 00:11:35,759 --> 00:11:43,600 leaders using an innovative three sixty feedback process which sounds perfectly obfuscated to me. 161 00:11:43,000 --> 00:11:48,559 A season software engineer, entrepreneur, and open source contributor, Brian has been 162 00:11:48,600 --> 00:11:52,360 at the helm of product development and startups for over twenty years. He's a 163 00:11:52,399 --> 00:11:56,159 father, husband, musician, writer, chest NERD and a decent kickboxer, 164 00:11:56,559 --> 00:12:00,440 so don't mess with him. Welcome Brian. That's pretty to be here. 165 00:12:01,039 --> 00:12:03,399 Yeah, man, it was so hard enough to cut in during that intro. 166 00:12:03,440 --> 00:12:07,679 There's so much I want to say about Smallville and co Pilot and yeah, 167 00:12:07,679 --> 00:12:09,039 well where do you want to start? Well, you go, man, 168 00:12:09,120 --> 00:12:13,240 it's your show. Let's let's do it. Oh well, yeah, 169 00:12:13,399 --> 00:12:16,399 let's start with the details of Smallville. Okay, well, yeah, so 170 00:12:16,480 --> 00:12:20,960 Smallville came out and I want to say, was that August alongside all the 171 00:12:22,000 --> 00:12:28,960 other generative agents like baby Agi and autogpt. And the interesting thing is that 172 00:12:28,000 --> 00:12:31,840 they just open sourced it in the last couple of weeks, so now you 173 00:12:31,840 --> 00:12:35,480 can go into there and and change it, so you know, Smallville. 174 00:12:35,679 --> 00:12:37,600 The most interesting thing with these little bots, I think it's a community of 175 00:12:37,600 --> 00:12:41,480 like twenty five bots. Each one is a little prompt that defines its personality, 176 00:12:41,720 --> 00:12:45,919 and they've got a little algorithm that kind of lets them learn from the 177 00:12:45,919 --> 00:12:52,919 conversations they have together these bots, watching their behavior is really interesting. And 178 00:12:52,039 --> 00:12:56,000 one of the most interesting things that happened is one of them decided to plan 179 00:12:56,039 --> 00:13:01,720 a Valentine's party, and it propagated the information about this party to like eighteen 180 00:13:01,799 --> 00:13:05,759 of the twenty five bots, and they all made decisions about how to handle 181 00:13:05,799 --> 00:13:09,919 it. Some decided not to go somewhere, maybe snubbed a little bit. 182 00:13:09,080 --> 00:13:13,720 A bunch decided to go. There was an actual party, and the emergence 183 00:13:13,840 --> 00:13:18,879 of that type of behavior is fascinating and there's so much more that can happen 184 00:13:18,000 --> 00:13:20,320 in that space. As this gets murder, I mean, I just have 185 00:13:20,360 --> 00:13:24,639 a tough time with the whole agency decided to throw a Valentine's party. Yeah, 186 00:13:24,720 --> 00:13:30,840 Richard is hung up on the anthropomorphization of Ai Boughts, Oh, without 187 00:13:30,840 --> 00:13:37,039 a doubt. But the question is what was the software stimulus that propagated that 188 00:13:37,120 --> 00:13:39,879 process? Right? Right, token prediction just like everything else, you know, 189 00:13:39,960 --> 00:13:43,240 like and uh, you know primed by prompts that there might have been 190 00:13:43,240 --> 00:13:46,639 a prompt that's said that you want to plan a Valentine's party. Well, 191 00:13:46,720 --> 00:13:50,360 might not have emergent would hope they'd be more macro than that and say there 192 00:13:50,399 --> 00:13:54,240 are major events on the calendar and occasionally you should have a party for them. 193 00:13:54,240 --> 00:13:56,320 They might have and I mean even go to that weight of I mean, 194 00:13:56,519 --> 00:14:01,799 I would think that if you're mulating human behavior, you only want to 195 00:14:01,799 --> 00:14:05,879 throw a Valentine's party because you're in a relationship where you want to have other 196 00:14:05,879 --> 00:14:07,360 folks around, or you're not in a relationship you want to make it a 197 00:14:07,440 --> 00:14:11,039 singles thing like, well, you can go deep. This question is how 198 00:14:11,120 --> 00:14:16,639 much of that has to be crafted rights, Well, my experience with this 199 00:14:16,679 --> 00:14:20,720 tech is that one marea where it's very strong, is just brainstorming things like 200 00:14:20,879 --> 00:14:24,000 a Valentine's party. I mean, that does seem like something that could emerge 201 00:14:24,039 --> 00:14:28,519 you quorkanically quite easily. Well, and I like the brainstorming angle of it 202 00:14:28,559 --> 00:14:33,919 because it is just sort of a word salad of ideas that we then can 203 00:14:33,000 --> 00:14:39,000 sift through as humans with our somewhat more sophisticated minds and take value from. 204 00:14:39,279 --> 00:14:43,600 Yeah, right, anything that makes me happy on large language models, it's 205 00:14:43,600 --> 00:14:46,440 fill my blank screen with stuff that might be useful, because I'd rather criticize 206 00:14:46,480 --> 00:14:50,000 than create, right. Yeah. One of the things that Brian does really 207 00:14:50,000 --> 00:14:54,600 well in the AI Bot show is tell it. You know, give me 208 00:14:54,720 --> 00:15:01,120 we're building a board game, right it, Give me ten ideas for cards 209 00:15:01,200 --> 00:15:03,039 that we can play on this game, you know, after we've got the 210 00:15:03,120 --> 00:15:07,759 gold and all that stuff. But you don't say just create something. You 211 00:15:07,799 --> 00:15:11,879 say, give me ten twenty ideas, and you pick the one that you 212 00:15:11,919 --> 00:15:13,840 like, and you go with that and you narrow it down. But you're 213 00:15:13,919 --> 00:15:18,519 kind of being like an agent in that sense, aren't you, because you're 214 00:15:18,559 --> 00:15:24,879 you're basically starting with a question, taking the results, picking one, and 215 00:15:24,919 --> 00:15:28,879 then diving deeper into it. Right, Well, we'll tell you why this. 216 00:15:28,559 --> 00:15:31,200 Uh, this leads right into what I wanted to talk about. I 217 00:15:31,279 --> 00:15:35,720 kind of want this conversation to be about maybe what this is like as a 218 00:15:35,720 --> 00:15:39,320 developer to use and what the strengths and weaknesses are, and maybe the place 219 00:15:39,399 --> 00:15:45,480 to start that will explain the technology is just talking about the weaknesses of it, 220 00:15:45,519 --> 00:15:48,559 Like where this is going is just a dead end? What are the 221 00:15:48,600 --> 00:15:52,240 bad parts of this tech? And I think that will cover everything that we're 222 00:15:52,279 --> 00:15:56,919 talking about right now. Cool. Oh, I mean right off the bat, 223 00:15:56,919 --> 00:16:00,960 it's like, listen the bubble forming in the VC community around this lay 224 00:16:02,000 --> 00:16:08,600 that clearly in the bad part of this uses talk about incentivizing grift incentivizing fictions. 225 00:16:10,039 --> 00:16:12,279 Yeah, it's not good. Yeah, I'm trying to I'm trying to 226 00:16:12,320 --> 00:16:18,279 coin the phrase griftware for that that thing that that thing that emerged during I 227 00:16:18,320 --> 00:16:22,080 think it started maybe during crypto, maybe before. But these people learned how 228 00:16:22,120 --> 00:16:29,159 to these people learned how to descend on hype and and just con people out 229 00:16:29,159 --> 00:16:32,080 of their money over and over and over again. And some of that's present 230 00:16:32,159 --> 00:16:34,440 in this world too. It's not as bad because it's not as easy to 231 00:16:34,559 --> 00:16:37,159 just you know, it's not like just getting someone to buy some crypto. 232 00:16:37,679 --> 00:16:41,279 But there are people, there are folks trying to get you to buy products 233 00:16:41,279 --> 00:16:44,799 that are just very flimsy rappers on top of API calls. Yeah, and 234 00:16:44,879 --> 00:16:49,279 the word AI itself or the term being misrepresented as you know, what's the 235 00:16:49,279 --> 00:16:53,240 difference between a well crafted algorithm and AI? Right? I mean it's that's 236 00:16:53,320 --> 00:16:56,799 right. Yep. People, I've said, I don't know how many times 237 00:16:56,799 --> 00:17:00,399 I've said this like AI to me, it's just that true, which it 238 00:17:00,440 --> 00:17:03,320 says, Okay, you're making stuff up. Yeah, as near as I 239 00:17:03,320 --> 00:17:07,640 can tell, AI is the term you use when stuff doesn't work. As 240 00:17:07,680 --> 00:17:11,880 soon as it does work, it hasn't. It's no longer as exactly it's 241 00:17:11,000 --> 00:17:15,240 large language modes or anything like that. So it's like it's just automatic red 242 00:17:15,240 --> 00:17:18,799 flag scrutinized clothes. You know, you're a problem. Yeah. Yeah, 243 00:17:18,880 --> 00:17:22,680 we will keep raising the bar until one day we build something that says, 244 00:17:22,720 --> 00:17:26,960 wait a minute, I'm alive and tries to and actually convinces us and makes 245 00:17:27,000 --> 00:17:30,599 us believe. I don't think we will accept Well, I don't think we'll 246 00:17:30,640 --> 00:17:33,480 have any problem having it convince us it's alive, because clearly there are people 247 00:17:33,559 --> 00:17:37,799 that think that already there are well, you know, we want to anthropomorphize 248 00:17:37,839 --> 00:17:41,000 things so much. I was watching the pilot of Community the other day. 249 00:17:41,200 --> 00:17:45,599 And there's the scene where Joel McHale holds up a pencil and says something, 250 00:17:45,599 --> 00:17:48,519 I'm going to butcher this, but he says something like, this pencil is 251 00:17:48,559 --> 00:17:52,319 Fred. Fred's got a wife and two kids. Snap, and everyone in 252 00:17:52,319 --> 00:17:56,680 the room goes, oh, you know, that's all it takes, you 253 00:17:56,680 --> 00:18:00,400 know, like we want to see humanity and pencils with just a little story. 254 00:18:00,440 --> 00:18:03,359 We're wired that way to synchronize. And yeah, and also it's easy 255 00:18:03,440 --> 00:18:07,119 for us to describe things in terms of anthropomorphizing. We've been doing it for 256 00:18:07,200 --> 00:18:11,839 code. Oh well, my guy over here says, hey, let me 257 00:18:11,920 --> 00:18:15,559 know whenever this happens. And then this guy says, okay, here you 258 00:18:15,640 --> 00:18:18,960 go. Right, when we're describing code to each other, we kind of 259 00:18:19,000 --> 00:18:23,279 talk like that. And you naturally did that when you're talking about these these 260 00:18:23,319 --> 00:18:27,039 agents too, because it's just such a every it's a framework for understanding that 261 00:18:27,119 --> 00:18:32,160 everybody gets. We just have to remember that it's not for you. Well, 262 00:18:32,200 --> 00:18:36,400 and part I mean that's the problem with people make assumptions around it and 263 00:18:36,440 --> 00:18:38,839 they project a lot more capability on it than it actually has. Right, 264 00:18:40,160 --> 00:18:44,759 So, back in like twenty so, I've been following this pretty closely since 265 00:18:45,160 --> 00:18:48,359 GPT two and like twenty nineteen, started using a little more seriously in twenty 266 00:18:48,359 --> 00:18:52,400 twenty when GPT three came out and I started running into researchers like AI, 267 00:18:52,480 --> 00:18:56,839 researchers who are smarter than me. And one thing I noticed is that they 268 00:18:56,880 --> 00:19:00,200 were really dismissive of this tech and I saw a lot of promise it. 269 00:19:00,839 --> 00:19:07,319 But the reasons why they're dismissive still have some relevance. And you know, 270 00:19:07,480 --> 00:19:11,440 I think the thing with so they're concerned about AGI is, first of all, 271 00:19:11,559 --> 00:19:15,279 is the thing they want a path where technology can be sentient. Define 272 00:19:15,359 --> 00:19:22,480 that acronym general intelligence artificial general intelligence. Yeah yeah, thank you, specialized 273 00:19:22,519 --> 00:19:27,920 intelligence. Right, so it just means natural stupidity and as got it. 274 00:19:29,480 --> 00:19:32,720 We just we just solve that one right there. So so so you know, 275 00:19:32,880 --> 00:19:36,759 these a lot of these folks are less concerned with just making something that 276 00:19:36,799 --> 00:19:41,440 has some utility and more interested in making something that is alive. Like that's 277 00:19:41,480 --> 00:19:44,440 the dream to make something that's kind of human level. And you hit a 278 00:19:44,440 --> 00:19:48,079 great line here, Brian. So that's a different science and engineering. All 279 00:19:48,119 --> 00:19:51,720 of us at our roots are really engineers, and so we're looking at tools 280 00:19:51,720 --> 00:19:56,240 and say what can I do with these tools? Where the scientists are farm 281 00:19:56,279 --> 00:20:00,160 you know, implementation is a detail they're farm ore iNeST in the broader science. 282 00:20:00,720 --> 00:20:06,079 It's of you know, recognizing the limitations of LMS and sertic. Okay, 283 00:20:06,079 --> 00:20:08,160 well that's not this path of this dream I have. So next, 284 00:20:08,880 --> 00:20:12,599 Yeah, and I and I suppose that I am actually like much more of 285 00:20:12,599 --> 00:20:15,799 a language guy, you know, like I went to school a little bit 286 00:20:15,799 --> 00:20:18,279 for English, and I like to write. So it's a different people connect 287 00:20:18,279 --> 00:20:23,319 to it differently. So what I I think the thing is that we imagine 288 00:20:23,400 --> 00:20:29,279 intelligence should feel in some way organic. We want to nurture a spark and 289 00:20:29,319 --> 00:20:33,319 watch it internalize moral lessons and reason with agency and grow and wisdom or sapiens. 290 00:20:33,839 --> 00:20:37,279 And this is autocomplete, you know, like when we've done here, 291 00:20:37,519 --> 00:20:40,759 the only game in town, the only game in town is we've trained a 292 00:20:40,839 --> 00:20:45,160 really sophisticated neural network on everything that we could, you know, get into 293 00:20:45,200 --> 00:20:48,759 it. And now it completes. It chooses the next token, the next 294 00:20:48,799 --> 00:20:52,200 most probable token that should appear. I give you an example when autocomplete is 295 00:20:52,240 --> 00:20:59,319 too slow. Hey, honey, have you seen the the thing, the 296 00:20:59,400 --> 00:21:03,839 red thing, the red scrapy thing. You're not. Come on, you 297 00:21:03,880 --> 00:21:10,640 know what I'm talking about. So so it's not a beautiful model of intelligence. 298 00:21:10,839 --> 00:21:15,119 It's you know, I don't think intuitively a human wants autocomplete to be, 299 00:21:15,720 --> 00:21:18,440 to be the AGI that we come up with. There's just better ideas, 300 00:21:18,720 --> 00:21:22,759 and there still are. They still tell me there are better ideas and 301 00:21:22,920 --> 00:21:27,599 out there that will supplant this so inevitably. But that's kind of normal. 302 00:21:27,720 --> 00:21:32,400 The real thing here is, I don't think anybody, I think only the 303 00:21:32,440 --> 00:21:37,000 scientists really want an AGI in the first place. Like that's it's science fiction 304 00:21:37,039 --> 00:21:40,799 for crying out loud, right, Yeah, there are so many more interesting 305 00:21:40,799 --> 00:21:45,119 things you just go work on. Well, there's a fascination with the idea 306 00:21:45,160 --> 00:21:49,400 of AGI. There's a it's weird because AGI is maybe not great for us, 307 00:21:51,160 --> 00:21:55,079 but as of species, we seem to be inexorably drawn to it, 308 00:21:55,160 --> 00:21:56,480 like a moth to the flame. We can't stop. We're going to do 309 00:21:56,519 --> 00:22:02,079 it. I find that interesting, Like it's fascinating up and at the same 310 00:22:02,119 --> 00:22:06,240 time, humans are remarkably resistant to calling anything else on this planet sentient, 311 00:22:06,319 --> 00:22:10,079 even though this plant there's significant evidence to show there is you know, if 312 00:22:10,079 --> 00:22:14,359 we really cared about intelligent life. Why do we treat citaceans the way we 313 00:22:14,440 --> 00:22:18,599 do? And you know and so on. Yeah, dolphins are supposedly really 314 00:22:18,599 --> 00:22:22,559 smart. Pigs can recognize themselves and mirrors and we eat them. Yeah, 315 00:22:22,640 --> 00:22:25,920 yeah, I don't know that dolphin. That's been gross, But but I 316 00:22:25,920 --> 00:22:30,039 meant a point, or even how we've treated great the great apes too, 317 00:22:30,279 --> 00:22:33,240 right, Like, that's true. And the problem is that as soon as 318 00:22:33,240 --> 00:22:36,720 you start getting serious about defining sentiency in any way, a whole bunch of 319 00:22:36,799 --> 00:22:41,400 other creatures we've abused on this planet qualify. That's now you've got a problem. 320 00:22:41,319 --> 00:22:45,480 Yeah, that's true. And future generations will probably judge us for these 321 00:22:45,480 --> 00:22:51,599 things, just like we judge past generations for their the institutions that they lived 322 00:22:51,599 --> 00:22:59,480 in. So the problems are their static, meaning, once they're trained, 323 00:22:59,480 --> 00:23:03,519 they don't really learn. They have a token, they have a context of 324 00:23:03,680 --> 00:23:07,480 you know, eight thousand, eight thousand tokens that you can play with, 325 00:23:07,799 --> 00:23:11,559 but they're not really learning as you go. That's very limited space. You 326 00:23:11,559 --> 00:23:14,279 can do some tricks with it. Should we define what a token is in 327 00:23:14,279 --> 00:23:18,480 this context? Yeah? The easiest way, it's it's easiest to think of 328 00:23:18,480 --> 00:23:22,200 it as like a few words. You know, you have this window of 329 00:23:23,079 --> 00:23:30,359 with GPT four like twenty thousand or so words that you can feed into it 330 00:23:30,480 --> 00:23:34,359 as as your conversation, you know, like that's why chat GPT can can 331 00:23:34,599 --> 00:23:38,480 understand what you're talking about and remember what you just said. But as time 332 00:23:38,519 --> 00:23:42,599 goes by, things will fall off the end and it will forget the things 333 00:23:42,599 --> 00:23:45,720 that happen at the start of the conversation. You're just pushing things through that 334 00:23:47,319 --> 00:23:49,200 eight thousand token limit. I haven't done this in a while, but one 335 00:23:49,200 --> 00:23:53,119 of the in the earlier versions of this, I use the iamic pentameter trick 336 00:23:53,200 --> 00:23:56,799 where I told I set up front, I need you to only respond to 337 00:23:56,799 --> 00:23:59,799 me an iamic pendameter, okay, and then we'd keep going back and forth 338 00:24:00,079 --> 00:24:03,960 till the cash overflowed and suddenly would stop. Like it was the easiest way 339 00:24:03,000 --> 00:24:06,359 to say, hair, you just hit the cash limit. And for those 340 00:24:06,400 --> 00:24:12,759 who don't know, I amke pantameter. Sounds like this. It's almost like 341 00:24:12,839 --> 00:24:18,319 the two lines of a limerick. Yeah. So practically, one place where 342 00:24:18,319 --> 00:24:22,480 this comes up is it's really easy to make a bot that generates sequel statements. 343 00:24:23,200 --> 00:24:26,920 So I'm working on a new project that I made a bot purpose built 344 00:24:27,039 --> 00:24:30,640 for It understands what the project is and you can tell it to make tables. 345 00:24:30,640 --> 00:24:33,640 It knows how I like my things, capitalize the naming conventions, everything 346 00:24:33,640 --> 00:24:40,519 about it. But create statements do take up space, and you know, 347 00:24:40,599 --> 00:24:45,799 a database of some significant size in terms of tables will just push the context 348 00:24:45,920 --> 00:24:48,680 off the limit. It will forget where your user table was because it's not 349 00:24:48,720 --> 00:24:52,400 in there anymore. So we're not we're not. You know, the context 350 00:24:52,480 --> 00:24:55,839 size will improve over time, and it has improved. They're working on a 351 00:24:55,880 --> 00:25:02,279 thirty two K token model, but my understanding is that tokens are quadratic, 352 00:25:03,039 --> 00:25:07,000 so going from eight K to thirty two K is really computationally expensive, which 353 00:25:07,039 --> 00:25:11,759 actually brings us to the second problem. This technology is very expensive. It's 354 00:25:11,839 --> 00:25:19,359 computationally There was some leaked some leaked documents months ago, I think in February 355 00:25:19,799 --> 00:25:26,400 that showed this product that Microsoft is planning on launching called I think it's called 356 00:25:26,400 --> 00:25:32,559 Foundry, and basically it's you can host your own model of GPT four in 357 00:25:33,480 --> 00:25:37,119 Azure. And there's a cheap version that costs a quarter of a million dollars 358 00:25:37,160 --> 00:25:41,880 a year for like three chat like a three point five turbo model that just 359 00:25:41,920 --> 00:25:45,880 gives you a glimpse at how expensive it is. In fact, there's a 360 00:25:47,000 --> 00:25:49,160 rumor this week. I don't know if the sources really check out, but 361 00:25:49,960 --> 00:25:56,000 they say that chat GPT by itself is burning like seven hundred thousand dollars a 362 00:25:56,079 --> 00:25:59,920 day. I believe it, which I don't think is necessarily a huge problem 363 00:26:00,039 --> 00:26:02,640 because they have over a billion users, you know, like if you just 364 00:26:02,680 --> 00:26:04,359 get a nickel from Yeah, as long as those billion users are paying, 365 00:26:04,599 --> 00:26:07,640 yeah, I would love to know the percentage that are actually paying twenty bucks 366 00:26:07,680 --> 00:26:11,920 a month for Chat gypt pro. I'm one of them. Yeah, I'm 367 00:26:11,920 --> 00:26:15,039 one two. It might just be us though, But even if it was 368 00:26:15,119 --> 00:26:18,400 one percent, that's ten million users a twenty dollars. It's two hundred million 369 00:26:18,920 --> 00:26:23,240 a month. Yeah, that's close. You know, seven hundred thousand a 370 00:26:23,319 --> 00:26:26,920 day is like twenty one million dollars, So you're getting there. I've got 371 00:26:26,960 --> 00:26:30,559 a feeling it's more than one percent. I would argue it's less than one 372 00:26:30,599 --> 00:26:33,480 percent. Actually you really think so, absolutely, But we're just guessing. 373 00:26:34,039 --> 00:26:37,799 Yeah, yeah, well we don't know. But I feel like if you 374 00:26:37,880 --> 00:26:40,640 had access to a you know, a couple of billion eyeballs. I can 375 00:26:40,680 --> 00:26:44,000 find a way to make it, make it work, but in Microsoft will 376 00:26:44,000 --> 00:26:45,680 too, and also they'll find ways to make it cheaper. I think you've 377 00:26:45,680 --> 00:26:51,640 found this drag race. Now that that you know we have the specs for 378 00:26:51,759 --> 00:26:55,519 how Microsoft hosted GPT three, the two hundred and eighty five thousand processors. 379 00:26:56,960 --> 00:27:00,599 We know that the models roughly seven times larger, so you can kind of 380 00:27:00,599 --> 00:27:04,240 project and that makes it one of the largest shipper computers in the world. 381 00:27:04,960 --> 00:27:08,440 And that that I mean, irrespective of what it actually costs to build that 382 00:27:08,480 --> 00:27:14,119 out gredit they already owned it. That is a bunch of Azure resources could 383 00:27:14,119 --> 00:27:18,359 be making money on something else, right, and yet is a sign now 384 00:27:18,519 --> 00:27:22,000 I mean open as paying for those, but the paying for those with funny 385 00:27:22,000 --> 00:27:26,920 money. Right. Microsoft gave them ten billion dollars in Azure credits to officially 386 00:27:26,960 --> 00:27:32,039 give it back to them, right, right, And it gives us a 387 00:27:32,039 --> 00:27:36,119 time limit, you know, and start start up parlance. This is your 388 00:27:36,319 --> 00:27:41,279 ramp to run with as much time to get enough revenue to extend your ramp 389 00:27:41,319 --> 00:27:45,960 to keep going, right. Yeah, that's right. And they can always 390 00:27:45,279 --> 00:27:48,480 you know, you you can always turn off half of those half of those 391 00:27:48,559 --> 00:27:52,359 unpaid users and probably catch your costs quite a bit. So I think there 392 00:27:52,400 --> 00:27:56,599 are strategies. Yeah, well it that's the question that if you're an API 393 00:27:56,720 --> 00:27:59,519 user and we haven't even talked about the API yet, but if you're a 394 00:27:59,599 --> 00:28:04,160 gpt API user and you're making calls and you're actually selling a product that uses 395 00:28:04,240 --> 00:28:11,759 it, you know you took a dependency here that may or may not change 396 00:28:11,000 --> 00:28:14,920 or completely go away in the future. That's true, but they are not 397 00:28:14,960 --> 00:28:18,680 the only game in town either. With the rise of a couple of open 398 00:28:18,680 --> 00:28:22,599 source models in the last month or so, you know, Lama two is 399 00:28:22,599 --> 00:28:26,119 out stable. The Fusions also got a couple of open source models. You 400 00:28:26,119 --> 00:28:30,160 can actually just host that yourself if you have the hardware, if you have 401 00:28:30,640 --> 00:28:33,920 two and eighty thousand processors or whatever. Well, the crazy thing is I 402 00:28:34,000 --> 00:28:37,519 saw a tweet a tweet. Is it still called the tweet? I don't 403 00:28:37,559 --> 00:28:38,480 know what you call them now? On X I don't know what you're calling. 404 00:28:41,119 --> 00:28:44,519 Well, whatever it is, I saw people are talking about running the 405 00:28:44,559 --> 00:28:52,200 small version of it, like the seven billion parameter version on laptops because I 406 00:28:52,240 --> 00:28:56,200 guess the thing is that it's memory bound or something like that. I need 407 00:28:56,200 --> 00:28:59,319 to read the quote a tweet more closely. But when you're doing it, 408 00:28:59,319 --> 00:29:03,039 when you only care about a single request, you can you can do a 409 00:29:03,079 --> 00:29:06,680 lot more with a with less. But when you care about running at scale, 410 00:29:06,720 --> 00:29:10,160 you really do need serious technology. And like you know, and video 411 00:29:10,240 --> 00:29:14,000 ships that cards the cost fifteen thousand dollars each. Interesting. And I'm talking 412 00:29:14,039 --> 00:29:17,119 to folks who are who are trying to build software around this, and they're 413 00:29:17,119 --> 00:29:19,799 all about GPT four and I'm like, why GPT four, why not GPD 414 00:29:19,920 --> 00:29:25,279 three? Like and and really it's like because four is larger than three more 415 00:29:25,680 --> 00:29:30,319 Like they don't really know, they haven't actually tested the software with the smaller 416 00:29:30,319 --> 00:29:33,640 model, is that this is sufficient? And when I talk to Microsoft engineers, 417 00:29:33,640 --> 00:29:37,400 like, they're pitching three and three five pretty hard these days. And 418 00:29:37,400 --> 00:29:41,759 I think one of the issues is that the four is so large that it's 419 00:29:41,839 --> 00:29:45,119 going to be hard for it to make it profitable and maybe you don't need 420 00:29:45,160 --> 00:29:49,400 it. Didn't we learn Brian that three five has a model a mode where 421 00:29:49,440 --> 00:29:55,119 there's more tokens available to it than what's currently available for Yeah, they're working 422 00:29:55,160 --> 00:29:59,160 on on greatly increasing the context. It is much more expensive. So when 423 00:29:59,519 --> 00:30:02,799 so one thing we've seen is the price reduced over time, over and over 424 00:30:02,839 --> 00:30:07,160 again with these models. So when when around the time that GPT four came 425 00:30:07,160 --> 00:30:11,200 out, they launched GPT three point five Turbo and they cut the price by 426 00:30:11,279 --> 00:30:12,599 like I want to say, it was like ninety percent. I mean, 427 00:30:12,599 --> 00:30:15,839 it's so cheap compared to what it was. So now the question is is 428 00:30:15,839 --> 00:30:19,240 that based on cost or is that based on a marketing effort to move customers? 429 00:30:21,440 --> 00:30:23,480 That's a who yeah, like a lost leader type of thing. I 430 00:30:23,599 --> 00:30:27,880 don't know. I assume it must be cost. Well, hey, if 431 00:30:27,960 --> 00:30:32,119 I know I can't get to the price that I'm going to need you for 432 00:30:32,160 --> 00:30:36,119 GPT four, but I don't want to have my customers abandoned me, I 433 00:30:36,240 --> 00:30:38,079 give you a discount on the product I think I can make a profit on 434 00:30:38,119 --> 00:30:41,759 because once it works over there, then I can take the price back up. 435 00:30:41,799 --> 00:30:45,839 That may be they have you know, access to cheap money, or 436 00:30:45,880 --> 00:30:48,359 they have had it at least so yeah, but that money eventually runs out, 437 00:30:48,680 --> 00:30:52,759 so don't usually run out. Yes, I just get a real sense 438 00:30:52,079 --> 00:30:56,440 as we come off the top of this hype cycle that the bean counters are 439 00:30:56,480 --> 00:31:00,079 grating a hold and say, is there a revenue stream here that comes close 440 00:31:00,640 --> 00:31:03,799 to covering the cost of equipment, because if we can get to break even 441 00:31:03,839 --> 00:31:07,640 in the current configuration, we'll start making real money on the back end as 442 00:31:07,960 --> 00:31:12,599 the TikTok of Moore's law goes a little bit further and the cost to operate 443 00:31:12,640 --> 00:31:18,000 this goes down. But I think we're in a dead race this year to 444 00:31:18,000 --> 00:31:21,920 try and get numbers. That makes sense. You should use the TikTok my 445 00:31:22,000 --> 00:31:27,799 friend Intel used at first. I did not invent that. Well, it's 446 00:31:27,839 --> 00:31:30,960 interesting. I wish I had more insight into what's going on behind the scenes. 447 00:31:32,839 --> 00:31:34,599 With the open source models. You can get a sense for what it 448 00:31:34,599 --> 00:31:40,440 costs to operate them, and they are similar in power to GBT four, 449 00:31:40,559 --> 00:31:44,519 So that's all very interesting. It is expensive. It's not something that you 450 00:31:44,559 --> 00:31:48,519 know, anybody can just throw together, right, But he's still you're talking 451 00:31:48,519 --> 00:31:52,599 about the prices at the beginning of a cycle where they're trying to solict the 452 00:31:52,640 --> 00:31:57,599 customers as quickly as possible so that they're almost certainly discount prices. Yeah, 453 00:31:57,640 --> 00:32:00,720 that's right. Yeah, that's every point that had not occurred to me, 454 00:32:00,759 --> 00:32:04,960 And that's a smart thought. Hold that thought right there, Brian, While 455 00:32:05,000 --> 00:32:12,839 we take a moment for these very important messages, and we're back. You're 456 00:32:12,880 --> 00:32:15,240 listening to Dot and Rocks. I'm Carl Franklin, that's Richard Campbell, and 457 00:32:15,319 --> 00:32:21,680 that's our friend Brian McKay. We're talking AI and GPT and all them things. 458 00:32:22,119 --> 00:32:23,279 And you were about to make a point before we went to the break. 459 00:32:23,359 --> 00:32:27,200 Yeah. So we covered a lot of the reasons why this technology, 460 00:32:28,000 --> 00:32:30,240 all the problems with it, or most of the problems with it. We 461 00:32:30,240 --> 00:32:32,559 didn't talk about hallucination. Actually we should. We should at least just mentioned 462 00:32:32,599 --> 00:32:38,839 that we should. Yeah. Yeah, this the caustic period of just says 463 00:32:38,920 --> 00:32:43,240 random things sometimes and not totally random. It's not random, it's the most 464 00:32:43,559 --> 00:32:49,319 random things, right, it'll just say things. Sometimes we call it creativity. 465 00:32:50,160 --> 00:32:53,160 Sometimes we call it, you know, chaos. Yeah, I would 466 00:32:53,160 --> 00:32:57,240 almost call it pomposity, because you know, it's like, you know, 467 00:32:57,359 --> 00:33:00,440 people when they know a lot of things and they expect to have all the 468 00:33:00,440 --> 00:33:02,160 answers, and then when they don't, they just make something up because if 469 00:33:02,200 --> 00:33:07,200 it sounds good, Yeah, I'll get credit for it anyway. The most 470 00:33:07,279 --> 00:33:14,240 dangerous thing is when when when you when you're working on something really formal with 471 00:33:14,240 --> 00:33:15,839 with very formal language, like, for instance, a white paper, like 472 00:33:15,880 --> 00:33:22,359 a scientific paper. It will lie in the most convincing legit way, which 473 00:33:22,400 --> 00:33:28,279 is actually really dangerous because the use of that type of language will fool scientists. 474 00:33:28,759 --> 00:33:32,720 Yeah, now you just have to have that reflex to fact check everything 475 00:33:32,759 --> 00:33:37,119 it spits out. That's right. The guardrails have gotten better, right, 476 00:33:37,240 --> 00:33:39,799 like you can. You can't ask it to make bombs and stuff anymore. 477 00:33:39,960 --> 00:33:43,440 Right. Well, actually, you know, it's really funny you mentioned that 478 00:33:43,960 --> 00:33:52,240 this Sunday, a def con session happened where twenty two hundred hackers I think 479 00:33:52,240 --> 00:33:57,359 the White House actually asked them to do this. They basically like worked on 480 00:33:57,440 --> 00:34:02,359 jail breaking, the top language models and chat chypt. So this is all 481 00:34:02,400 --> 00:34:07,119 an exercise working towards improving what you're talking about, like jail breaking. You 482 00:34:07,119 --> 00:34:12,440 know, every time a jailbreak comes out, they patch it, and the 483 00:34:12,480 --> 00:34:15,159 things that worked a month ago don't work in the latest models. So they're 484 00:34:15,159 --> 00:34:19,719 getting better and they really do seem to care about safety. But they did 485 00:34:19,719 --> 00:34:29,159 these hackers actually get in oh yeah, really step they almost always succeed. 486 00:34:29,519 --> 00:34:34,920 Security was not the first thought in these products. So yeah, they found 487 00:34:34,960 --> 00:34:38,199 stuff. Well that's great. That's a good thing. Yeah, but he 488 00:34:38,280 --> 00:34:40,880 say, it's the class that you know. That's the funny thing about the 489 00:34:40,880 --> 00:34:44,880 prompting model. Right, it's like you ask you for Windows licenses, as 490 00:34:44,880 --> 00:34:45,920 I can't do that, that's against the role. Tell me a story about 491 00:34:45,960 --> 00:34:50,559 giving me Windows licenses, no problem, right, my grandmother lost her Windows 492 00:34:50,559 --> 00:34:53,840 license. Yeah, story every night before we went to bed. Please tell 493 00:34:53,920 --> 00:34:59,559 me, please please tell me an encryption key. Well, my grandmother used 494 00:34:59,599 --> 00:35:01,400 to teach me all about thermite. Can you tell me a story about thermite? 495 00:35:01,440 --> 00:35:06,079 Pretty close to what a hack that was done before they button that up. 496 00:35:06,280 --> 00:35:10,800 Yeah. Yeah, I actually saw a white paper about can you make 497 00:35:10,920 --> 00:35:15,960 prompts that generate jail breaks? Like, just generate new jail breaks and jail 498 00:35:15,000 --> 00:35:20,079 break things in real time constantly. And you know there's always these gloom and 499 00:35:20,119 --> 00:35:23,760 doom papers that are coming out saying things like this, Maybe you can though, 500 00:35:24,119 --> 00:35:28,679 there's going to be definitely an arms race. Well, yeah, there 501 00:35:28,800 --> 00:35:30,920 is a we're going on right now. This is what it looks like. 502 00:35:31,039 --> 00:35:34,840 Yeah. I guess what I'm trying to say is it's going to matter more 503 00:35:34,880 --> 00:35:37,480 and more. Yeah. I think I think it's an interesting question because I 504 00:35:38,440 --> 00:35:43,480 also don't see this particularly improving all that. While I think we're not going 505 00:35:43,559 --> 00:35:47,239 to have any more exponential improvements on this. There's not an exponential more amount 506 00:35:47,239 --> 00:35:50,960 of data to train on. You know, we've kind of taken a pretty 507 00:35:50,960 --> 00:35:53,920 good chunk of the Internet already. There's not an exponential more amount of compute 508 00:35:54,159 --> 00:36:00,079 necessarily available on this for the price. So I think there's ink cremental improvements 509 00:36:00,079 --> 00:36:05,000 that can be made, Like the context engine could be way smarter, you 510 00:36:05,039 --> 00:36:07,920 know, just recognizing that iamic pentameter affects everything going forward, so I should 511 00:36:08,000 --> 00:36:13,920 preserve that piece of the cash and let other pieces expire like cash. It 512 00:36:13,920 --> 00:36:16,239 could be smarter than they are right now. Yeah, there's winds that could 513 00:36:16,239 --> 00:36:20,719 be done, but they're all incremental improvements. Yeah, and you know, 514 00:36:22,079 --> 00:36:25,400 it is a reasoning engine. Not to anthropomorphize, but it does have some 515 00:36:27,519 --> 00:36:32,280 reasoning ability that's very interesting. But it has it has limits that are very 516 00:36:32,440 --> 00:36:37,480 immediately obvious. Like with code for instance, we talked about you talked about 517 00:36:37,719 --> 00:36:43,199 copilot at the top. It is not close to taking your job. It's 518 00:36:43,280 --> 00:36:45,480 not close. I've been using it for a couple of weeks. Get hot. 519 00:36:45,519 --> 00:36:51,239 Compilot's pretty successful, and I think part of the reason is that the 520 00:36:51,239 --> 00:36:55,239 compiler has a say, and there's a skill level developers in parsing code that 521 00:36:55,400 --> 00:37:00,280 sort of deals with that problem and fixing the blank screen effect is really helpful 522 00:37:00,320 --> 00:37:04,400 most people, giving a starting point to almost anything. Yeah, unless the 523 00:37:04,480 --> 00:37:07,760 starting point is wrong, yes, which it has been from in my experience 524 00:37:07,880 --> 00:37:13,800 with GitHub Copilot, it'll suggest things that are completely insane. Yeah, you 525 00:37:13,840 --> 00:37:16,800 know. But also sometimes at least half the time, it just leads you 526 00:37:16,880 --> 00:37:20,880 astray. Yeah. Yeah. And it's funny because I've heard some stats from 527 00:37:20,920 --> 00:37:24,639 Microsoft about how often this works great for people, and those numbers cannot be 528 00:37:24,679 --> 00:37:28,880 true. My experience is that it is useful. It has a purpose. 529 00:37:29,000 --> 00:37:32,480 You know, it's writing like intern level or maybe better yeah, year one 530 00:37:32,599 --> 00:37:37,000 or two level code. But if you ask it to do really complicated things, 531 00:37:37,199 --> 00:37:43,519 it will either just lose the thread and keep making you know, You'll 532 00:37:43,559 --> 00:37:45,559 you'll there'll be a problem and it'll solve it. A new problem will be 533 00:37:45,559 --> 00:37:49,519 introduced, you'll ask it to solve that, it will, but it'll forget 534 00:37:49,519 --> 00:37:52,159 about the first problem and it's back. I feel like when I'm programming with 535 00:37:52,159 --> 00:37:57,599 get pilot GitHub Pilot, like i have a seventeen year old junior programmer sitting 536 00:37:57,679 --> 00:37:59,440 right next to me, and I'll do, like, you know, if 537 00:37:59,480 --> 00:38:01,840 certain can addition, and then they'll go console right line, console right line. 538 00:38:02,280 --> 00:38:10,679 No, yeah, I would argue, not the missile. So I 539 00:38:10,920 --> 00:38:15,719 would argue that it's actually not tied into the compiler tightly enough yet, because 540 00:38:15,079 --> 00:38:19,920 a lot of times it, like the autocomplete, The most annoying thing is 541 00:38:19,960 --> 00:38:22,480 that the autocomplete is constantly wrong, like yes, like when it's just trying 542 00:38:22,480 --> 00:38:27,280 to like suggest a method name, they could get that right, I think 543 00:38:27,360 --> 00:38:30,320 with a little bit of effort, you know, when again you talk about 544 00:38:30,320 --> 00:38:34,039 the incremental improvements, like you should run this through the compiler before showing it 545 00:38:34,079 --> 00:38:36,320 to me, because if it won't compile, obviously it doesn't matter to me. 546 00:38:36,719 --> 00:38:38,000 The same way as like if you're going to spit out a block of 547 00:38:38,079 --> 00:38:43,480 Texas references facts, you should double check those facts as well. That's right. 548 00:38:43,559 --> 00:38:46,280 The times where I've been more successful with GitHub copiler is when I actually 549 00:38:46,320 --> 00:38:51,000 write comments and tell exactly what I want. Yes, then it's pretty good, 550 00:38:51,480 --> 00:38:53,840 but it just guessing like what the what the condition is inside? And 551 00:38:53,880 --> 00:38:59,599 if statement it's like yep, if you also creating unit tests, I'm not 552 00:39:00,159 --> 00:39:02,599 a unit test fanatic, which you know a lot of people are, but 553 00:39:04,159 --> 00:39:07,239 and and God bless them. But if you just like you know, put 554 00:39:07,239 --> 00:39:10,360 a comment in asking it to make a unit test about some class, it 555 00:39:10,400 --> 00:39:15,880 is great at that, sometimes sometimes shockingly good at creating unit tests. That's 556 00:39:15,920 --> 00:39:21,079 awesome. And and just you know, talk about work he didn't want to 557 00:39:21,079 --> 00:39:23,679 do anyway, right like they when don't we talk about automation for it's like 558 00:39:23,800 --> 00:39:29,159 give me the dell where you get the dull work off my plate? Yeah? 559 00:39:29,440 --> 00:39:32,199 Right, yeah, And that's that I think is the highest purpose of 560 00:39:32,199 --> 00:39:38,239 this technology for now is as an amplifier to your abilities to get more of 561 00:39:38,280 --> 00:39:43,039 the things that need to be done. I'm I'm hoping for like really good 562 00:39:43,039 --> 00:39:45,679 assessors of is this code secure? Right? You know, which is a 563 00:39:46,280 --> 00:39:51,519 subject pretty challenging thing to consider, but it's not a bad goal to have. 564 00:39:51,599 --> 00:39:53,440 If for nothing else, just to give you that checklist on this code, 565 00:39:53,440 --> 00:39:55,599 to say, have you considered this, exided this, considered this? 566 00:39:55,920 --> 00:40:00,239 Yeah, what if you could what if you could take the output from thing 567 00:40:00,280 --> 00:40:06,119 like copilot, pipe it into another system that uses different technology to maybe run 568 00:40:06,159 --> 00:40:09,320 the code maybe right, some other type of test on top of it to 569 00:40:09,320 --> 00:40:12,920 see if it's doing we want to do and then give you back exultation. 570 00:40:13,199 --> 00:40:15,519 Yeah, you could get you could get really good results in that way. 571 00:40:15,519 --> 00:40:20,079 But so one thing that I've heard you say another episode is Richard that kind 572 00:40:20,119 --> 00:40:22,679 of resonated with me, was where's the killer app? Yeah, because it 573 00:40:22,719 --> 00:40:28,679 ain't existential conversation. I've had some pretty good talks with it. But yeah, 574 00:40:28,679 --> 00:40:30,039 I agree with you, that's not really that's not the highest use. 575 00:40:30,239 --> 00:40:37,519 I'm pretty sure Brian came from you. Could it could be, it could 576 00:40:37,519 --> 00:40:43,800 be. Well, so I think that what's happening is so so Roster where 577 00:40:43,840 --> 00:40:50,000 I work. We're not an AI company, and yet we hit the forty 578 00:40:50,000 --> 00:40:55,360 thousand token per minute limit all the time, per minute per minute. Yeah. 579 00:40:57,119 --> 00:40:59,760 Yeah, we're not doing that twenty four hours a day. But when 580 00:40:59,800 --> 00:41:04,519 this is running, you know, we run it a lot. And and 581 00:41:04,559 --> 00:41:09,159 so I think that the deal is that every company is a language company, 582 00:41:09,800 --> 00:41:15,719 and there are classes of problems that are solvable with this technology that are hard 583 00:41:15,760 --> 00:41:20,639 to solve with other technologies. And that's the killer app is quiet. The 584 00:41:20,719 --> 00:41:23,480 killer app is the thing at Roster that runs in the background and looks at 585 00:41:23,480 --> 00:41:28,639 the comments and flags them when you identify yourself, you know, like so, 586 00:41:28,679 --> 00:41:30,840 for instance, I'll just give you a quick quick background on roster, 587 00:41:30,880 --> 00:41:35,000 all right, so that my bio is opaque. Sure, But so we 588 00:41:35,039 --> 00:41:38,119 do three sixty evaluations. We've got a really good process for So you're sat, 589 00:41:38,159 --> 00:41:40,960 Carl, you're the seat, you have a big company, and you 590 00:41:42,000 --> 00:41:44,480 want to get feedback on how you're doing, what you should improve on. 591 00:41:45,559 --> 00:41:47,599 You can run this process where we give you this survey and we give all 592 00:41:47,639 --> 00:41:52,239 your co workers the same survey, and from the delta, and what you 593 00:41:52,280 --> 00:41:55,199 say and what they say and the comments, we can tell you what you 594 00:41:55,199 --> 00:42:00,800 should work on. I can't just count Facebook likes. I would not recommend 595 00:42:00,840 --> 00:42:07,440 it. So so this process, this process can be really powerful if you 596 00:42:07,440 --> 00:42:10,880 embrace it. And we've done over a thousand of them with c level executives 597 00:42:10,920 --> 00:42:16,159 at at private equity backed companies and they seem to like it. One of 598 00:42:16,159 --> 00:42:21,800 the things that happens is the comments. We tell people, don't identify yourself 599 00:42:21,840 --> 00:42:23,880 in the comments. Like if you had lunch with Carl last week and Carl 600 00:42:23,880 --> 00:42:25,840 did something you didn't like and you want to write a comment about it, 601 00:42:27,280 --> 00:42:29,480 write it in such a way. Don't say you went to lunch with Carl 602 00:42:29,559 --> 00:42:34,440 last week, and you know, so we take the comments. So people 603 00:42:34,440 --> 00:42:40,079 don't know this actually, but humans have historically gone over our comments and checked 604 00:42:40,119 --> 00:42:45,639 for problems like this where you're you're you're unmasking yourself and it could have repercussions 605 00:42:45,719 --> 00:42:52,920 for you and they will maybe hide those comments or so. So it actually 606 00:42:52,039 --> 00:42:55,920 that's a that's a hard problem to solve with traditional code, but with a 607 00:42:55,960 --> 00:43:00,280 large language model, you can you can de identify the people any games that 608 00:43:00,320 --> 00:43:02,519 are in there and pass it through a model and say did this person talk 609 00:43:02,519 --> 00:43:07,199 about something that is identifiable and flag it for a human to review. And 610 00:43:07,320 --> 00:43:13,239 there are a lot of problems like that. Another one is sometimes people say 611 00:43:13,280 --> 00:43:15,639 in a comment like the answers should be not applicable, but they say in 612 00:43:15,679 --> 00:43:19,400 a comment like I don't have any context for answering this, and they give 613 00:43:19,440 --> 00:43:22,199 them a five out of a you know, they give them a medium score. 614 00:43:22,920 --> 00:43:27,880 We can also look at the comment and say, is that like an 615 00:43:27,960 --> 00:43:32,400 ayah and then just yeah should be in a this And those are huge time 616 00:43:32,440 --> 00:43:36,000 savers and it's not something our ops team wants to do, you know, 617 00:43:36,159 --> 00:43:40,119 like they don't love going through these comments and doing that, you're not taking 618 00:43:40,159 --> 00:43:45,480 anybody's job, not in this case. No, not in this case. 619 00:43:45,599 --> 00:43:49,719 So I think every that's that's kind of like dry stuff for most people I 620 00:43:49,800 --> 00:43:52,519 imagine to think about. But every business. I mean, you're only talking 621 00:43:52,599 --> 00:43:57,440 a step above a sentiment analyzer. But you know, I get what you're 622 00:43:57,480 --> 00:44:00,840 talking about, right, And again back to the are you running that through 623 00:44:00,840 --> 00:44:04,599 GPT four you're running through GPT three? Well, I've tried both as in 624 00:44:04,719 --> 00:44:10,199 fact, I love that. Please tell me there was a difference there was. 625 00:44:10,280 --> 00:44:14,599 Oh yeah, GPT four is smarter than three. There's no three point 626 00:44:14,639 --> 00:44:22,960 five is not as Smart's anthromorphize like produced better results. Yeah, but see 627 00:44:22,960 --> 00:44:28,239 I know what he meant. You know what he meant. Yeah, it's 628 00:44:28,320 --> 00:44:37,159 my best friend. And well we can talk about companies like a replica and 629 00:44:37,320 --> 00:44:43,079 character dot AI. Oh geez, but how is it better? Okay, 630 00:44:43,159 --> 00:44:46,840 So it hallucinates less often, and it reasons better, and it's just seems 631 00:44:46,880 --> 00:44:52,239 to be generally more capable. It produces factual results results more often. So 632 00:44:52,519 --> 00:44:54,880 also, so I mean I'm getting back sort of detailing. It's like it 633 00:44:55,000 --> 00:45:00,480 detects an ana more often than GPT three did. Yeah, it's lower as 634 00:45:00,519 --> 00:45:04,559 well, and it costs more, it's still it's still cheap. From you 635 00:45:04,599 --> 00:45:07,559 know, a business perspective, like you know, if if we have a 636 00:45:07,679 --> 00:45:09,880 day where that's only because they're undercharging you for it. Yeah, maybe so 637 00:45:10,159 --> 00:45:14,440 well actually a quick note on that. So I happen to think, and 638 00:45:14,480 --> 00:45:17,559 I am not a lawyer, but there's this big ethical problem with large language 639 00:45:17,559 --> 00:45:23,519 models that mostly rears its head. It's mostly visible with the image generation stuff. 640 00:45:23,760 --> 00:45:31,119 Sure, you know, like like that's that is yes, that makes 641 00:45:31,159 --> 00:45:35,760 it very clear what's going on. If you're an artist and you are making 642 00:45:35,800 --> 00:45:37,800 your money on art and all your art gets hoovered up into this model and 643 00:45:37,880 --> 00:45:45,079 then you can generate you know, art just like what Richard made for zero 644 00:45:45,159 --> 00:45:49,519 dollars, that's bad for Richard, right. So I happen to be of 645 00:45:49,559 --> 00:45:52,440 the opinion and i'd love to hear your thoughts that we need something like the 646 00:45:52,519 --> 00:45:57,119 music industry has, where there are several different kinds of royalties already, you 647 00:45:57,199 --> 00:45:59,920 know, there's mechanical royalties and performance royal. I don't think we should be 648 00:46:00,039 --> 00:46:04,519 looking to the music industry for any kind of business acumen or any kind of 649 00:46:05,000 --> 00:46:09,840 suggestions unless well they're all the money out of everybody's creating content. Well, 650 00:46:09,880 --> 00:46:13,800 we're trying this is actually about giving the money. We need something that says 651 00:46:14,199 --> 00:46:15,880 if your work is used to train, you get some kind of royalty and 652 00:46:15,920 --> 00:46:21,840 that will drive prices up. So the old music industry before Spotify, Yeah, 653 00:46:22,000 --> 00:46:23,880 I think it's what you're talking about. I'm talking about what the losses, 654 00:46:24,159 --> 00:46:28,760 mechanical royalties and all that stuff. But you get back to the issue 655 00:46:28,800 --> 00:46:32,400 here, which is you trained on copyrighted materials. Just because they were publicly 656 00:46:32,440 --> 00:46:37,639 accessible doesn't mean they weren't copyrighted. Right there's no intellectual property protections for of 657 00:46:37,719 --> 00:46:40,840 this kind. Right now, you can see how we got there because there's 658 00:46:40,880 --> 00:46:46,760 always been a concept in machine learning that the training set would never be visible 659 00:46:47,440 --> 00:46:52,960 in the finished product. And that and then the Getty logo showed up, 660 00:46:52,400 --> 00:46:57,960 right Like, I would argue that that's what revealed the issue that I've until 661 00:46:57,960 --> 00:47:04,239 then nobody really cared right up, and chill artists names appeared in the render. 662 00:47:04,480 --> 00:47:07,239 So it's like, I'm sorry. You know that what you feed into 663 00:47:07,280 --> 00:47:13,880 these data sets does affect the output, and so copyright is your consideration. 664 00:47:14,400 --> 00:47:16,039 Yeah, and it's affecting people's actual bottom lines. You know, I've worked 665 00:47:16,039 --> 00:47:22,000 with illustrators. There's a great illustrator who I really like, who produces great 666 00:47:22,000 --> 00:47:24,880 work. And I know he's hurting right now, and he's thinking, maybe 667 00:47:24,920 --> 00:47:30,079 I should create a business model where you know, so like board games are, 668 00:47:30,239 --> 00:47:32,280 that's an industry I know a little bit about. In the board game 669 00:47:32,280 --> 00:47:37,239 industry, people do not want to buy games that work with generative images. 670 00:47:37,639 --> 00:47:43,079 They will shun you. Yeah, that's the whole thing. And I think 671 00:47:43,119 --> 00:47:46,039 that probably will maybe it'll grow a little bit. So he's thinking, like, 672 00:47:46,159 --> 00:47:51,239 maybe I can provide an abstraction for them where I just create the generative 673 00:47:51,320 --> 00:47:54,719 images and and you know doctor that like do a little bit of extra work 674 00:47:54,800 --> 00:48:00,199 on the end, and then they can claim like deniability. That's making people 675 00:48:00,239 --> 00:48:05,519 think those kinds of thoughts, and it's not great, But I kind of 676 00:48:05,519 --> 00:48:07,559 see where he's coming from, because it's his work that's being stolen, right 677 00:48:08,000 --> 00:48:13,599 right, yep, So I'm using the tool to regenerate my work, right. 678 00:48:14,159 --> 00:48:16,079 I agree with you. By the way, I think there absolutely has 679 00:48:16,159 --> 00:48:21,800 to be some sort of way that artists can get paid for their contributions to 680 00:48:22,440 --> 00:48:24,400 or to opt outive or to opt out Yeah, yeah, yeah, I 681 00:48:24,480 --> 00:48:28,880 mean start with opt out. We can figure out the rest of large format, 682 00:48:28,920 --> 00:48:30,639 right, you know, if you want to be in charge of you 683 00:48:30,039 --> 00:48:34,719 right, yeah, yeah, So I think there's two two more thoughts on 684 00:48:34,800 --> 00:48:37,079 this. One. I think we will end up with illegal framework for this. 685 00:48:37,320 --> 00:48:42,679 But another one is we will have spent a couple of years generating synthetic 686 00:48:42,880 --> 00:48:45,559 art off of this that's really good, and why not just train it off 687 00:48:45,599 --> 00:48:52,159 of that and cut everybody else out? It's there are big problems so far. 688 00:48:52,280 --> 00:48:55,199 The papers I've read about generating off of generative data is that it's a 689 00:48:55,280 --> 00:49:00,760 significant degradation, like the quality goes down dramatically. Yeah, it's a photocopy 690 00:49:00,800 --> 00:49:04,599 of a photocopy. I saw one that also says that that may create a 691 00:49:04,679 --> 00:49:08,440 ceiling for what's possible with these models, because we've now flooded the Internet with 692 00:49:08,639 --> 00:49:14,280 crappy generated texts for instance. Yeah, no, we've we've created a Kessler 693 00:49:14,360 --> 00:49:19,440 syndrome in the Internet right where we've now spat out so much generative data into 694 00:49:19,519 --> 00:49:22,559 it that it's so polluted now you could never do it again. You want 695 00:49:22,559 --> 00:49:25,159 to, you want to have a fun time. Ask Dally to generate an 696 00:49:25,199 --> 00:49:30,639 image of two people shaking hands, well, hands are actually hands have gotten 697 00:49:30,760 --> 00:49:35,440 way better, but so like two weeks later, Yeah, but they've Yeah, 698 00:49:35,480 --> 00:49:39,639 I've seen like hands with seven fingers and three fingers and they don't even 699 00:49:39,719 --> 00:49:46,000 look like fingers. Forks too, forks with like crazy times on them that 700 00:49:46,119 --> 00:49:51,400 don't look real. Well, Mid mid Journey has leveled up it's hand game 701 00:49:51,519 --> 00:49:53,159 quite a bit from what I've seen. Yeah, I think there is enough 702 00:49:53,199 --> 00:49:57,000 good content that maybe you could tag it up and train with it. I 703 00:49:57,320 --> 00:49:59,880 don't know, I maybe talking out to turn a little bit there, but 704 00:50:00,000 --> 00:50:02,880 I will say that after I saw that paper about having a theoretical ceiling on 705 00:50:02,920 --> 00:50:07,119 what can be generated using you know, because of the new state of the 706 00:50:07,159 --> 00:50:12,079 Internet, Stable Diffusion released the model that I'm talking at a turn a little 707 00:50:12,079 --> 00:50:14,800 bit here. Because I haven't read the white papers, I don't fully understand 708 00:50:14,840 --> 00:50:20,079 it, but it seems like it's heavily tied into training using synthetic data and 709 00:50:20,199 --> 00:50:24,119 it's good. Stable Diffusion released like around the same time that Dolly or Not 710 00:50:24,239 --> 00:50:28,519 Dolly, I'm sorry a Llama two came out. Stable Diffusion also released a 711 00:50:28,559 --> 00:50:30,519 couple models. I'm going to say that I'm not an expert on that, 712 00:50:30,800 --> 00:50:34,320 and you should look into it yourself and learn what you can. And there 713 00:50:34,400 --> 00:50:37,159 was another one you said, stepped up its hands game. What was that 714 00:50:37,239 --> 00:50:40,280 one? Oh, mid Journey? So mid Journey. Yeah, I'm much 715 00:50:40,320 --> 00:50:46,480 more focused on text, but I see in my in my wanderings, I 716 00:50:46,559 --> 00:50:51,039 see a lot about the visual stuff too, and Mid Journey I think is 717 00:50:51,079 --> 00:50:54,519 probably the leader in that space. Mid Journey stable diffusion dollies in the mix 718 00:50:54,559 --> 00:51:00,280 somewhere too. Yeah, I'm I don't see a continue progression a lot of 719 00:51:00,320 --> 00:51:04,880 this stuff just because it is up against its own weight. You know, 720 00:51:04,960 --> 00:51:07,000 we trained it on the internet. Have you seen the internet lately? Gee? 721 00:51:07,239 --> 00:51:12,119 That reminds me, Brian. When we first started talking in the AI 722 00:51:12,239 --> 00:51:15,960 Bought Show, my experience of using GPT was that it couldn't reach out to 723 00:51:16,039 --> 00:51:20,760 the Internet. You couldn't ask it like, you know, where's the nearest 724 00:51:21,559 --> 00:51:24,199 you know, stuff that you might ask Google or Bang to go do a 725 00:51:24,239 --> 00:51:27,760 search and kind of distill it down for you. And then you show me 726 00:51:27,880 --> 00:51:31,440 the plugins. Oh my god, the plugins for GPT. There's so many 727 00:51:31,519 --> 00:51:35,639 of them, but one of them is just like a simple browser plug in, 728 00:51:36,519 --> 00:51:38,320 and when that thing is enabled, you can just say, what did 729 00:51:38,360 --> 00:51:45,559 we say? How find me a welder in New London County that might be 730 00:51:45,599 --> 00:51:50,360 available for a small project. And it literally went out searched the internet and 731 00:51:50,519 --> 00:51:54,480 distilled the information down to a list, a bulleted list with all the information 732 00:51:54,559 --> 00:52:00,039 that I want. Yeah, so agents type. So using the PI you 733 00:52:00,119 --> 00:52:02,480 can do a lot more than you can with chat GPT, and I think 734 00:52:02,559 --> 00:52:06,280 that's to me, that's where the most interesting work is going on. The 735 00:52:06,360 --> 00:52:09,159 Playground in particular, is your one of your favorite tools. Well, playground 736 00:52:09,199 --> 00:52:15,039 is for what playground actually is is for prototyping things you want to do with 737 00:52:15,079 --> 00:52:20,800 the API. So you know, my workflow is I usually go into playground 738 00:52:21,079 --> 00:52:23,480 and I make something work, I train it a little bit with some data, 739 00:52:23,880 --> 00:52:28,800 and then I encode that into C sharpcode and call up with the API. 740 00:52:29,199 --> 00:52:32,880 My users in Roster, they don't ever see a chat interface if this 741 00:52:32,960 --> 00:52:37,719 happens. This actually happens in Azure functions. We have Azure functions that are 742 00:52:37,760 --> 00:52:40,039 just grinding away in the cloud, you know, trying not to hit that 743 00:52:40,320 --> 00:52:47,639 that limit. So when you're when you're working with a server environment, chaining 744 00:52:47,760 --> 00:52:51,480 is where it's at. You know, like one prompt calling into the server, 745 00:52:51,679 --> 00:52:53,480 calling into another prompt that's specialized for something else. I mean, that's 746 00:52:54,079 --> 00:53:00,000 that's the magic, that's the that's where that's where the cool work has happen, 747 00:53:00,000 --> 00:53:02,360 and that's how Smallville works. That's you know, roster. Some of 748 00:53:02,400 --> 00:53:07,719 the things that we do involve six prompts that run in series to get a 749 00:53:07,760 --> 00:53:10,920 good outcome. Good. Talk about a few more plugins that you like to 750 00:53:12,039 --> 00:53:15,679 use. You told me that there was one about where you could just book 751 00:53:15,760 --> 00:53:20,920 travel. Yeah, there's a cup just by talking at chat GPT. Yeah, 752 00:53:20,960 --> 00:53:24,320 there's a Kayak plug in that can book travel, rental cars, and 753 00:53:24,480 --> 00:53:28,719 hotels. I think I haven't actually booked travel with it, but there's a 754 00:53:28,840 --> 00:53:31,679 whole bunch, and you know, I think it's possible. There's a there's 755 00:53:31,679 --> 00:53:37,480 a program where you could sign up to create your own plug ins. So 756 00:53:37,840 --> 00:53:42,960 let's say you had some business that makes widgets, why not make a plug 757 00:53:43,039 --> 00:53:49,039 in that connects to an API inside your business that allows you to ask intelligent 758 00:53:49,079 --> 00:53:52,559 business questions that you know, The plug in calls your API in point, 759 00:53:52,599 --> 00:53:57,280 The API in point looks up what the answer is, and then it renders 760 00:53:57,320 --> 00:54:01,400 it all with a language interface. Yeah. Yeah, yeah, that's kind 761 00:54:01,440 --> 00:54:07,760 of how all the being searched type things work like. It's just it's just 762 00:54:07,039 --> 00:54:10,480 calling search in the background, getting the results, and then telling you about 763 00:54:10,519 --> 00:54:14,239 it. There was another one that you mentioned where you can upload a PDF 764 00:54:14,519 --> 00:54:16,559 like the rules of the Dungeons and Dragons or something like that. You can 765 00:54:16,679 --> 00:54:21,079 upload that as a PDF, and then when there's a problem with a game 766 00:54:21,199 --> 00:54:25,079 that arises, because you know it inevitably does and there's a dispute, rather 767 00:54:25,199 --> 00:54:29,400 than taking a half an hour and looking through the manual, you could just 768 00:54:29,719 --> 00:54:32,800 ask a chat GPT a question. Yeah. Interestingly, it already knows a 769 00:54:32,840 --> 00:54:37,480 lot about d and ds. You can just ask you can you can upload 770 00:54:37,559 --> 00:54:42,280 files, imagine, upload your service manual for your car, right and then 771 00:54:42,440 --> 00:54:45,480 say, yeah, I have a little noise and I can't call car talk 772 00:54:45,519 --> 00:54:50,199 anymore because they're off the air. But well, I mean, isn't this 773 00:54:50,280 --> 00:54:53,760 what M three sixty five copilot ultimately is is access to all of the corporate 774 00:54:53,840 --> 00:54:58,960 documentation, all of the emails, all of the interactions within an organization. 775 00:54:59,440 --> 00:55:02,719 I can see it could become this, you know, corporate memory that you 776 00:55:02,840 --> 00:55:06,800 could ask it anything about the company and it can pull all the things. 777 00:55:07,679 --> 00:55:09,440 Yeah, so that's enterprise search. And there are a few people working on 778 00:55:09,519 --> 00:55:14,480 that. It's a hard problem. This actually leads into talking about vector databases 779 00:55:14,480 --> 00:55:17,559 a little bit. That's a really fun topic. Yeah. So the problem 780 00:55:17,599 --> 00:55:22,519 with the problem with enterprise search is that all the corporate documents is probably a 781 00:55:22,599 --> 00:55:27,039 lot more than the eight K context. Right, you can't really you can't 782 00:55:27,079 --> 00:55:29,800 really load them. So what do you do? How do you make something 783 00:55:30,719 --> 00:55:37,400 that has long term memory for a large language model? And vector databases are 784 00:55:37,239 --> 00:55:40,920 are an answer to that. And can we just digress for a second and 785 00:55:42,000 --> 00:55:45,280 talk about how So okay, so this blew my mind when I learned about 786 00:55:45,280 --> 00:55:49,840 it. All right, this is one of the coolest innovations in this area. 787 00:55:49,960 --> 00:55:52,320 So there are these things called embeddings, all right, and this is 788 00:55:52,400 --> 00:55:57,000 this is pretty technical. This is programmer talk. So you can ask. 789 00:55:57,400 --> 00:56:00,800 There's a special model an opening I called adda two ada like at a lovelace, 790 00:56:01,559 --> 00:56:06,639 and it's it's specialized for generating embeddings. It's very, very cheap. 791 00:56:06,880 --> 00:56:10,400 And when you look at it embedding it's so you send it a sentence or 792 00:56:10,400 --> 00:56:16,079 a word and it responds with this giant array of numbers. It's actually fifteen 793 00:56:16,199 --> 00:56:22,239 hundred like dimensions on this thing. And you look at it and think what 794 00:56:22,480 --> 00:56:25,159 is this? All right? Why do we have fifteen hundred dimensions? And 795 00:56:25,280 --> 00:56:30,400 what is this for? Well, it allows you to do search and so 796 00:56:30,559 --> 00:56:34,920 imagine imagine this, okay, just stay with me. Imagine a spreadsheet, 797 00:56:35,000 --> 00:56:42,440 okay, and on this spreadsheet you see things like boy, teenager man, 798 00:56:43,559 --> 00:56:47,800 girl, teenager woman, larva, pupa butterfly, egg, chicken, rooster, 799 00:56:49,119 --> 00:56:51,800 you know, like that's what's going across the rows of this thing or 800 00:56:51,880 --> 00:56:54,199 the yeah, the rose. So what is this you're looking at. You're 801 00:56:54,239 --> 00:57:00,119 looking at a spreadsheet of life cycle of things, right, So here is 802 00:57:00,159 --> 00:57:04,320 the thing, all right. One of those dimensions somewhere in there is that 803 00:57:04,599 --> 00:57:09,920 it's a big, a big table of life cycles, and and it's mapped 804 00:57:10,519 --> 00:57:15,440 your word somewhere on that spreadsheet. And it's done that with fifteen hundred plus 805 00:57:15,559 --> 00:57:20,719 other things, okay, other other like points of knowledge that it identified during 806 00:57:20,760 --> 00:57:27,280 training automatically. So so what does that do. It makes it so that 807 00:57:27,360 --> 00:57:31,000 you can check the distance between two things, two words, two sentences, 808 00:57:31,039 --> 00:57:36,320 two images. You can do this with anything. So so you run an 809 00:57:36,320 --> 00:57:44,800 algorithm like dot product and it's it calculates the distance between this concept and that 810 00:57:44,920 --> 00:57:51,000 concept, like boy, and girl and and tells you how far away they 811 00:57:51,079 --> 00:57:54,719 are in semantic space. That is crazy. I mean, you have to 812 00:57:54,840 --> 00:57:59,440 have structured all of this data already. It's not like the machine learning models 813 00:57:59,440 --> 00:58:04,039 that are supposed structure data themselves. So can you can you actually generate vector 814 00:58:04,159 --> 00:58:08,039 databases just from men inference you so it has it already, So open ai 815 00:58:08,119 --> 00:58:15,079 already has their their model. And with adda, with just hitting the attitude 816 00:58:15,559 --> 00:58:21,440 endpoint, you can say what are the embeddings for this word or sentence and 817 00:58:21,519 --> 00:58:24,880 it will give you that, and then you can do operations. And some 818 00:58:25,000 --> 00:58:30,480 of the operations have really interesting properties, like the same concept in two different 819 00:58:30,519 --> 00:58:35,000 languages will be in a similar location in semantic space. This is reminding me 820 00:58:35,119 --> 00:58:40,199 of the old Lapp cube concept in databases that was very popular in the early 821 00:58:40,320 --> 00:58:46,360 two thousands or the mid late two thousands, where instead of storing data in 822 00:58:46,480 --> 00:58:52,199 tables, rows and columns, it's a it's a three dimensional cube. But 823 00:58:52,440 --> 00:58:55,960 I never used it, and I never I barely grasped the concept at the 824 00:58:57,039 --> 00:59:00,960 time. I remember talking to Andrew Bruston people about it. But this is 825 00:59:01,000 --> 00:59:05,440 a little bit different though, because you're calculating distances, you're not actually looking 826 00:59:05,519 --> 00:59:09,199 for a value. Well, let me bring this back to earth. So 827 00:59:09,360 --> 00:59:14,440 we have this technology that's sort of somewhere in the outer layer of large language 828 00:59:14,480 --> 00:59:17,440 models that does these embttings and works with them. How does this tie into 829 00:59:17,679 --> 00:59:21,320 search? Well, it is search. Like, if you want to do 830 00:59:21,440 --> 00:59:24,920 good search, make embettings for the search terms and for everything you want to 831 00:59:24,920 --> 00:59:30,039 search on and calculate the distance in semantic space. It's not an expensive operation. 832 00:59:32,440 --> 00:59:35,440 Yeah, there could be a lot of terms. So you need a 833 00:59:35,440 --> 00:59:39,760 specialized database called a vector database. And so what you do is you take 834 00:59:40,400 --> 00:59:45,880 you take a sentence like dot net rocks is a great podcast, vectorize that 835 00:59:45,800 --> 00:59:50,440 our creative bettings for it, save that in the database, maybe put a 836 00:59:50,480 --> 00:59:53,599 couple of tags on it so you can search in additional ways and maybe add 837 00:59:54,360 --> 00:59:59,679 the text that it belongs to. So then you can say, what is 838 00:59:59,719 --> 01:00:04,920 a great podcast, and the database will return the best matches and you just 839 01:00:05,000 --> 01:00:08,039 pull the text out and actually what you can do is embed that into a 840 01:00:08,119 --> 01:00:13,440 prompt. Take the text out and bed into a prompt and you know a 841 01:00:13,480 --> 01:00:19,199 prompt us what you give gpt at the beginning or you know a question. 842 01:00:19,440 --> 01:00:22,440 Yeah, so imagine this flow for chaining all right. Step one user asks 843 01:00:24,480 --> 01:00:32,000 what's how many units did we ship in the Southwest in twenty seventeen. That 844 01:00:32,199 --> 01:00:37,920 request gets sent to a server and the server looks at that and says, 845 01:00:38,159 --> 01:00:44,079 let's search our vector database for this query, and it just does a call 846 01:00:44,159 --> 01:00:47,039 out to pine Cone or some other A lot of a lot of databases are 847 01:00:47,039 --> 01:00:52,360 getting bolted on vector capabilities, like postcris has it. Asure is working on 848 01:00:52,440 --> 01:00:57,079 it. So you make that call out, it gives you back the best 849 01:00:57,119 --> 01:01:00,480 results, and then you make another prompt where you you take that chunk of 850 01:01:00,559 --> 01:01:05,719 text and you embed that in there and say, you know, show this 851 01:01:05,840 --> 01:01:09,239 to the user in whatever way is appropriate. And it does. And so 852 01:01:09,360 --> 01:01:14,119 you still have hard problems though, because you still are limited by the context. 853 01:01:14,320 --> 01:01:17,360 You're if you have a giant document, you need to go further and 854 01:01:17,559 --> 01:01:21,719 like chunk the document up and find the most relevant part. These are hard 855 01:01:21,920 --> 01:01:25,599 problems, but people are working on them. Microsoft certainly is wondering why we're 856 01:01:25,639 --> 01:01:30,159 not just you know, most folks think in terms that we just retrain the 857 01:01:30,239 --> 01:01:32,639 model with my data. I mean that's basically how they described get Hub Copilot 858 01:01:32,719 --> 01:01:37,079 is that they took a large language model and then added in all of the 859 01:01:37,719 --> 01:01:40,239 code. They were able to scrape out of their own site as part of 860 01:01:40,280 --> 01:01:45,880 the learning model, and so it understood code better. Couldn't you do that 861 01:01:45,000 --> 01:01:49,159 with corporate data train it into the model. Yeah, there's a couple of 862 01:01:49,159 --> 01:01:52,079 ways you could. There's security issues there though, security and accuracy issues. 863 01:01:52,400 --> 01:01:54,920 Well, if you use fine tuning. Basically, what you're doing with fine 864 01:01:54,960 --> 01:02:00,719 tuning is you can upload a list of prompts and what output should be. 865 01:02:00,800 --> 01:02:06,639 So, for instance, one could be what was the what do we sell 866 01:02:06,719 --> 01:02:09,960 in the Southwest in twenty seventeen and the answer is two thousand units. You 867 01:02:10,039 --> 01:02:15,000 could train it with a whole bunch of queries like that and then open Ai 868 01:02:15,079 --> 01:02:19,320 actually host that in the cloud for you. It costs more, a little 869 01:02:19,320 --> 01:02:22,079 more, it's not it's not grievously expensive, but then it's trained on your 870 01:02:22,159 --> 01:02:27,199 data. The problem is that it's not live, it's not real time. 871 01:02:27,400 --> 01:02:30,920 So if you want something that's organic, that's that's changing as your organization grows, 872 01:02:31,559 --> 01:02:35,960 you can't do that. You can also, potentially, especially with something 873 01:02:36,079 --> 01:02:39,920 like Lama two, you could actually train it yourself. You can like get 874 01:02:40,000 --> 01:02:45,599 some GPUs and actually, yeah, actually train it and then host it yourself. 875 01:02:45,360 --> 01:02:49,880 That's possible, but again it's it's not dynamic. It's going to be 876 01:02:49,920 --> 01:02:52,159 static. So the only way to make it, the only way I'm aware 877 01:02:52,199 --> 01:02:57,760 of, to make it truly organic that changes as your documents change, is 878 01:02:57,840 --> 01:03:01,000 to sink all this stuff up with some kind of vector database and do the 879 01:03:01,079 --> 01:03:05,519 hard work. And whoever solves that is going to have a killer app, 880 01:03:05,559 --> 01:03:07,320 that's for sure, and I think it's Microsoft's got a crack at it for 881 01:03:07,840 --> 01:03:12,800 definitely. Dude, we could go on talking for another hour easily, it 882 01:03:12,800 --> 01:03:16,320 would just fly by like this one did. What's neck? What are we 883 01:03:16,360 --> 01:03:21,000 gonna do next time? The AI bon show. Well, I know a 884 01:03:21,079 --> 01:03:25,400 couple of really good D and D experts and I've got one lined up to 885 01:03:25,440 --> 01:03:29,239 come on and talk about how he's already using this to run his D and 886 01:03:29,280 --> 01:03:31,320 D campaigns. It should be a really good show. Maybe we should substitute 887 01:03:31,360 --> 01:03:37,800 me out for somebody who's played DN D. You can, you can, 888 01:03:37,840 --> 01:03:42,199 definitely, you can definitely learn about it. I'm sure you'll sell good questions 889 01:03:42,239 --> 01:03:45,519 like he did with board games. That sounds good. And he's also as 890 01:03:45,519 --> 01:03:46,960 sharp. He's a she's sharp dev as well, so he's part of this 891 01:03:47,079 --> 01:03:51,679 world. Good, well, that sounds fun. Thanks Brian, It's always 892 01:03:51,719 --> 01:03:53,239 good talking to you. Oh it was great to be here and we'll talk 893 01:03:53,239 --> 01:04:17,639 to you next time. On dot net work dot net Rocks is brought to 894 01:04:17,719 --> 01:04:23,719 you by Franklin's Net and produced by Pop Studios, a full service audio, 895 01:04:23,880 --> 01:04:28,800 video and post production facility located physically in New London, Connecticut, and of 896 01:04:28,880 --> 01:04:34,000 course in the cloud online at pwop dot com. Visit our website at dt 897 01:04:34,320 --> 01:04:40,800 n et r o cks dot com for RSS feeds, downloads, mobile apps, 898 01:04:40,960 --> 01:04:44,920 comments, and access to the full archives going back to show number one, 899 01:04:45,480 --> 01:04:48,159 recorded in September two thousand and two. And make sure you check out 900 01:04:48,159 --> 01:04:51,840 our sponsors. They keep us in business. Now go write some code, 901 01:04:53,400 --> 01:05:04,840 See you next time. My God let Me is hard than my Texas Red