WEBVTT 1 00:00:01.000 --> 00:00:04.759 How'd you like to listen to dot net Rocks with no ads? Easy? 2 00:00:05.320 --> 00:00:09.400 Become a patron for just five dollars a month. You get access to a 3 00:00:09.480 --> 00:00:14.240 private RSS feed where all the shows have no ads. Twenty dollars a month, 4 00:00:14.240 --> 00:00:18.399 we'll get you that and a special dot net Rocks patron mug. Sign 5 00:00:18.440 --> 00:00:23.440 up now at Patreon dot dot net rocks dot com. Hey Carlin Richard. 6 00:00:23.480 --> 00:00:28.000 Here. As you may have heard, NDC is back offering their incredible in 7 00:00:28.120 --> 00:00:33.960 person conferences around the world. DC Porto is happening October sixteenth through the twentieth. 8 00:00:34.240 --> 00:00:38.280 Go to Eddcporto dot com to register and check out the full lineup of 9 00:00:38.320 --> 00:00:43.600 conferences at NDC Conferences dot com. Hey there, this is Jeff Fritz, 10 00:00:43.840 --> 00:00:48.159 the Purple Blazer guy from Microsoft, letting you in on a little secret about 11 00:00:48.240 --> 00:00:52.920 my friend Carl Franklin. You know the guy who started dot net Rocks, 12 00:00:52.960 --> 00:00:57.719 the first podcast about dot net in two thousand and two, The guy who's 13 00:00:57.719 --> 00:01:03.040 been teaching Blazer on YouTube since twenty Yeah, that Carl Franklin. Well, 14 00:01:03.400 --> 00:01:07.079 Carl's joined up with the folks from Code in a Castle to teach a week 15 00:01:07.159 --> 00:01:12.159 long hands on Blazer class at Are you ready to get this at a castle 16 00:01:12.439 --> 00:01:19.040 slash villa in Tuscany. It's sort of a luxury vacation with Blazer learning built 17 00:01:19.040 --> 00:01:26.879 in. Carl's calling it the Blazer master Class. You'll learn Blazer from the 18 00:01:26.920 --> 00:01:30.920 ground up, finishing the week with the ability to build and deploy Blazer applications. 19 00:01:33.120 --> 00:01:36.920 Since the training happens for only four hours in the morning over six days, 20 00:01:37.280 --> 00:01:41.920 you can bring your significant other your partner with you and you should right 21 00:01:42.400 --> 00:01:46.959 that this part of Italy is absolutely beautiful. There's so much to see and 22 00:01:47.159 --> 00:01:52.560 do and and Larry and Marco from Code in a Castle are organizing daily activities 23 00:01:52.640 --> 00:01:56.840 both at the castle and in the area. The castle is in the Marema, 24 00:01:57.280 --> 00:02:01.760 a less touristed region of Tuscany, offering both classic Tuscan hill country as 25 00:02:01.760 --> 00:02:07.479 well as easy access to the Etruscan Riviera, with sublime local food, wine 26 00:02:07.479 --> 00:02:13.560 and olive oil around every corner. Breakfast is included every day. There will 27 00:02:13.560 --> 00:02:16.919 be two communal dinners at the castle book ending the experience, and most other 28 00:02:17.000 --> 00:02:23.360 meals and all activities are included. And did I mention you'll learn Blazer in 29 00:02:23.479 --> 00:02:28.599 person from Carl Franklin Listen. Space is limited and for very good reason. 30 00:02:28.879 --> 00:02:34.439 This is quality training in a beautiful setting. Go to code in Acastle dot 31 00:02:34.479 --> 00:02:40.680 com slash Blazer twenty twenty three. That's bla z o R two zero two 32 00:02:40.840 --> 00:02:46.199 three to take advantage of this amazing opportunity to join Carl in Tuscany for an 33 00:02:46.280 --> 00:02:53.520 unforgettable week of La dolce vita while advancing your programming skills in this important new 34 00:02:53.599 --> 00:03:09.800 technology. Hey guess what it's time for dot net rocks. I'm Carl Franklin 35 00:03:09.879 --> 00:03:13.840 and I'm Richard Campbell, Brian McKay, my friend, our friend is here 36 00:03:13.879 --> 00:03:17.400 with us today talking about some AI stuff. But first, how you doing? 37 00:03:17.479 --> 00:03:21.000 Man? I am not that well. I mean this stuff you should 38 00:03:21.000 --> 00:03:23.280 talk about on Donna Rocks and stuff you shouldn't. But let's face it, 39 00:03:23.319 --> 00:03:25.240 after twenty something years, you guys know my life. Yeah, my father 40 00:03:25.319 --> 00:03:30.879 passed away this week. Yeah it sucks and it sucks, and but he 41 00:03:30.039 --> 00:03:35.639 was his lungs had failed him. There wasn't anything to be done. He 42 00:03:36.280 --> 00:03:38.479 passed with some dignity. All of us were they able to be there at 43 00:03:38.560 --> 00:03:44.599 least via zoom, including family from New Zealand. So yeah, I'm not 44 00:03:44.599 --> 00:03:46.479 gonna say it was really great because it was really awful, but at least 45 00:03:46.479 --> 00:03:50.199 we could all be together for that moment. Yeah, and it sounds like 46 00:03:50.240 --> 00:03:53.080 he was suffering a bit, so a little bit of relief. He was, 47 00:03:53.240 --> 00:03:57.560 and it was it was well, he died with some dignity and we 48 00:03:57.599 --> 00:03:59.879 could all hope to be so lucky. Well, sorry to hear that. 49 00:04:00.080 --> 00:04:01.879 He was a good man. Sorry to hear that. Buddy, we sounded 50 00:04:01.919 --> 00:04:04.080 so much alike. Let me tell you how much I like we sounded. 51 00:04:04.479 --> 00:04:09.039 I answered his phone more or more than one occasion at his house, and 52 00:04:09.080 --> 00:04:12.039 they would just and whoever was called was just talking to me like I was 53 00:04:12.120 --> 00:04:14.840 him and say no, no, I'm the Sun, and they would literally 54 00:04:14.879 --> 00:04:17.199 not believe me. Wow, It's like yeah, sure, Doug, and 55 00:04:17.240 --> 00:04:25.759 they would continue. He taught you how to do electronics and stuff. Yeah, 56 00:04:25.800 --> 00:04:29.079 yeah, no, he was an electrical engineer. He built electronic cashrowdisterers 57 00:04:29.199 --> 00:04:31.720 and you I think it was one time you had that experience where I took 58 00:04:31.759 --> 00:04:34.199 your seat pat machine and Bart went, oh that's what's wrong with that? 59 00:04:34.240 --> 00:04:39.240 Resolders of parts and here you go. Yeah, well he was having you 60 00:04:39.360 --> 00:04:46.079 resolder solder, see what I'm yeah, yeah, man, resoldering and unsoldering 61 00:04:46.240 --> 00:04:48.800 chips from boards that he was fixing or something he had. You're doing that 62 00:04:48.800 --> 00:04:51.079 when you were like seven or something, right, Yeah, yeah, you're 63 00:04:51.079 --> 00:04:54.959 exactly right. Yeah, had the soldering airand in my hand, my whole 64 00:04:54.959 --> 00:04:58.720 life. Wow. Well, we'll raise a glass to him. Indeed, 65 00:04:58.879 --> 00:05:02.879 cheers, and let's move on now with something a little more cheerful, Better 66 00:05:02.920 --> 00:05:15.079 No Framework, all right, man? When he got well, our friend 67 00:05:15.120 --> 00:05:17.759 Brian McKay gave me this one. He's got so many links to so many 68 00:05:17.839 --> 00:05:23.639 cool projects. This would be not his first contribution to Better No Framework, 69 00:05:23.720 --> 00:05:27.879 I'm correct, No, no, no, he has. I would say 70 00:05:27.920 --> 00:05:32.519 probably seven or eight or nine, ten or eleven or twelve stories thirteen stories 71 00:05:33.879 --> 00:05:41.079 provided by provided by Brian. But anyway, this is Smallville and it's generative 72 00:05:41.240 --> 00:05:45.720 agents for video games, and we just did an AI bought show on agents. 73 00:05:46.000 --> 00:05:53.879 And agents are these things that use gpt APIs and things like that LM 74 00:05:53.959 --> 00:05:58.439 models to do things and then refine them so you can shame them together. 75 00:05:58.600 --> 00:06:00.959 So the first thing might do when you give it a problem is break it 76 00:06:00.959 --> 00:06:05.360 down into parts and then ask itself how to define those parts, and keep 77 00:06:05.360 --> 00:06:10.600 going down until it gets very detailed. For just for one example. But 78 00:06:10.680 --> 00:06:16.600 this is generative agents that are virtual characters that can store memories and dynamically react 79 00:06:16.680 --> 00:06:21.720 to their environment. And so they're able to observe their surrounding, store memories, 80 00:06:21.720 --> 00:06:25.839 and react to state changes in the world. So you basically give them 81 00:06:25.839 --> 00:06:29.399 a personality in your program and you let them free and there's a virtual world 82 00:06:30.000 --> 00:06:32.360 and they go around and they do life. Now I know you're not a 83 00:06:32.480 --> 00:06:36.839 video game guy. Yeah it used to be. Yeah, spend more time 84 00:06:36.920 --> 00:06:43.759 programming, I guess. I mean we talk about like really contemporary games and 85 00:06:44.279 --> 00:06:46.959 the and the animated characters and so forth. There's a game called Assassin's Creed 86 00:06:47.040 --> 00:06:50.279 and there was there was an ancient Greek version of the game, and it's 87 00:06:50.319 --> 00:06:54.879 an interesting there's a whole larger subtext to all of this and so forth. 88 00:06:55.319 --> 00:06:57.959 But I stopped playing the game. I just started hanging out in the world 89 00:06:58.040 --> 00:07:00.920 because the world was that cool, right, So there'd be things like I 90 00:07:00.920 --> 00:07:04.160 remember one time just following around an elderly woman in one of these villages and 91 00:07:04.199 --> 00:07:09.040 she didn't get up in the morning, go down to the market, buy 92 00:07:09.439 --> 00:07:14.079 flower, take it home, make it into bread. Wow, right, 93 00:07:14.120 --> 00:07:16.639 Like that's how the game had such a state when you think about a generative 94 00:07:16.639 --> 00:07:20.879 agent in this equation, the idea that she would remember interacting with me, 95 00:07:21.399 --> 00:07:25.360 that perhaps if I had been aggressive to her in anyway, she'd be afraid 96 00:07:25.360 --> 00:07:27.399 of me, like she'd see me and move away from me. Like that, 97 00:07:27.439 --> 00:07:32.199 You could permanently change these NPC characters, or affect these NBC characters without 98 00:07:32.199 --> 00:07:34.360 them having to read a law of software for it. That that's just be 99 00:07:34.439 --> 00:07:39.560 an intrinsic part of the game, right, that you just let them develop 100 00:07:39.600 --> 00:07:43.800 how they how they will normally exactly right. That that's just it's fascinating to 101 00:07:43.800 --> 00:07:47.480 me in the play experience that you would have an impact in the game. 102 00:07:47.560 --> 00:07:50.199 Like man, you know, often in those kinds of games, you have 103 00:07:50.240 --> 00:07:54.759 an event where like everybody's there, there's a big crowd, and you do 104 00:07:54.839 --> 00:07:58.439 something extraordinary, you know, you behead the king whatever. That may be 105 00:07:58.720 --> 00:08:01.600 the idea that you'd never directly interrupted with that character, but that character had 106 00:08:01.639 --> 00:08:07.800 been there and had seen that thing, and you had affected their behavior towards 107 00:08:07.800 --> 00:08:09.959 you. Yeah, Like, I don't know if games they can do that 108 00:08:11.120 --> 00:08:16.079 right now, Well, Smallville's looks like it's going to be in that camp. 109 00:08:16.439 --> 00:08:18.879 We'll let Brian talk a little bit more about that. Well, I'm 110 00:08:18.879 --> 00:08:22.480 sure, yes, yeah, But first I guess people are talking to us 111 00:08:22.480 --> 00:08:26.279 today, Richard. Sorry, people talk to us most days. Friend. 112 00:08:26.439 --> 00:08:30.519 And this is from show eighteen forty eight, fairly recent. That's the one 113 00:08:30.560 --> 00:08:33.480 we did at Techaram and Antwerp with Jody Birchill and we talked about the no 114 00:08:33.600 --> 00:08:39.000 free lunch and machine waitning and I was really great to talk to her because 115 00:08:39.000 --> 00:08:43.200 she literally is a professional in this space and I think helped us ground a 116 00:08:43.240 --> 00:08:48.759 bit more what's happening with the generative machine learning models in This comment comes from 117 00:08:48.879 --> 00:08:52.080 Lucas, who says, very interesting episode. Maybe you missed an opportunity to 118 00:08:52.120 --> 00:08:58.559 talk about what probably most interesting listeners, which is CHATTYBT and code. Based 119 00:08:58.600 --> 00:09:01.639 on how you described it it works, I still don't get how it's able 120 00:09:01.679 --> 00:09:05.279 to produce reasonable unit tests from a random block and code I've paste into it, 121 00:09:05.639 --> 00:09:09.000 Explain how a chunk of code works, how it could translate code from 122 00:09:09.000 --> 00:09:11.679 one language to another. It seems to be much more than just glowing together 123 00:09:13.159 --> 00:09:16.679 related sentences words of gods from the Internet. And this is my favorite sentence 124 00:09:16.720 --> 00:09:20.679 of the whole thing. The Lucas said, the write a review of prompts 125 00:09:20.720 --> 00:09:24.759 and pseudos philosophical conversation you could have with it, make fun anecdotes, but 126 00:09:24.879 --> 00:09:28.679 I don't think it's the most interesting part for developers. Yeah, and I 127 00:09:28.720 --> 00:09:31.440 really appreciate that, Lucas. I mean, one of the reasons we didn't 128 00:09:31.440 --> 00:09:35.279 focus on code with Jody is that she was a machine learning professional, and 129 00:09:35.279 --> 00:09:39.480 I really wanted to talk more broadly about what was going on with these technologies 130 00:09:39.759 --> 00:09:45.440 and her concerns around it because she was a professional. It is interesting obviously 131 00:09:45.519 --> 00:09:48.679 you're referencing get a co Pilot more than anything, but chat GYBT it applies 132 00:09:48.679 --> 00:09:52.600 with us as well, which is that And I think Jody talked a bit 133 00:09:52.639 --> 00:09:56.799 about this. The tokenization of language is an important part of the power of 134 00:09:56.919 --> 00:10:00.960 large language models because it also creates a sense of bide to actionality that it's 135 00:10:01.720 --> 00:10:05.799 not only does it know from words what code you might want because of those 136 00:10:05.840 --> 00:10:09.279 sentence relationships, but it can work the other way that when presented code, 137 00:10:09.360 --> 00:10:13.279 it knows what language to produce for you to describe that code. It's not 138 00:10:13.399 --> 00:10:16.519 always right, according to you know, get hub compile itself. They're still 139 00:10:16.559 --> 00:10:22.720 batting less than fifty percent of compilable code on the initial prompt. But it 140 00:10:22.799 --> 00:10:26.360 is interesting to see that consistently that number goes up, although not by very 141 00:10:26.480 --> 00:10:33.759 much with multiple refined prompts, so it still has a way to go. 142 00:10:33.879 --> 00:10:35.919 But there are plenty of shows, believe me, where we're going to talk 143 00:10:35.960 --> 00:10:39.919 about large language models and code. So yeah, this is only the beginning 144 00:10:41.600 --> 00:10:46.080 and wherever it goes from there, I mean large language plus plus I imagine. 145 00:10:46.159 --> 00:10:50.679 And they now have plugins for accessing the Internet, as I learned from 146 00:10:50.679 --> 00:10:52.159 Brian. Well you know what, I'm just gonna we'll get there, introduced 147 00:10:52.159 --> 00:10:54.919 Brian and let him talk to about all that stuff. Okay, yeah, 148 00:10:56.000 --> 00:10:58.120 hey, Lucas, thank you so much for your comment. And a copy 149 00:10:58.159 --> 00:11:00.039 of us to cod Buy is on its way to un If you'd like a 150 00:11:00.120 --> 00:11:01.799 copy of Music Code By, I write a comment on the website at Dona 151 00:11:01.960 --> 00:11:05.320 Rocks dot com or on the facebooks. We publish every show there, and 152 00:11:05.360 --> 00:11:07.000 if you comment there when you're reading the show, we'll send your copy mused 153 00:11:07.000 --> 00:11:09.720 to go by and you can follow us on Twitter or x or whatever the 154 00:11:09.720 --> 00:11:13.960 hell they're calling it these days. But the real fun is over on Mastodon. 155 00:11:13.480 --> 00:11:18.720 I'm at Carl Franklin at tech Hub dot Social, and I'm Rich Campbell 156 00:11:18.799 --> 00:11:22.840 at Masadondo Social Sensitude. We'd like to hear from you, of course over 157 00:11:22.879 --> 00:11:26.000 there and share our stories and all that stuff. Awesome. So let's introduce 158 00:11:26.039 --> 00:11:31.480 Brian. Brian McKay is the co host of The AI Bought Show alongside myself, 159 00:11:31.879 --> 00:11:35.639 and also serves as the CTO of Roster, a company dedicated to transforming 160 00:11:35.759 --> 00:11:43.600 leaders using an innovative three sixty feedback process which sounds perfectly obfuscated to me. 161 00:11:43.000 --> 00:11:48.559 A season software engineer, entrepreneur, and open source contributor, Brian has been 162 00:11:48.600 --> 00:11:52.360 at the helm of product development and startups for over twenty years. He's a 163 00:11:52.399 --> 00:11:56.159 father, husband, musician, writer, chest NERD and a decent kickboxer, 164 00:11:56.559 --> 00:12:00.440 so don't mess with him. Welcome Brian. That's pretty to be here. 165 00:12:01.039 --> 00:12:03.399 Yeah, man, it was so hard enough to cut in during that intro. 166 00:12:03.440 --> 00:12:07.679 There's so much I want to say about Smallville and co Pilot and yeah, 167 00:12:07.679 --> 00:12:09.039 well where do you want to start? Well, you go, man, 168 00:12:09.120 --> 00:12:13.240 it's your show. Let's let's do it. Oh well, yeah, 169 00:12:13.399 --> 00:12:16.399 let's start with the details of Smallville. Okay, well, yeah, so 170 00:12:16.480 --> 00:12:20.960 Smallville came out and I want to say, was that August alongside all the 171 00:12:22.000 --> 00:12:28.960 other generative agents like baby Agi and autogpt. And the interesting thing is that 172 00:12:28.000 --> 00:12:31.840 they just open sourced it in the last couple of weeks, so now you 173 00:12:31.840 --> 00:12:35.480 can go into there and and change it, so you know, Smallville. 174 00:12:35.679 --> 00:12:37.600 The most interesting thing with these little bots, I think it's a community of 175 00:12:37.600 --> 00:12:41.480 like twenty five bots. Each one is a little prompt that defines its personality, 176 00:12:41.720 --> 00:12:45.919 and they've got a little algorithm that kind of lets them learn from the 177 00:12:45.919 --> 00:12:52.919 conversations they have together these bots, watching their behavior is really interesting. And 178 00:12:52.039 --> 00:12:56.000 one of the most interesting things that happened is one of them decided to plan 179 00:12:56.039 --> 00:13:01.720 a Valentine's party, and it propagated the information about this party to like eighteen 180 00:13:01.799 --> 00:13:05.759 of the twenty five bots, and they all made decisions about how to handle 181 00:13:05.799 --> 00:13:09.919 it. Some decided not to go somewhere, maybe snubbed a little bit. 182 00:13:09.080 --> 00:13:13.720 A bunch decided to go. There was an actual party, and the emergence 183 00:13:13.840 --> 00:13:18.879 of that type of behavior is fascinating and there's so much more that can happen 184 00:13:18.000 --> 00:13:20.320 in that space. As this gets murder, I mean, I just have 185 00:13:20.360 --> 00:13:24.639 a tough time with the whole agency decided to throw a Valentine's party. Yeah, 186 00:13:24.720 --> 00:13:30.840 Richard is hung up on the anthropomorphization of Ai Boughts, Oh, without 187 00:13:30.840 --> 00:13:37.039 a doubt. But the question is what was the software stimulus that propagated that 188 00:13:37.120 --> 00:13:39.879 process? Right? Right, token prediction just like everything else, you know, 189 00:13:39.960 --> 00:13:43.240 like and uh, you know primed by prompts that there might have been 190 00:13:43.240 --> 00:13:46.639 a prompt that's said that you want to plan a Valentine's party. Well, 191 00:13:46.720 --> 00:13:50.360 might not have emergent would hope they'd be more macro than that and say there 192 00:13:50.399 --> 00:13:54.240 are major events on the calendar and occasionally you should have a party for them. 193 00:13:54.240 --> 00:13:56.320 They might have and I mean even go to that weight of I mean, 194 00:13:56.519 --> 00:14:01.799 I would think that if you're mulating human behavior, you only want to 195 00:14:01.799 --> 00:14:05.879 throw a Valentine's party because you're in a relationship where you want to have other 196 00:14:05.879 --> 00:14:07.360 folks around, or you're not in a relationship you want to make it a 197 00:14:07.440 --> 00:14:11.039 singles thing like, well, you can go deep. This question is how 198 00:14:11.120 --> 00:14:16.639 much of that has to be crafted rights, Well, my experience with this 199 00:14:16.679 --> 00:14:20.720 tech is that one marea where it's very strong, is just brainstorming things like 200 00:14:20.879 --> 00:14:24.000 a Valentine's party. I mean, that does seem like something that could emerge 201 00:14:24.039 --> 00:14:28.519 you quorkanically quite easily. Well, and I like the brainstorming angle of it 202 00:14:28.559 --> 00:14:33.919 because it is just sort of a word salad of ideas that we then can 203 00:14:33.000 --> 00:14:39.000 sift through as humans with our somewhat more sophisticated minds and take value from. 204 00:14:39.279 --> 00:14:43.600 Yeah, right, anything that makes me happy on large language models, it's 205 00:14:43.600 --> 00:14:46.440 fill my blank screen with stuff that might be useful, because I'd rather criticize 206 00:14:46.480 --> 00:14:50.000 than create, right. Yeah. One of the things that Brian does really 207 00:14:50.000 --> 00:14:54.600 well in the AI Bot show is tell it. You know, give me 208 00:14:54.720 --> 00:15:01.120 we're building a board game, right it, Give me ten ideas for cards 209 00:15:01.200 --> 00:15:03.039 that we can play on this game, you know, after we've got the 210 00:15:03.120 --> 00:15:07.759 gold and all that stuff. But you don't say just create something. You 211 00:15:07.799 --> 00:15:11.879 say, give me ten twenty ideas, and you pick the one that you 212 00:15:11.919 --> 00:15:13.840 like, and you go with that and you narrow it down. But you're 213 00:15:13.919 --> 00:15:18.519 kind of being like an agent in that sense, aren't you, because you're 214 00:15:18.559 --> 00:15:24.879 you're basically starting with a question, taking the results, picking one, and 215 00:15:24.919 --> 00:15:28.879 then diving deeper into it. Right, Well, we'll tell you why this. 216 00:15:28.559 --> 00:15:31.200 Uh, this leads right into what I wanted to talk about. I 217 00:15:31.279 --> 00:15:35.720 kind of want this conversation to be about maybe what this is like as a 218 00:15:35.720 --> 00:15:39.320 developer to use and what the strengths and weaknesses are, and maybe the place 219 00:15:39.399 --> 00:15:45.480 to start that will explain the technology is just talking about the weaknesses of it, 220 00:15:45.519 --> 00:15:48.559 Like where this is going is just a dead end? What are the 221 00:15:48.600 --> 00:15:52.240 bad parts of this tech? And I think that will cover everything that we're 222 00:15:52.279 --> 00:15:56.919 talking about right now. Cool. Oh, I mean right off the bat, 223 00:15:56.919 --> 00:16:00.960 it's like, listen the bubble forming in the VC community around this lay 224 00:16:02.000 --> 00:16:08.600 that clearly in the bad part of this uses talk about incentivizing grift incentivizing fictions. 225 00:16:10.039 --> 00:16:12.279 Yeah, it's not good. Yeah, I'm trying to I'm trying to 226 00:16:12.320 --> 00:16:18.279 coin the phrase griftware for that that thing that that thing that emerged during I 227 00:16:18.320 --> 00:16:22.080 think it started maybe during crypto, maybe before. But these people learned how 228 00:16:22.120 --> 00:16:29.159 to these people learned how to descend on hype and and just con people out 229 00:16:29.159 --> 00:16:32.080 of their money over and over and over again. And some of that's present 230 00:16:32.159 --> 00:16:34.440 in this world too. It's not as bad because it's not as easy to 231 00:16:34.559 --> 00:16:37.159 just you know, it's not like just getting someone to buy some crypto. 232 00:16:37.679 --> 00:16:41.279 But there are people, there are folks trying to get you to buy products 233 00:16:41.279 --> 00:16:44.799 that are just very flimsy rappers on top of API calls. Yeah, and 234 00:16:44.879 --> 00:16:49.279 the word AI itself or the term being misrepresented as you know, what's the 235 00:16:49.279 --> 00:16:53.240 difference between a well crafted algorithm and AI? Right? I mean it's that's 236 00:16:53.320 --> 00:16:56.799 right. Yep. People, I've said, I don't know how many times 237 00:16:56.799 --> 00:17:00.399 I've said this like AI to me, it's just that true, which it 238 00:17:00.440 --> 00:17:03.320 says, Okay, you're making stuff up. Yeah, as near as I 239 00:17:03.320 --> 00:17:07.640 can tell, AI is the term you use when stuff doesn't work. As 240 00:17:07.680 --> 00:17:11.880 soon as it does work, it hasn't. It's no longer as exactly it's 241 00:17:11.000 --> 00:17:15.240 large language modes or anything like that. So it's like it's just automatic red 242 00:17:15.240 --> 00:17:18.799 flag scrutinized clothes. You know, you're a problem. Yeah. Yeah, 243 00:17:18.880 --> 00:17:22.680 we will keep raising the bar until one day we build something that says, 244 00:17:22.720 --> 00:17:26.960 wait a minute, I'm alive and tries to and actually convinces us and makes 245 00:17:27.000 --> 00:17:30.599 us believe. I don't think we will accept Well, I don't think we'll 246 00:17:30.640 --> 00:17:33.480 have any problem having it convince us it's alive, because clearly there are people 247 00:17:33.559 --> 00:17:37.799 that think that already there are well, you know, we want to anthropomorphize 248 00:17:37.839 --> 00:17:41.000 things so much. I was watching the pilot of Community the other day. 249 00:17:41.200 --> 00:17:45.599 And there's the scene where Joel McHale holds up a pencil and says something, 250 00:17:45.599 --> 00:17:48.519 I'm going to butcher this, but he says something like, this pencil is 251 00:17:48.559 --> 00:17:52.319 Fred. Fred's got a wife and two kids. Snap, and everyone in 252 00:17:52.319 --> 00:17:56.680 the room goes, oh, you know, that's all it takes, you 253 00:17:56.680 --> 00:18:00.400 know, like we want to see humanity and pencils with just a little story. 254 00:18:00.440 --> 00:18:03.359 We're wired that way to synchronize. And yeah, and also it's easy 255 00:18:03.440 --> 00:18:07.119 for us to describe things in terms of anthropomorphizing. We've been doing it for 256 00:18:07.200 --> 00:18:11.839 code. Oh well, my guy over here says, hey, let me 257 00:18:11.920 --> 00:18:15.559 know whenever this happens. And then this guy says, okay, here you 258 00:18:15.640 --> 00:18:18.960 go. Right, when we're describing code to each other, we kind of 259 00:18:19.000 --> 00:18:23.279 talk like that. And you naturally did that when you're talking about these these 260 00:18:23.319 --> 00:18:27.039 agents too, because it's just such a every it's a framework for understanding that 261 00:18:27.119 --> 00:18:32.160 everybody gets. We just have to remember that it's not for you. Well, 262 00:18:32.200 --> 00:18:36.400 and part I mean that's the problem with people make assumptions around it and 263 00:18:36.440 --> 00:18:38.839 they project a lot more capability on it than it actually has. Right, 264 00:18:40.160 --> 00:18:44.759 So, back in like twenty so, I've been following this pretty closely since 265 00:18:45.160 --> 00:18:48.359 GPT two and like twenty nineteen, started using a little more seriously in twenty 266 00:18:48.359 --> 00:18:52.400 twenty when GPT three came out and I started running into researchers like AI, 267 00:18:52.480 --> 00:18:56.839 researchers who are smarter than me. And one thing I noticed is that they 268 00:18:56.880 --> 00:19:00.200 were really dismissive of this tech and I saw a lot of promise it. 269 00:19:00.839 --> 00:19:07.319 But the reasons why they're dismissive still have some relevance. And you know, 270 00:19:07.480 --> 00:19:11.440 I think the thing with so they're concerned about AGI is, first of all, 271 00:19:11.559 --> 00:19:15.279 is the thing they want a path where technology can be sentient. Define 272 00:19:15.359 --> 00:19:22.480 that acronym general intelligence artificial general intelligence. Yeah yeah, thank you, specialized 273 00:19:22.519 --> 00:19:27.920 intelligence. Right, so it just means natural stupidity and as got it. 274 00:19:29.480 --> 00:19:32.720 We just we just solve that one right there. So so so you know, 275 00:19:32.880 --> 00:19:36.759 these a lot of these folks are less concerned with just making something that 276 00:19:36.799 --> 00:19:41.440 has some utility and more interested in making something that is alive. Like that's 277 00:19:41.480 --> 00:19:44.440 the dream to make something that's kind of human level. And you hit a 278 00:19:44.440 --> 00:19:48.079 great line here, Brian. So that's a different science and engineering. All 279 00:19:48.119 --> 00:19:51.720 of us at our roots are really engineers, and so we're looking at tools 280 00:19:51.720 --> 00:19:56.240 and say what can I do with these tools? Where the scientists are farm 281 00:19:56.279 --> 00:20:00.160 you know, implementation is a detail they're farm ore iNeST in the broader science. 282 00:20:00.720 --> 00:20:06.079 It's of you know, recognizing the limitations of LMS and sertic. Okay, 283 00:20:06.079 --> 00:20:08.160 well that's not this path of this dream I have. So next, 284 00:20:08.880 --> 00:20:12.599 Yeah, and I and I suppose that I am actually like much more of 285 00:20:12.599 --> 00:20:15.799 a language guy, you know, like I went to school a little bit 286 00:20:15.799 --> 00:20:18.279 for English, and I like to write. So it's a different people connect 287 00:20:18.279 --> 00:20:23.319 to it differently. So what I I think the thing is that we imagine 288 00:20:23.400 --> 00:20:29.279 intelligence should feel in some way organic. We want to nurture a spark and 289 00:20:29.319 --> 00:20:33.319 watch it internalize moral lessons and reason with agency and grow and wisdom or sapiens. 290 00:20:33.839 --> 00:20:37.279 And this is autocomplete, you know, like when we've done here, 291 00:20:37.519 --> 00:20:40.759 the only game in town, the only game in town is we've trained a 292 00:20:40.839 --> 00:20:45.160 really sophisticated neural network on everything that we could, you know, get into 293 00:20:45.200 --> 00:20:48.759 it. And now it completes. It chooses the next token, the next 294 00:20:48.799 --> 00:20:52.200 most probable token that should appear. I give you an example when autocomplete is 295 00:20:52.240 --> 00:20:59.319 too slow. Hey, honey, have you seen the the thing, the 296 00:20:59.400 --> 00:21:03.839 red thing, the red scrapy thing. You're not. Come on, you 297 00:21:03.880 --> 00:21:10.640 know what I'm talking about. So so it's not a beautiful model of intelligence. 298 00:21:10.839 --> 00:21:15.119 It's you know, I don't think intuitively a human wants autocomplete to be, 299 00:21:15.720 --> 00:21:18.440 to be the AGI that we come up with. There's just better ideas, 300 00:21:18.720 --> 00:21:22.759 and there still are. They still tell me there are better ideas and 301 00:21:22.920 --> 00:21:27.599 out there that will supplant this so inevitably. But that's kind of normal. 302 00:21:27.720 --> 00:21:32.400 The real thing here is, I don't think anybody, I think only the 303 00:21:32.440 --> 00:21:37.000 scientists really want an AGI in the first place. Like that's it's science fiction 304 00:21:37.039 --> 00:21:40.799 for crying out loud, right, Yeah, there are so many more interesting 305 00:21:40.799 --> 00:21:45.119 things you just go work on. Well, there's a fascination with the idea 306 00:21:45.160 --> 00:21:49.400 of AGI. There's a it's weird because AGI is maybe not great for us, 307 00:21:51.160 --> 00:21:55.079 but as of species, we seem to be inexorably drawn to it, 308 00:21:55.160 --> 00:21:56.480 like a moth to the flame. We can't stop. We're going to do 309 00:21:56.519 --> 00:22:02.079 it. I find that interesting, Like it's fascinating up and at the same 310 00:22:02.119 --> 00:22:06.240 time, humans are remarkably resistant to calling anything else on this planet sentient, 311 00:22:06.319 --> 00:22:10.079 even though this plant there's significant evidence to show there is you know, if 312 00:22:10.079 --> 00:22:14.359 we really cared about intelligent life. Why do we treat citaceans the way we 313 00:22:14.440 --> 00:22:18.599 do? And you know and so on. Yeah, dolphins are supposedly really 314 00:22:18.599 --> 00:22:22.559 smart. Pigs can recognize themselves and mirrors and we eat them. Yeah, 315 00:22:22.640 --> 00:22:25.920 yeah, I don't know that dolphin. That's been gross, But but I 316 00:22:25.920 --> 00:22:30.039 meant a point, or even how we've treated great the great apes too, 317 00:22:30.279 --> 00:22:33.240 right, Like, that's true. And the problem is that as soon as 318 00:22:33.240 --> 00:22:36.720 you start getting serious about defining sentiency in any way, a whole bunch of 319 00:22:36.799 --> 00:22:41.400 other creatures we've abused on this planet qualify. That's now you've got a problem. 320 00:22:41.319 --> 00:22:45.480 Yeah, that's true. And future generations will probably judge us for these 321 00:22:45.480 --> 00:22:51.599 things, just like we judge past generations for their the institutions that they lived 322 00:22:51.599 --> 00:22:59.480 in. So the problems are their static, meaning, once they're trained, 323 00:22:59.480 --> 00:23:03.519 they don't really learn. They have a token, they have a context of 324 00:23:03.680 --> 00:23:07.480 you know, eight thousand, eight thousand tokens that you can play with, 325 00:23:07.799 --> 00:23:11.559 but they're not really learning as you go. That's very limited space. You 326 00:23:11.559 --> 00:23:14.279 can do some tricks with it. Should we define what a token is in 327 00:23:14.279 --> 00:23:18.480 this context? Yeah? The easiest way, it's it's easiest to think of 328 00:23:18.480 --> 00:23:22.200 it as like a few words. You know, you have this window of 329 00:23:23.079 --> 00:23:30.359 with GPT four like twenty thousand or so words that you can feed into it 330 00:23:30.480 --> 00:23:34.359 as as your conversation, you know, like that's why chat GPT can can 331 00:23:34.599 --> 00:23:38.480 understand what you're talking about and remember what you just said. But as time 332 00:23:38.519 --> 00:23:42.599 goes by, things will fall off the end and it will forget the things 333 00:23:42.599 --> 00:23:45.720 that happen at the start of the conversation. You're just pushing things through that 334 00:23:47.319 --> 00:23:49.200 eight thousand token limit. I haven't done this in a while, but one 335 00:23:49.200 --> 00:23:53.119 of the in the earlier versions of this, I use the iamic pentameter trick 336 00:23:53.200 --> 00:23:56.799 where I told I set up front, I need you to only respond to 337 00:23:56.799 --> 00:23:59.799 me an iamic pendameter, okay, and then we'd keep going back and forth 338 00:24:00.079 --> 00:24:03.960 till the cash overflowed and suddenly would stop. Like it was the easiest way 339 00:24:03.000 --> 00:24:06.359 to say, hair, you just hit the cash limit. And for those 340 00:24:06.400 --> 00:24:12.759 who don't know, I amke pantameter. Sounds like this. It's almost like 341 00:24:12.839 --> 00:24:18.319 the two lines of a limerick. Yeah. So practically, one place where 342 00:24:18.319 --> 00:24:22.480 this comes up is it's really easy to make a bot that generates sequel statements. 343 00:24:23.200 --> 00:24:26.920 So I'm working on a new project that I made a bot purpose built 344 00:24:27.039 --> 00:24:30.640 for It understands what the project is and you can tell it to make tables. 345 00:24:30.640 --> 00:24:33.640 It knows how I like my things, capitalize the naming conventions, everything 346 00:24:33.640 --> 00:24:40.519 about it. But create statements do take up space, and you know, 347 00:24:40.599 --> 00:24:45.799 a database of some significant size in terms of tables will just push the context 348 00:24:45.920 --> 00:24:48.680 off the limit. It will forget where your user table was because it's not 349 00:24:48.720 --> 00:24:52.400 in there anymore. So we're not we're not. You know, the context 350 00:24:52.480 --> 00:24:55.839 size will improve over time, and it has improved. They're working on a 351 00:24:55.880 --> 00:25:02.279 thirty two K token model, but my understanding is that tokens are quadratic, 352 00:25:03.039 --> 00:25:07.000 so going from eight K to thirty two K is really computationally expensive, which 353 00:25:07.039 --> 00:25:11.759 actually brings us to the second problem. This technology is very expensive. It's 354 00:25:11.839 --> 00:25:19.359 computationally There was some leaked some leaked documents months ago, I think in February 355 00:25:19.799 --> 00:25:26.400 that showed this product that Microsoft is planning on launching called I think it's called 356 00:25:26.400 --> 00:25:32.559 Foundry, and basically it's you can host your own model of GPT four in 357 00:25:33.480 --> 00:25:37.119 Azure. And there's a cheap version that costs a quarter of a million dollars 358 00:25:37.160 --> 00:25:41.880 a year for like three chat like a three point five turbo model that just 359 00:25:41.920 --> 00:25:45.880 gives you a glimpse at how expensive it is. In fact, there's a 360 00:25:47.000 --> 00:25:49.160 rumor this week. I don't know if the sources really check out, but 361 00:25:49.960 --> 00:25:56.000 they say that chat GPT by itself is burning like seven hundred thousand dollars a 362 00:25:56.079 --> 00:25:59.920 day. I believe it, which I don't think is necessarily a huge problem 363 00:26:00.039 --> 00:26:02.640 because they have over a billion users, you know, like if you just 364 00:26:02.680 --> 00:26:04.359 get a nickel from Yeah, as long as those billion users are paying, 365 00:26:04.599 --> 00:26:07.640 yeah, I would love to know the percentage that are actually paying twenty bucks 366 00:26:07.680 --> 00:26:11.920 a month for Chat gypt pro. I'm one of them. Yeah, I'm 367 00:26:11.920 --> 00:26:15.039 one two. It might just be us though, But even if it was 368 00:26:15.119 --> 00:26:18.400 one percent, that's ten million users a twenty dollars. It's two hundred million 369 00:26:18.920 --> 00:26:23.240 a month. Yeah, that's close. You know, seven hundred thousand a 370 00:26:23.319 --> 00:26:26.920 day is like twenty one million dollars, So you're getting there. I've got 371 00:26:26.960 --> 00:26:30.559 a feeling it's more than one percent. I would argue it's less than one 372 00:26:30.599 --> 00:26:33.480 percent. Actually you really think so, absolutely, But we're just guessing. 373 00:26:34.039 --> 00:26:37.799 Yeah, yeah, well we don't know. But I feel like if you 374 00:26:37.880 --> 00:26:40.640 had access to a you know, a couple of billion eyeballs. I can 375 00:26:40.680 --> 00:26:44.000 find a way to make it, make it work, but in Microsoft will 376 00:26:44.000 --> 00:26:45.680 too, and also they'll find ways to make it cheaper. I think you've 377 00:26:45.680 --> 00:26:51.640 found this drag race. Now that that you know we have the specs for 378 00:26:51.759 --> 00:26:55.519 how Microsoft hosted GPT three, the two hundred and eighty five thousand processors. 379 00:26:56.960 --> 00:27:00.599 We know that the models roughly seven times larger, so you can kind of 380 00:27:00.599 --> 00:27:04.240 project and that makes it one of the largest shipper computers in the world. 381 00:27:04.960 --> 00:27:08.440 And that that I mean, irrespective of what it actually costs to build that 382 00:27:08.480 --> 00:27:14.119 out gredit they already owned it. That is a bunch of Azure resources could 383 00:27:14.119 --> 00:27:18.359 be making money on something else, right, and yet is a sign now 384 00:27:18.519 --> 00:27:22.000 I mean open as paying for those, but the paying for those with funny 385 00:27:22.000 --> 00:27:26.920 money. Right. Microsoft gave them ten billion dollars in Azure credits to officially 386 00:27:26.960 --> 00:27:32.039 give it back to them, right, right, And it gives us a 387 00:27:32.039 --> 00:27:36.119 time limit, you know, and start start up parlance. This is your 388 00:27:36.319 --> 00:27:41.279 ramp to run with as much time to get enough revenue to extend your ramp 389 00:27:41.319 --> 00:27:45.960 to keep going, right. Yeah, that's right. And they can always 390 00:27:45.279 --> 00:27:48.480 you know, you you can always turn off half of those half of those 391 00:27:48.559 --> 00:27:52.359 unpaid users and probably catch your costs quite a bit. So I think there 392 00:27:52.400 --> 00:27:56.599 are strategies. Yeah, well it that's the question that if you're an API 393 00:27:56.720 --> 00:27:59.519 user and we haven't even talked about the API yet, but if you're a 394 00:27:59.599 --> 00:28:04.160 gpt API user and you're making calls and you're actually selling a product that uses 395 00:28:04.240 --> 00:28:11.759 it, you know you took a dependency here that may or may not change 396 00:28:11.000 --> 00:28:14.920 or completely go away in the future. That's true, but they are not 397 00:28:14.960 --> 00:28:18.680 the only game in town either. With the rise of a couple of open 398 00:28:18.680 --> 00:28:22.599 source models in the last month or so, you know, Lama two is 399 00:28:22.599 --> 00:28:26.119 out stable. The Fusions also got a couple of open source models. You 400 00:28:26.119 --> 00:28:30.160 can actually just host that yourself if you have the hardware, if you have 401 00:28:30.640 --> 00:28:33.920 two and eighty thousand processors or whatever. Well, the crazy thing is I 402 00:28:34.000 --> 00:28:37.519 saw a tweet a tweet. Is it still called the tweet? I don't 403 00:28:37.559 --> 00:28:38.480 know what you call them now? On X I don't know what you're calling. 404 00:28:41.119 --> 00:28:44.519 Well, whatever it is, I saw people are talking about running the 405 00:28:44.559 --> 00:28:52.200 small version of it, like the seven billion parameter version on laptops because I 406 00:28:52.240 --> 00:28:56.200 guess the thing is that it's memory bound or something like that. I need 407 00:28:56.200 --> 00:28:59.319 to read the quote a tweet more closely. But when you're doing it, 408 00:28:59.319 --> 00:29:03.039 when you only care about a single request, you can you can do a 409 00:29:03.079 --> 00:29:06.680 lot more with a with less. But when you care about running at scale, 410 00:29:06.720 --> 00:29:10.160 you really do need serious technology. And like you know, and video 411 00:29:10.240 --> 00:29:14.000 ships that cards the cost fifteen thousand dollars each. Interesting. And I'm talking 412 00:29:14.039 --> 00:29:17.119 to folks who are who are trying to build software around this, and they're 413 00:29:17.119 --> 00:29:19.799 all about GPT four and I'm like, why GPT four, why not GPD 414 00:29:19.920 --> 00:29:25.279 three? Like and and really it's like because four is larger than three more 415 00:29:25.680 --> 00:29:30.319 Like they don't really know, they haven't actually tested the software with the smaller 416 00:29:30.319 --> 00:29:33.640 model, is that this is sufficient? And when I talk to Microsoft engineers, 417 00:29:33.640 --> 00:29:37.400 like, they're pitching three and three five pretty hard these days. And 418 00:29:37.400 --> 00:29:41.759 I think one of the issues is that the four is so large that it's 419 00:29:41.839 --> 00:29:45.119 going to be hard for it to make it profitable and maybe you don't need 420 00:29:45.160 --> 00:29:49.400 it. Didn't we learn Brian that three five has a model a mode where 421 00:29:49.440 --> 00:29:55.119 there's more tokens available to it than what's currently available for Yeah, they're working 422 00:29:55.160 --> 00:29:59.160 on on greatly increasing the context. It is much more expensive. So when 423 00:29:59.519 --> 00:30:02.799 so one thing we've seen is the price reduced over time, over and over 424 00:30:02.839 --> 00:30:07.160 again with these models. So when when around the time that GPT four came 425 00:30:07.160 --> 00:30:11.200 out, they launched GPT three point five Turbo and they cut the price by 426 00:30:11.279 --> 00:30:12.599 like I want to say, it was like ninety percent. I mean, 427 00:30:12.599 --> 00:30:15.839 it's so cheap compared to what it was. So now the question is is 428 00:30:15.839 --> 00:30:19.240 that based on cost or is that based on a marketing effort to move customers? 429 00:30:21.440 --> 00:30:23.480 That's a who yeah, like a lost leader type of thing. I 430 00:30:23.599 --> 00:30:27.880 don't know. I assume it must be cost. Well, hey, if 431 00:30:27.960 --> 00:30:32.119 I know I can't get to the price that I'm going to need you for 432 00:30:32.160 --> 00:30:36.119 GPT four, but I don't want to have my customers abandoned me, I 433 00:30:36.240 --> 00:30:38.079 give you a discount on the product I think I can make a profit on 434 00:30:38.119 --> 00:30:41.759 because once it works over there, then I can take the price back up. 435 00:30:41.799 --> 00:30:45.839 That may be they have you know, access to cheap money, or 436 00:30:45.880 --> 00:30:48.359 they have had it at least so yeah, but that money eventually runs out, 437 00:30:48.680 --> 00:30:52.759 so don't usually run out. Yes, I just get a real sense 438 00:30:52.079 --> 00:30:56.440 as we come off the top of this hype cycle that the bean counters are 439 00:30:56.480 --> 00:31:00.079 grating a hold and say, is there a revenue stream here that comes close 440 00:31:00.640 --> 00:31:03.799 to covering the cost of equipment, because if we can get to break even 441 00:31:03.839 --> 00:31:07.640 in the current configuration, we'll start making real money on the back end as 442 00:31:07.960 --> 00:31:12.599 the TikTok of Moore's law goes a little bit further and the cost to operate 443 00:31:12.640 --> 00:31:18.000 this goes down. But I think we're in a dead race this year to 444 00:31:18.000 --> 00:31:21.920 try and get numbers. That makes sense. You should use the TikTok my 445 00:31:22.000 --> 00:31:27.799 friend Intel used at first. I did not invent that. Well, it's 446 00:31:27.839 --> 00:31:30.960 interesting. I wish I had more insight into what's going on behind the scenes. 447 00:31:32.839 --> 00:31:34.599 With the open source models. You can get a sense for what it 448 00:31:34.599 --> 00:31:40.440 costs to operate them, and they are similar in power to GBT four, 449 00:31:40.559 --> 00:31:44.519 So that's all very interesting. It is expensive. It's not something that you 450 00:31:44.559 --> 00:31:48.519 know, anybody can just throw together, right, But he's still you're talking 451 00:31:48.519 --> 00:31:52.599 about the prices at the beginning of a cycle where they're trying to solict the 452 00:31:52.640 --> 00:31:57.599 customers as quickly as possible so that they're almost certainly discount prices. Yeah, 453 00:31:57.640 --> 00:32:00.720 that's right. Yeah, that's every point that had not occurred to me, 454 00:32:00.759 --> 00:32:04.960 And that's a smart thought. Hold that thought right there, Brian, While 455 00:32:05.000 --> 00:32:12.839 we take a moment for these very important messages, and we're back. You're 456 00:32:12.880 --> 00:32:15.240 listening to Dot and Rocks. I'm Carl Franklin, that's Richard Campbell, and 457 00:32:15.319 --> 00:32:21.680 that's our friend Brian McKay. We're talking AI and GPT and all them things. 458 00:32:22.119 --> 00:32:23.279 And you were about to make a point before we went to the break. 459 00:32:23.359 --> 00:32:27.200 Yeah. So we covered a lot of the reasons why this technology, 460 00:32:28.000 --> 00:32:30.240 all the problems with it, or most of the problems with it. We 461 00:32:30.240 --> 00:32:32.559 didn't talk about hallucination. Actually we should. We should at least just mentioned 462 00:32:32.599 --> 00:32:38.839 that we should. Yeah. Yeah, this the caustic period of just says 463 00:32:38.920 --> 00:32:43.240 random things sometimes and not totally random. It's not random, it's the most 464 00:32:43.559 --> 00:32:49.319 random things, right, it'll just say things. Sometimes we call it creativity. 465 00:32:50.160 --> 00:32:53.160 Sometimes we call it, you know, chaos. Yeah, I would 466 00:32:53.160 --> 00:32:57.240 almost call it pomposity, because you know, it's like, you know, 467 00:32:57.359 --> 00:33:00.440 people when they know a lot of things and they expect to have all the 468 00:33:00.440 --> 00:33:02.160 answers, and then when they don't, they just make something up because if 469 00:33:02.200 --> 00:33:07.200 it sounds good, Yeah, I'll get credit for it anyway. The most 470 00:33:07.279 --> 00:33:14.240 dangerous thing is when when when you when you're working on something really formal with 471 00:33:14.240 --> 00:33:15.839 with very formal language, like, for instance, a white paper, like 472 00:33:15.880 --> 00:33:22.359 a scientific paper. It will lie in the most convincing legit way, which 473 00:33:22.400 --> 00:33:28.279 is actually really dangerous because the use of that type of language will fool scientists. 474 00:33:28.759 --> 00:33:32.720 Yeah, now you just have to have that reflex to fact check everything 475 00:33:32.759 --> 00:33:37.119 it spits out. That's right. The guardrails have gotten better, right, 476 00:33:37.240 --> 00:33:39.799 like you can. You can't ask it to make bombs and stuff anymore. 477 00:33:39.960 --> 00:33:43.440 Right. Well, actually, you know, it's really funny you mentioned that 478 00:33:43.960 --> 00:33:52.240 this Sunday, a def con session happened where twenty two hundred hackers I think 479 00:33:52.240 --> 00:33:57.359 the White House actually asked them to do this. They basically like worked on 480 00:33:57.440 --> 00:34:02.359 jail breaking, the top language models and chat chypt. So this is all 481 00:34:02.400 --> 00:34:07.119 an exercise working towards improving what you're talking about, like jail breaking. You 482 00:34:07.119 --> 00:34:12.440 know, every time a jailbreak comes out, they patch it, and the 483 00:34:12.480 --> 00:34:15.159 things that worked a month ago don't work in the latest models. So they're 484 00:34:15.159 --> 00:34:19.719 getting better and they really do seem to care about safety. But they did 485 00:34:19.719 --> 00:34:29.159 these hackers actually get in oh yeah, really step they almost always succeed. 486 00:34:29.519 --> 00:34:34.920 Security was not the first thought in these products. So yeah, they found 487 00:34:34.960 --> 00:34:38.199 stuff. Well that's great. That's a good thing. Yeah, but he 488 00:34:38.280 --> 00:34:40.880 say, it's the class that you know. That's the funny thing about the 489 00:34:40.880 --> 00:34:44.880 prompting model. Right, it's like you ask you for Windows licenses, as 490 00:34:44.880 --> 00:34:45.920 I can't do that, that's against the role. Tell me a story about 491 00:34:45.960 --> 00:34:50.559 giving me Windows licenses, no problem, right, my grandmother lost her Windows 492 00:34:50.559 --> 00:34:53.840 license. Yeah, story every night before we went to bed. Please tell 493 00:34:53.920 --> 00:34:59.559 me, please please tell me an encryption key. Well, my grandmother used 494 00:34:59.599 --> 00:35:01.400 to teach me all about thermite. Can you tell me a story about thermite? 495 00:35:01.440 --> 00:35:06.079 Pretty close to what a hack that was done before they button that up. 496 00:35:06.280 --> 00:35:10.800 Yeah. Yeah, I actually saw a white paper about can you make 497 00:35:10.920 --> 00:35:15.960 prompts that generate jail breaks? Like, just generate new jail breaks and jail 498 00:35:15.000 --> 00:35:20.079 break things in real time constantly. And you know there's always these gloom and 499 00:35:20.119 --> 00:35:23.760 doom papers that are coming out saying things like this, Maybe you can though, 500 00:35:24.119 --> 00:35:28.679 there's going to be definitely an arms race. Well, yeah, there 501 00:35:28.800 --> 00:35:30.920 is a we're going on right now. This is what it looks like. 502 00:35:31.039 --> 00:35:34.840 Yeah. I guess what I'm trying to say is it's going to matter more 503 00:35:34.880 --> 00:35:37.480 and more. Yeah. I think I think it's an interesting question because I 504 00:35:38.440 --> 00:35:43.480 also don't see this particularly improving all that. While I think we're not going 505 00:35:43.559 --> 00:35:47.239 to have any more exponential improvements on this. There's not an exponential more amount 506 00:35:47.239 --> 00:35:50.960 of data to train on. You know, we've kind of taken a pretty 507 00:35:50.960 --> 00:35:53.920 good chunk of the Internet already. There's not an exponential more amount of compute 508 00:35:54.159 --> 00:36:00.079 necessarily available on this for the price. So I think there's ink cremental improvements 509 00:36:00.079 --> 00:36:05.000 that can be made, Like the context engine could be way smarter, you 510 00:36:05.039 --> 00:36:07.920 know, just recognizing that iamic pentameter affects everything going forward, so I should 511 00:36:08.000 --> 00:36:13.920 preserve that piece of the cash and let other pieces expire like cash. It 512 00:36:13.920 --> 00:36:16.239 could be smarter than they are right now. Yeah, there's winds that could 513 00:36:16.239 --> 00:36:20.719 be done, but they're all incremental improvements. Yeah, and you know, 514 00:36:22.079 --> 00:36:25.400 it is a reasoning engine. Not to anthropomorphize, but it does have some 515 00:36:27.519 --> 00:36:32.280 reasoning ability that's very interesting. But it has it has limits that are very 516 00:36:32.440 --> 00:36:37.480 immediately obvious. Like with code for instance, we talked about you talked about 517 00:36:37.719 --> 00:36:43.199 copilot at the top. It is not close to taking your job. It's 518 00:36:43.280 --> 00:36:45.480 not close. I've been using it for a couple of weeks. Get hot. 519 00:36:45.519 --> 00:36:51.239 Compilot's pretty successful, and I think part of the reason is that the 520 00:36:51.239 --> 00:36:55.239 compiler has a say, and there's a skill level developers in parsing code that 521 00:36:55.400 --> 00:37:00.280 sort of deals with that problem and fixing the blank screen effect is really helpful 522 00:37:00.320 --> 00:37:04.400 most people, giving a starting point to almost anything. Yeah, unless the 523 00:37:04.480 --> 00:37:07.760 starting point is wrong, yes, which it has been from in my experience 524 00:37:07.880 --> 00:37:13.800 with GitHub Copilot, it'll suggest things that are completely insane. Yeah, you 525 00:37:13.840 --> 00:37:16.800 know. But also sometimes at least half the time, it just leads you 526 00:37:16.880 --> 00:37:20.880 astray. Yeah. Yeah. And it's funny because I've heard some stats from 527 00:37:20.920 --> 00:37:24.639 Microsoft about how often this works great for people, and those numbers cannot be 528 00:37:24.679 --> 00:37:28.880 true. My experience is that it is useful. It has a purpose. 529 00:37:29.000 --> 00:37:32.480 You know, it's writing like intern level or maybe better yeah, year one 530 00:37:32.599 --> 00:37:37.000 or two level code. But if you ask it to do really complicated things, 531 00:37:37.199 --> 00:37:43.519 it will either just lose the thread and keep making you know, You'll 532 00:37:43.559 --> 00:37:45.559 you'll there'll be a problem and it'll solve it. A new problem will be 533 00:37:45.559 --> 00:37:49.519 introduced, you'll ask it to solve that, it will, but it'll forget 534 00:37:49.519 --> 00:37:52.159 about the first problem and it's back. I feel like when I'm programming with 535 00:37:52.159 --> 00:37:57.599 get pilot GitHub Pilot, like i have a seventeen year old junior programmer sitting 536 00:37:57.679 --> 00:37:59.440 right next to me, and I'll do, like, you know, if 537 00:37:59.480 --> 00:38:01.840 certain can addition, and then they'll go console right line, console right line. 538 00:38:02.280 --> 00:38:10.679 No, yeah, I would argue, not the missile. So I 539 00:38:10.920 --> 00:38:15.719 would argue that it's actually not tied into the compiler tightly enough yet, because 540 00:38:15.079 --> 00:38:19.920 a lot of times it, like the autocomplete, The most annoying thing is 541 00:38:19.960 --> 00:38:22.480 that the autocomplete is constantly wrong, like yes, like when it's just trying 542 00:38:22.480 --> 00:38:27.280 to like suggest a method name, they could get that right, I think 543 00:38:27.360 --> 00:38:30.320 with a little bit of effort, you know, when again you talk about 544 00:38:30.320 --> 00:38:34.039 the incremental improvements, like you should run this through the compiler before showing it 545 00:38:34.079 --> 00:38:36.320 to me, because if it won't compile, obviously it doesn't matter to me. 546 00:38:36.719 --> 00:38:38.000 The same way as like if you're going to spit out a block of 547 00:38:38.079 --> 00:38:43.480 Texas references facts, you should double check those facts as well. That's right. 548 00:38:43.559 --> 00:38:46.280 The times where I've been more successful with GitHub copiler is when I actually 549 00:38:46.320 --> 00:38:51.000 write comments and tell exactly what I want. Yes, then it's pretty good, 550 00:38:51.480 --> 00:38:53.840 but it just guessing like what the what the condition is inside? And 551 00:38:53.880 --> 00:38:59.599 if statement it's like yep, if you also creating unit tests, I'm not 552 00:39:00.159 --> 00:39:02.599 a unit test fanatic, which you know a lot of people are, but 553 00:39:04.159 --> 00:39:07.239 and and God bless them. But if you just like you know, put 554 00:39:07.239 --> 00:39:10.360 a comment in asking it to make a unit test about some class, it 555 00:39:10.400 --> 00:39:15.880 is great at that, sometimes sometimes shockingly good at creating unit tests. That's 556 00:39:15.920 --> 00:39:21.079 awesome. And and just you know, talk about work he didn't want to 557 00:39:21.079 --> 00:39:23.679 do anyway, right like they when don't we talk about automation for it's like 558 00:39:23.800 --> 00:39:29.159 give me the dell where you get the dull work off my plate? Yeah? 559 00:39:29.440 --> 00:39:32.199 Right, yeah, And that's that I think is the highest purpose of 560 00:39:32.199 --> 00:39:38.239 this technology for now is as an amplifier to your abilities to get more of 561 00:39:38.280 --> 00:39:43.039 the things that need to be done. I'm I'm hoping for like really good 562 00:39:43.039 --> 00:39:45.679 assessors of is this code secure? Right? You know, which is a 563 00:39:46.280 --> 00:39:51.519 subject pretty challenging thing to consider, but it's not a bad goal to have. 564 00:39:51.599 --> 00:39:53.440 If for nothing else, just to give you that checklist on this code, 565 00:39:53.440 --> 00:39:55.599 to say, have you considered this, exided this, considered this? 566 00:39:55.920 --> 00:40:00.239 Yeah, what if you could what if you could take the output from thing 567 00:40:00.280 --> 00:40:06.119 like copilot, pipe it into another system that uses different technology to maybe run 568 00:40:06.159 --> 00:40:09.320 the code maybe right, some other type of test on top of it to 569 00:40:09.320 --> 00:40:12.920 see if it's doing we want to do and then give you back exultation. 570 00:40:13.199 --> 00:40:15.519 Yeah, you could get you could get really good results in that way. 571 00:40:15.519 --> 00:40:20.079 But so one thing that I've heard you say another episode is Richard that kind 572 00:40:20.119 --> 00:40:22.679 of resonated with me, was where's the killer app? Yeah, because it 573 00:40:22.719 --> 00:40:28.679 ain't existential conversation. I've had some pretty good talks with it. But yeah, 574 00:40:28.679 --> 00:40:30.039 I agree with you, that's not really that's not the highest use. 575 00:40:30.239 --> 00:40:37.519 I'm pretty sure Brian came from you. Could it could be, it could 576 00:40:37.519 --> 00:40:43.800 be. Well, so I think that what's happening is so so Roster where 577 00:40:43.840 --> 00:40:50.000 I work. We're not an AI company, and yet we hit the forty 578 00:40:50.000 --> 00:40:55.360 thousand token per minute limit all the time, per minute per minute. Yeah. 579 00:40:57.119 --> 00:40:59.760 Yeah, we're not doing that twenty four hours a day. But when 580 00:40:59.800 --> 00:41:04.519 this is running, you know, we run it a lot. And and 581 00:41:04.559 --> 00:41:09.159 so I think that the deal is that every company is a language company, 582 00:41:09.800 --> 00:41:15.719 and there are classes of problems that are solvable with this technology that are hard 583 00:41:15.760 --> 00:41:20.639 to solve with other technologies. And that's the killer app is quiet. The 584 00:41:20.719 --> 00:41:23.480 killer app is the thing at Roster that runs in the background and looks at 585 00:41:23.480 --> 00:41:28.639 the comments and flags them when you identify yourself, you know, like so, 586 00:41:28.679 --> 00:41:30.840 for instance, I'll just give you a quick quick background on roster, 587 00:41:30.880 --> 00:41:35.000 all right, so that my bio is opaque. Sure, But so we 588 00:41:35.039 --> 00:41:38.119 do three sixty evaluations. We've got a really good process for So you're sat, 589 00:41:38.159 --> 00:41:40.960 Carl, you're the seat, you have a big company, and you 590 00:41:42.000 --> 00:41:44.480 want to get feedback on how you're doing, what you should improve on. 591 00:41:45.559 --> 00:41:47.599 You can run this process where we give you this survey and we give all 592 00:41:47.639 --> 00:41:52.239 your co workers the same survey, and from the delta, and what you 593 00:41:52.280 --> 00:41:55.199 say and what they say and the comments, we can tell you what you 594 00:41:55.199 --> 00:42:00.800 should work on. I can't just count Facebook likes. I would not recommend 595 00:42:00.840 --> 00:42:07.440 it. So so this process, this process can be really powerful if you 596 00:42:07.440 --> 00:42:10.880 embrace it. And we've done over a thousand of them with c level executives 597 00:42:10.920 --> 00:42:16.159 at at private equity backed companies and they seem to like it. One of 598 00:42:16.159 --> 00:42:21.800 the things that happens is the comments. We tell people, don't identify yourself 599 00:42:21.840 --> 00:42:23.880 in the comments. Like if you had lunch with Carl last week and Carl 600 00:42:23.880 --> 00:42:25.840 did something you didn't like and you want to write a comment about it, 601 00:42:27.280 --> 00:42:29.480 write it in such a way. Don't say you went to lunch with Carl 602 00:42:29.559 --> 00:42:34.440 last week, and you know, so we take the comments. So people 603 00:42:34.440 --> 00:42:40.079 don't know this actually, but humans have historically gone over our comments and checked 604 00:42:40.119 --> 00:42:45.639 for problems like this where you're you're you're unmasking yourself and it could have repercussions 605 00:42:45.719 --> 00:42:52.920 for you and they will maybe hide those comments or so. So it actually 606 00:42:52.039 --> 00:42:55.920 that's a that's a hard problem to solve with traditional code, but with a 607 00:42:55.960 --> 00:43:00.280 large language model, you can you can de identify the people any games that 608 00:43:00.320 --> 00:43:02.519 are in there and pass it through a model and say did this person talk 609 00:43:02.519 --> 00:43:07.199 about something that is identifiable and flag it for a human to review. And 610 00:43:07.320 --> 00:43:13.239 there are a lot of problems like that. Another one is sometimes people say 611 00:43:13.280 --> 00:43:15.639 in a comment like the answers should be not applicable, but they say in 612 00:43:15.679 --> 00:43:19.400 a comment like I don't have any context for answering this, and they give 613 00:43:19.440 --> 00:43:22.199 them a five out of a you know, they give them a medium score. 614 00:43:22.920 --> 00:43:27.880 We can also look at the comment and say, is that like an 615 00:43:27.960 --> 00:43:32.400 ayah and then just yeah should be in a this And those are huge time 616 00:43:32.440 --> 00:43:36.000 savers and it's not something our ops team wants to do, you know, 617 00:43:36.159 --> 00:43:40.119 like they don't love going through these comments and doing that, you're not taking 618 00:43:40.159 --> 00:43:45.480 anybody's job, not in this case. No, not in this case. 619 00:43:45.599 --> 00:43:49.719 So I think every that's that's kind of like dry stuff for most people I 620 00:43:49.800 --> 00:43:52.519 imagine to think about. But every business. I mean, you're only talking 621 00:43:52.599 --> 00:43:57.440 a step above a sentiment analyzer. But you know, I get what you're 622 00:43:57.480 --> 00:44:00.840 talking about, right, And again back to the are you running that through 623 00:44:00.840 --> 00:44:04.599 GPT four you're running through GPT three? Well, I've tried both as in 624 00:44:04.719 --> 00:44:10.199 fact, I love that. Please tell me there was a difference there was. 625 00:44:10.280 --> 00:44:14.599 Oh yeah, GPT four is smarter than three. There's no three point 626 00:44:14.639 --> 00:44:22.960 five is not as Smart's anthromorphize like produced better results. Yeah, but see 627 00:44:22.960 --> 00:44:28.239 I know what he meant. You know what he meant. Yeah, it's 628 00:44:28.320 --> 00:44:37.159 my best friend. And well we can talk about companies like a replica and 629 00:44:37.320 --> 00:44:43.079 character dot AI. Oh geez, but how is it better? Okay, 630 00:44:43.159 --> 00:44:46.840 So it hallucinates less often, and it reasons better, and it's just seems 631 00:44:46.880 --> 00:44:52.239 to be generally more capable. It produces factual results results more often. So 632 00:44:52.519 --> 00:44:54.880 also, so I mean I'm getting back sort of detailing. It's like it 633 00:44:55.000 --> 00:45:00.480 detects an ana more often than GPT three did. Yeah, it's lower as 634 00:45:00.519 --> 00:45:04.559 well, and it costs more, it's still it's still cheap. From you 635 00:45:04.599 --> 00:45:07.559 know, a business perspective, like you know, if if we have a 636 00:45:07.679 --> 00:45:09.880 day where that's only because they're undercharging you for it. Yeah, maybe so 637 00:45:10.159 --> 00:45:14.440 well actually a quick note on that. So I happen to think, and 638 00:45:14.480 --> 00:45:17.559 I am not a lawyer, but there's this big ethical problem with large language 639 00:45:17.559 --> 00:45:23.519 models that mostly rears its head. It's mostly visible with the image generation stuff. 640 00:45:23.760 --> 00:45:31.119 Sure, you know, like like that's that is yes, that makes 641 00:45:31.159 --> 00:45:35.760 it very clear what's going on. If you're an artist and you are making 642 00:45:35.800 --> 00:45:37.800 your money on art and all your art gets hoovered up into this model and 643 00:45:37.880 --> 00:45:45.079 then you can generate you know, art just like what Richard made for zero 644 00:45:45.159 --> 00:45:49.519 dollars, that's bad for Richard, right. So I happen to be of 645 00:45:49.559 --> 00:45:52.440 the opinion and i'd love to hear your thoughts that we need something like the 646 00:45:52.519 --> 00:45:57.119 music industry has, where there are several different kinds of royalties already, you 647 00:45:57.199 --> 00:45:59.920 know, there's mechanical royalties and performance royal. I don't think we should be 648 00:46:00.039 --> 00:46:04.519 looking to the music industry for any kind of business acumen or any kind of 649 00:46:05.000 --> 00:46:09.840 suggestions unless well they're all the money out of everybody's creating content. Well, 650 00:46:09.880 --> 00:46:13.800 we're trying this is actually about giving the money. We need something that says 651 00:46:14.199 --> 00:46:15.880 if your work is used to train, you get some kind of royalty and 652 00:46:15.920 --> 00:46:21.840 that will drive prices up. So the old music industry before Spotify, Yeah, 653 00:46:22.000 --> 00:46:23.880 I think it's what you're talking about. I'm talking about what the losses, 654 00:46:24.159 --> 00:46:28.760 mechanical royalties and all that stuff. But you get back to the issue 655 00:46:28.800 --> 00:46:32.400 here, which is you trained on copyrighted materials. Just because they were publicly 656 00:46:32.440 --> 00:46:37.639 accessible doesn't mean they weren't copyrighted. Right there's no intellectual property protections for of 657 00:46:37.719 --> 00:46:40.840 this kind. Right now, you can see how we got there because there's 658 00:46:40.880 --> 00:46:46.760 always been a concept in machine learning that the training set would never be visible 659 00:46:47.440 --> 00:46:52.960 in the finished product. And that and then the Getty logo showed up, 660 00:46:52.400 --> 00:46:57.960 right Like, I would argue that that's what revealed the issue that I've until 661 00:46:57.960 --> 00:47:04.239 then nobody really cared right up, and chill artists names appeared in the render. 662 00:47:04.480 --> 00:47:07.239 So it's like, I'm sorry. You know that what you feed into 663 00:47:07.280 --> 00:47:13.880 these data sets does affect the output, and so copyright is your consideration. 664 00:47:14.400 --> 00:47:16.039 Yeah, and it's affecting people's actual bottom lines. You know, I've worked 665 00:47:16.039 --> 00:47:22.000 with illustrators. There's a great illustrator who I really like, who produces great 666 00:47:22.000 --> 00:47:24.880 work. And I know he's hurting right now, and he's thinking, maybe 667 00:47:24.920 --> 00:47:30.079 I should create a business model where you know, so like board games are, 668 00:47:30.239 --> 00:47:32.280 that's an industry I know a little bit about. In the board game 669 00:47:32.280 --> 00:47:37.239 industry, people do not want to buy games that work with generative images. 670 00:47:37.639 --> 00:47:43.079 They will shun you. Yeah, that's the whole thing. And I think 671 00:47:43.119 --> 00:47:46.039 that probably will maybe it'll grow a little bit. So he's thinking, like, 672 00:47:46.159 --> 00:47:51.239 maybe I can provide an abstraction for them where I just create the generative 673 00:47:51.320 --> 00:47:54.719 images and and you know doctor that like do a little bit of extra work 674 00:47:54.800 --> 00:48:00.199 on the end, and then they can claim like deniability. That's making people 675 00:48:00.239 --> 00:48:05.519 think those kinds of thoughts, and it's not great, But I kind of 676 00:48:05.519 --> 00:48:07.559 see where he's coming from, because it's his work that's being stolen, right 677 00:48:08.000 --> 00:48:13.599 right, yep, So I'm using the tool to regenerate my work, right. 678 00:48:14.159 --> 00:48:16.079 I agree with you. By the way, I think there absolutely has 679 00:48:16.159 --> 00:48:21.800 to be some sort of way that artists can get paid for their contributions to 680 00:48:22.440 --> 00:48:24.400 or to opt outive or to opt out Yeah, yeah, yeah, I 681 00:48:24.480 --> 00:48:28.880 mean start with opt out. We can figure out the rest of large format, 682 00:48:28.920 --> 00:48:30.639 right, you know, if you want to be in charge of you 683 00:48:30.039 --> 00:48:34.719 right, yeah, yeah, So I think there's two two more thoughts on 684 00:48:34.800 --> 00:48:37.079 this. One. I think we will end up with illegal framework for this. 685 00:48:37.320 --> 00:48:42.679 But another one is we will have spent a couple of years generating synthetic 686 00:48:42.880 --> 00:48:45.559 art off of this that's really good, and why not just train it off 687 00:48:45.599 --> 00:48:52.159 of that and cut everybody else out? It's there are big problems so far. 688 00:48:52.280 --> 00:48:55.199 The papers I've read about generating off of generative data is that it's a 689 00:48:55.280 --> 00:49:00.760 significant degradation, like the quality goes down dramatically. Yeah, it's a photocopy 690 00:49:00.800 --> 00:49:04.599 of a photocopy. I saw one that also says that that may create a 691 00:49:04.679 --> 00:49:08.440 ceiling for what's possible with these models, because we've now flooded the Internet with 692 00:49:08.639 --> 00:49:14.280 crappy generated texts for instance. Yeah, no, we've we've created a Kessler 693 00:49:14.360 --> 00:49:19.440 syndrome in the Internet right where we've now spat out so much generative data into 694 00:49:19.519 --> 00:49:22.559 it that it's so polluted now you could never do it again. You want 695 00:49:22.559 --> 00:49:25.159 to, you want to have a fun time. Ask Dally to generate an 696 00:49:25.199 --> 00:49:30.639 image of two people shaking hands, well, hands are actually hands have gotten 697 00:49:30.760 --> 00:49:35.440 way better, but so like two weeks later, Yeah, but they've Yeah, 698 00:49:35.480 --> 00:49:39.639 I've seen like hands with seven fingers and three fingers and they don't even 699 00:49:39.719 --> 00:49:46.000 look like fingers. Forks too, forks with like crazy times on them that 700 00:49:46.119 --> 00:49:51.400 don't look real. Well, Mid mid Journey has leveled up it's hand game 701 00:49:51.519 --> 00:49:53.159 quite a bit from what I've seen. Yeah, I think there is enough 702 00:49:53.199 --> 00:49:57.000 good content that maybe you could tag it up and train with it. I 703 00:49:57.320 --> 00:49:59.880 don't know, I maybe talking out to turn a little bit there, but 704 00:50:00.000 --> 00:50:02.880 I will say that after I saw that paper about having a theoretical ceiling on 705 00:50:02.920 --> 00:50:07.119 what can be generated using you know, because of the new state of the 706 00:50:07.159 --> 00:50:12.079 Internet, Stable Diffusion released the model that I'm talking at a turn a little 707 00:50:12.079 --> 00:50:14.800 bit here. Because I haven't read the white papers, I don't fully understand 708 00:50:14.840 --> 00:50:20.079 it, but it seems like it's heavily tied into training using synthetic data and 709 00:50:20.199 --> 00:50:24.119 it's good. Stable Diffusion released like around the same time that Dolly or Not 710 00:50:24.239 --> 00:50:28.519 Dolly, I'm sorry a Llama two came out. Stable Diffusion also released a 711 00:50:28.559 --> 00:50:30.519 couple models. I'm going to say that I'm not an expert on that, 712 00:50:30.800 --> 00:50:34.320 and you should look into it yourself and learn what you can. And there 713 00:50:34.400 --> 00:50:37.159 was another one you said, stepped up its hands game. What was that 714 00:50:37.239 --> 00:50:40.280 one? Oh, mid Journey? So mid Journey. Yeah, I'm much 715 00:50:40.320 --> 00:50:46.480 more focused on text, but I see in my in my wanderings, I 716 00:50:46.559 --> 00:50:51.039 see a lot about the visual stuff too, and Mid Journey I think is 717 00:50:51.079 --> 00:50:54.519 probably the leader in that space. Mid Journey stable diffusion dollies in the mix 718 00:50:54.559 --> 00:51:00.280 somewhere too. Yeah, I'm I don't see a continue progression a lot of 719 00:51:00.320 --> 00:51:04.880 this stuff just because it is up against its own weight. You know, 720 00:51:04.960 --> 00:51:07.000 we trained it on the internet. Have you seen the internet lately? Gee? 721 00:51:07.239 --> 00:51:12.119 That reminds me, Brian. When we first started talking in the AI 722 00:51:12.239 --> 00:51:15.960 Bought Show, my experience of using GPT was that it couldn't reach out to 723 00:51:16.039 --> 00:51:20.760 the Internet. You couldn't ask it like, you know, where's the nearest 724 00:51:21.559 --> 00:51:24.199 you know, stuff that you might ask Google or Bang to go do a 725 00:51:24.239 --> 00:51:27.760 search and kind of distill it down for you. And then you show me 726 00:51:27.880 --> 00:51:31.440 the plugins. Oh my god, the plugins for GPT. There's so many 727 00:51:31.519 --> 00:51:35.639 of them, but one of them is just like a simple browser plug in, 728 00:51:36.519 --> 00:51:38.320 and when that thing is enabled, you can just say, what did 729 00:51:38.360 --> 00:51:45.559 we say? How find me a welder in New London County that might be 730 00:51:45.599 --> 00:51:50.360 available for a small project. And it literally went out searched the internet and 731 00:51:50.519 --> 00:51:54.480 distilled the information down to a list, a bulleted list with all the information 732 00:51:54.559 --> 00:52:00.039 that I want. Yeah, so agents type. So using the PI you 733 00:52:00.119 --> 00:52:02.480 can do a lot more than you can with chat GPT, and I think 734 00:52:02.559 --> 00:52:06.280 that's to me, that's where the most interesting work is going on. The 735 00:52:06.360 --> 00:52:09.159 Playground in particular, is your one of your favorite tools. Well, playground 736 00:52:09.199 --> 00:52:15.039 is for what playground actually is is for prototyping things you want to do with 737 00:52:15.079 --> 00:52:20.800 the API. So you know, my workflow is I usually go into playground 738 00:52:21.079 --> 00:52:23.480 and I make something work, I train it a little bit with some data, 739 00:52:23.880 --> 00:52:28.800 and then I encode that into C sharpcode and call up with the API. 740 00:52:29.199 --> 00:52:32.880 My users in Roster, they don't ever see a chat interface if this 741 00:52:32.960 --> 00:52:37.719 happens. This actually happens in Azure functions. We have Azure functions that are 742 00:52:37.760 --> 00:52:40.039 just grinding away in the cloud, you know, trying not to hit that 743 00:52:40.320 --> 00:52:47.639 that limit. So when you're when you're working with a server environment, chaining 744 00:52:47.760 --> 00:52:51.480 is where it's at. You know, like one prompt calling into the server, 745 00:52:51.679 --> 00:52:53.480 calling into another prompt that's specialized for something else. I mean, that's 746 00:52:54.079 --> 00:53:00.000 that's the magic, that's the that's where that's where the cool work has happen, 747 00:53:00.000 --> 00:53:02.360 and that's how Smallville works. That's you know, roster. Some of 748 00:53:02.400 --> 00:53:07.719 the things that we do involve six prompts that run in series to get a 749 00:53:07.760 --> 00:53:10.920 good outcome. Good. Talk about a few more plugins that you like to 750 00:53:12.039 --> 00:53:15.679 use. You told me that there was one about where you could just book 751 00:53:15.760 --> 00:53:20.920 travel. Yeah, there's a cup just by talking at chat GPT. Yeah, 752 00:53:20.960 --> 00:53:24.320 there's a Kayak plug in that can book travel, rental cars, and 753 00:53:24.480 --> 00:53:28.719 hotels. I think I haven't actually booked travel with it, but there's a 754 00:53:28.840 --> 00:53:31.679 whole bunch, and you know, I think it's possible. There's a there's 755 00:53:31.679 --> 00:53:37.480 a program where you could sign up to create your own plug ins. So 756 00:53:37.840 --> 00:53:42.960 let's say you had some business that makes widgets, why not make a plug 757 00:53:43.039 --> 00:53:49.039 in that connects to an API inside your business that allows you to ask intelligent 758 00:53:49.079 --> 00:53:52.559 business questions that you know, The plug in calls your API in point, 759 00:53:52.599 --> 00:53:57.280 The API in point looks up what the answer is, and then it renders 760 00:53:57.320 --> 00:54:01.400 it all with a language interface. Yeah. Yeah, yeah, that's kind 761 00:54:01.440 --> 00:54:07.760 of how all the being searched type things work like. It's just it's just 762 00:54:07.039 --> 00:54:10.480 calling search in the background, getting the results, and then telling you about 763 00:54:10.519 --> 00:54:14.239 it. There was another one that you mentioned where you can upload a PDF 764 00:54:14.519 --> 00:54:16.559 like the rules of the Dungeons and Dragons or something like that. You can 765 00:54:16.679 --> 00:54:21.079 upload that as a PDF, and then when there's a problem with a game 766 00:54:21.199 --> 00:54:25.079 that arises, because you know it inevitably does and there's a dispute, rather 767 00:54:25.199 --> 00:54:29.400 than taking a half an hour and looking through the manual, you could just 768 00:54:29.719 --> 00:54:32.800 ask a chat GPT a question. Yeah. Interestingly, it already knows a 769 00:54:32.840 --> 00:54:37.480 lot about d and ds. You can just ask you can you can upload 770 00:54:37.559 --> 00:54:42.280 files, imagine, upload your service manual for your car, right and then 771 00:54:42.440 --> 00:54:45.480 say, yeah, I have a little noise and I can't call car talk 772 00:54:45.519 --> 00:54:50.199 anymore because they're off the air. But well, I mean, isn't this 773 00:54:50.280 --> 00:54:53.760 what M three sixty five copilot ultimately is is access to all of the corporate 774 00:54:53.840 --> 00:54:58.960 documentation, all of the emails, all of the interactions within an organization. 775 00:54:59.440 --> 00:55:02.719 I can see it could become this, you know, corporate memory that you 776 00:55:02.840 --> 00:55:06.800 could ask it anything about the company and it can pull all the things. 777 00:55:07.679 --> 00:55:09.440 Yeah, so that's enterprise search. And there are a few people working on 778 00:55:09.519 --> 00:55:14.480 that. It's a hard problem. This actually leads into talking about vector databases 779 00:55:14.480 --> 00:55:17.559 a little bit. That's a really fun topic. Yeah. So the problem 780 00:55:17.599 --> 00:55:22.519 with the problem with enterprise search is that all the corporate documents is probably a 781 00:55:22.599 --> 00:55:27.039 lot more than the eight K context. Right, you can't really you can't 782 00:55:27.079 --> 00:55:29.800 really load them. So what do you do? How do you make something 783 00:55:30.719 --> 00:55:37.400 that has long term memory for a large language model? And vector databases are 784 00:55:37.239 --> 00:55:40.920 are an answer to that. And can we just digress for a second and 785 00:55:42.000 --> 00:55:45.280 talk about how So okay, so this blew my mind when I learned about 786 00:55:45.280 --> 00:55:49.840 it. All right, this is one of the coolest innovations in this area. 787 00:55:49.960 --> 00:55:52.320 So there are these things called embeddings, all right, and this is 788 00:55:52.400 --> 00:55:57.000 this is pretty technical. This is programmer talk. So you can ask. 789 00:55:57.400 --> 00:56:00.800 There's a special model an opening I called adda two ada like at a lovelace, 790 00:56:01.559 --> 00:56:06.639 and it's it's specialized for generating embeddings. It's very, very cheap. 791 00:56:06.880 --> 00:56:10.400 And when you look at it embedding it's so you send it a sentence or 792 00:56:10.400 --> 00:56:16.079 a word and it responds with this giant array of numbers. It's actually fifteen 793 00:56:16.199 --> 00:56:22.239 hundred like dimensions on this thing. And you look at it and think what 794 00:56:22.480 --> 00:56:25.159 is this? All right? Why do we have fifteen hundred dimensions? And 795 00:56:25.280 --> 00:56:30.400 what is this for? Well, it allows you to do search and so 796 00:56:30.559 --> 00:56:34.920 imagine imagine this, okay, just stay with me. Imagine a spreadsheet, 797 00:56:35.000 --> 00:56:42.440 okay, and on this spreadsheet you see things like boy, teenager man, 798 00:56:43.559 --> 00:56:47.800 girl, teenager woman, larva, pupa butterfly, egg, chicken, rooster, 799 00:56:49.119 --> 00:56:51.800 you know, like that's what's going across the rows of this thing or 800 00:56:51.880 --> 00:56:54.199 the yeah, the rose. So what is this you're looking at. You're 801 00:56:54.239 --> 00:57:00.119 looking at a spreadsheet of life cycle of things, right, So here is 802 00:57:00.159 --> 00:57:04.320 the thing, all right. One of those dimensions somewhere in there is that 803 00:57:04.599 --> 00:57:09.920 it's a big, a big table of life cycles, and and it's mapped 804 00:57:10.519 --> 00:57:15.440 your word somewhere on that spreadsheet. And it's done that with fifteen hundred plus 805 00:57:15.559 --> 00:57:20.719 other things, okay, other other like points of knowledge that it identified during 806 00:57:20.760 --> 00:57:27.280 training automatically. So so what does that do. It makes it so that 807 00:57:27.360 --> 00:57:31.000 you can check the distance between two things, two words, two sentences, 808 00:57:31.039 --> 00:57:36.320 two images. You can do this with anything. So so you run an 809 00:57:36.320 --> 00:57:44.800 algorithm like dot product and it's it calculates the distance between this concept and that 810 00:57:44.920 --> 00:57:51.000 concept, like boy, and girl and and tells you how far away they 811 00:57:51.079 --> 00:57:54.719 are in semantic space. That is crazy. I mean, you have to 812 00:57:54.840 --> 00:57:59.440 have structured all of this data already. It's not like the machine learning models 813 00:57:59.440 --> 00:58:04.039 that are supposed structure data themselves. So can you can you actually generate vector 814 00:58:04.159 --> 00:58:08.039 databases just from men inference you so it has it already, So open ai 815 00:58:08.119 --> 00:58:15.079 already has their their model. And with adda, with just hitting the attitude 816 00:58:15.559 --> 00:58:21.440 endpoint, you can say what are the embeddings for this word or sentence and 817 00:58:21.519 --> 00:58:24.880 it will give you that, and then you can do operations. And some 818 00:58:25.000 --> 00:58:30.480 of the operations have really interesting properties, like the same concept in two different 819 00:58:30.519 --> 00:58:35.000 languages will be in a similar location in semantic space. This is reminding me 820 00:58:35.119 --> 00:58:40.199 of the old Lapp cube concept in databases that was very popular in the early 821 00:58:40.320 --> 00:58:46.360 two thousands or the mid late two thousands, where instead of storing data in 822 00:58:46.480 --> 00:58:52.199 tables, rows and columns, it's a it's a three dimensional cube. But 823 00:58:52.440 --> 00:58:55.960 I never used it, and I never I barely grasped the concept at the 824 00:58:57.039 --> 00:59:00.960 time. I remember talking to Andrew Bruston people about it. But this is 825 00:59:01.000 --> 00:59:05.440 a little bit different though, because you're calculating distances, you're not actually looking 826 00:59:05.519 --> 00:59:09.199 for a value. Well, let me bring this back to earth. So 827 00:59:09.360 --> 00:59:14.440 we have this technology that's sort of somewhere in the outer layer of large language 828 00:59:14.480 --> 00:59:17.440 models that does these embttings and works with them. How does this tie into 829 00:59:17.679 --> 00:59:21.320 search? Well, it is search. Like, if you want to do 830 00:59:21.440 --> 00:59:24.920 good search, make embettings for the search terms and for everything you want to 831 00:59:24.920 --> 00:59:30.039 search on and calculate the distance in semantic space. It's not an expensive operation. 832 00:59:32.440 --> 00:59:35.440 Yeah, there could be a lot of terms. So you need a 833 00:59:35.440 --> 00:59:39.760 specialized database called a vector database. And so what you do is you take 834 00:59:40.400 --> 00:59:45.880 you take a sentence like dot net rocks is a great podcast, vectorize that 835 00:59:45.800 --> 00:59:50.440 our creative bettings for it, save that in the database, maybe put a 836 00:59:50.480 --> 00:59:53.599 couple of tags on it so you can search in additional ways and maybe add 837 00:59:54.360 --> 00:59:59.679 the text that it belongs to. So then you can say, what is 838 00:59:59.719 --> 01:00:04.920 a great podcast, and the database will return the best matches and you just 839 01:00:05.000 --> 01:00:08.039 pull the text out and actually what you can do is embed that into a 840 01:00:08.119 --> 01:00:13.440 prompt. Take the text out and bed into a prompt and you know a 841 01:00:13.480 --> 01:00:19.199 prompt us what you give gpt at the beginning or you know a question. 842 01:00:19.440 --> 01:00:22.440 Yeah, so imagine this flow for chaining all right. Step one user asks 843 01:00:24.480 --> 01:00:32.000 what's how many units did we ship in the Southwest in twenty seventeen. That 844 01:00:32.199 --> 01:00:37.920 request gets sent to a server and the server looks at that and says, 845 01:00:38.159 --> 01:00:44.079 let's search our vector database for this query, and it just does a call 846 01:00:44.159 --> 01:00:47.039 out to pine Cone or some other A lot of a lot of databases are 847 01:00:47.039 --> 01:00:52.360 getting bolted on vector capabilities, like postcris has it. Asure is working on 848 01:00:52.440 --> 01:00:57.079 it. So you make that call out, it gives you back the best 849 01:00:57.119 --> 01:01:00.480 results, and then you make another prompt where you you take that chunk of 850 01:01:00.559 --> 01:01:05.719 text and you embed that in there and say, you know, show this 851 01:01:05.840 --> 01:01:09.239 to the user in whatever way is appropriate. And it does. And so 852 01:01:09.360 --> 01:01:14.119 you still have hard problems though, because you still are limited by the context. 853 01:01:14.320 --> 01:01:17.360 You're if you have a giant document, you need to go further and 854 01:01:17.559 --> 01:01:21.719 like chunk the document up and find the most relevant part. These are hard 855 01:01:21.920 --> 01:01:25.599 problems, but people are working on them. Microsoft certainly is wondering why we're 856 01:01:25.639 --> 01:01:30.159 not just you know, most folks think in terms that we just retrain the 857 01:01:30.239 --> 01:01:32.639 model with my data. I mean that's basically how they described get Hub Copilot 858 01:01:32.719 --> 01:01:37.079 is that they took a large language model and then added in all of the 859 01:01:37.719 --> 01:01:40.239 code. They were able to scrape out of their own site as part of 860 01:01:40.280 --> 01:01:45.880 the learning model, and so it understood code better. Couldn't you do that 861 01:01:45.000 --> 01:01:49.159 with corporate data train it into the model. Yeah, there's a couple of 862 01:01:49.159 --> 01:01:52.079 ways you could. There's security issues there though, security and accuracy issues. 863 01:01:52.400 --> 01:01:54.920 Well, if you use fine tuning. Basically, what you're doing with fine 864 01:01:54.960 --> 01:02:00.719 tuning is you can upload a list of prompts and what output should be. 865 01:02:00.800 --> 01:02:06.639 So, for instance, one could be what was the what do we sell 866 01:02:06.719 --> 01:02:09.960 in the Southwest in twenty seventeen and the answer is two thousand units. You 867 01:02:10.039 --> 01:02:15.000 could train it with a whole bunch of queries like that and then open Ai 868 01:02:15.079 --> 01:02:19.320 actually host that in the cloud for you. It costs more, a little 869 01:02:19.320 --> 01:02:22.079 more, it's not it's not grievously expensive, but then it's trained on your 870 01:02:22.159 --> 01:02:27.199 data. The problem is that it's not live, it's not real time. 871 01:02:27.400 --> 01:02:30.920 So if you want something that's organic, that's that's changing as your organization grows, 872 01:02:31.559 --> 01:02:35.960 you can't do that. You can also, potentially, especially with something 873 01:02:36.079 --> 01:02:39.920 like Lama two, you could actually train it yourself. You can like get 874 01:02:40.000 --> 01:02:45.599 some GPUs and actually, yeah, actually train it and then host it yourself. 875 01:02:45.360 --> 01:02:49.880 That's possible, but again it's it's not dynamic. It's going to be 876 01:02:49.920 --> 01:02:52.159 static. So the only way to make it, the only way I'm aware 877 01:02:52.199 --> 01:02:57.760 of, to make it truly organic that changes as your documents change, is 878 01:02:57.840 --> 01:03:01.000 to sink all this stuff up with some kind of vector database and do the 879 01:03:01.079 --> 01:03:05.519 hard work. And whoever solves that is going to have a killer app, 880 01:03:05.559 --> 01:03:07.320 that's for sure, and I think it's Microsoft's got a crack at it for 881 01:03:07.840 --> 01:03:12.800 definitely. Dude, we could go on talking for another hour easily, it 882 01:03:12.800 --> 01:03:16.320 would just fly by like this one did. What's neck? What are we 883 01:03:16.360 --> 01:03:21.000 gonna do next time? The AI bon show. Well, I know a 884 01:03:21.079 --> 01:03:25.400 couple of really good D and D experts and I've got one lined up to 885 01:03:25.440 --> 01:03:29.239 come on and talk about how he's already using this to run his D and 886 01:03:29.280 --> 01:03:31.320 D campaigns. It should be a really good show. Maybe we should substitute 887 01:03:31.360 --> 01:03:37.800 me out for somebody who's played DN D. You can, you can, 888 01:03:37.840 --> 01:03:42.199 definitely, you can definitely learn about it. I'm sure you'll sell good questions 889 01:03:42.239 --> 01:03:45.519 like he did with board games. That sounds good. And he's also as 890 01:03:45.519 --> 01:03:46.960 sharp. He's a she's sharp dev as well, so he's part of this 891 01:03:47.079 --> 01:03:51.679 world. Good, well, that sounds fun. Thanks Brian, It's always 892 01:03:51.719 --> 01:03:53.239 good talking to you. Oh it was great to be here and we'll talk 893 01:03:53.239 --> 01:04:17.639 to you next time. On dot net work dot net Rocks is brought to 894 01:04:17.719 --> 01:04:23.719 you by Franklin's Net and produced by Pop Studios, a full service audio, 895 01:04:23.880 --> 01:04:28.800 video and post production facility located physically in New London, Connecticut, and of 896 01:04:28.880 --> 01:04:34.000 course in the cloud online at pwop dot com. Visit our website at dt 897 01:04:34.320 --> 01:04:40.800 n et r o cks dot com for RSS feeds, downloads, mobile apps, 898 01:04:40.960 --> 01:04:44.920 comments, and access to the full archives going back to show number one, 899 01:04:45.480 --> 01:04:48.159 recorded in September two thousand and two. And make sure you check out 900 01:04:48.159 --> 01:04:51.840 our sponsors. They keep us in business. Now go write some code, 901 01:04:53.400 --> 01:05:04.840 See you next time. My God let Me is hard than my Texas Red