1 00:00:01,080 --> 00:00:05,679 How'd you like to listen to dot NetRocks with no ads? Easy? Become 2 00:00:05,679 --> 00:00:09,839 a patron for just five dollars a month. You get access to a private 3 00:00:10,000 --> 00:00:14,279 RSS feed where all the shows have no ads. Twenty dollars a month, 4 00:00:14,279 --> 00:00:18,839 we'll get you that and a special dot NetRocks patron mug. Sign up now 5 00:00:18,839 --> 00:00:36,159 at Patreon dot dot NetRocks dot com. Welcome back to dot net rocks. 6 00:00:36,200 --> 00:00:40,079 I'm Carl Franklin and I'm Richard Campbell, and we are in our respective places 7 00:00:40,079 --> 00:00:45,320 on opposite sides of the North American continent, absolutely as we usually are. 8 00:00:45,840 --> 00:00:51,000 But why don't know you caught us in the middle of three or four conferences 9 00:00:51,039 --> 00:00:55,320 here listener. Yeah, well, in the last few shows and get in 10 00:00:55,359 --> 00:00:58,280 time shifting being hilarious. We're all shows. We did it in Porto, 11 00:00:58,640 --> 00:01:03,239 and some awesome conversations. Really neat folks, folks who never talked to before. 12 00:01:03,239 --> 00:01:06,400 I always tend to when we're going to do in person shows, I 13 00:01:06,400 --> 00:01:08,480 try to go after guests we've never had on before. Right, Yeah, 14 00:01:08,560 --> 00:01:12,200 new stuff, all good, I would point out, I am. You 15 00:01:12,239 --> 00:01:15,400 know, it's getting towards the end of the year, so you know what 16 00:01:15,439 --> 00:01:19,079 I'm working on the end of the year, geek out. Yeah, yeah, 17 00:01:19,120 --> 00:01:22,560 I know it's close to the end of the year when I have to 18 00:01:22,599 --> 00:01:26,599 open up the script my script files and start pulling stuff forward for space and 19 00:01:26,640 --> 00:01:29,799 for energy, and I can't wait. There's been so much, it's so 20 00:01:30,000 --> 00:01:33,359 much, so much good stuff. They're both they're both going to be I'm 21 00:01:33,359 --> 00:01:36,599 just committing now. They're going to be long. I mean, you could 22 00:01:36,640 --> 00:01:40,840 just do a whole one on the JWT right, oh, JWST and the 23 00:01:40,879 --> 00:01:46,560 crisis in cosmology, like the current proposal they're floating around. Maybe we're long 24 00:01:46,599 --> 00:01:49,719 about the length of the age of the universe. Maybe it's twice as long 25 00:01:49,760 --> 00:01:53,840 as we thought. It's twenty six billion years instead of thirteen billion, wow, 26 00:01:53,879 --> 00:01:57,280 which just is stunning. I mean, I love the fact that this 27 00:01:57,400 --> 00:02:01,079 is why that instrument went up there, right, it was to test all 28 00:02:01,120 --> 00:02:07,319 of the assumptions we've been making and it has delivered. Like do we talk 29 00:02:07,359 --> 00:02:09,800 to Amber Strong about that, like this was one of the site and she's 30 00:02:09,879 --> 00:02:15,400 just like, Okay, there are things we're finding that cannot be there. 31 00:02:15,520 --> 00:02:20,439 There's something we don't know and it's a big something. And Vishwaz is looking 32 00:02:20,479 --> 00:02:23,360 at us like did I just parachute in here from another planet, land on 33 00:02:23,360 --> 00:02:27,360 the right show anyway, don't get me started. Oh wait, you did, 34 00:02:27,960 --> 00:02:30,159 I did. I'm sorry you late. I can't wait. I can't 35 00:02:30,159 --> 00:02:35,360 wait, I can't wait. So uh yeah, well let's get started with 36 00:02:35,400 --> 00:02:46,280 better no framework awesome? All right. Well I've talked about this show I've 37 00:02:46,280 --> 00:02:50,960 done with Brian McKay, the a iBOT Show before we had them on. 38 00:02:51,240 --> 00:02:57,360 Yes, yeah, and we have just started getting into code and using c 39 00:02:57,560 --> 00:03:02,319 sharp with the API to do you know, the chaining, you know, 40 00:03:02,439 --> 00:03:08,319 get the results of one prompt chain it into another. But we attempted to 41 00:03:08,360 --> 00:03:12,719 do one last week and by last week, I mean you know a few 42 00:03:12,719 --> 00:03:15,479 weeks ago, I think it was the twenty fifth of October, we recorded 43 00:03:15,919 --> 00:03:21,960 with a guest and for some reason Riverside FM, which is what we record 44 00:03:22,039 --> 00:03:29,840 with, decided to completely trash the are not finish uploading the screen share on 45 00:03:29,919 --> 00:03:31,960 top of that, so we're going to record it again. That's that's the 46 00:03:32,719 --> 00:03:36,520 thing. So the episode eleven, watch for that because it's going to be 47 00:03:36,560 --> 00:03:44,719 amazing. We're basically trying to break the or push GPT four V image recognition 48 00:03:45,080 --> 00:03:46,639 to its limits. Oh wow, that's what we're trying to do. But 49 00:03:47,120 --> 00:03:52,360 in the course of doing that, Brian actually got some results and we're going 50 00:03:52,400 --> 00:03:54,719 to rerecord it. So I don't know what's going to happen the second time 51 00:03:54,719 --> 00:03:59,840 of the first time. First time he got results that he didn't expect, 52 00:04:00,039 --> 00:04:05,759 and he tried these same prompts with the same images and got completely different responses. 53 00:04:06,199 --> 00:04:11,759 And it turned out either somebody's watching or it's just learning, Like, 54 00:04:12,360 --> 00:04:15,120 no, that wasn't a good response, Like there might be something else looking 55 00:04:15,199 --> 00:04:20,279 at the responses and tuning the algorithm, like in a matter of twenty four 56 00:04:20,319 --> 00:04:24,800 hours. Well, there's also another possibility, which is that generate of AI 57 00:04:24,959 --> 00:04:28,959 models have a variability that allow for a range of answers. Yeah, and 58 00:04:29,040 --> 00:04:32,560 I kind of got into it with him and his guest as well. You 59 00:04:32,600 --> 00:04:39,920 know, the whole idea of do these large language models reason, And my 60 00:04:40,079 --> 00:04:44,560 stance right now is no, they don't. And you know, when presented 61 00:04:44,600 --> 00:04:46,800 with the facts, this is what they do. They take this, they 62 00:04:46,800 --> 00:04:50,519 take this. How is that not reasoning? My position is well, because 63 00:04:50,519 --> 00:04:55,680 it gives you inconsistent answers. Well, more partly it's just a statistical model, 64 00:04:55,759 --> 00:04:58,560 right, Like, there's no reason to be had. It's not weighing 65 00:04:58,600 --> 00:05:02,240 any factors at all. Gives a statistical result, right and and you know, 66 00:05:02,319 --> 00:05:06,079 but if you want to just look at it empirically, when you ask 67 00:05:06,240 --> 00:05:11,879 somebody what's one plus one and on Tuesday they say two and on Thursday they 68 00:05:11,920 --> 00:05:15,040 say nine, Well you asked the same question over and over again, it'll 69 00:05:15,040 --> 00:05:17,439 give you different answers like yeah, that's it. Yeah, and so there 70 00:05:17,480 --> 00:05:20,319 therefore you can't count on it. It's not reliable and it's not what you 71 00:05:20,319 --> 00:05:25,120 think it is. You don't ask You don't ask a sarcastic parrot for facts. 72 00:05:25,120 --> 00:05:30,360 That's not what it's meant for. I'm sure Vishwas has lots of things 73 00:05:30,360 --> 00:05:33,959 to say on this, but anyway, tune into the Aibot show dot com. 74 00:05:34,319 --> 00:05:39,800 Uh. Starting at about episode nine, I think is when we started 75 00:05:39,839 --> 00:05:44,079 using code. But in episode eleven is where we start really pushing the boundaries 76 00:05:44,079 --> 00:05:46,519 with GPT four v Love it, don't learn it, Love it Cool. 77 00:05:46,519 --> 00:05:49,399 Boo's Talkers Today, Richard grabbed a comment of a show eighteen twenty nine when 78 00:05:49,439 --> 00:05:54,600 we did just earlier this year with our friend Vishwaz when we were talking about 79 00:05:54,639 --> 00:05:58,160 fusion development, which apparently is a term that's now fallen out of favor. 80 00:05:58,680 --> 00:06:02,439 But this is about the idea that the power platform presents an awesome way to 81 00:06:02,480 --> 00:06:08,720 build heetrically need as clients against cloud back ends, and that there's a role 82 00:06:08,839 --> 00:06:12,920 for traditional net developers in there and building some of those back end services and 83 00:06:12,920 --> 00:06:15,040 even building client side components. And that was the idea of Fusion, that 84 00:06:15,079 --> 00:06:18,399 we can all work together. I don't know how else caught on with folks, 85 00:06:18,560 --> 00:06:21,519 but not that hip. But we did get a comment off that show 86 00:06:21,519 --> 00:06:26,480 from Jim, who I believes are regular He said, every time I hear 87 00:06:26,560 --> 00:06:29,480 someone talk about how easy it is to build with power apps for a power 88 00:06:29,600 --> 00:06:31,839 user, I shake my head and I have to think to myself that we've 89 00:06:31,879 --> 00:06:36,120 lost something. In the past twenty years. I went from it support dude 90 00:06:36,120 --> 00:06:40,639 with no real software development training to a software development leader, in large part 91 00:06:40,680 --> 00:06:45,680 because of visual Basic for applications, which made it easy to learn to program. 92 00:06:46,040 --> 00:06:49,120 We didn't need a solution that put hard to maintain Excel formulas into web 93 00:06:49,120 --> 00:06:53,519 app. We had a problem that needed to get solved and get back quickly 94 00:06:53,600 --> 00:06:58,000 to developing more apps. We need tools that get us to the equivalent of 95 00:06:58,000 --> 00:07:00,959 the late nineties and early two thousand desktops development experience, not things that try 96 00:07:01,000 --> 00:07:04,319 and mimic access. I mean, it's one click to create a project that 97 00:07:04,319 --> 00:07:09,040 I could start dropping UI elements into and then writing clean cut with a minimum 98 00:07:09,040 --> 00:07:13,199 of ceremony, not hiding logic behind textbox dates out of textboxes. And I'm 99 00:07:13,199 --> 00:07:15,839 going to turn off my rant before it turns into a blog post now. 100 00:07:15,920 --> 00:07:19,720 I mean, Jim, I would say you're misremembering some stuff too, like 101 00:07:20,040 --> 00:07:23,519 first thing that I came to appreciate. I said this the other day on 102 00:07:23,560 --> 00:07:26,839 another show, where I said, I think wind Forms is the exception. 103 00:07:28,879 --> 00:07:32,399 It's the only product that ever had a great designer. Every other attempt to 104 00:07:32,399 --> 00:07:38,000 make a great designer look at you, front Page or dream Weaver, much 105 00:07:38,079 --> 00:07:42,800 less WPF. Every other attempt to make a great designer has fallen on hard 106 00:07:42,879 --> 00:07:46,160 times. Only wind Forms had a great designer. There is a reason for 107 00:07:46,199 --> 00:07:50,879 that. The reason is wind Forms is a pixel based grid, and all 108 00:07:50,920 --> 00:07:57,800 of these other things are dynamic in don't use yeah, and that's why can't 109 00:07:57,920 --> 00:08:01,079 You can't have a visual designer when you're not pixel based. Well, and 110 00:08:01,120 --> 00:08:03,519 the downside to the pixel based is then you go to higher risk monitors and 111 00:08:03,560 --> 00:08:07,560 oops, or you want to work at mobile oops, or you want to 112 00:08:07,720 --> 00:08:11,720 scale. Yeah, all of those problems are introduced, and so that's why 113 00:08:11,759 --> 00:08:15,360 we have this one draggy, droppy designer that people fell in love with back 114 00:08:15,399 --> 00:08:20,480 in the day. Yeah, but the idea that that vb ever built clean 115 00:08:20,519 --> 00:08:24,079 code of any kind and with minimum ceremony, Like it was good at getting 116 00:08:24,120 --> 00:08:26,399 stuff done, but boy it was. You had to practice carefully to make 117 00:08:26,439 --> 00:08:31,000 maintainable projects. Yes you did. It's a bit of a nostalgic effect to 118 00:08:31,000 --> 00:08:35,080 think all things were so much simpler them. Well they were because we had 119 00:08:35,159 --> 00:08:39,720 VGA displays, one type of machine to deal with, and you know, 120 00:08:39,879 --> 00:08:41,519 it more or less work that you were writing code on the same kind of 121 00:08:41,559 --> 00:08:45,480 computer you were going to run it on. Like life was simpler in that 122 00:08:45,559 --> 00:08:48,200 respect. You know. I had a flashback to the nineties when I had 123 00:08:48,200 --> 00:08:52,679 my little kids and we used to play DVD games right on the computers of 124 00:08:52,720 --> 00:08:56,360 the day. Dug out some of those DVDs, you know, like played 125 00:08:56,399 --> 00:09:01,360 them, brought them up, and I was thinking, oh, this is 126 00:09:01,360 --> 00:09:03,240 going to be great. You know, we get to do these cartoony mathey 127 00:09:03,320 --> 00:09:07,519 things, right. Yeah, it's like the size of a postage stamp on 128 00:09:07,600 --> 00:09:09,679 my screen. Your screens just a little bit higher resolution now, yeah, 129 00:09:09,720 --> 00:09:13,200 and you blow it up and it does not look good, you know. 130 00:09:13,279 --> 00:09:16,159 For the for the history of dot net, I have all of those great 131 00:09:16,159 --> 00:09:20,559 slides of the conferences from two thousand, like the original pds A tech ed 132 00:09:20,600 --> 00:09:24,200 when dot net was first announced, and like Mixo six and so forth. 133 00:09:24,559 --> 00:09:28,840 You know, I had to get all of those graphics remade because they were 134 00:09:28,879 --> 00:09:33,600 like three twenty two eighty yeah, six hundred like back then, screens weren't 135 00:09:33,600 --> 00:09:37,879 the same resolution, and they they looked terrible. Tretty tramble on eighty ten 136 00:09:37,919 --> 00:09:39,679 eighty. So I literally sent copies of them to designers, said remake these 137 00:09:39,720 --> 00:09:43,440 as ten eighty for me. So I have the only ones. I have 138 00:09:43,480 --> 00:09:46,279 the only good looking mix O six graphic, Like, it's me. I 139 00:09:46,399 --> 00:09:50,200 made it, it's mine. That's very cool. Anyway, Jim, thank 140 00:09:50,240 --> 00:09:52,600 you so much for your comment and a copy of music Coobu. It's on 141 00:09:52,639 --> 00:09:54,679 its way to you. And if you'd like a copy of Musiccobeu, I 142 00:09:54,759 --> 00:09:58,679 write a comment on the website at dot net rocks dot com or on the 143 00:09:58,679 --> 00:10:01,399 facebooks we publish every show there and if you comment there and I read it 144 00:10:01,399 --> 00:10:03,720 on the show, we'll send you copy of us to go by. And 145 00:10:03,759 --> 00:10:05,919 you know, you can follow us on Twitter if you like to, or 146 00:10:05,120 --> 00:10:09,440 X or whatever the hell it is today. It's going to be Z tomorrow, 147 00:10:09,480 --> 00:10:15,440 I'm sure. But yeah, we're still there, but the cool kids 148 00:10:15,480 --> 00:10:20,000 are hanging out. I'm mastadon from from my perspective anyway, I'm at Carl 149 00:10:20,000 --> 00:10:24,440 Franklin at tech hubs Social, and I'm Rich Campbell at Macedon dot social. 150 00:10:24,600 --> 00:10:28,360 Yeah, send us a two. Maybe you'll get a copy of music to 151 00:10:28,360 --> 00:10:31,600 go buy. There you go. It's another way that you can leave a 152 00:10:31,600 --> 00:10:37,919 comment. And that brings us to our good friend Vishwas. Vishwa Leiley has 153 00:10:37,080 --> 00:10:43,879 been on the show some thirteen or fourteen times. He is an Azure an 154 00:10:43,879 --> 00:10:48,279 AI expert. His bio is long, but I won't be labored that here. 155 00:10:48,279 --> 00:10:52,799 Uh he's a real great guy as well. Welcome back, Vishwas. 156 00:10:52,039 --> 00:11:00,200 Thank you what you've been up to, what's doing well? Doing the Like 157 00:11:00,240 --> 00:11:07,320 everybody else, got started with chagpt open ai, Azure open Ei. Specifically 158 00:11:07,840 --> 00:11:11,799 for asure developers like me. It was an easy sort of way to get 159 00:11:11,840 --> 00:11:16,840 into generative AI I've been working on some aim al stuff previously, but generative 160 00:11:16,840 --> 00:11:24,399 AI, of course has changed everything that we knew before. Now you're in 161 00:11:24,480 --> 00:11:26,639 watching BC I think. I think a lot of your customers or governments like, 162 00:11:28,320 --> 00:11:33,000 aren't they pretty careful about adopting new technologies. I'm always impressed that you're 163 00:11:33,039 --> 00:11:37,679 in these things so early on. Yeah. So, yes, Richard, 164 00:11:37,759 --> 00:11:41,919 that's true. But I do work with government and commercial customers both. You 165 00:11:41,960 --> 00:11:46,120 know, the the idea is that can we can we try out some solutions 166 00:11:46,120 --> 00:11:50,360 in the commercial world bring those some of those solutions to the government community. 167 00:11:50,879 --> 00:11:56,039 So that's that's been the pattern. But you'll be surprised the government customers. 168 00:11:56,120 --> 00:11:58,960 And you know, Microsoft has Azure Open the Eye service that was fed ramped, 169 00:12:00,600 --> 00:12:03,919 right, so it has available in a certain secure setting. That was 170 00:12:03,919 --> 00:12:09,240 an announcement that came a few months ago. So there are civilian agencies who 171 00:12:09,399 --> 00:12:18,320 have lots of needs for automation of processes and processing of documents, and they're 172 00:12:18,320 --> 00:12:24,399 definitely looking at generative AI models to be able to expedite some of that work. 173 00:12:24,559 --> 00:12:28,879 Well, I know the government is currently sharpening its clause to to put 174 00:12:28,919 --> 00:12:33,799 some restrictions on AI even sort of be fuddled about this whole thing from the 175 00:12:33,879 --> 00:12:37,600 very beginning, and I'm not so sure how well they understand it. But 176 00:12:39,519 --> 00:12:43,799 clearly there's some legislation needs to happen, especially around copyright protection and that kind 177 00:12:43,840 --> 00:12:48,799 of stuff licensing. But what are your thoughts on that. Have you heard 178 00:12:48,799 --> 00:12:52,559 anything being in DC about what they've planned to do? Well, you know, 179 00:12:52,679 --> 00:12:56,120 like you, there's been news that White House is preparing for a major 180 00:12:56,120 --> 00:13:00,559 announcement. I don't know if it's a good thing or a bad thing for 181 00:13:01,320 --> 00:13:05,519 You're right, I think we do need some guidance there, but I always 182 00:13:05,559 --> 00:13:13,799 worry about what kind of guidance and will impede the amazing amount of overall. 183 00:13:13,919 --> 00:13:16,159 I mean, they are negatives and there are positives, like everything else. 184 00:13:18,279 --> 00:13:22,080 I do think that this is just an amazing development that we will look back 185 00:13:22,120 --> 00:13:26,200 five years from now and say, oh, something new started in twenty twenty 186 00:13:26,240 --> 00:13:33,360 three. So I'm hoping that the legislation that they're coming up with, I'm 187 00:13:33,360 --> 00:13:41,600 hoping with the industry leaders is a responsible step but does not curtail the progress 188 00:13:41,639 --> 00:13:45,559 and the amazing possibilities that we see from this technology. Yeah, when I 189 00:13:45,600 --> 00:13:50,159 think about the logical roles, it's like it would be good to have some 190 00:13:50,200 --> 00:13:54,080 privacy legislation. You've been needing that for ages, you know, large language 191 00:13:54,120 --> 00:13:58,240 models notwithstanding. But to me, the one area that you want from a 192 00:13:58,240 --> 00:14:01,840 public perspective care about is source of data full stop. If you're going to 193 00:14:01,840 --> 00:14:05,799 put make an LLM available to the public, even for a fee, you 194 00:14:05,799 --> 00:14:09,440 have to tell the where you get your data from. That's an easy one, 195 00:14:09,519 --> 00:14:13,080 and it's and you know, there's nothing profound there. It's not a 196 00:14:13,120 --> 00:14:16,919 secret. Presumably, it certainly deals with it starts to tackle the cocky right 197 00:14:16,960 --> 00:14:18,960 issue, but it's just like, hey, you know, you want to 198 00:14:18,960 --> 00:14:22,200 compete, make distinctions between different lms, talk about source of data, right, 199 00:14:22,320 --> 00:14:26,159 So, Richard, I think source of data. And there's been a 200 00:14:26,159 --> 00:14:30,519 debate about you know, what do open source models mean? Right? Is 201 00:14:30,559 --> 00:14:33,960 it just publishing the weights of your model, Does it make it open source? 202 00:14:33,679 --> 00:14:39,720 Or does announcing to the world what sources that your models were trained on. 203 00:14:41,080 --> 00:14:43,720 Yeah, that's the definition of open source. So you're absolutely right. 204 00:14:45,480 --> 00:14:50,039 These companies have to at some point tell us what are the source of data. 205 00:14:50,080 --> 00:14:56,200 But that's different from citations. Once the response is being generated at a 206 00:14:56,240 --> 00:15:00,879 sentence level, that's that's a very hard problem to do to spid well. 207 00:15:00,919 --> 00:15:03,159 And that's what you really need, right, I mean, nobody wants a 208 00:15:03,440 --> 00:15:11,080 you know, fifty terrorbyte list of sources, general sources that it's been trained 209 00:15:11,120 --> 00:15:13,120 on. But what I want to know is when it tells me something, 210 00:15:13,399 --> 00:15:16,720 where did you get that fact? You know? But as you said, 211 00:15:16,759 --> 00:15:22,600 it's very hard because it's a statistic model that predicts the next word that's going 212 00:15:22,679 --> 00:15:24,320 to come. How does it know it? Arguably, it's not pulling from 213 00:15:24,360 --> 00:15:28,399 any one source. It's the points of the data that it processed against. 214 00:15:28,559 --> 00:15:33,000 Right, Yes, that is true. We can sort of break my mind, 215 00:15:33,039 --> 00:15:37,759 break the generative way technology into two parts. The ability to comprehend what 216 00:15:37,799 --> 00:15:41,240 you're saying. You know, so it's throwing it a prompt and it is 217 00:15:41,279 --> 00:15:45,279 trying to comprehend and trying to give you an answer. And the second part 218 00:15:45,320 --> 00:15:48,279 is the general knowledge that it has captured through the training process. Right. 219 00:15:48,320 --> 00:15:52,440 If you think of I think of generative Way in two terms, right from 220 00:15:52,559 --> 00:15:56,200 and I think we're going to talk get into this more. I look at 221 00:15:56,240 --> 00:16:02,159 it from the perspective of how can generate your way I make me a better 222 00:16:02,200 --> 00:16:07,679 developer? How can I solve everyday problems? That are there in the businesses 223 00:16:07,720 --> 00:16:11,200 in it. How can I solve that? So from my perspective, I 224 00:16:11,320 --> 00:16:17,399 trend to break, like I said, into two parts, the part about 225 00:16:17,440 --> 00:16:21,639 comprehending and sort of listening and reasoning, and then part about general knowledge. 226 00:16:22,159 --> 00:16:26,159 I tend to leave the general knowledge part out if I can. When I'm 227 00:16:26,159 --> 00:16:30,039 asking for a prompt or asking for a response, I'm going to provide you 228 00:16:30,080 --> 00:16:33,440 a context and you should only reply based on the information that I am providing 229 00:16:33,480 --> 00:16:38,639 you commonly referred to as the RAG pattern, and that is very important to 230 00:16:38,720 --> 00:16:45,480 me. So Richard, your question about sources and citation is very important. 231 00:16:45,519 --> 00:16:49,720 But in the kind of solutions that I'm building, I am providing it a 232 00:16:49,759 --> 00:16:55,279 context and saying I'm going to give you these ten pages of information and you 233 00:16:55,399 --> 00:16:59,559 need to process that, and that way, the hallucination and the accuracy is 234 00:16:59,639 --> 00:17:03,240 much high to begin with, because I'm controlling the source. I agree, 235 00:17:03,519 --> 00:17:08,680 and I think that code in general, you know, code generation or help 236 00:17:08,759 --> 00:17:15,839 with coding is where AI. You know, this large language model absolutely shines. 237 00:17:15,119 --> 00:17:19,599 Sure, and nobody's going to complain that, oh, that algorithm was 238 00:17:19,640 --> 00:17:26,160 lifted out of my open source project. Well it probably wasn't it was probably 239 00:17:26,279 --> 00:17:32,359 put together just by statistics, but but I've found it. That's that's where 240 00:17:32,400 --> 00:17:37,720 it really shines. And that's not where people are complaining. People are complaining 241 00:17:37,759 --> 00:17:42,400 because it's taking their art, artwork and appropriating it. You know, artists, 242 00:17:42,480 --> 00:17:45,599 like all the artists are crying about this, right, you know, 243 00:17:45,920 --> 00:17:51,559 and this is why people don't trust it. But as you know, anyone 244 00:17:51,599 --> 00:17:56,759 who's writing complex documents that are in or code that are in a specific as 245 00:17:56,799 --> 00:18:02,240 you said, context, right, that's where it really really shines. And 246 00:18:03,200 --> 00:18:07,400 before I stop talking here, the other things that I like about it are 247 00:18:07,720 --> 00:18:15,359 the new chat GPT being web interface, so instead of using the data that 248 00:18:15,400 --> 00:18:22,599 it's trained on, you can do essentially what is a web search, but 249 00:18:22,039 --> 00:18:26,359 it goes out, finds multiple sources, puts things together, and if it 250 00:18:26,400 --> 00:18:32,839 finds the answer, it sort of gives you a nice concise answer instead of 251 00:18:32,880 --> 00:18:37,319 having to scrape it out of different sites. And also if it can't find 252 00:18:37,319 --> 00:18:41,039 an answer, it'll say, you know what, I searched all over the 253 00:18:41,039 --> 00:18:45,880 web and I couldn't find that thing that you're looking for. Maybe it doesn't 254 00:18:45,920 --> 00:18:48,799 exist, and maybe here's some documentation that you can go look at and see 255 00:18:48,839 --> 00:18:52,680 if you can figure it out yourself. Yeah, that integration because all of 256 00:18:52,680 --> 00:18:56,960 these language models have a knowledge cutoff date December twenty twenty one or some date 257 00:18:56,119 --> 00:19:00,599 like that. So if you ask a question that goes beyond that, of 258 00:19:00,640 --> 00:19:03,160 course, having this big integration where you can do go do a web search, 259 00:19:04,200 --> 00:19:07,680 you could do that. I guess with ch AGPT too, with some 260 00:19:07,759 --> 00:19:11,720 plug ins, but beings sort of gives you that. What I've found is 261 00:19:11,759 --> 00:19:15,839 it generally works, but there are there are situations where the citations are not 262 00:19:15,960 --> 00:19:21,880 complete, and that merging of real time look up versus what the model has 263 00:19:21,920 --> 00:19:26,200 been trained on does not work as seamlessly. You're right as you were copect, 264 00:19:26,480 --> 00:19:30,480 and especially for things like c sharp like you're better off using the default 265 00:19:32,319 --> 00:19:36,880 model that it was trained on to to do some code figure you know, 266 00:19:37,400 --> 00:19:41,359 right, a routine that does X, then you are using the bang stuff. 267 00:19:41,599 --> 00:19:45,000 The other thing I like about it now is I can say, here's 268 00:19:45,000 --> 00:19:48,960 a u r L to a PDF, digest this, and then tell me 269 00:19:48,039 --> 00:19:52,720 why, you know, my fridge doesn't work. That's right, and it 270 00:19:52,759 --> 00:19:56,200 will actually do a good job of diagnosing, And you can take pictures and 271 00:19:56,359 --> 00:20:00,720 upload it and it'll understand what it's looking at in the picture most of the 272 00:20:00,799 --> 00:20:03,519 time. This is what we're finding out anyway, cal that's a very good 273 00:20:03,519 --> 00:20:07,559 example. Give it a fifty or one hundred page document and let it read 274 00:20:07,599 --> 00:20:11,279 it, and then have a conversation with it, and say, because you 275 00:20:11,279 --> 00:20:15,400 know this, this fifty page document presumably is talking about different problems with the 276 00:20:15,480 --> 00:20:18,559 fridge. You don't want to go through that. You can say, Okay, 277 00:20:18,599 --> 00:20:22,519 now that you've read it, I'm going to tell you two symptoms that 278 00:20:22,599 --> 00:20:26,839 I'm seeing with the problem that I'm having, and can you narrow it down? 279 00:20:26,880 --> 00:20:30,200 And then having a conversation with that. That's just amazing. It is 280 00:20:30,240 --> 00:20:33,279 amazing. That's aig. It speaks to the strength of this model. It 281 00:20:33,400 --> 00:20:38,079 is a good summarizer to be able to ream through a lot of information. 282 00:20:38,279 --> 00:20:44,640 And plus you can validate you know it got it from those hundred pages. 283 00:20:44,680 --> 00:20:48,359 So once it gives a statement like you should do this, you can take 284 00:20:48,400 --> 00:20:52,920 that phrase and search in that document and probably find exactly the reference you need 285 00:20:52,680 --> 00:20:56,400 to say, yeah, the document really does say this. I found I 286 00:20:56,440 --> 00:21:00,720 did this once in my amplifier and it actually gave me the answer and said 287 00:21:00,759 --> 00:21:04,839 on page fifty three in this section here's what it says. So it actually 288 00:21:06,000 --> 00:21:10,000 told me where in the document the answer lied. And this was a real 289 00:21:10,039 --> 00:21:11,759 experiment that I did. And I had a real problem with a piece of 290 00:21:11,799 --> 00:21:15,799 gear, and I literally told her to read the PDF of the manual and 291 00:21:17,119 --> 00:21:21,359 it in like five seconds gave me the answer the challenge that you know, 292 00:21:21,480 --> 00:21:23,920 let's say if your one hundred page PDF document, either it can read it 293 00:21:25,000 --> 00:21:27,240 completely, that's one thing. But you know what if you have thousands of 294 00:21:27,440 --> 00:21:32,079 fifty page documents, Yeah, then you're trying to sort of come up with 295 00:21:32,119 --> 00:21:36,960 an answer, and you're eventually going to run into a limit. Even though 296 00:21:37,000 --> 00:21:40,359 GBT four gives you thirty to two K models, which is what about two 297 00:21:40,400 --> 00:21:42,759 hundred and fifty pages or so what if you had more pages? And then 298 00:21:42,799 --> 00:21:47,559 now you can't feed that into the model, right, and now you have 299 00:21:47,599 --> 00:21:49,559 to do some sort of a search, a vector search of some sort, 300 00:21:49,880 --> 00:21:55,240 get the right content in and pass that as a context you prompt you said 301 00:21:55,279 --> 00:22:00,759 the V word vector databases. I find these baffle and I'm not so sure 302 00:22:00,799 --> 00:22:04,519 that you know, Joe web user is going to be able to figure them 303 00:22:04,559 --> 00:22:08,039 out. Can you briefly explain what they are? And sure, sure, 304 00:22:08,119 --> 00:22:11,599 sure, and I shouldn't have used the words without explaining it first. No, 305 00:22:11,720 --> 00:22:17,039 of course you should. How are we going to ask you questions? 306 00:22:17,519 --> 00:22:22,440 I mean, yeah, what's our role here? No? True? So 307 00:22:22,839 --> 00:22:27,079 let's take a step back. So you started out with a great example of 308 00:22:27,279 --> 00:22:32,839 you know, having this great fifty page PDF about your amplifier and it just 309 00:22:33,319 --> 00:22:36,839 you. You had the model read that completely, and you were able to 310 00:22:36,880 --> 00:22:44,000 do that because that fifty pages fit into the size of that that the model 311 00:22:44,000 --> 00:22:48,640 supports, which is thirty two. Right. What if you had thousands of 312 00:22:48,680 --> 00:22:53,519 fifty page documents and you're trying to build an answer. You can't feed all 313 00:22:53,599 --> 00:22:56,720 of those to the models, at least not now. Maybe GPT five GPT 314 00:22:56,799 --> 00:23:00,319 six would allow us to do that, who knows. So in the absence 315 00:23:00,359 --> 00:23:03,640 of that, what you have to do is you have to go do a 316 00:23:03,680 --> 00:23:10,880 search against those thousand documents. And the problem is that if you did some 317 00:23:11,000 --> 00:23:15,960 keyboard search, as we've been used to doing, right, it is not 318 00:23:15,200 --> 00:23:22,000 that keyboard search is really not conveying the intent of what you're trying to search. 319 00:23:22,079 --> 00:23:25,559 You might search for some words and you might get good hits, no 320 00:23:25,680 --> 00:23:29,160 question about it. But what if you're trying to just describe the problem. 321 00:23:29,240 --> 00:23:33,400 You're just saying, my amplifier works this way in certain conditions, but I'm 322 00:23:33,440 --> 00:23:37,799 noticing these problems. I can guarantee you there will be no keyboard hits when 323 00:23:37,799 --> 00:23:42,599 you write a statement like that. But in the you're describing the problem. 324 00:23:44,160 --> 00:23:48,200 And what if you were able to take this two word two lines worth the 325 00:23:48,440 --> 00:23:56,279 description and find a paragraph of texts that closely resembles this text. And that's 326 00:23:56,279 --> 00:23:59,400 what a vector database is. What we are trying to do, is we're 327 00:23:59,480 --> 00:24:07,519 trying to take every piece of text in that large corpus of documents and we 328 00:24:07,559 --> 00:24:14,519 are assigning some statistical values to it so that we can place certain sentences in 329 00:24:14,559 --> 00:24:18,960 a certain manner. So think of every word having a certain number of attributes, 330 00:24:19,440 --> 00:24:22,880 and then words that are closely aligned will have similar number of attributes. 331 00:24:22,920 --> 00:24:27,720 Things like that. So now you take that sentence of your description of the 332 00:24:27,799 --> 00:24:33,000 problem, you create an embedding out of it, and you know, you 333 00:24:33,000 --> 00:24:37,039 can use any of the models to create an embedding, which is you're getting 334 00:24:37,039 --> 00:24:41,680 a mathematical representation of those two sentences. And then you are and this going 335 00:24:41,720 --> 00:24:47,319 back to high school math, you're doing a coscience search of this mathematical representation 336 00:24:47,960 --> 00:24:52,400 against the mathematical representation of the entire corpus. Maybe you get two or three 337 00:24:52,519 --> 00:24:57,319 hits which generally talk about this problem, and then you bring that text back 338 00:24:57,359 --> 00:25:03,640 and only send that text to the model because now you've compressed that information. 339 00:25:03,240 --> 00:25:08,319 So that's if you're able to do that, then you can get answers to 340 00:25:08,319 --> 00:25:14,400 the question even if you have one thousand documents. And I'll talk about later 341 00:25:14,440 --> 00:25:19,519 on the product or application that I'm working on, which basically uses the pattern 342 00:25:19,559 --> 00:25:23,160 that I just talked about. I mean, we don't have enough time for 343 00:25:23,200 --> 00:25:27,519 you to explain this, but in general, how does one go about creating 344 00:25:27,880 --> 00:25:36,359 a vector database from a tome of documents? Right? So you know, 345 00:25:36,599 --> 00:25:41,359 let's just take the simple example of you those two lines of description of your 346 00:25:41,400 --> 00:25:47,799 problem. Right, you can go to any number of embedding models that are 347 00:25:47,839 --> 00:25:52,680 available. You can say, hey, please give me a vector representation of 348 00:25:52,720 --> 00:25:56,359 these three lines of text, and it literally gives you back a vector representation 349 00:25:56,440 --> 00:26:03,119 of those three lines of text. So, because these models have been trained, 350 00:26:03,160 --> 00:26:08,119 like I said, you play this prediction game what comes next? What 351 00:26:08,279 --> 00:26:14,200 comes next? You play this prediction game, and by playing this game, 352 00:26:14,319 --> 00:26:19,279 millions and billions of times you have assigned a certain attribute to a certain word 353 00:26:19,359 --> 00:26:25,599 in a certain phrase of that sentence. And then by placing these values in 354 00:26:25,640 --> 00:26:29,480 a multi dimensional space, you can say these words together mean something, These 355 00:26:29,519 --> 00:26:33,839 words together mean something. And you know, we all remember the twenty fourteen 356 00:26:33,920 --> 00:26:37,039 word to wac paper that Google came out with, you know, twenty fourteen, 357 00:26:37,039 --> 00:26:42,079 and we were so amazed when it could do simple things like king minus 358 00:26:42,119 --> 00:26:47,920 man plus woman equals what oh queen, And we were so happy that you 359 00:26:47,920 --> 00:26:52,240 know, these models have now some semantic understanding of English language. Right that 360 00:26:52,400 --> 00:26:56,680 was twenty fourteen when the word to wac paper came out. And of course, 361 00:26:56,839 --> 00:27:03,079 you know, we've given the complexity of the models, the transformer model, 362 00:27:03,119 --> 00:27:04,920 the availability of GPUs. Now we can do so much more. We 363 00:27:04,920 --> 00:27:11,440 were embedding words literally, now we're embedding paragraphs and pages essentially, so that 364 00:27:14,480 --> 00:27:19,680 to your listeners, essentially you have taken your corpus of text and you've created 365 00:27:19,680 --> 00:27:26,480 a mathematical representation that tells a model. The model is not sentient, it 366 00:27:26,519 --> 00:27:30,839 does not know what it means. But because of mathematical representation, can take 367 00:27:30,839 --> 00:27:36,759 the problem description and say there are these four paragraphs out of these thousand documents 368 00:27:37,000 --> 00:27:41,680 that seem to be talking about something similar, and you go and pull those 369 00:27:41,720 --> 00:27:45,000 paragraphs, and then you go to your prompt and say, I'm going to 370 00:27:45,039 --> 00:27:52,000 give you four paragraphs worth of relevant information about my amplifier, and I'm going 371 00:27:52,039 --> 00:27:56,319 to give you some symptoms. Your job is to take that symptom and try 372 00:27:56,319 --> 00:28:02,960 to come up with a plausible answer for the problem that I'm having. That's 373 00:28:03,359 --> 00:28:08,200 that's essentially what's happened. I get that with the paragraphs that you generate for 374 00:28:08,279 --> 00:28:14,119 the prompt. But how do you know if I have thousands and thousands of 375 00:28:14,160 --> 00:28:18,920 pages of documents, is there like literally an app that I can just drag 376 00:28:18,960 --> 00:28:22,279 and drop those into and proof its a vector database or is there a lot 377 00:28:22,319 --> 00:28:25,480 of manual work on the part of the of the user. That's a great 378 00:28:25,559 --> 00:28:30,799 question. So Microsoft sixty sixty five Copilot is going to come out. The 379 00:28:30,839 --> 00:28:34,039 release data is November first or something like that, if I'm read correctly. 380 00:28:34,839 --> 00:28:40,119 So let's just use that as an example to answer your question. Card. 381 00:28:40,279 --> 00:28:45,480 Sure, so Microsoft is sixty five copilot would be Now you have inside Word, 382 00:28:45,559 --> 00:28:49,480 PowerPoint, Excel, whatever you will be able to ask. You can 383 00:28:49,519 --> 00:28:56,160 go into Word and say, hey, can you please generate me an invoice 384 00:28:56,640 --> 00:29:02,079 based on the last five invoices that I sent out? Okay, you'd be 385 00:29:02,079 --> 00:29:03,640 able to do something like this. How do you be able to do that? 386 00:29:03,680 --> 00:29:08,559 Well, maybe you have thousands of invoices stored on your one drive somewhere, 387 00:29:10,200 --> 00:29:15,599 and when you start training the semantic index, which is you know, 388 00:29:15,599 --> 00:29:18,240 which is a component of the Microsoft sixty sixty five copilot, what they would 389 00:29:18,279 --> 00:29:22,480 do is they would go through your one drive, open up every document, 390 00:29:23,160 --> 00:29:27,160 and most commonly this is what people are doing. Open up every document at 391 00:29:27,200 --> 00:29:32,759 the page break, Take the one page so or some fixed number of words 392 00:29:32,799 --> 00:29:37,680 or tokens. Read that from your document, create an embedding, store it. 393 00:29:40,039 --> 00:29:42,960 Take the next page, create an embedding using the AI model that I 394 00:29:42,960 --> 00:29:47,519 talked about. It could be any text model, and store it. And 395 00:29:47,839 --> 00:29:52,799 that's how Microsoft internally is creating the semantic index. Of course, cognitive search 396 00:29:52,000 --> 00:29:56,759 now has vector database built into it. Right, Cognitive search used to be 397 00:29:56,839 --> 00:30:03,039 text search, now has vector database built into it. So now you've got 398 00:30:03,079 --> 00:30:07,240 through this whole process of ingestion, which can take lots of hours, yes, 399 00:30:07,279 --> 00:30:10,759 because you could have thousands of documents. But now you've taken your entire 400 00:30:10,799 --> 00:30:15,000 corpus and broken it down into a set of embeddings. So I take it 401 00:30:15,039 --> 00:30:19,400 then this isn't something that Joe programmer would do with a bunch of data, 402 00:30:19,519 --> 00:30:23,119 like you need some sort of powerful tool on the back end to do this 403 00:30:23,279 --> 00:30:30,079 for you. This is something I'm time to understand your question. Maybe this 404 00:30:30,200 --> 00:30:33,680 is the time to sort of talk about the application I'm building and see if 405 00:30:33,680 --> 00:30:37,079 I answered that question. And I'll come back right all right, but hold 406 00:30:37,119 --> 00:30:41,480 that thought. We'll do that right after this break for these very important messages, 407 00:30:45,680 --> 00:30:48,359 and we're back. It's starting in Rocks and Carl Franklin. That's Richie 408 00:30:48,359 --> 00:30:52,279 Campbell, and that's vishwas Lele and he was just about to start telling us 409 00:30:52,279 --> 00:30:56,440 about this amazing application that he's working on. Please do go ahead, sir, 410 00:30:56,880 --> 00:31:00,640 thank you. So I don't know amazing, but there's an application that 411 00:31:00,880 --> 00:31:03,559 we've been trying to build the last six months that has taken up a lot 412 00:31:04,079 --> 00:31:08,640 of my time. So we talked about the Microsoft THY sixty five copilot. 413 00:31:08,720 --> 00:31:17,440 We are building a similar capability but to help proposal developers. So imagine you're 414 00:31:17,480 --> 00:31:22,759 getting a fifty one hundred page RFP or RFI or RFQ or something like that, 415 00:31:23,559 --> 00:31:29,960 and that's a very time consuming and expensive process. Right, wouldn't it 416 00:31:30,000 --> 00:31:33,279 be nice if you were able to take all your past performances so you're responding 417 00:31:33,279 --> 00:31:41,640 to these kinds of RFPs over years prior. Can you take those documents and 418 00:31:41,720 --> 00:31:45,920 create some sort of a vector database and then use that to generate a draft 419 00:31:47,079 --> 00:31:52,519 of a future RFP. So to going back to your question, Carl, 420 00:31:53,279 --> 00:31:57,440 as a developer, what we had to do was to write those ingestion routines 421 00:31:59,319 --> 00:32:05,559 where we take those past proposal responses that you may have submitted to the government 422 00:32:05,640 --> 00:32:08,799 or some agency or a state or local or what have you, and write 423 00:32:08,799 --> 00:32:15,119 that ingestion routine where we take that document split it up. Now, there 424 00:32:15,160 --> 00:32:19,680 is a lot of work that is happening which says, should you be breaking 425 00:32:19,720 --> 00:32:22,920 the document on the page boundary? Is that enough? Because think about it, 426 00:32:23,359 --> 00:32:30,200 if you just break it on a page boundary, and maybe the important 427 00:32:30,640 --> 00:32:35,319 idea or concept is transcending that page boundary, right, break it up by 428 00:32:35,359 --> 00:32:37,960 topics, right, break it up by topics, But then how do you 429 00:32:38,000 --> 00:32:45,119 break it up by topics table of contents. That's a good point, but 430 00:32:46,039 --> 00:32:52,640 sometimes you know, most in our case, at least the PDF documents that 431 00:32:52,680 --> 00:32:55,599 are usually make up these responses. The table of contents are not that great 432 00:32:55,640 --> 00:33:00,359 and PDF is not a great possible format in any case. Right, So 433 00:33:00,440 --> 00:33:06,519 you have to use in some cases language models to be able to even passe 434 00:33:06,599 --> 00:33:13,039 them out and to break the document down into some sort of a section in 435 00:33:13,160 --> 00:33:17,920 some classification format so that you can use it for embedding. So you store, 436 00:33:19,880 --> 00:33:22,000 maybe not do it on page boundaries, maybe do it on some sort 437 00:33:22,000 --> 00:33:27,839 of a lexical boundary. But then you also think about what if the table 438 00:33:27,880 --> 00:33:31,559 of contents has some sort of inheritance built into it, is it important for 439 00:33:31,599 --> 00:33:36,640 you to capture that inheritance because that might come in handy as you're trying to 440 00:33:36,680 --> 00:33:42,319 respond to a question. Right, So those are all important things to do. 441 00:33:44,079 --> 00:33:45,799 So going back to your question, Yes, as a developer, you 442 00:33:45,920 --> 00:33:52,440 have to think about where your embedding boundaries lie because when it comes time to 443 00:33:52,519 --> 00:33:58,880 generate content, the more precise context that you can provide to the language model, 444 00:33:59,119 --> 00:34:01,519 the better response are going to get. And the more precise context you 445 00:34:01,559 --> 00:34:07,480 can get by thinking deeply about how you're going to break your corpus into the 446 00:34:07,519 --> 00:34:09,840 right set of embeddings. Isn't Microsoft Copilt going to do a lot of this 447 00:34:09,920 --> 00:34:14,440 for us, Like, this is a solve problem if you just are all 448 00:34:14,440 --> 00:34:17,000 in on them three sixty five and pay whatever per month they're going to ask 449 00:34:17,039 --> 00:34:22,960 for. That's a great question. And let me tell you the difference is 450 00:34:23,039 --> 00:34:31,199 so Microsoft Phrisks have Copilot is going to be a general purpose answering engine, 451 00:34:31,440 --> 00:34:37,199 right, so you can ask it and say, hey, please draft me 452 00:34:37,840 --> 00:34:40,480 the text for an invoice based on the last five invoices, and I'm sure 453 00:34:40,519 --> 00:34:45,800 it'll do a great job. You've seen demos of that. It also has 454 00:34:45,840 --> 00:34:50,239 access to the graph, so you knows which people you're collaborating with, and 455 00:34:50,920 --> 00:34:57,199 it can take into account that. But if you have a complex writing problem 456 00:34:57,400 --> 00:35:01,320 where the questions are not single line questions, right, the RFP questions are. 457 00:35:01,360 --> 00:35:06,880 Sometimes the questions themselves are maybe a page long or half a page long. 458 00:35:07,440 --> 00:35:12,960 Right, how do you sort of create a prompt? So in case 459 00:35:12,960 --> 00:35:15,920 of Microsoft, there are sixty five you're essentially writing a prompt. You're saying, 460 00:35:15,920 --> 00:35:19,519 hey, genade mean invoice which is based on the past five invoices. 461 00:35:19,639 --> 00:35:22,519 Right, So much context in that statement that needs to be known that you 462 00:35:22,599 --> 00:35:28,119 have in mind. Right, in our case, the context is enormous, 463 00:35:28,239 --> 00:35:32,960 right, Richard, Because if you're trying to respond to a complex question, 464 00:35:34,960 --> 00:35:37,960 you have to infuse a lot of attributes into that. Right. You have 465 00:35:37,000 --> 00:35:45,199 to think about what are the pain points that the buyer is expressing in this 466 00:35:45,320 --> 00:35:49,320 description? What are the then themes? How am I the one that they 467 00:35:49,360 --> 00:35:53,199 should be selecting and not somebody else? Is the buyer cost conscious? Have 468 00:35:53,320 --> 00:35:55,840 we done this work? Like? You know, it's great to say that 469 00:35:55,880 --> 00:36:00,440 we are the greatest company it can do agile development, But want it be 470 00:36:00,519 --> 00:36:02,639 nice if you can provide some proof points that we have done this and we've 471 00:36:02,719 --> 00:36:07,039 had success. So I've just given you three examples. There are many such 472 00:36:07,079 --> 00:36:13,679 attributes, So when you are crafting a prompt, you need to take these 473 00:36:13,719 --> 00:36:19,559 considerations into account. So in case of sixty five copilot, you are writing 474 00:36:19,599 --> 00:36:23,760 the prompt, but in our situation you're writing a prompt that is far more 475 00:36:23,760 --> 00:36:30,039 complex. So we are trying to do some prompt engineering for on behalf of 476 00:36:30,079 --> 00:36:32,800 the user where they can tell us at a high level what they want to 477 00:36:32,840 --> 00:36:38,679 do with their response, and for us, it is very much about about 478 00:36:39,840 --> 00:36:44,480 That's why it's a copilot. It's not a pilot or an autopilot. You're 479 00:36:44,599 --> 00:36:49,320 leveraging the experience of experienced proposal writers. The experienced proposal writers who spent ten 480 00:36:49,400 --> 00:36:52,519 twenty thirty years writing this, they know exactly how you should be responding. 481 00:36:52,800 --> 00:36:58,719 We are simply augmenting them and providing the first draft. Right. All of 482 00:36:58,760 --> 00:37:02,960 the innovativeness of our engine will not help you win any proposals. Our engine 483 00:37:04,000 --> 00:37:07,559 will help you do sort of the mundane steps quickly so that you can spend 484 00:37:07,679 --> 00:37:13,920 more time adding innovation to your response. So you know, if you if 485 00:37:13,920 --> 00:37:17,039 you have to generate this prompt, we think that every user is not going 486 00:37:17,079 --> 00:37:22,159 to be able to generate those levels of prompts. And I'll tell you why, 487 00:37:22,280 --> 00:37:27,360 because as good as GBT is, if you give a certain size prompt, 488 00:37:27,440 --> 00:37:30,960 it starts forgetting some instructions that you give in the middle, right, 489 00:37:32,639 --> 00:37:37,239 so you have to repeat it. So we have to have sort of layers 490 00:37:37,280 --> 00:37:40,599 of prompts that go in. And this is why you know. The team 491 00:37:40,639 --> 00:37:44,880 that I'm working with has data scientists in them, and people ask me sometimes, 492 00:37:44,880 --> 00:37:47,760 hey, generative AI is pre trained, do I need a data scientist? 493 00:37:47,880 --> 00:37:52,559 Well, you need a data scientist if you're if you're crafting this sophisticated 494 00:37:52,599 --> 00:37:57,559 prompts on behalf of the user by taking some input from them. So, 495 00:37:58,280 --> 00:38:04,039 Richard, in that manner, we think that this is a very specialized example 496 00:38:04,559 --> 00:38:07,039 of what we are trying to use the language model, and this is sort 497 00:38:07,039 --> 00:38:10,920 of the difference with the M sixty five copilot, and there are a few 498 00:38:10,920 --> 00:38:16,880 other differences for us. Hallucination is very very important. Don't please don't go 499 00:38:16,960 --> 00:38:22,920 out and say that you did a five thousand SharePoint node migration when you've only 500 00:38:22,960 --> 00:38:25,559 done fifty or five hundred, right, that we'll be a problem. Right, 501 00:38:27,000 --> 00:38:34,519 So we do explicitly accuracy verification, and there are many techniques that your 502 00:38:34,599 --> 00:38:37,719 listeners may be aware of that. You know, you take the generator to 503 00:38:37,719 --> 00:38:42,960 response, you try to extract entities from that and lage language models. Right, 504 00:38:43,199 --> 00:38:46,400 So any definitive statements in there, yes, yes, and language models 505 00:38:46,400 --> 00:38:50,840 are great or extracting entities. And then you take those entities and then you 506 00:38:51,119 --> 00:38:53,760 do a search against your knowledge repository and say, do I see a five 507 00:38:53,840 --> 00:38:59,199 hundred node migration ever mentioned here? And if it is not, then let's 508 00:38:59,199 --> 00:39:02,159 just flag it and let a human proposal writer go check this. We're doing 509 00:39:02,199 --> 00:39:09,559 those kinds of things peer, if you I'll just just extend this by saying 510 00:39:09,599 --> 00:39:13,840 that I don't know if if your listeners have seen this study from MIT, 511 00:39:14,559 --> 00:39:16,920 where there are many studies out there which said, okay, let's just try 512 00:39:16,960 --> 00:39:22,360 to measure the productivity improvements or something like chat depety. Right, So what 513 00:39:22,400 --> 00:39:25,960 they did was they took four hundred people college degrees good writers, broke them 514 00:39:25,960 --> 00:39:30,760 into groups of into two groups two hundred each, gave them a writing assignment, 515 00:39:31,360 --> 00:39:36,199 and then people who were evaluating them had no idea whether they were using 516 00:39:36,199 --> 00:39:38,639 some tools or not. So they gave them on writing assignment where nobody was 517 00:39:38,679 --> 00:39:43,760 allowed to use any tools. Say, got the baseline. Then they give 518 00:39:43,800 --> 00:39:46,719 them another writing assignment, but this time half the group was only allowed to 519 00:39:47,199 --> 00:39:51,159 use chat deputy. The other group was not allowed to use chat jpety. 520 00:39:52,000 --> 00:39:55,800 And now once again the evaluators had no idea who was doing what. And 521 00:39:55,840 --> 00:39:59,960 if you look at the results, pretty amazing results. Right. The group 522 00:40:00,159 --> 00:40:06,400 that was using chat GPT completed their work in thirty or thirty three percent less 523 00:40:06,400 --> 00:40:09,039 time. And I can point you to the study. So my numbers may 524 00:40:09,039 --> 00:40:13,920 be off. But more importantly, more importantly, the people who are using 525 00:40:14,000 --> 00:40:19,719 chat GPT scored a thirty three or thirty five percent better grades than people who 526 00:40:19,760 --> 00:40:23,280 are not using Okay, so a third better grade in a third less time, 527 00:40:23,360 --> 00:40:27,639 third less time. I mean that's amazing. And then what was it 528 00:40:27,679 --> 00:40:31,639 fact checked? Because that's my problem. You might be able to split up 529 00:40:31,639 --> 00:40:37,800 beautiful text, but is it really right? Yes, and human evaluators evaluated 530 00:40:37,920 --> 00:40:42,440 that, right, It was just like you know, people testing that that's 531 00:40:42,480 --> 00:40:45,480 always a case, right, And we know that human evaluators make mistakes too 532 00:40:46,320 --> 00:40:50,840 as well well, and we don't know exactly what they were trying to write, 533 00:40:50,880 --> 00:40:53,320 like as soon as you get away from non factual writing, like just 534 00:40:53,400 --> 00:40:59,760 exploratory fiction based writing and stuff, it's not the same set of problems right, 535 00:41:00,239 --> 00:41:04,039 right, And you know what is interesting is that you know, remember 536 00:41:04,079 --> 00:41:07,760 I said that they were all given a task where nobody was allowed to use 537 00:41:07,760 --> 00:41:12,159 in external tools. So that was to establish a baseline, right. What 538 00:41:12,280 --> 00:41:17,880 is interesting in the study is that writer folks who did not have great writing 539 00:41:17,920 --> 00:41:23,400 ability, right, even they benefited from when they were put in a pool 540 00:41:23,400 --> 00:41:28,400 where they could use chatgputy. They benefited from. Their grades went up significantly. 541 00:41:28,480 --> 00:41:30,440 I mean I would argue they would benefit more, they would manafit more 542 00:41:30,559 --> 00:41:37,280 right. But you know, there's very very interesting thing that about that study 543 00:41:37,639 --> 00:41:40,559 that I'd like to bring back to your question, Richard. It was very 544 00:41:40,559 --> 00:41:44,440 interesting in the study that there's something called, and I'm going to use a 545 00:41:44,440 --> 00:41:46,800 fancy term that I did not know until I read this m study called the 546 00:41:47,320 --> 00:41:55,039 human machine complementarity. Okay. So what they did was they evaluated the grades 547 00:41:55,079 --> 00:41:59,760 at two levels. So the team that was remembered the team that was using 548 00:41:59,840 --> 00:42:05,519 cheat gpty. It took the output of chat gpt and graded it, and 549 00:42:05,599 --> 00:42:09,920 then they took the output where humans did some prompt engineering on top of chat 550 00:42:09,960 --> 00:42:16,079 GPT and generated an output and evaluated that too, okay, and they found 551 00:42:16,079 --> 00:42:20,679 that there was not much difference, right. So what is interesting is, 552 00:42:20,679 --> 00:42:24,599 so where is this human machine complementarity coming in? So people looked at the 553 00:42:24,639 --> 00:42:29,599 chat gepety output, made a few tweaks, and then they submitted it. 554 00:42:30,159 --> 00:42:37,760 So, tying it back to your question, Richard, my theory is that 555 00:42:38,440 --> 00:42:43,320 the reason there was not much different from the chat gept and the human machine 556 00:42:43,599 --> 00:42:49,519 output was people were not thinking carefully about the bronze and actually writing detailed prons, 557 00:42:49,920 --> 00:42:54,679 right, And we think that by creating these layers of prompt for them 558 00:42:55,159 --> 00:42:59,679 you can get a benefit on top of so everybody is going to get that 559 00:42:59,800 --> 00:43:02,159 third percent benefit. That's going to be at some point table stakes, right, 560 00:43:04,519 --> 00:43:08,440 was it? Steve Snowski I was reading a post from him recently said 561 00:43:08,480 --> 00:43:15,119 every textbox is going to be LLLM enabled soon and it's going to be like 562 00:43:15,239 --> 00:43:17,320 you start writing and it will just start giving you some stuff. So no, 563 00:43:19,159 --> 00:43:22,960 everybody is going to benefit from thirty percent or twenty eight percent or whatever 564 00:43:22,039 --> 00:43:27,840 that is, right, So how do you go beyond that? Well you 565 00:43:27,920 --> 00:43:32,480 go beyond that through sophisticated prompt engineering. So that's been my journey to build 566 00:43:32,519 --> 00:43:37,239 this tool. Take your past responses, and I have to say this. 567 00:43:37,679 --> 00:43:40,480 People ask me this all the time. This is not what we are building. 568 00:43:40,519 --> 00:43:44,199 Is not cheating as a service, right, I tell people, if 569 00:43:44,239 --> 00:43:46,800 you don't, if you don't have good past responses, if you don't have 570 00:43:46,840 --> 00:43:52,000 good architects who can come up with a good solution, if you don't have 571 00:43:52,280 --> 00:43:59,280 creative solutions, then what we produce will be pretty you know, poor and 572 00:43:59,840 --> 00:44:02,360 not something that you want. But if you have good past performances that you've 573 00:44:02,400 --> 00:44:07,480 done this SharePoint migration work and you can come up with a you have architects 574 00:44:07,480 --> 00:44:10,960 who can come up with a solution, then this is going to benefit. 575 00:44:12,039 --> 00:44:19,599 So really, from a responsibilii standpoint, it is about taking your knowledge and 576 00:44:19,639 --> 00:44:25,280 then generating that quickly and then accurately checking it that you're not hallucinating and saying 577 00:44:25,280 --> 00:44:30,159 you did five thousand not server migration, right. I love it as a 578 00:44:30,239 --> 00:44:35,800 service, chees. But also you know the old the old comside comment of 579 00:44:35,920 --> 00:44:39,760 gigo right of garbaging garbage out. Yes, like we still have data quality 580 00:44:39,800 --> 00:44:43,920 problems like this comes down to the quality the data you load the set with 581 00:44:44,039 --> 00:44:46,559 and how well you load it. But it is an automation tool, and 582 00:44:49,079 --> 00:44:52,760 all automations stuff from the same problem, which is they're only going to amplify 583 00:44:53,440 --> 00:44:57,000 what you tell it to amplify. And if you tell it to amplify your 584 00:44:57,039 --> 00:45:01,559 stupidity, it's going to amplify your stupidity. You're you're absolutely liter Richard my 585 00:45:02,320 --> 00:45:07,400 sort of in my experience in six months working on this project, it's clear 586 00:45:07,480 --> 00:45:14,239 that you spend a lot of time doing data wrangling. You talked about garbage 587 00:45:14,239 --> 00:45:16,559 and garbage out. Do you worry about what data is getting in? So 588 00:45:16,559 --> 00:45:22,320 so be prepared to do those kinds of things. Even though these models are 589 00:45:22,360 --> 00:45:28,280 great, But I would say one thing, these models are far more accepting 590 00:45:28,320 --> 00:45:36,079 of the mistakes. So I was just doing this example recently where you know, 591 00:45:36,119 --> 00:45:40,280 we had to calculate some accuracy or some data and we went out to 592 00:45:40,400 --> 00:45:46,719 three or four external services to go validate this data. And each of the 593 00:45:46,800 --> 00:45:51,480 services returned the data in a different format, right, one return in a 594 00:45:51,559 --> 00:45:55,320 certain Jason structure. Somebody gave us XML back, and previously I would have 595 00:45:55,400 --> 00:45:59,440 to write a lot of data wrangling code to sort of pars out that data, 596 00:45:59,599 --> 00:46:04,159 get this and then compare it sort of very format it's yeah, to 597 00:46:04,199 --> 00:46:07,400 get them into very imperative code, very much imperative code that I had to 598 00:46:07,480 --> 00:46:10,320 write, yeah, right, and then immediately throw away like you're only going 599 00:46:10,400 --> 00:46:15,440 to do throw it because it's going to And in case of language models, 600 00:46:15,480 --> 00:46:22,239 you can say, hey, I'm you know, I'm interested in validating this 601 00:46:22,639 --> 00:46:27,920 source information that I have. I went out and made a call to four 602 00:46:28,039 --> 00:46:31,400 external data sources, and they're going to give us some data back. And 603 00:46:31,480 --> 00:46:35,119 this data may not be exactly what you want, might be wrapped in some 604 00:46:35,320 --> 00:46:39,599 XML or JSON. Please try to make sense of the data and it actually 605 00:46:39,719 --> 00:46:45,480 tries to extract entities whether it is XML or JSON right and does the comparison 606 00:46:45,559 --> 00:46:49,719 for you. So so I found that writing that data wrangling code was much 607 00:46:49,840 --> 00:46:53,400 quicker than previously as possible. So there are these examples. So he was 608 00:46:53,480 --> 00:46:59,440 able to discriminate a tag from content it will not it will yeah, so 609 00:47:00,559 --> 00:47:06,639 as is the case with the imperative program. So I do feel that as 610 00:47:06,840 --> 00:47:13,960 a software developer, as we think about these problems, and Carl you talked 611 00:47:14,000 --> 00:47:19,000 about, you know, being a coding assistant is important, and I tend 612 00:47:19,079 --> 00:47:22,400 to use most of the times, I tend to use chat jupety for my 613 00:47:22,559 --> 00:47:27,920 coding stuff because it is you know, of course I can use the get 614 00:47:28,000 --> 00:47:30,559 ub copilot and get ub copilot. You know, the benefit of course is 615 00:47:30,599 --> 00:47:35,719 that you know ID you're based in the ide right there as you're coding. 616 00:47:35,760 --> 00:47:38,599 It is showing. But for me, even more important thing is I go 617 00:47:38,679 --> 00:47:43,639 to chat gepet and say I'm trying to write this routine and this is generally 618 00:47:43,760 --> 00:47:45,760 my idea about this routine. I can describe it in plain English. It 619 00:47:45,840 --> 00:47:50,320 comes back with some code and maybe I take that code, I paste it 620 00:47:50,440 --> 00:47:52,800 in there and then I'll test it and maybe'll give me an error, or 621 00:47:52,840 --> 00:47:55,599 maybe the logic is not quite accurate. I can go back and have a 622 00:47:55,679 --> 00:47:59,480 conversation that, hey, you gave me some code, but it is not 623 00:47:59,599 --> 00:48:05,000 handling this edge condition. Oh that's let me handle that edge condition for you. 624 00:48:05,119 --> 00:48:07,840 Go modify these three lines of code from here and then bring it back 625 00:48:08,119 --> 00:48:14,159 to my ID and change it. So in this manner, it's an iterative 626 00:48:14,199 --> 00:48:20,480 development model. So that is a significant timesaver. I had this exact experience 627 00:48:20,519 --> 00:48:25,280 you're talking about, without going too far into it, Kelly. My wife 628 00:48:25,320 --> 00:48:30,360 and I like to do word search puzzles, right, and we were getting 629 00:48:30,360 --> 00:48:31,639 it in the paper once a week, and we're like, you know, 630 00:48:32,280 --> 00:48:35,639 maybe I had to go get a book, and the books were like terrible. 631 00:48:35,840 --> 00:48:39,280 So I actually wrote a word search using chatch ept to help me do 632 00:48:39,400 --> 00:48:45,320 it, and it was completely iterative, and I probably should publish the conversation 633 00:48:45,440 --> 00:48:49,440 that I had to get the code. But it started just by putting them 634 00:48:49,519 --> 00:48:52,719 all in a line, you know, and I'm like, can you mix 635 00:48:52,760 --> 00:48:57,519 them up? Some backwards some diagonal some backwards diagonals? Like sure, So 636 00:48:57,639 --> 00:49:04,159 we just went through this reiterative thing and then it actually was great. I 637 00:49:04,239 --> 00:49:07,679 mean I used, first of all, I use a GPTAPI to search for 638 00:49:07,880 --> 00:49:15,760 words around the topic of X camping right. And I tried, first of 639 00:49:15,800 --> 00:49:20,360 all, I tried all these other libraries that try to do semantic you know, 640 00:49:20,639 --> 00:49:23,159 matching and searching and stuff, and they just fell flat. But you 641 00:49:23,239 --> 00:49:28,119 know, the GPTAPI was like, no problem. It just produced these words 642 00:49:28,159 --> 00:49:30,719 and I could tell it maximum word length, minimum word length, how many 643 00:49:30,760 --> 00:49:35,719 words to come back with, you know, don't use dashes or apostrophes like. 644 00:49:36,360 --> 00:49:39,320 It was really really an easy and it was just a single sentence of 645 00:49:39,440 --> 00:49:45,960 English that I had to perfect in this API as a query. So yeah, 646 00:49:45,000 --> 00:49:52,639 I know exactly what you're saying. The iterative process of using GPT chat 647 00:49:52,760 --> 00:49:57,079 GPT in particular to come up with some solution is really really powerful. I 648 00:49:57,159 --> 00:50:00,679 knew the model is able to reflect sometimes and say hey, I asked you 649 00:50:00,719 --> 00:50:06,280 to write a poem and which I let's say, I want I wanted to 650 00:50:06,320 --> 00:50:08,599 make sure that this is does not rhyme, and you give me an output 651 00:50:08,639 --> 00:50:12,199 which is rhyming, and say hey, this does not make sense, and 652 00:50:12,199 --> 00:50:15,079 it will say, hey, sorry, I apologize, I don't I met 653 00:50:15,119 --> 00:50:20,000 your requirement. It's very polite. It's polite, but Richard, I do 654 00:50:20,119 --> 00:50:22,960 want to go back to you said it is a stochastic parrot. Right, 655 00:50:24,039 --> 00:50:29,519 I would challenge that a little bit and I put a put a video out. 656 00:50:29,599 --> 00:50:32,119 I'll send you the link that is not based on my work. That's 657 00:50:32,559 --> 00:50:38,320 based on some great research that was done something called the Othello GPT and your 658 00:50:38,400 --> 00:50:43,320 listeners may know Othello as a game called reversy. It's an eight by board, 659 00:50:43,880 --> 00:50:46,679 right, So somebody did this, this awesome research, and I have 660 00:50:46,719 --> 00:50:50,480 an eight or ten minute video that I talk about it, and I especially 661 00:50:50,480 --> 00:50:52,000 if you want to reference that, because you know, we always think, 662 00:50:52,440 --> 00:50:57,480 hey, this is a stochastic parrot. You know, it just comes up 663 00:50:57,559 --> 00:51:00,880 with something based on what it has heard. And then you start typing and 664 00:51:00,960 --> 00:51:04,960 you start talking or you give a response, and it says, oh, 665 00:51:05,119 --> 00:51:07,000 these eight words, the ninth word must be this, and the tenth word 666 00:51:07,280 --> 00:51:14,280 must be this. So what the creators of the Athology GPD paper did was 667 00:51:14,719 --> 00:51:17,320 they said, Okay, let's constrain the problem as any good engineer would do. 668 00:51:19,000 --> 00:51:23,400 Rather than having a fifty thousand vocabulary of English words and then having billions 669 00:51:23,440 --> 00:51:28,599 of parameter models, let's just use the same technology, use a transform on 670 00:51:28,760 --> 00:51:34,119 network. But let's constrain the problem down to the game of Othallo. So 671 00:51:34,440 --> 00:51:39,440 Othello is a by board and you know, for limited number of moves, 672 00:51:39,639 --> 00:51:44,679 as you can imagine. Right, let's just take the sequences. Let's just 673 00:51:44,800 --> 00:51:51,039 go to the World Championship Othallo website and let's just download all the game scripts. 674 00:51:51,519 --> 00:51:53,719 And now your vocabulary is not fifty thousand words, it's just the limited 675 00:51:53,840 --> 00:51:58,880 number of moves that you can make in that game. And now train the 676 00:51:59,079 --> 00:52:04,960 model. Train a language model based on the scripts of Othello game. Okay, 677 00:52:05,960 --> 00:52:09,400 And now there's a technique in in neural networks that has been around for 678 00:52:09,519 --> 00:52:15,800 some time called the probes where you can go to a certain layer of this 679 00:52:15,320 --> 00:52:21,519 neural network and say, what have you learned? What up until now? 680 00:52:22,119 --> 00:52:25,039 What is your understanding? And what the creators of this paper did was, 681 00:52:25,760 --> 00:52:31,559 so let's say two players are playing this game and then you let's say you're 682 00:52:31,599 --> 00:52:37,599 they are halfway through the game or at whatever level they are, And what 683 00:52:37,760 --> 00:52:43,760 they wanted to know was, does the model really develop an emerging representation of 684 00:52:43,880 --> 00:52:47,519 the board. It's not just predicting the next move, right, And to 685 00:52:47,679 --> 00:52:52,719 prove that point, what they did was what if I'm going to suddenly change 686 00:52:52,840 --> 00:53:00,519 the board positioning, I can move these these these moves around or the pieces 687 00:53:00,559 --> 00:53:07,400 around, and does it now change the prediction based on that? So two 688 00:53:07,480 --> 00:53:12,840 things I said, apply approbe figure out the emerging representation of that othello game. 689 00:53:13,280 --> 00:53:17,519 So now it is predicting based on that. Now go in and manually 690 00:53:17,760 --> 00:53:23,760 change the position of these pieces and now see if it predicts something accurately. 691 00:53:24,159 --> 00:53:29,320 So what this paper tries to do is try to say that, you know, 692 00:53:29,400 --> 00:53:35,159 it's not just a stochastic prediction. In training this over and over and 693 00:53:35,320 --> 00:53:40,199 over again, it has developed an emerging representation of the world, and can 694 00:53:40,280 --> 00:53:46,480 we just try and apply this to CHATGPT In some ways it is developing an 695 00:53:46,559 --> 00:53:52,800 emerging representation of the world. Now there are people who absolutely disagree, Ian 696 00:53:52,920 --> 00:53:55,559 Lacun being one of them. But there are data scientists who who think that 697 00:53:55,800 --> 00:54:02,599 these models are learning something more than at giving them credit for. So that's 698 00:54:02,599 --> 00:54:07,239 a debate. But you know, I'll point to that. TELLOGPT if you 699 00:54:07,239 --> 00:54:09,320 want to read the paper, I found the link and I'll included in the 700 00:54:09,320 --> 00:54:14,199 show notes. Yes on your post and yeah it's in I mean, yeah, 701 00:54:14,199 --> 00:54:16,639 I'm you know me I follow in firmly in the Angkoon space of this 702 00:54:16,840 --> 00:54:21,519 is there are no emergency property emergent property here except the ones that we project 703 00:54:21,639 --> 00:54:28,960 upon them. It's software, but you can't argue with its ability to summarize, 704 00:54:29,599 --> 00:54:35,320 And so is this a kind of summarization, right? And a good 705 00:54:35,400 --> 00:54:38,519 summary also represents an emerging understanding. Right? But keep in mind, you 706 00:54:38,599 --> 00:54:43,119 know, the model has no notion that you're playing a game. It has 707 00:54:43,159 --> 00:54:45,760 no notion of who's winning, who's losing. Right, are there any of 708 00:54:45,800 --> 00:54:50,320 those things are important? Right? And that's there in nice part of the 709 00:54:50,360 --> 00:54:54,000 problem like this is such the opposite of common sense that you could possibly imagine. 710 00:54:54,559 --> 00:54:58,199 Right. But at the same time, the eight by eight board is 711 00:54:58,280 --> 00:55:00,400 complex enough that it cannot memory the moves. You can't keep it in your 712 00:55:00,440 --> 00:55:04,599 head. You can not keep it in your head. So there is some 713 00:55:04,840 --> 00:55:09,320 emerging representation the rules of the game. Some strategy is emerging that you know, 714 00:55:09,639 --> 00:55:13,559 anybody who's played othello knows that you know you want to take the coroner 715 00:55:13,639 --> 00:55:20,280 positions quickly to block your opponent. It is developing those emerging strategies. Nobody 716 00:55:20,320 --> 00:55:24,000 said you should do this, So there's something to be said, I found 717 00:55:24,079 --> 00:55:28,639 that paper so fascinating that I had to read for your listeners. I had 718 00:55:28,639 --> 00:55:31,920 to read it five times to understand that paper. These papers are complex to 719 00:55:32,039 --> 00:55:37,400 read, and then I tried to synthesize the meaning as I understood it in 720 00:55:37,480 --> 00:55:40,480 a ten or fifteen minute video. Looks like you found a link to that. 721 00:55:40,880 --> 00:55:45,719 Yeah, it's twenty minutes, but okay, it was complicated, so 722 00:55:45,880 --> 00:55:49,760 it was a little longer. But I appreciate your thinking on this, and 723 00:55:49,840 --> 00:55:52,480 it's great to see you building stuff here because it does give us all encouragement 724 00:55:52,559 --> 00:55:58,639 that there is space for some useful products if you put the proper parameters around 725 00:55:58,639 --> 00:56:04,000 it, like the capiats are important, covets are important, and think about 726 00:56:04,039 --> 00:56:07,239 it in this manner. Right again, going back to I'm always looking for 727 00:56:07,360 --> 00:56:13,599 how can I build application faster? So think about the three applications type that 728 00:56:13,719 --> 00:56:16,239 are emerging. Right of course, three sixty five copilot, you have productivity 729 00:56:16,320 --> 00:56:20,159 tools they are adding lllms to. That makes sense, right, But then 730 00:56:20,360 --> 00:56:23,840 maybe you have some sort of a line of business application or complex claims processing 731 00:56:23,960 --> 00:56:30,039 system that most of your listeners can relate to. There's one piece of functionality 732 00:56:30,119 --> 00:56:35,119 that we've never been able to you know, imperatively code. Maybe that functionality 733 00:56:35,239 --> 00:56:38,360 can be we can embed LLMS into your existing line of business applicant, right 734 00:56:38,599 --> 00:56:45,800 right, we you know, we get some large discounts because if we keep 735 00:56:45,920 --> 00:56:52,039 track of all of the invoices and certain products from different vendors, if we 736 00:56:52,119 --> 00:56:55,320 can somehow put them together, maybe we can come up with a way and 737 00:56:55,519 --> 00:57:00,400 negotiate a contract with somebody. Right, very hard problem to do. We 738 00:57:00,480 --> 00:57:05,519 have never been able to add that to over invoice them. Maybe now we 739 00:57:05,599 --> 00:57:08,760 can inject LLLMS to do that. So that's the other type of application. 740 00:57:09,000 --> 00:57:12,719 And then of course the third type is a native you know, problems that 741 00:57:12,800 --> 00:57:15,679 we have not been able to solve without generative AI, and now people are 742 00:57:15,760 --> 00:57:20,519 coming up with applications and new applications come out every day. No, and 743 00:57:20,599 --> 00:57:24,119 I appreciate all of that sort of truth that as we narrow in on what 744 00:57:24,280 --> 00:57:29,039 these tools are reliable for, it is a new class of automation and it 745 00:57:29,119 --> 00:57:32,840 will take certain workloads off the table that just we're bloody hard. I think 746 00:57:32,880 --> 00:57:37,159 the easy win is the raising of the average Like you talked about the writing 747 00:57:37,320 --> 00:57:43,360 side, that yeah, LMS will mean that your ordinary writer is a better 748 00:57:43,440 --> 00:57:45,880 writer, one would argue, the very best will still be better than any 749 00:57:46,000 --> 00:57:52,440 tool, but most of us aren't the very best, but far more interesting 750 00:57:52,840 --> 00:57:55,239 will be this true. We couldn't have solved this any other way, Right, 751 00:57:55,920 --> 00:57:59,840 you were mentioning a fellow and immediately go to the deep mind stuff that 752 00:58:00,079 --> 00:58:06,039 that went after go. And while that's interesting, deep mind's also tackling protein 753 00:58:06,159 --> 00:58:12,599 folding and is now producing results that we're going to take years to validate because 754 00:58:12,599 --> 00:58:15,559 they've tackled such a complicated problem. Like at this point they're like, well, 755 00:58:15,599 --> 00:58:19,320 it's now come back with some protein for whole routines we've never seen before, 756 00:58:20,039 --> 00:58:22,280 and until we prove them, we don't know if they're right or not, 757 00:58:22,599 --> 00:58:25,599 and it's actually hard for us to even prove it. Is that's interesting 758 00:58:25,719 --> 00:58:30,199 is the IBM Watson Chess program and AI. Well, it's a decision tree 759 00:58:30,239 --> 00:58:35,239 model built in the nineties. Like it's way more primitive and compremitive. But 760 00:58:35,400 --> 00:58:38,920 if you look at what it's just, it predates the deep learning models that 761 00:58:39,000 --> 00:58:44,159 we use in generated AI, but it uses just pure brute force to go 762 00:58:44,320 --> 00:58:47,280 through those trees in every possible combination. Yeah, well, it has a 763 00:58:47,320 --> 00:58:52,519 triage because you can get too deep. But one of the variations on Alpha 764 00:58:53,000 --> 00:58:59,960 GO they actually use the same training model to train against chess, and its 765 00:59:00,199 --> 00:59:04,519 slaughtered the best digital chess players out there, and it learned no time at 766 00:59:04,519 --> 00:59:07,719 all. Like it just if there was an indication that for this kind of 767 00:59:07,800 --> 00:59:13,719 problem space, it was wicked good at it. And also that, unlike 768 00:59:13,960 --> 00:59:19,800 the Decision model was taught against historical games, that Alpha my model literally was 769 00:59:19,800 --> 00:59:22,280 just given the rules and played itself, right, which is what they ended 770 00:59:22,360 --> 00:59:25,199 up with GO. Originally, they with the original version of the Go they 771 00:59:25,280 --> 00:59:30,880 trained against classical games and then made a much more efficient version that just trained 772 00:59:30,880 --> 00:59:35,719 against itself. So, I mean the space is where these genera AI models 773 00:59:35,800 --> 00:59:42,199 make sense, none of which is fact based, especially especially when it comes 774 00:59:42,239 --> 00:59:46,000 about extracting facts from the Internet. You know, that place that really lacks 775 00:59:46,039 --> 00:59:50,880 a lot of facts. Yeah, so you know, it's as long as 776 00:59:50,920 --> 00:59:54,599 you use a tool appropriately and you don't fall into the trap of trusting it 777 00:59:54,719 --> 01:00:00,440 as an authority, right any more than you trust your calculator being authority calcuators, 778 01:00:00,239 --> 01:00:04,000 Well, at least the calculator comes up with the same answer every time. 779 01:00:04,440 --> 01:00:07,159 Oh, the input it's are the same, for sure, don't discount 780 01:00:07,199 --> 01:00:09,119 the user error issue with calculators, right, I mean it, you know 781 01:00:09,199 --> 01:00:13,000 you have to follow the same sequence of keys, but yeah, you get 782 01:00:13,519 --> 01:00:16,719 Well, how many times we run into companies where they based their actions on 783 01:00:16,840 --> 01:00:22,679 a on a spreadsheet that had bad math in it. You know it's easy 784 01:00:22,760 --> 01:00:25,840 to do. Uh. The hour flies by with you visual, I know, 785 01:00:27,239 --> 01:00:30,840 kind like I want to keep going, but I could easily go another 786 01:00:30,880 --> 01:00:35,119 hour. Oh well, thank you for this. It's I'm really grateful for 787 01:00:35,440 --> 01:00:38,239 the work you've done. Yeah, me too, And and especially I really 788 01:00:38,320 --> 01:00:46,159 appreciate your business oriented mind of using AI where and that's something that I'm I'm 789 01:00:46,559 --> 01:00:51,920 just now starting to, you know, wrap my mind around. How how 790 01:00:52,000 --> 01:00:53,840 are real businesses going to use this? I mean you mentioned the Azure thing, 791 01:00:54,000 --> 01:01:01,000 but also Google Bard now can search Drive and Gmail and everything else. 792 01:01:01,320 --> 01:01:07,280 Gmail search was always really good, but all these business tools are coming out, 793 01:01:07,559 --> 01:01:14,119 and therein lies the opportunity I think. Yeah, So thanks again, 794 01:01:14,239 --> 01:01:19,159 Vishwas and dear listener, we will talk to you again next time on dot 795 01:01:19,239 --> 01:01:44,360 net rocks. Dot net Rocks is brought to you by Franklin's Net and produced 796 01:01:44,360 --> 01:01:49,960 by Pop Studios, a full service audio, video and post production facility located 797 01:01:50,000 --> 01:01:53,599 physically in New London, Connecticut, and of course in the cloud online at 798 01:01:53,679 --> 01:01:59,480 pwop dot com. Visit our website at d O T N E t R 799 01:01:59,480 --> 01:02:05,000 O c as dot com for RSS feeds, downloads, mobile apps, comments, 800 01:02:05,360 --> 01:02:08,440 and access to the full archives going back to show number one, recorded 801 01:02:08,480 --> 01:02:13,360 in September two thousand and two. And make sure you check out our sponsors. 802 01:02:13,559 --> 01:02:16,159 They keep us in business. Now go write some code. See you 803 01:02:16,239 --> 01:02:20,360 next time. You got Jack Middle Vans