WEBVTT 1 00:00:01.080 --> 00:00:05.679 How'd you like to listen to dot NetRocks with no ads? Easy? Become 2 00:00:05.679 --> 00:00:09.839 a patron for just five dollars a month. You get access to a private 3 00:00:10.000 --> 00:00:14.279 RSS feed where all the shows have no ads. Twenty dollars a month, 4 00:00:14.279 --> 00:00:18.839 we'll get you that and a special dot NetRocks patron mug. Sign up now 5 00:00:18.839 --> 00:00:36.159 at Patreon dot dot NetRocks dot com. Welcome back to dot net rocks. 6 00:00:36.200 --> 00:00:40.079 I'm Carl Franklin and I'm Richard Campbell, and we are in our respective places 7 00:00:40.079 --> 00:00:45.320 on opposite sides of the North American continent, absolutely as we usually are. 8 00:00:45.840 --> 00:00:51.000 But why don't know you caught us in the middle of three or four conferences 9 00:00:51.039 --> 00:00:55.320 here listener. Yeah, well, in the last few shows and get in 10 00:00:55.359 --> 00:00:58.280 time shifting being hilarious. We're all shows. We did it in Porto, 11 00:00:58.640 --> 00:01:03.239 and some awesome conversations. Really neat folks, folks who never talked to before. 12 00:01:03.239 --> 00:01:06.400 I always tend to when we're going to do in person shows, I 13 00:01:06.400 --> 00:01:08.480 try to go after guests we've never had on before. Right, Yeah, 14 00:01:08.560 --> 00:01:12.200 new stuff, all good, I would point out, I am. You 15 00:01:12.239 --> 00:01:15.400 know, it's getting towards the end of the year, so you know what 16 00:01:15.439 --> 00:01:19.079 I'm working on the end of the year, geek out. Yeah, yeah, 17 00:01:19.120 --> 00:01:22.560 I know it's close to the end of the year when I have to 18 00:01:22.599 --> 00:01:26.599 open up the script my script files and start pulling stuff forward for space and 19 00:01:26.640 --> 00:01:29.799 for energy, and I can't wait. There's been so much, it's so 20 00:01:30.000 --> 00:01:33.359 much, so much good stuff. They're both they're both going to be I'm 21 00:01:33.359 --> 00:01:36.599 just committing now. They're going to be long. I mean, you could 22 00:01:36.640 --> 00:01:40.840 just do a whole one on the JWT right, oh, JWST and the 23 00:01:40.879 --> 00:01:46.560 crisis in cosmology, like the current proposal they're floating around. Maybe we're long 24 00:01:46.599 --> 00:01:49.719 about the length of the age of the universe. Maybe it's twice as long 25 00:01:49.760 --> 00:01:53.840 as we thought. It's twenty six billion years instead of thirteen billion, wow, 26 00:01:53.879 --> 00:01:57.280 which just is stunning. I mean, I love the fact that this 27 00:01:57.400 --> 00:02:01.079 is why that instrument went up there, right, it was to test all 28 00:02:01.120 --> 00:02:07.319 of the assumptions we've been making and it has delivered. Like do we talk 29 00:02:07.359 --> 00:02:09.800 to Amber Strong about that, like this was one of the site and she's 30 00:02:09.879 --> 00:02:15.400 just like, Okay, there are things we're finding that cannot be there. 31 00:02:15.520 --> 00:02:20.439 There's something we don't know and it's a big something. And Vishwaz is looking 32 00:02:20.479 --> 00:02:23.360 at us like did I just parachute in here from another planet, land on 33 00:02:23.360 --> 00:02:27.360 the right show anyway, don't get me started. Oh wait, you did, 34 00:02:27.960 --> 00:02:30.159 I did. I'm sorry you late. I can't wait. I can't 35 00:02:30.159 --> 00:02:35.360 wait, I can't wait. So uh yeah, well let's get started with 36 00:02:35.400 --> 00:02:46.280 better no framework awesome? All right. Well I've talked about this show I've 37 00:02:46.280 --> 00:02:50.960 done with Brian McKay, the a iBOT Show before we had them on. 38 00:02:51.240 --> 00:02:57.360 Yes, yeah, and we have just started getting into code and using c 39 00:02:57.560 --> 00:03:02.319 sharp with the API to do you know, the chaining, you know, 40 00:03:02.439 --> 00:03:08.319 get the results of one prompt chain it into another. But we attempted to 41 00:03:08.360 --> 00:03:12.719 do one last week and by last week, I mean you know a few 42 00:03:12.719 --> 00:03:15.479 weeks ago, I think it was the twenty fifth of October, we recorded 43 00:03:15.919 --> 00:03:21.960 with a guest and for some reason Riverside FM, which is what we record 44 00:03:22.039 --> 00:03:29.840 with, decided to completely trash the are not finish uploading the screen share on 45 00:03:29.919 --> 00:03:31.960 top of that, so we're going to record it again. That's that's the 46 00:03:32.719 --> 00:03:36.520 thing. So the episode eleven, watch for that because it's going to be 47 00:03:36.560 --> 00:03:44.719 amazing. We're basically trying to break the or push GPT four V image recognition 48 00:03:45.080 --> 00:03:46.639 to its limits. Oh wow, that's what we're trying to do. But 49 00:03:47.120 --> 00:03:52.360 in the course of doing that, Brian actually got some results and we're going 50 00:03:52.400 --> 00:03:54.719 to rerecord it. So I don't know what's going to happen the second time 51 00:03:54.719 --> 00:03:59.840 of the first time. First time he got results that he didn't expect, 52 00:04:00.039 --> 00:04:05.759 and he tried these same prompts with the same images and got completely different responses. 53 00:04:06.199 --> 00:04:11.759 And it turned out either somebody's watching or it's just learning, Like, 54 00:04:12.360 --> 00:04:15.120 no, that wasn't a good response, Like there might be something else looking 55 00:04:15.199 --> 00:04:20.279 at the responses and tuning the algorithm, like in a matter of twenty four 56 00:04:20.319 --> 00:04:24.800 hours. Well, there's also another possibility, which is that generate of AI 57 00:04:24.959 --> 00:04:28.959 models have a variability that allow for a range of answers. Yeah, and 58 00:04:29.040 --> 00:04:32.560 I kind of got into it with him and his guest as well. You 59 00:04:32.600 --> 00:04:39.920 know, the whole idea of do these large language models reason, And my 60 00:04:40.079 --> 00:04:44.560 stance right now is no, they don't. And you know, when presented 61 00:04:44.600 --> 00:04:46.800 with the facts, this is what they do. They take this, they 62 00:04:46.800 --> 00:04:50.519 take this. How is that not reasoning? My position is well, because 63 00:04:50.519 --> 00:04:55.680 it gives you inconsistent answers. Well, more partly it's just a statistical model, 64 00:04:55.759 --> 00:04:58.560 right, Like, there's no reason to be had. It's not weighing 65 00:04:58.600 --> 00:05:02.240 any factors at all. Gives a statistical result, right and and you know, 66 00:05:02.319 --> 00:05:06.079 but if you want to just look at it empirically, when you ask 67 00:05:06.240 --> 00:05:11.879 somebody what's one plus one and on Tuesday they say two and on Thursday they 68 00:05:11.920 --> 00:05:15.040 say nine, Well you asked the same question over and over again, it'll 69 00:05:15.040 --> 00:05:17.439 give you different answers like yeah, that's it. Yeah, and so there 70 00:05:17.480 --> 00:05:20.319 therefore you can't count on it. It's not reliable and it's not what you 71 00:05:20.319 --> 00:05:25.120 think it is. You don't ask You don't ask a sarcastic parrot for facts. 72 00:05:25.120 --> 00:05:30.360 That's not what it's meant for. I'm sure Vishwas has lots of things 73 00:05:30.360 --> 00:05:33.959 to say on this, but anyway, tune into the Aibot show dot com. 74 00:05:34.319 --> 00:05:39.800 Uh. Starting at about episode nine, I think is when we started 75 00:05:39.839 --> 00:05:44.079 using code. But in episode eleven is where we start really pushing the boundaries 76 00:05:44.079 --> 00:05:46.519 with GPT four v Love it, don't learn it, Love it Cool. 77 00:05:46.519 --> 00:05:49.399 Boo's Talkers Today, Richard grabbed a comment of a show eighteen twenty nine when 78 00:05:49.439 --> 00:05:54.600 we did just earlier this year with our friend Vishwaz when we were talking about 79 00:05:54.639 --> 00:05:58.160 fusion development, which apparently is a term that's now fallen out of favor. 80 00:05:58.680 --> 00:06:02.439 But this is about the idea that the power platform presents an awesome way to 81 00:06:02.480 --> 00:06:08.720 build heetrically need as clients against cloud back ends, and that there's a role 82 00:06:08.839 --> 00:06:12.920 for traditional net developers in there and building some of those back end services and 83 00:06:12.920 --> 00:06:15.040 even building client side components. And that was the idea of Fusion, that 84 00:06:15.079 --> 00:06:18.399 we can all work together. I don't know how else caught on with folks, 85 00:06:18.560 --> 00:06:21.519 but not that hip. But we did get a comment off that show 86 00:06:21.519 --> 00:06:26.480 from Jim, who I believes are regular He said, every time I hear 87 00:06:26.560 --> 00:06:29.480 someone talk about how easy it is to build with power apps for a power 88 00:06:29.600 --> 00:06:31.839 user, I shake my head and I have to think to myself that we've 89 00:06:31.879 --> 00:06:36.120 lost something. In the past twenty years. I went from it support dude 90 00:06:36.120 --> 00:06:40.639 with no real software development training to a software development leader, in large part 91 00:06:40.680 --> 00:06:45.680 because of visual Basic for applications, which made it easy to learn to program. 92 00:06:46.040 --> 00:06:49.120 We didn't need a solution that put hard to maintain Excel formulas into web 93 00:06:49.120 --> 00:06:53.519 app. We had a problem that needed to get solved and get back quickly 94 00:06:53.600 --> 00:06:58.000 to developing more apps. We need tools that get us to the equivalent of 95 00:06:58.000 --> 00:07:00.959 the late nineties and early two thousand desktops development experience, not things that try 96 00:07:01.000 --> 00:07:04.319 and mimic access. I mean, it's one click to create a project that 97 00:07:04.319 --> 00:07:09.040 I could start dropping UI elements into and then writing clean cut with a minimum 98 00:07:09.040 --> 00:07:13.199 of ceremony, not hiding logic behind textbox dates out of textboxes. And I'm 99 00:07:13.199 --> 00:07:15.839 going to turn off my rant before it turns into a blog post now. 100 00:07:15.920 --> 00:07:19.720 I mean, Jim, I would say you're misremembering some stuff too, like 101 00:07:20.040 --> 00:07:23.519 first thing that I came to appreciate. I said this the other day on 102 00:07:23.560 --> 00:07:26.839 another show, where I said, I think wind Forms is the exception. 103 00:07:28.879 --> 00:07:32.399 It's the only product that ever had a great designer. Every other attempt to 104 00:07:32.399 --> 00:07:38.000 make a great designer look at you, front Page or dream Weaver, much 105 00:07:38.079 --> 00:07:42.800 less WPF. Every other attempt to make a great designer has fallen on hard 106 00:07:42.879 --> 00:07:46.160 times. Only wind Forms had a great designer. There is a reason for 107 00:07:46.199 --> 00:07:50.879 that. The reason is wind Forms is a pixel based grid, and all 108 00:07:50.920 --> 00:07:57.800 of these other things are dynamic in don't use yeah, and that's why can't 109 00:07:57.920 --> 00:08:01.079 You can't have a visual designer when you're not pixel based. Well, and 110 00:08:01.120 --> 00:08:03.519 the downside to the pixel based is then you go to higher risk monitors and 111 00:08:03.560 --> 00:08:07.560 oops, or you want to work at mobile oops, or you want to 112 00:08:07.720 --> 00:08:11.720 scale. Yeah, all of those problems are introduced, and so that's why 113 00:08:11.759 --> 00:08:15.360 we have this one draggy, droppy designer that people fell in love with back 114 00:08:15.399 --> 00:08:20.480 in the day. Yeah, but the idea that that vb ever built clean 115 00:08:20.519 --> 00:08:24.079 code of any kind and with minimum ceremony, Like it was good at getting 116 00:08:24.120 --> 00:08:26.399 stuff done, but boy it was. You had to practice carefully to make 117 00:08:26.439 --> 00:08:31.000 maintainable projects. Yes you did. It's a bit of a nostalgic effect to 118 00:08:31.000 --> 00:08:35.080 think all things were so much simpler them. Well they were because we had 119 00:08:35.159 --> 00:08:39.720 VGA displays, one type of machine to deal with, and you know, 120 00:08:39.879 --> 00:08:41.519 it more or less work that you were writing code on the same kind of 121 00:08:41.559 --> 00:08:45.480 computer you were going to run it on. Like life was simpler in that 122 00:08:45.559 --> 00:08:48.200 respect. You know. I had a flashback to the nineties when I had 123 00:08:48.200 --> 00:08:52.679 my little kids and we used to play DVD games right on the computers of 124 00:08:52.720 --> 00:08:56.360 the day. Dug out some of those DVDs, you know, like played 125 00:08:56.399 --> 00:09:01.360 them, brought them up, and I was thinking, oh, this is 126 00:09:01.360 --> 00:09:03.240 going to be great. You know, we get to do these cartoony mathey 127 00:09:03.320 --> 00:09:07.519 things, right. Yeah, it's like the size of a postage stamp on 128 00:09:07.600 --> 00:09:09.679 my screen. Your screens just a little bit higher resolution now, yeah, 129 00:09:09.720 --> 00:09:13.200 and you blow it up and it does not look good, you know. 130 00:09:13.279 --> 00:09:16.159 For the for the history of dot net, I have all of those great 131 00:09:16.159 --> 00:09:20.559 slides of the conferences from two thousand, like the original pds A tech ed 132 00:09:20.600 --> 00:09:24.200 when dot net was first announced, and like Mixo six and so forth. 133 00:09:24.559 --> 00:09:28.840 You know, I had to get all of those graphics remade because they were 134 00:09:28.879 --> 00:09:33.600 like three twenty two eighty yeah, six hundred like back then, screens weren't 135 00:09:33.600 --> 00:09:37.879 the same resolution, and they they looked terrible. Tretty tramble on eighty ten 136 00:09:37.919 --> 00:09:39.679 eighty. So I literally sent copies of them to designers, said remake these 137 00:09:39.720 --> 00:09:43.440 as ten eighty for me. So I have the only ones. I have 138 00:09:43.480 --> 00:09:46.279 the only good looking mix O six graphic, Like, it's me. I 139 00:09:46.399 --> 00:09:50.200 made it, it's mine. That's very cool. Anyway, Jim, thank 140 00:09:50.240 --> 00:09:52.600 you so much for your comment and a copy of music Coobu. It's on 141 00:09:52.639 --> 00:09:54.679 its way to you. And if you'd like a copy of Musiccobeu, I 142 00:09:54.759 --> 00:09:58.679 write a comment on the website at dot net rocks dot com or on the 143 00:09:58.679 --> 00:10:01.399 facebooks we publish every show there and if you comment there and I read it 144 00:10:01.399 --> 00:10:03.720 on the show, we'll send you copy of us to go by. And 145 00:10:03.759 --> 00:10:05.919 you know, you can follow us on Twitter if you like to, or 146 00:10:05.120 --> 00:10:09.440 X or whatever the hell it is today. It's going to be Z tomorrow, 147 00:10:09.480 --> 00:10:15.440 I'm sure. But yeah, we're still there, but the cool kids 148 00:10:15.480 --> 00:10:20.000 are hanging out. I'm mastadon from from my perspective anyway, I'm at Carl 149 00:10:20.000 --> 00:10:24.440 Franklin at tech hubs Social, and I'm Rich Campbell at Macedon dot social. 150 00:10:24.600 --> 00:10:28.360 Yeah, send us a two. Maybe you'll get a copy of music to 151 00:10:28.360 --> 00:10:31.600 go buy. There you go. It's another way that you can leave a 152 00:10:31.600 --> 00:10:37.919 comment. And that brings us to our good friend Vishwas. Vishwa Leiley has 153 00:10:37.080 --> 00:10:43.879 been on the show some thirteen or fourteen times. He is an Azure an 154 00:10:43.879 --> 00:10:48.279 AI expert. His bio is long, but I won't be labored that here. 155 00:10:48.279 --> 00:10:52.799 Uh he's a real great guy as well. Welcome back, Vishwas. 156 00:10:52.039 --> 00:11:00.200 Thank you what you've been up to, what's doing well? Doing the Like 157 00:11:00.240 --> 00:11:07.320 everybody else, got started with chagpt open ai, Azure open Ei. Specifically 158 00:11:07.840 --> 00:11:11.799 for asure developers like me. It was an easy sort of way to get 159 00:11:11.840 --> 00:11:16.840 into generative AI I've been working on some aim al stuff previously, but generative 160 00:11:16.840 --> 00:11:24.399 AI, of course has changed everything that we knew before. Now you're in 161 00:11:24.480 --> 00:11:26.639 watching BC I think. I think a lot of your customers or governments like, 162 00:11:28.320 --> 00:11:33.000 aren't they pretty careful about adopting new technologies. I'm always impressed that you're 163 00:11:33.039 --> 00:11:37.679 in these things so early on. Yeah. So, yes, Richard, 164 00:11:37.759 --> 00:11:41.919 that's true. But I do work with government and commercial customers both. You 165 00:11:41.960 --> 00:11:46.120 know, the the idea is that can we can we try out some solutions 166 00:11:46.120 --> 00:11:50.360 in the commercial world bring those some of those solutions to the government community. 167 00:11:50.879 --> 00:11:56.039 So that's that's been the pattern. But you'll be surprised the government customers. 168 00:11:56.120 --> 00:11:58.960 And you know, Microsoft has Azure Open the Eye service that was fed ramped, 169 00:12:00.600 --> 00:12:03.919 right, so it has available in a certain secure setting. That was 170 00:12:03.919 --> 00:12:09.240 an announcement that came a few months ago. So there are civilian agencies who 171 00:12:09.399 --> 00:12:18.320 have lots of needs for automation of processes and processing of documents, and they're 172 00:12:18.320 --> 00:12:24.399 definitely looking at generative AI models to be able to expedite some of that work. 173 00:12:24.559 --> 00:12:28.879 Well, I know the government is currently sharpening its clause to to put 174 00:12:28.919 --> 00:12:33.799 some restrictions on AI even sort of be fuddled about this whole thing from the 175 00:12:33.879 --> 00:12:37.600 very beginning, and I'm not so sure how well they understand it. But 176 00:12:39.519 --> 00:12:43.799 clearly there's some legislation needs to happen, especially around copyright protection and that kind 177 00:12:43.840 --> 00:12:48.799 of stuff licensing. But what are your thoughts on that. Have you heard 178 00:12:48.799 --> 00:12:52.559 anything being in DC about what they've planned to do? Well, you know, 179 00:12:52.679 --> 00:12:56.120 like you, there's been news that White House is preparing for a major 180 00:12:56.120 --> 00:13:00.559 announcement. I don't know if it's a good thing or a bad thing for 181 00:13:01.320 --> 00:13:05.519 You're right, I think we do need some guidance there, but I always 182 00:13:05.559 --> 00:13:13.799 worry about what kind of guidance and will impede the amazing amount of overall. 183 00:13:13.919 --> 00:13:16.159 I mean, they are negatives and there are positives, like everything else. 184 00:13:18.279 --> 00:13:22.080 I do think that this is just an amazing development that we will look back 185 00:13:22.120 --> 00:13:26.200 five years from now and say, oh, something new started in twenty twenty 186 00:13:26.240 --> 00:13:33.360 three. So I'm hoping that the legislation that they're coming up with, I'm 187 00:13:33.360 --> 00:13:41.600 hoping with the industry leaders is a responsible step but does not curtail the progress 188 00:13:41.639 --> 00:13:45.559 and the amazing possibilities that we see from this technology. Yeah, when I 189 00:13:45.600 --> 00:13:50.159 think about the logical roles, it's like it would be good to have some 190 00:13:50.200 --> 00:13:54.080 privacy legislation. You've been needing that for ages, you know, large language 191 00:13:54.120 --> 00:13:58.240 models notwithstanding. But to me, the one area that you want from a 192 00:13:58.240 --> 00:14:01.840 public perspective care about is source of data full stop. If you're going to 193 00:14:01.840 --> 00:14:05.799 put make an LLM available to the public, even for a fee, you 194 00:14:05.799 --> 00:14:09.440 have to tell the where you get your data from. That's an easy one, 195 00:14:09.519 --> 00:14:13.080 and it's and you know, there's nothing profound there. It's not a 196 00:14:13.120 --> 00:14:16.919 secret. Presumably, it certainly deals with it starts to tackle the cocky right 197 00:14:16.960 --> 00:14:18.960 issue, but it's just like, hey, you know, you want to 198 00:14:18.960 --> 00:14:22.200 compete, make distinctions between different lms, talk about source of data, right, 199 00:14:22.320 --> 00:14:26.159 So, Richard, I think source of data. And there's been a 200 00:14:26.159 --> 00:14:30.519 debate about you know, what do open source models mean? Right? Is 201 00:14:30.559 --> 00:14:33.960 it just publishing the weights of your model, Does it make it open source? 202 00:14:33.679 --> 00:14:39.720 Or does announcing to the world what sources that your models were trained on. 203 00:14:41.080 --> 00:14:43.720 Yeah, that's the definition of open source. So you're absolutely right. 204 00:14:45.480 --> 00:14:50.039 These companies have to at some point tell us what are the source of data. 205 00:14:50.080 --> 00:14:56.200 But that's different from citations. Once the response is being generated at a 206 00:14:56.240 --> 00:15:00.879 sentence level, that's that's a very hard problem to do to spid well. 207 00:15:00.919 --> 00:15:03.159 And that's what you really need, right, I mean, nobody wants a 208 00:15:03.440 --> 00:15:11.080 you know, fifty terrorbyte list of sources, general sources that it's been trained 209 00:15:11.120 --> 00:15:13.120 on. But what I want to know is when it tells me something, 210 00:15:13.399 --> 00:15:16.720 where did you get that fact? You know? But as you said, 211 00:15:16.759 --> 00:15:22.600 it's very hard because it's a statistic model that predicts the next word that's going 212 00:15:22.679 --> 00:15:24.320 to come. How does it know it? Arguably, it's not pulling from 213 00:15:24.360 --> 00:15:28.399 any one source. It's the points of the data that it processed against. 214 00:15:28.559 --> 00:15:33.000 Right, Yes, that is true. We can sort of break my mind, 215 00:15:33.039 --> 00:15:37.759 break the generative way technology into two parts. The ability to comprehend what 216 00:15:37.799 --> 00:15:41.240 you're saying. You know, so it's throwing it a prompt and it is 217 00:15:41.279 --> 00:15:45.279 trying to comprehend and trying to give you an answer. And the second part 218 00:15:45.320 --> 00:15:48.279 is the general knowledge that it has captured through the training process. Right. 219 00:15:48.320 --> 00:15:52.440 If you think of I think of generative Way in two terms, right from 220 00:15:52.559 --> 00:15:56.200 and I think we're going to talk get into this more. I look at 221 00:15:56.240 --> 00:16:02.159 it from the perspective of how can generate your way I make me a better 222 00:16:02.200 --> 00:16:07.679 developer? How can I solve everyday problems? That are there in the businesses 223 00:16:07.720 --> 00:16:11.200 in it. How can I solve that? So from my perspective, I 224 00:16:11.320 --> 00:16:17.399 trend to break, like I said, into two parts, the part about 225 00:16:17.440 --> 00:16:21.639 comprehending and sort of listening and reasoning, and then part about general knowledge. 226 00:16:22.159 --> 00:16:26.159 I tend to leave the general knowledge part out if I can. When I'm 227 00:16:26.159 --> 00:16:30.039 asking for a prompt or asking for a response, I'm going to provide you 228 00:16:30.080 --> 00:16:33.440 a context and you should only reply based on the information that I am providing 229 00:16:33.480 --> 00:16:38.639 you commonly referred to as the RAG pattern, and that is very important to 230 00:16:38.720 --> 00:16:45.480 me. So Richard, your question about sources and citation is very important. 231 00:16:45.519 --> 00:16:49.720 But in the kind of solutions that I'm building, I am providing it a 232 00:16:49.759 --> 00:16:55.279 context and saying I'm going to give you these ten pages of information and you 233 00:16:55.399 --> 00:16:59.559 need to process that, and that way, the hallucination and the accuracy is 234 00:16:59.639 --> 00:17:03.240 much high to begin with, because I'm controlling the source. I agree, 235 00:17:03.519 --> 00:17:08.680 and I think that code in general, you know, code generation or help 236 00:17:08.759 --> 00:17:15.839 with coding is where AI. You know, this large language model absolutely shines. 237 00:17:15.119 --> 00:17:19.599 Sure, and nobody's going to complain that, oh, that algorithm was 238 00:17:19.640 --> 00:17:26.160 lifted out of my open source project. Well it probably wasn't it was probably 239 00:17:26.279 --> 00:17:32.359 put together just by statistics, but but I've found it. That's that's where 240 00:17:32.400 --> 00:17:37.720 it really shines. And that's not where people are complaining. People are complaining 241 00:17:37.759 --> 00:17:42.400 because it's taking their art, artwork and appropriating it. You know, artists, 242 00:17:42.480 --> 00:17:45.599 like all the artists are crying about this, right, you know, 243 00:17:45.920 --> 00:17:51.559 and this is why people don't trust it. But as you know, anyone 244 00:17:51.599 --> 00:17:56.759 who's writing complex documents that are in or code that are in a specific as 245 00:17:56.799 --> 00:18:02.240 you said, context, right, that's where it really really shines. And 246 00:18:03.200 --> 00:18:07.400 before I stop talking here, the other things that I like about it are 247 00:18:07.720 --> 00:18:15.359 the new chat GPT being web interface, so instead of using the data that 248 00:18:15.400 --> 00:18:22.599 it's trained on, you can do essentially what is a web search, but 249 00:18:22.039 --> 00:18:26.359 it goes out, finds multiple sources, puts things together, and if it 250 00:18:26.400 --> 00:18:32.839 finds the answer, it sort of gives you a nice concise answer instead of 251 00:18:32.880 --> 00:18:37.319 having to scrape it out of different sites. And also if it can't find 252 00:18:37.319 --> 00:18:41.039 an answer, it'll say, you know what, I searched all over the 253 00:18:41.039 --> 00:18:45.880 web and I couldn't find that thing that you're looking for. Maybe it doesn't 254 00:18:45.920 --> 00:18:48.799 exist, and maybe here's some documentation that you can go look at and see 255 00:18:48.839 --> 00:18:52.680 if you can figure it out yourself. Yeah, that integration because all of 256 00:18:52.680 --> 00:18:56.960 these language models have a knowledge cutoff date December twenty twenty one or some date 257 00:18:56.119 --> 00:19:00.599 like that. So if you ask a question that goes beyond that, of 258 00:19:00.640 --> 00:19:03.160 course, having this big integration where you can do go do a web search, 259 00:19:04.200 --> 00:19:07.680 you could do that. I guess with ch AGPT too, with some 260 00:19:07.759 --> 00:19:11.720 plug ins, but beings sort of gives you that. What I've found is 261 00:19:11.759 --> 00:19:15.839 it generally works, but there are there are situations where the citations are not 262 00:19:15.960 --> 00:19:21.880 complete, and that merging of real time look up versus what the model has 263 00:19:21.920 --> 00:19:26.200 been trained on does not work as seamlessly. You're right as you were copect, 264 00:19:26.480 --> 00:19:30.480 and especially for things like c sharp like you're better off using the default 265 00:19:32.319 --> 00:19:36.880 model that it was trained on to to do some code figure you know, 266 00:19:37.400 --> 00:19:41.359 right, a routine that does X, then you are using the bang stuff. 267 00:19:41.599 --> 00:19:45.000 The other thing I like about it now is I can say, here's 268 00:19:45.000 --> 00:19:48.960 a u r L to a PDF, digest this, and then tell me 269 00:19:48.039 --> 00:19:52.720 why, you know, my fridge doesn't work. That's right, and it 270 00:19:52.759 --> 00:19:56.200 will actually do a good job of diagnosing, And you can take pictures and 271 00:19:56.359 --> 00:20:00.720 upload it and it'll understand what it's looking at in the picture most of the 272 00:20:00.799 --> 00:20:03.519 time. This is what we're finding out anyway, cal that's a very good 273 00:20:03.519 --> 00:20:07.559 example. Give it a fifty or one hundred page document and let it read 274 00:20:07.599 --> 00:20:11.279 it, and then have a conversation with it, and say, because you 275 00:20:11.279 --> 00:20:15.400 know this, this fifty page document presumably is talking about different problems with the 276 00:20:15.480 --> 00:20:18.559 fridge. You don't want to go through that. You can say, Okay, 277 00:20:18.599 --> 00:20:22.519 now that you've read it, I'm going to tell you two symptoms that 278 00:20:22.599 --> 00:20:26.839 I'm seeing with the problem that I'm having, and can you narrow it down? 279 00:20:26.880 --> 00:20:30.200 And then having a conversation with that. That's just amazing. It is 280 00:20:30.240 --> 00:20:33.279 amazing. That's aig. It speaks to the strength of this model. It 281 00:20:33.400 --> 00:20:38.079 is a good summarizer to be able to ream through a lot of information. 282 00:20:38.279 --> 00:20:44.640 And plus you can validate you know it got it from those hundred pages. 283 00:20:44.680 --> 00:20:48.359 So once it gives a statement like you should do this, you can take 284 00:20:48.400 --> 00:20:52.920 that phrase and search in that document and probably find exactly the reference you need 285 00:20:52.680 --> 00:20:56.400 to say, yeah, the document really does say this. I found I 286 00:20:56.440 --> 00:21:00.720 did this once in my amplifier and it actually gave me the answer and said 287 00:21:00.759 --> 00:21:04.839 on page fifty three in this section here's what it says. So it actually 288 00:21:06.000 --> 00:21:10.000 told me where in the document the answer lied. And this was a real 289 00:21:10.039 --> 00:21:11.759 experiment that I did. And I had a real problem with a piece of 290 00:21:11.799 --> 00:21:15.799 gear, and I literally told her to read the PDF of the manual and 291 00:21:17.119 --> 00:21:21.359 it in like five seconds gave me the answer the challenge that you know, 292 00:21:21.480 --> 00:21:23.920 let's say if your one hundred page PDF document, either it can read it 293 00:21:25.000 --> 00:21:27.240 completely, that's one thing. But you know what if you have thousands of 294 00:21:27.440 --> 00:21:32.079 fifty page documents, Yeah, then you're trying to sort of come up with 295 00:21:32.119 --> 00:21:36.960 an answer, and you're eventually going to run into a limit. Even though 296 00:21:37.000 --> 00:21:40.359 GBT four gives you thirty to two K models, which is what about two 297 00:21:40.400 --> 00:21:42.759 hundred and fifty pages or so what if you had more pages? And then 298 00:21:42.799 --> 00:21:47.559 now you can't feed that into the model, right, and now you have 299 00:21:47.599 --> 00:21:49.559 to do some sort of a search, a vector search of some sort, 300 00:21:49.880 --> 00:21:55.240 get the right content in and pass that as a context you prompt you said 301 00:21:55.279 --> 00:22:00.759 the V word vector databases. I find these baffle and I'm not so sure 302 00:22:00.799 --> 00:22:04.519 that you know, Joe web user is going to be able to figure them 303 00:22:04.559 --> 00:22:08.039 out. Can you briefly explain what they are? And sure, sure, 304 00:22:08.119 --> 00:22:11.599 sure, and I shouldn't have used the words without explaining it first. No, 305 00:22:11.720 --> 00:22:17.039 of course you should. How are we going to ask you questions? 306 00:22:17.519 --> 00:22:22.440 I mean, yeah, what's our role here? No? True? So 307 00:22:22.839 --> 00:22:27.079 let's take a step back. So you started out with a great example of 308 00:22:27.279 --> 00:22:32.839 you know, having this great fifty page PDF about your amplifier and it just 309 00:22:33.319 --> 00:22:36.839 you. You had the model read that completely, and you were able to 310 00:22:36.880 --> 00:22:44.000 do that because that fifty pages fit into the size of that that the model 311 00:22:44.000 --> 00:22:48.640 supports, which is thirty two. Right. What if you had thousands of 312 00:22:48.680 --> 00:22:53.519 fifty page documents and you're trying to build an answer. You can't feed all 313 00:22:53.599 --> 00:22:56.720 of those to the models, at least not now. Maybe GPT five GPT 314 00:22:56.799 --> 00:23:00.319 six would allow us to do that, who knows. So in the absence 315 00:23:00.359 --> 00:23:03.640 of that, what you have to do is you have to go do a 316 00:23:03.680 --> 00:23:10.880 search against those thousand documents. And the problem is that if you did some 317 00:23:11.000 --> 00:23:15.960 keyboard search, as we've been used to doing, right, it is not 318 00:23:15.200 --> 00:23:22.000 that keyboard search is really not conveying the intent of what you're trying to search. 319 00:23:22.079 --> 00:23:25.559 You might search for some words and you might get good hits, no 320 00:23:25.680 --> 00:23:29.160 question about it. But what if you're trying to just describe the problem. 321 00:23:29.240 --> 00:23:33.400 You're just saying, my amplifier works this way in certain conditions, but I'm 322 00:23:33.440 --> 00:23:37.799 noticing these problems. I can guarantee you there will be no keyboard hits when 323 00:23:37.799 --> 00:23:42.599 you write a statement like that. But in the you're describing the problem. 324 00:23:44.160 --> 00:23:48.200 And what if you were able to take this two word two lines worth the 325 00:23:48.440 --> 00:23:56.279 description and find a paragraph of texts that closely resembles this text. And that's 326 00:23:56.279 --> 00:23:59.400 what a vector database is. What we are trying to do, is we're 327 00:23:59.480 --> 00:24:07.519 trying to take every piece of text in that large corpus of documents and we 328 00:24:07.559 --> 00:24:14.519 are assigning some statistical values to it so that we can place certain sentences in 329 00:24:14.559 --> 00:24:18.960 a certain manner. So think of every word having a certain number of attributes, 330 00:24:19.440 --> 00:24:22.880 and then words that are closely aligned will have similar number of attributes. 331 00:24:22.920 --> 00:24:27.720 Things like that. So now you take that sentence of your description of the 332 00:24:27.799 --> 00:24:33.000 problem, you create an embedding out of it, and you know, you 333 00:24:33.000 --> 00:24:37.039 can use any of the models to create an embedding, which is you're getting 334 00:24:37.039 --> 00:24:41.680 a mathematical representation of those two sentences. And then you are and this going 335 00:24:41.720 --> 00:24:47.319 back to high school math, you're doing a coscience search of this mathematical representation 336 00:24:47.960 --> 00:24:52.400 against the mathematical representation of the entire corpus. Maybe you get two or three 337 00:24:52.519 --> 00:24:57.319 hits which generally talk about this problem, and then you bring that text back 338 00:24:57.359 --> 00:25:03.640 and only send that text to the model because now you've compressed that information. 339 00:25:03.240 --> 00:25:08.319 So that's if you're able to do that, then you can get answers to 340 00:25:08.319 --> 00:25:14.400 the question even if you have one thousand documents. And I'll talk about later 341 00:25:14.440 --> 00:25:19.519 on the product or application that I'm working on, which basically uses the pattern 342 00:25:19.559 --> 00:25:23.160 that I just talked about. I mean, we don't have enough time for 343 00:25:23.200 --> 00:25:27.519 you to explain this, but in general, how does one go about creating 344 00:25:27.880 --> 00:25:36.359 a vector database from a tome of documents? Right? So you know, 345 00:25:36.599 --> 00:25:41.359 let's just take the simple example of you those two lines of description of your 346 00:25:41.400 --> 00:25:47.799 problem. Right, you can go to any number of embedding models that are 347 00:25:47.839 --> 00:25:52.680 available. You can say, hey, please give me a vector representation of 348 00:25:52.720 --> 00:25:56.359 these three lines of text, and it literally gives you back a vector representation 349 00:25:56.440 --> 00:26:03.119 of those three lines of text. So, because these models have been trained, 350 00:26:03.160 --> 00:26:08.119 like I said, you play this prediction game what comes next? What 351 00:26:08.279 --> 00:26:14.200 comes next? You play this prediction game, and by playing this game, 352 00:26:14.319 --> 00:26:19.279 millions and billions of times you have assigned a certain attribute to a certain word 353 00:26:19.359 --> 00:26:25.599 in a certain phrase of that sentence. And then by placing these values in 354 00:26:25.640 --> 00:26:29.480 a multi dimensional space, you can say these words together mean something, These 355 00:26:29.519 --> 00:26:33.839 words together mean something. And you know, we all remember the twenty fourteen 356 00:26:33.920 --> 00:26:37.039 word to wac paper that Google came out with, you know, twenty fourteen, 357 00:26:37.039 --> 00:26:42.079 and we were so amazed when it could do simple things like king minus 358 00:26:42.119 --> 00:26:47.920 man plus woman equals what oh queen, And we were so happy that you 359 00:26:47.920 --> 00:26:52.240 know, these models have now some semantic understanding of English language. Right that 360 00:26:52.400 --> 00:26:56.680 was twenty fourteen when the word to wac paper came out. And of course, 361 00:26:56.839 --> 00:27:03.079 you know, we've given the complexity of the models, the transformer model, 362 00:27:03.119 --> 00:27:04.920 the availability of GPUs. Now we can do so much more. We 363 00:27:04.920 --> 00:27:11.440 were embedding words literally, now we're embedding paragraphs and pages essentially, so that 364 00:27:14.480 --> 00:27:19.680 to your listeners, essentially you have taken your corpus of text and you've created 365 00:27:19.680 --> 00:27:26.480 a mathematical representation that tells a model. The model is not sentient, it 366 00:27:26.519 --> 00:27:30.839 does not know what it means. But because of mathematical representation, can take 367 00:27:30.839 --> 00:27:36.759 the problem description and say there are these four paragraphs out of these thousand documents 368 00:27:37.000 --> 00:27:41.680 that seem to be talking about something similar, and you go and pull those 369 00:27:41.720 --> 00:27:45.000 paragraphs, and then you go to your prompt and say, I'm going to 370 00:27:45.039 --> 00:27:52.000 give you four paragraphs worth of relevant information about my amplifier, and I'm going 371 00:27:52.039 --> 00:27:56.319 to give you some symptoms. Your job is to take that symptom and try 372 00:27:56.319 --> 00:28:02.960 to come up with a plausible answer for the problem that I'm having. That's 373 00:28:03.359 --> 00:28:08.200 that's essentially what's happened. I get that with the paragraphs that you generate for 374 00:28:08.279 --> 00:28:14.119 the prompt. But how do you know if I have thousands and thousands of 375 00:28:14.160 --> 00:28:18.920 pages of documents, is there like literally an app that I can just drag 376 00:28:18.960 --> 00:28:22.279 and drop those into and proof its a vector database or is there a lot 377 00:28:22.319 --> 00:28:25.480 of manual work on the part of the of the user. That's a great 378 00:28:25.559 --> 00:28:30.799 question. So Microsoft sixty sixty five Copilot is going to come out. The 379 00:28:30.839 --> 00:28:34.039 release data is November first or something like that, if I'm read correctly. 380 00:28:34.839 --> 00:28:40.119 So let's just use that as an example to answer your question. Card. 381 00:28:40.279 --> 00:28:45.480 Sure, so Microsoft is sixty five copilot would be Now you have inside Word, 382 00:28:45.559 --> 00:28:49.480 PowerPoint, Excel, whatever you will be able to ask. You can 383 00:28:49.519 --> 00:28:56.160 go into Word and say, hey, can you please generate me an invoice 384 00:28:56.640 --> 00:29:02.079 based on the last five invoices that I sent out? Okay, you'd be 385 00:29:02.079 --> 00:29:03.640 able to do something like this. How do you be able to do that? 386 00:29:03.680 --> 00:29:08.559 Well, maybe you have thousands of invoices stored on your one drive somewhere, 387 00:29:10.200 --> 00:29:15.599 and when you start training the semantic index, which is you know, 388 00:29:15.599 --> 00:29:18.240 which is a component of the Microsoft sixty sixty five copilot, what they would 389 00:29:18.279 --> 00:29:22.480 do is they would go through your one drive, open up every document, 390 00:29:23.160 --> 00:29:27.160 and most commonly this is what people are doing. Open up every document at 391 00:29:27.200 --> 00:29:32.759 the page break, Take the one page so or some fixed number of words 392 00:29:32.799 --> 00:29:37.680 or tokens. Read that from your document, create an embedding, store it. 393 00:29:40.039 --> 00:29:42.960 Take the next page, create an embedding using the AI model that I 394 00:29:42.960 --> 00:29:47.519 talked about. It could be any text model, and store it. And 395 00:29:47.839 --> 00:29:52.799 that's how Microsoft internally is creating the semantic index. Of course, cognitive search 396 00:29:52.000 --> 00:29:56.759 now has vector database built into it. Right, Cognitive search used to be 397 00:29:56.839 --> 00:30:03.039 text search, now has vector database built into it. So now you've got 398 00:30:03.079 --> 00:30:07.240 through this whole process of ingestion, which can take lots of hours, yes, 399 00:30:07.279 --> 00:30:10.759 because you could have thousands of documents. But now you've taken your entire 400 00:30:10.799 --> 00:30:15.000 corpus and broken it down into a set of embeddings. So I take it 401 00:30:15.039 --> 00:30:19.400 then this isn't something that Joe programmer would do with a bunch of data, 402 00:30:19.519 --> 00:30:23.119 like you need some sort of powerful tool on the back end to do this 403 00:30:23.279 --> 00:30:30.079 for you. This is something I'm time to understand your question. Maybe this 404 00:30:30.200 --> 00:30:33.680 is the time to sort of talk about the application I'm building and see if 405 00:30:33.680 --> 00:30:37.079 I answered that question. And I'll come back right all right, but hold 406 00:30:37.119 --> 00:30:41.480 that thought. We'll do that right after this break for these very important messages, 407 00:30:45.680 --> 00:30:48.359 and we're back. It's starting in Rocks and Carl Franklin. That's Richie 408 00:30:48.359 --> 00:30:52.279 Campbell, and that's vishwas Lele and he was just about to start telling us 409 00:30:52.279 --> 00:30:56.440 about this amazing application that he's working on. Please do go ahead, sir, 410 00:30:56.880 --> 00:31:00.640 thank you. So I don't know amazing, but there's an application that 411 00:31:00.880 --> 00:31:03.559 we've been trying to build the last six months that has taken up a lot 412 00:31:04.079 --> 00:31:08.640 of my time. So we talked about the Microsoft THY sixty five copilot. 413 00:31:08.720 --> 00:31:17.440 We are building a similar capability but to help proposal developers. So imagine you're 414 00:31:17.480 --> 00:31:22.759 getting a fifty one hundred page RFP or RFI or RFQ or something like that, 415 00:31:23.559 --> 00:31:29.960 and that's a very time consuming and expensive process. Right, wouldn't it 416 00:31:30.000 --> 00:31:33.279 be nice if you were able to take all your past performances so you're responding 417 00:31:33.279 --> 00:31:41.640 to these kinds of RFPs over years prior. Can you take those documents and 418 00:31:41.720 --> 00:31:45.920 create some sort of a vector database and then use that to generate a draft 419 00:31:47.079 --> 00:31:52.519 of a future RFP. So to going back to your question, Carl, 420 00:31:53.279 --> 00:31:57.440 as a developer, what we had to do was to write those ingestion routines 421 00:31:59.319 --> 00:32:05.559 where we take those past proposal responses that you may have submitted to the government 422 00:32:05.640 --> 00:32:08.799 or some agency or a state or local or what have you, and write 423 00:32:08.799 --> 00:32:15.119 that ingestion routine where we take that document split it up. Now, there 424 00:32:15.160 --> 00:32:19.680 is a lot of work that is happening which says, should you be breaking 425 00:32:19.720 --> 00:32:22.920 the document on the page boundary? Is that enough? Because think about it, 426 00:32:23.359 --> 00:32:30.200 if you just break it on a page boundary, and maybe the important 427 00:32:30.640 --> 00:32:35.319 idea or concept is transcending that page boundary, right, break it up by 428 00:32:35.359 --> 00:32:37.960 topics, right, break it up by topics, But then how do you 429 00:32:38.000 --> 00:32:45.119 break it up by topics table of contents. That's a good point, but 430 00:32:46.039 --> 00:32:52.640 sometimes you know, most in our case, at least the PDF documents that 431 00:32:52.680 --> 00:32:55.599 are usually make up these responses. The table of contents are not that great 432 00:32:55.640 --> 00:33:00.359 and PDF is not a great possible format in any case. Right, So 433 00:33:00.440 --> 00:33:06.519 you have to use in some cases language models to be able to even passe 434 00:33:06.599 --> 00:33:13.039 them out and to break the document down into some sort of a section in 435 00:33:13.160 --> 00:33:17.920 some classification format so that you can use it for embedding. So you store, 436 00:33:19.880 --> 00:33:22.000 maybe not do it on page boundaries, maybe do it on some sort 437 00:33:22.000 --> 00:33:27.839 of a lexical boundary. But then you also think about what if the table 438 00:33:27.880 --> 00:33:31.559 of contents has some sort of inheritance built into it, is it important for 439 00:33:31.599 --> 00:33:36.640 you to capture that inheritance because that might come in handy as you're trying to 440 00:33:36.680 --> 00:33:42.319 respond to a question. Right, So those are all important things to do. 441 00:33:44.079 --> 00:33:45.799 So going back to your question, Yes, as a developer, you 442 00:33:45.920 --> 00:33:52.440 have to think about where your embedding boundaries lie because when it comes time to 443 00:33:52.519 --> 00:33:58.880 generate content, the more precise context that you can provide to the language model, 444 00:33:59.119 --> 00:34:01.519 the better response are going to get. And the more precise context you 445 00:34:01.559 --> 00:34:07.480 can get by thinking deeply about how you're going to break your corpus into the 446 00:34:07.519 --> 00:34:09.840 right set of embeddings. Isn't Microsoft Copilt going to do a lot of this 447 00:34:09.920 --> 00:34:14.440 for us, Like, this is a solve problem if you just are all 448 00:34:14.440 --> 00:34:17.000 in on them three sixty five and pay whatever per month they're going to ask 449 00:34:17.039 --> 00:34:22.960 for. That's a great question. And let me tell you the difference is 450 00:34:23.039 --> 00:34:31.199 so Microsoft Phrisks have Copilot is going to be a general purpose answering engine, 451 00:34:31.440 --> 00:34:37.199 right, so you can ask it and say, hey, please draft me 452 00:34:37.840 --> 00:34:40.480 the text for an invoice based on the last five invoices, and I'm sure 453 00:34:40.519 --> 00:34:45.800 it'll do a great job. You've seen demos of that. It also has 454 00:34:45.840 --> 00:34:50.239 access to the graph, so you knows which people you're collaborating with, and 455 00:34:50.920 --> 00:34:57.199 it can take into account that. But if you have a complex writing problem 456 00:34:57.400 --> 00:35:01.320 where the questions are not single line questions, right, the RFP questions are. 457 00:35:01.360 --> 00:35:06.880 Sometimes the questions themselves are maybe a page long or half a page long. 458 00:35:07.440 --> 00:35:12.960 Right, how do you sort of create a prompt? So in case 459 00:35:12.960 --> 00:35:15.920 of Microsoft, there are sixty five you're essentially writing a prompt. You're saying, 460 00:35:15.920 --> 00:35:19.519 hey, genade mean invoice which is based on the past five invoices. 461 00:35:19.639 --> 00:35:22.519 Right, So much context in that statement that needs to be known that you 462 00:35:22.599 --> 00:35:28.119 have in mind. Right, in our case, the context is enormous, 463 00:35:28.239 --> 00:35:32.960 right, Richard, Because if you're trying to respond to a complex question, 464 00:35:34.960 --> 00:35:37.960 you have to infuse a lot of attributes into that. Right. You have 465 00:35:37.000 --> 00:35:45.199 to think about what are the pain points that the buyer is expressing in this 466 00:35:45.320 --> 00:35:49.320 description? What are the then themes? How am I the one that they 467 00:35:49.360 --> 00:35:53.199 should be selecting and not somebody else? Is the buyer cost conscious? Have 468 00:35:53.320 --> 00:35:55.840 we done this work? Like? You know, it's great to say that 469 00:35:55.880 --> 00:36:00.440 we are the greatest company it can do agile development, But want it be 470 00:36:00.519 --> 00:36:02.639 nice if you can provide some proof points that we have done this and we've 471 00:36:02.719 --> 00:36:07.039 had success. So I've just given you three examples. There are many such 472 00:36:07.079 --> 00:36:13.679 attributes, So when you are crafting a prompt, you need to take these 473 00:36:13.719 --> 00:36:19.559 considerations into account. So in case of sixty five copilot, you are writing 474 00:36:19.599 --> 00:36:23.760 the prompt, but in our situation you're writing a prompt that is far more 475 00:36:23.760 --> 00:36:30.039 complex. So we are trying to do some prompt engineering for on behalf of 476 00:36:30.079 --> 00:36:32.800 the user where they can tell us at a high level what they want to 477 00:36:32.840 --> 00:36:38.679 do with their response, and for us, it is very much about about 478 00:36:39.840 --> 00:36:44.480 That's why it's a copilot. It's not a pilot or an autopilot. You're 479 00:36:44.599 --> 00:36:49.320 leveraging the experience of experienced proposal writers. The experienced proposal writers who spent ten 480 00:36:49.400 --> 00:36:52.519 twenty thirty years writing this, they know exactly how you should be responding. 481 00:36:52.800 --> 00:36:58.719 We are simply augmenting them and providing the first draft. Right. All of 482 00:36:58.760 --> 00:37:02.960 the innovativeness of our engine will not help you win any proposals. Our engine 483 00:37:04.000 --> 00:37:07.559 will help you do sort of the mundane steps quickly so that you can spend 484 00:37:07.679 --> 00:37:13.920 more time adding innovation to your response. So you know, if you if 485 00:37:13.920 --> 00:37:17.039 you have to generate this prompt, we think that every user is not going 486 00:37:17.079 --> 00:37:22.159 to be able to generate those levels of prompts. And I'll tell you why, 487 00:37:22.280 --> 00:37:27.360 because as good as GBT is, if you give a certain size prompt, 488 00:37:27.440 --> 00:37:30.960 it starts forgetting some instructions that you give in the middle, right, 489 00:37:32.639 --> 00:37:37.239 so you have to repeat it. So we have to have sort of layers 490 00:37:37.280 --> 00:37:40.599 of prompts that go in. And this is why you know. The team 491 00:37:40.639 --> 00:37:44.880 that I'm working with has data scientists in them, and people ask me sometimes, 492 00:37:44.880 --> 00:37:47.760 hey, generative AI is pre trained, do I need a data scientist? 493 00:37:47.880 --> 00:37:52.559 Well, you need a data scientist if you're if you're crafting this sophisticated 494 00:37:52.599 --> 00:37:57.559 prompts on behalf of the user by taking some input from them. So, 495 00:37:58.280 --> 00:38:04.039 Richard, in that manner, we think that this is a very specialized example 496 00:38:04.559 --> 00:38:07.039 of what we are trying to use the language model, and this is sort 497 00:38:07.039 --> 00:38:10.920 of the difference with the M sixty five copilot, and there are a few 498 00:38:10.920 --> 00:38:16.880 other differences for us. Hallucination is very very important. Don't please don't go 499 00:38:16.960 --> 00:38:22.920 out and say that you did a five thousand SharePoint node migration when you've only 500 00:38:22.960 --> 00:38:25.559 done fifty or five hundred, right, that we'll be a problem. Right, 501 00:38:27.000 --> 00:38:34.519 So we do explicitly accuracy verification, and there are many techniques that your 502 00:38:34.599 --> 00:38:37.719 listeners may be aware of that. You know, you take the generator to 503 00:38:37.719 --> 00:38:42.960 response, you try to extract entities from that and lage language models. Right, 504 00:38:43.199 --> 00:38:46.400 So any definitive statements in there, yes, yes, and language models 505 00:38:46.400 --> 00:38:50.840 are great or extracting entities. And then you take those entities and then you 506 00:38:51.119 --> 00:38:53.760 do a search against your knowledge repository and say, do I see a five 507 00:38:53.840 --> 00:38:59.199 hundred node migration ever mentioned here? And if it is not, then let's 508 00:38:59.199 --> 00:39:02.159 just flag it and let a human proposal writer go check this. We're doing 509 00:39:02.199 --> 00:39:09.559 those kinds of things peer, if you I'll just just extend this by saying 510 00:39:09.599 --> 00:39:13.840 that I don't know if if your listeners have seen this study from MIT, 511 00:39:14.559 --> 00:39:16.920 where there are many studies out there which said, okay, let's just try 512 00:39:16.960 --> 00:39:22.360 to measure the productivity improvements or something like chat depety. Right, So what 513 00:39:22.400 --> 00:39:25.960 they did was they took four hundred people college degrees good writers, broke them 514 00:39:25.960 --> 00:39:30.760 into groups of into two groups two hundred each, gave them a writing assignment, 515 00:39:31.360 --> 00:39:36.199 and then people who were evaluating them had no idea whether they were using 516 00:39:36.199 --> 00:39:38.639 some tools or not. So they gave them on writing assignment where nobody was 517 00:39:38.679 --> 00:39:43.760 allowed to use any tools. Say, got the baseline. Then they give 518 00:39:43.800 --> 00:39:46.719 them another writing assignment, but this time half the group was only allowed to 519 00:39:47.199 --> 00:39:51.159 use chat deputy. The other group was not allowed to use chat jpety. 520 00:39:52.000 --> 00:39:55.800 And now once again the evaluators had no idea who was doing what. And 521 00:39:55.840 --> 00:39:59.960 if you look at the results, pretty amazing results. Right. The group 522 00:40:00.159 --> 00:40:06.400 that was using chat GPT completed their work in thirty or thirty three percent less 523 00:40:06.400 --> 00:40:09.039 time. And I can point you to the study. So my numbers may 524 00:40:09.039 --> 00:40:13.920 be off. But more importantly, more importantly, the people who are using 525 00:40:14.000 --> 00:40:19.719 chat GPT scored a thirty three or thirty five percent better grades than people who 526 00:40:19.760 --> 00:40:23.280 are not using Okay, so a third better grade in a third less time, 527 00:40:23.360 --> 00:40:27.639 third less time. I mean that's amazing. And then what was it 528 00:40:27.679 --> 00:40:31.639 fact checked? Because that's my problem. You might be able to split up 529 00:40:31.639 --> 00:40:37.800 beautiful text, but is it really right? Yes, and human evaluators evaluated 530 00:40:37.920 --> 00:40:42.440 that, right, It was just like you know, people testing that that's 531 00:40:42.480 --> 00:40:45.480 always a case, right, And we know that human evaluators make mistakes too 532 00:40:46.320 --> 00:40:50.840 as well well, and we don't know exactly what they were trying to write, 533 00:40:50.880 --> 00:40:53.320 like as soon as you get away from non factual writing, like just 534 00:40:53.400 --> 00:40:59.760 exploratory fiction based writing and stuff, it's not the same set of problems right, 535 00:41:00.239 --> 00:41:04.039 right, And you know what is interesting is that you know, remember 536 00:41:04.079 --> 00:41:07.760 I said that they were all given a task where nobody was allowed to use 537 00:41:07.760 --> 00:41:12.159 in external tools. So that was to establish a baseline, right. What 538 00:41:12.280 --> 00:41:17.880 is interesting in the study is that writer folks who did not have great writing 539 00:41:17.920 --> 00:41:23.400 ability, right, even they benefited from when they were put in a pool 540 00:41:23.400 --> 00:41:28.400 where they could use chatgputy. They benefited from. Their grades went up significantly. 541 00:41:28.480 --> 00:41:30.440 I mean I would argue they would benefit more, they would manafit more 542 00:41:30.559 --> 00:41:37.280 right. But you know, there's very very interesting thing that about that study 543 00:41:37.639 --> 00:41:40.559 that I'd like to bring back to your question, Richard. It was very 544 00:41:40.559 --> 00:41:44.440 interesting in the study that there's something called, and I'm going to use a 545 00:41:44.440 --> 00:41:46.800 fancy term that I did not know until I read this m study called the 546 00:41:47.320 --> 00:41:55.039 human machine complementarity. Okay. So what they did was they evaluated the grades 547 00:41:55.079 --> 00:41:59.760 at two levels. So the team that was remembered the team that was using 548 00:41:59.840 --> 00:42:05.519 cheat gpty. It took the output of chat gpt and graded it, and 549 00:42:05.599 --> 00:42:09.920 then they took the output where humans did some prompt engineering on top of chat 550 00:42:09.960 --> 00:42:16.079 GPT and generated an output and evaluated that too, okay, and they found 551 00:42:16.079 --> 00:42:20.679 that there was not much difference, right. So what is interesting is, 552 00:42:20.679 --> 00:42:24.599 so where is this human machine complementarity coming in? So people looked at the 553 00:42:24.639 --> 00:42:29.599 chat gepety output, made a few tweaks, and then they submitted it. 554 00:42:30.159 --> 00:42:37.760 So, tying it back to your question, Richard, my theory is that 555 00:42:38.440 --> 00:42:43.320 the reason there was not much different from the chat gept and the human machine 556 00:42:43.599 --> 00:42:49.519 output was people were not thinking carefully about the bronze and actually writing detailed prons, 557 00:42:49.920 --> 00:42:54.679 right, And we think that by creating these layers of prompt for them 558 00:42:55.159 --> 00:42:59.679 you can get a benefit on top of so everybody is going to get that 559 00:42:59.800 --> 00:43:02.159 third percent benefit. That's going to be at some point table stakes, right, 560 00:43:04.519 --> 00:43:08.440 was it? Steve Snowski I was reading a post from him recently said 561 00:43:08.480 --> 00:43:15.119 every textbox is going to be LLLM enabled soon and it's going to be like 562 00:43:15.239 --> 00:43:17.320 you start writing and it will just start giving you some stuff. So no, 563 00:43:19.159 --> 00:43:22.960 everybody is going to benefit from thirty percent or twenty eight percent or whatever 564 00:43:22.039 --> 00:43:27.840 that is, right, So how do you go beyond that? Well you 565 00:43:27.920 --> 00:43:32.480 go beyond that through sophisticated prompt engineering. So that's been my journey to build 566 00:43:32.519 --> 00:43:37.239 this tool. Take your past responses, and I have to say this. 567 00:43:37.679 --> 00:43:40.480 People ask me this all the time. This is not what we are building. 568 00:43:40.519 --> 00:43:44.199 Is not cheating as a service, right, I tell people, if 569 00:43:44.239 --> 00:43:46.800 you don't, if you don't have good past responses, if you don't have 570 00:43:46.840 --> 00:43:52.000 good architects who can come up with a good solution, if you don't have 571 00:43:52.280 --> 00:43:59.280 creative solutions, then what we produce will be pretty you know, poor and 572 00:43:59.840 --> 00:44:02.360 not something that you want. But if you have good past performances that you've 573 00:44:02.400 --> 00:44:07.480 done this SharePoint migration work and you can come up with a you have architects 574 00:44:07.480 --> 00:44:10.960 who can come up with a solution, then this is going to benefit. 575 00:44:12.039 --> 00:44:19.599 So really, from a responsibilii standpoint, it is about taking your knowledge and 576 00:44:19.639 --> 00:44:25.280 then generating that quickly and then accurately checking it that you're not hallucinating and saying 577 00:44:25.280 --> 00:44:30.159 you did five thousand not server migration, right. I love it as a 578 00:44:30.239 --> 00:44:35.800 service, chees. But also you know the old the old comside comment of 579 00:44:35.920 --> 00:44:39.760 gigo right of garbaging garbage out. Yes, like we still have data quality 580 00:44:39.800 --> 00:44:43.920 problems like this comes down to the quality the data you load the set with 581 00:44:44.039 --> 00:44:46.559 and how well you load it. But it is an automation tool, and 582 00:44:49.079 --> 00:44:52.760 all automations stuff from the same problem, which is they're only going to amplify 583 00:44:53.440 --> 00:44:57.000 what you tell it to amplify. And if you tell it to amplify your 584 00:44:57.039 --> 00:45:01.559 stupidity, it's going to amplify your stupidity. You're you're absolutely liter Richard my 585 00:45:02.320 --> 00:45:07.400 sort of in my experience in six months working on this project, it's clear 586 00:45:07.480 --> 00:45:14.239 that you spend a lot of time doing data wrangling. You talked about garbage 587 00:45:14.239 --> 00:45:16.559 and garbage out. Do you worry about what data is getting in? So 588 00:45:16.559 --> 00:45:22.320 so be prepared to do those kinds of things. Even though these models are 589 00:45:22.360 --> 00:45:28.280 great, But I would say one thing, these models are far more accepting 590 00:45:28.320 --> 00:45:36.079 of the mistakes. So I was just doing this example recently where you know, 591 00:45:36.119 --> 00:45:40.280 we had to calculate some accuracy or some data and we went out to 592 00:45:40.400 --> 00:45:46.719 three or four external services to go validate this data. And each of the 593 00:45:46.800 --> 00:45:51.480 services returned the data in a different format, right, one return in a 594 00:45:51.559 --> 00:45:55.320 certain Jason structure. Somebody gave us XML back, and previously I would have 595 00:45:55.400 --> 00:45:59.440 to write a lot of data wrangling code to sort of pars out that data, 596 00:45:59.599 --> 00:46:04.159 get this and then compare it sort of very format it's yeah, to 597 00:46:04.199 --> 00:46:07.400 get them into very imperative code, very much imperative code that I had to 598 00:46:07.480 --> 00:46:10.320 write, yeah, right, and then immediately throw away like you're only going 599 00:46:10.400 --> 00:46:15.440 to do throw it because it's going to And in case of language models, 600 00:46:15.480 --> 00:46:22.239 you can say, hey, I'm you know, I'm interested in validating this 601 00:46:22.639 --> 00:46:27.920 source information that I have. I went out and made a call to four 602 00:46:28.039 --> 00:46:31.400 external data sources, and they're going to give us some data back. And 603 00:46:31.480 --> 00:46:35.119 this data may not be exactly what you want, might be wrapped in some 604 00:46:35.320 --> 00:46:39.599 XML or JSON. Please try to make sense of the data and it actually 605 00:46:39.719 --> 00:46:45.480 tries to extract entities whether it is XML or JSON right and does the comparison 606 00:46:45.559 --> 00:46:49.719 for you. So so I found that writing that data wrangling code was much 607 00:46:49.840 --> 00:46:53.400 quicker than previously as possible. So there are these examples. So he was 608 00:46:53.480 --> 00:46:59.440 able to discriminate a tag from content it will not it will yeah, so 609 00:47:00.559 --> 00:47:06.639 as is the case with the imperative program. So I do feel that as 610 00:47:06.840 --> 00:47:13.960 a software developer, as we think about these problems, and Carl you talked 611 00:47:14.000 --> 00:47:19.000 about, you know, being a coding assistant is important, and I tend 612 00:47:19.079 --> 00:47:22.400 to use most of the times, I tend to use chat jupety for my 613 00:47:22.559 --> 00:47:27.920 coding stuff because it is you know, of course I can use the get 614 00:47:28.000 --> 00:47:30.559 ub copilot and get ub copilot. You know, the benefit of course is 615 00:47:30.599 --> 00:47:35.719 that you know ID you're based in the ide right there as you're coding. 616 00:47:35.760 --> 00:47:38.599 It is showing. But for me, even more important thing is I go 617 00:47:38.679 --> 00:47:43.639 to chat gepet and say I'm trying to write this routine and this is generally 618 00:47:43.760 --> 00:47:45.760 my idea about this routine. I can describe it in plain English. It 619 00:47:45.840 --> 00:47:50.320 comes back with some code and maybe I take that code, I paste it 620 00:47:50.440 --> 00:47:52.800 in there and then I'll test it and maybe'll give me an error, or 621 00:47:52.840 --> 00:47:55.599 maybe the logic is not quite accurate. I can go back and have a 622 00:47:55.679 --> 00:47:59.480 conversation that, hey, you gave me some code, but it is not 623 00:47:59.599 --> 00:48:05.000 handling this edge condition. Oh that's let me handle that edge condition for you. 624 00:48:05.119 --> 00:48:07.840 Go modify these three lines of code from here and then bring it back 625 00:48:08.119 --> 00:48:14.159 to my ID and change it. So in this manner, it's an iterative 626 00:48:14.199 --> 00:48:20.480 development model. So that is a significant timesaver. I had this exact experience 627 00:48:20.519 --> 00:48:25.280 you're talking about, without going too far into it, Kelly. My wife 628 00:48:25.320 --> 00:48:30.360 and I like to do word search puzzles, right, and we were getting 629 00:48:30.360 --> 00:48:31.639 it in the paper once a week, and we're like, you know, 630 00:48:32.280 --> 00:48:35.639 maybe I had to go get a book, and the books were like terrible. 631 00:48:35.840 --> 00:48:39.280 So I actually wrote a word search using chatch ept to help me do 632 00:48:39.400 --> 00:48:45.320 it, and it was completely iterative, and I probably should publish the conversation 633 00:48:45.440 --> 00:48:49.440 that I had to get the code. But it started just by putting them 634 00:48:49.519 --> 00:48:52.719 all in a line, you know, and I'm like, can you mix 635 00:48:52.760 --> 00:48:57.519 them up? Some backwards some diagonal some backwards diagonals? Like sure, So 636 00:48:57.639 --> 00:49:04.159 we just went through this reiterative thing and then it actually was great. I 637 00:49:04.239 --> 00:49:07.679 mean I used, first of all, I use a GPTAPI to search for 638 00:49:07.880 --> 00:49:15.760 words around the topic of X camping right. And I tried, first of 639 00:49:15.800 --> 00:49:20.360 all, I tried all these other libraries that try to do semantic you know, 640 00:49:20.639 --> 00:49:23.159 matching and searching and stuff, and they just fell flat. But you 641 00:49:23.239 --> 00:49:28.119 know, the GPTAPI was like, no problem. It just produced these words 642 00:49:28.159 --> 00:49:30.719 and I could tell it maximum word length, minimum word length, how many 643 00:49:30.760 --> 00:49:35.719 words to come back with, you know, don't use dashes or apostrophes like. 644 00:49:36.360 --> 00:49:39.320 It was really really an easy and it was just a single sentence of 645 00:49:39.440 --> 00:49:45.960 English that I had to perfect in this API as a query. So yeah, 646 00:49:45.000 --> 00:49:52.639 I know exactly what you're saying. The iterative process of using GPT chat 647 00:49:52.760 --> 00:49:57.079 GPT in particular to come up with some solution is really really powerful. I 648 00:49:57.159 --> 00:50:00.679 knew the model is able to reflect sometimes and say hey, I asked you 649 00:50:00.719 --> 00:50:06.280 to write a poem and which I let's say, I want I wanted to 650 00:50:06.320 --> 00:50:08.599 make sure that this is does not rhyme, and you give me an output 651 00:50:08.639 --> 00:50:12.199 which is rhyming, and say hey, this does not make sense, and 652 00:50:12.199 --> 00:50:15.079 it will say, hey, sorry, I apologize, I don't I met 653 00:50:15.119 --> 00:50:20.000 your requirement. It's very polite. It's polite, but Richard, I do 654 00:50:20.119 --> 00:50:22.960 want to go back to you said it is a stochastic parrot. Right, 655 00:50:24.039 --> 00:50:29.519 I would challenge that a little bit and I put a put a video out. 656 00:50:29.599 --> 00:50:32.119 I'll send you the link that is not based on my work. That's 657 00:50:32.559 --> 00:50:38.320 based on some great research that was done something called the Othello GPT and your 658 00:50:38.400 --> 00:50:43.320 listeners may know Othello as a game called reversy. It's an eight by board, 659 00:50:43.880 --> 00:50:46.679 right, So somebody did this, this awesome research, and I have 660 00:50:46.719 --> 00:50:50.480 an eight or ten minute video that I talk about it, and I especially 661 00:50:50.480 --> 00:50:52.000 if you want to reference that, because you know, we always think, 662 00:50:52.440 --> 00:50:57.480 hey, this is a stochastic parrot. You know, it just comes up 663 00:50:57.559 --> 00:51:00.880 with something based on what it has heard. And then you start typing and 664 00:51:00.960 --> 00:51:04.960 you start talking or you give a response, and it says, oh, 665 00:51:05.119 --> 00:51:07.000 these eight words, the ninth word must be this, and the tenth word 666 00:51:07.280 --> 00:51:14.280 must be this. So what the creators of the Athology GPD paper did was 667 00:51:14.719 --> 00:51:17.320 they said, Okay, let's constrain the problem as any good engineer would do. 668 00:51:19.000 --> 00:51:23.400 Rather than having a fifty thousand vocabulary of English words and then having billions 669 00:51:23.440 --> 00:51:28.599 of parameter models, let's just use the same technology, use a transform on 670 00:51:28.760 --> 00:51:34.119 network. But let's constrain the problem down to the game of Othallo. So 671 00:51:34.440 --> 00:51:39.440 Othello is a by board and you know, for limited number of moves, 672 00:51:39.639 --> 00:51:44.679 as you can imagine. Right, let's just take the sequences. Let's just 673 00:51:44.800 --> 00:51:51.039 go to the World Championship Othallo website and let's just download all the game scripts. 674 00:51:51.519 --> 00:51:53.719 And now your vocabulary is not fifty thousand words, it's just the limited 675 00:51:53.840 --> 00:51:58.880 number of moves that you can make in that game. And now train the 676 00:51:59.079 --> 00:52:04.960 model. Train a language model based on the scripts of Othello game. Okay, 677 00:52:05.960 --> 00:52:09.400 And now there's a technique in in neural networks that has been around for 678 00:52:09.519 --> 00:52:15.800 some time called the probes where you can go to a certain layer of this 679 00:52:15.320 --> 00:52:21.519 neural network and say, what have you learned? What up until now? 680 00:52:22.119 --> 00:52:25.039 What is your understanding? And what the creators of this paper did was, 681 00:52:25.760 --> 00:52:31.559 so let's say two players are playing this game and then you let's say you're 682 00:52:31.599 --> 00:52:37.599 they are halfway through the game or at whatever level they are, And what 683 00:52:37.760 --> 00:52:43.760 they wanted to know was, does the model really develop an emerging representation of 684 00:52:43.880 --> 00:52:47.519 the board. It's not just predicting the next move, right, And to 685 00:52:47.679 --> 00:52:52.719 prove that point, what they did was what if I'm going to suddenly change 686 00:52:52.840 --> 00:53:00.519 the board positioning, I can move these these these moves around or the pieces 687 00:53:00.559 --> 00:53:07.400 around, and does it now change the prediction based on that? So two 688 00:53:07.480 --> 00:53:12.840 things I said, apply approbe figure out the emerging representation of that othello game. 689 00:53:13.280 --> 00:53:17.519 So now it is predicting based on that. Now go in and manually 690 00:53:17.760 --> 00:53:23.760 change the position of these pieces and now see if it predicts something accurately. 691 00:53:24.159 --> 00:53:29.320 So what this paper tries to do is try to say that, you know, 692 00:53:29.400 --> 00:53:35.159 it's not just a stochastic prediction. In training this over and over and 693 00:53:35.320 --> 00:53:40.199 over again, it has developed an emerging representation of the world, and can 694 00:53:40.280 --> 00:53:46.480 we just try and apply this to CHATGPT In some ways it is developing an 695 00:53:46.559 --> 00:53:52.800 emerging representation of the world. Now there are people who absolutely disagree, Ian 696 00:53:52.920 --> 00:53:55.559 Lacun being one of them. But there are data scientists who who think that 697 00:53:55.800 --> 00:54:02.599 these models are learning something more than at giving them credit for. So that's 698 00:54:02.599 --> 00:54:07.239 a debate. But you know, I'll point to that. TELLOGPT if you 699 00:54:07.239 --> 00:54:09.320 want to read the paper, I found the link and I'll included in the 700 00:54:09.320 --> 00:54:14.199 show notes. Yes on your post and yeah it's in I mean, yeah, 701 00:54:14.199 --> 00:54:16.639 I'm you know me I follow in firmly in the Angkoon space of this 702 00:54:16.840 --> 00:54:21.519 is there are no emergency property emergent property here except the ones that we project 703 00:54:21.639 --> 00:54:28.960 upon them. It's software, but you can't argue with its ability to summarize, 704 00:54:29.599 --> 00:54:35.320 And so is this a kind of summarization, right? And a good 705 00:54:35.400 --> 00:54:38.519 summary also represents an emerging understanding. Right? But keep in mind, you 706 00:54:38.599 --> 00:54:43.119 know, the model has no notion that you're playing a game. It has 707 00:54:43.159 --> 00:54:45.760 no notion of who's winning, who's losing. Right, are there any of 708 00:54:45.800 --> 00:54:50.320 those things are important? Right? And that's there in nice part of the 709 00:54:50.360 --> 00:54:54.000 problem like this is such the opposite of common sense that you could possibly imagine. 710 00:54:54.559 --> 00:54:58.199 Right. But at the same time, the eight by eight board is 711 00:54:58.280 --> 00:55:00.400 complex enough that it cannot memory the moves. You can't keep it in your 712 00:55:00.440 --> 00:55:04.599 head. You can not keep it in your head. So there is some 713 00:55:04.840 --> 00:55:09.320 emerging representation the rules of the game. Some strategy is emerging that you know, 714 00:55:09.639 --> 00:55:13.559 anybody who's played othello knows that you know you want to take the coroner 715 00:55:13.639 --> 00:55:20.280 positions quickly to block your opponent. It is developing those emerging strategies. Nobody 716 00:55:20.320 --> 00:55:24.000 said you should do this, So there's something to be said, I found 717 00:55:24.079 --> 00:55:28.639 that paper so fascinating that I had to read for your listeners. I had 718 00:55:28.639 --> 00:55:31.920 to read it five times to understand that paper. These papers are complex to 719 00:55:32.039 --> 00:55:37.400 read, and then I tried to synthesize the meaning as I understood it in 720 00:55:37.480 --> 00:55:40.480 a ten or fifteen minute video. Looks like you found a link to that. 721 00:55:40.880 --> 00:55:45.719 Yeah, it's twenty minutes, but okay, it was complicated, so 722 00:55:45.880 --> 00:55:49.760 it was a little longer. But I appreciate your thinking on this, and 723 00:55:49.840 --> 00:55:52.480 it's great to see you building stuff here because it does give us all encouragement 724 00:55:52.559 --> 00:55:58.639 that there is space for some useful products if you put the proper parameters around 725 00:55:58.639 --> 00:56:04.000 it, like the capiats are important, covets are important, and think about 726 00:56:04.039 --> 00:56:07.239 it in this manner. Right again, going back to I'm always looking for 727 00:56:07.360 --> 00:56:13.599 how can I build application faster? So think about the three applications type that 728 00:56:13.719 --> 00:56:16.239 are emerging. Right of course, three sixty five copilot, you have productivity 729 00:56:16.320 --> 00:56:20.159 tools they are adding lllms to. That makes sense, right, But then 730 00:56:20.360 --> 00:56:23.840 maybe you have some sort of a line of business application or complex claims processing 731 00:56:23.960 --> 00:56:30.039 system that most of your listeners can relate to. There's one piece of functionality 732 00:56:30.119 --> 00:56:35.119 that we've never been able to you know, imperatively code. Maybe that functionality 733 00:56:35.239 --> 00:56:38.360 can be we can embed LLMS into your existing line of business applicant, right 734 00:56:38.599 --> 00:56:45.800 right, we you know, we get some large discounts because if we keep 735 00:56:45.920 --> 00:56:52.039 track of all of the invoices and certain products from different vendors, if we 736 00:56:52.119 --> 00:56:55.320 can somehow put them together, maybe we can come up with a way and 737 00:56:55.519 --> 00:57:00.400 negotiate a contract with somebody. Right, very hard problem to do. We 738 00:57:00.480 --> 00:57:05.519 have never been able to add that to over invoice them. Maybe now we 739 00:57:05.599 --> 00:57:08.760 can inject LLLMS to do that. So that's the other type of application. 740 00:57:09.000 --> 00:57:12.719 And then of course the third type is a native you know, problems that 741 00:57:12.800 --> 00:57:15.679 we have not been able to solve without generative AI, and now people are 742 00:57:15.760 --> 00:57:20.519 coming up with applications and new applications come out every day. No, and 743 00:57:20.599 --> 00:57:24.119 I appreciate all of that sort of truth that as we narrow in on what 744 00:57:24.280 --> 00:57:29.039 these tools are reliable for, it is a new class of automation and it 745 00:57:29.119 --> 00:57:32.840 will take certain workloads off the table that just we're bloody hard. I think 746 00:57:32.880 --> 00:57:37.159 the easy win is the raising of the average Like you talked about the writing 747 00:57:37.320 --> 00:57:43.360 side, that yeah, LMS will mean that your ordinary writer is a better 748 00:57:43.440 --> 00:57:45.880 writer, one would argue, the very best will still be better than any 749 00:57:46.000 --> 00:57:52.440 tool, but most of us aren't the very best, but far more interesting 750 00:57:52.840 --> 00:57:55.239 will be this true. We couldn't have solved this any other way, Right, 751 00:57:55.920 --> 00:57:59.840 you were mentioning a fellow and immediately go to the deep mind stuff that 752 00:58:00.079 --> 00:58:06.039 that went after go. And while that's interesting, deep mind's also tackling protein 753 00:58:06.159 --> 00:58:12.599 folding and is now producing results that we're going to take years to validate because 754 00:58:12.599 --> 00:58:15.559 they've tackled such a complicated problem. Like at this point they're like, well, 755 00:58:15.599 --> 00:58:19.320 it's now come back with some protein for whole routines we've never seen before, 756 00:58:20.039 --> 00:58:22.280 and until we prove them, we don't know if they're right or not, 757 00:58:22.599 --> 00:58:25.599 and it's actually hard for us to even prove it. Is that's interesting 758 00:58:25.719 --> 00:58:30.199 is the IBM Watson Chess program and AI. Well, it's a decision tree 759 00:58:30.239 --> 00:58:35.239 model built in the nineties. Like it's way more primitive and compremitive. But 760 00:58:35.400 --> 00:58:38.920 if you look at what it's just, it predates the deep learning models that 761 00:58:39.000 --> 00:58:44.159 we use in generated AI, but it uses just pure brute force to go 762 00:58:44.320 --> 00:58:47.280 through those trees in every possible combination. Yeah, well, it has a 763 00:58:47.320 --> 00:58:52.519 triage because you can get too deep. But one of the variations on Alpha 764 00:58:53.000 --> 00:58:59.960 GO they actually use the same training model to train against chess, and its 765 00:59:00.199 --> 00:59:04.519 slaughtered the best digital chess players out there, and it learned no time at 766 00:59:04.519 --> 00:59:07.719 all. Like it just if there was an indication that for this kind of 767 00:59:07.800 --> 00:59:13.719 problem space, it was wicked good at it. And also that, unlike 768 00:59:13.960 --> 00:59:19.800 the Decision model was taught against historical games, that Alpha my model literally was 769 00:59:19.800 --> 00:59:22.280 just given the rules and played itself, right, which is what they ended 770 00:59:22.360 --> 00:59:25.199 up with GO. Originally, they with the original version of the Go they 771 00:59:25.280 --> 00:59:30.880 trained against classical games and then made a much more efficient version that just trained 772 00:59:30.880 --> 00:59:35.719 against itself. So, I mean the space is where these genera AI models 773 00:59:35.800 --> 00:59:42.199 make sense, none of which is fact based, especially especially when it comes 774 00:59:42.239 --> 00:59:46.000 about extracting facts from the Internet. You know, that place that really lacks 775 00:59:46.039 --> 00:59:50.880 a lot of facts. Yeah, so you know, it's as long as 776 00:59:50.920 --> 00:59:54.599 you use a tool appropriately and you don't fall into the trap of trusting it 777 00:59:54.719 --> 01:00:00.440 as an authority, right any more than you trust your calculator being authority calcuators, 778 01:00:00.239 --> 01:00:04.000 Well, at least the calculator comes up with the same answer every time. 779 01:00:04.440 --> 01:00:07.159 Oh, the input it's are the same, for sure, don't discount 780 01:00:07.199 --> 01:00:09.119 the user error issue with calculators, right, I mean it, you know 781 01:00:09.199 --> 01:00:13.000 you have to follow the same sequence of keys, but yeah, you get 782 01:00:13.519 --> 01:00:16.719 Well, how many times we run into companies where they based their actions on 783 01:00:16.840 --> 01:00:22.679 a on a spreadsheet that had bad math in it. You know it's easy 784 01:00:22.760 --> 01:00:25.840 to do. Uh. The hour flies by with you visual, I know, 785 01:00:27.239 --> 01:00:30.840 kind like I want to keep going, but I could easily go another 786 01:00:30.880 --> 01:00:35.119 hour. Oh well, thank you for this. It's I'm really grateful for 787 01:00:35.440 --> 01:00:38.239 the work you've done. Yeah, me too, And and especially I really 788 01:00:38.320 --> 01:00:46.159 appreciate your business oriented mind of using AI where and that's something that I'm I'm 789 01:00:46.559 --> 01:00:51.920 just now starting to, you know, wrap my mind around. How how 790 01:00:52.000 --> 01:00:53.840 are real businesses going to use this? I mean you mentioned the Azure thing, 791 01:00:54.000 --> 01:01:01.000 but also Google Bard now can search Drive and Gmail and everything else. 792 01:01:01.320 --> 01:01:07.280 Gmail search was always really good, but all these business tools are coming out, 793 01:01:07.559 --> 01:01:14.119 and therein lies the opportunity I think. Yeah, So thanks again, 794 01:01:14.239 --> 01:01:19.159 Vishwas and dear listener, we will talk to you again next time on dot 795 01:01:19.239 --> 01:01:44.360 net rocks. Dot net Rocks is brought to you by Franklin's Net and produced 796 01:01:44.360 --> 01:01:49.960 by Pop Studios, a full service audio, video and post production facility located 797 01:01:50.000 --> 01:01:53.599 physically in New London, Connecticut, and of course in the cloud online at 798 01:01:53.679 --> 01:01:59.480 pwop dot com. Visit our website at d O T N E t R 799 01:01:59.480 --> 01:02:05.000 O c as dot com for RSS feeds, downloads, mobile apps, comments, 800 01:02:05.360 --> 01:02:08.440 and access to the full archives going back to show number one, recorded 801 01:02:08.480 --> 01:02:13.360 in September two thousand and two. And make sure you check out our sponsors. 802 01:02:13.559 --> 01:02:16.159 They keep us in business. Now go write some code. See you 803 01:02:16.239 --> 01:02:20.360 next time. You got Jack Middle Vans