WEBVTT 1 00:00:04.519 --> 00:00:07.960 Well, welcome to another episode of the Ruby Rovers podcast. I'm your host 2 00:00:07.960 --> 00:00:12.000 today, Valentino Stole, and we're joined by a very special guest, Scott 3 00:00:12.039 --> 00:00:15.800 Werner. Scott, do you want to introduce yourself? Tell us why you're 4 00:00:15.880 --> 00:00:21.839 Ruby famous? Here? Ruby famous? I wish someday. Thanks. Thanks 5 00:00:21.879 --> 00:00:25.920 for having me, Valentino. It's super exciting to be here and be a 6 00:00:25.960 --> 00:00:30.679 part of this. Yeah, so I'm here. I'm the CEO and founder 7 00:00:30.719 --> 00:00:37.600 and co founder of sub Layer, where we're building a self assembling AI agent 8 00:00:37.640 --> 00:00:46.520 framework in Ruby and kind of supporting tools and infrastructure for uh to make something 9 00:00:46.560 --> 00:00:53.799 like that possible. And yeah, basically been programming with AI and Ruby for 10 00:00:54.280 --> 00:00:59.320 and we were just talking earlier basically exclusively trying to get AI to write all 11 00:00:59.320 --> 00:01:06.319 my code for me and me actually write the code and primarily Ruby. That's 12 00:01:06.400 --> 00:01:11.959 awesome. So just so word clear, you're talking about sub Layer is a 13 00:01:11.120 --> 00:01:15.959 project. Sorry, Yes, the sub layer is the GM. Sub Layer's 14 00:01:15.959 --> 00:01:21.599 the GYM, and so yeah, it's uh, you know it said. 15 00:01:21.760 --> 00:01:26.480 You know, I've been describing it as a self assembling AI agent framework, 16 00:01:26.239 --> 00:01:30.920 and kind of what I mean by that is that each of the each of 17 00:01:30.959 --> 00:01:34.680 the components that we've built are designed so that it's easy for an l l 18 00:01:34.799 --> 00:01:41.280 M to generate new additional components that you might need. So the the the 19 00:01:41.280 --> 00:01:46.519 framework is very minimal, and it's really about kind of defining the the conventions, 20 00:01:46.879 --> 00:01:52.000 the interfaces and then kind of letting you lose to to expand on that 21 00:01:52.079 --> 00:01:56.560 for whatever you want your application. I'm super curious here, like, uh, 22 00:01:56.879 --> 00:01:59.319 where do you where do you start something like that? Like how did 23 00:01:59.319 --> 00:02:02.560 you get involved than this? Right? Like, uh, how do you 24 00:02:02.640 --> 00:02:09.599 just stop coding? Uh? It was tough, you know, I think 25 00:02:10.000 --> 00:02:14.879 I think a lot of us were really inspired when we saw chat GPT for 26 00:02:15.479 --> 00:02:20.879 Katt. We saw GPT four come out and people using chat GPT and you 27 00:02:20.879 --> 00:02:25.680 know, posting videos of I use this pro throw this prompt into chat GPT. 28 00:02:25.840 --> 00:02:30.159 I copy and pasted the code into x code and I have a running 29 00:02:30.199 --> 00:02:36.240 app and I've never coded before, and uh, you know I saw that 30 00:02:36.479 --> 00:02:38.879 and immediately it was like, oh, well, those people are going to 31 00:02:38.919 --> 00:02:46.759 want a way to add functionality and modify that software over time. And uh 32 00:02:46.199 --> 00:02:55.879 kind of realized that a lot of the stuff that we do in XP around 33 00:02:55.919 --> 00:03:01.479 like TDD and and the different refactory as we do lend themselves really well to 34 00:03:01.520 --> 00:03:07.840 working with LM. So breaking things out into small pieces, the smaller requests, 35 00:03:07.879 --> 00:03:10.039 the simpler request you have for the LLLM that you the more likely get 36 00:03:10.120 --> 00:03:15.240 you something back. That's that's good, you know. And also kind of 37 00:03:15.280 --> 00:03:17.199 had this, I had this conversation with people like, you know, you 38 00:03:17.240 --> 00:03:20.840 would be mad at me if I gave you a thousand nine PR, but 39 00:03:20.840 --> 00:03:23.280 at least you could come over to my desk and yell at me. Right 40 00:03:23.719 --> 00:03:28.080 if you've got a thousand nine PR from chatchypt that would be a big problem. 41 00:03:28.639 --> 00:03:30.960 And so kind of with that line of thinking is where we kind of 42 00:03:31.240 --> 00:03:37.400 started our journey. That's really funny. And I just think of the costs 43 00:03:37.400 --> 00:03:44.919 associated with that too, for something that you don't want right right, that's 44 00:03:44.919 --> 00:03:49.639 really cool. So I've definitely checked out cub lair. I've used it to 45 00:03:50.080 --> 00:03:53.759 generate specs because I don't like write writing specs. I'm not a test driven 46 00:03:54.039 --> 00:04:00.960 person, but I do drive tests, so I'm more of a backwards test 47 00:04:01.039 --> 00:04:06.439 driven developer whatever you would call that, and so I use it. Yeah, 48 00:04:06.479 --> 00:04:10.159 it was a little script that I run sub layer and I say, 49 00:04:10.159 --> 00:04:13.919 hey, generator, you know our spec test for this Ruby file and for 50 00:04:13.919 --> 00:04:18.360 the most part it does pretty good. You know. Yeah, so I 51 00:04:18.680 --> 00:04:25.759 love the kind of syntax for just like, hey, here's a generator uses 52 00:04:25.759 --> 00:04:30.639 this prompt to do it. It's super cool. Ah. Thanks, So 53 00:04:30.759 --> 00:04:34.480 why don't you give us like the high level like how does this all work? 54 00:04:34.839 --> 00:04:40.920 Right? Yeah, a little you know a little bit of background like 55 00:04:41.040 --> 00:04:45.079 how and why it works is uh, you know, we started calling it 56 00:04:45.240 --> 00:04:49.920 promptable architecture, but the core idea was that they were going to be patterns, 57 00:04:50.160 --> 00:04:56.000 styles, ways that you build your application that lend itself well to l 58 00:04:56.040 --> 00:05:01.000 o ms taking a short request and giving you something back. And you know, 59 00:05:01.120 --> 00:05:03.720 some of the things we started out with things we kind of already know 60 00:05:03.879 --> 00:05:09.199 of, you know, clear interfaces, single responsibility principle, uh, loose 61 00:05:09.240 --> 00:05:13.439 coupling, and kind of came about those through you know, reasoning of you 62 00:05:13.480 --> 00:05:16.759 know, it's going to be much easier to add code to your app if 63 00:05:16.879 --> 00:05:21.079 you can just drop that new file in rather than have to update a whole 64 00:05:21.079 --> 00:05:28.480 bunch of other different files, and you know, other things like having these 65 00:05:28.519 --> 00:05:31.360 conventions that you can throw into the prompt and the LLLM will just believe you 66 00:05:31.399 --> 00:05:39.279 that that's your convention and follow it. And so that's that's been a lot 67 00:05:39.360 --> 00:05:42.560 of kind of our trial and error figuring out what are these what are these 68 00:05:42.560 --> 00:05:46.600 patterns? How do we how are we going to structure our applications differently to 69 00:05:46.720 --> 00:05:49.720 take advantage of this, And so, you know, one of the main 70 00:05:49.759 --> 00:05:54.959 things around the sublayer gem is, you know, every single component is very 71 00:05:54.959 --> 00:05:59.000 loosely coupled that you know, you could have one hundred generators, you could 72 00:05:59.000 --> 00:06:03.079 have a thousand generator and they won't affect each other, and so you can 73 00:06:03.360 --> 00:06:05.600 you know, one of the one of the things that we've been playing with 74 00:06:05.800 --> 00:06:10.560 is like maybe you know, don't repeat yourself or duplication in your code base. 75 00:06:10.720 --> 00:06:14.839 Isn't that bad if it takes you know, in the dots you can 76 00:06:14.839 --> 00:06:17.160 see that you can generate a generator in like, you know, ten seconds, 77 00:06:18.839 --> 00:06:20.959 Maybe it's not that bad if you know you and I both work on 78 00:06:21.000 --> 00:06:25.920 the same story. If you know, it's us just describing it and getting 79 00:06:26.000 --> 00:06:31.120 the code and it doesn't affect anything else in the code base. So so 80 00:06:31.160 --> 00:06:35.079 there's there's promptable architecture and so like that's been kind of our drive for this 81 00:06:35.480 --> 00:06:41.680 GEM, and one of the things that we've found works really well with that 82 00:06:42.040 --> 00:06:46.800 is that you really only need to give the LM one example to generate something 83 00:06:46.839 --> 00:06:51.480 new and so generate a variation off of that, which is where our kind 84 00:06:51.480 --> 00:06:57.680 of main tool blueprints comes in and basically what it does, which I think 85 00:06:57.759 --> 00:07:02.160 is you know, has been pretty common experience from a lot of the developers 86 00:07:02.160 --> 00:07:06.000 that I've talked to. Nine times out of ten, when you're working on 87 00:07:06.040 --> 00:07:10.519 a new user story, there's somewhere else in the code base that you do 88 00:07:10.680 --> 00:07:15.879 something similar, or there's a pattern you probably want to reuse, so you 89 00:07:15.879 --> 00:07:18.199 go, you look at that file, you repurpose some of the code, 90 00:07:18.199 --> 00:07:23.240 maybe copy and paste, and you know, keep the code based very similar 91 00:07:23.240 --> 00:07:29.079 to what you're doing, which we kind of realized is a manual, human 92 00:07:29.240 --> 00:07:33.240 version of RAG, which you know, I made a joke to somebody that 93 00:07:33.319 --> 00:07:39.079 like, my development style is retrieval, augmented generation, very good at googling 94 00:07:39.120 --> 00:07:43.199 and looking around the code base, and you know, it was kind of 95 00:07:43.800 --> 00:07:46.519 mostly joking, but then kind of took a step back and was like, 96 00:07:46.639 --> 00:07:50.879 actually, maybe there's something to that that's really funny. I never thought of 97 00:07:50.920 --> 00:07:55.360 it like that, but yeah, we are just we're just doing RAG. 98 00:07:58.120 --> 00:08:01.959 Right and so real, so you can actually kind of operationalize that and the 99 00:08:03.839 --> 00:08:05.720 you know, one of the things that I think for you join a new 100 00:08:05.759 --> 00:08:13.160 team and what takes so long to ramp up is that experience of, oh, 101 00:08:13.240 --> 00:08:15.560 yeah, when we when we run into that problem, we do this 102 00:08:15.639 --> 00:08:18.360 pattern over here, that's in you know, this file on you know this 103 00:08:18.519 --> 00:08:22.800 line. But like you only get that from months years of working on that 104 00:08:22.839 --> 00:08:28.079 team. That intuition whereas you index these things, you put them in a 105 00:08:28.160 --> 00:08:33.799 vector database and you query them, you can find them and uh, you 106 00:08:33.799 --> 00:08:39.240 know, I we built that. We originally actually on the sublayer GEM thought 107 00:08:39.240 --> 00:08:43.960 we were going to build a whole bunch of different you know, components and 108 00:08:43.000 --> 00:08:48.159 generators and stuff. But uh, on our product planning meeting one Monday, 109 00:08:48.759 --> 00:08:52.279 we'd like, you know, we basically thought we were going to do about 110 00:08:52.279 --> 00:08:56.960 this much work this week and had the idea of maybe we'll just generate one 111 00:08:56.000 --> 00:09:01.639 of these generators. And then a whole work weeks worth of work was done 112 00:09:01.799 --> 00:09:03.480 in a day, and it was like, take a step back, what 113 00:09:03.519 --> 00:09:09.320 did we just kind of like discover so like you know, building, we've 114 00:09:09.320 --> 00:09:16.200 found like building in that method building this method of like very loosely coupled components. 115 00:09:16.200 --> 00:09:22.120 Small components can be very powerful and quick. I didn't realize that you 116 00:09:22.279 --> 00:09:28.879 have a list of of pre defined blueprints on the site. That's really cool. 117 00:09:30.799 --> 00:09:33.600 Yeah, that's super cool. I was gonna say that, you know 118 00:09:33.639 --> 00:09:37.600 that this mentality is makes so much sense, right, Like, I feel 119 00:09:37.639 --> 00:09:43.200 like I've been at places before where they have had just like generators in general, 120 00:09:43.279 --> 00:09:46.759 like reils generators right where Okay, you just like want to start up 121 00:09:46.759 --> 00:09:50.159 a new rails app, like you use a generator, Like you want to 122 00:09:50.200 --> 00:09:54.440 create a new model that has all of the conventions, use the rails generator 123 00:09:54.440 --> 00:09:58.600 for that. Like it definitely has helped, like in a consultancy for example, 124 00:09:58.720 --> 00:10:03.159 Like you know where you have all this boilerplate stuff that you just want 125 00:10:03.200 --> 00:10:05.759 to build off of. It's like, well, you use the pre existing 126 00:10:05.799 --> 00:10:09.159 stuff and then at least you have the shell there and it's all hooked up 127 00:10:09.159 --> 00:10:13.679 and wired correctly, and then you can build off of that. But this 128 00:10:13.799 --> 00:10:16.960 is like the next step after that, right, like because then you can 129 00:10:16.000 --> 00:10:22.279 then start filling out the stuff in that boiler plate right to adapt to whatever 130 00:10:22.320 --> 00:10:28.840 it may be, which is really cool. And so like how does it 131 00:10:28.159 --> 00:10:31.480 how well does it work? Right? Like, I think that's like maybe 132 00:10:33.159 --> 00:10:35.039 what a lot of people hesitate to jump into all this with, right, 133 00:10:35.200 --> 00:10:41.480 Like you know, okay, so you have all this source material for it 134 00:10:41.519 --> 00:10:46.799 to use, Like, isn't it just making mistakes? Though? So it 135 00:10:46.960 --> 00:10:50.240 can Actually one of the things that I've kind of realized and been trying to 136 00:10:50.279 --> 00:10:54.399 figure out how to how to describe, right, But it's almost like by 137 00:10:54.679 --> 00:11:01.919 intentionally designing your classes and interfaces a certain way, you can almost like target 138 00:11:01.960 --> 00:11:07.000 the hallucinations to where you want them. Which you know it makes mistakes, 139 00:11:07.440 --> 00:11:13.600 but they a lot of times are mistakes in the style that you would probably 140 00:11:13.200 --> 00:11:18.320 want anyway, which I know it doesn't really make sense, but like so 141 00:11:18.960 --> 00:11:22.840 in the actually I think, what do you what do you mean by that? 142 00:11:24.840 --> 00:11:30.399 So like, well, you know, one of the things, if 143 00:11:30.399 --> 00:11:33.480 you you know, go ahead and generate something on the doc site, you'll 144 00:11:33.480 --> 00:11:37.600 see that we have kind of this idea of an l M output adapter and 145 00:11:37.679 --> 00:11:41.279 right now it says, you know, there's a symbol for a single string. 146 00:11:41.399 --> 00:11:46.200 So we're basically the simplest generator you can make is here's a string. 147 00:11:46.799 --> 00:11:50.559 I'm gonna send a prompt to an l M and I want this specific string 148 00:11:50.679 --> 00:11:54.879 back, and like you said, like your your example is the string could 149 00:11:54.879 --> 00:12:00.120 be our spectats for you know, some Ruby code that I've got, But 150 00:12:00.159 --> 00:12:03.960 if you ask it to build a generator that maybe gives you a list of 151 00:12:07.399 --> 00:12:11.639 a list of refactors to make in that Ruby file, it might hallucinate that 152 00:12:11.720 --> 00:12:16.000 it needs an L and output generator that generates a list And while that might 153 00:12:16.039 --> 00:12:22.440 not exist, it'll still be in the conventions and the style of that of 154 00:12:22.480 --> 00:12:26.879 the library, which might exist but also might point to Okay, well, 155 00:12:26.879 --> 00:12:30.360 here's the next feature we need to build. We need to flesh out what 156 00:12:30.399 --> 00:12:37.679 it means to get a list back. And so having those spots where there's 157 00:12:37.759 --> 00:12:41.000 nothing different from the base example to the new example you're trying to make, 158 00:12:41.679 --> 00:12:46.639 but the place is where it's different, you can basically design your code so 159 00:12:46.639 --> 00:12:50.639 that those are the pieces that change. Output adapter might be a little more 160 00:12:50.639 --> 00:12:54.039 complex. The thing that I think you run into a lot is it just 161 00:12:54.120 --> 00:12:58.600 generates a new prompt for what you're trying to do and doesn't touch anything else 162 00:12:58.639 --> 00:13:03.759 in the rest of the code. And so I'm thinking about it like I'm 163 00:13:03.840 --> 00:13:07.720 targeting the hallucination to that prompt or to the to the initializer rather than anything 164 00:13:07.720 --> 00:13:11.720 else in the in the code base or in the in the file. Gotcha, 165 00:13:11.840 --> 00:13:18.159 Yeah, that makes sense. I have so many questions around the blueprints 166 00:13:18.200 --> 00:13:22.919 aspect of it, and I mean I guess, like where does where does 167 00:13:22.919 --> 00:13:28.200 somebody even get started? Like is blueprints a good place to get started with 168 00:13:28.320 --> 00:13:31.960 the general code generation aspects of it? Or is that like a first test 169 00:13:33.039 --> 00:13:37.360 the waters with sub layer and get used to the generator's aspect of it? 170 00:13:37.480 --> 00:13:41.480 Like where do you suggest somebody? Like that's hey, I've got a team 171 00:13:41.519 --> 00:13:43.960 of people, we have a conventions in place we want to start adopting this 172 00:13:45.159 --> 00:13:50.159 process. Like where do you start? So, yeah, I guess one 173 00:13:50.480 --> 00:13:54.679 place you can start We just you saw on the site we just rolled out 174 00:13:54.679 --> 00:13:58.200 some new functionality where you can sign up you can start to put some code 175 00:13:58.200 --> 00:14:03.080 examples in privately and start to generate from them to see how it how it 176 00:14:03.159 --> 00:14:07.879 acts, and that that could be a great place to kind of start to 177 00:14:07.279 --> 00:14:11.960 start to play with UH promptability, which we've got a whole bunch of features 178 00:14:13.000 --> 00:14:18.559 we're going to roll out, kind of being able to score or point out 179 00:14:18.720 --> 00:14:26.639 areas where you know where it might make the code more promptable, and so 180 00:14:26.639 --> 00:14:28.600 so that's one aspect if you if you're curious about playing with with your own 181 00:14:28.639 --> 00:14:33.480 code. And then on the sub layer docks, we actually and I'm sorry 182 00:14:33.519 --> 00:14:39.879 this is very meta, but we've actually created this this interactive docks product which 183 00:14:39.919 --> 00:14:43.879 we're going to be rolling out as well, where live in our docks, 184 00:14:43.879 --> 00:14:48.480 where you can come and go to any of the components we have generators and 185 00:14:48.559 --> 00:14:54.919 actions right now and see it in action. Just write in what you're trying 186 00:14:54.960 --> 00:15:01.159 to do and it will give it give you reasonable There might be some things 187 00:15:01.320 --> 00:15:05.440 depending on what you're trying to do, might not be implemented yet. But 188 00:15:05.639 --> 00:15:09.279 one of the things that we're doing, if you go to the running there's 189 00:15:09.320 --> 00:15:16.639 a there's a guide on the left about running local, running a local l 190 00:15:16.799 --> 00:15:24.000 M with Lamaphile, And what Andrew on our team does is actually in the 191 00:15:24.039 --> 00:15:26.679 demo of setting it up, go to the dock site, generate the code 192 00:15:26.759 --> 00:15:33.000 for the demo, runs it on the video. So I'm sorry we're doing 193 00:15:35.240 --> 00:15:39.080 I was on earlier because I had looked at blueprints a while ago and I 194 00:15:39.519 --> 00:15:43.200 hadn't seen some player's latest stuff and this is so cool. Yeah, you 195 00:15:43.279 --> 00:15:46.120 just go on to the site and you can like, uh, you have 196 00:15:46.360 --> 00:15:52.480 just generate this stuff. It's so cool. Sorry if we're inflating your bill 197 00:15:52.519 --> 00:16:00.600 here, but that's maybe why it's there. Yeah, so I'm curious, 198 00:16:00.639 --> 00:16:06.240 like, uh, you know, definitely local inference is uh, you know, 199 00:16:06.360 --> 00:16:08.440 a hot topic and people are trying to you know, it's getting faster 200 00:16:08.799 --> 00:16:15.399 and MacBooks are running it quicker. Is that something that you recommend people start 201 00:16:15.440 --> 00:16:21.080 with? Though? Like, is it fast enough to like make make like 202 00:16:21.120 --> 00:16:27.799 the development flow smooth. It's it's not yet. I would probably recommend starting 203 00:16:27.840 --> 00:16:34.159 with something like, uh, you know, I think for for some some 204 00:16:34.200 --> 00:16:40.840 of the stuff that we're doing, you know, five dollars in uh in 205 00:16:41.799 --> 00:16:47.320 open Ai API credits can kind of help you get started. I think Google's 206 00:16:47.399 --> 00:16:51.639 has a free tier or even Gemini one point five flash, which is super 207 00:16:51.679 --> 00:16:56.480 fast. You know, we're we have we're using you know, some of 208 00:16:56.519 --> 00:17:02.559 the some of the models on our site, and you know you can also 209 00:17:02.639 --> 00:17:07.559 just go and play with way play with stuff in with private blueprints for the 210 00:17:07.640 --> 00:17:17.480 local models. I I kind of do think that's where it's where it's probably 211 00:17:17.519 --> 00:17:26.640 going for for at least a lot of stuff. The problem I think the 212 00:17:26.839 --> 00:17:32.720 The prohibitive thing about that is that you need a very powerful laptop to really 213 00:17:32.960 --> 00:17:37.000 be able to run the local models fast enough and powerful enough to actually get 214 00:17:37.279 --> 00:17:41.759 good results back. Yeah, four thousand and five thousand dollars a book pro 215 00:17:42.319 --> 00:17:48.799 or five dollars into open API or opening API credits. I remember I first 216 00:17:48.880 --> 00:17:52.480 saw the you know, the first Open models, and I go and I, 217 00:17:52.559 --> 00:17:56.079 you know, install on my Raspberry Pie is I'm thinking, oh, 218 00:17:56.160 --> 00:18:00.519 yeah, you know, like I'll just throw everything a chain of Rasberry pies. 219 00:18:00.599 --> 00:18:03.240 And it turns out you can't really like spread imprints like that. It's 220 00:18:03.279 --> 00:18:10.480 not it's not really there yet. And it's also super slow on and anything 221 00:18:10.640 --> 00:18:17.160 like even on like a nice m one, right, So it's kind of 222 00:18:17.200 --> 00:18:21.160 funny like you sit there and you're like watching reminds me of the old days 223 00:18:21.160 --> 00:18:23.839 of like waiting for an image to load on dial up or something, right, 224 00:18:23.920 --> 00:18:30.920 like you're waiting for the sentence to complete. Yeah, so I think 225 00:18:30.920 --> 00:18:33.119 it also, you know, the risky run is that it gives you the 226 00:18:33.119 --> 00:18:37.240 wrong impression, like if you're if you're just getting started. It's definitely good 227 00:18:37.319 --> 00:18:41.960 to throw five dollars at some of the like frontier models like cloud three point 228 00:18:41.960 --> 00:18:48.440 five GPT four to oh or or Gemini, just to see what's possible, 229 00:18:49.200 --> 00:18:52.319 because you can get the wrong impression if you're like like this one of the 230 00:18:52.319 --> 00:18:56.799 thing just like went off the rails and started talking to talking about something unrelated, 231 00:18:57.039 --> 00:19:02.319 right, right, Yeah, So I'm interested, like, have you 232 00:19:02.400 --> 00:19:07.359 found like certain models performed better for like certain tasks like related to code generation, 233 00:19:08.160 --> 00:19:12.640 or are the ones that are specifically better at generating Ruby code or like 234 00:19:12.960 --> 00:19:19.680 genering certain kinds of Ruby code, or even just like are there certain tasks 235 00:19:19.920 --> 00:19:25.599 with code generation like generating the specs for X or something like that where some 236 00:19:25.640 --> 00:19:30.359 models just do a better job than others, or is it kind of like 237 00:19:30.599 --> 00:19:36.960 just use any one of the top ones. I my opinion and my approach 238 00:19:37.039 --> 00:19:41.400 is kind of really just used the top ones because they're really those are going 239 00:19:41.440 --> 00:19:44.680 to be the slowest, most expensive, worst models we're ever going to have. 240 00:19:45.000 --> 00:19:48.119 Again, you know, it's just getting better and it's just scaling up, 241 00:19:48.200 --> 00:19:52.599 so like getting you know, getting used to what's what's possible, and 242 00:19:53.680 --> 00:19:56.480 you know, even if you run to the wrong run into an issue, 243 00:19:56.519 --> 00:20:00.440 like it's more about the UX and like what is this, you know, 244 00:20:00.480 --> 00:20:04.759 what does this imply? If you can do this. I was telling talking 245 00:20:04.799 --> 00:20:10.000 to somebody about you know, remember remember when everybody's like, oh ai, 246 00:20:10.079 --> 00:20:12.400 image generation is never gonna work because it keeps getting people seven fingers, and 247 00:20:12.480 --> 00:20:15.079 like, I can't remember the last time that was, you know, part 248 00:20:15.119 --> 00:20:18.839 of the conversation, but it was all about, like, here's a great 249 00:20:18.839 --> 00:20:23.240 way to generate an image. Ignore the hands for it for now. That 250 00:20:23.359 --> 00:20:30.279 reminds me. I saw recently, Uh, some some university has the the 251 00:20:30.400 --> 00:20:38.079 third thumb or you're gonna actually like you use your feet to like you like 252 00:20:38.200 --> 00:20:42.079 to get like another thumb like prosthetic on your hand so that you can like, 253 00:20:42.640 --> 00:20:45.599 you know, hold stuff while you're like doing something else with one hand. 254 00:20:47.799 --> 00:20:51.880 They showed peeling a banana with one hand. You know that guy, 255 00:20:52.079 --> 00:20:56.720 That guy's clearly aerated. Oh my gosh, it's so funny. But I 256 00:20:56.720 --> 00:21:00.319 guess it's more to your point. And actually one of the things we've been 257 00:21:00.319 --> 00:21:03.960 trying to figure out the right way to present this and maybe it'll come out 258 00:21:03.000 --> 00:21:11.279 as something interactive on the blueprint site. But we've found that different attributes are 259 00:21:11.279 --> 00:21:17.640 different, like uh, different different attributes of code quality that we've talked about 260 00:21:17.640 --> 00:21:26.039 for a long time are actually kind of quantifiable now where one of the experiments 261 00:21:26.079 --> 00:21:30.240 we were running was you know, having having really descriptive good names and then 262 00:21:30.359 --> 00:21:37.319 generating off of that blueprint that you know, GPT three point five did really 263 00:21:37.359 --> 00:21:41.640 well. When you made those names worse or less descriptive, it had a 264 00:21:41.680 --> 00:21:48.440 harder time figuring it out, but GPT four could. And so like you 265 00:21:48.440 --> 00:21:51.640 know, that kind of goes the follows a lot of the scaling for the 266 00:21:51.680 --> 00:21:56.319 other models as well, where if you give it an easier problem to solve, 267 00:21:56.599 --> 00:21:59.079 it can solve it. But as soon as that problem gets harder, 268 00:22:00.079 --> 00:22:04.640 you need a more powerful model, which I've been testing out with some people, 269 00:22:04.799 --> 00:22:08.640 like we might be able to put a dollar value on technical debt because 270 00:22:08.839 --> 00:22:14.359 it costs you a dollar per thousand tokens versus you know, ten cents per 271 00:22:14.400 --> 00:22:18.400 thousand tokens. That's really funny now, sourcing the jobs we don't want to 272 00:22:18.440 --> 00:22:25.960 do to AI, it's all a cost per token. At this point, 273 00:22:27.359 --> 00:22:33.519 it's always been. That's that's an interesting idea. Like, you know, 274 00:22:33.599 --> 00:22:38.920 definitely refactoring makes a lot of sense. I have a special GPT I made 275 00:22:40.119 --> 00:22:44.039 like that, you know, it has a bunch of resources that I've collected 276 00:22:44.039 --> 00:22:48.720 over the years at its disposal to like help me refractor stuff specific to Ruby, 277 00:22:48.960 --> 00:22:52.000 and it works great. And having that extra knowledge base like is awesome 278 00:22:52.880 --> 00:22:56.720 and I'd use it all the time just for like rubber duck sessions, right, 279 00:22:56.519 --> 00:23:00.039 which I know the next step would be to then it actually changed the 280 00:23:00.119 --> 00:23:04.880 stuff, which I haven't spent the time to do. But like now that 281 00:23:06.160 --> 00:23:07.559 you know you've made it so easy with sub layer, I'm definitely going to 282 00:23:07.640 --> 00:23:14.039 revisit it because that's definitely a task where like you could just say, hey, 283 00:23:14.480 --> 00:23:18.119 like generate a pull request for me, like you know, and all 284 00:23:18.160 --> 00:23:21.440 that get up APIs are there, like it could use it as a disposal 285 00:23:21.759 --> 00:23:26.880 Like have you played like with tool usage in that way yet As far as 286 00:23:26.920 --> 00:23:33.480 like the sub layer connection, uh yeah, And actually I guess even to 287 00:23:33.559 --> 00:23:37.559 that that task is is kind of the where we see things going that. 288 00:23:38.440 --> 00:23:45.200 Yeah, we we're kind of looking at it, like you know, the 289 00:23:45.400 --> 00:23:48.799 those kind of automations. Once you have that LM and you have you know, 290 00:23:48.839 --> 00:23:52.839 an easy way to take get some data, send it to an l 291 00:23:52.920 --> 00:23:56.920 LM in a specific way and then get something structured back, you can build 292 00:23:56.279 --> 00:24:00.559 things with that and you can build things with that very quickly, like that, 293 00:24:00.680 --> 00:24:04.519 you know, take this, take these refactoring or these you know, 294 00:24:04.880 --> 00:24:10.680 these suggestions, make a pull request or or you know, kind of anything 295 00:24:10.680 --> 00:24:14.880 you can think of. We've been we've been toying with UH with one idea 296 00:24:14.960 --> 00:24:18.720 where it kind of sits in our UH sits in our video chat for our 297 00:24:18.880 --> 00:24:25.160 planning meeting in the beginning of the week that UH watches the video, check 298 00:24:26.039 --> 00:24:30.480 manages the transcript, analyze the transcript, and then anytime, you know, 299 00:24:30.960 --> 00:24:33.000 you know, when somebody asked a question about a story and then you get 300 00:24:33.039 --> 00:24:36.000 an answer, and then you forget to put it in the story, and 301 00:24:36.000 --> 00:24:37.920 then like we you know, a few days later, we decide, here, 302 00:24:38.920 --> 00:24:41.440 you can just build an automation to kind of pull that out and put 303 00:24:41.480 --> 00:24:45.400 that in the story for you. And like that's a super quick task that 304 00:24:45.960 --> 00:24:53.920 doesn't take that much work anymore, right, And and so very long winded 305 00:24:53.920 --> 00:24:57.960 way of saying, we've explored the tool usage, but I think we're looking 306 00:24:59.039 --> 00:25:06.519 at it a little differently then how the providers are offering it. You know, 307 00:25:07.119 --> 00:25:14.839 we we'd absolutely rely on function calling, but only for UH, only 308 00:25:14.880 --> 00:25:18.839 for like the structured output aspects of it, and relying on you know, 309 00:25:18.920 --> 00:25:22.279 things that we're good at and things that we can do deterministically and reliably, 310 00:25:22.319 --> 00:25:30.599 which is you know, call APIs and uh yeah, pull make make requests 311 00:25:30.599 --> 00:25:33.720 and make make get requests and get make books requests and it files locally that 312 00:25:33.839 --> 00:25:41.160 kind of thing. But it matters to like get the parameters that you're going 313 00:25:41.200 --> 00:25:45.119 to send to those APIs and then we can do that deterministically. But like 314 00:25:45.559 --> 00:25:48.079 have the have the l ll M do what it's good at, and have 315 00:25:48.720 --> 00:25:56.599 us do what we're good at. Yeah, that makes a lot of sense. 316 00:25:56.839 --> 00:26:03.279 Uh do you see it more of like moving in the way that kind 317 00:26:03.319 --> 00:26:10.400 of GitHub is pushing with like configuration steps or is it like I'm curious if 318 00:26:10.440 --> 00:26:15.079 you've played with like the opposite of like the object oriented aspect of like okay, 319 00:26:15.079 --> 00:26:18.880 here's a prompt and you know it can have all these extra functions and 320 00:26:18.880 --> 00:26:23.039 stuff wrapped into the code aspect versus like you know, the configuration outline where 321 00:26:23.200 --> 00:26:27.200 right where it seems get hubs going that way to fit better into their CI 322 00:26:27.359 --> 00:26:32.640 structure to be honest, right where you just have this YAML file and then 323 00:26:32.839 --> 00:26:37.279 it has prompts and can perform actions and stuff in it in that way, 324 00:26:38.359 --> 00:26:47.079 like where is that flexibility more advantageous in like putting it in your code versus 325 00:26:47.119 --> 00:26:57.000 like in something like a configuration file. That's a good question, you know. 326 00:26:57.319 --> 00:27:04.799 I we were definitely coming at it from a different perspective. You know, 327 00:27:04.880 --> 00:27:12.480 I think a lot of the multifunction calling the giftub workspace are very kind 328 00:27:12.480 --> 00:27:17.559 of conversation driven, and so you have those functions that are structured ways for 329 00:27:17.599 --> 00:27:19.720 it to get in more information to do the next step in the conversation, 330 00:27:22.519 --> 00:27:29.480 which can be powerful, can be very powerful, and I mean we've seen 331 00:27:29.519 --> 00:27:36.680 a great bunch of examples of the power that is possible. But I think 332 00:27:36.799 --> 00:27:45.599 from maybe this is just you know, the thinking, wishful thinking about software 333 00:27:45.599 --> 00:27:48.799 engine the future of software engineering, that you know, a lot of the 334 00:27:48.799 --> 00:27:56.279 principles that we've found wild building, while not exactly right when working with an 335 00:27:56.359 --> 00:28:02.559 l M, still matter. And so you know, one one thing that 336 00:28:02.599 --> 00:28:06.400 I'm nervous about and haven't why I haven't really gone the route of like multiple 337 00:28:06.400 --> 00:28:11.160 different functions for it to choose from, is uh kind of the same reason 338 00:28:11.200 --> 00:28:15.440 why you know, it's it's bad to have multiple it's bad to have a 339 00:28:15.480 --> 00:28:18.880 dozen parameters in your function. Right, there was a there's a quote that's 340 00:28:18.880 --> 00:28:22.440 something like if you have if you're if your function has ten parameters, you 341 00:28:22.519 --> 00:28:30.240 probably forgot one that. Like a lot of those things while folks see and 342 00:28:30.359 --> 00:28:36.039 kind of intuition and not like backed by any science. Really, I don't 343 00:28:36.039 --> 00:28:41.079 know, still feel right to me. I still feel like you want to 344 00:28:41.160 --> 00:28:44.039 know that, like this is going to happen, This is going to happen, 345 00:28:44.079 --> 00:28:47.119 and this is going to happen, even if there's some you know, 346 00:28:47.279 --> 00:28:56.839 fuzzy non deterministic thing happening here. Yeah, I agree with that. Uh 347 00:28:56.920 --> 00:29:03.559 huh. I'm personally against the step by steps with with LLLM usage in general, 348 00:29:04.880 --> 00:29:14.599 mostly because like you kind of want observability into that and and kind of 349 00:29:14.599 --> 00:29:18.079 like a way to intercept those steps. Like as an example, I see 350 00:29:18.119 --> 00:29:23.440 like the usage over time starts to become more like reactive than it does like 351 00:29:23.720 --> 00:29:30.359 input output. So like especially once you start using agents more or things that 352 00:29:30.440 --> 00:29:33.480 are like can communicate with each other, it becomes like an observability thing, 353 00:29:33.559 --> 00:29:37.319 right, Like you end up having things that like are watching you know what 354 00:29:37.440 --> 00:29:42.400 is happening and can help redirect right if things start going wrong or interrupt, 355 00:29:44.279 --> 00:29:47.079 and when you have that step by step process, it starts to make it 356 00:29:47.160 --> 00:29:53.079 harder to change that, right because it's it's got the code flow and like 357 00:29:53.119 --> 00:29:56.279 okay, well you tell it to go to a different point in the code 358 00:29:56.319 --> 00:29:59.559 flow. Like then it's like instruction jumping, Like what are we back to 359 00:29:59.599 --> 00:30:03.480 see ag in? Right? Like assembly, Like I use Ruby so that 360 00:30:03.519 --> 00:30:07.039 I don't have to know assembly, right, like I think like Aaron Patterson 361 00:30:07.119 --> 00:30:11.920 is known notoriously known for his joke like you know, I do assembly so 362 00:30:11.000 --> 00:30:18.480 you don't have to write like uh, and so I see you know, 363 00:30:18.759 --> 00:30:22.680 the the prompt usage kind of going in that direction too, right, Like 364 00:30:22.519 --> 00:30:26.240 people write natural language and it does stuff for them, right, Like that's 365 00:30:26.319 --> 00:30:30.599 the whole driving force of all of this. It's like, you know, 366 00:30:30.200 --> 00:30:34.000 we had like a product manager come in and make some changes to get better 367 00:30:34.039 --> 00:30:38.480 outputs, right, and like then our end users see the result of that 368 00:30:40.240 --> 00:30:44.640 and like very specific you know tweaks like general people can make, right, 369 00:30:44.720 --> 00:30:48.519 Like I work at doc Simony and it's like a social network for physicians and 370 00:30:48.559 --> 00:30:53.440 they go on the chat shept and they use chat shept to like solve problems 371 00:30:53.440 --> 00:30:57.000 for themselves, right, Like they'll go and they'll you know, generate like 372 00:30:57.519 --> 00:31:03.000 a denial appeal letter for some insurance provider, and they just like then they 373 00:31:03.079 --> 00:31:06.039 use it, right, and so they use natural language to make that happen, 374 00:31:06.440 --> 00:31:08.160 and like obviously like they have some issues like that they have. There's 375 00:31:08.160 --> 00:31:14.640 a lot of handholding there, right, and and so like the whole point 376 00:31:14.720 --> 00:31:18.680 is like to help drive people to use the natural language. And then s 377 00:31:18.759 --> 00:31:25.960 as developers are like to introduce like the advances of like extending it, right, 378 00:31:26.119 --> 00:31:29.400 like okay, chat, chap, all these models like yeah, they 379 00:31:29.440 --> 00:31:33.119 give you text Like, well, because of that, you can have to 380 00:31:33.200 --> 00:31:36.960 generate any number of things because we're coders, right, Like we can start 381 00:31:37.000 --> 00:31:41.359 pipelining and obfuscating that you know, input output that the alms give us to 382 00:31:41.920 --> 00:31:45.839 mutate it in different ways, right, And the users don't have to see 383 00:31:45.839 --> 00:31:48.519 that, right, but they could still give it the same inputs that us 384 00:31:48.519 --> 00:31:52.519 as developers would even give it. Right. That's like that's where I see, 385 00:31:52.559 --> 00:31:55.440 like, all right, we got to get away from this, like 386 00:31:55.480 --> 00:31:59.880 all right, step by step instruction Like that's a programmer like mentality, right, 387 00:32:00.240 --> 00:32:05.119 Like we're starting the design evolution is going to be really interesting to watch 388 00:32:05.200 --> 00:32:08.880 for developers. I think I'm curious. I'm curious, like what what your 389 00:32:08.920 --> 00:32:15.240 thoughts are on that aspect, Like where do you see like like the design 390 00:32:15.440 --> 00:32:21.680 driving force of a lot of this stuff moving? You know. So my 391 00:32:21.799 --> 00:32:27.720 previous startup was actually a a prototyping tool for voice apps, and so this 392 00:32:27.839 --> 00:32:31.440 was like Rata's Google, Google Home and Alexa were coming out. We built 393 00:32:31.480 --> 00:32:37.480 a tool for uh, you know, prototyping out different conversation flows and that 394 00:32:37.680 --> 00:32:43.119 same thing, a step by step back and forth kind of conversation. And 395 00:32:43.279 --> 00:32:49.759 you know, one of the things that we kind of realized is that conversation 396 00:32:49.920 --> 00:32:57.160 is kind of just like one one modality, and we got acquired by Adobe 397 00:32:57.359 --> 00:33:00.240 built that into XD because we, you know, firmly believe that you know, 398 00:33:00.920 --> 00:33:06.559 you you want multiple modalities for inputting information to a machine. Sometimes it's 399 00:33:06.559 --> 00:33:09.759 a voice because you're in a you know, your your hands are busy and 400 00:33:09.759 --> 00:33:14.440 you can't actually press a button, or you know, you're cooking or something. 401 00:33:15.799 --> 00:33:21.519 But other times you're on the subway and like asking asking your phone for 402 00:33:21.599 --> 00:33:24.559 your bank account information, it's probably not a great idea, but like pressing 403 00:33:24.559 --> 00:33:30.079 the buttons on your phone's probably fine. And so all that being said, 404 00:33:30.160 --> 00:33:34.240 I think I think where things are going. I don't think we're going to 405 00:33:34.279 --> 00:33:38.000 throw out like the last However, many decades of UX research that we have 406 00:33:38.160 --> 00:33:45.480 into like interfacing with machines, that there are going to be a lot of 407 00:33:45.480 --> 00:33:49.960 times where you have buttons and forms too, you know that people can understand 408 00:33:50.039 --> 00:33:54.960 and intuitively grasp right like can you can you imagine going to Facebook and having 409 00:33:55.039 --> 00:33:59.519 to like write a sequel career or asque like the Facebook bot to add somebody 410 00:33:59.559 --> 00:34:04.920 as a friend versus you know, just clicking a friend button. So I 411 00:34:04.920 --> 00:34:07.119 think it's going to be I think it's going to be a mix, even 412 00:34:07.160 --> 00:34:13.079 if behind the scenes there's going to be you know, this non deterministic natural 413 00:34:13.159 --> 00:34:19.880 language API that that way that we've been using, but like from a user 414 00:34:19.920 --> 00:34:24.000 standpoint, not requiring them to understand the right way to prompt that makes sense. 415 00:34:24.119 --> 00:34:30.599 Yeah, that's interesting. I mean prompt I hate to say prompt engineering, 416 00:34:30.599 --> 00:34:36.119 but trying to get get the right thing to do what you want is 417 00:34:36.159 --> 00:34:42.159 still frustrating to me. It is, like it's so it reminds me of 418 00:34:42.199 --> 00:34:45.239 like when I first started to learn how to code, to be honest, 419 00:34:45.719 --> 00:34:52.000 which is a little funny. Likes so much fun to bring that excitement back, 420 00:34:52.079 --> 00:34:55.239 right, it really is. Yeah, And uh, I don't know, 421 00:34:55.280 --> 00:34:59.360 are there other tools that you use for that, Like, you know, 422 00:34:59.480 --> 00:35:09.159 prompt is aspect that you'd recommend to people. I radical, I definitely 423 00:35:09.480 --> 00:35:15.519 kind of out out there on the prompt design stuff. I you know, 424 00:35:15.679 --> 00:35:21.440 you we've got an open source server for blueprints. We've got sub layers open 425 00:35:21.440 --> 00:35:25.519 source so you can see the prompts that we're using. We're very minimal most 426 00:35:25.559 --> 00:35:34.159 of the prompt design for us, which it's maybe it's prompt designed, but 427 00:35:34.199 --> 00:35:37.119 it's all about like what code are we going to give it and like making 428 00:35:37.119 --> 00:35:44.119 the problem easy. And so we haven't really used many tools because the there's 429 00:35:44.159 --> 00:35:50.840 maybe ten lines of prompt that we rely on. But I think I think 430 00:35:50.880 --> 00:35:57.159 thinking about it like programmers is helpful that you know, if you have you 431 00:35:57.239 --> 00:36:00.280 know, for you know, we're talk a little bit about this of just 432 00:36:00.320 --> 00:36:06.639 like what what makes code promptable that if you have to say, like I 433 00:36:06.679 --> 00:36:08.239 need this input, this input, this input, then do this other thing 434 00:36:08.320 --> 00:36:13.239 and then you know, do that, it's probably bad code. It's also 435 00:36:13.280 --> 00:36:19.559 probably not very promptible code. And like taking that further, if you're asking 436 00:36:19.599 --> 00:36:23.840 the LLM to do this and this and this and that, it's probably going 437 00:36:23.880 --> 00:36:28.079 to screw up somewhere, and so like break those apart the same way that 438 00:36:28.119 --> 00:36:30.599 you you know, you would much rather not have a thousand line method. 439 00:36:31.039 --> 00:36:37.159 You break it out into individual pieces that you can individually test are individually reliable, 440 00:36:37.440 --> 00:36:39.159 and then piece those together. Yeah, that's kind of my approach to 441 00:36:42.280 --> 00:36:46.639 I do use like sometimes we'll use like, you know, the models against 442 00:36:46.639 --> 00:36:52.079 each other to build and iterate on, which is helpful, like how would 443 00:36:52.079 --> 00:36:53.880 you improve this? And then go to the other one and say, well, 444 00:36:54.119 --> 00:36:57.639 what do you think about that? Like how would you improve this from? 445 00:36:57.719 --> 00:37:02.159 It's like they often will, like you know it do better if you 446 00:37:02.320 --> 00:37:07.199 use the like a competing like model against uh the other, which is a 447 00:37:07.199 --> 00:37:12.599 little funny, and especially if you tell it like oh, like you're in 448 00:37:12.599 --> 00:37:19.800 a competition to like make the best prompt, right, Yeah, I mean 449 00:37:19.840 --> 00:37:22.159 I haven't tried to like take the deep dive to be like this is a 450 00:37:22.199 --> 00:37:25.599 matter of life or death, right like, uh, and you can also 451 00:37:25.639 --> 00:37:30.800 win a million dollars, right, like just shove all of the you know, 452 00:37:30.960 --> 00:37:32.800 prompt tricks, which is kind of funny, like those are going away, 453 00:37:34.000 --> 00:37:37.800 like it's getting harder and harder to make those improvements work. But yeah, 454 00:37:38.559 --> 00:37:44.039 I don't know if they're they're doing that training, but but yeah a 455 00:37:44.039 --> 00:37:47.480 little funny. Yeah, yeah, that brings me to my next point, 456 00:37:47.559 --> 00:37:52.239 like, uh, you know, blueprint seems like it's like ripe for fine 457 00:37:52.280 --> 00:37:57.239 tuning, right, Like you have these pre existing conditions and pre existing formatting. 458 00:37:57.800 --> 00:38:00.400 Uh, you know, it seems like this would make a great flow 459 00:38:00.559 --> 00:38:05.480 to fine tune the models to give the outputs in more desirable formats. Like 460 00:38:05.760 --> 00:38:10.440 do you even see that as like being a meaningful improvement or is it are 461 00:38:10.480 --> 00:38:15.519 the better like just you pay to use better models of this one? I 462 00:38:15.559 --> 00:38:21.719 mean, the my my thinking right now is the is you know, until 463 00:38:21.760 --> 00:38:25.000 we until we get to the you know, the end of this curve of 464 00:38:25.199 --> 00:38:30.639 you know, the costs dropping. I think they drop by the quarter every 465 00:38:31.199 --> 00:38:37.159 eighteen months or something like that, where you know, it keeps getting cheaper 466 00:38:37.519 --> 00:38:45.360 and new models come out, which I think until until that stops or slows 467 00:38:45.400 --> 00:38:52.239 down to really just just keep going. And you know, I wrote a 468 00:38:52.239 --> 00:39:00.320 post a little while ago titled waste Inferences, which basically kind of that call 469 00:39:00.440 --> 00:39:04.760 or just like you know, we can actually build very simple applications and do 470 00:39:05.039 --> 00:39:09.039 very simple things even if yeah, it costs a little bit more to you 471 00:39:09.039 --> 00:39:13.920 know, use GPT four oh or Cloud three point five today, but six 472 00:39:13.920 --> 00:39:15.559 months from now it's going to cost half as much, and then six months 473 00:39:15.559 --> 00:39:20.239 from that it's going to cost half as much as that. And so like 474 00:39:21.039 --> 00:39:25.679 if you if you over optimize today, that's time you could be spent. 475 00:39:27.400 --> 00:39:32.079 Time you could be spending finding new patterns for when when these things cost nothing. 476 00:39:34.320 --> 00:39:37.960 I don't if you remember. I mean I remember early on you used 477 00:39:37.000 --> 00:39:42.239 to get charged. Some people still get charged for bandwidth, but like the 478 00:39:42.679 --> 00:39:45.239 like entry tier for any kind of web hosting was also a bandwidth charge, 479 00:39:45.280 --> 00:39:49.440 and like if somebody linked to an image on your site could go bankrupt. 480 00:39:50.599 --> 00:39:53.880 But like I haven't seen anybody you know, really talk about three. But 481 00:39:54.039 --> 00:40:00.320 like the costs are so minuscule now that it's not even like called out funny 482 00:40:00.360 --> 00:40:05.119 you mentioned that. I remember being on a call there was like some kind 483 00:40:05.159 --> 00:40:08.679 of like a I don't know, like presentation or something that a lot of 484 00:40:08.679 --> 00:40:13.440 the Open Eye engineers that were giving to a bunch of companies. I was 485 00:40:13.519 --> 00:40:16.480 like enough to join in, and you know a lot of people were asking 486 00:40:16.639 --> 00:40:23.039 like, you know, it is fine tuning like worth it for these models, 487 00:40:23.119 --> 00:40:28.119 like and they're you know, how do you get even like your pre 488 00:40:28.119 --> 00:40:32.239 existing machine learning teams on board to start using this stuff right, And what 489 00:40:32.280 --> 00:40:37.079 they were basically saying bluntly was like, you know, the models are getting 490 00:40:37.159 --> 00:40:43.280 so good that fine tuning is really not going to be worth it over the 491 00:40:43.320 --> 00:40:49.639 long run in general, and that like basically it would be better for your 492 00:40:49.760 --> 00:40:54.639 like machine learning team even to get used to using the large language models then 493 00:40:54.679 --> 00:41:00.880 it would be to have them like you know, do like machine learning to 494 00:41:01.280 --> 00:41:06.639 like for a recommendation system or something like that, like over time, Like 495 00:41:07.199 --> 00:41:09.519 you would be more advantageous to like basically get your foot in the door to 496 00:41:09.880 --> 00:41:16.760 the models and work through prompting than it would be to like get trained data 497 00:41:16.840 --> 00:41:22.719 on some kind of like you know, PyTorch or something like that for whatever 498 00:41:22.760 --> 00:41:25.239 you're trying to do, like because over the long run, you still have 499 00:41:25.239 --> 00:41:29.559 to maintain all that, like you have to worry about the same things like 500 00:41:29.760 --> 00:41:35.599 is it being effective doing its job right and then also improving those aspects of 501 00:41:35.639 --> 00:41:42.159 it, whereas the models themselves, like they already are improving themselves. And 502 00:41:42.199 --> 00:41:44.599 you get and you can make it, like you are, right, like 503 00:41:44.639 --> 00:41:52.000 where you can get it to improve itself like iteratively, and so it's interesting 504 00:41:52.079 --> 00:41:55.280 to see, you know, it's I'm kind of torn on it though, 505 00:41:57.400 --> 00:42:02.000 because I do I have worked with some fine tuning aspects to like get it 506 00:42:02.039 --> 00:42:07.480 to conform to very specific outputs as an example, it and it does like 507 00:42:07.559 --> 00:42:14.719 perform much better for those very like the you know, Obie's book is great 508 00:42:14.800 --> 00:42:19.280 like describing the narrow Path, which is like the perfect example of like how 509 00:42:19.440 --> 00:42:22.239 like all of this works really right, It's like the smaller and narrower the 510 00:42:22.280 --> 00:42:25.719 scope you can make the task, like, the better that it performs, 511 00:42:27.159 --> 00:42:30.079 and like, you know, the problem with it now is like GBT four 512 00:42:30.199 --> 00:42:36.440 is like such a vast like knowledge base, right that like it starts to 513 00:42:36.559 --> 00:42:40.480 perform poorly the greater the tasks that you ask of it, or the more 514 00:42:40.480 --> 00:42:43.920 that you ask of it. So sure it can like start to distill and 515 00:42:43.960 --> 00:42:46.480 do a pretty good great job most of the time, but like the more 516 00:42:46.559 --> 00:42:50.440 you start to like ask for it, like as an example, like you 517 00:42:50.480 --> 00:42:55.079 know, what which of these categories does this content fall into? Right, 518 00:42:55.199 --> 00:42:59.639 It's gonna make a lot of mistakes because you're asking it to like match up 519 00:42:59.639 --> 00:43:01.320 too many things at once, right, But if you're just like, well 520 00:43:01.440 --> 00:43:05.679 is it this category yes or no, it's like gonna get almost one hundred 521 00:43:05.679 --> 00:43:09.000 percent of the time, right, And so like it's more think thinking like 522 00:43:09.360 --> 00:43:13.960 those aspects of it, which is kind of funny because like the pricing model 523 00:43:14.039 --> 00:43:16.840 is like perfect for them, right, Like the more that you use it, 524 00:43:17.119 --> 00:43:21.000 uh, you know, the more money they make. But also like 525 00:43:21.360 --> 00:43:23.599 you know, the the more that you need to use it, the times 526 00:43:23.639 --> 00:43:28.920 wise, right, starts to increase just by the nature of the design, 527 00:43:28.960 --> 00:43:35.440 which is kind of funny. I just realize I put a disclaimer on that 528 00:43:35.480 --> 00:43:46.599 waste not being ab I that's funny. But I guess the one thing I 529 00:43:46.599 --> 00:43:52.440 would say on that, and like even to to your question, and you 530 00:43:52.440 --> 00:43:55.920 know, maybe started could have started the I said this a lot earlier that 531 00:43:57.039 --> 00:44:00.079 like a lot of these these these things are so new, right that like 532 00:44:00.679 --> 00:44:05.480 a lot of what we're doing is a lot of what we're doing. There's 533 00:44:05.519 --> 00:44:07.800 a lot of you know, papers coming out around a lot of these techniques. 534 00:44:08.440 --> 00:44:13.320 But I think everybody's just really you know, they have the theories of 535 00:44:13.320 --> 00:44:16.880 why these things work. They can get it to work, and that might 536 00:44:16.960 --> 00:44:21.280 work for them, it might work for this. There might be pieces of 537 00:44:21.320 --> 00:44:27.280 what people are doing that are right or wrong. I don't think anybody knows 538 00:44:27.320 --> 00:44:30.599 for sure, and like especially how uncertain the future is that I think. 539 00:44:31.840 --> 00:44:36.960 I think it's you know, these opinions are mostly mine from the way that 540 00:44:36.960 --> 00:44:39.960 we're approaching it, and I could be completely wrong. I've seen successes and 541 00:44:40.000 --> 00:44:45.039 I have seen like evidence that we're right about some things, but that doesn't 542 00:44:45.360 --> 00:44:49.440 mean that there are a whole bunch of other techniques and patterns and ways to 543 00:44:49.880 --> 00:44:54.800 do it. And so like that's the's you know, at this also kind 544 00:44:54.800 --> 00:45:00.440 of terrifying that, like, you know, but it's also super exciting, 545 00:45:00.519 --> 00:45:05.480 right, Like it's so fun to see, you know, to like expand 546 00:45:05.519 --> 00:45:07.280 the way that you're thinking about it when you see somebody try something that you 547 00:45:07.280 --> 00:45:12.679 wouldn't have thought to try or tried and failed and then see them succeed with 548 00:45:12.719 --> 00:45:17.760 it, and you know, expand your understanding of these things. So if 549 00:45:19.039 --> 00:45:22.679 something you know, for anybody listening, like if something interests you and like 550 00:45:22.039 --> 00:45:27.119 you, it's super easy to try out. So like don't let don't let 551 00:45:27.119 --> 00:45:30.440 somebody say saying like this doesn't work or this does work stop you from just 552 00:45:30.519 --> 00:45:35.239 trying it, because it's usually like a two cent API call to find out 553 00:45:35.480 --> 00:45:39.559 if your idea there's something there. Yeah, for sure. Now it's never 554 00:45:39.559 --> 00:45:44.559 been a greater time to experiment because you can just prove people wrong so easily 555 00:45:45.039 --> 00:45:49.960 or or prove yourself right. You know, like it's it's very easy to 556 00:45:50.039 --> 00:45:53.440 test that out, which is honestly great. With all the new tools that 557 00:45:53.480 --> 00:45:58.880 are out there, you know, perfect time to plug the Ruby AI builders 558 00:45:58.880 --> 00:46:02.039 discord. You know, people just like drop in the wildest stuff and you're 559 00:46:02.079 --> 00:46:07.679 like, oh, I could do that, like, you know, definitely 560 00:46:07.800 --> 00:46:13.639 come join us. Yeah, it was a couple of weeks ago and somebody 561 00:46:13.679 --> 00:46:21.760 found that it could you could send it Base sixty four and coded information and 562 00:46:21.800 --> 00:46:24.159 it would be fun. And then so that was like a really fun thing 563 00:46:24.519 --> 00:46:29.920 on a Friday night night. How does how is it able to do this? 564 00:46:30.519 --> 00:46:34.519 That's awesome? Yeah, I wonder if you can like squirt around some 565 00:46:34.559 --> 00:46:40.320 of the token issues compression. I don't know, I wonder if it makes 566 00:46:42.519 --> 00:46:50.920 yeah, it any slower at responding. I haven't noticed. So I wanted 567 00:46:50.960 --> 00:46:57.519 to briefly touch on you know, for those that don't know you've started hosting 568 00:46:57.559 --> 00:47:01.280 a what what seems to be a repeat event in New York City, Uh, 569 00:47:01.519 --> 00:47:06.599 the Ruby AI meet up? How did you get started with that? 570 00:47:06.920 --> 00:47:10.199 Like? What what prompted it? Like, you know, I attended the 571 00:47:10.239 --> 00:47:15.199 last one. I thought it was really really great to meet new Rubias like 572 00:47:15.239 --> 00:47:17.239 in the AI space, but even non Rubius were there, which was pretty 573 00:47:17.239 --> 00:47:22.559 cool to see. Yeah, how did you get involved in that? I've 574 00:47:22.559 --> 00:47:27.199 gotta you know, I've got a backstory of course, so you know, 575 00:47:27.840 --> 00:47:30.840 I one of the we got asked a lot, you know, as we 576 00:47:30.840 --> 00:47:37.239 were building this startup why Ruby? And I am as you can maybe if 577 00:47:37.280 --> 00:47:38.920 you can see some of the books here, a little bit of a software 578 00:47:38.960 --> 00:47:44.599 historian and really love looking and digging deep into like the reason for, uh, 579 00:47:44.880 --> 00:47:49.559 why we do what we do and kind of realized that, you know, 580 00:47:49.719 --> 00:47:53.880 every time there's a new platform shift or new kind of thing we can 581 00:47:53.920 --> 00:47:59.199 do with computers or as things change, there's like there's kind of a pendulum 582 00:47:59.239 --> 00:48:02.159 swing, and you you know, you kind of see like object oriented programming 583 00:48:02.199 --> 00:48:07.519 and the guy kind of coming out of small talk and then you know that's 584 00:48:07.559 --> 00:48:12.559 more of the informal kind of like fuzzy reasoning kind of approach, which then 585 00:48:12.840 --> 00:48:15.960 you know, you got it got more formalized through like the eighties and nineties 586 00:48:16.000 --> 00:48:21.519 when with like C plus plus and Java, and then you know, database 587 00:48:21.559 --> 00:48:28.000 back web applications became a thing, and we found that the dynamic language is 588 00:48:28.079 --> 00:48:31.400 Ruby, Python. JavaScript made it possible to test out a whole bunch of 589 00:48:31.440 --> 00:48:36.760 ideas and find the pattern and find what we how what it means to have 590 00:48:36.840 --> 00:48:40.320 like these dynamic web applications. And then through the twenty tens or so, 591 00:48:40.639 --> 00:48:45.760 it's become more formalized with Typescript and Rust, and we're not really trying out 592 00:48:45.760 --> 00:48:49.679 all these different variations we've got. We know what we want to build and 593 00:48:49.719 --> 00:48:52.119 we want that to be stable and scalable. But now with all lems, 594 00:48:52.159 --> 00:48:58.119 nobody knows right. And so my thinking was, you know, I need 595 00:48:58.119 --> 00:49:00.119 a dynamic language where it's going to get out of the way and I can 596 00:49:00.159 --> 00:49:07.519 test out and interface a DSL and idea super quickly. And started thinking back 597 00:49:07.559 --> 00:49:12.800 to like when I was getting into Ruby, how there's just like this explosion 598 00:49:13.079 --> 00:49:15.880 of different gems, people trying things out, Like why the Lucky Stiff was 599 00:49:15.880 --> 00:49:22.679 putting things out like rapid fire, and you know, something like camping comes 600 00:49:22.719 --> 00:49:28.199 about which inspires Sinatra. I don't know if this is direct, but in 601 00:49:28.239 --> 00:49:31.199 my mind this is this is my head canon that camping inspires Sinatra, which 602 00:49:31.199 --> 00:49:35.519 inspires Flask and express, which is like kind of what a lot of things 603 00:49:35.519 --> 00:49:42.800 are built on now and we kind of are in that period now where like 604 00:49:43.719 --> 00:49:45.880 we want to get bring people together and kind of like what I said before, 605 00:49:46.119 --> 00:49:49.960 just like I don't know, you know, maybe I'm right about some 606 00:49:50.000 --> 00:49:52.039 things, I'm wrong about some things, but like we need to get together 607 00:49:52.280 --> 00:49:55.840 and share those ideas, bounce those ideas off each other, get into arguments, 608 00:49:57.920 --> 00:50:00.280 and like sho show each other cool stuff that were building. And so 609 00:50:00.440 --> 00:50:04.559 that's where the Happy Hour came about. It was like I feel like, 610 00:50:04.880 --> 00:50:07.800 I feel like Ruby is right for this. I feel like there is uh, 611 00:50:09.000 --> 00:50:12.960 there is you know, from talking to people one on one, there 612 00:50:13.039 --> 00:50:17.159 is definitely that feeling of like where did that magic go? So I was 613 00:50:17.159 --> 00:50:20.199 like, ah, let's see if we can let's see if we can bring 614 00:50:20.239 --> 00:50:24.440 some rubies together in New York and uh, you know, test that test 615 00:50:24.480 --> 00:50:30.360 that theory out and so we did and it was it was a huge I 616 00:50:30.360 --> 00:50:32.559 feel like it was a huge success. Like you know, got a chance 617 00:50:32.559 --> 00:50:36.920 to meet you. We had people coming from all over the East Coast to 618 00:50:37.000 --> 00:50:43.000 come come to it and yeah, got uh you know, we've sponsors for 619 00:50:43.039 --> 00:50:46.840 the next one coming up in July July twenty fourth test, double infield fire 620 00:50:46.920 --> 00:50:52.440 hydrant. Saw the you know, saw the excitement, saw the energy, 621 00:50:53.119 --> 00:50:58.320 and uh, yeah, we're trying to make the next one even bigger, 622 00:50:58.400 --> 00:51:02.320 bringing more people together. I think another head canon. I don't know if 623 00:51:02.360 --> 00:51:06.639 this is exactly right. The story you hear about like GitHub starting was you 624 00:51:06.679 --> 00:51:09.719 know, the you know, happened at a booth at a sports bar after 625 00:51:10.639 --> 00:51:13.519 after a meet up where they were like, you know, we should try 626 00:51:13.519 --> 00:51:16.199 this, and I don't know, I wasn't there. I don't know the 627 00:51:16.199 --> 00:51:20.159 conversation, but like that's the story you hear, and I wanted to try 628 00:51:20.199 --> 00:51:25.960 to, Yeah, see if we can recreate some of that magic. Yeah 629 00:51:27.000 --> 00:51:29.440 that's awesome. You know, I appreciate it, like I know a lot 630 00:51:29.440 --> 00:51:32.840 of people appreciate it. Like, yeah, that was the first Ruby event 631 00:51:32.840 --> 00:51:37.400 I've been to in a Quinton quite a while. And just because it was 632 00:51:37.440 --> 00:51:42.480 close and you know, like there wasn't you know it, it reminded me 633 00:51:42.519 --> 00:51:46.960 of the Unconference tracks right where you know, they're just like come hang out 634 00:51:47.039 --> 00:51:51.440 at a conference like and not go to any talk, and it like removes 635 00:51:51.480 --> 00:51:53.519 all of the barriers to like meet people, right like, which is kind 636 00:51:53.519 --> 00:51:57.599 of funny because you go to the pay for the conference so that you can 637 00:51:57.639 --> 00:52:02.440 listen to the talks, but then they end up having like you know, 638 00:52:02.920 --> 00:52:07.280 somebody's just like it ends up being like a lightning talks but like side side 639 00:52:07.320 --> 00:52:10.280 tracks, right, which is kind of funny. But it has that kind 640 00:52:10.280 --> 00:52:14.360 of like vibe to it, right where like people are just like talking about 641 00:52:14.400 --> 00:52:16.440 like all of their excitement, which is like all this stuff is so exciting, 642 00:52:16.719 --> 00:52:20.400 and where do you start and like what are people working on? Like 643 00:52:20.840 --> 00:52:24.000 you know, you don't know until and like that. The meetups was like 644 00:52:24.320 --> 00:52:28.760 definitely the culmination of everybody just being like, here's what we're working on. 645 00:52:29.039 --> 00:52:31.800 Like this is exciting, you know, and it was super cool to see. 646 00:52:35.559 --> 00:52:39.679 So I'm looking forward to going again. Yeah yeah, actually I guess 647 00:52:39.719 --> 00:52:45.920 I can. I don't have the the access to the comments, but send 648 00:52:45.920 --> 00:52:50.719 this to you link to the link to the event with anybody. Oh awesome, 649 00:52:50.920 --> 00:52:54.079 yep, we'll drop it in here. Yeah, come join us. 650 00:52:54.679 --> 00:53:00.000 It was so much fun and yeah, like you said people travel for it. 651 00:53:00.119 --> 00:53:05.199 You know, it's really cool to see. Yeah, it's funny. 652 00:53:05.440 --> 00:53:12.320 You know you don't realize like how yeah, how tough the last few years 653 00:53:12.360 --> 00:53:16.679 have been being you know, remote and hybrid that like you don't really get 654 00:53:16.679 --> 00:53:21.800 a chance to like as much, I guess anymore, just like meet and 655 00:53:21.840 --> 00:53:27.400 hang out. And yeah, got a lot of feedback of just like like 656 00:53:27.559 --> 00:53:29.960 that same you know, your same comment, like I haven't been to a 657 00:53:30.000 --> 00:53:31.599 Roovie event in a long time. I haven't, you know, I haven't 658 00:53:31.599 --> 00:53:37.079 seen this many people at a Ruby event for years. Like there's definitely you 659 00:53:37.119 --> 00:53:44.239 know, yeah, something we've lost by like going mostly remote and hybrid, 660 00:53:44.360 --> 00:53:50.159 but being in person kind of brings back for me anyway. Yeah, yeah, 661 00:53:50.320 --> 00:53:54.960 I remember my first Reels comp or was it in Maryland? And uh 662 00:53:57.280 --> 00:54:01.639 yeah, it's definitely even intimidating in a larger setting and having smaller groups, 663 00:54:02.360 --> 00:54:07.920 you know, it definitely is easier to to chat chat with people and social 664 00:54:07.000 --> 00:54:13.320 ats, I think. But you know, one thing, I think we 665 00:54:13.320 --> 00:54:15.639 we had this moment when we first met. It was like I know your 666 00:54:15.760 --> 00:54:20.199 name, but like I know you're from the Avatar and discord right right, 667 00:54:22.639 --> 00:54:27.480 uh bridge that gap a little bit yeah. I mean to be honest, 668 00:54:27.559 --> 00:54:31.320 if extras had like a you know, print your avatar, like I feel 669 00:54:31.320 --> 00:54:35.360 like a lot it would make like, you know, these social connections easier 670 00:54:35.400 --> 00:54:44.400 to see. Uh, just slap that on. That's funny. So you 671 00:54:44.400 --> 00:54:46.480 where are you going next with this? Like where where's sub layer heading? 672 00:54:46.519 --> 00:54:52.920 Where's blueprints? Like where do you where do you see like your next phase 673 00:54:52.840 --> 00:54:58.760 in all of this? I think the next thing and you know, with 674 00:54:58.880 --> 00:55:02.639 the topic of this is Ruby as a sleeping giant for AI application development. 675 00:55:02.840 --> 00:55:07.079 Like I think it's you know, we've kind of laid the foundation here with 676 00:55:07.599 --> 00:55:10.920 the framework, new version of trying to get the new version out last night. 677 00:55:10.920 --> 00:55:15.400 But I had a couple of things I had to do. Lay the 678 00:55:15.400 --> 00:55:19.960 foundation with the framework, lay the foundation with blueprints, and then really kind 679 00:55:20.000 --> 00:55:24.800 of show how easy it is to show and like spread the word of how 680 00:55:24.840 --> 00:55:30.679 easy it is to build these l empowered applications. That changes a lot of 681 00:55:31.000 --> 00:55:37.760 you know, changes the way when do we think about what's possible? Right, 682 00:55:39.039 --> 00:55:45.199 I think there's still a lot of it's hard to it's hard to go 683 00:55:45.280 --> 00:55:50.440 from a place where you know a project in twenty eighteen even you're like, 684 00:55:50.559 --> 00:55:53.239 okay, that'll be about a team of like fifteen people. It'll take you 685 00:55:53.320 --> 00:55:57.760 know, a year or two to do, turning into like a ten cent 686 00:55:57.800 --> 00:56:02.239 API call like that. Mind it's like it's it's wild right now. I'm 687 00:56:02.239 --> 00:56:06.880 sure you've you've experienced it with just like some of the things these models can 688 00:56:06.920 --> 00:56:14.119 do, and so really like our next steps are bringing more and more of 689 00:56:14.159 --> 00:56:20.159 that to uh to the forefront of just like what all what are all of 690 00:56:20.159 --> 00:56:23.519 these things that you can do that take you half a day to automate, 691 00:56:23.559 --> 00:56:29.239 where like previously probably just ignored it because it was so costly and so boring 692 00:56:30.199 --> 00:56:35.559 and mind numbing. And so building out the framework more bringing bringing more use 693 00:56:35.599 --> 00:56:40.360 cases to the forefront, and then you know, expanding blueprints to make that 694 00:56:40.480 --> 00:56:45.199 even faster so that more and more of these examples that you have, the 695 00:56:45.280 --> 00:56:52.039 more things they can do and more things they can generate. And then so 696 00:56:52.119 --> 00:56:55.719 trying to experiment with the interactive docks piece getting ready to roll out kind of 697 00:56:57.400 --> 00:57:00.800 making it possible for anybody with API docs to uh have what we have in 698 00:57:00.840 --> 00:57:05.519 our docks uh see if see if there's interest there, So even if you're 699 00:57:05.559 --> 00:57:08.960 not even if you're not on the forefront of like trying to turn million dollar 700 00:57:09.039 --> 00:57:15.920 projects into tens on API calls. You can still see some benefit today. 701 00:57:16.880 --> 00:57:21.039 You know. What I would love to see is uh, just being able 702 00:57:21.079 --> 00:57:24.519 to give like a GitHub link to like lines or something like that and say, 703 00:57:24.599 --> 00:57:29.920 like do something with this. I think that could make a really cool 704 00:57:30.039 --> 00:57:32.320 like example of hey, like here are the lines to some code, like 705 00:57:32.840 --> 00:57:37.480 you know, change this or something or make it say something silly. I 706 00:57:37.519 --> 00:57:43.079 don't know, Yeah, I don't know. Yeah, like there's but there's 707 00:57:43.079 --> 00:57:47.000 so much stuff you could do like that, Like it's so easy. Yeah, 708 00:57:47.079 --> 00:57:51.079 I would love to see more, like more ease of use, right, 709 00:57:51.199 --> 00:57:54.079 Like it's it's already easy, but like still people don't you know know 710 00:57:54.079 --> 00:57:58.880 where to start, uh, which is which is a little bizarre at the 711 00:57:58.920 --> 00:58:01.039 same time, right, like, uh, it is as easy as just 712 00:58:01.280 --> 00:58:07.760 typing, right. I hate to like distill it down to just that, 713 00:58:07.000 --> 00:58:09.920 but like, uh, you know, at this point you can just go 714 00:58:10.000 --> 00:58:15.559 to like you know, Microsoft dot com and like use chat GPT for free, 715 00:58:15.679 --> 00:58:22.360 you know, like it is that easy to get started, which unfortunately 716 00:58:22.440 --> 00:58:27.280 you got to buy into that, right right, Yeah, I mean I 717 00:58:27.280 --> 00:58:30.039 think that's that's the thing, and I think there's you know, one of 718 00:58:30.039 --> 00:58:32.920 the things that I've been trying to figure out is how to I've been trying 719 00:58:32.920 --> 00:58:37.559 with that that example of like you know you're already doing rat rag is just 720 00:58:37.599 --> 00:58:40.880 you googling for like the docs, finding the dots and using it and using 721 00:58:40.920 --> 00:58:45.480 it in your task. That Like, I feel like there are a lot 722 00:58:45.519 --> 00:58:52.760 of like very big, scary, like technical terms that you know, when 723 00:58:52.760 --> 00:58:57.960 you're getting ready to do a side project, it's like I'm gonna have to 724 00:58:57.960 --> 00:59:00.400 like read all these dogs to figure out what these things mean, when like 725 00:59:00.639 --> 00:59:05.159 what they really mean is like put an example in the prompt or like in 726 00:59:05.199 --> 00:59:07.239 context learning, it's just like give the they give some examples in the prompt 727 00:59:07.400 --> 00:59:10.679 right and right, and it figures out how to do it like that. 728 00:59:12.239 --> 00:59:17.159 I think like that simplification I really want to try to find or work with 729 00:59:17.199 --> 00:59:22.480 people to help find because I've been talking about this with with like rails, 730 00:59:22.519 --> 00:59:24.239 where there used to be like, oh, you have to know all this 731 00:59:24.360 --> 00:59:29.800 stuff about like database normalization and third normal form and and this and that, 732 00:59:29.960 --> 00:59:32.440 but it's like it really boils down to, oh, you mean like customers 733 00:59:32.480 --> 00:59:39.559 have many purchases, got it right? So I think I think there are 734 00:59:40.039 --> 00:59:44.159 that I think is one of the things that's going to happen need to happen 735 00:59:44.199 --> 00:59:50.599 next. So just like the like easier description like distill this down, there's 736 00:59:50.719 --> 00:59:55.559 you know, ah, there's very deep uh you know, deep explanations to 737 00:59:55.599 --> 00:59:59.719 take it further and make it more and more powerful. But to like get 738 00:59:59.719 --> 01:00:02.960 started, it's just like you mean, you like just copy and paste that 739 01:00:04.000 --> 01:00:07.840 into it, right, Yeah, the dummies guide to prompting, you know. 740 01:00:08.480 --> 01:00:12.800 But to be honest, like I never understood why like they came up 741 01:00:12.800 --> 01:00:15.079 with the phrases they did, like you know, zero shot or a few 742 01:00:15.199 --> 01:00:19.519 shot, Like what does all that mean? Like there was it's almost like 743 01:00:19.599 --> 01:00:23.239 nonsensical to like say that when you just mean examples, right or not examples? 744 01:00:23.360 --> 01:00:30.000 Like, yeah, I think that's what's one of the things that you've 745 01:00:30.159 --> 01:00:32.760 been talking about with you know, Andre and I have been talking Andre from 746 01:00:35.119 --> 01:00:39.400 Lang Chain RB I've been talking about is that, you know a lot of 747 01:00:39.440 --> 01:00:43.960 this stuff is coming out of research and coming out of academia and like those 748 01:00:43.960 --> 01:00:50.320 things, those those terms do matter for like those those domains, but from 749 01:00:50.719 --> 01:00:53.519 you know, an application engineering side of things that are more like you know, 750 01:00:53.599 --> 01:01:00.320 applied we can have our own descriptions of these and why they work, 751 01:01:00.000 --> 01:01:05.679 and that are separate from you know, the science of the theory and more 752 01:01:05.719 --> 01:01:10.400 of the practice. Right was the you in In theory, there's no difference 753 01:01:10.440 --> 01:01:14.440 between theory and practice, but in practice there is. Well, is there 754 01:01:14.719 --> 01:01:17.840 anything else you wanted to cover to today before we jump in the picks here? 755 01:01:21.440 --> 01:01:23.760 Yeah? Well, I you know, if if anybody listening is going 756 01:01:23.760 --> 01:01:30.880 to Madison Ruby, I'm going to be giving a talk, uh going more 757 01:01:30.920 --> 01:01:37.880 into using l MS in uh, a little bit of the philosophy on you 758 01:01:37.880 --> 01:01:40.599 know, how to how to deal with l and generated code. The talks 759 01:01:40.639 --> 01:01:46.960 called going post l you know, won't give any spoilers away, but you 760 01:01:47.000 --> 01:01:51.920 know, leaning very heavily, which Obi talks about Postel's law in his book, 761 01:01:52.519 --> 01:01:58.760 but you know, being building your systems so that they can be liberal 762 01:01:58.760 --> 01:02:02.360 and what they expect liberal and what they accept and conservative and what they send, 763 01:02:04.000 --> 01:02:07.440 uh, just a little bit more liberal and what you what you accept 764 01:02:07.519 --> 01:02:13.039 than than what you're used to. And then you know, there's a lot 765 01:02:13.079 --> 01:02:19.440 of things that Ruby can do easily any language to do anything right, but 766 01:02:19.880 --> 01:02:24.119 like the meta programming things that you can do in Ruby to change your application 767 01:02:24.199 --> 01:02:32.400 at run time. Ah, I think are under explored, so be going 768 01:02:32.440 --> 01:02:37.440 into going into that a lot at the talk. That's awesome. Yeah, 769 01:02:37.559 --> 01:02:43.840 I share your your sentiment there. I I've definitely tried to have the lll 770 01:02:43.960 --> 01:02:47.920 M generate Ruby methods and run them and then reuse them. It's it's a 771 01:02:47.960 --> 01:02:54.440 lot of fun, a lot very dangerous. But yeah, definitely under explored 772 01:02:54.920 --> 01:03:00.280 for sure. So that's cool. Yeah, people go check out Madison Ruby 773 01:03:00.320 --> 01:03:04.840 if you're not going. Uh, I'm excited to watch that talk remotely at 774 01:03:04.920 --> 01:03:12.239 least. Yeah, hear what you got. So if people want to reach 775 01:03:12.280 --> 01:03:15.679 out to you or find you on the web, you know where can they 776 01:03:15.679 --> 01:03:22.559 do that? Sub layer dot com. Uh primary Scott at sublayer dot com 777 01:03:22.960 --> 01:03:30.480 if you'd like to email me uh at Scott wernerd dot or at Twitter Scott 778 01:03:30.519 --> 01:03:37.880 Wernerd at Twitter, and let's see we've all. You know. I'm also 779 01:03:37.360 --> 01:03:42.559 like we talked about in the Ruby A I Builders discord, uh, you 780 01:03:42.599 --> 01:03:47.840 know, sharing you know, we uh discussion the big the big discussions right 781 01:03:47.840 --> 01:03:51.880 now are on that that arc A g I challenge which we've all been kind 782 01:03:51.880 --> 01:03:58.280 of racking our brains about. And then you know the link here on the 783 01:03:58.320 --> 01:04:00.960 sub layer site. We have a we have an access to our discord, 784 01:04:00.960 --> 01:04:05.519 which is a little bit more you know, about our releases and maybe some 785 01:04:08.119 --> 01:04:14.159 less Ruby specific stuff and more kind of the thinking on what is how do 786 01:04:14.199 --> 01:04:19.440 we how do we think about these these AI tools and use them for building 787 01:04:19.440 --> 01:04:24.880 applications just in general. Yeah, I've been enjoying that channel as well. 788 01:04:26.199 --> 01:04:30.960 It's super interesting the stuff gets that gets posted, and you would think it 789 01:04:30.960 --> 01:04:33.239 would get shared in more places, to be honest, A lot of the 790 01:04:33.320 --> 01:04:36.519 yeah, a lot of the papers I get shared. Maybe people just aren't 791 01:04:36.559 --> 01:04:41.400 reading as much as I am. I don't know. I don't think so, 792 01:04:41.440 --> 01:04:45.000 but it feels like an unlimited amount of stuff to read. There's so 793 01:04:45.199 --> 01:04:51.559 much to read. Wake up every morning, where did I go? Yeah? 794 01:04:51.599 --> 01:04:55.119 I feel like the paper I read last week is just like already like 795 01:04:55.360 --> 01:05:02.480 you know, dated, Like like how is that possible? All right, 796 01:05:02.719 --> 01:05:06.239 Well, let's jump in the picks. We've been talking about so much great 797 01:05:06.239 --> 01:05:11.800 stuff. Picks our segment where we just pick literally anything. It could be 798 01:05:12.079 --> 01:05:15.559 code if you want to, It doesn't have to be, but it's typically 799 01:05:15.559 --> 01:05:19.440 what I pick, and if you need a minute, I can go first. 800 01:05:21.039 --> 01:05:29.920 So I've decided to to fine tune a large language model for Ruby, 801 01:05:31.400 --> 01:05:39.880 mostly just for the experiment than anything, but I'm calling it Ruby lang dot 802 01:05:39.920 --> 01:05:46.079 AI. I'm gonna start building it and open and just like learning how all 803 01:05:46.119 --> 01:05:51.760 this stuff works and how it might be used to source all of the awesome 804 01:05:51.760 --> 01:05:58.039 open source Ruby that's out there that is designed incredibly and make use of that 805 01:05:58.079 --> 01:06:03.000 and take advantage of it and make a a new language model that is very 806 01:06:03.119 --> 01:06:06.679 Ruby centric. Uh. So I'm gonna see how it goes. I'm hoping 807 01:06:06.719 --> 01:06:10.400 that I'm successful in it, but I may not be. There may just 808 01:06:10.400 --> 01:06:15.119 be a bunch of lessons learned, but we'll see. You can follow my 809 01:06:15.199 --> 01:06:20.360 progress Ruby lane that ai. Yeah, I mean I saw that you posted 810 01:06:20.360 --> 01:06:24.599 about it either yesterday or the day before, and yeah, excited to see 811 01:06:24.639 --> 01:06:29.840 where that goes. Yeah. You know, I got this stupid massive GPU 812 01:06:29.920 --> 01:06:33.079 server at home and I was just using it for imprints and I'm like, 813 01:06:33.320 --> 01:06:36.800 uh, you know, I was just wasting away over here like running imprints 814 01:06:36.800 --> 01:06:41.960 on it, and so I thought like I should fine tune something, and 815 01:06:42.800 --> 01:06:48.239 this seems like a perfect use case. So here we go. That's great. 816 01:06:51.039 --> 01:06:56.320 Yeah, so I guess I've got I've got two things. One you 817 01:06:56.360 --> 01:07:00.760 know, the I talked a little bit around the like the formal informal kind 818 01:07:00.760 --> 01:07:04.039 of pendulum swing. You know. One of the places where I got that 819 01:07:04.119 --> 01:07:11.679 from was an AVD Grim talk the Soul of Software from Ah it was a 820 01:07:11.679 --> 01:07:15.920 long it was a long time ago at this point, but it's there are 821 01:07:15.920 --> 01:07:21.320 a couple versions of it up on YouTube. He goes into more of the 822 01:07:21.480 --> 01:07:28.039 kind of the informal mindset and how there are those two splits of software and 823 01:07:28.880 --> 01:07:39.119 how neither one is particularly absolutely right, but more it's each each one gives 824 01:07:39.119 --> 01:07:42.559 you different different things depending on what you need. So like that was that 825 01:07:42.639 --> 01:07:46.559 was very impactful for me, especially now given the you know, the approach 826 01:07:46.599 --> 01:07:53.199 we're taking and the other thing read a lot of substacks and there's one called 827 01:07:53.280 --> 01:07:59.280 strange loop Cannon which he had to post a little while back about what lllms 828 01:07:59.360 --> 01:08:03.239 can't do, which you know is also a trap a lot of times, 829 01:08:03.280 --> 01:08:06.320 because as soon as you say it's not possible to do this, everybody on 830 01:08:06.360 --> 01:08:11.719 Twitter tries to prove you wrong. And in his post he had one about 831 01:08:12.599 --> 01:08:16.279 it can't do Conway's Game of Life, and I think like within a week 832 01:08:16.399 --> 01:08:20.079 somebody had proven him wrong. He had tried fine tuning all these different things, 833 01:08:20.279 --> 01:08:26.800 couldn't figure it out, and the collective wisdom of Twitter came up with 834 01:08:26.840 --> 01:08:30.560 a solution, I think. But then he has he has one recently. 835 01:08:31.920 --> 01:08:34.960 It's very good. I highly recommend just the sub stack in general. That 836 01:08:35.000 --> 01:08:39.039 one is very good. And then the latest one, seeing like a network 837 01:08:39.119 --> 01:08:48.319 is very was very very good as well. That's awesome. Well, I 838 01:08:48.560 --> 01:08:53.920 appreciate you coming on Scott and talking about all this code generation stuff and the 839 01:08:54.439 --> 01:09:00.319 love of Rubai and you know, it definitely is a sleeping giant. And 840 01:09:00.399 --> 01:09:01.520 I think we're just going to see more and more of why that is. 841 01:09:03.840 --> 01:09:08.319 And you know, we'll have to have you on again after you're you know, 842 01:09:09.000 --> 01:09:15.720 you start creating many companies menalize all of this movie code generation. Uh, 843 01:09:15.960 --> 01:09:18.920 you know, I could definitely foresee a future where you're just like, 844 01:09:19.520 --> 01:09:21.800 oh, you know, create a Reil sap that does this, and it 845 01:09:21.920 --> 01:09:29.279 just does it. It seems seems easy enough, simple. Yeah. No, 846 01:09:29.439 --> 01:09:30.199 thank you for having me. It's been a lot of fun. It's 847 01:09:30.239 --> 01:09:33.520 been awesome, uh, you know, chatting and catching up and excited to 848 01:09:33.520 --> 01:09:39.800 see you in person a month. Yeah, totally all right, Well, 849 01:09:40.199 --> 01:09:44.960 until next time, folks, I'm out here, and come come visit us 850 01:09:44.960 --> 01:09:45.359 next time.