WEBVTT 1 00:00:05.320 --> 00:00:08.039 Cla command kind of became the test with a front 2 00:00:08.119 --> 00:00:09.640 end and the back end. You need to actually write 3 00:00:09.679 --> 00:00:13.519 some tests lean towards end and integration type tests that 4 00:00:13.560 --> 00:00:16.559 test the whole flow instead of testing small parts. 5 00:00:16.280 --> 00:00:16.800 Of your coach. 6 00:00:16.839 --> 00:00:19.079 Your codal change very quickly, so you have to rewrite 7 00:00:19.079 --> 00:00:19.879 those tests a lot. 8 00:00:20.039 --> 00:00:24.640 But does that mean that? But that means, if understand 9 00:00:24.679 --> 00:00:28.239 your description correctly, that you're effectively manually testing it. I 10 00:00:28.320 --> 00:00:31.960 mean you're you're running a certain script, but then you're look, 11 00:00:32.119 --> 00:00:35.679 you're you're you are the one who's actually checking to 12 00:00:35.719 --> 00:00:37.759 see that you get the expected results. 13 00:00:37.840 --> 00:00:40.240 Well, it's still a I mean, it's automated because you 14 00:00:40.280 --> 00:00:43.240 wrote a test. The test itself is going through steps 15 00:00:43.280 --> 00:00:45.200 that you would manually do yourself. 16 00:00:45.719 --> 00:00:47.799 But like I said, it's like it's like an end 17 00:00:47.799 --> 00:00:48.399 to end test. 18 00:00:49.439 --> 00:00:52.079 Yeah, it's not an end to end test. Yeah, that's 19 00:00:52.119 --> 00:00:57.320 what that was aiming for. So so yeah, so so 20 00:00:57.560 --> 00:01:00.840 vibe coding. I guess is because the system is so 21 00:01:00.920 --> 00:01:06.319 dynamic and everything is changing so rapidly, you're effectively also 22 00:01:06.519 --> 00:01:10.000 kind of acting, as to an extent, the manual QA 23 00:01:10.239 --> 00:01:11.480 for what you're developing. 24 00:01:11.680 --> 00:01:13.840 Yes, yeah, I think that's that's a good way to 25 00:01:13.959 --> 00:01:14.519 look at it. 26 00:01:14.560 --> 00:01:17.040 And as much as you can build in ways what 27 00:01:17.079 --> 00:01:21.159 a system can qa itself, you should there's always going 28 00:01:21.200 --> 00:01:22.799 to be at least some sort of points in time 29 00:01:22.840 --> 00:01:25.079 where you have to step outside and actually just like 30 00:01:25.439 --> 00:01:27.719 try it and see if it works or not. So 31 00:01:27.719 --> 00:01:28.799 I think there's a lot of the same kind of 32 00:01:28.840 --> 00:01:31.040 issues that we get just with testing in general, like 33 00:01:31.040 --> 00:01:32.719 what's it efficient way to test it work not? 34 00:01:33.000 --> 00:01:34.879 You know, in a lot of cases when I've done it, 35 00:01:34.920 --> 00:01:37.200 I just loaded in the browser and just click around 36 00:01:38.159 --> 00:01:40.920 and then I'll have it, I'll vibe code it and 37 00:01:41.239 --> 00:01:45.079 prompt it for unit tests. Did you but that it 38 00:01:45.159 --> 00:01:47.599 really does depend on what I'm building, because sometimes that's 39 00:01:47.719 --> 00:01:49.519 not the right way to do it if it's you know, 40 00:01:49.959 --> 00:01:51.480 completely command line based or something. 41 00:01:51.480 --> 00:01:54.920 By the way, coding, what do you usually generate, Anthony? 42 00:01:55.000 --> 00:01:56.840 What do I in terms of like what LM am 43 00:01:56.879 --> 00:01:57.239 I using? 44 00:01:57.319 --> 00:02:00.920 Or no, what you're using the l ELM generate code? 45 00:02:01.079 --> 00:02:05.280 Is it React code, straight on, dumb code, jQuery code, 46 00:02:05.519 --> 00:02:09.759 HTMX code. Yeah, I know you're generating a. 47 00:02:09.719 --> 00:02:10.400 Lot of stuff. 48 00:02:10.719 --> 00:02:12.879 Like I said, I first started by building a CLI 49 00:02:13.719 --> 00:02:16.439 and then I extended that functionality to a back end 50 00:02:16.439 --> 00:02:20.039 and front end which have gone through many different tech stacks. 51 00:02:20.199 --> 00:02:22.000 You know, I was a big framework. 52 00:02:21.599 --> 00:02:24.159 Guy, so I'm always kind of trying out different tools. 53 00:02:24.520 --> 00:02:27.479 My current stack in terms of what I'm deploying the 54 00:02:27.520 --> 00:02:30.159 auto show app with, which is the app I vibe 55 00:02:30.199 --> 00:02:35.680 coded is Astro with solid as the templating language, and 56 00:02:35.919 --> 00:02:39.400 Tailwind for the styling. And yeah, so that's the that's 57 00:02:39.439 --> 00:02:41.479 basically the back end and the front end. I've done 58 00:02:41.479 --> 00:02:45.240 stuff with like a noe JS Fastify server, I've done 59 00:02:45.240 --> 00:02:50.319 stuff with React. Really, any kind of popular technology that 60 00:02:50.360 --> 00:02:52.240 has a lot of usage and a lot of docs 61 00:02:52.400 --> 00:02:56.240 will probably do pretty well. Some people say you should 62 00:02:56.280 --> 00:02:59.800 always use next JS and super base, but like I think, 63 00:02:59.840 --> 00:03:02.240 as as all as you're not trying to use something 64 00:03:02.319 --> 00:03:05.800 really obscure and really new, it tends to work pretty well. 65 00:03:05.840 --> 00:03:08.479 If it is very obscure, very new, you should create 66 00:03:08.919 --> 00:03:11.599 like a just a single file that kind of has 67 00:03:11.639 --> 00:03:13.520 all the docs in it and drop that into the 68 00:03:13.560 --> 00:03:14.919 context of the LM. 69 00:03:15.000 --> 00:03:17.039 That will kind of help short change that issue. 70 00:03:17.280 --> 00:03:20.560 I think these days you also have various mcps that 71 00:03:20.680 --> 00:03:24.039 make it very easy to get content off of GitHub 72 00:03:24.120 --> 00:03:24.759 for examples. 73 00:03:25.800 --> 00:03:28.840 Yes, yeah, yeah, mcps at a whole other level of 74 00:03:28.879 --> 00:03:29.599 things for sure. 75 00:03:29.719 --> 00:03:33.719 Yeah, but when you're using the LL no, I just 76 00:03:34.039 --> 00:03:38.080 curious because you're asking about the tools are you doing 77 00:03:38.120 --> 00:03:43.360 this with like copilot connected to Gemini or GPT or 78 00:03:43.479 --> 00:03:46.800 Claude or using something like cloud code, or how are 79 00:03:46.840 --> 00:03:49.639 you doing that night. I'll also point out the copilot 80 00:03:49.680 --> 00:03:52.199 if you put it into agent mode instead of I 81 00:03:52.199 --> 00:03:54.960 can't remember the other mode, it'll actually put the code 82 00:03:54.960 --> 00:03:57.319 into your ID for you. 83 00:03:57.800 --> 00:04:00.479 And so I'm a little curious, like what level are 84 00:04:00.560 --> 00:04:01.280 you working at there? 85 00:04:01.520 --> 00:04:05.159 Yeah, so I've tried co pilot and cursor, I've tried 86 00:04:05.199 --> 00:04:08.919 clock code, I've tried you know, things like bolt and 87 00:04:09.159 --> 00:04:12.080 v zero. I think that is what Versailles thing is called. 88 00:04:12.240 --> 00:04:14.080 And then like when you have the agent loop, like 89 00:04:14.120 --> 00:04:16.160 you said, was actually changing your code for you. That 90 00:04:16.319 --> 00:04:18.279 is the full Vibe code experience. Like that is Steve 91 00:04:18.279 --> 00:04:24.800 Sworzneimer for sure. So I tend to it's kind of nice, It's. 92 00:04:24.199 --> 00:04:26.079 It can be very very nice. Yeah, totally. 93 00:04:26.120 --> 00:04:28.279 I think for like personal projects, that's definitely what I 94 00:04:28.279 --> 00:04:31.240 would do. I have an I have a flow that 95 00:04:31.240 --> 00:04:33.720 some people will consider it kind of strange and inefficient. 96 00:04:33.839 --> 00:04:37.040 I had used this tool called rebomix, which is a 97 00:04:37.040 --> 00:04:40.439 way to kind of intelligently take your code base and 98 00:04:40.519 --> 00:04:42.959 smush it into something that is small enough to fit 99 00:04:43.040 --> 00:04:46.439 in the context of these lllms. And then it also 100 00:04:46.480 --> 00:04:48.639 includes instructions of how I want my code to be 101 00:04:48.720 --> 00:04:49.680 styled and written. 102 00:04:50.160 --> 00:04:52.160 And then I just write what. 103 00:04:52.040 --> 00:04:54.079 I want to do as a line at the bottom 104 00:04:54.120 --> 00:04:56.519 of that whole thing, and then I PLoP that into 105 00:04:56.800 --> 00:05:00.639 either shatshept, Claude, or Gemini. I use all three kind 106 00:05:00.639 --> 00:05:02.920 of depending on which was going to work best, and 107 00:05:02.959 --> 00:05:05.000 then it gives it gives you the output, and then 108 00:05:05.040 --> 00:05:07.199 I put it into my file. And as I'm doing that, 109 00:05:07.240 --> 00:05:08.959 I'll quickly kind of look over the code and I'll 110 00:05:08.959 --> 00:05:10.839 look at the diffs. So I'm not doing a pure 111 00:05:10.920 --> 00:05:13.560 vibe coding experience that I am paying attention to my cokes. 112 00:05:13.639 --> 00:05:16.560 I'm doing a production app that has you know, payments 113 00:05:16.600 --> 00:05:19.439 and log in and stuff. So that gives me a 114 00:05:19.480 --> 00:05:22.040 step to kind of slow down, look at the code 115 00:05:22.040 --> 00:05:24.800 that's being written, look at what is changing in my project, 116 00:05:25.120 --> 00:05:27.560 get a sense of what's happening, and then I can 117 00:05:27.560 --> 00:05:28.959 then test it and see if it works. 118 00:05:29.600 --> 00:05:33.879 So do you generally, when when the NLM is suggesting stuff, 119 00:05:33.959 --> 00:05:37.360 do you usually approve it one by one, or do 120 00:05:37.399 --> 00:05:41.079 you just approve it all and then look at I 121 00:05:41.079 --> 00:05:43.600 don't know the get diffs or whatever. 122 00:05:43.959 --> 00:05:46.279 So I PLoP in all the changes that it gives me, 123 00:05:46.319 --> 00:05:49.560 and I tell to write codes files in full, so 124 00:05:49.600 --> 00:05:52.360 I just click copy paste for each of them, and 125 00:05:52.480 --> 00:05:54.680 the way my projits are structured usually only has to 126 00:05:54.759 --> 00:05:57.600 change a handful of files to make the change if 127 00:05:57.600 --> 00:06:00.480 it's atomic enough. And then after I've put all the 128 00:06:00.519 --> 00:06:02.519 files and all that, look at the diffs on each 129 00:06:02.560 --> 00:06:04.279 of those files. So if there's three files, I also 130 00:06:04.319 --> 00:06:06.680 go click over to the deff you're on vs code 131 00:06:06.920 --> 00:06:09.360 kind of look at all those and then I'll test 132 00:06:09.399 --> 00:06:12.759 the application with either a test scripture or just clicking through. 133 00:06:13.439 --> 00:06:17.279 Now, you mentioned that you use the project itself as 134 00:06:17.439 --> 00:06:20.480 the context or put it in the context in order 135 00:06:20.560 --> 00:06:25.480 to obviously create consistency within the code base. Yep. For 136 00:06:25.839 --> 00:06:29.759 consistency is only worthwhile once you have some code, and 137 00:06:29.920 --> 00:06:34.199 also assuming the code is structured the way that you 138 00:06:34.439 --> 00:06:38.720 like it. So do you also manually create or have 139 00:06:38.879 --> 00:06:41.800 created some sort of rule files or something along these 140 00:06:41.839 --> 00:06:45.160 lines as kind of a starting point or a way 141 00:06:45.240 --> 00:06:49.040 to keep aligning your code with a certain desired outcome. 142 00:06:49.240 --> 00:06:49.480 Yeah. 143 00:06:49.519 --> 00:06:54.120 So with repomix, it includes an instructions file that gets 144 00:06:54.120 --> 00:06:56.839 appended to each one, and for in there, I have 145 00:06:57.399 --> 00:06:59.759 things like how I want the logging to be done 146 00:07:00.279 --> 00:07:04.040 JavaScript kind of styling, so always using imports, not you know. 147 00:07:04.040 --> 00:07:07.399 Require kind of like cursor rules. 148 00:07:07.800 --> 00:07:09.800 Yeah, they're exactly that curse rules, Yeah, totally. 149 00:07:09.879 --> 00:07:11.720 So yeah, I think everyone should have those, and they 150 00:07:11.720 --> 00:07:14.639 should make sure like get down in writing how they 151 00:07:14.639 --> 00:07:17.120 actually want their code to look and behave and the 152 00:07:17.439 --> 00:07:18.800 coding styles that they want. 153 00:07:19.560 --> 00:07:23.160 So for me, it's interesting. So I've recently I'm working 154 00:07:23.199 --> 00:07:28.079 on several legacy projects at work that are there are 155 00:07:28.160 --> 00:07:31.720 definitely legacy, but there's still very much undergoing development by 156 00:07:31.720 --> 00:07:36.839 several developers, and I introduced rule files into all of them. 157 00:07:36.879 --> 00:07:40.439 And it was a very interesting experience for me in 158 00:07:40.480 --> 00:07:43.519 a lot of ways. It was my first real encounter 159 00:07:43.639 --> 00:07:46.639 with prompt engineering, you might say. So, first of all, 160 00:07:46.680 --> 00:07:49.439 it was really odd for me, being an old timer 161 00:07:49.480 --> 00:07:54.240 in tech, to effectively write configuration as English in a 162 00:07:54.319 --> 00:07:58.560 markdown file rather than in Jason or Yama or anything 163 00:07:58.639 --> 00:08:01.959 that you normally would expe configuration to be in. It's 164 00:08:02.000 --> 00:08:05.399 a very like it's a very strange experience the first 165 00:08:05.439 --> 00:08:10.399 time you do it, Like instead of saying, you know, 166 00:08:10.959 --> 00:08:16.920 yeslint colon, on, you say I want you to use yeslint. 167 00:08:17.000 --> 00:08:20.560 It's it's it's it's a very odd experience. But the 168 00:08:20.680 --> 00:08:24.839 other thing that obviously was a very significant thing for 169 00:08:24.959 --> 00:08:31.120 me was saying how seemingly very small changes that I 170 00:08:31.240 --> 00:08:36.120 would make in the rule files could have very significant 171 00:08:36.200 --> 00:08:41.159 impact on the generated code. And it wasn't always very 172 00:08:41.320 --> 00:08:44.960 trivial to totally understand why M. 173 00:08:46.879 --> 00:08:50.440 Is that The experience as one, Yeah, for me, this 174 00:08:50.519 --> 00:08:54.000 whole the rules came around very iteratively where I was 175 00:08:54.039 --> 00:08:56.559 working a lot with the lms, so it was generating 176 00:08:56.600 --> 00:08:58.360 a lot of code, and there would be a lot 177 00:08:58.360 --> 00:09:01.200 of time doing something consistently that I didn't want, and 178 00:09:01.200 --> 00:09:03.600 then every time that would happen, I would write a rule, 179 00:09:04.000 --> 00:09:06.480 and then I would make sure that rule actually change 180 00:09:06.480 --> 00:09:08.639 the thing I wanted to change, didn't change anything else. 181 00:09:09.120 --> 00:09:11.120 So you kind of need to do this. It takes 182 00:09:11.120 --> 00:09:11.480 some time. 183 00:09:11.519 --> 00:09:13.720 There's really no shortcut to doing it, and you have 184 00:09:13.799 --> 00:09:15.879 to just kind of get familiar with the llms and 185 00:09:15.919 --> 00:09:18.960 how they generate code, and when you're changing your rules 186 00:09:19.279 --> 00:09:22.480 trying it's another kind of like iterative process, where you 187 00:09:22.480 --> 00:09:24.320 get the outputs you see, are they what you want? 188 00:09:24.600 --> 00:09:27.240 Tweak the prompt if it's not what you want, and 189 00:09:27.440 --> 00:09:29.120 once you get it where you want, try not to 190 00:09:29.159 --> 00:09:30.600 mess too much with your rules file. 191 00:09:31.399 --> 00:09:35.759 Yeah. So, so just to finish that point check for me, 192 00:09:35.879 --> 00:09:39.720 what I literally did was I started without any rule files. 193 00:09:39.759 --> 00:09:42.879 I would pick a file, let's say, that was in 194 00:09:42.919 --> 00:09:46.559 a style that I didn't like, and literally gave just 195 00:09:46.639 --> 00:09:51.480 the one command refactor, saw what got generated, then put 196 00:09:51.519 --> 00:09:55.440 in some rules, did it again, saw the difference, tweak them, 197 00:09:55.519 --> 00:09:57.759 did it again, And like you said, it was an 198 00:09:57.799 --> 00:10:02.279 iterative process. What I I also discovered was sometimes even 199 00:10:02.320 --> 00:10:06.840 though I gave certain instructions, and even though the atherlam 200 00:10:07.039 --> 00:10:12.639 very explicitly stated that it was following those instructions, it 201 00:10:12.720 --> 00:10:15.639 still didn't or didn't fully right. 202 00:10:15.879 --> 00:10:18.519 And this is where having examples, like if the rules 203 00:10:18.639 --> 00:10:21.919 match what your code already does, then it will be 204 00:10:22.039 --> 00:10:23.960 it'll be better at doing that because they'll both have 205 00:10:24.080 --> 00:10:27.120 the English language to explain it an example of what 206 00:10:27.159 --> 00:10:29.759 it actually looks like. So you're right, sometimes when you're 207 00:10:29.759 --> 00:10:32.039 just writing in English, it won't always get it. You 208 00:10:32.080 --> 00:10:34.879 actually need a file that's written with that style, you 209 00:10:34.919 --> 00:10:36.879 can show it to be like, no, do it like this. 210 00:10:37.559 --> 00:10:40.000 So for example, for me, it was I wanted it 211 00:10:40.120 --> 00:10:43.519 to not introduce in any slint errors or. 212 00:10:43.480 --> 00:10:47.039 Warnings your s link configuration is. 213 00:10:47.159 --> 00:10:49.759 Yeah, and I obviously gave it the PAS to the 214 00:10:49.960 --> 00:10:54.120 slint configuration, and it would literally say I found some 215 00:10:54.279 --> 00:10:57.000 e s slint issues and fixing them, and then I 216 00:10:57.080 --> 00:11:00.279 found some more and fixing them, and even then when 217 00:11:00.279 --> 00:11:03.480 it was all done, it still had some issues. 218 00:11:03.799 --> 00:11:06.200 So yeah, and that's because you're you're putting a tool 219 00:11:06.279 --> 00:11:09.080 in the middle that it has to understand well enough 220 00:11:09.080 --> 00:11:12.960 to be able to do that, instead of just knowing 221 00:11:13.080 --> 00:11:15.320 enough about code in general to know like what import 222 00:11:15.399 --> 00:11:20.080 versus require is. Anytime you're eye in these extra tools 223 00:11:20.080 --> 00:11:24.360 and actual layers of indirection, that it's more chances for 224 00:11:24.360 --> 00:11:24.960 the LM to. 225 00:11:24.960 --> 00:11:26.559 Get confused and do stuff wrong. 226 00:11:27.360 --> 00:11:29.759 Yeah, because it has to know about es lint in 227 00:11:29.799 --> 00:11:32.360 its latent space because you're not teaching it. 228 00:11:33.440 --> 00:11:36.440 So if you had a doctor or an es lint 229 00:11:36.639 --> 00:11:39.639 you know file that actually has es lint contacts and 230 00:11:39.679 --> 00:11:42.200 explains what the s lint stuff means and your configuration, 231 00:11:42.320 --> 00:11:44.480 how that maps to eslint, how it works, that would 232 00:11:44.480 --> 00:11:46.360 be an additional piece of context you could give it 233 00:11:46.399 --> 00:11:47.919 that would help it figure that out. 234 00:11:48.159 --> 00:11:50.200 Right, then you are teaching it how to do that. 235 00:11:52.519 --> 00:11:54.879 So I have a question because you guys are talking 236 00:11:54.879 --> 00:11:57.879 about rule files and this is something that I haven't 237 00:11:57.919 --> 00:12:01.519 really used typically. I'm getting in and I'm saying, look, 238 00:12:01.799 --> 00:12:03.440 I want you to do this, I want you to 239 00:12:03.519 --> 00:12:05.960 use these tools. I want you to and yeah, I 240 00:12:05.960 --> 00:12:08.159 have to keep reminding it when I'm when I'm doing 241 00:12:08.200 --> 00:12:08.720 the code. 242 00:12:08.799 --> 00:12:12.240 So where do you put those into your tools? 243 00:12:12.360 --> 00:12:15.720 For me, everything goes through repomix, So this is a 244 00:12:15.799 --> 00:12:18.240 huge part of my toolbase. And if you're using cursor, 245 00:12:18.360 --> 00:12:21.039 you won't do this, you won't have this workflow at all. 246 00:12:21.120 --> 00:12:24.240 But for me, I just have literally, uh, the way 247 00:12:24.279 --> 00:12:26.039 the way reap wakes works, you just have a separate 248 00:12:26.039 --> 00:12:27.919 markdown file with your rules written in it. 249 00:12:28.120 --> 00:12:29.840 It just grabs it and depends it. I have a 250 00:12:30.120 --> 00:12:30.919 customs script that. 251 00:12:30.919 --> 00:12:32.679 Does a whole bunch of crap, but it's really it's 252 00:12:32.720 --> 00:12:34.720 just a It's just a hug of markdown is all 253 00:12:34.720 --> 00:12:38.080 it is. And that gets added to the project's contexts 254 00:12:38.080 --> 00:12:39.639 that repomix creates for me. 255 00:12:40.639 --> 00:12:42.879 And that's essentially the same way that it works. And 256 00:12:42.919 --> 00:12:46.320 cursor just takes it from a different location. And yeah, 257 00:12:46.480 --> 00:12:49.279 the whole point and the whole point is exactly what 258 00:12:49.320 --> 00:12:53.879 you said, Chuck, to avoid needing to manually put those 259 00:12:54.039 --> 00:12:56.679 in the stuff that the same stuff in the prompt 260 00:12:56.720 --> 00:12:58.879 again and again and again each and every time. 261 00:12:58.799 --> 00:13:02.639 Right, Yeah, and it's usually the ongoing stuff. Right, So 262 00:13:02.759 --> 00:13:05.440 it's oh, just a remind No, I'm using tailwind four 263 00:13:05.519 --> 00:13:07.519 not tailwind three, right, or. 264 00:13:07.440 --> 00:13:11.440 First of all it's I'm using Yeah, first of all, 265 00:13:11.440 --> 00:13:14.639 it's I'm using tailwind rather than let's say, see a sense. 266 00:13:14.679 --> 00:13:17.679 It might figure it out from your existing content, or 267 00:13:17.720 --> 00:13:20.279 it might not. Right if you're starting a new project, 268 00:13:20.279 --> 00:13:22.600 then how would it know? So you would need to 269 00:13:22.679 --> 00:13:25.960 remember to tell it use tailwind, But if you put 270 00:13:25.960 --> 00:13:28.000 it in your in your root five, you don't. 271 00:13:29.559 --> 00:13:29.799 Yeah. 272 00:13:29.840 --> 00:13:32.840 This also gets to know something about conversation length. The 273 00:13:32.919 --> 00:13:35.919 longer you talk to an LM and a single conversation, 274 00:13:36.039 --> 00:13:38.519 the dumber it will get. This is something that's highly 275 00:13:38.600 --> 00:13:41.720 unintuitive that a lot of people do not understand about lms. 276 00:13:41.759 --> 00:13:44.480 That's why I like repo mix because every time I'm 277 00:13:44.519 --> 00:13:47.720 making a change, I'm kind of one shotting it getting 278 00:13:47.720 --> 00:13:49.720 code back if it doesn't work. I'll have a very 279 00:13:49.759 --> 00:13:51.799 quick back and forth to fix the bug, and then 280 00:13:51.799 --> 00:13:54.000 I'm immediately going to go to a new context, a 281 00:13:54.039 --> 00:13:55.559 new conversation every time. 282 00:13:55.759 --> 00:13:57.840 So that was the thing that I wanted to bring 283 00:13:57.919 --> 00:14:00.559 up with repo mix, and I was going to ask earlier, 284 00:14:00.559 --> 00:14:02.639 but you kind of already answered it by saying repo 285 00:14:02.759 --> 00:14:05.600 mix and that is is you know, like when I'm 286 00:14:05.639 --> 00:14:08.399 starting out, it can kind of keep most of the 287 00:14:08.440 --> 00:14:11.960 app and in the context, but as I build more 288 00:14:12.000 --> 00:14:15.240 things in, it obviously gets to the point where it can't. 289 00:14:15.879 --> 00:14:19.399 Yeah, so it makes more like test files and things 290 00:14:19.440 --> 00:14:21.320 like that. You can configure it to say I just 291 00:14:21.360 --> 00:14:24.799 want my front end files to be in this configuration 292 00:14:25.000 --> 00:14:26.799 and my back end files to be in this. So 293 00:14:27.120 --> 00:14:29.360 if you have an appic it is very very large, 294 00:14:29.639 --> 00:14:32.120 you just start to scope it two different sections that 295 00:14:32.159 --> 00:14:34.200 will have the context it needs to make the change 296 00:14:34.200 --> 00:14:35.159 you need you want to make. 297 00:14:35.759 --> 00:14:38.320 So do you have to tell it I'm working in 298 00:14:38.360 --> 00:14:41.240 this section of code, and so only care about because 299 00:14:41.279 --> 00:14:44.200 because That's what I'm wonderings on it. 300 00:14:44.960 --> 00:14:46.879 I have a script where I create a bunch of 301 00:14:46.919 --> 00:14:51.000 different configurations that allow repomix to generate different types of 302 00:14:51.000 --> 00:14:54.440 context and I'll just run a command like repo back 303 00:14:54.559 --> 00:14:58.159 for my back end or something. It's there's a lot 304 00:14:58.200 --> 00:15:00.399 of ways which you can configure it. In terms of 305 00:15:00.399 --> 00:15:02.120 how many different ways you need to configure it will 306 00:15:02.159 --> 00:15:04.240 kind of depend on how larger project is and how 307 00:15:04.240 --> 00:15:06.399 many different sections there are, and how easily it will 308 00:15:06.440 --> 00:15:09.200 get confused if you give it the whole project versus 309 00:15:09.200 --> 00:15:12.559 just the parts of the project it needs. But since 310 00:15:12.600 --> 00:15:15.200 I'm vibe coding it, every time I want repo mix command, 311 00:15:15.240 --> 00:15:16.759 I just give it my code and say, hey, I 312 00:15:16.879 --> 00:15:18.679 have this repo mix command to cover this part of 313 00:15:18.679 --> 00:15:19.279 a project. 314 00:15:19.320 --> 00:15:20.320 It gives me the new command. 315 00:15:21.200 --> 00:15:25.879 Okay, you mentioned before that the NLM gets dumber the 316 00:15:26.000 --> 00:15:30.879 longer the conversation continues. Yeah, why is that so? 317 00:15:30.919 --> 00:15:35.039 It's just because of what context length itself is. Context 318 00:15:35.120 --> 00:15:38.399 length basically means that there's a certain number of tokens 319 00:15:38.720 --> 00:15:41.039 that the LM is able to kind of keep in 320 00:15:41.120 --> 00:15:44.240 its like working memory as a certain point as I 321 00:15:44.240 --> 00:15:47.039 guess longer longer, longer II at full. So we'll have 322 00:15:47.080 --> 00:15:50.559 to start bumping off text from the beginning, and then 323 00:15:50.639 --> 00:15:52.840 that means it will lose important context if it made 324 00:15:52.879 --> 00:15:55.360 some changes at the very beginning and then you get 325 00:15:55.360 --> 00:15:57.759 to a very long conversation, it won't be able to 326 00:15:57.759 --> 00:15:59.679 be aware of those at the very beginning. 327 00:15:59.840 --> 00:16:02.720 Why you have to keep it within the. 328 00:16:03.240 --> 00:16:06.320 L ELM's context window, not just for the first message, 329 00:16:06.320 --> 00:16:08.639 so it doesn't say this is too long, I can't respond, 330 00:16:09.159 --> 00:16:12.440 So it doesn't then as the claude actually does something, 331 00:16:12.519 --> 00:16:14.480 smart cloud will tell you at a certain point in 332 00:16:14.519 --> 00:16:16.480 time when your conversation is done, and they will not 333 00:16:16.600 --> 00:16:18.200 let you keep using They will tell you have to 334 00:16:18.240 --> 00:16:20.360 create a new chat. I'm not sure if any of 335 00:16:20.399 --> 00:16:22.279 the lms do that right now. Most of them just 336 00:16:22.360 --> 00:16:24.639 let let you keep going forever, and then it will 337 00:16:24.720 --> 00:16:26.480 just be that's where you get the hallucinations. 338 00:16:26.480 --> 00:16:27.759 That's where you get a lot of these errors. 339 00:16:27.759 --> 00:16:29.519 People think about when they think of l elms as 340 00:16:29.559 --> 00:16:33.120 once you exceed the context window in your conversation. 341 00:16:33.279 --> 00:16:36.600 So it's basically to a certain point in time, you're 342 00:16:36.679 --> 00:16:39.000 kind of getting lost in the weeds, as it were, 343 00:16:39.159 --> 00:16:43.440 with all the noise that accumulated throughout that entire conversation, 344 00:16:43.559 --> 00:16:47.200 all the dead ends and wrong paths remain in the 345 00:16:47.240 --> 00:16:50.799 context and actually bump out the more important stuff that 346 00:16:50.840 --> 00:16:53.399 you probably gave it the very beginning of the conversation, 347 00:16:53.480 --> 00:16:56.960 as you said, like the goal that you're working towards, 348 00:16:56.960 --> 00:16:59.399 et cetera. By the way, we have a question from 349 00:16:59.440 --> 00:17:04.599 the audience. Does he always try all three models Claude, Gemini, Chajipt, 350 00:17:05.200 --> 00:17:08.119 Which one tends to work best for which types of code? 351 00:17:08.960 --> 00:17:09.240 Sure? 352 00:17:09.359 --> 00:17:11.839 So I won't switch to different models if the first 353 00:17:11.839 --> 00:17:14.079 one I tried isn't working, Like if it gives me 354 00:17:14.119 --> 00:17:16.000 broken code and then I tell it to fix it 355 00:17:16.039 --> 00:17:18.279 and it gives me code that's still broken in the 356 00:17:18.319 --> 00:17:20.960 same way or broken in a different way. I treat 357 00:17:21.000 --> 00:17:24.480 most of these things pretty pretty pragmatically. And the question 358 00:17:24.519 --> 00:17:26.200 more so is like, so what cause if you do 359 00:17:26.319 --> 00:17:28.119 all three you other then check all three and if 360 00:17:28.119 --> 00:17:29.640 they all work, then you just wasted, you know, a 361 00:17:29.680 --> 00:17:30.200 ton of time. 362 00:17:30.359 --> 00:17:32.200 The bigger question is which one do you start with? 363 00:17:32.240 --> 00:17:33.839 Which is one is most likely you give be correct 364 00:17:33.880 --> 00:17:35.680 answer the first time, so you don't have to use 365 00:17:35.720 --> 00:17:36.599 another model. 366 00:17:36.839 --> 00:17:39.319 And that has changed a lot for me over time. 367 00:17:39.359 --> 00:17:41.559 I'm constantly kind of switching back and forth as new 368 00:17:41.599 --> 00:17:44.640 models come out because the space is so competitive right 369 00:17:44.680 --> 00:17:47.559 now that almost no one is able to hold onto 370 00:17:47.599 --> 00:17:49.480 the best model for more than a couple of months. 371 00:17:49.960 --> 00:17:54.480 So My go to right now is Claude for Sonnet, 372 00:17:54.839 --> 00:17:56.519 not Claude for Opus. 373 00:17:56.799 --> 00:18:00.680 This is a rare case where this quote unquote best biggest, 374 00:18:00.759 --> 00:18:03.920 newest model is not really the best son It is 375 00:18:03.920 --> 00:18:06.559 better than Opus because one, it has a longer context window, 376 00:18:06.559 --> 00:18:08.440 WHI I don't know why they made opens a smaller 377 00:18:08.480 --> 00:18:13.880 contact window. It also is slightly faster, and it just 378 00:18:14.559 --> 00:18:16.559 doesn't have as many kind of does it seem times 379 00:18:16.640 --> 00:18:19.039 much downtime. There's one time where Opus was broken and 380 00:18:19.079 --> 00:18:20.640 then I realized I could just switch to sign it 381 00:18:20.680 --> 00:18:23.160 and sign it wasn't so like sometimes if Claude is down, 382 00:18:23.319 --> 00:18:25.799 a Claude model is down, not Claude the whole thing. 383 00:18:26.480 --> 00:18:28.200 So would cloth Horse sign it is my go to 384 00:18:28.400 --> 00:18:28.960 right now. 385 00:18:29.319 --> 00:18:32.759 Then if that doesn't work, I'll try Gemini two point five, 386 00:18:33.000 --> 00:18:35.160 and then if that doesn't work, I'll try Chat. 387 00:18:35.200 --> 00:18:39.839 Should be t relating starting, Yeah, I I just kind 388 00:18:39.839 --> 00:18:43.160 of start with whatever I'm sitting on and then I'll 389 00:18:43.200 --> 00:18:44.319 just change it when I need to. 390 00:18:45.200 --> 00:18:47.200 That's I don't start with any one of them. I 391 00:18:47.319 --> 00:18:47.880 just rotate. 392 00:18:47.960 --> 00:18:50.640 It's like this isn't doing as well, and I also 393 00:18:50.640 --> 00:18:53.839 don't follow along so much with the there's a new 394 00:18:53.880 --> 00:18:56.680 model out. I want to try it, I just I'll 395 00:18:56.680 --> 00:18:59.319 wind up switching to it when it's like okay for 396 00:18:59.400 --> 00:19:01.440 whatever reason, and this isn't doing what I want. 397 00:19:01.599 --> 00:19:03.960 My question though, is how much are you paying for this? 398 00:19:04.799 --> 00:19:07.880 Because as you use the different models, you typically have 399 00:19:07.920 --> 00:19:10.000 to pay some of them. It's per usage, right, So 400 00:19:10.039 --> 00:19:12.079 it's like I put so many tokens. 401 00:19:11.680 --> 00:19:15.559