WEBVTT 1 00:00:05.200 --> 00:00:08.119 Hey, folks, welcome back to another episode of the Ruby 2 00:00:08.199 --> 00:00:12.919 Rogues podcast. This week, on our panel we have Valentino Stole. Hey, now, 3 00:00:13.039 --> 00:00:15.839 I'm Charles Maxwoot from Top End Devs And yeah, it's 4 00:00:15.880 --> 00:00:18.879 been a while Valentino, we've kind of had a lot 5 00:00:18.920 --> 00:00:21.879 going on, just picking things back up. Do you want 6 00:00:21.920 --> 00:00:23.800 to give people a quick update as to where you're 7 00:00:23.839 --> 00:00:25.320 at and then I'll do the same thing and then 8 00:00:25.320 --> 00:00:27.960 we can dive into the rest of this topic for today. 9 00:00:28.640 --> 00:00:28.960 Yeah. 10 00:00:29.039 --> 00:00:31.399 Yeah, you know, earlier in the year I moved gigs, 11 00:00:31.760 --> 00:00:34.119 I was at doc Semity and I switched over to 12 00:00:34.759 --> 00:00:38.159 the Lovely team Agusta working on AI related stuff. 13 00:00:38.240 --> 00:00:40.200 Pretty wild thing's happening. 14 00:00:40.399 --> 00:00:43.600 Yeah, you know, lots of So when you say you're 15 00:00:43.600 --> 00:00:47.520 working on AI stuff for Gusto, are you integrating AI 16 00:00:48.079 --> 00:00:52.079 based features into what Gusto does or you know, well, 17 00:00:52.119 --> 00:00:53.960 what does that look like? I guess it seems like 18 00:00:54.039 --> 00:00:55.840 there's a wide world of things that people are doing 19 00:00:55.840 --> 00:00:56.240 with AI. 20 00:00:57.320 --> 00:00:59.560 There is a wide world of things. So they have 21 00:00:59.679 --> 00:01:04.079 a you know, Chapot Gus who's like the assistant for 22 00:01:04.159 --> 00:01:08.280 your business, your small business, and uh, you know, they're 23 00:01:08.319 --> 00:01:12.239 basically just integrating their entire ecosystem into the chato okay, 24 00:01:13.159 --> 00:01:14.040 amongst other things. 25 00:01:14.359 --> 00:01:16.359 There's a lot of other initiatives. That's that's the one 26 00:01:16.400 --> 00:01:18.319 I've been focused on in my op okay. 27 00:01:18.359 --> 00:01:20.319 So yeah, pretty much, anything that you would want to 28 00:01:20.359 --> 00:01:23.239 do as a small business, think about what an AI 29 00:01:23.319 --> 00:01:26.359 assistant might help out with that, especially when you have 30 00:01:26.400 --> 00:01:28.959 a small business platform you can plug into. 31 00:01:30.239 --> 00:01:34.040 It's it's kind of exciting, very very cool. I'd love 32 00:01:34.079 --> 00:01:36.680 to dig into that. It looks like a lot of fun. 33 00:01:36.799 --> 00:01:39.239 So yeah, So for me, about the same time you 34 00:01:39.400 --> 00:01:43.319 transition over to Gusto, I dropped my contract and went 35 00:01:43.359 --> 00:01:45.159 and worked for price Picks and so I've been at 36 00:01:45.200 --> 00:01:49.280 price Picks since March and I've spent quite a bit 37 00:01:49.319 --> 00:01:51.359 of time working on a lot of the social features. 38 00:01:51.400 --> 00:01:53.159 So if you're if you get into the app and 39 00:01:53.200 --> 00:01:57.040 you're you want to follow another member's profile or you 40 00:01:57.079 --> 00:01:59.439 want to they have the feeds where you can see 41 00:01:59.439 --> 00:02:02.239 what other people or wagering or stuff like that. You know, 42 00:02:02.319 --> 00:02:04.760 it kind of makes it fun and you know you're 43 00:02:04.799 --> 00:02:06.480 able to kind of see more of the things that 44 00:02:06.519 --> 00:02:08.719 other people are doing and you know kind of build 45 00:02:08.759 --> 00:02:11.039 stuff off of that. And anyway, I've I've been spending 46 00:02:11.039 --> 00:02:13.039 a lot of time on that and on the profile stuff. 47 00:02:13.080 --> 00:02:16.159 So that's where a lot of my work has come. 48 00:02:16.199 --> 00:02:18.879 It's actually kind of funny because my wife and I'll 49 00:02:18.879 --> 00:02:21.319 be sitting watching TV and a price picks that will 50 00:02:21.360 --> 00:02:23.280 come up, and it's, you know, it's like, hey, did 51 00:02:23.280 --> 00:02:26.039 you know that you could you know whatever, and I 52 00:02:26.080 --> 00:02:27.560 and I look at her and I go, yeah, I 53 00:02:27.599 --> 00:02:30.080 built that, right, at least I built them back end of. 54 00:02:30.080 --> 00:02:32.719 It, right, are you on there? You know, pick making 55 00:02:32.759 --> 00:02:33.280 your picks? 56 00:02:33.360 --> 00:02:33.560 You know? 57 00:02:35.000 --> 00:02:35.080 No. 58 00:02:35.439 --> 00:02:39.960 In fact, my track record on picking the right stuff 59 00:02:40.039 --> 00:02:44.080 to win is abysmal. I mean, you know, obviously as 60 00:02:44.080 --> 00:02:46.759 an employee, I can't actually play for real money, but 61 00:02:47.039 --> 00:02:49.000 you know, if I want to play, I can play, 62 00:02:49.039 --> 00:02:51.319 you know, and it's kind of fun. I just you know, 63 00:02:51.400 --> 00:02:57.280 I can't. I can't depositor withdraw money and so anyway, 64 00:02:57.319 --> 00:03:00.639 it's just it's just yeah, it's so so it's all 65 00:03:00.639 --> 00:03:03.639 in fun. But yeah, I almost never win. 66 00:03:03.800 --> 00:03:07.120 So it is funny to see, like, you know, ads 67 00:03:07.159 --> 00:03:09.439 pop up and you're like, hey, yeah, I know that. 68 00:03:10.280 --> 00:03:13.360 Yeah, yeah. But my deal is is I don't really 69 00:03:13.360 --> 00:03:16.599 follow NFL or NBA or you know, any of the 70 00:03:17.080 --> 00:03:21.080 big sports that people are are in there playing on 71 00:03:21.360 --> 00:03:25.080 So I think I think I'd have a better shot 72 00:03:25.159 --> 00:03:28.120 if I was like a die hard Eagles fan or something, 73 00:03:28.240 --> 00:03:30.719 you know. And so it's like, I'm going to put 74 00:03:30.759 --> 00:03:33.800 something in on every Eagles game, but I don't because 75 00:03:33.800 --> 00:03:36.599 I just don't care. Yeah, but yeah, So we were 76 00:03:36.639 --> 00:03:40.520 talking before the we got recording, and you said that 77 00:03:40.560 --> 00:03:45.120 you've been working on this autogenesis stuff and Gym. 78 00:03:45.400 --> 00:03:48.599 It's a term I came across called autogenetic, which basically 79 00:03:48.639 --> 00:03:54.599 just means self generating, and Okay, I've started to explore 80 00:03:54.680 --> 00:03:58.039 like what it might look like for these AI things 81 00:03:58.080 --> 00:04:02.159 to just like assemble the themselves things that let them 82 00:04:02.199 --> 00:04:04.879 do more rather than having me decide. 83 00:04:04.599 --> 00:04:05.520 What it should do more. 84 00:04:06.080 --> 00:04:06.400 Mhm. 85 00:04:07.360 --> 00:04:10.080 And that part of my exploration is this gem called 86 00:04:10.199 --> 00:04:12.800 it's a Ruby gem called Agentic. I started as just 87 00:04:12.879 --> 00:04:16.360 like I wanted a way to do plan and execution 88 00:04:16.519 --> 00:04:20.319 the workflows. So if you're not familiar with those, it's 89 00:04:20.360 --> 00:04:24.360 a way for you to have an AI thing an LLM, 90 00:04:24.439 --> 00:04:27.480 create a plan and then execute that plan. And so 91 00:04:27.959 --> 00:04:31.480 you can think cloud code right, is a perfect example 92 00:04:31.519 --> 00:04:36.560 of how this kind of started in a more practical way. 93 00:04:36.800 --> 00:04:40.240 There there were others before them, but you know, a 94 00:04:40.279 --> 00:04:43.600 plan execute is very like common pattern for AI stuff. 95 00:04:43.800 --> 00:04:45.279 And as I was. 96 00:04:45.199 --> 00:04:50.720 Building it, I had these like, you know, agents that 97 00:04:51.519 --> 00:04:58.040 I was like manually creating, and I thought, why am 98 00:04:58.079 --> 00:05:01.000 I creating these things? There has to be a that 99 00:05:01.079 --> 00:05:03.800 the plan could just be like, okay, what kinds of 100 00:05:03.920 --> 00:05:06.480 you know, what do I need in order to accomplish 101 00:05:06.720 --> 00:05:07.240 this goal? 102 00:05:08.000 --> 00:05:08.279 Right? 103 00:05:08.839 --> 00:05:14.560 And then I have you know, if I introduced concepts 104 00:05:14.839 --> 00:05:19.279 to the LLM that it knew how it could use 105 00:05:19.319 --> 00:05:22.439 them and how it could build them, could it do 106 00:05:22.480 --> 00:05:25.120 it effectively? And so the obvious one is agent, right, 107 00:05:25.240 --> 00:05:29.120 Like you have instructions and it can do things, it 108 00:05:29.120 --> 00:05:32.199 has tools available to it, and so can an LLM 109 00:05:32.839 --> 00:05:37.480 with just knowledge about like that construct build its own 110 00:05:37.879 --> 00:05:42.319 and assemble its own agents to accomplished tasks that it 111 00:05:42.360 --> 00:05:43.160 has to do. 112 00:05:43.279 --> 00:05:44.759 And so that's where I kind of like really. 113 00:05:44.680 --> 00:05:49.040 Dug into this agentic gem and started I created this 114 00:05:49.199 --> 00:05:53.199 like way to assemble agents based on just like instructions 115 00:05:53.199 --> 00:05:56.360 and giving it a role, and with that like primitive 116 00:05:56.519 --> 00:05:59.519 giving it a name, a role and instructions, throwing a 117 00:05:59.560 --> 00:06:02.639 task it and be like, hey, like assemble whatever agents 118 00:06:02.680 --> 00:06:05.480 you need to accomplish this task and give me the 119 00:06:05.480 --> 00:06:06.160 result of that. 120 00:06:06.399 --> 00:06:08.879 Right, So it reminds me a little bit of some 121 00:06:08.920 --> 00:06:12.480 of the conversations we've had with other people about AI 122 00:06:12.600 --> 00:06:17.319 agents and lms, where effectively, I think kind of the 123 00:06:17.360 --> 00:06:21.199 basic version is you have one agent or one program 124 00:06:21.279 --> 00:06:22.920 and you give it a task and it just kind 125 00:06:22.959 --> 00:06:25.920 of runs until it gets the stuff done right. And 126 00:06:25.959 --> 00:06:29.240 so then we've talked specifically, I'm thinking of when we 127 00:06:29.319 --> 00:06:32.519 talked to Obi about Ray where he said, you know, 128 00:06:32.639 --> 00:06:35.439 I've got multiple agents, right, So I've got like a 129 00:06:35.480 --> 00:06:38.959 calendar scheduler agent, I've got this agent, I've got that agent, 130 00:06:39.000 --> 00:06:40.639 I've got another agent, right, and so they all kind 131 00:06:40.639 --> 00:06:43.600 of specialize, and then he's got one or two that 132 00:06:43.800 --> 00:06:46.920 kind of orchestrate things at different levels and so and 133 00:06:46.959 --> 00:06:48.879 a lot of that was done deliberately, right. It's like, 134 00:06:48.959 --> 00:06:51.120 I'm going to have a calendar agent, I'm going to 135 00:06:51.160 --> 00:06:54.519 have a chat agent, I'm going to have a whatever 136 00:06:54.560 --> 00:06:57.720 else agent. And what you're saying is is in your case, 137 00:06:57.839 --> 00:07:01.839 you're saying, Okay, I just need this task done right, 138 00:07:01.920 --> 00:07:04.800 and so you make up what the agents are, right, 139 00:07:04.839 --> 00:07:08.560 and so you're giving that or that level of design 140 00:07:08.600 --> 00:07:10.519 and orchestration to the LLM as well. 141 00:07:11.199 --> 00:07:13.040 Exactly because. 142 00:07:14.240 --> 00:07:18.240 Because this object oriented, right, and all these types constructs 143 00:07:18.279 --> 00:07:22.279 are defined in code, I could save the artifacts of those, right. 144 00:07:22.360 --> 00:07:26.759 So like, as it's building out these agents to accomplish tasks, 145 00:07:27.120 --> 00:07:29.079 it's saving them off for use later. 146 00:07:29.920 --> 00:07:30.079 Right. 147 00:07:30.160 --> 00:07:31.839 So like if it goes. 148 00:07:31.600 --> 00:07:34.720 And if it has like a bigger plan that it's 149 00:07:34.759 --> 00:07:37.040 trying to get through and a goal is trying to accomplish, 150 00:07:37.160 --> 00:07:40.040 and it's creating all these subtasks and making agents and 151 00:07:40.120 --> 00:07:43.199 then reuse those agents as it goes through the plan, 152 00:07:43.639 --> 00:07:46.519 and so like let's say it's like just like a 153 00:07:46.560 --> 00:07:49.759 research agent that it decides to make, and it goes 154 00:07:50.040 --> 00:07:54.160 and it researches some content and then pulls back information 155 00:07:54.240 --> 00:07:56.360 and saves it in a file, and that's like what 156 00:07:56.399 --> 00:07:58.759 it was built to do. And then it comes across 157 00:07:58.800 --> 00:08:00.959 another task it is like I need to research about 158 00:08:00.959 --> 00:08:03.879 this new thing. You can just reuse that agent and 159 00:08:04.480 --> 00:08:09.399 because like it then becomes available to the system as like, hey, 160 00:08:09.439 --> 00:08:12.839 this is these are agents that are available right at 161 00:08:12.879 --> 00:08:15.759 every given turn in the task, you can choose to 162 00:08:15.959 --> 00:08:18.399 pick one of those existing ones or build a new one. 163 00:08:18.439 --> 00:08:21.839 And sometimes you know, if the research is if the 164 00:08:22.439 --> 00:08:25.199 task is not related enough to what is trying to 165 00:08:25.240 --> 00:08:27.839 do like let's say it was like just generic research 166 00:08:27.879 --> 00:08:29.920 on the web, but it needed to research in a 167 00:08:29.959 --> 00:08:32.759 specific file that it would create a new agent for 168 00:08:32.960 --> 00:08:36.639 file research. All right, So it's it's kind of interesting 169 00:08:36.679 --> 00:08:42.200 to watch this thing kind of bloom, right, uh huh 170 00:08:42.240 --> 00:08:43.840 as you give it different things to do. 171 00:08:44.440 --> 00:08:47.480 So one thing that I'm wondering about is it seemed 172 00:08:47.519 --> 00:08:51.679 like some of the sub jobs or subclasses of agent 173 00:08:51.799 --> 00:08:54.039 or however you want to look at, that they had 174 00:08:54.120 --> 00:08:57.519 pretty specific functions that they needed, right, So for example, 175 00:08:57.559 --> 00:09:01.279 talking to a calendar or you know, connecting to a 176 00:09:01.320 --> 00:09:05.120 particular service or things like that. Does it create those 177 00:09:05.159 --> 00:09:08.759 two and actually write the function code for those sometimes? 178 00:09:11.799 --> 00:09:13.639 Right? So that's the beauty of this is like it's 179 00:09:13.679 --> 00:09:15.480 an experiment and things that all. 180 00:09:15.559 --> 00:09:17.200 It really is coming from my job. 181 00:09:17.360 --> 00:09:20.720 Like, yeah, sometimes it does effectively do that, you know. 182 00:09:20.919 --> 00:09:24.799 Huh when it gets runs there are cases where it 183 00:09:25.080 --> 00:09:27.159 like you know, because I do have limitters on it 184 00:09:27.200 --> 00:09:32.600 that like don't let it like recursively you know, get 185 00:09:32.600 --> 00:09:33.120 to itself. 186 00:09:33.120 --> 00:09:34.080 So if it fails to. 187 00:09:34.039 --> 00:09:36.039 Do something, it doesn't get stuck and it could just 188 00:09:36.120 --> 00:09:38.399 move on and say I wasn't able to do that, 189 00:09:38.519 --> 00:09:40.360 And so it does do that, but yeah, I would 190 00:09:40.360 --> 00:09:43.000 say the more complicated things, it tries to just break 191 00:09:43.039 --> 00:09:46.360 it apart into smaller things and try and accomplish those. 192 00:09:46.480 --> 00:09:48.639 And so I haven't really pushed the limits. 193 00:09:48.279 --> 00:09:51.399 Of it yet, right, So, I guess one thing that 194 00:09:51.440 --> 00:09:53.759 I'm curious about with some of this, because I guess 195 00:09:53.759 --> 00:09:55.960 you could also just say, you know, here are some 196 00:09:56.039 --> 00:09:59.360 functions that I just have available, right, and then it 197 00:09:59.399 --> 00:10:03.600 could build the agents around those however it wants. But 198 00:10:03.679 --> 00:10:06.120 I guess I'm wondering, you know, how granular does it get? 199 00:10:06.720 --> 00:10:09.879 And I'm also curious what the implications are for Let's 200 00:10:09.919 --> 00:10:12.679 say I want to actually build and design something like 201 00:10:12.720 --> 00:10:14.600 this on my own. I don't want to do the 202 00:10:14.639 --> 00:10:18.360 auto genetic stuff. I just want to, you know, I 203 00:10:18.720 --> 00:10:22.960 want to decide where the boundaries are for for my agents. 204 00:10:23.000 --> 00:10:25.720 You know, does this inform those decisions for you? But 205 00:10:26.480 --> 00:10:30.039 let's let's back up real quick, like how how granular 206 00:10:30.080 --> 00:10:32.320 does it get? And how you know, how does it 207 00:10:32.360 --> 00:10:35.240 handle some of that stuff? Because I'm imagining, for example, 208 00:10:35.480 --> 00:10:38.320 let's say it did need to do some scheduling or 209 00:10:38.799 --> 00:10:42.720 you know, calendar management, right, it could have a connect 210 00:10:42.759 --> 00:10:45.120 to Google Calendar, and I have these six functions that 211 00:10:45.159 --> 00:10:48.120 it can do, or it could say, you know, I'm 212 00:10:48.240 --> 00:10:52.919 just a busy checker and I'm an appointment updater and 213 00:10:52.960 --> 00:10:55.480 I'm right, And so you could end up with agents 214 00:10:55.480 --> 00:10:57.919 that do a bunch of different things as opposed to 215 00:10:57.919 --> 00:11:01.200 one that just kind of generically and it is Google Calendar. 216 00:11:02.080 --> 00:11:02.679 Yeah, I get that. 217 00:11:02.759 --> 00:11:05.480 I mean I built this in a very modular way 218 00:11:05.480 --> 00:11:08.639 because I wanted the ability to have like agents that 219 00:11:08.679 --> 00:11:11.759 I specifically made with certain things. And so you can 220 00:11:12.080 --> 00:11:16.399 build your own agent, and there's even like you can 221 00:11:16.440 --> 00:11:19.120 give it like specifications if you don't want to fill 222 00:11:19.159 --> 00:11:21.559 in all the details, so you can create like kind 223 00:11:21.600 --> 00:11:22.960 of a spec for how the. 224 00:11:22.960 --> 00:11:24.759 Agent should be built. 225 00:11:25.039 --> 00:11:28.320 Okay, but yeah, there's like a register, so you could 226 00:11:28.360 --> 00:11:30.639 like register your agents and then it would make use 227 00:11:30.679 --> 00:11:33.320 of those like in the normal process as it's looking 228 00:11:33.360 --> 00:11:36.879 to accomplish tasks, and you can bypass all have to 229 00:11:36.960 --> 00:11:40.159 use Yeah you don't have to use self assembly. 230 00:11:40.440 --> 00:11:44.039 Yeah right, but but what how how far does the 231 00:11:44.080 --> 00:11:44.840 self assembly go? 232 00:11:45.039 --> 00:11:46.759 It goes pretty deep? Yeah, I mean it. 233 00:11:47.399 --> 00:11:51.360 So I built like a capability system I call it, 234 00:11:51.919 --> 00:11:55.440 where like let's say, you know, a capability could be 235 00:11:55.559 --> 00:12:00.080 most similar to like tools, I guess in the modern realm, 236 00:12:00.120 --> 00:12:01.919 but like I think we'd more of like okay, be 237 00:12:01.960 --> 00:12:05.840 able to search the web, like read a file, generate 238 00:12:05.879 --> 00:12:10.120 a pdf. Like these are all capabilities, right, And so 239 00:12:10.159 --> 00:12:12.919 I made like kind of an agent capability system that 240 00:12:13.000 --> 00:12:16.399 you can register with the agentic system and just say 241 00:12:16.440 --> 00:12:21.480 okay you have and it's like all defined abstractly with okay, 242 00:12:21.559 --> 00:12:25.879 you have this capability, and here's the implementation function and 243 00:12:25.919 --> 00:12:29.519 it takes inputs and it generates a normalized response. 244 00:12:29.639 --> 00:12:32.000 And these like. 245 00:12:32.039 --> 00:12:35.919 Specifications that you're generating for the capability help inform the 246 00:12:36.480 --> 00:12:40.200 agentic system as well what it can do, right as 247 00:12:40.240 --> 00:12:44.919 it's building agents too, So like you can basically define 248 00:12:45.000 --> 00:12:49.039 kind of like what what is capable for agents to 249 00:12:49.120 --> 00:12:51.879 be built with, right, And so it's not just gonna 250 00:12:52.320 --> 00:12:54.559 go in like say oh I can go and like 251 00:12:54.720 --> 00:12:58.919 connect Google Calendar and like automate a huge pipeline. There's 252 00:12:58.960 --> 00:13:01.240 only so many capability is that you can register at 253 00:13:01.279 --> 00:13:03.879 a time. And so like I made it kind of 254 00:13:04.039 --> 00:13:06.799 with that informed decision of like I don't want to 255 00:13:07.399 --> 00:13:08.399 just make stuff up. 256 00:13:08.360 --> 00:13:10.919 Because it was at first. 257 00:13:10.799 --> 00:13:13.000 Not really doing too great, but as soon as you 258 00:13:13.080 --> 00:13:15.440 put some like kind of guardrails around it and give 259 00:13:15.480 --> 00:13:17.120 it like, oh, you can only. 260 00:13:17.000 --> 00:13:19.840 Kind of do these things. It performs really well. 261 00:13:20.080 --> 00:13:23.039 And so like when you especially for you know, it's 262 00:13:23.080 --> 00:13:26.840 funny now that like research a deep research is a thing, right, 263 00:13:27.039 --> 00:13:29.279 Like that's like the easiest thing to do with all 264 00:13:29.320 --> 00:13:32.519 of these agents. It's just like right, the other report 265 00:13:32.720 --> 00:13:37.799 and like search the web, right, like so easy to do, right, yeah, 266 00:13:37.799 --> 00:13:39.440 And so like what happens when you want to do 267 00:13:39.559 --> 00:13:42.679 like next things and have it like yeah, like you said, 268 00:13:42.759 --> 00:13:44.919 talk to APIs or things like that. 269 00:13:45.879 --> 00:13:48.799 It's yeah, it's interesting to see how it evolves. 270 00:13:49.600 --> 00:13:53.799 Yeah, it does? Does it ever evolve? I'm gonna I, 271 00:13:53.879 --> 00:13:55.559 you know, I'm going to kind of build my own 272 00:13:55.840 --> 00:13:58.519 tools or my own capabilities or do you just lock 273 00:13:58.600 --> 00:14:00.919 that down? So it's like you have to ask. 274 00:14:00.799 --> 00:14:05.840 So I've tried, uh my, my preliminited. My initial goal 275 00:14:06.039 --> 00:14:07.919 was could I get it to build itself? 276 00:14:08.120 --> 00:14:11.840 Right? So, like if I I through just like a 277 00:14:12.159 --> 00:14:14.399 plan a goal at it, like hey, I want. 278 00:14:14.240 --> 00:14:18.320 To create like a self assembling agent system in Ruby, right, 279 00:14:18.399 --> 00:14:19.200 and could I. 280 00:14:19.159 --> 00:14:21.440 Get that to actually happen? 281 00:14:22.440 --> 00:14:22.559 Right? 282 00:14:22.639 --> 00:14:25.799 And what right? Like what agents would it built? 283 00:14:26.000 --> 00:14:26.200 You know? 284 00:14:26.200 --> 00:14:30.720 I created a coding agent for Ruby. Right, it created 285 00:14:30.879 --> 00:14:37.039 like uh, you know a I don't know, PM like agents, 286 00:14:37.240 --> 00:14:40.480 like managed a project. Right, It did all these things, 287 00:14:40.799 --> 00:14:41.639 and it got like. 288 00:14:43.240 --> 00:14:43.840 It got to a. 289 00:14:43.799 --> 00:14:46.159 Certain point where it was just like, all right, it 290 00:14:46.200 --> 00:14:49.559 just spent way too long, and it like it said, oh, 291 00:14:49.759 --> 00:14:52.000 I've gone through too many turns, and now I can't 292 00:14:52.440 --> 00:14:55.720 like I say it was decent. You know, it wasn't 293 00:14:55.720 --> 00:14:56.080 what I. 294 00:14:56.039 --> 00:15:01.360 Would code, right, but it's a little rising how far 295 00:15:01.440 --> 00:15:03.279 it did get and the agents that it built, and 296 00:15:03.360 --> 00:15:06.320 so I've actually reused and repurposed some of the agents 297 00:15:06.360 --> 00:15:09.399 that it made in my own interesting, which is kind 298 00:15:09.440 --> 00:15:12.159 of fun. And you know, the probably the biggest benefit 299 00:15:12.200 --> 00:15:15.360 to me personally is I got it to use Olama locally, 300 00:15:17.279 --> 00:15:19.960 so I could just use starcoder or something like that 301 00:15:20.279 --> 00:15:22.480 to run on my own machines and not have to 302 00:15:22.559 --> 00:15:25.279 spend any more. Yeah, but I'm trying to evolve it. 303 00:15:25.279 --> 00:15:29.799 It has this like extension system that I'm trying to 304 00:15:30.240 --> 00:15:33.639 like adapt where you can give it a domain, so 305 00:15:33.720 --> 00:15:35.679 like if you want it to be scoped on healthcare, 306 00:15:35.919 --> 00:15:39.639 then you can add like specific knowledge to that domain 307 00:15:39.879 --> 00:15:43.240 and have it use it in its assembly and execution. 308 00:15:43.519 --> 00:15:48.440 And I have like this initial like self learning system 309 00:15:48.480 --> 00:15:51.519 that I've started to to explore, where it keeps track 310 00:15:51.559 --> 00:15:55.960 of records of itself and tries to create patterns and 311 00:15:56.039 --> 00:16:00.559 like strategies based on how you use it, so it'll 312 00:16:00.600 --> 00:16:05.639 like slowly evolve in different ways based on how you 313 00:16:05.879 --> 00:16:09.399 like train it ultimately or tell it how to learn. 314 00:16:11.679 --> 00:16:13.799 And so I've been exploring That's probably where I want 315 00:16:13.799 --> 00:16:16.600 to spend more of my time. Next is kind of 316 00:16:16.600 --> 00:16:20.080 getting it to recognize things that it does well and 317 00:16:20.200 --> 00:16:24.120 doesn't and how do I tell it and inform it 318 00:16:24.200 --> 00:16:28.000 when it's doing something that it shouldn't be doing or not. 319 00:16:28.519 --> 00:16:32.240 I haven't really figured out that like human intervention aspect 320 00:16:32.919 --> 00:16:34.919 right at the moment. It's kind of just like making 321 00:16:34.919 --> 00:16:35.759 that up on its own. 322 00:16:35.960 --> 00:16:37.919 So it's kind of interesting. 323 00:16:37.480 --> 00:16:40.720 To explore what models do and what they learn on 324 00:16:40.759 --> 00:16:43.679 their own because they are all different. It's kind of 325 00:16:43.679 --> 00:16:46.879 fun to rerun the same experiments with different models and 326 00:16:47.000 --> 00:16:49.120 see how it evolves differently. 327 00:16:49.440 --> 00:16:54.720 Yeah, I bet so. I'm a little curious, you know, 328 00:16:54.759 --> 00:16:57.600 as you get into this, like what is it showing 329 00:16:57.639 --> 00:17:00.720 you or telling you about using agents or building agents? 330 00:17:00.759 --> 00:17:03.159 Like what has it taught you any lessons that it's 331 00:17:03.240 --> 00:17:05.880 like when I do this on this other project, I 332 00:17:05.880 --> 00:17:08.480 ought to build my agents more this way or that way. 333 00:17:08.839 --> 00:17:11.319 It's yeah, I would say the biggest thing I've kind 334 00:17:11.319 --> 00:17:14.240 of taken away is that lms are not good at 335 00:17:14.279 --> 00:17:20.039 like managing their own artifacts, right, Like if you tell 336 00:17:20.079 --> 00:17:23.559 it to like do something and like keep track of something, 337 00:17:24.160 --> 00:17:27.559 it's going to do it differently almost every time, and 338 00:17:27.720 --> 00:17:30.599 like you might see some normalization, but like it's going 339 00:17:30.680 --> 00:17:32.400 to take a lot of prompting to get it to 340 00:17:32.559 --> 00:17:36.000 like be any like deterministic at all. 341 00:17:36.200 --> 00:17:39.880 And so I've I've found that like trying to trying to. 342 00:17:39.839 --> 00:17:44.680 Help it avoid maybe some of those pathways to generate 343 00:17:44.720 --> 00:17:48.119 specific things in certain formats to just avoid those. Getting 344 00:17:48.119 --> 00:17:50.119 back to the lessons, it's hard to tell what is 345 00:17:50.160 --> 00:17:53.480 a good lesson or not right because so much of 346 00:17:53.519 --> 00:17:56.319 it is like the LM's deciding what to do. And 347 00:17:56.359 --> 00:18:00.279 I guess what I found is the more struck sure 348 00:18:00.680 --> 00:18:04.720 and reils that you give to llms, the more deterministic 349 00:18:05.359 --> 00:18:10.880 you can make what it does. And so it doesn't 350 00:18:10.960 --> 00:18:17.279 know anything, but if you if you can help kind 351 00:18:17.279 --> 00:18:20.519 of visualize things for it, it seems. 352 00:18:20.240 --> 00:18:22.240 To produce much better results. 353 00:18:23.200 --> 00:18:26.400 What I mean by that is like you know it 354 00:18:26.640 --> 00:18:29.400 has a ton of training data, so you know you 355 00:18:29.519 --> 00:18:31.400 kind of know, like you have a good idea and 356 00:18:31.559 --> 00:18:33.799 understanding of what training data has just by asking you 357 00:18:33.880 --> 00:18:37.759 a bunch of questions, right, and but you can get 358 00:18:37.799 --> 00:18:42.279 an idea for like what concepts it has too, right, 359 00:18:42.480 --> 00:18:45.720 And so I like to like poke the models all 360 00:18:45.759 --> 00:18:49.559 the time on what concepts it can recognize, right, And 361 00:18:49.599 --> 00:18:51.759 if you give it a new concept, can it like 362 00:18:52.759 --> 00:18:58.480 continue to like reading about that concept and like you know, 363 00:18:58.599 --> 00:19:01.960 mutate it at as like you would right, Like if 364 00:19:02.000 --> 00:19:05.119 you're going to create like a user object in rails, right, 365 00:19:05.279 --> 00:19:07.960 like and you wanted to add a new attribute to it? 366 00:19:08.039 --> 00:19:09.599 Is like that straightforward? 367 00:19:09.680 --> 00:19:12.240 Like would you be like, well, why would I add, right, 368 00:19:12.359 --> 00:19:15.680 like a shopping cart to a user? Maybe you would, right, 369 00:19:15.799 --> 00:19:18.440 But like would you would you add like a I 370 00:19:18.440 --> 00:19:21.319 don't know something super unrelated like a I don't know 371 00:19:21.359 --> 00:19:23.839 what Maybe that's a bad example. Let's say you had 372 00:19:23.839 --> 00:19:26.920 a school, would you add like a grocery store to it? 373 00:19:27.079 --> 00:19:28.240 Right? Right? Uh? 374 00:19:28.440 --> 00:19:31.279 Can the LLM like know that kind of thing? 375 00:19:31.759 --> 00:19:35.000 The answer is yes, the lms are very good at 376 00:19:35.000 --> 00:19:38.200 those connections of things, right, And so like, uh, the 377 00:19:38.240 --> 00:19:42.079 further away that the concepts are actually better to the genet, 378 00:19:42.839 --> 00:19:46.359 like the generations will get right, because that's what it's doing. 379 00:19:46.400 --> 00:19:49.839 It's like trying to hone in on very like similar things, 380 00:19:49.960 --> 00:19:52.119 right and like what the next things are. And so 381 00:19:52.160 --> 00:19:56.079 if you can give it more like funneling going back 382 00:19:56.359 --> 00:20:00.039 to Obi's like you know, narrowing the path, right, the 383 00:20:00.079 --> 00:20:02.440 more you can do that, even on a conceptual level, 384 00:20:02.759 --> 00:20:06.480 the better, right. And so that's where I've kind of 385 00:20:06.519 --> 00:20:09.839 like taken, like been blown away by how well this works. 386 00:20:10.160 --> 00:20:12.480 Is like I can just give it more and more. 387 00:20:12.759 --> 00:20:17.119 Concrete concepts for it to reason about, and it can 388 00:20:17.160 --> 00:20:20.359 figure out how they work together, right, And then okay, well, 389 00:20:20.400 --> 00:20:23.480 if you then help it, like tell it how those 390 00:20:23.480 --> 00:20:26.559 things connect and work together, it doesn't even better. And 391 00:20:26.599 --> 00:20:29.000 so like and the more things that you can like 392 00:20:29.440 --> 00:20:33.000 you know, glue together and firm up, like, the better 393 00:20:33.079 --> 00:20:35.440 that it performs, and the more determinists. 394 00:20:34.839 --> 00:20:36.519 That you can get it. And so I guess that's 395 00:20:36.640 --> 00:20:37.279 kind of what I've. 396 00:20:37.200 --> 00:20:41.279 Taken, is like if you can decouple that idea and 397 00:20:41.400 --> 00:20:44.240 like make a bunch of things so that you can 398 00:20:44.279 --> 00:20:48.119 define those like concepts and how they work together, then 399 00:20:48.160 --> 00:20:49.759 you'll get the best result. 400 00:20:49.480 --> 00:20:55.160 From all of this, gotcha, So which is the most 401 00:20:55.160 --> 00:20:58.039