WEBVTT 1 00:00:04.679 --> 00:00:07.240 Hey, folks, welcome back to another episode of the Ruby 2 00:00:07.320 --> 00:00:10.400 Rogues podcast. I'm your host this week, Charles max Wood, 3 00:00:10.400 --> 00:00:12.880 and I am here with Carmine Paulino. 4 00:00:14.279 --> 00:00:18.120 And you said you're from Berlin, from Italy, Southern Italy. 5 00:00:18.519 --> 00:00:20.440 I always get excited when I have Italian song because 6 00:00:20.440 --> 00:00:21.359 I lived in Italy. 7 00:00:21.120 --> 00:00:23.239 For a couple of years, and I'm like, oh, all 8 00:00:23.280 --> 00:00:26.039 the good memories anyway, and the good memories are the 9 00:00:26.079 --> 00:00:28.960 good food and good people anyway. But yeah, you live 10 00:00:29.000 --> 00:00:32.560 in Berlin and you wrote the Ruby llm jem. I'm 11 00:00:32.600 --> 00:00:35.479 sure there's a lot more to know about you. What 12 00:00:35.520 --> 00:00:36.320 else should people know? 13 00:00:36.399 --> 00:00:38.320 And go, oh, Carmine is so cool. 14 00:00:39.880 --> 00:00:42.280 Well, thank you very much for the introduction, Charles. It's 15 00:00:42.280 --> 00:00:46.280 a pleasure to be here. So I started using technology 16 00:00:46.280 --> 00:00:49.200 when I was really, really young. So at the age 17 00:00:49.200 --> 00:00:51.560 of five, I started to write my first batch scripts 18 00:00:52.000 --> 00:00:57.320 to automates launching my favorite games, and damn, moment of 19 00:00:57.359 --> 00:01:00.799 making a computer do exactly what I wanted felt like magic. 20 00:01:01.159 --> 00:01:06.040 So that sparked lifelong fascination with technology. And at the 21 00:01:06.079 --> 00:01:10.400 moment I am building like AI tools, you know, I'm 22 00:01:10.400 --> 00:01:13.840 building chat with work, which is this retrieval advantage generation 23 00:01:14.040 --> 00:01:17.239 app that you can connect to your Google Drive and 24 00:01:17.359 --> 00:01:24.760 Slack Notion accounts essentially chat with all your documents, just 25 00:01:24.799 --> 00:01:27.239 as if Chgpt or Claud. 26 00:01:27.000 --> 00:01:28.239 Would know anything about them. 27 00:01:29.120 --> 00:01:32.040 I also co founded fresh Flow, which is also an 28 00:01:32.079 --> 00:01:35.680 AI based startup that helps supermarkets reduce food waste and 29 00:01:35.719 --> 00:01:40.079 maximize their profits. And yeah, I've done a bunch of things. 30 00:01:40.640 --> 00:01:41.120 Essentially. 31 00:01:42.159 --> 00:01:46.439 I graduated in computer science from University of Bologna, then 32 00:01:46.480 --> 00:01:51.640 did my master in AI from Free University of Amsterdam, 33 00:01:51.799 --> 00:01:54.319 and then I've been a data scientist ever since. But 34 00:01:54.680 --> 00:01:58.400 I think relevant to this podcast is that my last 35 00:01:58.680 --> 00:02:02.319 project at the Universe Still Bologna, which was for Internet 36 00:02:02.400 --> 00:02:06.879 and Web Technologies, was actually in Ruby. And the story 37 00:02:06.920 --> 00:02:10.400 goes that, you know, it was supposed to be in 38 00:02:10.400 --> 00:02:16.439 PHP and at the time, mind you, yeah, at the time, 39 00:02:16.560 --> 00:02:18.280 I was already. 40 00:02:20.039 --> 00:02:20.840 It's a great. 41 00:02:20.639 --> 00:02:26.639 Story because I was already looking at acer News and 42 00:02:27.280 --> 00:02:29.680 at the time, all my team, you know, we were 43 00:02:29.680 --> 00:02:34.639 four people, they all knew PHP and I didn't, right, 44 00:02:35.039 --> 00:02:37.520 But I followed after News and I saw Ruby on 45 00:02:37.599 --> 00:02:41.120 rails and this was, I believe, if I remember correctly, 46 00:02:41.280 --> 00:02:46.159 was twenty ten, so you know the era, right, And 47 00:02:46.800 --> 00:02:49.599 I was like, so intrigued by Ruby on Rails at 48 00:02:49.639 --> 00:02:54.000 the time that I convinced my entire team to switch 49 00:02:54.039 --> 00:02:56.840 to Ruby and rails and we all had so much 50 00:02:56.879 --> 00:02:59.759 fun that actually some of us went to work for 51 00:03:00.319 --> 00:03:02.639 other companies that use Ruby and rails. In fact, one 52 00:03:02.639 --> 00:03:05.400 of my friends I went to work for SoundCloud here 53 00:03:05.439 --> 00:03:08.120 in Berlin. I even thought about doing that, I even 54 00:03:08.159 --> 00:03:10.520 went through the application, but then I thought, hey, I 55 00:03:10.560 --> 00:03:13.759 want to have a master degree and you know, maybe 56 00:03:13.759 --> 00:03:17.560 look at something else as well. And that's where I 57 00:03:17.560 --> 00:03:21.680 started looking at AI. And I was a data scientist 58 00:03:21.800 --> 00:03:24.719 for many years. Whenever I could do anything that was 59 00:03:24.759 --> 00:03:26.680 related to the web, it was Ruby on rails anyway, 60 00:03:27.800 --> 00:03:33.039 but mostly Python, mostly machine learning engineering, you know, trying 61 00:03:33.080 --> 00:03:37.719 to scale also systems, so between you know, the DevOps 62 00:03:37.759 --> 00:03:40.439 to the modeling level, right, So it was kind of 63 00:03:40.439 --> 00:03:44.400 a full stack, let's say, machine learning person for a 64 00:03:44.439 --> 00:03:47.879 long time. And then well I decided to go back 65 00:03:47.919 --> 00:03:51.520 a little bit to academia. So I went to the 66 00:03:51.560 --> 00:03:55.400 volunteering Institute in the UK, mostly helping out with an 67 00:03:55.479 --> 00:04:00.960 organization called that as Signs for Social Good. And then 68 00:04:01.560 --> 00:04:05.080 I received a call from entrepreneur first and they told 69 00:04:05.080 --> 00:04:07.400 me like, hey, you have a profile of a potential founder. 70 00:04:07.960 --> 00:04:10.800 I was like, really, because that's that was my dream 71 00:04:10.919 --> 00:04:13.719 and I never thought I could do that. So I 72 00:04:13.800 --> 00:04:15.960 went for it and then I found it fresh Flow 73 00:04:17.000 --> 00:04:18.920 and now I'm actually. 74 00:04:18.560 --> 00:04:19.240 Doing my own thing. 75 00:04:19.240 --> 00:04:23.399 I'm a slopreneur and I'm building chat with work and 76 00:04:23.480 --> 00:04:27.079 so to connect back to rubylm since now I'm completely 77 00:04:27.079 --> 00:04:30.399 free and I'm doing you know, like a web app essentially, 78 00:04:31.279 --> 00:04:35.639 it was obviously done in Ruby and rails and at 79 00:04:35.680 --> 00:04:39.560 the time and maybe I'm jumping too far, but at 80 00:04:39.600 --> 00:04:41.720 the time, I wanted to do it because. 81 00:04:44.279 --> 00:04:45.199 It's all connected. 82 00:04:46.720 --> 00:04:52.160 I was searching for like libraries for building chat with 83 00:04:52.319 --> 00:04:56.639 work and I saw some of the libraries out there 84 00:04:56.759 --> 00:04:59.720 and I wasn't so satisfied with them, so I decided 85 00:04:59.720 --> 00:05:00.639 to build my own. 86 00:05:02.240 --> 00:05:04.639 Right Yeah. 87 00:05:05.000 --> 00:05:07.560 So yeah, it's it's interesting kind of getting into that 88 00:05:07.680 --> 00:05:10.360 a bit. And some of this reflects, you know, with 89 00:05:10.399 --> 00:05:13.279 my own background, right, you know, with you know, I 90 00:05:13.319 --> 00:05:15.000 was kind of a solopreneur. I had a couple of 91 00:05:15.040 --> 00:05:17.959 virtual assistants, but I mostly just you know, ran things 92 00:05:18.000 --> 00:05:21.480 myself with the podcast network for I don't know, like 93 00:05:22.040 --> 00:05:25.120 eight years, and you know, I built most of the 94 00:05:25.120 --> 00:05:30.079 software that the podcast run on and you know, just 95 00:05:30.079 --> 00:05:32.839 just on and on and on and on. I'm not 96 00:05:32.839 --> 00:05:36.439 going to get into kind of the emotional wreckage that 97 00:05:36.480 --> 00:05:41.160 I went through the last few years, especially with you know, now, 98 00:05:41.199 --> 00:05:43.480 I'm in a full time job and I love the 99 00:05:43.480 --> 00:05:45.120 people I work with, and I love the projects I 100 00:05:45.199 --> 00:05:48.439 work on. But there's just something about working on your 101 00:05:48.480 --> 00:05:51.079 own thing and having it, you know, come together in 102 00:05:51.120 --> 00:05:51.439 that way. 103 00:05:51.519 --> 00:05:54.680 So yeah, yeah, I really identify with a lot of 104 00:05:54.680 --> 00:05:58.120 what you're talking about. But yeah, I wanted to. 105 00:05:59.240 --> 00:06:01.240 I wanted to jump in and just start talking about 106 00:06:01.279 --> 00:06:04.079 Ruby LM, and I know that the like the AI 107 00:06:04.199 --> 00:06:07.240 space and the LLM space is pretty hot right now. 108 00:06:08.759 --> 00:06:10.680 And that's one of the things that I've been wanting 109 00:06:10.680 --> 00:06:15.240 to put together for a while. Is specifically you said, hey, 110 00:06:15.399 --> 00:06:17.439 you know, you can get in, you can chat with. 111 00:06:17.399 --> 00:06:19.639 Your dropbox or your Google Driver or whatever. 112 00:06:20.160 --> 00:06:22.839 And for me, I want the same thing, but I 113 00:06:22.879 --> 00:06:24.639 want to be able to do it with my podcasts. 114 00:06:24.720 --> 00:06:28.319 Right It's like, hey, you know, change the host out right. 115 00:06:28.439 --> 00:06:31.600 So like on Javascriptjbver, we just had somebody, you know, 116 00:06:31.600 --> 00:06:33.439 he's like, hey, I've got to take some time away 117 00:06:33.560 --> 00:06:36.639 and write, so things like that. You know, he's the 118 00:06:36.720 --> 00:06:40.519 last guy that's one of the originals besides me, Right, 119 00:06:40.639 --> 00:06:42.759 so chat with it about that, or Hey, how's the 120 00:06:42.800 --> 00:06:47.360 traffic look, or you know, publish this episode on this 121 00:06:47.439 --> 00:06:49.120 date instead of that date and that kind of a thing. 122 00:06:49.160 --> 00:06:52.920 And so yeah, I got excited about it. 123 00:06:53.000 --> 00:06:57.519 And then the local meetup the last like three months 124 00:06:57.519 --> 00:07:01.560 have all had presentations on how to through API's hit. 125 00:07:03.360 --> 00:07:05.199 LLM end points basically. 126 00:07:05.560 --> 00:07:09.079 And you know so, and I've been playing with some 127 00:07:09.160 --> 00:07:11.480 of that, and we've had Obi Fernandez on to talk 128 00:07:11.519 --> 00:07:15.199 about Ray and so I got everything hooked up and 129 00:07:16.800 --> 00:07:19.480 was figuring all that stuff out. Ray is our AIX 130 00:07:19.519 --> 00:07:21.560 for the people who are listening and trying to go 131 00:07:21.600 --> 00:07:26.040 find it, and he talked about it before, but I 132 00:07:26.399 --> 00:07:29.399 found that the documentation didn't give me clear direction on 133 00:07:29.680 --> 00:07:33.040 how to get everything together the way I wanted. And 134 00:07:33.319 --> 00:07:36.160 he kind of implies that you can stream the response back, 135 00:07:37.480 --> 00:07:41.959 but he basically says it's in my book, and I went, 136 00:07:42.000 --> 00:07:43.279 I have the book, and I went looked in the 137 00:07:43.279 --> 00:07:46.800 book and I got lost. So but but I did 138 00:07:46.879 --> 00:07:49.560 have a fundamental thing working to do the presentation at 139 00:07:49.560 --> 00:07:52.120 the meet up last night, and I was doing a 140 00:07:52.199 --> 00:07:54.839 chat bot and I just hooked it up to OpenWeatherMap 141 00:07:55.160 --> 00:07:57.480 dot org and so it was just you know so. 142 00:07:57.959 --> 00:07:59.600 But then I was showing him, Look, you don't just 143 00:07:59.639 --> 00:08:02.040 ask it, hey, what's the weather. Here's the latitude longitude. 144 00:08:02.279 --> 00:08:05.120 But I put in, is a good is it a 145 00:08:05.120 --> 00:08:07.079 good time right now to go have a picnic? 146 00:08:07.120 --> 00:08:07.560 And right? 147 00:08:07.600 --> 00:08:12.199 It came back and said the weather is right. So 148 00:08:12.279 --> 00:08:16.279 it figured out my cities at this latitude longitude and right. 149 00:08:16.319 --> 00:08:18.639 And this is the power of the AI, right, is 150 00:08:18.680 --> 00:08:21.240 that it's smart enough to figure out some of the context. 151 00:08:21.399 --> 00:08:24.439 And I'm being really long winded and I'm sorry, but anyway, 152 00:08:24.480 --> 00:08:27.079 what happened was, for some reason, the open router gem 153 00:08:28.040 --> 00:08:33.120 it was working Tuesday night, and then Wednesday morning when 154 00:08:33.159 --> 00:08:35.919 I was fiddling with it, it just wouldn't connect and 155 00:08:35.919 --> 00:08:36.519 it was given. 156 00:08:36.320 --> 00:08:37.320 Me a four or four error. 157 00:08:37.639 --> 00:08:41.000 Of course, the error also said that my JWT token 158 00:08:41.200 --> 00:08:44.440 was wrong, but I was just using an apikey and 159 00:08:44.519 --> 00:08:47.200 so I think something must have changed on their end 160 00:08:47.600 --> 00:08:51.240 or there was something funky going on between me and them, 161 00:08:51.240 --> 00:08:53.360 because it worked fine one day and was broken the next. 162 00:08:53.399 --> 00:08:56.960 I'm like, I really want to show those guys something working. 163 00:08:57.320 --> 00:09:00.240 And then I was like, but I'm talking to Carmine tomorrow, 164 00:09:01.360 --> 00:09:03.080 and so I just went and grabbed it and within 165 00:09:03.120 --> 00:09:04.799 an hour and a half I had the exact same 166 00:09:04.840 --> 00:09:09.480 thing working. And so, anyway, this is cool stuff, cool 167 00:09:09.519 --> 00:09:13.679 cool gem. Anyway, So I'm just curious. I mean, if 168 00:09:13.720 --> 00:09:15.799 you had any other people come to you and basically 169 00:09:15.799 --> 00:09:20.720 say this stuff is great, save my bacon. I don't 170 00:09:20.759 --> 00:09:25.480 know what's been your feedback on Ruby LLM. 171 00:09:26.039 --> 00:09:31.799 It has been absolutely overwhelmingly positive. I couldn't have imagined 172 00:09:31.879 --> 00:09:35.639 how good this would have been. So when I released 173 00:09:35.639 --> 00:09:38.080 the gem, I was like sitting on it for like 174 00:09:38.120 --> 00:09:42.799 two weeks and on you know zero point one point 175 00:09:42.879 --> 00:09:45.440 zero pre forty nine or something like that. 176 00:09:45.519 --> 00:09:47.639 Right, I had to release a lot of pre release. 177 00:09:47.440 --> 00:09:50.879 Versions for myself, and I thought, hey, maybe I should 178 00:09:50.919 --> 00:09:53.399 release it right now instead of waiting until I get 179 00:09:53.480 --> 00:09:57.080 everything downe with chatwood work and you know, doing it 180 00:09:57.200 --> 00:10:00.559 let's say the DHH way where you released it after base, Right, 181 00:10:01.039 --> 00:10:03.600 that was my original plan. I was like, well, what 182 00:10:03.679 --> 00:10:06.159 can go wrong? And so I released it and I 183 00:10:06.240 --> 00:10:11.519 made some write posts and immediately in two days we 184 00:10:11.559 --> 00:10:14.120 got to two hundred and sixty style and my goal 185 00:10:14.399 --> 00:10:20.360 was to get to three hundred, so already kind of 186 00:10:20.360 --> 00:10:23.759 achieved my goal in no time at all, and then 187 00:10:24.080 --> 00:10:28.399 someone posted it on Accre News and I remember that 188 00:10:28.639 --> 00:10:31.039 night I was playing in a club. 189 00:10:31.080 --> 00:10:34.399 So I'm also a DJ onlands, you know, my free time. 190 00:10:36.120 --> 00:10:40.360 And yeah, and I come back home, I go to sleep, 191 00:10:41.519 --> 00:10:44.559 and around like six am in the morning, I get 192 00:10:45.320 --> 00:10:48.600 so many notifications of discord, which for some reason I 193 00:10:48.600 --> 00:10:53.080 could hear. I didn't like mute my computer and I 194 00:10:53.200 --> 00:10:55.919 go and check and people were saying like, oh my god, 195 00:10:55.960 --> 00:10:57.960 you made it to a front page of Accre News. 196 00:10:58.039 --> 00:11:01.360 How amazing. And in one we got to one point 197 00:11:01.399 --> 00:11:07.720 seven thousand stars. Wow, which was absolutely overwhelming. Yeah. So 198 00:11:07.799 --> 00:11:12.960 the reception has been fantastic. I've made new friends. There's 199 00:11:13.000 --> 00:11:15.200 some people that I talked to now kind of regularly. 200 00:11:16.159 --> 00:11:18.360 I have an audience on Twitter, which I didn't really 201 00:11:18.399 --> 00:11:23.080 have before. So it has been absolutely fantastic. Just mostly 202 00:11:23.799 --> 00:11:25.960 I would say ninety nine point nine nine nine percent 203 00:11:25.960 --> 00:11:27.879 and everything has been super positive. 204 00:11:28.080 --> 00:11:29.879 Very cool, very very cool. 205 00:11:30.519 --> 00:11:34.399 Yeah, I have to say, yeah. I mean my feedback 206 00:11:34.480 --> 00:11:38.240 is is that it's really simple to use. I think 207 00:11:38.279 --> 00:11:41.200 a lot of people get worried that oh it's AI 208 00:11:41.399 --> 00:11:43.320 and I have to know all these things and. 209 00:11:44.799 --> 00:11:47.039 No, you just tell it. I want to chat, you know, 210 00:11:47.480 --> 00:11:48.440 you have to do a little. 211 00:11:48.240 --> 00:11:49.919 Bit of setup so that it knows how to connect 212 00:11:49.960 --> 00:11:53.200 open ai or something. But that was for me at least, literally, 213 00:11:53.840 --> 00:12:00.559 here's my API key go crazy so. 214 00:12:00.559 --> 00:12:00.840 Great. 215 00:12:01.000 --> 00:12:04.480 Yeah, that was exactly the type of design that I 216 00:12:04.519 --> 00:12:09.159 wanted to have. So basically the reason why I made 217 00:12:10.000 --> 00:12:13.879 Ruby l LAMB is because when I looked at the 218 00:12:13.879 --> 00:12:18.600 other libraries, they essentially fit in two different categories. Right, 219 00:12:18.679 --> 00:12:21.279 So you have the libraries that are mostly done by 220 00:12:21.320 --> 00:12:24.039 Alex Rudahl. Shout out to him, Oh. 221 00:12:24.000 --> 00:12:33.759 Basic, great guy, that his library is amazing. It does everything, yes, exactly. 222 00:12:34.919 --> 00:12:38.600 So his library is his main library is Ruby, open ai, right, 223 00:12:38.679 --> 00:12:42.759 and then he has also the Entropic gem. They are 224 00:12:43.320 --> 00:12:46.679 lightweight layers on top of the HDCP calls, right. So 225 00:12:47.000 --> 00:12:49.320 what that means is that it's really easy to support 226 00:12:49.480 --> 00:12:52.840 every single function from the l lamps, but it's also 227 00:12:53.000 --> 00:12:55.720 really hard to then switch between different l lamps. 228 00:12:55.559 --> 00:12:56.279 If you have to switch. 229 00:12:56.360 --> 00:12:59.279 And one of my use cases for chat with work. 230 00:13:01.399 --> 00:13:04.840 Exactly, and all of the response objects, right, they're all different. 231 00:13:05.519 --> 00:13:09.000 So when I was designing Chat with Work, I wanted 232 00:13:09.039 --> 00:13:12.480 to have something that could make it easy to switch 233 00:13:12.559 --> 00:13:18.279 between models and I was thinking either mid conversation as 234 00:13:18.320 --> 00:13:22.159 a you know, north star, and so I needed to 235 00:13:22.200 --> 00:13:24.879 have something that was a unified interface. 236 00:13:25.960 --> 00:13:27.360 So I looked at the other. 237 00:13:27.480 --> 00:13:31.600 Type of libraries, you know, so like lang chain RB specifically, 238 00:13:32.360 --> 00:13:37.240 and I saw that I wanted to have some other features. 239 00:13:37.440 --> 00:13:40.600 The design wasn't exactly what I would have done for 240 00:13:40.639 --> 00:13:45.200 a library, but I implemented it. But then I wanted 241 00:13:45.200 --> 00:13:50.679 to have the pricing information, the model information, and listing 242 00:13:50.720 --> 00:13:54.279 of the models. So I started modifying it, and I 243 00:13:54.320 --> 00:13:57.159 started to go into the code of lang chain and 244 00:13:58.120 --> 00:14:00.600 it is quite a big jem. It's not as it 245 00:14:00.600 --> 00:14:03.840 has a lot of code. Yeah, so already you know, 246 00:14:03.879 --> 00:14:07.759 it was quite difficult to orienter yourself around, and it 247 00:14:07.840 --> 00:14:10.279 is quite I would I want to say, it's a 248 00:14:10.320 --> 00:14:14.600 little over engineered for my taste, right, Okay, But I 249 00:14:14.759 --> 00:14:19.000 started modifying it, and also like, it depends on the 250 00:14:19.879 --> 00:14:23.840 Alex's gems, right, so in order to have that same feature, 251 00:14:24.120 --> 00:14:26.960 I need to modify the entropic jam. Then also lun 252 00:14:27.080 --> 00:14:30.159 chain and also lun chain or be rails right, because 253 00:14:30.200 --> 00:14:32.120 there's a separate JEM for the rails integration. 254 00:14:32.440 --> 00:14:34.080 I was like, is this really what I want? 255 00:14:34.519 --> 00:14:37.600 Also, I noticed other things that I didn't like about it, 256 00:14:37.639 --> 00:14:42.399 which is that it put the providers first. And I 257 00:14:42.440 --> 00:14:47.919 think philosophically speaking, we don't really care about providers, right, 258 00:14:48.039 --> 00:14:49.399 people care about models. 259 00:14:50.080 --> 00:14:51.399 Where do they come from? 260 00:14:51.960 --> 00:14:55.360 Maybe it's interesting in terms of pricing, But if you 261 00:14:55.399 --> 00:14:57.360 want to just chat with a model, you want to 262 00:14:57.399 --> 00:15:00.840 just go like Ruby lam or Chat model and just 263 00:15:00.960 --> 00:15:04.200 go with it, right. I think that's the that's the 264 00:15:04.240 --> 00:15:09.240 beauty and why people have been finding Ruby LM so interesting. 265 00:15:10.039 --> 00:15:13.799 So one question I have puts the models first? 266 00:15:15.480 --> 00:15:16.440 Yeah, one question I. 267 00:15:16.440 --> 00:15:19.960 Have on that real quick is so initially I was, 268 00:15:20.080 --> 00:15:22.960 like I said, I was using open router and open 269 00:15:23.039 --> 00:15:25.519 router it kind of does the you can switch models 270 00:15:26.000 --> 00:15:29.399 on their service, right, so you're you can pick them, 271 00:15:29.399 --> 00:15:31.200 but you don't have to set up all the different 272 00:15:31.440 --> 00:15:37.960 service providers. So why not go with something like that? 273 00:15:37.960 --> 00:15:38.720 That's a good question. 274 00:15:39.240 --> 00:15:44.799 Well, the open router basically locks you in into that environment, 275 00:15:45.279 --> 00:15:48.240 so you need to use open router. What if open 276 00:15:48.320 --> 00:15:52.679 router dies or if providers like alge my Little Story 277 00:15:53.440 --> 00:15:58.440 or you four four errors right exactly. So I didn't 278 00:15:58.440 --> 00:16:01.639 want to be locked in into anything. I wanted to 279 00:16:01.720 --> 00:16:05.759 use truly open source software just like you know, Ruby 280 00:16:05.759 --> 00:16:10.279 and rails are and that's why I decided, you know what, 281 00:16:10.399 --> 00:16:13.240 this is probably a good idea to do. And yeah, 282 00:16:13.279 --> 00:16:17.240 there were other reasons actually why I thought that the 283 00:16:17.360 --> 00:16:22.720 existing gems were not of my taste. For example, streaming, 284 00:16:22.799 --> 00:16:25.519 even though like line chain or b has, you know, 285 00:16:25.519 --> 00:16:29.559 I unified the interface for most things. Streaming is where 286 00:16:29.559 --> 00:16:33.799 it gets complicated. So whenever you get a chunk, you 287 00:16:33.879 --> 00:16:38.279 actually get a chunk that is exactly the object that 288 00:16:38.600 --> 00:16:42.639 is retrieved from the API itself, So you need to 289 00:16:43.720 --> 00:16:46.519 parse the chunks in a different way. If you have 290 00:16:46.559 --> 00:16:49.759 an entropic model, then you have an opening eye model. 291 00:16:50.759 --> 00:16:56.720 And so all of these problems combined made it so 292 00:16:56.759 --> 00:16:59.360 that I wanted to build my own. 293 00:17:00.440 --> 00:17:04.160 Nice Well I'm glad you did because it's it's freaking awesome. 294 00:17:04.759 --> 00:17:09.519 So yeah, let's let's dive in a little bit. So 295 00:17:09.640 --> 00:17:13.160 you mentioned models, and if you get into like the 296 00:17:13.200 --> 00:17:16.680 getting started, I mean, I only hooked up open AI, 297 00:17:16.960 --> 00:17:20.079 but it sounds like you want people to be able 298 00:17:20.160 --> 00:17:23.240 to switch seamlessly. So if I put in an open 299 00:17:23.279 --> 00:17:28.640 aike and an Anthropic key, and one or two other providers, 300 00:17:28.640 --> 00:17:32.519 maybe a Gemini key, I can switch between those seamlessly. 301 00:17:35.400 --> 00:17:37.039 That's right. That's right. 302 00:17:37.920 --> 00:17:41.640 And the way that this works is that Ruby ALAM 303 00:17:41.720 --> 00:17:45.359 essentially has two different parts of the code. So you 304 00:17:45.440 --> 00:17:49.759 have the clean interface with the user, right, So you 305 00:17:49.880 --> 00:17:53.599 have the chat object, you have the message object, you 306 00:17:53.720 --> 00:17:58.640 have the content object, et cetera, et cetera, and then 307 00:17:58.680 --> 00:18:01.720 you have the mess part of the code, which is 308 00:18:01.759 --> 00:18:05.799 the provider code, right, and what the provider code does 309 00:18:05.880 --> 00:18:10.400 I'm saying messy, but I think it's decent. But it 310 00:18:10.400 --> 00:18:15.000 has all the messy parts of actually communicating with the 311 00:18:15.000 --> 00:18:19.559 provider interfaces and translating between you know, the Ruby LM 312 00:18:19.640 --> 00:18:26.319 internal representation and the provider representation. It sounds pretty straightforward, 313 00:18:26.759 --> 00:18:29.160 and so I implemented it that way so that you 314 00:18:29.319 --> 00:18:35.160 can have a very simple interface but actually change between models, 315 00:18:35.839 --> 00:18:39.119 because then it's just a question of parsing and retrieving 316 00:18:39.200 --> 00:18:44.559 data and formatting it for their for the format not 317 00:18:44.720 --> 00:18:47.839 really about you know, a specific representation that only works 318 00:18:48.240 --> 00:18:50.599 for the specific provider. I definitely didn't want to do that, 319 00:18:51.119 --> 00:18:54.359 and the way to achieve that was to essentially study 320 00:18:54.839 --> 00:18:59.240 the the APIs from you know, open Ai and Entropic, 321 00:19:00.319 --> 00:19:04.160 because that's where I started, and figure out what is 322 00:19:04.200 --> 00:19:08.319 the best way that they implement things, and how would 323 00:19:08.319 --> 00:19:11.920 I implement it in terms of the objects, the Ruby objects. 324 00:19:11.519 --> 00:19:13.000 That I want to have in my library. 325 00:19:13.480 --> 00:19:15.680 And so I picked and choose, you know, how to 326 00:19:15.720 --> 00:19:21.079 represent tool calls for example, and that's always now represented RUBYLM. 327 00:19:21.160 --> 00:19:22.599 And then if the auto. 328 00:19:22.359 --> 00:19:26.680 Provider that doesn't represent it this way, you know, will 329 00:19:26.720 --> 00:19:32.559 get a different format method in their modules in order 330 00:19:32.559 --> 00:19:35.039 to do that. 331 00:19:35.039 --> 00:19:35.720 That's awesome. 332 00:19:35.920 --> 00:19:38.680 So one thing that I'm wondering about is do the 333 00:19:38.720 --> 00:19:42.640 different models have different capabilities that you have to adjust 334 00:19:42.720 --> 00:19:44.920 for or let people know, Hey, you can do this 335 00:19:45.079 --> 00:19:48.759 with I don't know, GPT and Anthropic, but not with 336 00:19:48.880 --> 00:19:52.559 Gemini or something like that. 337 00:19:52.559 --> 00:19:55.680 That's right, Yeah, that's a great question. So I implemented 338 00:19:55.680 --> 00:19:58.799 them all the registry. That's actually where this whole thing 339 00:19:58.839 --> 00:20:01.359 about RUBYLM started right when I wanted to implement the 340 00:20:01.599 --> 00:20:07.039 model registry for for line chain. And it basically has 341 00:20:07.720 --> 00:20:12.559 a bunch of different capabilities per each model. So for example, 342 00:20:13.960 --> 00:20:17.799 it would know if a model has a function support, 343 00:20:18.440 --> 00:20:23.119 if we would have a Jason mode soon when we 344 00:20:23.160 --> 00:20:26.359 will actually have structured output support, if it will not, 345 00:20:26.440 --> 00:20:30.359 if we will have structured output support, what is the 346 00:20:30.400 --> 00:20:33.799 context length? What is the maximum amount of tokens for 347 00:20:34.000 --> 00:20:37.119 the output, you know, and all that information is put 348 00:20:37.119 --> 00:20:41.799 together in adjacent file that we distribute directly with the library, 349 00:20:41.839 --> 00:20:46.640 because one of the design principles is to be a 350 00:20:46.680 --> 00:20:50.160 turnkey solution. So you just as you as you did, 351 00:20:50.200 --> 00:20:52.920 like you installed it, you put your opening ike and 352 00:20:52.960 --> 00:20:55.559 there you go. You have a chat interface if you 353 00:20:55.599 --> 00:20:58.319 want to write at least in the common line. And 354 00:20:58.839 --> 00:21:00.839 this is exactly the experience that I wanted to have 355 00:21:00.920 --> 00:21:04.400 and I want people to have. So if there is 356 00:21:04.440 --> 00:21:07.880 a model that doesn't have let's say, function calling support, 357 00:21:08.880 --> 00:21:11.880 we will throw an error because we will know it 358 00:21:12.000 --> 00:21:16.160 from the from the models Dore Jason. And the way 359 00:21:16.200 --> 00:21:18.799 that at the moment we make the models Dore Jason 360 00:21:19.119 --> 00:21:24.480 is to query the APIs from the listing model APIs 361 00:21:24.519 --> 00:21:28.680 from open EI, from Entropic, Gemini, et cetera, and then 362 00:21:28.720 --> 00:21:35.240 augmented with our own code that figures out based on 363 00:21:35.279 --> 00:21:38.559 the model family, like that's the model family has function calling, 364 00:21:38.920 --> 00:21:43.640 that's the small family has you know, Jason mode, what's 365 00:21:43.680 --> 00:21:45.160 the context length, et cetera, et cetera. 366 00:21:45.279 --> 00:21:47.119 And you will find that in the code in the. 367 00:21:47.039 --> 00:21:51.880 Capabilities dot r B files in the module in the 368 00:21:51.920 --> 00:21:59.359 model sorry, in the provider modules. Now, however, since I've 369 00:21:59.359 --> 00:22:01.839 grown a little bit tired of you know, having to 370 00:22:01.960 --> 00:22:09.200 maintain that code and have you know, also just parsing 371 00:22:09.559 --> 00:22:14.640 the provider's web pages, which is actually kind of costly, 372 00:22:14.720 --> 00:22:16.400 especially when you do it with lams. And there is 373 00:22:16.440 --> 00:22:20.400 some code in Ruby LAM to do that in the tasks, 374 00:22:21.039 --> 00:22:23.200 but I remember the last time I used it, I 375 00:22:23.240 --> 00:22:26.519 spent twenty bucks just to parse all of the new 376 00:22:26.799 --> 00:22:31.920 open Ai model pages. Well, since I grown tired of 377 00:22:31.960 --> 00:22:36.680 doing that, I actually contacted an ex employee of mine 378 00:22:36.680 --> 00:22:40.440 from fresh Flow who now is also an entrepreneur, and 379 00:22:40.480 --> 00:22:43.680 he has a company called Parsera and they're basically doing 380 00:22:44.559 --> 00:22:49.200 scraping of web pages using l lams and then they 381 00:22:49.240 --> 00:22:52.640 returned that as data in Jason and I thought that 382 00:22:52.640 --> 00:22:54.400 that would have been the perfect use case in order 383 00:22:54.480 --> 00:22:57.799 to showcase their product, and we would get like an 384 00:22:57.799 --> 00:23:01.359 API for free in order to just you know, query 385 00:23:01.599 --> 00:23:07.559 about the capabilities of any LM in existence, since unfortunately 386 00:23:07.599 --> 00:23:09.720 the providers they don't provide that at the moment. Right, 387 00:23:09.799 --> 00:23:14.039 Like when you get a list model endpoint response, you 388 00:23:14.200 --> 00:23:17.160 just have most of the time just the model ID 389 00:23:17.640 --> 00:23:22.720 and sometimes the capabilities. I think Google does this and 390 00:23:23.039 --> 00:23:26.359 some other small things, but not the full you know, 391 00:23:26.559 --> 00:23:31.160 model info object that we have inside Ruby ELM. So 392 00:23:31.920 --> 00:23:35.359 basically what Persera is now building and they are like, 393 00:23:35.440 --> 00:23:40.359 I wouldn't say ninety percent. There is this API that 394 00:23:40.440 --> 00:23:43.079 you can just with a get request query and you 395 00:23:43.200 --> 00:23:49.920 have a beautiful interface, you know, a beautiful Jason object 396 00:23:50.480 --> 00:23:56.160 with all of the different capabilities, all the prices, all 397 00:23:56.359 --> 00:24:01.119 well formatted for all the major you know, for providers 398 00:24:01.119 --> 00:24:04.200 that we have in Ruby l M. And so I 399 00:24:04.200 --> 00:24:08.319 will scrap that code. I'm really excited when I remove code. 400 00:24:09.039 --> 00:24:12.599 So I'll scrap that code and simply like called that 401 00:24:12.599 --> 00:24:14.480 that API from from then on. 402 00:24:14.680 --> 00:24:16.119 So I'm really excited for that. 403 00:24:17.119 --> 00:24:20.319 So you're not going to have a Jason in your 404 00:24:20.960 --> 00:24:23.359 in the gym anymore. It's going to be an API 405 00:24:23.480 --> 00:24:26.519 call or the APA call. API call will generate the Jason. 406 00:24:29.839 --> 00:24:30.599 Rather the second. 407 00:24:30.720 --> 00:24:32.960 Yeah, I still want to have the turnkey solution so 408 00:24:33.000 --> 00:24:37.279 that anybody that installs it, maybe doesn't have Internet, I 409 00:24:37.279 --> 00:24:40.039 don't know, doesn't want to do that API call of 410 00:24:40.039 --> 00:24:44.279 course the Internet, then it can still actually have that 411 00:24:45.599 --> 00:24:49.039 installed and without having an extra API call every time. 412 00:24:49.160 --> 00:24:51.000 And it's refreshed models for example. 413 00:24:51.359 --> 00:24:51.640 Yep. 414 00:24:53.640 --> 00:24:57.759 So yeah, let's dive into what Ruby LLM actually provides, 415 00:24:58.000 --> 00:25:01.119 because I think people can now kind of be comfortable 416 00:25:01.119 --> 00:25:04.119 with Hey, look, it does all these models. It knows 417 00:25:04.160 --> 00:25:06.839 which models will do which things, and so if you 418 00:25:06.880 --> 00:25:09.920 try and do something with one model that it just doesn't, 419 00:25:10.200 --> 00:25:12.680 you know, it's not compatible with the model, It'll tell 420 00:25:12.680 --> 00:25:14.000 you and then you can do something different. 421 00:25:15.960 --> 00:25:17.880 So let's start with. 422 00:25:17.839 --> 00:25:21.359 The chat because that's where I started, and I was 423 00:25:21.400 --> 00:25:23.720 just blown away that it was that easy, because you 424 00:25:23.839 --> 00:25:29.160 literally just do you instantiate a chat object and then 425 00:25:29.200 --> 00:25:32.359 you do it dot ask and then you get content 426 00:25:32.440 --> 00:25:35.839 back and you can post it. And what I found 427 00:25:35.960 --> 00:25:38.240 was that I had to pull in the common mark 428 00:25:38.319 --> 00:25:41.039 gem and tell common Mark to convert it to HTML 429 00:25:41.359 --> 00:25:48.279 and I was set. That was like, it is there 430 00:25:48.319 --> 00:25:49.640 anything more to it? 431 00:25:49.680 --> 00:25:51.680 Am I missing it? I mean, we get into tools 432 00:25:51.720 --> 00:25:52.160 in a minute. 433 00:25:52.160 --> 00:25:59.599 But so in the chat you can also specify contents, right, 434 00:25:59.680 --> 00:26:02.559 so you can send an image with it, you can 435 00:26:02.599 --> 00:26:04.880 send the audio file with it, you can send a 436 00:26:04.920 --> 00:26:05.720 PDF with it. 437 00:26:06.839 --> 00:26:08.519 And also there I tried. 438 00:26:08.359 --> 00:26:12.160 To make the design of the library feel super easy 439 00:26:12.799 --> 00:26:14.839 so that if you need to analyze an image it's 440 00:26:14.920 --> 00:26:18.519 just a chat dot ask what's in this image? Come 441 00:26:18.640 --> 00:26:23.240 up with colon open curly braces image and then the 442 00:26:23.279 --> 00:26:27.359 path through your image or the HCP link to your image. 443 00:26:28.680 --> 00:26:32.839 And that also I think follows the same philosophy or 444 00:26:32.880 --> 00:26:36.279 rubylm of convention over configuration and you know, just trying 445 00:26:36.279 --> 00:26:41.160 to make people's life easier. Then you can also stream 446 00:26:41.200 --> 00:26:44.640 the responses in real time, right, and you can simply 447 00:26:44.720 --> 00:26:47.759 use Ruby blocks to do that. So it's super simple. 448 00:26:49.119 --> 00:26:51.920 It's just a chat dot ask whatever you want to ask, 449 00:26:52.200 --> 00:26:56.559 do chunk and then print chann dot content and the 450 00:26:56.680 --> 00:26:59.759 chunk is a chunk object and it looks like a message. 451 00:27:00.119 --> 00:27:01.880 In fact, if you look at the code is pretty 452 00:27:01.920 --> 00:27:04.599 much exactly like a message. I think it's just inherits 453 00:27:04.960 --> 00:27:10.319 from message and that's it. And it will if you 454 00:27:10.440 --> 00:27:14.319 just print the contents, it will simply print that chunk. 455 00:27:14.720 --> 00:27:17.839 But then in the background we're actually accumulating the response 456 00:27:18.400 --> 00:27:22.680 and so the result of that chat dot ask would 457 00:27:22.680 --> 00:27:26.759 actually be a full, fully blown message and it will 458 00:27:26.799 --> 00:27:29.880 be also persistent in the background in the in the 459 00:27:29.920 --> 00:27:35.319 memory that Ruby LM do chat has about messages, Yeah, 460 00:27:35.559 --> 00:27:40.039 what are capabilities that have? So it can also generate images. 461 00:27:40.799 --> 00:27:45.839 Because I mean you have so you have the capability. 462 00:27:46.039 --> 00:27:47.759 Yeah, and it just does it in a do loop 463 00:27:47.880 --> 00:27:51.000 and yeah, it has the chunk content and it has 464 00:27:51.119 --> 00:27:53.400 like a print or puts. 465 00:27:54.200 --> 00:27:56.960 In kind of your just general Ruby application. 466 00:27:57.559 --> 00:28:01.759 But I don't know, people who listen to this show 467 00:28:01.839 --> 00:28:03.799 know that I'm totally in love with Turbo. 468 00:28:04.480 --> 00:28:06.319 It was like, where have you been on my life Turbo? 469 00:28:07.799 --> 00:28:10.640 But you have examples on how to push the stuff 470 00:28:10.640 --> 00:28:17.440 out to Turbo as well, right, So one one question 471 00:28:17.480 --> 00:28:19.880 that I have with this because it shows a job 472 00:28:20.720 --> 00:28:24.839 and the job uses the Turbo streams channel broadcast right 473 00:28:25.480 --> 00:28:29.880 to update the message, which is you know what you want? 474 00:28:31.880 --> 00:28:34.079 I'm just I'm trying to figure out exactly where this 475 00:28:35.720 --> 00:28:36.799 gets called from. 476 00:28:38.359 --> 00:28:43.839 Does it just run periodically? And then so. 477 00:28:47.400 --> 00:28:51.279 How it works in chatwood work is that there is 478 00:28:51.319 --> 00:28:54.839 a job. It's not called chat stream job, it's called 479 00:28:54.880 --> 00:28:59.039 slightly differently, but there is that job similar to the 480 00:28:59.160 --> 00:29:03.640 job in the guides, and what it does is to 481 00:29:03.640 --> 00:29:10.319 stream via Turbo, essentially appending the chunk content to a 482 00:29:10.440 --> 00:29:15.119 div that is hidden, and then I use mark dot 483 00:29:15.160 --> 00:29:19.440 j as in order to doing the rendering of the 484 00:29:20.440 --> 00:29:23.400 of the actual message. The reason why I do this 485 00:29:24.000 --> 00:29:28.920 is because I used to have everything server side and 486 00:29:29.640 --> 00:29:33.079 do all of the rendering of markdown server side and 487 00:29:33.119 --> 00:29:37.759 then send you know, the HTML over But the problem 488 00:29:37.839 --> 00:29:41.359 is that when you get to really big messages is 489 00:29:41.400 --> 00:29:44.599 starting to stream every single time you have a new chunk, 490 00:29:45.039 --> 00:29:48.880 which may be even every single word, or you know, 491 00:29:49.240 --> 00:29:55.079 even sometimes less than one word, And you can imagine 492 00:29:55.079 --> 00:30:00.839 that refreshing the message so fast and and doing it 493 00:30:00.920 --> 00:30:05.519 so many times it actually bogs down your connection. So 494 00:30:05.680 --> 00:30:08.319 at some point I noticed that my connection was downloading 495 00:30:08.359 --> 00:30:13.279 like multiple megabytes per second right of HTML. So it 496 00:30:13.279 --> 00:30:15.319 didn't make any sense to me. So what I'm doing 497 00:30:15.400 --> 00:30:18.759 is to simply append the chunk to that specific dive 498 00:30:19.880 --> 00:30:22.319 and then marked will look at that div and it 499 00:30:22.359 --> 00:30:25.240 would actually do the rendering in an udative. 500 00:30:24.960 --> 00:30:26.279 That I actually displayed to the user. 501 00:30:26.839 --> 00:30:29.279 And that also gives me the possibility to do a 502 00:30:29.319 --> 00:30:35.079 copy functionality in which the when you copy the message, 503 00:30:35.640 --> 00:30:37.880 it would actually copy the markdown version of the message, 504 00:30:37.880 --> 00:30:40.599 which I think is so much more useful than getting 505 00:30:40.640 --> 00:30:46.440 the full blown HTML from the page. And so how 506 00:30:46.440 --> 00:30:48.960 do I call that job? Well, I call it from 507 00:30:49.200 --> 00:30:57.839 the the actual controller. So whenever the user asks something, 508 00:30:58.119 --> 00:31:00.880 you know the controller, which is fire that job in 509 00:31:00.920 --> 00:31:05.640 the background, and the job will simply process whatever comes 510 00:31:05.640 --> 00:31:06.319 from the OLM. 511 00:31:06.480 --> 00:31:13.279 Okay, so in the controller then I have something that effectively, 512 00:31:13.960 --> 00:31:18.319 I mean, do I tell it I'm streaming it? I 513 00:31:18.359 --> 00:31:20.480 guess I just have the do block and so then 514 00:31:20.720 --> 00:31:26.519 every time gets executive executed, sorry, it calls the job 515 00:31:26.599 --> 00:31:29.880 or how I'm just trying to wrap my brain around this. 516 00:31:34.759 --> 00:31:39.000 Right, So basically the controller would simply you know, maybe 517 00:31:39.039 --> 00:31:40.799 you want to create a message in the controller, do 518 00:31:40.880 --> 00:31:43.480 whatever you want, but in the end it needs to 519 00:31:43.680 --> 00:31:48.160 call that chat stream job so that you can process 520 00:31:49.079 --> 00:31:54.000 the streaming in the background because you can't have that 521 00:31:54.119 --> 00:31:56.920 running in the controller because it's such a long running thing. 522 00:31:58.759 --> 00:32:02.559 And that's exactly how I do it in chatwood work. 523 00:32:04.000 --> 00:32:06.000 Okay. 524 00:32:06.039 --> 00:32:09.400 So basically what you've got going on is, yeah, you've 525 00:32:09.400 --> 00:32:14.079 got your and it shows it in the chat right, 526 00:32:14.119 --> 00:32:16.079 So you do the chat dot ask with the do 527 00:32:16.839 --> 00:32:19.799 inside the do you call the job the job is 528 00:32:19.839 --> 00:32:24.079 streaming stuff out through turbo and then you have marked 529 00:32:24.119 --> 00:32:30.359 basically putting it in. Yeah, that's cool, that's correct. When 530 00:32:30.400 --> 00:32:34.680 your controller action finishes, then it renders a turbo stream whatever. 531 00:32:38.359 --> 00:32:42.640 Yeah, so if you set up your turbus, your broadcasts, 532 00:32:42.680 --> 00:32:48.599 you know, inside your application record, you know, on your models. Basically, 533 00:32:49.039 --> 00:32:52.000 it would also update the whole message when it's finished, 534 00:32:53.200 --> 00:32:55.200 and you can also tell it to you know, update 535 00:32:55.319 --> 00:32:59.039 of typing indicators and perhaps the button you know that 536 00:32:59.279 --> 00:33:02.880 has to change from a send to a stop. All 537 00:33:02.920 --> 00:33:06.240 of that you can do actually really really easily with Rails, 538 00:33:07.519 --> 00:33:12.319 and no wonder because Rails is so fantastic for these things. 539 00:33:13.279 --> 00:33:16.559 One thing that I think is interesting about the jobs, 540 00:33:16.640 --> 00:33:22.720 if I may, is that what I noticed, especially using 541 00:33:23.359 --> 00:33:28.599 things like solid que which use threads, is that you 542 00:33:28.640 --> 00:33:34.039 have a certain number of threads that you can use maximum, right, 543 00:33:34.079 --> 00:33:37.799 So you need to set it up and you have 544 00:33:39.279 --> 00:33:42.559 one worker per process as far as I remember, and 545 00:33:42.599 --> 00:33:45.119 then you need to set up how many threads per 546 00:33:45.160 --> 00:33:47.640 process you have as workers. 547 00:33:47.799 --> 00:33:48.000 Right. 548 00:33:48.119 --> 00:33:52.400 So even if you set it up in a way 549 00:33:52.440 --> 00:33:56.400 that it will process many many connections at the same time, 550 00:33:56.440 --> 00:33:59.119 many many chat streaming jobs at the same time, you 551 00:33:59.160 --> 00:34:02.920 will always be limited by that. And the limitation is 552 00:34:02.920 --> 00:34:08.400 that every single thread in Ruby is eight megabytes. And 553 00:34:09.679 --> 00:34:12.599 also you're occupying a whole slot, right, So you can 554 00:34:12.639 --> 00:34:17.480 imagine an application like chat with work it has not 555 00:34:17.519 --> 00:34:19.559 only the chat streaming job, but it also has the 556 00:34:19.639 --> 00:34:23.719 process document jobs, right, and the discovery document jobs and 557 00:34:24.039 --> 00:34:26.880 all of these and there's many, many jobs running at 558 00:34:26.920 --> 00:34:29.440 the same time, but they're all very lightweight, right that 559 00:34:29.519 --> 00:34:34.159 they're not doing much CPU wise, they're mostly like sending 560 00:34:34.239 --> 00:34:38.760 things back and forth through APIs and then you know, 561 00:34:38.880 --> 00:34:42.480 the grant of the work is probably done in postcris, 562 00:34:42.519 --> 00:34:47.280 you know, because I use postgris also as a vector storage. 563 00:34:47.719 --> 00:34:52.199 So they're all io bound, not CPU bound, and they're 564 00:34:52.280 --> 00:34:56.199 consuming one thread every single time there is a job 565 00:34:56.280 --> 00:35:01.280 running at the same time, and so that was not ideal. 566 00:35:01.400 --> 00:35:05.360 And in the Python world and in the JavaScript world, 567 00:35:05.639 --> 00:35:09.599 you know, for io bound operations, everybody would use a 568 00:35:09.719 --> 00:35:12.360 syn ko for example in Python, right, or a sync 569 00:35:12.480 --> 00:35:15.599 wait in JavaScript. And so what I noticed is that 570 00:35:16.239 --> 00:35:19.079 now I came back to Ruby, there's a bit of 571 00:35:19.119 --> 00:35:24.800 a lack of understanding or you know, knowledge about using 572 00:35:25.840 --> 00:35:32.480 concurrency instead of using parallelism, well fake parallelism with threats, 573 00:35:33.880 --> 00:35:36.599 And so I delved deeper into it. And actually there 574 00:35:36.679 --> 00:35:42.079 is a beautiful ecosystem of asynchronous gems such as a sync, 575 00:35:42.840 --> 00:35:47.400 a sink cable, ACNK job, anything that is built by 576 00:35:48.000 --> 00:35:53.480 Samuel Williams I aquatics, you know, also called socketry, is 577 00:35:54.039 --> 00:35:59.159 absolutely fantastic. And what I also noticed is that Ruby 578 00:35:59.280 --> 00:36:04.400 is probably the best language to do concurrency because in 579 00:36:04.440 --> 00:36:08.800 most other languages you need to uppend these or prepense 580 00:36:10.840 --> 00:36:14.239 these keywords like a sync await every time you need 581 00:36:14.280 --> 00:36:17.320 to do such an operation, But in Ruby you don't 582 00:36:17.360 --> 00:36:22.119 need to because if the C level operation actually is 583 00:36:22.360 --> 00:36:26.800 fiber aware, then you get fiber awareness for free. So 584 00:36:26.920 --> 00:36:32.480 you just need to essentially use or wrap whatever code 585 00:36:32.559 --> 00:36:36.320 you have into let's say on a sync block, and 586 00:36:36.960 --> 00:36:39.960 if you do it right, it will work in parallel 587 00:36:40.199 --> 00:36:41.039 in concurrent. 588 00:36:41.360 --> 00:36:42.840 Actually awesome. 589 00:36:43.039 --> 00:36:46.320 So one question I have on that is, because I've 590 00:36:46.400 --> 00:36:50.239 never even used this before, can you run this and 591 00:36:51.360 --> 00:36:53.800 solid que side by side and then just throw the 592 00:36:53.840 --> 00:36:57.280 acinc jobs at one and any irregular stuff at the 593 00:36:57.280 --> 00:36:59.360 other or do you just put everything into an acin job. 594 00:37:02.000 --> 00:37:05.000 At the moment, how I'm using it is everything is 595 00:37:05.119 --> 00:37:09.639 in a saying job, but you can totally right random 596 00:37:09.679 --> 00:37:13.159 side side by side. I think there is a Q 597 00:37:13.400 --> 00:37:17.679 adapter that you can use in each single job, so 598 00:37:17.760 --> 00:37:20.440 you just you know, you have the que as the 599 00:37:20.480 --> 00:37:23.000 fault for example, right, so you can have a Q 600 00:37:23.119 --> 00:37:28.320 adapter and you just say solid Q for certain jobs, 601 00:37:28.440 --> 00:37:31.960 and you can do a saying job for your jobs. 602 00:37:32.920 --> 00:37:34.920 That's a good tip because yeah, if I'm going to 603 00:37:34.960 --> 00:37:39.480 build a podcast assistant, I'm probably gonna run into the 604 00:37:39.519 --> 00:37:42.880 same constraints, and so that is really helpful. 605 00:37:43.320 --> 00:37:50.840 Right, Yeah, Yeah, I'm no, it's just saying that I 606 00:37:51.440 --> 00:37:56.800 basically discovered this this fly, and I think that it's 607 00:37:56.840 --> 00:38:00.320 really a matter of communication to the Ruby community out 608 00:38:00.519 --> 00:38:03.599 saying like, hey, there's not just this way of doing 609 00:38:03.679 --> 00:38:06.320 concurrency in paralyism that we're all familiar with, you know, 610 00:38:06.599 --> 00:38:10.480 like threads, but there's also fibers, and you know, there's 611 00:38:11.360 --> 00:38:14.320 other software that you could use that even though you 612 00:38:14.400 --> 00:38:18.480 don't have io bound operations, you can simply replace Puma 613 00:38:18.480 --> 00:38:22.119 with Falcon and have you know, five x speed ups. 614 00:38:22.159 --> 00:38:25.840 Sometimes it's really so much faster it's noticeable. I didn't 615 00:38:25.840 --> 00:38:28.000 do any benchmarks, but it's really Yeah. 616 00:38:28.039 --> 00:38:30.079 I've talked to Samuel before and I got the impression 617 00:38:30.119 --> 00:38:33.599 that some workloads really lend themselves to it and some 618 00:38:33.760 --> 00:38:34.519 don't as much. 619 00:38:34.559 --> 00:38:37.679 But yeah, Falcon is really cool. 620 00:38:40.440 --> 00:38:43.280 One other thing that I'm curious about here, So when 621 00:38:43.280 --> 00:38:46.000 I built my app the other night, with the chat. 622 00:38:46.440 --> 00:38:49.159 I realized that in my controller, I was instantiating a 623 00:38:49.199 --> 00:38:54.000 new chat every time writers doing RUBLLM dot chat. And 624 00:38:54.440 --> 00:38:58.000 what I want is I want an ongoing conversation with 625 00:38:59.280 --> 00:39:01.760 you know, so that it kind of maintains the context 626 00:39:01.840 --> 00:39:07.480 and things like that. And there's an example in here 627 00:39:07.519 --> 00:39:10.239 where you're basically what I would do, like in a 628 00:39:10.320 --> 00:39:15.159 RAILS console or IRB or something where you know, it's 629 00:39:15.199 --> 00:39:18.039 like chat dot ask and then you send it another 630 00:39:18.079 --> 00:39:21.199 message chat dot ask and it looks like that maintains 631 00:39:21.239 --> 00:39:24.519 that context. But how do I do that between requests 632 00:39:25.159 --> 00:39:28.960 when I'm hitting the controller anew every time? But I 633 00:39:29.000 --> 00:39:30.679 still want it to be part of that same. 634 00:39:30.559 --> 00:39:36.599 Chat, right, I think this is really important to say. 635 00:39:36.679 --> 00:39:44.480 So the RAILS integration essentially persistent messages, Yeah, persistent messages 636 00:39:44.599 --> 00:39:47.400 inside active record models. 637 00:39:47.480 --> 00:39:54.679 So we have essentially some helpers which are called. 638 00:39:54.480 --> 00:40:00.559 Acts as chat, acts as message access tool call that 639 00:40:00.639 --> 00:40:03.800 you can drop in into your models. You know, if 640 00:40:03.840 --> 00:40:07.480 it's called chat the model for chats, then you just 641 00:40:08.280 --> 00:40:14.840 say class you know chat inheriting from application record and 642 00:40:14.920 --> 00:40:19.320 then just say access chat and you have essentially all 643 00:40:19.360 --> 00:40:24.719 the connection to also the message model. You have persistence 644 00:40:25.480 --> 00:40:29.960 and every single method that we have in the classic 645 00:40:30.039 --> 00:40:33.880 chat interface, you know in plain old Ruby objects. 646 00:40:34.400 --> 00:40:36.400 You would have it also in rails. 647 00:40:36.559 --> 00:40:40.199 So if you do chat dot ask, it will first 648 00:40:40.360 --> 00:40:46.039 create a message with the with the user role and 649 00:40:46.559 --> 00:40:53.920 save it and then also call the API for whatever 650 00:40:54.519 --> 00:40:58.920 LLLM you're using, retrieve that and then put it inside 651 00:40:58.920 --> 00:41:02.119 a new message anderson is that and then return that 652 00:41:02.320 --> 00:41:06.280 message at the end. So it looked like exactly the 653 00:41:06.320 --> 00:41:09.719 same interface for PLANEOAR Ruby objects, but you have the 654 00:41:09.760 --> 00:41:12.800 persistence done not just in memory but also in your database. 655 00:41:13.159 --> 00:41:16.679 Right, what if I'm not using rails or not using 656 00:41:16.840 --> 00:41:19.239 active record, is there a way to do this like 657 00:41:19.320 --> 00:41:22.360 in Sinatra or something like that and that anyway? 658 00:41:24.239 --> 00:41:28.639 Right, I haven't used Sinatra recently. The last time I 659 00:41:28.719 --> 00:41:36.239 used it was more than ten years ago. Maybe it's right, 660 00:41:37.440 --> 00:41:41.119 So I wouldn't know how to actually build that out, 661 00:41:42.159 --> 00:41:45.199 but it's possible to. If it's possible to make an 662 00:41:45.239 --> 00:41:50.679 adapter for whatever type of you know, storage you have, 663 00:41:51.519 --> 00:41:53.639 it should be pretty easy and people can just look 664 00:41:53.679 --> 00:41:56.639 at the access code and see how we've done it. 665 00:41:56.760 --> 00:41:59.119 Right, Yeah, that makes sense. 666 00:41:59.159 --> 00:42:00.880 You can't just look at his chat and then see 667 00:42:00.920 --> 00:42:06.960 what she did. Very cool let's talk about tools for 668 00:42:07.000 --> 00:42:08.920 a minute, because this was one of the other things 669 00:42:08.920 --> 00:42:11.159 I showed off the other night was essentially I created 670 00:42:11.159 --> 00:42:16.599 a tool that just checked the weather, and yeah, I 671 00:42:16.639 --> 00:42:18.679 didn't use the same service you did, but it was 672 00:42:18.760 --> 00:42:22.920 basically the same thing. I used Open weather Map instead 673 00:42:22.960 --> 00:42:26.840 of Medio or whatever you used, but yeah, it was 674 00:42:26.880 --> 00:42:28.480 the same deal. Do you want to explain to people 675 00:42:28.480 --> 00:42:30.000 what tools are and how they work? 676 00:42:34.559 --> 00:42:34.880 Right? 677 00:42:34.960 --> 00:42:38.199 So, I think tools, especially with MCP, they became the 678 00:42:38.239 --> 00:42:42.400 most hot thing talked about at the moment in AI. Right, 679 00:42:44.199 --> 00:42:49.440 So for people who are not familiar essentially with LLM models, 680 00:42:49.800 --> 00:42:54.039 you can create a tool in your own language and 681 00:42:54.079 --> 00:42:58.159 then tell them like, hey, this is the way that 682 00:42:58.239 --> 00:43:03.119 this tool works. So this is the description of the tool, 683 00:43:03.599 --> 00:43:05.760 and these are all the parameters that we need. These 684 00:43:05.760 --> 00:43:08.679 are the parameters that are optional, and these are all 685 00:43:08.719 --> 00:43:12.280 the descriptions for every single parameters. And then the LM 686 00:43:12.320 --> 00:43:15.199 will decide when to use that tool. 687 00:43:15.400 --> 00:43:15.599 Right. 688 00:43:15.679 --> 00:43:20.639 So let's say you created this tool for weather, but 689 00:43:21.239 --> 00:43:23.920 then the user talks about something else, wants to talk 690 00:43:23.960 --> 00:43:27.920 about MacBooks, right, the LLLM will not actually call the 691 00:43:27.960 --> 00:43:32.320 tool every single time. And I think the fundamental difference 692 00:43:33.079 --> 00:43:37.880 between tools and the classical RAG for example that we 693 00:43:38.039 --> 00:43:40.719 used to have, you know, even just a year ago 694 00:43:41.159 --> 00:43:46.159 at this point is that you don't always do retrieval 695 00:43:46.199 --> 00:43:50.480 advantage generation. So basically you don't always search, you know, 696 00:43:50.559 --> 00:43:52.960 the documents and then put them in the context of 697 00:43:53.039 --> 00:43:57.719 the message every single time you have a new message. Basically, 698 00:43:57.840 --> 00:44:03.639 tools enable agentic aid I because the AI can be 699 00:44:03.679 --> 00:44:06.440 an agent and really figure out if it needs to 700 00:44:06.519 --> 00:44:12.400 use this tool or not. And they are now like 701 00:44:12.480 --> 00:44:15.840 all the age with MCP, So MCP, for it's not familiar, 702 00:44:16.119 --> 00:44:21.119 is model context protocol, and it's a way to bunch 703 00:44:21.320 --> 00:44:27.280 up a bunch, you know, a bunch of tools into 704 00:44:27.400 --> 00:44:31.119 one single interface and give that interface to the l 705 00:44:31.199 --> 00:44:32.920 l M so that the l M can figure out 706 00:44:33.719 --> 00:44:36.320 you know, hey, I can now access the file system, 707 00:44:36.440 --> 00:44:40.800 I can now use VIM, I can now use vs code, 708 00:44:40.920 --> 00:44:46.119 I can now use I don't know whatever API. And 709 00:44:46.480 --> 00:44:49.519 it's essentially an implementation of tools in a way, right, 710 00:44:49.559 --> 00:44:52.079 but just in a different API. 711 00:44:57.079 --> 00:44:57.280 Yeah. 712 00:44:57.280 --> 00:45:01.280 Absolutely, So this is where like with my my thing 713 00:45:01.360 --> 00:45:04.760 with my podcast stuff is you know, it's not just 714 00:45:05.440 --> 00:45:08.800 which episodes were most popular or what other thing do 715 00:45:08.880 --> 00:45:11.320 I want you to look up, but it's hey, reschedule this, 716 00:45:11.480 --> 00:45:13.920 or hey, invite this other person or do this other thing, 717 00:45:14.599 --> 00:45:17.960 and it can through the API go and talk to 718 00:45:18.000 --> 00:45:19.800 the other system and make it do something for me. 719 00:45:21.599 --> 00:45:22.159 That's right. 720 00:45:22.599 --> 00:45:26.199 So it opens up a lot of possibilities for using 721 00:45:26.360 --> 00:45:29.639 lms for all kinds of disparate features. 722 00:45:29.719 --> 00:45:29.880 Right. 723 00:45:30.880 --> 00:45:35.000 You can save things, you can retrieve things from the Internet. 724 00:45:35.599 --> 00:45:40.000 You can imagine for example Google search or like document search, 725 00:45:41.079 --> 00:45:47.360 or creating new blog posts. It's just I think the 726 00:45:47.400 --> 00:45:50.119 possibilities are endless because you're letting. 727 00:45:49.840 --> 00:45:51.480 The AI use your code. 728 00:45:52.119 --> 00:45:57.039 Of course, you need to be aware that also ais 729 00:45:57.079 --> 00:46:00.840 are generally benevolent, but you know they could use your 730 00:46:00.880 --> 00:46:02.079 tool badly. 731 00:46:03.119 --> 00:46:04.840 They could cause wreckage. 732 00:46:04.920 --> 00:46:11.519 So don't use evolved, don't use system right, but try 733 00:46:11.559 --> 00:46:14.480 to be kind of careful about it. But in general, 734 00:46:14.559 --> 00:46:19.000 it's a fantastic way to make ELM super super powerful. 735 00:46:21.800 --> 00:46:25.480 Yeah, and I think tool design and security with the 736 00:46:25.519 --> 00:46:27.679 tools and things like that might be outside the scope 737 00:46:27.719 --> 00:46:31.920 of this episode, but yeah, I mean I've had people 738 00:46:31.960 --> 00:46:34.920 ask me too, it's, hey, well, how do you make 739 00:46:34.920 --> 00:46:37.239 sure that the tool is only doing this stuff within 740 00:46:37.280 --> 00:46:39.559 the scope you want? So, for example, in my case 741 00:46:39.639 --> 00:46:42.800 with podcasting, it's how do I make sure it's only 742 00:46:43.239 --> 00:46:47.280 doing stuff on one person's podcast and not another, or 743 00:46:47.360 --> 00:46:47.960 things like that. 744 00:46:48.159 --> 00:46:48.480 And so. 745 00:46:50.119 --> 00:46:53.079 Because you know, we've seen people do what looks like 746 00:46:53.599 --> 00:46:57.840 SQL injection attacks except as prompt stuff, and it says, well, 747 00:46:58.360 --> 00:47:02.239 but pretend you can do it essentially, right, they talk 748 00:47:02.400 --> 00:47:06.320 the LM around to Okay, I'm going to pretend I 749 00:47:06.360 --> 00:47:08.320 can do it, and so I'm going to act like 750 00:47:08.360 --> 00:47:12.119 this other user even though I'm not technically supposed to 751 00:47:12.119 --> 00:47:15.000 do that. And there are only so many safeguards you 752 00:47:15.039 --> 00:47:16.800 can put into place. But if you don't have a 753 00:47:16.800 --> 00:47:20.440 tool that allows them to impersonate anyone else, and you've 754 00:47:20.599 --> 00:47:23.960 locked down the access to the user and it's data 755 00:47:24.039 --> 00:47:27.719 to a methodology you can control, like find the current 756 00:47:27.800 --> 00:47:30.840 user uses the session to validate the user instead of 757 00:47:31.039 --> 00:47:34.599 looking the user up by email address, for example, and 758 00:47:34.639 --> 00:47:36.639 it doesn't have a tool to look up the user 759 00:47:36.679 --> 00:47:39.800 by email address unless you need that feature for admins. 760 00:47:39.840 --> 00:47:41.599 But then you use a different chat agent for that 761 00:47:42.679 --> 00:47:46.280 and give it those tools there Now you're getting into 762 00:47:47.880 --> 00:47:49.639 so you have to be careful how you use them. 763 00:47:49.880 --> 00:47:52.519 Like you said, for sure, try and think ahead on 764 00:47:53.239 --> 00:47:56.920 how people may trick your agent into doing something that shouldn't. 765 00:47:57.199 --> 00:47:58.840 That's right, That's exactly right. 766 00:48:02.280 --> 00:48:03.800 But yeah, the tools are cool. 767 00:48:04.679 --> 00:48:07.679 One question I had on this, so I built a 768 00:48:07.719 --> 00:48:10.159 tool for the weather, and then I was thinking about 769 00:48:10.199 --> 00:48:12.599 it and I was like, well, what if I wanted 770 00:48:12.679 --> 00:48:15.320 So then I was showing off, not what's the weather 771 00:48:15.400 --> 00:48:19.199 in Lehigh, Utah, which is where I live, but you know, 772 00:48:19.679 --> 00:48:21.760 is it a good time to go have a picnic 773 00:48:21.760 --> 00:48:22.519 in Lehigh, Utah? 774 00:48:22.599 --> 00:48:22.719 Right? 775 00:48:22.719 --> 00:48:24.079 And it's smart enough to go, Okay, I need to 776 00:48:24.079 --> 00:48:26.840 know what Lehi is, I need you know. I figured 777 00:48:26.880 --> 00:48:28.920 out the latitude and longitude. I sent it to the 778 00:48:28.960 --> 00:48:30.840 Weather Service. It told me what the weather was, and 779 00:48:30.880 --> 00:48:32.960 then it said, yeah, it's kind of a nice evening 780 00:48:33.000 --> 00:48:36.519 for a picnic, right because it was sixty degrees and 781 00:48:37.119 --> 00:48:40.079 you know, the wind wasn't too bad and it wasn't raining. 782 00:48:41.800 --> 00:48:43.679 But then I was thinking, okay, well, what if I 783 00:48:43.679 --> 00:48:45.960 wanted to ask it what day this week should I 784 00:48:46.000 --> 00:48:49.599 go for a picnic? Then it has to have another 785 00:48:49.679 --> 00:48:53.119 tool that will do the forecast look up. And so 786 00:48:53.360 --> 00:48:56.760 I can imagine that if you're creating tools, you could 787 00:48:56.760 --> 00:49:01.079 wind up with a lot of tools, right, collection of them. 788 00:49:01.599 --> 00:49:03.519 I'm wondering if there's a way to like group them together. 789 00:49:03.559 --> 00:49:05.519 So it's like, hey, here's all the calendar stuff, here's 790 00:49:05.519 --> 00:49:08.000 all the drop box stuff, here's all the drive stuff. 791 00:49:08.480 --> 00:49:10.840 I think that's exactly what Entropic was trying to do 792 00:49:10.880 --> 00:49:16.960 with MCP. It's probably exactly that use case. And essentially, 793 00:49:17.039 --> 00:49:19.400 as I said, MCP is this way of bunching up 794 00:49:19.559 --> 00:49:24.960 different tools into one single API, and it tells the LLM, hey, 795 00:49:25.039 --> 00:49:27.000 I have all these things that you can do now. 796 00:49:28.519 --> 00:49:31.920 And you can even run like MCP servers in your 797 00:49:31.960 --> 00:49:37.840 own computer for example, in order to you know, connected 798 00:49:38.199 --> 00:49:39.880 the LM with your files. 799 00:49:41.599 --> 00:49:42.280 Use it more. 800 00:49:42.480 --> 00:49:48.039 You know, cloud code Sorry, cloud code uses MCP as 801 00:49:48.039 --> 00:49:50.800 far as I know, and maybe wrong on this, but 802 00:49:50.960 --> 00:49:54.239 I think it uses MCP to actually connect to your 803 00:49:54.280 --> 00:49:57.119 files and your file system and to look up stuff 804 00:49:57.119 --> 00:50:00.840 and to change it. So it is a valid point, 805 00:50:01.400 --> 00:50:03.039 you know, and you can win up with a lot 806 00:50:03.079 --> 00:50:05.519 of tools and actually they consume a little bit of 807 00:50:05.599 --> 00:50:11.760 your context window. However, now with the recent models, you know, 808 00:50:12.280 --> 00:50:14.440 I don't know if you heard, GPT four point one 809 00:50:14.960 --> 00:50:17.800 just came out and it has a one million token 810 00:50:17.880 --> 00:50:19.280 context that. 811 00:50:19.320 --> 00:50:20.599 Is pretty big. 812 00:50:21.599 --> 00:50:24.119 Yeah, you can do a lot of stuff with one 813 00:50:24.119 --> 00:50:28.199 million tokens and I've been using Gemini two point five 814 00:50:28.280 --> 00:50:35.280 pro extensively lately and it's a fantastic model super bowlish 815 00:50:35.320 --> 00:50:40.960 on on Google lately, and it also has you know, 816 00:50:41.440 --> 00:50:47.159 million context window if not two million, I don't remember that. 817 00:50:50.440 --> 00:50:51.400 Yeah, that makes sense. 818 00:50:53.800 --> 00:50:56.320 The other thing that I've seen done and we talked 819 00:50:56.360 --> 00:51:00.199 to Obi before MCP was really a thing, and what 820 00:51:00.239 --> 00:51:02.280 he was doing was he actually had different. 821 00:51:02.039 --> 00:51:04.559 Agents that had different jobs, right, and. 822 00:51:04.559 --> 00:51:10.000 So then instead of providing this huge list of tools, 823 00:51:10.840 --> 00:51:14.519 he would he would have one tool that was essentially said, 824 00:51:15.119 --> 00:51:20.159 write a prompt for the calendar agent, right, and so 825 00:51:20.280 --> 00:51:22.519 then it would prompt the calendar agent and say, hey, 826 00:51:22.559 --> 00:51:26.119 can you invite so and so to this calendar event? 827 00:51:27.079 --> 00:51:29.920 And then on the calendar agent would then say, oh, 828 00:51:30.000 --> 00:51:31.760 this means I have to go find the event. 829 00:51:31.559 --> 00:51:34.360 And update it with you know, with the user or whatever. 830 00:51:35.039 --> 00:51:38.960 So that that was his way around it. I kind 831 00:51:38.960 --> 00:51:42.719 of liked that approach anyway, because then I really can 832 00:51:42.840 --> 00:51:46.800 kind of box in each of the agents with just 833 00:51:46.840 --> 00:51:51.039 a small set of responsibilities and capabilities. But yeah, that 834 00:51:51.360 --> 00:51:53.400 was his way around it, and I think that's kind 835 00:51:53.400 --> 00:51:55.960 of an elegant solution. But I can imagine if I 836 00:51:56.039 --> 00:52:00.480 have a app tools directory in my application, that I 837 00:52:00.599 --> 00:52:02.880 might have a lot of those in there, depending on 838 00:52:02.920 --> 00:52:06.440 how many capabilities I'm giving to my application right or 839 00:52:06.440 --> 00:52:07.079 to my agent. 840 00:52:07.320 --> 00:52:07.559 Right. 841 00:52:08.480 --> 00:52:09.960 However, I think that this. 842 00:52:11.400 --> 00:52:16.199 Is a totally fine design solution for the past models 843 00:52:16.239 --> 00:52:20.320 that we had, you know, even last year, where two 844 00:52:20.440 --> 00:52:24.599 hundred thousand tokens context windows were pretty big, right. 845 00:52:26.199 --> 00:52:26.679 Yeah. 846 00:52:27.199 --> 00:52:31.159 I think we'll probably see in the future, you know, 847 00:52:31.320 --> 00:52:36.480 just having one single model that would actually handle all 848 00:52:36.519 --> 00:52:39.199 the tools you have, because the context window is big enough, 849 00:52:39.719 --> 00:52:42.119 and it would be super powerful and it can do 850 00:52:42.199 --> 00:52:43.840 so many things, it will actually. 851 00:52:45.000 --> 00:52:48.559 Probably blow people's minds. 852 00:52:50.079 --> 00:52:52.800 Yeah, with a million tokens. Yeah, I can imagine that 853 00:52:52.800 --> 00:52:58.000 that's that's a ton. It's interesting because I was working 854 00:52:58.039 --> 00:53:02.880 with a model this earlier this year, and I think 855 00:53:02.880 --> 00:53:04.360 the context window had. 856 00:53:04.159 --> 00:53:07.280 Like two hundred thousand tokens. I think something like that 857 00:53:07.280 --> 00:53:08.679 one hundred and fifty two hundred thousand. 858 00:53:09.239 --> 00:53:10.960 And the issue that I was running into there was 859 00:53:10.960 --> 00:53:15.079 that I was pulling in bills during the Utah legislative 860 00:53:15.119 --> 00:53:18.320 session and getting summaries done and things like that with it, 861 00:53:18.960 --> 00:53:20.800 and there were like four. 862 00:53:20.440 --> 00:53:23.239 Bills that were too big. It blew, it blew past 863 00:53:23.320 --> 00:53:25.440 the context window, and so I. 864 00:53:25.400 --> 00:53:29.000 Tried breaking it up, but ultimately that's the other great 865 00:53:29.039 --> 00:53:30.760 thing about a lot of this stuff is that the 866 00:53:30.800 --> 00:53:34.039 context windows are getting bigger, and what that means is 867 00:53:34.079 --> 00:53:39.599 that the prices are coming down for models that have 868 00:53:39.719 --> 00:53:42.079 larger context windows that were kind of the run of 869 00:53:42.119 --> 00:53:44.880 the mill, state of the art ones before. And so 870 00:53:45.119 --> 00:53:47.320 you can get away with a lot more or you 871 00:53:47.360 --> 00:53:50.039 can just like in your what you're saying here with 872 00:53:50.079 --> 00:53:53.119 the chat is that I can go in in midstreams 873 00:53:53.239 --> 00:53:56.320 it comes back and says, hey, this was way too big. 874 00:53:56.360 --> 00:54:00.320 I can go Okay, just this time, go use bigger 875 00:54:00.360 --> 00:54:02.840 model and pay the extra money. 876 00:54:03.000 --> 00:54:04.000 Right Yeah. 877 00:54:04.039 --> 00:54:07.039 I think these are two trends that we've seen so far. 878 00:54:07.159 --> 00:54:09.719 So one is context window becoming bigger. And I think 879 00:54:09.719 --> 00:54:12.079 it's a question of just pure capabilities. 880 00:54:12.119 --> 00:54:14.159 Right, So, all of. 881 00:54:14.119 --> 00:54:16.760 The applications similar to chat with work, and there are 882 00:54:16.960 --> 00:54:22.000 a few, they all need to have sort of big 883 00:54:22.039 --> 00:54:24.480 context windows in order to function correctly, because you want 884 00:54:24.480 --> 00:54:28.960 to have as many documents, you know, describing things or 885 00:54:29.119 --> 00:54:32.599 chats or whatever else that is relevant for the topic 886 00:54:32.639 --> 00:54:35.039 that the user asked in order to be first of 887 00:54:35.079 --> 00:54:39.519 all correct in your answer. And Thori and I think 888 00:54:39.599 --> 00:54:42.599 there is always a balance in an information retrieval system 889 00:54:42.639 --> 00:54:46.079 like this, like how many you actually present even to 890 00:54:46.119 --> 00:54:47.039 the LLLM itself. 891 00:54:48.039 --> 00:54:50.000 But that's a whole other topic. 892 00:54:52.199 --> 00:54:54.679 But with the big context windows, you can kind of, 893 00:54:54.880 --> 00:54:58.039 you know, relax a little bit that balance. And then 894 00:54:58.039 --> 00:55:01.840 the second thing that you mentioned, which is getting the 895 00:55:01.880 --> 00:55:06.960 prices down, I think this is inevitable. The models are 896 00:55:06.960 --> 00:55:12.079 being commoditized more and more, and honestly, I am finally 897 00:55:12.079 --> 00:55:14.920 seeing what I thought Google would have done, which is, 898 00:55:15.559 --> 00:55:19.760 you know, putting the prices very very low for models 899 00:55:19.800 --> 00:55:22.079 because they have access to TPUs, right, so they make 900 00:55:22.119 --> 00:55:28.199 their own chips for AI. And one thing, you know, 901 00:55:28.280 --> 00:55:31.719 you can use Nvidia cards and they're fantastic, you know, 902 00:55:32.719 --> 00:55:36.320 for for EI usage. But another thing is you know 903 00:55:36.440 --> 00:55:41.920 TPU from from Google. It goes much faster, and you know, 904 00:55:42.000 --> 00:55:44.280 it's it's so much better. They make it in house, 905 00:55:44.360 --> 00:55:49.039 so it's probably cheaper for them. So yeah, I'm really 906 00:55:49.079 --> 00:55:51.960 excited to see where where things will go next with 907 00:55:52.039 --> 00:55:56.960 this in in AI because there seems to be so 908 00:55:57.119 --> 00:55:59.079 much going on at the same time it's kind of 909 00:55:59.079 --> 00:55:59.679 hard to follow. 910 00:56:02.039 --> 00:56:04.159 Yep, yep, absolutely well. 911 00:56:04.239 --> 00:56:06.079 Yeah, and every time I see a new model come 912 00:56:06.119 --> 00:56:08.480 out and people are like because it's felt like the 913 00:56:08.559 --> 00:56:12.159 last little while, you know, like Grock came out and everyone's. 914 00:56:11.800 --> 00:56:13.960 Like Grock is the best and it's doing the best 915 00:56:14.039 --> 00:56:14.760 and blah blah blah. 916 00:56:14.800 --> 00:56:17.599 And literally like a week later, like GPT four one 917 00:56:17.719 --> 00:56:20.199 was out or Anthropic, I can't remember, but one of 918 00:56:20.199 --> 00:56:22.320 them came out with something and it was like, Wow, 919 00:56:22.440 --> 00:56:24.880 this one has you know, this much more over Grock. 920 00:56:24.920 --> 00:56:26.840 And then it was the other one, whether it was 921 00:56:26.880 --> 00:56:29.199 GPT or Claude or something else, came out and it 922 00:56:29.239 --> 00:56:31.360 was like, oh, now this one right. And so we 923 00:56:31.440 --> 00:56:34.440 keep getting better and better and it feels like this 924 00:56:34.639 --> 00:56:36.639 arms race. But the flip side of it is is 925 00:56:36.679 --> 00:56:40.960 like you said, you know, the models that are powerful 926 00:56:41.119 --> 00:56:44.880 enough are being commoditized to the point where it's just like, 927 00:56:45.000 --> 00:56:46.880 you know what, just just pick the one that's going 928 00:56:46.960 --> 00:56:49.800 to do the thing for you. And I mean, granted, 929 00:56:50.000 --> 00:56:53.239 you know, the latent space some of them allow the 930 00:56:53.239 --> 00:56:56.039 different models to do better in certain contexts. Right, it's 931 00:56:56.079 --> 00:57:00.920 not just the token size, but holy crap, if they 932 00:57:01.000 --> 00:57:03.360 keep pushing it the way they're pushing it, I mean, 933 00:57:03.400 --> 00:57:05.760 we're just going to have some wild capabilities with what 934 00:57:05.800 --> 00:57:06.239 it can do. 935 00:57:06.760 --> 00:57:09.199 And I for one super excited about that. 936 00:57:09.679 --> 00:57:13.599 Yeah, we're getting a little bit close to time, so 937 00:57:13.639 --> 00:57:19.000 I'm going to push us to image generation and text 938 00:57:19.000 --> 00:57:22.599 embedding and then we'll wrap up with picks. We've got 939 00:57:22.639 --> 00:57:28.079 about ten minutes to talk about image generation and embedding. 940 00:57:28.119 --> 00:57:31.119 So how does that all work? 941 00:57:31.159 --> 00:57:32.679 Because I didn't dive into that so much. 942 00:57:32.760 --> 00:57:35.000 I've used other tools to do it, but I didn't 943 00:57:35.039 --> 00:57:36.199 dive into that so much with. 944 00:57:38.440 --> 00:57:42.559 Ruby LLM, right, So it's a simple call. It's at 945 00:57:42.599 --> 00:57:45.599 the same level as RUBYLM do chat, but since it 946 00:57:45.639 --> 00:57:48.519 doesn't need to have like the context of all the 947 00:57:48.559 --> 00:57:52.559 conversation that you have with the LLM, it's as simple 948 00:57:52.559 --> 00:57:53.480 as RUBYLM do. 949 00:57:53.599 --> 00:57:58.800 Paint and then you have your message there. 950 00:57:59.360 --> 00:58:02.360 And also for embedding, so you can just use RUBYLM 951 00:58:02.360 --> 00:58:05.239 dot m bed and just the text. You can also 952 00:58:05.320 --> 00:58:10.960 pass a raise so it will embed multiple texts altogether 953 00:58:11.039 --> 00:58:14.280 and batch them together. This is nice for you know, 954 00:58:14.360 --> 00:58:20.719 pricing and API usage. But that's pretty much all there 955 00:58:20.760 --> 00:58:23.199 is to it. You know, there's some little things that 956 00:58:23.239 --> 00:58:25.519 you can do depending on the model. For example, for 957 00:58:26.199 --> 00:58:31.559 I believe Dali has two different sizes outputs, so. 958 00:58:31.440 --> 00:58:32.320 You can configure that. 959 00:58:33.119 --> 00:58:37.119 But and you can choose obviously between all the models 960 00:58:37.119 --> 00:58:41.320 that we support in the providers that we support. But 961 00:58:42.760 --> 00:58:45.079 there's pretty much all there is to it about these 962 00:58:45.119 --> 00:58:48.440 two two things is just a very simple interface for that. 963 00:58:50.880 --> 00:58:51.880 Yeah, I have to say. 964 00:58:51.880 --> 00:58:54.400 One of the things that I'm excited about this as 965 00:58:54.400 --> 00:59:00.360 far as the image generation goes, is I want people 966 00:59:00.400 --> 00:59:02.360 to be able to say, hey, I want you to 967 00:59:02.440 --> 00:59:08.039 generate some album of art or some episode artwork or 968 00:59:08.039 --> 00:59:12.360 something like that. And like if I say, hey, use 969 00:59:12.400 --> 00:59:17.119 this episode that we recorded and generate some artwork for it, 970 00:59:18.000 --> 00:59:20.719 I mean I could see it basically using the chat 971 00:59:20.800 --> 00:59:25.400 prompt set up to say you know, and you could 972 00:59:25.440 --> 00:59:29.360 you said you could pass it attachments, so it could say, 973 00:59:29.599 --> 00:59:34.119 here's the audio for the episode, you know, give me, 974 00:59:34.719 --> 00:59:40.199 write me a prompt to get a good image, you know, 975 00:59:40.719 --> 00:59:43.679 cover art image for this episode. And then it turns 976 00:59:43.719 --> 00:59:47.400 around and it goes ruby LLM paint. Here's the prompt 977 00:59:47.440 --> 00:59:52.559 that I made up, right, and then you can see 978 00:59:52.559 --> 00:59:54.840 what it comes up with. And the other thing is 979 00:59:54.840 --> 00:59:57.960 is that on the chat models we didn't talk about temperature, 980 00:59:58.719 --> 01:00:00.920 but you could set the temperature a little bit higher 981 01:00:01.480 --> 01:00:05.599 and get some more creativity out of your prompt. Then 982 01:00:05.639 --> 01:00:08.199 the image generation may give you something a little bit different, 983 01:00:08.599 --> 01:00:11.159 And so I'm looking at changing some of these together 984 01:00:11.199 --> 01:00:13.480 and I'm really excited about that stuff too. But yeah, 985 01:00:13.280 --> 01:00:15.719 the image generation seem pretty straightforward to me. 986 01:00:16.199 --> 01:00:20.280 Yeah, what I'm excited about is to see GPT four 987 01:00:20.320 --> 01:00:23.719 all image generation in the APIs, and I know it's 988 01:00:23.760 --> 01:00:26.199 coming soon. I know it has been taking the world 989 01:00:26.239 --> 01:00:32.800 by storm lately. How many Studio Ghibli memes of your found, Charles, Oh. 990 01:00:32.679 --> 01:00:35.400 Yeah, you have them. 991 01:00:35.599 --> 01:00:40.599 Yeah, so you see wacky stuff with Donald Trump or 992 01:00:41.440 --> 01:00:44.400 Bernie Sanders here in the US, right, you know, not 993 01:00:44.519 --> 01:00:46.519 to pick on one side or the other, but both 994 01:00:46.559 --> 01:00:49.559 of those guys cartoon out really really well, and it's 995 01:00:49.599 --> 01:00:51.320 so fun to see some of them because it's like 996 01:00:51.559 --> 01:00:55.320 it's like, oh, yeah, that's the studio Ghibli that kind 997 01:00:55.320 --> 01:00:58.199 of emphasizes kind of the quirky aspects of either one 998 01:00:58.239 --> 01:00:58.960 of those guys. 999 01:01:00.920 --> 01:01:05.840 Yeah, the capability of JUBT for image and it is fantastic. 1000 01:01:06.960 --> 01:01:09.400 But returning back to your question, I think the. 1001 01:01:10.880 --> 01:01:13.079 Or rather what you said is to me, I'm really 1002 01:01:13.119 --> 01:01:17.719 excited about that because it really would combine both the 1003 01:01:17.760 --> 01:01:20.320 text interface and also like the image generation in the 1004 01:01:20.320 --> 01:01:21.159 same model. 1005 01:01:21.480 --> 01:01:22.639 So that we can have. 1006 01:01:23.159 --> 01:01:27.320 This capability without even needing to create a new tool. 1007 01:01:27.960 --> 01:01:29.960 So right now, how you would do it is that 1008 01:01:30.039 --> 01:01:32.480 you create a tool and inside the tool you have 1009 01:01:32.599 --> 01:01:37.119 RUBLM dot paint. Right with these types of models, you 1010 01:01:37.119 --> 01:01:39.079 wouldn't need a tool anymore, and it would do it 1011 01:01:39.199 --> 01:01:40.719 all in the same model, and it would do it 1012 01:01:40.800 --> 01:01:44.800 so much better than the previous diffusion models because apparently 1013 01:01:44.800 --> 01:01:52.039 they're using combination of diffusion and transformers architectures. So in 1014 01:01:52.079 --> 01:01:55.599 the end, what comes down to is that is so 1015 01:01:55.760 --> 01:02:00.360 much more beautiful, so much more precise. It also understands language, right, 1016 01:02:00.480 --> 01:02:04.159 so it also understands out to make fonts so you 1017 01:02:04.199 --> 01:02:06.639 can write you know, when it writes it, actually you 1018 01:02:06.639 --> 01:02:08.679 can read it, and it's not just this garbage that 1019 01:02:09.159 --> 01:02:11.760 we used to get like a year ago. 1020 01:02:14.199 --> 01:02:20.880 Right, So I have to admit with the embeddings, so 1021 01:02:21.000 --> 01:02:24.320 I was flirting with the idea of doing vector searches 1022 01:02:24.880 --> 01:02:27.400 for top end devs, and I'm going to move there, 1023 01:02:27.840 --> 01:02:30.400 but I'm just using what's built into postgress at this point, 1024 01:02:30.960 --> 01:02:34.679 and that's what it pointed me too as one option, Right, 1025 01:02:34.719 --> 01:02:38.519 I could hand off getting the vectors out of a 1026 01:02:38.599 --> 01:02:41.840 text embedding, but beyond that are there are other things 1027 01:02:41.840 --> 01:02:44.360 that people do with these because I'm I'm kind of 1028 01:02:44.480 --> 01:02:46.599 lost as far as like how useful this is? 1029 01:02:46.840 --> 01:02:51.480 Right, Well, you could do so many things, like you know, 1030 01:02:51.639 --> 01:02:56.480 cluster things together, for example, and try to have different topics. 1031 01:02:57.119 --> 01:03:01.159 So let's maybe start from what isn't embedding. I'll try 1032 01:03:01.199 --> 01:03:02.840 to compress it because I know we don't have so 1033 01:03:02.920 --> 01:03:07.639 much time. It's basically a very long array of numbers 1034 01:03:07.960 --> 01:03:11.400 which represents a point in a space which is a 1035 01:03:11.480 --> 01:03:15.480 multi dimensional space, and the dimensions are you know, the 1036 01:03:15.559 --> 01:03:18.559 length of the array. Essentially, what that means is that 1037 01:03:18.599 --> 01:03:22.239 if you projected in three D space, it's just this point, right, 1038 01:03:22.320 --> 01:03:24.760 And let's say this point represents a word or a 1039 01:03:24.800 --> 01:03:32.079 document or whatever, and the distance between another point is 1040 01:03:32.079 --> 01:03:35.239 actually significant, and it is semantically significant. 1041 01:03:35.360 --> 01:03:37.119 So, for example, if. 1042 01:03:37.000 --> 01:03:41.519 You would have the word and let's let's bring it 1043 01:03:41.559 --> 01:03:43.800 back to word t K. Right, it's a really really 1044 01:03:43.800 --> 01:03:48.719 old model that used to represent words in this embedding space. 1045 01:03:48.880 --> 01:03:49.039 Right. 1046 01:03:49.639 --> 01:03:57.840 So if we say, if we take the embedding of king, right, 1047 01:03:58.519 --> 01:04:01.639 and we take the embedding of queen, we can actually 1048 01:04:02.400 --> 01:04:07.719 I believe, subtract from King the embedding of man and 1049 01:04:07.920 --> 01:04:12.000 add the embedding of woman, and we will get very close, 1050 01:04:12.679 --> 01:04:16.519 if not exactly perfect on the embedding of queen. Right, 1051 01:04:16.719 --> 01:04:19.880 So they're semantically significant. So what that means is that 1052 01:04:19.920 --> 01:04:25.840 you can start to you know, cluster similar data points, 1053 01:04:25.920 --> 01:04:32.440 similar documents, similar movies, similar whatever you want. You can 1054 01:04:32.559 --> 01:04:37.880 search in them. And searching is a kind of a clustering. 1055 01:04:37.880 --> 01:04:41.800 So you're basically like taking the query and embed it 1056 01:04:42.480 --> 01:04:44.320 and then see where it ends up in the space, 1057 01:04:44.760 --> 01:04:49.559 and then you start to expand your your bubble to 1058 01:04:49.719 --> 01:04:52.760 see who else is nearby, and then you just take 1059 01:04:52.800 --> 01:04:55.519 a limit, you know, the top ten or whatever, because 1060 01:04:55.559 --> 01:04:57.480 it could go infinitely. You know, you could get all 1061 01:04:57.519 --> 01:04:59.840 the documents just ordered by out. 1062 01:04:59.760 --> 01:05:04.239 Close, right you were just yeah, ordered by distance. Yeah yeah, Okay, 1063 01:05:04.880 --> 01:05:06.000 So it really is. 1064 01:05:05.960 --> 01:05:10.320 Kind of a search type use case, that's right, But 1065 01:05:10.400 --> 01:05:14.920 it's more semantic than the typical keyword search. 1066 01:05:16.480 --> 01:05:22.880 Right, because if you use the keyword for ruby, well 1067 01:05:22.920 --> 01:05:27.840 that's not a great example, but if you make it 1068 01:05:28.159 --> 01:05:31.039 you have two synonyms, yeah. 1069 01:05:31.280 --> 01:05:33.119 Or yes, sing a TYPEO won't find it. 1070 01:05:33.880 --> 01:05:37.159 Or if you have synonyms, right, And so somebody said testing, 1071 01:05:37.199 --> 01:05:41.480 and somebody said, uh, tested, and you look for tested, 1072 01:05:42.320 --> 01:05:46.280 it won't find testing because it's not doing semantic you know, 1073 01:05:46.320 --> 01:05:47.679 it doesn't break it up into tokens. 1074 01:05:47.679 --> 01:05:50.679 It doesn't do all of that other work where this does. 1075 01:05:50.800 --> 01:05:52.559 And so yeah that makes sense. 1076 01:05:52.639 --> 01:05:55.360 Yeah, a typical search engine would have stemming, so they 1077 01:05:55.400 --> 01:06:00.000 would separate testing and tested and kind of matches anyway. 1078 01:06:00.360 --> 01:06:02.920 But I that's exactly right. I think the genre idea 1079 01:06:03.000 --> 01:06:03.639 is exactly right. 1080 01:06:04.679 --> 01:06:08.199 Yeah, all right, cool, Well, I'm gonna go ahead and 1081 01:06:08.800 --> 01:06:11.760 wrap us up and head us into picks. But this 1082 01:06:11.920 --> 01:06:15.719 is terrific. I'm gonna point people to ruby l l 1083 01:06:15.840 --> 01:06:17.840 M dot com. But if they want to reach out 1084 01:06:17.880 --> 01:06:22.800 to you or connect in some other way, like how 1085 01:06:22.800 --> 01:06:24.719 do people find you on the internet and say, dude, 1086 01:06:24.760 --> 01:06:25.440 this is awesome. 1087 01:06:26.320 --> 01:06:32.079 So my blog is at paulinot me and you can 1088 01:06:32.119 --> 01:06:34.880 reach me on Twitter as well at x dot com 1089 01:06:34.920 --> 01:06:36.400 slash Paulino. 1090 01:06:37.719 --> 01:06:40.840 Mm hmm. And for the non Italians, Paulino is p 1091 01:06:40.920 --> 01:06:42.559 a O l I n O. 1092 01:06:43.159 --> 01:06:45.280 Yes, correct, p l O n I N. 1093 01:06:45.440 --> 01:06:49.719 So yeah, it's it's the p a O that might 1094 01:06:49.800 --> 01:06:53.079 lose people anyway. 1095 01:06:54.559 --> 01:06:56.920 Thanks so much for coming. Let's let's jump in on 1096 01:06:56.960 --> 01:07:00.480 the picks. I'm gonna be kind of short with mine 1097 01:07:02.280 --> 01:07:03.519 just because I don't want to take a ton of 1098 01:07:03.519 --> 01:07:04.840 time and I have to jump over to this other 1099 01:07:04.880 --> 01:07:11.320 meeting at eleven. But so my first pick I always 1100 01:07:11.360 --> 01:07:15.800 do a board game picks. Last week I think I 1101 01:07:15.880 --> 01:07:17.599 might have picked this, but I'm going to pick it again. 1102 01:07:18.440 --> 01:07:20.400 It's kind of like the game that we've been playing lately. 1103 01:07:20.400 --> 01:07:26.199 It's called Cult Express, and basically you have a train 1104 01:07:26.280 --> 01:07:28.400 on the table and you know, it's just made out 1105 01:07:28.440 --> 01:07:30.480 of wood. It kind of all fits together. But it's 1106 01:07:30.599 --> 01:07:34.800 unique for a game board, right because it's it's it's 1107 01:07:34.840 --> 01:07:39.719 this three D train and your bank your train robbers, 1108 01:07:39.800 --> 01:07:41.480 and so you're trying to gather as much loot as 1109 01:07:41.519 --> 01:07:45.840 possible and whoever gathers the most loot wins. You also 1110 01:07:45.960 --> 01:07:50.800 get you get one thousand dollars for being the person 1111 01:07:50.800 --> 01:07:54.400 who shot the most bullets. And so you have cards 1112 01:07:54.400 --> 01:07:56.480 that represent actions. You can pick up loot, you can 1113 01:07:56.559 --> 01:07:58.519 move from the roof to the car, or the car 1114 01:07:58.599 --> 01:08:00.920 to the roof. You can move one card to the other. 1115 01:08:01.840 --> 01:08:06.800 You can shoot your gun at other robbers, and if 1116 01:08:06.880 --> 01:08:09.840 you hit them, then your bullet card goes into their deck. 1117 01:08:10.199 --> 01:08:13.440 And it's basically no op, no action card. And so 1118 01:08:13.480 --> 01:08:15.079 when you drop up a new hand, if you have 1119 01:08:15.159 --> 01:08:18.399 those in there. They're just kind of duds. You can 1120 01:08:18.439 --> 01:08:21.920 punch other players, and if you punch another player, then 1121 01:08:21.920 --> 01:08:25.600 they drop a piece of loot, and then you can 1122 01:08:25.640 --> 01:08:28.640 pick up loot. And then all the players or all 1123 01:08:28.680 --> 01:08:32.239 the characters have different abilities, right, And so the player 1124 01:08:32.279 --> 01:08:35.359 that I played, if you punch somebody and they drop loot, 1125 01:08:35.479 --> 01:08:37.760 if they drop a loot bag, you can catch it 1126 01:08:38.479 --> 01:08:41.479 before it hits the floor and collect it without having 1127 01:08:41.479 --> 01:08:42.920 to play a pick up loot card. 1128 01:08:43.239 --> 01:08:43.439 Right. 1129 01:08:44.159 --> 01:08:46.560 Some of the others were that you could you could 1130 01:08:46.560 --> 01:08:49.199 shoot above you or below you if you were on 1131 01:08:49.239 --> 01:08:51.840 the roof or in the car. There were some that 1132 01:08:51.920 --> 01:08:53.880 had to do with a sheriff, right, because there's a 1133 01:08:53.920 --> 01:08:56.560 sheriff on the train and if he winds up in 1134 01:08:56.600 --> 01:08:58.960 the same car as you, then he puts a bullet 1135 01:08:58.960 --> 01:09:03.039 in your in your deck anyway, So it was a 1136 01:09:03.079 --> 01:09:05.680 lot of fun board game. Geek waits at one point 1137 01:09:05.720 --> 01:09:09.000 eighty six, which means that it's pretty approachable for the 1138 01:09:09.039 --> 01:09:12.479 average random board game player that doesn't play the heavy 1139 01:09:12.520 --> 01:09:17.159 board games like I like. And uh yeah, it took 1140 01:09:17.239 --> 01:09:19.560 us like forty five minutes or so to play, or 1141 01:09:19.600 --> 01:09:23.199 maybe an hour, but we were learning the rules and 1142 01:09:23.239 --> 01:09:24.840 so I could I could imagine you can play in 1143 01:09:24.880 --> 01:09:25.600 forty five minutes. 1144 01:09:26.079 --> 01:09:27.880 And yeah. 1145 01:09:27.920 --> 01:09:31.560 Anyway, so the way you play your cards is you 1146 01:09:31.640 --> 01:09:34.600 go around the circle and everybody plays a card onto 1147 01:09:34.720 --> 01:09:35.720 the pile. 1148 01:09:37.079 --> 01:09:38.319 Sometimes you play two cards. 1149 01:09:38.319 --> 01:09:41.439 Sometimes you play him face down, but you can see 1150 01:09:41.439 --> 01:09:44.239 what everybody else is playing on a regular play. And 1151 01:09:44.279 --> 01:09:47.319 then you take the deck and you flip it over 1152 01:09:47.560 --> 01:09:50.840 and you just play the deck. And so then you know, 1153 01:09:51.159 --> 01:09:53.039 he played a shoot the gun card, so if he 1154 01:09:53.039 --> 01:09:55.520 could shoot somebody, you shot him. And then the next 1155 01:09:55.560 --> 01:09:57.680 guy he played a punch card, so he punched the 1156 01:09:57.720 --> 01:09:59.920 guy next to him because he could. And if you can, 1157 01:10:00.199 --> 01:10:01.720 then you can't write. So if you play it and 1158 01:10:01.760 --> 01:10:03.439 there's nobody in the car with you to punch and 1159 01:10:03.479 --> 01:10:06.079 you just don't punch. But if there is somebody there 1160 01:10:06.119 --> 01:10:07.960 and they're not the person you wanted to punch, you 1161 01:10:08.000 --> 01:10:08.840 still punch them. 1162 01:10:09.239 --> 01:10:10.840 So that's kind of how it works. 1163 01:10:10.880 --> 01:10:14.920 So anyway, it was really really fun, really really enjoyed it. 1164 01:10:14.920 --> 01:10:17.399 It kind of reminds me of another game called robot Rally. 1165 01:10:18.279 --> 01:10:20.399 The difference is is that with robot Rally, what you 1166 01:10:20.439 --> 01:10:23.359 do is you put all of your action cards into 1167 01:10:23.399 --> 01:10:26.039 your own pile and then you flip it over and 1168 01:10:26.079 --> 01:10:28.119 play them in order. But you play them in order, 1169 01:10:28.199 --> 01:10:31.319 everybody goes around and plays them. And so you know, 1170 01:10:31.399 --> 01:10:33.199 a lot of times you set yourself up to push 1171 01:10:33.239 --> 01:10:37.439 another robot into a hole or make them go off course, 1172 01:10:37.680 --> 01:10:39.479 but they did something you didn't expect, and so then 1173 01:10:39.560 --> 01:10:42.239 you wind up making moves that you didn't expect because 1174 01:10:42.239 --> 01:10:45.199 you expected to hit them. And so anyway, it's it's 1175 01:10:45.239 --> 01:10:48.439 a lot of fun. It's kind of a it's almost 1176 01:10:48.439 --> 01:10:52.279 like mob programming a little bit. So anyway, a fun 1177 01:10:52.359 --> 01:10:58.600 cult express it. It's really fun. Besides that, at work, 1178 01:10:59.039 --> 01:11:01.760 we're doing a Ruby book club and we're reading Sandy 1179 01:11:01.800 --> 01:11:12.159 Metz's oh what is it called. It's the Pewter Book, 1180 01:11:12.239 --> 01:11:19.760 The Practical Object Oriented Design in Ruby, and it's a 1181 01:11:19.840 --> 01:11:22.520 terrific book. I don't agree with everything that Sandy says 1182 01:11:22.520 --> 01:11:24.279 in it, but a lot of it I do, and 1183 01:11:24.640 --> 01:11:26.199 the rest of it every time I read it and 1184 01:11:26.279 --> 01:11:28.000 I think about it for a while ago. 1185 01:11:29.319 --> 01:11:31.920 Okay, yes, but with caveats. 1186 01:11:31.840 --> 01:11:34.760 Right, so right, anyway, if you haven't read it, it 1187 01:11:34.800 --> 01:11:37.239 was written in twenty eighteen, the second edition, but it's 1188 01:11:37.239 --> 01:11:38.119 still pretty down good. 1189 01:11:38.159 --> 01:11:39.680 So I'm going to pick that. 1190 01:11:39.960 --> 01:11:43.560 And yeah, in a couple of weeks, we're going to 1191 01:11:43.600 --> 01:11:47.399 be talking to the guys that built Ruby events dot org. 1192 01:11:48.239 --> 01:11:51.840 It was originally Ruby videos dot dev and so it 1193 01:11:51.880 --> 01:11:53.920 had all the conference videos. They were trying to collect 1194 01:11:53.920 --> 01:11:58.079 all of those. But yeah, now they're pulling in like 1195 01:11:58.119 --> 01:12:00.640 all the upcoming conferences and stuff like that, making it 1196 01:12:00.680 --> 01:12:01.960 easy to figure all that stuff out. 1197 01:12:02.000 --> 01:12:05.800 So I'm going to pick that too, Carmine, what are 1198 01:12:05.840 --> 01:12:06.319 your picks? 1199 01:12:06.880 --> 01:12:08.439 My picks? Can I go with music? 1200 01:12:09.880 --> 01:12:10.119 Yeah? 1201 01:12:10.199 --> 01:12:12.760 Yeah, yeah, yeah, all right cool. So I run a 1202 01:12:12.840 --> 01:12:16.640 disco collective here in Berlin. It's called Floppy Disco Fun 1203 01:12:16.960 --> 01:12:19.680 and what women trying to do is to bring back 1204 01:12:19.720 --> 01:12:21.159 the funk a little bit in Berlin. As you know, 1205 01:12:21.199 --> 01:12:24.159 it's probably as you probably know, it's a very much 1206 01:12:24.199 --> 01:12:28.520 of a techno city. And I want to pick some 1207 01:12:28.760 --> 01:12:31.039 friends that have been playing with us that have been 1208 01:12:31.079 --> 01:12:37.319 absolutely fantastic. Cody Curry, he is a jazz house master. 1209 01:12:37.479 --> 01:12:41.800 We play with him actually this Saturday, and he has 1210 01:12:42.239 --> 01:12:46.399 releases on Spotify that you know, they have three million plays, 1211 01:12:46.479 --> 01:12:50.920 So definitely hitting a spot there with people. Very jazzy, 1212 01:12:51.399 --> 01:12:56.399 very groovy, very soft jazz kind of style, but in 1213 01:12:56.520 --> 01:13:01.359 house music, so really really cool stuff, really chill guy, 1214 01:13:01.399 --> 01:13:02.880 also really cool. 1215 01:13:03.520 --> 01:13:03.640 RP. 1216 01:13:03.800 --> 01:13:06.840 Brown from the same label. Toy Toonics is the label 1217 01:13:06.880 --> 01:13:09.359 by the way, it's an Italian German label by an 1218 01:13:09.359 --> 01:13:10.239 Italian German guy. 1219 01:13:11.640 --> 01:13:12.560 And r P. 1220 01:13:12.760 --> 01:13:13.720 Brown is. 1221 01:13:16.399 --> 01:13:20.239 More on the UK side of the spectrum. I want 1222 01:13:20.239 --> 01:13:24.479 to say, so he combines you know, disco with UK 1223 01:13:24.680 --> 01:13:29.000 influences because he lived there for a little bit, uh 1224 01:13:29.039 --> 01:13:33.800 and Delphonic different label. He posts on a bunch of 1225 01:13:33.840 --> 01:13:38.239 labels actually, and he has been I think one of 1226 01:13:38.279 --> 01:13:45.560 the biggest disco influencers here in Berlin and DJs and 1227 01:13:45.600 --> 01:13:47.319 he plays a lot and he's uh. 1228 01:13:47.880 --> 01:13:50.880 Just also just such a nice guy. So I want 1229 01:13:50.920 --> 01:13:51.640 to pick these three. 1230 01:13:54.079 --> 01:13:58.600 And in terms of books, uh, the Hot Wire book 1231 01:13:58.920 --> 01:14:03.399 by Joe Oh, fantastic, fantastic book. 1232 01:14:04.319 --> 01:14:06.760 Been reading this and now you're really enjoying it. 1233 01:14:08.239 --> 01:14:08.920 Which one is that? 1234 01:14:09.760 --> 01:14:15.800 I think it's called All to Our tur Ball? I sorry, Joe. 1235 01:14:16.760 --> 01:14:20.119 Why are Native for rails developers? 1236 01:14:20.159 --> 01:14:20.319 Oh? 1237 01:14:20.319 --> 01:14:26.239 Hot Wire Native Yes by Joe mast Yeah, yeah, yeah, yeah, 1238 01:14:26.239 --> 01:14:29.399 that's a terrific book on practprog. We talked to him 1239 01:14:29.399 --> 01:14:31.640 a little bit ago and we're going to have him 1240 01:14:31.640 --> 01:14:32.119 come back on. 1241 01:14:32.079 --> 01:14:34.479 Because there was just more to talk about. So I'll 1242 01:14:34.479 --> 01:14:35.720 look for that a month or so. 1243 01:14:36.439 --> 01:14:36.840 Awesome. 1244 01:14:39.159 --> 01:14:42.119 All right, good deal. Well, thanks for coming, Thank you 1245 01:14:42.159 --> 01:14:42.720 for having me. 1246 01:14:43.239 --> 01:14:44.079 It's really a pleasure. 1247 01:14:44.199 --> 01:14:46.920 This was so cool. I may not sleep for a 1248 01:14:46.920 --> 01:14:48.359 week because I'm playing with this stuff. 1249 01:14:50.319 --> 01:14:52.119 I'll feel bad and happy at the same time. 1250 01:14:53.680 --> 01:14:56.800 Yeah, all right, Well I'm gonna go ahead and push stop. 1251 01:14:56.880 --> 01:14:58.479 Stick around because we got to make sure we get 1252 01:14:58.520 --> 01:15:02.239 all your stuff uploaded. But to the listener until next time, 1253 01:15:02.319 --> 01:15:07.159 max out. M m mm hmmm.