WEBVTT 1 00:00:05.000 --> 00:00:06.599 To like already click the button. 2 00:00:09.720 --> 00:00:11.359 And that's how we're starting the episode. 3 00:00:13.240 --> 00:00:15.960 Oh for those of you just getting started, actually you 4 00:00:16.000 --> 00:00:17.800 are just getting started because I just clicked on the 5 00:00:17.920 --> 00:00:23.480 record button. But hey, welcome, thanks for joining the podcast. Warren. 6 00:00:23.519 --> 00:00:24.000 How are you? 7 00:00:24.679 --> 00:00:26.640 I'm I'm great. You know, I feel like, you know, 8 00:00:26.679 --> 00:00:29.239 we should go back to this is Adventures and DevOps. 9 00:00:29.280 --> 00:00:31.440 You know, for the first time listeners, if this is 10 00:00:31.519 --> 00:00:35.079 just the one episode they happen to click on, then 11 00:00:35.359 --> 00:00:37.399 they're in for a real treat today because I'm really 12 00:00:37.439 --> 00:00:38.840 interested in today's topic. 13 00:00:39.920 --> 00:00:45.520 Yeah, and today's topic, we have gorkam Erkan with us 14 00:00:45.600 --> 00:00:49.799 and we're gonna be talking about how to how to 15 00:00:49.840 --> 00:00:56.479 integrate AI and mL into teams using tooling that's a 16 00:00:56.520 --> 00:00:59.359 little bit more aligned with how we do DevOps. 17 00:00:59.479 --> 00:01:00.520 That sound of out right? 18 00:01:01.679 --> 00:01:02.880 Yeah? That sounds right? 19 00:01:03.200 --> 00:01:03.640 All right? 20 00:01:03.960 --> 00:01:04.280 Cool? 21 00:01:05.280 --> 00:01:08.200 Before we dig into the details of that, can you 22 00:01:08.239 --> 00:01:11.319 share with our listeners a little bit about your background? 23 00:01:12.799 --> 00:01:18.079 Sure. I am the CEA of josu U. The josu 24 00:01:18.200 --> 00:01:22.079 is a company that is trying to make it easier 25 00:01:22.079 --> 00:01:30.000 for enterprises to adopt AI and mL in into their applications. Uh. 26 00:01:30.040 --> 00:01:34.359 And prior to JOSU I was with red Hat for many, 27 00:01:34.400 --> 00:01:37.120 many years. I was as a distinguished engineer with red 28 00:01:37.159 --> 00:01:46.040 Hat worked on developer tools, which in red hat it 29 00:01:46.319 --> 00:01:49.840 covers anything from I D E S to C C 30 00:01:50.000 --> 00:01:54.920 D pipelines to them op so it's it's all everything 31 00:01:55.480 --> 00:01:59.480 that you can think of that is open source and 32 00:01:59.560 --> 00:02:04.200 that will tool My group kind of got involved with 33 00:02:04.239 --> 00:02:10.560 those as part of that. I did Java tools early 34 00:02:10.560 --> 00:02:18.479 in my career for something called J two E if 35 00:02:18.520 --> 00:02:25.120 anyone remembers that. So and then I actually had the 36 00:02:25.240 --> 00:02:32.080 great idea of integrating Java tooling into vs code. The 37 00:02:32.439 --> 00:02:37.120 current Java tooling is something that I have done in 38 00:02:37.479 --> 00:02:40.400 a month or so and that has grown to what 39 00:02:40.479 --> 00:02:47.000 it is today. My group got involved with things like 40 00:02:47.159 --> 00:02:50.000 tech Tile, CD Foundation and so and so forth. So 41 00:02:50.000 --> 00:02:53.879 I've been doing devobs for the last probably five years, 42 00:02:54.199 --> 00:02:58.159 or tools for DeVos for five years, and tools in general. 43 00:02:58.360 --> 00:03:05.599 Before that, before joining red Hat, I was working for Nokia, 44 00:03:06.280 --> 00:03:09.879 the phone company, so I was in the middle of 45 00:03:09.919 --> 00:03:21.360 the mobile revolution and again mostly doing open source projects 46 00:03:21.680 --> 00:03:23.400 at Noukia at the time. 47 00:03:23.639 --> 00:03:27.240 When you say open source and Java and red Hat, 48 00:03:27.280 --> 00:03:31.400 my mind immediately goes to Jenkins and I don't have 49 00:03:31.479 --> 00:03:34.919 a particular soft spot in my heart for that product, 50 00:03:34.960 --> 00:03:36.599 but I can imagine you've had to spend a lot 51 00:03:36.639 --> 00:03:38.199 of time with it. Is that what they were using. 52 00:03:39.960 --> 00:03:43.840 So yeah, we did start with Jenkins. We do support, 53 00:03:43.960 --> 00:03:47.800 We did support or red Hats still supports Jenkins to 54 00:03:48.000 --> 00:03:53.080 some degree. But at the time when we tried to 55 00:03:53.159 --> 00:03:57.520 do a project, code open shift that I owe and 56 00:03:57.639 --> 00:03:59.919 one of the things that we tried to do at 57 00:04:00.080 --> 00:04:06.159 the time was to run Jenkins in a manner that 58 00:04:06.199 --> 00:04:09.520 it is not supposed to be run at the time. 59 00:04:09.960 --> 00:04:13.560 I think Jenkins is doing better now, but at the time, 60 00:04:13.879 --> 00:04:16.839 Jenkins was created as a server, so it was meant 61 00:04:16.879 --> 00:04:19.240 to run as a server. But then when you're on 62 00:04:19.279 --> 00:04:23.399 the cloud trying to run Jenkins, one thing that you 63 00:04:23.439 --> 00:04:25.360 want to do is you want to try to run 64 00:04:25.399 --> 00:04:30.360 its servers, like if I need to go and build something, 65 00:04:30.879 --> 00:04:35.680 especially if you're doing this as a SAS service, you 66 00:04:35.759 --> 00:04:39.160 want to just on demand start a Jenkins instance, do 67 00:04:39.319 --> 00:04:41.560 your bills, and then shut it down so that you're 68 00:04:41.560 --> 00:04:47.000 not spending too much on the cloud resources. That didn't 69 00:04:47.040 --> 00:04:51.040 work well for us. So one of the things that 70 00:04:51.240 --> 00:04:55.560 happened at the time was the Canadia project. If you're 71 00:04:55.600 --> 00:04:59.800 familiar with it, and as part of the Canadia project 72 00:05:00.040 --> 00:05:06.240 first code drop before even it even got public, it 73 00:05:06.279 --> 00:05:09.279 was open to some of the redheaders, including myself, and 74 00:05:09.319 --> 00:05:11.839 one of the things that we noticed was there was 75 00:05:11.879 --> 00:05:15.279 a built piece in the k native code which was 76 00:05:15.319 --> 00:05:21.120 actually doing server spiels essentially. So the whole idea of oh, 77 00:05:21.360 --> 00:05:25.439 can we actually make this and turned this into something 78 00:05:26.879 --> 00:05:31.240 It turned out to be the techtile project later and 79 00:05:31.279 --> 00:05:35.920 then it became the tech Time's cd c CD system 80 00:05:36.560 --> 00:05:40.839 that we know today. So it actually we got involved 81 00:05:40.839 --> 00:05:46.879 with that because of Jenkins because we couldn't actually make 82 00:05:46.920 --> 00:05:50.879 it cloud native at the time, so it felt like, oh, 83 00:05:50.920 --> 00:05:54.600 we should actually, you know, do something cloud native instead. 84 00:05:56.120 --> 00:06:01.680 So that's that's how involvement with Jane. But on the 85 00:06:01.720 --> 00:06:05.800 other hand, I would still bet that a big chunk 86 00:06:05.839 --> 00:06:11.279 of workloads on kimbernites in the world is actually Jenkins. 87 00:06:16.160 --> 00:06:16.959 Survey to edit. 88 00:06:17.240 --> 00:06:19.680 I mean, I don't know, I don't have un versa. 89 00:06:19.720 --> 00:06:23.079 I like that would be my guess. It's it's it's 90 00:06:23.120 --> 00:06:25.240 in double digits. I'm pretty sure enough. 91 00:06:25.680 --> 00:06:28.279 I feel like I feel like this is where ignorance 92 00:06:28.399 --> 00:06:30.680 is bliss. Like I don't I'm going to pretend that's 93 00:06:30.720 --> 00:06:34.000 not happening and just continue living my life in whatever 94 00:06:34.279 --> 00:06:36.000 delusion I have currently going on. 95 00:06:37.079 --> 00:06:41.199 Unfortunately, it is true. It's there, It's it's in many enterprises. 96 00:06:41.240 --> 00:06:47.839 You can't ignore Jenkins. It's hundreds of instances for sure. Yeah. 97 00:06:48.040 --> 00:06:53.399 I have a love hate relationship with Jenkins. I hate 98 00:06:53.480 --> 00:06:56.000 working with it, but I love how it can just 99 00:06:56.079 --> 00:06:58.519 do absolutely anything you needed to do. 100 00:07:00.519 --> 00:07:07.120 And the ecosystem, Yeah, like the ecosystem behind Jenkins. You know, 101 00:07:07.360 --> 00:07:11.759 I think nowadays get up actions maybe at that level, 102 00:07:11.839 --> 00:07:19.399 but the ecosystem behind Jenkins, it's, it's it's pretty big. 103 00:07:19.560 --> 00:07:22.360 It's very important. You can find a plug in for 104 00:07:22.399 --> 00:07:26.600 anything essentially, yeah, or anything that you care about, right 105 00:07:27.920 --> 00:07:28.639 for sure. 106 00:07:29.959 --> 00:07:34.839 So let's talk a little bit about aim L since 107 00:07:34.879 --> 00:07:38.720 that's our topic for today. And there's like, you know, 108 00:07:38.959 --> 00:07:43.879 I whenever I think about this, the most common scenarios 109 00:07:43.920 --> 00:07:47.120 that I see coming up are people using chat, GPT 110 00:07:47.759 --> 00:07:50.560 or tools like that, And it seems like a very 111 00:07:53.240 --> 00:07:59.680 early stage product. So to get that enterprise ready, like, 112 00:07:59.720 --> 00:08:02.360 what the what are the challenges that people are facing? 113 00:08:03.879 --> 00:08:07.639 Yeah, so I think check GPT, Like what we are 114 00:08:07.720 --> 00:08:10.920 going through right now is that that in every enterprise 115 00:08:11.000 --> 00:08:14.920 there is one or two let's call it experiments that 116 00:08:15.040 --> 00:08:18.079 is going on. And usually when you're starting the experiment, 117 00:08:18.560 --> 00:08:20.560 the first thing that you do is like what is 118 00:08:20.600 --> 00:08:27.680 the least resistance that you can face, and that is 119 00:08:28.360 --> 00:08:31.319 use the check GPT API s right, the open API 120 00:08:31.600 --> 00:08:37.320 or Microsoft has Azure APIs as well, So use those 121 00:08:37.759 --> 00:08:40.679 and do whatever you need to do and come up 122 00:08:40.720 --> 00:08:44.120 with your first early experiments. And when you look at 123 00:08:44.159 --> 00:08:49.600 those experiments in many of the organizations, what you see 124 00:08:49.720 --> 00:08:52.759 is there are a lot of success that is coming in. 125 00:08:53.120 --> 00:08:55.639 There's a lot of learning that they are going through, 126 00:08:56.159 --> 00:09:00.679 but there's also successful stories as well coming out of them. Uh. 127 00:09:01.240 --> 00:09:04.360 The one thing about AI is in deterministic, right, It's 128 00:09:04.399 --> 00:09:07.960 like the first time that you do something with AI, 129 00:09:08.200 --> 00:09:11.720 you're going to fail, and then the second you're going 130 00:09:11.759 --> 00:09:16.120 to do another iteration, and because you're going to understand 131 00:09:16.200 --> 00:09:19.519 that behavior, and then you're going to do another iteration. 132 00:09:19.639 --> 00:09:23.240 I think check GPT and all the other APIs out there, 133 00:09:23.360 --> 00:09:30.519 of the open API and similar AI API is out there. 134 00:09:30.600 --> 00:09:36.639 I think it provides them this opportunity to do the experimentation. 135 00:09:37.360 --> 00:09:39.720 But the way I see it is this is this 136 00:09:39.759 --> 00:09:43.639 is temporary, right, This is just dipping your toes into 137 00:09:43.679 --> 00:09:48.879 the water. When you actually want to start doing something 138 00:09:50.519 --> 00:09:53.799 that will change your business and try to get some 139 00:09:54.000 --> 00:09:58.240 financial benefits to your company. That means integrating with many 140 00:09:58.320 --> 00:10:01.759 main systems. That means like it's like we have seen 141 00:10:01.799 --> 00:10:04.159 it with cloud, we have seen it with mobile. Right, 142 00:10:04.200 --> 00:10:07.240 It's like it's not a oh, here's one application that 143 00:10:07.399 --> 00:10:11.679 is mobile, or here's one application that runs on the cloud. 144 00:10:12.159 --> 00:10:15.600 You're starting to adopt it to your organization or if 145 00:10:15.639 --> 00:10:18.440 you are even older, there was a time where there 146 00:10:18.559 --> 00:10:23.200 was we talked about paper this office. Right, So it's 147 00:10:23.240 --> 00:10:26.360 like it takes a little bit of time, it's a process, 148 00:10:26.440 --> 00:10:30.159 and you actually need to internalize it into your organization. 149 00:10:30.240 --> 00:10:33.840 I think the next step for many of these organizations 150 00:10:34.360 --> 00:10:37.960 is to start to internalize it. And when you're internalizing it, 151 00:10:38.000 --> 00:10:43.639 you cannot really depend on a third party service. And 152 00:10:43.759 --> 00:10:48.559 some of the industries cannot even share their data give 153 00:10:48.600 --> 00:10:54.320 their data to these services where there is concerns for 154 00:10:54.440 --> 00:10:58.279 privacy and security and so on and so forth. So 155 00:10:58.320 --> 00:11:06.600 they need to start running these essentially base models internally 156 00:11:07.000 --> 00:11:11.919 in their infrastructure or in their cloud infrastructure so that 157 00:11:12.759 --> 00:11:16.000 they can get the benefits. And the one thing that 158 00:11:16.480 --> 00:11:21.480 I see is when you go to your data scientists 159 00:11:21.639 --> 00:11:33.720 or data data scientists or mL engineer. They are separate 160 00:11:34.480 --> 00:11:38.960 from the DevOps engineers in the company. They are not 161 00:11:39.080 --> 00:11:43.600 well integrated. They have a very different tool set. It's 162 00:11:43.679 --> 00:11:47.279 just because of the way the AI has been developed 163 00:11:47.320 --> 00:11:51.080 and coming into the world. And one thing that you 164 00:11:51.200 --> 00:11:54.600 notice is there is no shortage of open source projects 165 00:11:54.600 --> 00:11:57.639 and tools in the AI world, but there is a 166 00:11:57.679 --> 00:12:03.720 shortage of projects and open source tools in the AI 167 00:12:03.840 --> 00:12:08.799 world that are standards. You can do. Just think of 168 00:12:08.919 --> 00:12:12.240 any subject and you can find ten alternates to do it. 169 00:12:13.000 --> 00:12:16.799 None of them talks to each each other. Nothing is standard. 170 00:12:17.840 --> 00:12:21.559 So this is one of the things that we have noticed. 171 00:12:21.559 --> 00:12:26.200 We actually were trying to solve this for ourselves as well, 172 00:12:26.600 --> 00:12:29.840 and one of the things we decided what was okay, 173 00:12:29.919 --> 00:12:34.919 So I have so many ways of storing a model 174 00:12:35.919 --> 00:12:39.039 and moving that to production. I have so many ways 175 00:12:39.080 --> 00:12:43.759 of storing a data set and moving that to inference 176 00:12:43.919 --> 00:12:46.720 or training and testing and so on and so forth, 177 00:12:47.120 --> 00:12:50.519 but none of it is standard, like this should be 178 00:12:50.639 --> 00:12:53.559 easier than what it is right now. So what we 179 00:12:53.600 --> 00:12:57.480 did is we came up with an open source project 180 00:12:57.519 --> 00:13:02.200 called ki toops dot mL. And what kittops dot mL 181 00:13:02.360 --> 00:13:06.519 provides is what we call model kits. A model kit 182 00:13:06.679 --> 00:13:12.840 is essentially an OCI artifact and an OCI artifact like 183 00:13:13.039 --> 00:13:16.120 a container image. It's not a container image itself, but 184 00:13:16.200 --> 00:13:19.399 it's an OCI artifact. The advantage of being an OCI 185 00:13:19.639 --> 00:13:25.279 artifact is it can be stored in any OCI registry 186 00:13:25.600 --> 00:13:32.120 like docer hub ECR as you're wherever your image registry is, 187 00:13:32.720 --> 00:13:36.960 and using the information that is stored in the model kit, 188 00:13:37.519 --> 00:13:44.759 you can actually move your models as part of your 189 00:13:44.840 --> 00:13:49.240 c c D pipelines much much efficient because your c 190 00:13:49.480 --> 00:13:53.120 c D pipelines actually knows how to usually knows how 191 00:13:53.120 --> 00:13:57.919 to talk to OCI registrict. For instance, when today we 192 00:13:58.080 --> 00:14:04.120 have a a registry that we are running ourselves in 193 00:14:04.279 --> 00:14:09.919 jose dot mL and you will see many signed model 194 00:14:10.000 --> 00:14:13.759 kits in there. The tool that we are using to 195 00:14:13.840 --> 00:14:17.279 sign model kits is cosign, which is what you would 196 00:14:17.440 --> 00:14:25.519 use for signing an imagery, a regular container image. So 197 00:14:26.720 --> 00:14:33.320 the idea behind having the OCI artifacts was to introduce 198 00:14:33.399 --> 00:14:38.879 a standard way of storing and standard way of sharing 199 00:14:39.799 --> 00:14:48.879 your AI artifacts between data scientists and mL engineers and 200 00:14:49.240 --> 00:14:50.600 DevOps engineers. 201 00:14:51.240 --> 00:14:55.519 You said something really interesting in there, and that's I 202 00:14:55.519 --> 00:14:57.960 haven't heard of put it this way before that your 203 00:14:58.080 --> 00:15:02.000 first foray into using any sort of AI or mL 204 00:15:02.120 --> 00:15:05.080 models in your company will be an experiment that ends 205 00:15:05.120 --> 00:15:09.200 in failure. And I, you know, I think there's something 206 00:15:09.639 --> 00:15:12.200 really to that, because I can imagine if you like 207 00:15:12.320 --> 00:15:15.600 walked around and put like a hammer on every engineer's 208 00:15:15.639 --> 00:15:20.039 desk and said, you will now start to develop with 209 00:15:20.120 --> 00:15:23.759 this hammer instead of what you were using before. I 210 00:15:23.799 --> 00:15:26.159 feel like it's a ridiculous analogy, but I think there's 211 00:15:26.200 --> 00:15:28.080 a lot of sense there that like that for sure 212 00:15:28.159 --> 00:15:31.000 will not end in success there, Like, however you thought 213 00:15:31.840 --> 00:15:35.919 you could utilize it internally is probably not going to 214 00:15:35.960 --> 00:15:39.399 be super effective. And that obviously extends into the area 215 00:15:39.519 --> 00:15:43.080 of if you are building AI into your product, not 216 00:15:43.159 --> 00:15:46.679 just using it as a as a tool for development, 217 00:15:46.840 --> 00:15:49.320 and you're really talking about this next level which isn't 218 00:15:49.360 --> 00:15:53.039 just using it but actually creating it to be effective 219 00:15:53.039 --> 00:15:54.840 as part of the product offering. 220 00:15:56.279 --> 00:15:59.840 Yeah, And the reason for that is it's really in 221 00:15:59.879 --> 00:16:04.120 the terministic right with the AI itself. So you may 222 00:16:04.360 --> 00:16:09.559 actually think that they're like when we are integrating applications together, 223 00:16:10.960 --> 00:16:15.120 the rule based the applications, the outcome is very deterministic. 224 00:16:15.240 --> 00:16:18.159 We know that these are the inputs and these will 225 00:16:18.159 --> 00:16:21.759 be the outputs. In the case of AI, that's not 226 00:16:21.799 --> 00:16:26.679 the case, right, So when you go through your first 227 00:16:26.799 --> 00:16:29.759 initial cycle and go through your you know, you receive 228 00:16:29.799 --> 00:16:32.440 your inputs, and then you start to get your outputs, 229 00:16:32.679 --> 00:16:35.960 you start to see these outputs that you haven't expected before. 230 00:16:36.559 --> 00:16:39.840 And the usually, like, for instance, it's very likely that 231 00:16:39.919 --> 00:16:43.879 you're missing actually some data that you're feeding into the model, 232 00:16:44.440 --> 00:16:48.840 and you're not getting the outputs that you're expecting. Because 233 00:16:49.240 --> 00:16:55.120 let's say that you're giving the customers can data, but 234 00:16:55.159 --> 00:16:58.440 then you when you give the eleventh data, the AI 235 00:16:58.639 --> 00:17:01.080 is now able to make the code nection or or 236 00:17:01.200 --> 00:17:04.440 or make the analysis better and give it a better result. 237 00:17:04.759 --> 00:17:07.559 So you will start to notice things like that, and 238 00:17:07.640 --> 00:17:10.680 it is not very easy to notice this at the 239 00:17:10.759 --> 00:17:16.079 design phase. It's not even possible to notice these things 240 00:17:16.079 --> 00:17:19.200 at the design phase. So that's why I'm saying that 241 00:17:19.480 --> 00:17:22.799 your first experiment is going to fail, and you're gonna 242 00:17:22.839 --> 00:17:26.759 do another one with more data or less data or 243 00:17:27.240 --> 00:17:30.440 changed data, and then you will get to a state 244 00:17:30.519 --> 00:17:33.319 where you're you're you're happy with the outputs. 245 00:17:33.559 --> 00:17:35.960 I feel I have a fear here now that, like 246 00:17:36.039 --> 00:17:38.799 you know, if I'm incorporating it into one of our products. 247 00:17:39.039 --> 00:17:41.680 You know, we have a couple and we have tried 248 00:17:41.799 --> 00:17:44.440 going down the route of adding some AI into into 249 00:17:44.480 --> 00:17:47.799 one of them. Uh, there is this to only say 250 00:17:47.839 --> 00:17:50.880 non deterministic. We mean like literally the same inputs gives 251 00:17:50.920 --> 00:17:55.119 us different output. And exactly, I wonder if we're training 252 00:17:55.200 --> 00:17:58.039 our customers to believe that they should just you know, 253 00:17:59.440 --> 00:18:01.720 turn it off and turn it on again in a sense, right, 254 00:18:01.759 --> 00:18:03.839 Like you know, you have to keep repeating the same 255 00:18:03.920 --> 00:18:06.880 question over and over again to get the answer you want. 256 00:18:06.920 --> 00:18:09.799 And that just that feels like a dangerous thing to 257 00:18:09.799 --> 00:18:13.319 push that expectation onto the users of our products. 258 00:18:15.400 --> 00:18:19.519 Yeah, there is a Yeah, I see what you're you're saying, 259 00:18:19.559 --> 00:18:23.920 and and and I think that's that's that's the part 260 00:18:23.960 --> 00:18:27.759 of the AI that that still needs to be enhanced 261 00:18:28.000 --> 00:18:33.480 a little bit. You know, the the whole subject of 262 00:18:34.039 --> 00:18:42.519 guardrails on AI is a little bit it feels early. Oh, 263 00:18:42.559 --> 00:18:44.640 I'm told, I'm totally I'm totally with you. 264 00:18:44.640 --> 00:18:47.279 No, there was actually a study that was released just recently, 265 00:18:47.319 --> 00:18:50.240 and I think we'll kind of appreciate this that it's 266 00:18:50.279 --> 00:18:54.720 not being like prompt engineering isn't being used for malicious purposes. 267 00:18:54.799 --> 00:18:58.279 It's generally being used by power users to actually get 268 00:18:58.359 --> 00:19:01.400 value out of the system more than anything, which I 269 00:19:01.400 --> 00:19:04.599 think is really surprising that these systems aren't being abused 270 00:19:04.640 --> 00:19:09.000 in that way realistically yet, And so adding in guardrails 271 00:19:09.039 --> 00:19:12.240 of anything is just preventing those early adopters from being 272 00:19:12.240 --> 00:19:15.240 able to utilize your product more effectively, rather than preventing 273 00:19:15.279 --> 00:19:16.240 malicious attackers. 274 00:19:16.319 --> 00:19:19.640 Right. And we actually like since you said prompt Engini, 275 00:19:19.720 --> 00:19:26.039 we actually talked with an enterprise recently and one of 276 00:19:26.119 --> 00:19:33.279 their needs was to share their prompts, Like, uh, think 277 00:19:33.319 --> 00:19:35.559 of it this way. You have a database of all 278 00:19:35.599 --> 00:19:40.519 your all your enterprises data, all your customer data, and 279 00:19:40.559 --> 00:19:43.279 so on and so forth. In the past, the sequel 280 00:19:43.519 --> 00:19:47.680 statements that you have used for finance was very different 281 00:19:47.720 --> 00:19:51.000 from the psychoal statesments that you have used for sales, 282 00:19:51.240 --> 00:19:56.640 so you'd never really actually care to share the psycholos statements. 283 00:19:57.079 --> 00:19:59.440 But now what they're seeing is, as you said, there 284 00:19:59.440 --> 00:20:03.079 are these users who are using these prompts and getting 285 00:20:03.119 --> 00:20:07.039 really good results out of those prompts that are querying 286 00:20:07.079 --> 00:20:10.640 there or that are connected to their their data. Right, 287 00:20:11.079 --> 00:20:13.000 So what they want to be able to do is 288 00:20:13.039 --> 00:20:16.519 to to be able to share these prompts like between 289 00:20:16.680 --> 00:20:21.079 finance and and and HR and and and sales and 290 00:20:21.119 --> 00:20:23.920 so on and so forth, so that they don't have 291 00:20:24.039 --> 00:20:29.440 to reinvent the veil every time. But I think that 292 00:20:29.680 --> 00:20:32.720 we're starting to see a different picture now in the 293 00:20:33.079 --> 00:20:36.200 in the AI world where with with these kind of things. 294 00:20:36.240 --> 00:20:42.160 But when you are opening your data through and an 295 00:20:42.160 --> 00:20:48.119 AI model directed to your customers, I think there's still 296 00:20:48.359 --> 00:20:55.359 work that needs to be done on the guard for this. Yes, yeah, 297 00:20:55.400 --> 00:20:55.720 for sure. 298 00:20:55.759 --> 00:20:59.759 It reminds me a lot of there's two things here 299 00:20:59.799 --> 00:21:03.000 that that seem like there's lessons we can learn here 300 00:21:03.039 --> 00:21:08.559 from other engineering or other industries. Like whenever I work 301 00:21:08.599 --> 00:21:12.960 with early stage startups, I always say that like the 302 00:21:13.000 --> 00:21:19.319 goal of your first product as a startup is to launch, 303 00:21:20.599 --> 00:21:24.480 like the let me rephrase that. To become a successful startup, 304 00:21:24.519 --> 00:21:27.319 you have to identify the difference between the product that 305 00:21:27.400 --> 00:21:30.759 you built and the product that your customers thought you 306 00:21:30.880 --> 00:21:34.000 built and close that gap. And it feels like this 307 00:21:34.240 --> 00:21:40.440 is very similar to that. You know, because AI will 308 00:21:40.519 --> 00:21:43.920 answer whatever question you ask it, and then you have 309 00:21:44.039 --> 00:21:49.279 to figure out what assumptions it made that you didn't 310 00:21:49.359 --> 00:21:53.599 want it to make. And that's where I think really 311 00:21:53.640 --> 00:21:57.240 sharing those prompt those prompt models can be helpful. For 312 00:21:57.279 --> 00:21:59.759 a while there on X there was a trend where 313 00:21:59.759 --> 00:22:03.799 peopleeople were you know, sharing what they were doing with 314 00:22:03.839 --> 00:22:06.640 different AI models and then also sharing the prompts that 315 00:22:06.640 --> 00:22:09.319 they used too, And I found that really interesting to 316 00:22:09.400 --> 00:22:13.200 see the different caveats that they would give in the 317 00:22:13.279 --> 00:22:16.880 prompt to keep it from wandering off some other path 318 00:22:17.079 --> 00:22:19.960 that gave you an answer that looked right, but as 319 00:22:20.279 --> 00:22:21.920 wasn't actually the answer that you needed. 320 00:22:22.440 --> 00:22:24.559 No, you say that and it's like, actually, really ridiculous. 321 00:22:25.160 --> 00:22:27.519 Like I'm going through right now. I'm making a presentation 322 00:22:27.720 --> 00:22:32.480 for the European as reinvent that I'm speaking at, and 323 00:22:33.799 --> 00:22:38.400 I want some pictures, and I will ask I'm using 324 00:22:38.799 --> 00:22:40.400 one of the tools, and I will ask the same 325 00:22:40.440 --> 00:22:42.079 question multiple times. But when I get a picture that 326 00:22:42.119 --> 00:22:43.799 I actually like, I will take the prompt that I 327 00:22:43.960 --> 00:22:45.880 used for it, and I will save it as the 328 00:22:45.920 --> 00:22:49.000 file name of the picture so that I can get like, 329 00:22:49.039 --> 00:22:51.400 I mean, it's a ridiculous, but that's the name for me, 330 00:22:51.519 --> 00:22:53.400 that really is the thing that identifies it, so that 331 00:22:53.440 --> 00:22:55.720 I can, you know, potentially go back later and be like, 332 00:22:55.759 --> 00:22:58.960 what did I use to actually generate that image? 333 00:23:00.279 --> 00:23:04.039 So let me complicate that problem a little bit. So 334 00:23:04.160 --> 00:23:08.480 you have you're you're using a check GPT for instance, like. 335 00:23:09.680 --> 00:23:11.960 It's not the one from open Ai, but yeah, for sure, 336 00:23:11.960 --> 00:23:13.559 it's an l you're using. 337 00:23:14.359 --> 00:23:18.559 You're using and it's actually a check giv A service. 338 00:23:19.240 --> 00:23:24.599 So you don't really know the version number. Really, you 339 00:23:24.599 --> 00:23:27.720 don't really have to know the data sets that has 340 00:23:27.799 --> 00:23:30.599 been used and retrained and so and so forth. So 341 00:23:30.839 --> 00:23:33.279 for some of the organizations that's not going to work. 342 00:23:33.720 --> 00:23:36.559 So some organizations are going to say, hey, you know what, 343 00:23:37.160 --> 00:23:41.119 I need to know exactly the model, the data set 344 00:23:41.200 --> 00:23:44.240 that it was trained on, and I need to know 345 00:23:44.960 --> 00:23:50.480 where my fine tuning is coming from, where my RAGS 346 00:23:50.680 --> 00:23:55.000 is coming from. So I need all the lineage of 347 00:23:55.000 --> 00:24:02.480 of of this when this prompt was issued and right. 348 00:24:02.880 --> 00:24:10.359 So you need systems that are capable of telling precisely 349 00:24:11.000 --> 00:24:14.599 what was the data set, what is the model, what 350 00:24:14.759 --> 00:24:18.599 is the fine tuning data set? Right? And what is 351 00:24:18.839 --> 00:24:21.319 if you have been using RAG, what is what is 352 00:24:22.119 --> 00:24:26.799 the version of the vector database or the lineage of 353 00:24:26.839 --> 00:24:33.440 the vectors. All of this stored and available somewhere because 354 00:24:35.000 --> 00:24:38.240 something may happen at any point saying that, oh, you 355 00:24:38.319 --> 00:24:41.640 need to go back and figure out what went wrong 356 00:24:41.920 --> 00:24:47.079 with this? Answer what went wrong with this today? Like 357 00:24:47.319 --> 00:24:51.559 if you're using an API, you can't really say what 358 00:24:51.559 --> 00:24:59.559 what went wrong? And so if you think about it 359 00:24:59.559 --> 00:25:03.440 this way, we actually have a system that was used 360 00:25:03.480 --> 00:25:07.319 for this purpose. We used s BOMs to be able 361 00:25:07.359 --> 00:25:12.880 to say exactly what we have used in production for 362 00:25:13.000 --> 00:25:18.200 our applications. Right, so can we actually utilize the same 363 00:25:18.319 --> 00:25:24.000 system with AI as well? Like, oh, I'm using the 364 00:25:24.039 --> 00:25:28.279 application x y Z, it's built out of these dependencies, 365 00:25:28.359 --> 00:25:30.920 these libraries and so on and so forth. An AI 366 00:25:31.079 --> 00:25:36.559 application is not two different. It's basically saying that, oh, 367 00:25:36.680 --> 00:25:41.559 I'm using the same application, the same dependencies. I'm using 368 00:25:41.960 --> 00:25:45.279 the inference engine x y Z, and then I'm using 369 00:25:45.279 --> 00:25:49.000 the model weights that are coming from here. I'm using 370 00:25:49.000 --> 00:25:52.960 a data set that was used in that is coming 371 00:25:53.039 --> 00:25:57.720 from here. And these are all in an s bomb 372 00:25:57.799 --> 00:26:03.799 which existing systems know how to pars and get results from. 373 00:26:04.200 --> 00:26:06.839 And we know how to sign s bomps, we know 374 00:26:06.960 --> 00:26:11.799 how to store s bomps, you know CI artifacts. So 375 00:26:13.079 --> 00:26:16.680 I think that's kind of you know, is the missing 376 00:26:16.720 --> 00:26:21.200 link nowadays where we are doing these things in our enterprises, 377 00:26:22.799 --> 00:26:28.680 adopting models based models from somewhere, and none of the 378 00:26:28.799 --> 00:26:31.440 data sets that we are using is enough. So some 379 00:26:32.160 --> 00:26:37.000 one of our engineers just goes to Google and searches 380 00:26:37.000 --> 00:26:40.200 for a data set and uses that to you know, 381 00:26:40.319 --> 00:26:43.400 do some of the training. You don't know what is 382 00:26:43.440 --> 00:26:45.720 in that data set. So it's it's a little bit 383 00:26:45.759 --> 00:26:50.559 wild west at the moment with the AI adoption. So 384 00:26:51.200 --> 00:26:56.680 and when it was wild west for the regular applications, 385 00:26:57.200 --> 00:27:02.359 you know, we we ended up with things like, uh, 386 00:27:02.480 --> 00:27:05.920 the we ended up with things like what was the 387 00:27:06.359 --> 00:27:13.599 look for j right? So how many how many days 388 00:27:14.119 --> 00:27:17.000 any Joba they will prespend trying to figure out if 389 00:27:17.000 --> 00:27:19.440 they have used the wrong version of look for Jack 390 00:27:20.039 --> 00:27:25.920 when that came out right, So, uh so things happened. 391 00:27:26.759 --> 00:27:30.720 Uh and I think we need to be prepared for it. 392 00:27:30.799 --> 00:27:33.240 And it's a little bit wild wild West at the 393 00:27:33.279 --> 00:27:47.519 moment with the with the I amount. 394 00:27:41.319 --> 00:27:43.960 It's you know, as we talk about this more, it 395 00:27:44.079 --> 00:27:48.039 kind of feels like there is it's almost like a 396 00:27:48.119 --> 00:27:52.720 communication problem. One of the things you said was knowing 397 00:27:52.759 --> 00:27:55.720 the lineage of the data and with when it comes 398 00:27:55.799 --> 00:27:59.759 to these AI models, we have a very human feeling 399 00:28:00.119 --> 00:28:04.240 interface to it, and so we I think we naturally 400 00:28:05.319 --> 00:28:09.799 try to leverage that. But the difference being, if if 401 00:28:09.839 --> 00:28:14.400