1 00:00:01,080 --> 00:00:04,799 How'd you like to listen to dot net rocks with no ads? Easy? 2 00:00:05,360 --> 00:00:09,439 Become a patron for just five dollars a month. You get access to a 3 00:00:09,480 --> 00:00:14,240 private RSS feed where all the shows have no ads. Twenty dollars a month. 4 00:00:14,279 --> 00:00:18,440 We'll get you that and a special dot net Rocks patron mug. Sign 5 00:00:18,519 --> 00:00:35,000 up now at Patreon dot dot NetRocks dot com. Hey guess what, it's 6 00:00:35,039 --> 00:00:39,159 dot net rocks. I'm Carl Franklin and I'm Richard Campbell here for another hour 7 00:00:39,200 --> 00:00:43,320 of Meret Gordon Nurse. Yes, because we were talking about meat before the 8 00:00:43,359 --> 00:00:46,640 show. Yes, yes, we were, as you do. So you 9 00:00:46,679 --> 00:00:51,240 know, every time we end a show and it's like solid awesome, which 10 00:00:51,280 --> 00:00:54,920 is most of them, Richard always says, well, that was an hour 11 00:00:55,000 --> 00:01:03,679 of media goodness. Well, and we're digging into AP making APIs better, 12 00:01:03,759 --> 00:01:06,120 then this is going to be all medi goodness for sure. Oh, it's 13 00:01:06,159 --> 00:01:08,959 totally gonna be all medi goodness. Anthony Alaibe is here. We're going to 14 00:01:10,000 --> 00:01:12,280 talk to him in a few minutes. But first I have some more goodness 15 00:01:12,400 --> 00:01:26,599 from Microsoft for better no framework awesome. So I just saw this as a 16 00:01:26,640 --> 00:01:37,280 blog post from Dan Roth. Yeah, introducing this experimental U tool or feature 17 00:01:37,400 --> 00:01:45,680 components called smart components, and these are essentially AI wrapped edit controls. So 18 00:01:46,000 --> 00:01:49,920 one of them is a smart paste, so it's takes the place of an 19 00:01:49,920 --> 00:01:53,200 input textbox and by smart pace. You know, if you go to a 20 00:01:53,239 --> 00:01:56,799 form and it's got all these fields name, address, phone number YadA, 21 00:01:56,840 --> 00:02:00,920 YadA, YadA, credit card number YadA, well you could copy, like 22 00:02:00,280 --> 00:02:07,680 text from notepad or something that has all those details arranged in paragraphs or whatever, 23 00:02:07,879 --> 00:02:10,599 copy and then paste it into the form, and poof, everything goes 24 00:02:10,639 --> 00:02:15,800 in the right place. Nice. Are you interested? I am? This 25 00:02:15,840 --> 00:02:19,599 is really cool. No, no, you know, this is what I 26 00:02:19,639 --> 00:02:22,960 figured all along. As these technologies mature, we're just going to incorporate them. 27 00:02:22,960 --> 00:02:24,400 We're not going to make apps about them. We're just going to incorporate 28 00:02:24,400 --> 00:02:28,960 them in our apps exactly. And that's what I love about this. It's 29 00:02:28,960 --> 00:02:36,039 a great it's a great application of a high technology to make every day ordinary 30 00:02:36,120 --> 00:02:39,680 things easier. Absolutely, and that's what They also have a text area, 31 00:02:40,319 --> 00:02:45,599 so it will do sort of you put some details of things. You can 32 00:02:45,639 --> 00:02:50,400 train it very quickly on just some sentences that are in the style of and 33 00:02:50,439 --> 00:02:53,879 then it will suggest to you like autocomplete, you know, like Gmail does 34 00:02:53,919 --> 00:02:58,479 when you're but with real details. And then there's also a combo box, 35 00:02:59,199 --> 00:03:04,960 so when you're pasting components, it will pick the right value from a dropdown 36 00:03:05,039 --> 00:03:07,680 combo box if one is there. Nice little things like that. So this 37 00:03:07,719 --> 00:03:12,879 is all good. So we're gonna link to it. It's eighteen ninety one 38 00:03:12,960 --> 00:03:17,680 dot pop, dot me, or you could just search for dot net smart 39 00:03:17,680 --> 00:03:22,919 components. Daniel Roth and Steve Sanderson does a great demo. And by the 40 00:03:22,919 --> 00:03:28,240 way, he is getting really animated lately in his videos, like he's turning 41 00:03:28,280 --> 00:03:31,800 into like Joe actor, like he I was really like a used car salesman, 42 00:03:31,919 --> 00:03:37,120 but not quite as not quite as catchy. But he's really selling it. 43 00:03:37,639 --> 00:03:40,520 Oh wow, that's interesting because he's only so reserved. He's subdued most 44 00:03:40,560 --> 00:03:46,319 of the time. I think that was Glenn Block, but yes, yeah, 45 00:03:46,360 --> 00:03:49,960 he is a very reserved guy. But he's really excited about this and 46 00:03:50,479 --> 00:03:53,159 it's good, such a good idea, it's very cool, fantastic. So 47 00:03:53,520 --> 00:03:58,199 eighteen ninety one dot pop, dot me, go check it out. Who's 48 00:03:58,240 --> 00:04:01,639 talking to us Richard gradyment of Show eighteen o two. So I went back 49 00:04:01,680 --> 00:04:05,240 a bit that's July of twenty two when we were at ANC Lennon. We 50 00:04:05,360 --> 00:04:10,840 talked to Victoria al Mazova, who we should talk to again because she's so 51 00:04:10,919 --> 00:04:15,599 fishit and smart. That was the whole measuring of dev secops, which is 52 00:04:15,599 --> 00:04:17,279 a bit of a non sequity of this comment from Rob Garner, one of 53 00:04:17,279 --> 00:04:21,600 our longtime listeners. Hey Rob, thanks for commenting again, even though middlely 54 00:04:21,600 --> 00:04:26,160 a couple of years ago, because I love what he said here. He 55 00:04:26,160 --> 00:04:29,720 says, Okay, someone has to say it. It's bound to result in 56 00:04:29,759 --> 00:04:32,839 the loss of cool points. But as youre sir, seems like it'd be 57 00:04:32,879 --> 00:04:36,079 hard to use at times. Sometimes I don't think I was supposed to say 58 00:04:36,079 --> 00:04:42,240 that, but I'm saying it. And my example is functions. They're supposed 59 00:04:42,279 --> 00:04:45,920 to really the developer of the complexity of thinking about the structure of an API, 60 00:04:46,480 --> 00:04:49,399 to free you to just write code. And the Microsoft examples were great 61 00:04:49,480 --> 00:04:54,959 right up until you need to apply the repository pattern. Then you have to 62 00:04:55,040 --> 00:04:59,199 use the pendency injection. They discover you'd be better using the correct dependency approach 63 00:04:59,399 --> 00:05:02,600 for your version of functions or whether you're in or out of process, and 64 00:05:02,600 --> 00:05:05,079 then you have to dig through all the examples to figure out what you're doing, 65 00:05:05,120 --> 00:05:10,720 because the approach is change several sigiment ways for each version of Azure function. 66 00:05:10,800 --> 00:05:13,759 And at that point you should be asking yourself, is a function really 67 00:05:13,759 --> 00:05:15,839 what I want here? Yeah? You know, and because that's a code 68 00:05:15,839 --> 00:05:18,439 smell right well. And it's the thing that made me jump, was like 69 00:05:18,600 --> 00:05:23,800 not thinking about the structure API, because you should always think about the structure 70 00:05:23,879 --> 00:05:26,199 API. Yeah. I think we're going to talk about that a lot today. 71 00:05:26,279 --> 00:05:29,639 I think so. But Rob, you know, you said the thing 72 00:05:29,639 --> 00:05:31,279 we're not allowed to say out loud. And part of this is just the 73 00:05:31,360 --> 00:05:35,720 twitching of Azure right Like Microsoft still figuring out how to deliver these things to 74 00:05:35,800 --> 00:05:39,519 us too, which makes it really makes it tough to write code against it 75 00:05:39,519 --> 00:05:43,319 because you don't know what's going to happen next. And I don't know if 76 00:05:43,439 --> 00:05:46,240 I'm put if I'm making that API available to more than just me, I 77 00:05:46,279 --> 00:05:49,920 need to think a lot about making it right and what that looks like. 78 00:05:50,240 --> 00:05:53,360 Indeed, So Rob, thank you so much for comment. To copy of 79 00:05:53,439 --> 00:05:55,439 music code by is on its way to you. And if you'd like a 80 00:05:55,480 --> 00:05:58,120 copy of music Code buy, I write a comment on the website at Donna 81 00:05:58,240 --> 00:06:00,759 rocks dot com or on the facebooks. We publish it show there, an 82 00:06:00,759 --> 00:06:02,079 if you comment there and I read on the show, will send you copy 83 00:06:02,120 --> 00:06:04,560 of us to go by, and you can follow us on ex Twitter, 84 00:06:04,600 --> 00:06:08,399 Twitter x or whatever the hell they're calling it these days. But the cool 85 00:06:08,439 --> 00:06:12,639 kids are hanging out on mastadon. I'm at Carl Franklin at techob dot social, 86 00:06:12,720 --> 00:06:15,920 and I'm Rich Campbell at mastadon dot social, and of course you can 87 00:06:15,959 --> 00:06:18,600 find all the ways to get in touch with me at Carl Franklin dot com. 88 00:06:19,279 --> 00:06:24,680 All right, let's introduce Anthony. This is his first time on dot 89 00:06:24,720 --> 00:06:29,160 net rocks, so let me give them the formal introduction. Anthony Alaibe has 90 00:06:29,160 --> 00:06:33,600 spent over a decade building software at companies like Opera, Parody, Polka Dot, 91 00:06:33,680 --> 00:06:39,519 and Delivery Hero. He has faced his fair share of dealing with unreliability 92 00:06:39,560 --> 00:06:46,040 and breaking changes and APIs, including losing over two million dollars in orders to 93 00:06:46,199 --> 00:06:51,439 such API issues at Delivery Hero, which prompted him to build API toolkit dot 94 00:06:51,480 --> 00:06:58,560 io to help companies navigate their back end and API landscape. He has an 95 00:06:58,600 --> 00:07:01,519 academic background in medicine and it is currently based in Berlin. When he's not 96 00:07:01,519 --> 00:07:06,360 writing code, you can find him binging an anime series or thinking about how 97 00:07:06,399 --> 00:07:13,120 principles from the medical world can help us build software better. So from the 98 00:07:13,160 --> 00:07:19,319 medical world, So you're saying we should measure software using the metric system. 99 00:07:19,879 --> 00:07:24,600 Obviously the rest of the world does you guys need to catch up. Hey, 100 00:07:24,920 --> 00:07:29,480 I didn't start the fire, Okay, I just live here, make 101 00:07:29,519 --> 00:07:31,800 the freaking rules. In fact, I got an A on my metric test 102 00:07:31,879 --> 00:07:45,759 in sixth grade in moose Humper. Okay, silliness. So, yeah, 103 00:07:45,800 --> 00:07:50,399 what did you think of the comment that Richard wrote about functions being I think 104 00:07:50,399 --> 00:07:55,079 I only read it. I didn't read it. Yeah, you wrote it 105 00:07:55,079 --> 00:07:59,519 with your mouth kind of. I made noises about it. It's gonna be 106 00:07:59,519 --> 00:08:05,360 one of those Yeah ahead, Yeah, I mean I think Richard said it 107 00:08:05,600 --> 00:08:09,759 said a nice right, we want to use functions functions as much as possible. 108 00:08:09,879 --> 00:08:16,079 You're small constrained, But then soon you want to access it that's a 109 00:08:16,160 --> 00:08:20,319 base or you want to access access to the dependencies, and it doesn't work 110 00:08:20,319 --> 00:08:28,319 out anymore because yeah, you need dependencies somehow. Pretty soon you're just holding 111 00:08:28,319 --> 00:08:33,200 on to functions. Is for the purpose of having a function exactly, but 112 00:08:33,440 --> 00:08:39,279 I mean so at the code level, I think when we can, you 113 00:08:39,320 --> 00:08:43,600 know, functions can be pretty or I think they might preferred way you know, 114 00:08:43,720 --> 00:08:50,080 to go when I can, just because they're constrained, you know exactly 115 00:08:50,080 --> 00:08:54,080 what goes in here. You don't expect so much magic, especially if it's 116 00:08:54,159 --> 00:08:58,200 like, you know, some kind of pure simple function. You know, 117 00:08:58,279 --> 00:09:03,480 it's like maybe not gonna reach out to data days without extra Also, I 118 00:09:03,480 --> 00:09:07,240 find that functions are best used when they're used a lot, so they don't 119 00:09:07,279 --> 00:09:11,120 go to sleep. Yeah you know what I mean, Because when a function 120 00:09:11,200 --> 00:09:13,480 goes to sleep and the next user comes around and wakes it up, they 121 00:09:13,519 --> 00:09:18,919 gotta they might take the wrap is broken. Yeah, I what fund did 122 00:09:18,919 --> 00:09:26,360 I be talking about? I think Azure functions. Functions? Yeah, okay, 123 00:09:26,360 --> 00:09:31,559 okay, I was talking about beyond the functions somehow. Yeah, I'm 124 00:09:31,600 --> 00:09:35,799 sorry, that's fine. So, yeah, we've done a bunch of shows 125 00:09:35,840 --> 00:09:39,480 around the whole concept of the modular monolith, and to me, it's like 126 00:09:39,080 --> 00:09:43,600 the function is where you put the the badly behaved piece of your monolith. 127 00:09:45,480 --> 00:09:48,279 That it's the one that's being called all the time, and every time it 128 00:09:48,320 --> 00:09:54,159 needs to change, it resonates the whole uh block a day of app like, 129 00:09:54,679 --> 00:09:56,000 so you want to put it in its own wrap, or it's gonna 130 00:09:56,039 --> 00:10:01,399 always be active or most of the time active. You need to scale independently 131 00:10:01,440 --> 00:10:05,279 of everything else, Like why why I have to light up more instead of 132 00:10:05,320 --> 00:10:09,440 app service when you could just have this noisy thing running in it in a 133 00:10:09,519 --> 00:10:15,840 function exactly. Also when you want to when you're already dip in an ecosystem, 134 00:10:15,960 --> 00:10:22,120 for example, when you want to you want to react to events, 135 00:10:22,360 --> 00:10:28,360 say, you know, like streaming events, do something and do something else 136 00:10:28,360 --> 00:10:33,279 and trigger something. Gain action functions are very good for that because they just 137 00:10:33,320 --> 00:10:37,240 get caught when that event happens, especially if it's an event that doesn't happen 138 00:10:39,360 --> 00:10:43,519 constantly. You know. Yeah, hey, can you tell us the delivery 139 00:10:43,559 --> 00:10:52,039 hero story? Because I love hearing about people losing millions of dollars. I 140 00:10:52,240 --> 00:10:56,840 know, I think the delivery hero story is not that. I think it's 141 00:10:56,840 --> 00:11:01,559 a story most of us have experience, maybe you know, at different scales, 142 00:11:01,039 --> 00:11:07,039 and what happened was that? Wait? Wait, what is delivery hero? 143 00:11:07,159 --> 00:11:09,519 Because I don't know. This is a popular in the United States, 144 00:11:09,679 --> 00:11:15,720 right, but elsewhere this is like this uber eats kind of thing. Yeah, 145 00:11:16,000 --> 00:11:20,000 So deliver Hero is this company of companies. That's why the brand is 146 00:11:20,039 --> 00:11:24,399 not as popular. But it's a company that owns I would say, most 147 00:11:24,440 --> 00:11:28,919 of the food delivery brands outside of China. So if you think of Global 148 00:11:30,200 --> 00:11:33,080 or food Panda, you know, depending on where you are food, but 149 00:11:33,159 --> 00:11:39,639 they only deliver Hero sandwiches. No, most of the brands that are owned 150 00:11:39,639 --> 00:11:43,639 by deliver here, Like there's an online pizza brand and I think they only 151 00:11:43,960 --> 00:11:50,759 deliver pizza. But then the other regular Uber eats style right brands? Yeah, 152 00:11:50,360 --> 00:11:52,639 yeah, and I see, like this is not North America, this 153 00:11:52,759 --> 00:11:58,759 is South America, Europe, Africa, Middle East, Southeast Asia. Exactly. 154 00:12:00,519 --> 00:12:03,320 Interesting, So what were the API issues that lost two million dollars? 155 00:12:05,039 --> 00:12:11,000 That specifically was a migration, so it had to migrate. We were writing 156 00:12:11,080 --> 00:12:15,559 a service, so we're writing just taking out a little bit of a monolith 157 00:12:16,120 --> 00:12:22,639 to be a separate service. And this service was the sech and discovery experience 158 00:12:22,720 --> 00:12:26,600 where you could set for food for most of these apps and get the list 159 00:12:26,639 --> 00:12:33,519 of dishes. And so we implemented this. So my team then we implemented 160 00:12:33,559 --> 00:12:39,080 this service like you would write, you know, we we tried to get 161 00:12:39,120 --> 00:12:45,360 the specification of the endpoint that we're migrating. But you know, with legacies 162 00:12:45,679 --> 00:12:50,399 services that documentation might not always be the best, so well, that's a 163 00:12:50,519 --> 00:12:58,759 very polite way to put it. So yeah, so we of course did 164 00:12:58,840 --> 00:13:01,919 the manual a way as well, which would be that you you know, 165 00:13:03,000 --> 00:13:07,679 copy the request and maybe difits use that as as the infut to implement this 166 00:13:07,759 --> 00:13:13,879 new service. And when it was time to roll out, we even went 167 00:13:13,960 --> 00:13:20,639 to step further that we would we run both of them on staging and you 168 00:13:20,679 --> 00:13:24,159 know, differing, so when a request comes, to send it to the 169 00:13:24,279 --> 00:13:28,879 legacy service and also send it to the new service, and then put requests 170 00:13:28,879 --> 00:13:31,320 together to make sure that you know, we was doing exactly what was right. 171 00:13:31,360 --> 00:13:35,000 So you're you're literally just white boxing this. You can't see what the 172 00:13:35,039 --> 00:13:39,000 original code is. You're just saying, if I do am I getting the 173 00:13:39,039 --> 00:13:41,559 same outputs for the same input on the old one as the new one, 174 00:13:43,919 --> 00:13:50,399 exactly. So this the old code was really it just it's just some I 175 00:13:50,440 --> 00:13:52,600 don't like you said the word legacy, but it was one of these services 176 00:13:52,639 --> 00:13:58,320 where most of the original authors were no longer in the company. They're gone, 177 00:13:58,440 --> 00:14:03,320 Yeah, yeah, they're gone. Totally normal exactly, and it's difficult 178 00:14:03,360 --> 00:14:09,159 to you know, to get dip into the weeds properly. But even with 179 00:14:09,200 --> 00:14:15,159 this care, defering all the requests, writing as much tests as possible, 180 00:14:15,759 --> 00:14:20,639 when we went to production, thinks they went crazy, you know, So 181 00:14:20,720 --> 00:14:26,559 did you know right away there was a problem or did it just was losing 182 00:14:26,639 --> 00:14:30,639 data missing things? So when we went to production, obviously we you know, 183 00:14:30,879 --> 00:14:35,720 we have the deployments checklists, you know, check that things are still 184 00:14:35,759 --> 00:14:41,720 working as expected in as many countries as possible, but we didn't check every 185 00:14:41,720 --> 00:14:43,840 country because we were perating were at the time we were put it in fifty 186 00:14:43,879 --> 00:14:50,000 five countries. You know, we're not gonna test fifty five countries. Yeah, 187 00:14:50,960 --> 00:14:54,960 And so what happened was that in scandy In some of these countries, 188 00:14:56,000 --> 00:15:03,000 specifically in Scandinavia, they have a leg lego we requirements that food delivery companies 189 00:15:03,159 --> 00:15:09,480 have to include the dietary requirements of any dishes that they offer. Right, 190 00:15:09,639 --> 00:15:13,799 none of us knew about this requirements, Like, I had no idea requirements 191 00:15:13,879 --> 00:15:20,720 what requirements? So are these nutritional nutrition values all that kind of stuff, 192 00:15:20,120 --> 00:15:26,679 exact cast whatever, and also allergies and things like that. Okay, yes, 193 00:15:26,919 --> 00:15:31,080 but this was a requirement only in Scandinavia. The other countries couldn't curt 194 00:15:31,200 --> 00:15:35,039 us. So everything worked perfectly in the other countries of course. So you 195 00:15:35,120 --> 00:15:39,879 just you just turned off. So did it fail in Scandinavia or did you 196 00:15:39,000 --> 00:15:46,279 not provide the data or did you get sued? Yes, we're just just 197 00:15:46,639 --> 00:15:54,799 crashing in Scandinavia, so wine exactly. So a couple one hundred thousand people 198 00:15:54,799 --> 00:16:03,279 didn't get their Ludificit deliveries. Yeah, So that that's basically what happened. 199 00:16:03,360 --> 00:16:07,200 And it seemed like a selly thing, you know, just a couple of 200 00:16:07,240 --> 00:16:11,679 missing fields. We had pretty good monitoring set up we have, you know, 201 00:16:11,720 --> 00:16:17,399 we try to do things carefully with this migration, but that did not 202 00:16:17,840 --> 00:16:21,559 prevent us from running into this trouble and maybe start thinking, you know, 203 00:16:22,600 --> 00:16:26,679 why can't we automate catching you know, breaking changes or any kind of issues. 204 00:16:27,799 --> 00:16:30,200 So that is basically what happened. You know, we most of us 205 00:16:30,240 --> 00:16:34,799 have been there, so yeah, yeah, I know the pain. Maybe 206 00:16:34,799 --> 00:16:38,360 not that much money, but I know the pain. Yeah, we have 207 00:16:38,399 --> 00:16:42,080 to be very careful, do you like that's the whole point, Like, 208 00:16:42,200 --> 00:16:47,879 be more careful next time. Isn't a good strategy? Like we should be 209 00:16:47,960 --> 00:16:52,200 able to not be careful and we can't do harm. You know, what's 210 00:16:52,200 --> 00:16:56,240 a good idea is to have a lunch and pass around the Arizona missions insurance 211 00:16:56,279 --> 00:17:00,320 policy that your company has, and it tells you what you're on the hook 212 00:17:00,399 --> 00:17:04,200 for. And you know, typically in one of those policies, it'll list 213 00:17:04,279 --> 00:17:11,759 right there failure to prevent a denial of service attacks for example. So if 214 00:17:11,759 --> 00:17:18,400 you're publishing an API and you don't have any kind of throttling on that API, 215 00:17:18,599 --> 00:17:23,599 you could be sued for one of your downstream customers having a denial of 216 00:17:23,640 --> 00:17:30,480 service attack. How about that? I'm just shocked at Jillie. That is 217 00:17:30,039 --> 00:17:33,079 in a lot of contracts. Well, it's in an insurance policy. The 218 00:17:33,160 --> 00:17:37,200 kinds of things that it would cover. Yeah, well, I mean first 219 00:17:37,200 --> 00:17:42,640 problem with reading insurance policies is staying awake. Yeah. The second problem is 220 00:17:42,799 --> 00:17:48,119 it will just strike you with fear at all the possible risks. You know, 221 00:17:48,160 --> 00:17:49,759 but that's a good thing to know, Like, what what are the 222 00:17:49,799 --> 00:17:55,519 things that the insurance companies are concerned about in terms of aris and omissions that 223 00:17:55,640 --> 00:18:00,039 you should kind of you know, think about. Yeah, not the litigating 224 00:18:00,119 --> 00:18:04,640 is ever is likely to provide a useful outcome. Now I don't want to 225 00:18:04,640 --> 00:18:10,799 avoid that. Yeah, winning a lawsuits like winning an earthquake, right, 226 00:18:10,880 --> 00:18:14,960 Like, there's no good outcomes there, right, So that the key is 227 00:18:15,119 --> 00:18:18,359 to not get suited in the first place. Yeah, try and try and 228 00:18:18,400 --> 00:18:22,000 work this out. So is was this the story that led you towards API 229 00:18:22,119 --> 00:18:29,559 toolkit? Yeah, it is precisely this. But you know, all through 230 00:18:29,799 --> 00:18:34,079 our careers, we've we've faced a lot of these little issues that kind of 231 00:18:34,119 --> 00:18:38,440 stack up in the back of our minds before even a delivery. Here. 232 00:18:38,599 --> 00:18:44,519 I used to work in a at a basically a digital bank, you know, 233 00:18:44,680 --> 00:18:48,440 and with these digital banks we have, we still have to work with 234 00:18:48,519 --> 00:18:55,880 dead parties. M So the dead parties could be other aps like MasterCard or 235 00:18:55,880 --> 00:18:57,839 other banks. In this case, it was in Nigeria, so we had 236 00:18:57,839 --> 00:19:04,279 to communic get with other bands and the national into band settlement system. And 237 00:19:04,640 --> 00:19:10,559 it was very common that, you know, these tech parties would either just 238 00:19:10,799 --> 00:19:15,880 go down or they would just introduce breaking changes. It could be something as 239 00:19:15,880 --> 00:19:19,119 simple as we're naming a field, you know, maybe someone named the field 240 00:19:19,200 --> 00:19:25,400 messages before and then some new engineer says, why is this messages it's always 241 00:19:25,440 --> 00:19:30,759 one message and then changes it to a message harmlets from his perspective. But 242 00:19:30,039 --> 00:19:33,680 you know, no one was prepared for that. I worked on a project 243 00:19:33,720 --> 00:19:40,079 once where the architect had every field be an array instead of just a single 244 00:19:40,160 --> 00:19:45,839 value. In the event that someday down the road, perhaps perhaps you might 245 00:19:45,880 --> 00:19:52,160 want to pass two of those values instead of one. Yeah, that was 246 00:19:52,160 --> 00:19:56,160 a lot of fun, to be fair, We don't know the trauma or 247 00:19:56,240 --> 00:20:02,119 that experience before that made him do that. Yeah, that's that's but you 248 00:20:02,160 --> 00:20:06,359 know, there's there's preparing for trauma, and then there's just common sense being 249 00:20:06,400 --> 00:20:11,119 agile, right, all right, So we haven't really talked about what API 250 00:20:11,200 --> 00:20:15,279 toolk io is. If you go the API toolk io, it says AI 251 00:20:15,440 --> 00:20:22,559 powered API observability, quality assurance, and insights. So give us the elevator 252 00:20:22,559 --> 00:20:26,000 pitch there. Yeah, that's a lot of words, but the whole idea 253 00:20:26,039 --> 00:20:30,559 is, you know, we already have a lot we have our monitoring platforms 254 00:20:30,920 --> 00:20:37,160 already, but can't we go step lower you the most monitoring platforms would catch 255 00:20:37,319 --> 00:20:41,839 several issues, maybe some logs and any errors you put in your lugs, 256 00:20:41,920 --> 00:20:47,720 maybe, but do we go step for them, actually analyze the payloads you 257 00:20:47,759 --> 00:20:53,039 know, will requests body and response body that your customers are making to you, 258 00:20:53,200 --> 00:20:57,680 but also that you are making to tear party integrations. And it is 259 00:20:59,319 --> 00:21:03,559 and the reason is that and not every error is going to be an error 260 00:21:03,559 --> 00:21:07,079 in your log or an exception that you can catch. Some errors could just 261 00:21:07,119 --> 00:21:14,400 be you know, missing data, something was renamed, or an error that 262 00:21:14,480 --> 00:21:17,920 is coming from a dependence. I can give a story about this dependency because 263 00:21:17,920 --> 00:21:26,440 there's something that happened. A customer had to deal with a library. You 264 00:21:26,480 --> 00:21:30,200 know, it is simple date library, you know, and this date library 265 00:21:30,400 --> 00:21:41,839 was created by an American, so but the diffault but it diffaulted you know, 266 00:21:41,920 --> 00:21:47,359 the US date formats, which is I think it's month day and then 267 00:21:47,440 --> 00:21:52,720 yeah, and there was this very long conversation in this on this library jub 268 00:21:52,839 --> 00:21:57,079 that hey, we should use the international default of the day a month year. 269 00:21:57,200 --> 00:22:00,440 And it went on for I don't know, maybe with a for years. 270 00:22:00,960 --> 00:22:03,240 Then the author gave in and said, okay, let's make this change. 271 00:22:03,920 --> 00:22:10,240 And there was I think a six months depplication. Notice it's just a 272 00:22:10,240 --> 00:22:14,319 live you know, everyone no one's reading the git hop issues of all the 273 00:22:14,400 --> 00:22:21,839 libries they use. So when this update happened. Eventually, it caused subtle 274 00:22:22,599 --> 00:22:27,799 changes were for some customers the date field, you know, was now like 275 00:22:30,119 --> 00:22:37,200 the international dates type. But that is a hard thing to identify it because 276 00:22:37,240 --> 00:22:42,599 some days of the year are basically the same on both the US date and 277 00:22:42,680 --> 00:22:51,839 the international and like fest of General eleven eleven ten exactly. I'm really surprised 278 00:22:51,839 --> 00:22:56,759 that out of that meeting didn't come the solution, which is to separate the 279 00:22:56,839 --> 00:23:02,480 date into three arrays, one with the day, one with the month, 280 00:23:02,799 --> 00:23:11,000 one with the ear because that's clearly the solution that might have walked you, 281 00:23:11,960 --> 00:23:18,559 to be honest. But in any case, it was a breaking change and 282 00:23:18,839 --> 00:23:22,480 it causes company quite some money. So it was not the kind of change 283 00:23:22,480 --> 00:23:26,640 that you would blame a developer. You know, maybe some developer was working 284 00:23:26,680 --> 00:23:32,200 on a completely different task, decided to update some dependencies and now there's an 285 00:23:32,240 --> 00:23:36,119 incident that he has no idea where it came from. So those are the 286 00:23:36,160 --> 00:23:38,799 kind of issues I was thinking, can we catch them at the level? 287 00:23:40,920 --> 00:23:44,440 Wow? Yeah, IO eighty six O one exists for a reason. It's 288 00:23:44,559 --> 00:23:48,960 year, month, day just at least, you know, however you want 289 00:23:48,000 --> 00:23:52,279 to display it after that, knock yourself out story. It is year, 290 00:23:52,359 --> 00:23:59,000 month, day, right, what could happen? Yeah, it's a classic 291 00:23:59,000 --> 00:24:03,720 words. So you feel like you could catch this from at an API level 292 00:24:03,839 --> 00:24:07,599 to say, hey, this API is measuring up dates. Now it's doing 293 00:24:07,640 --> 00:24:12,680 something down with dates exactly. So what what what we're doing is, it's 294 00:24:12,799 --> 00:24:18,319 imagine a middleware on your service that every time a request comes into your service, 295 00:24:18,359 --> 00:24:22,240 it just takes a copy, so it doesn't really impact your run time 296 00:24:22,400 --> 00:24:26,359 or whatever. It just copies it and then you know, sends it somewhere 297 00:24:26,519 --> 00:24:30,640 and that in that somewhere, which is a kid m we're like just learning. 298 00:24:30,640 --> 00:24:34,160 We build a model of the API, so we learn what all the 299 00:24:34,240 --> 00:24:41,079 fields are, whether all the different permutations of fields that are possible on this 300 00:24:41,160 --> 00:24:44,160 API. You know what at the type is this field supposed to be a 301 00:24:44,279 --> 00:24:48,880 uu i D or is this some random number? And with this model a 302 00:24:48,880 --> 00:24:53,640 lot of things become possible. For example, if this representation changes, if 303 00:24:53,680 --> 00:24:56,400 some field used to be a uu i D and all of a sudden it's 304 00:24:56,440 --> 00:25:00,440 just a number or it's a text, right, and someone gets notified and 305 00:25:00,480 --> 00:25:04,000 say, hey, this fields changed. Was it what you intended? If 306 00:25:04,000 --> 00:25:10,039 it was what you intended, then just acknowledge it, but sometimes it's a 307 00:25:10,039 --> 00:25:12,920 bug and someone can fix it. But I like the idea of you just 308 00:25:14,079 --> 00:25:18,240 snap this onto the system and let it run for a month and it should 309 00:25:18,279 --> 00:25:22,240 build up a pretty good catalog of all the shape of API calls out there. 310 00:25:22,279 --> 00:25:25,799 Like in a month, you're going to figure out if you misunderstood the 311 00:25:25,839 --> 00:25:30,720 day, feel exactly exactly. And it's like it's it's open to the developers 312 00:25:30,720 --> 00:25:33,960 because you can just go. You can go and see all your endpoints. 313 00:25:34,279 --> 00:25:37,079 Pick one of the endpoints and see all the different ships. You know, 314 00:25:37,599 --> 00:25:41,759 at the end of the day, most endpoints don't have quite so many ships. 315 00:25:42,240 --> 00:25:47,279 You know, maybe there's a lot of different status codes and then different 316 00:25:47,319 --> 00:25:49,359 responses. I'm sure eighty percent of them are fine. You know, they're 317 00:25:49,400 --> 00:25:52,559 all the same, and maybe it's even ninety five percent of the fine. 318 00:25:52,640 --> 00:26:04,319 It's all the edges that matter. Sometimes you've got Scandinavians that requirements just to 319 00:26:04,400 --> 00:26:07,880 highlight that, hey, there's an additional chunk of data in anything. Going 320 00:26:07,920 --> 00:26:12,240 to these end points would have been enough flag for you at least look to 321 00:26:12,400 --> 00:26:17,000 know, oh that's something weird there. We should go look there. Yeah, 322 00:26:17,599 --> 00:26:22,079 appreciate, So we basically did that, and along the way we realized, 323 00:26:22,119 --> 00:26:26,839 oh, now we're analyzing and building this model, there's much more we 324 00:26:26,839 --> 00:26:30,559 can do, for example, So when I started to think, okay, 325 00:26:30,559 --> 00:26:34,839 what what can we do? For example, we can generate open API specifications 326 00:26:34,880 --> 00:26:40,960 from this model that we have now. And the next step is we can 327 00:26:41,319 --> 00:26:49,480 generate simplistic h kind of API tests. You know, because you understand the 328 00:26:49,640 --> 00:26:53,680 users journey on the back end, sure you can say okay, use our 329 00:26:53,720 --> 00:26:59,319 logs in. Then it does. I can just generate some tests that can 330 00:26:59,359 --> 00:27:03,359 now be auto make it and say, okay, make this call, make 331 00:27:03,400 --> 00:27:07,160 this synthetic monitor call every one hour, you know, just behave like a 332 00:27:07,279 --> 00:27:12,960 user and check that everything is as it should be. That's basically it. 333 00:27:14,000 --> 00:27:17,279 So it's the It's the thing I wish we had at delivery here. Yeah. 334 00:27:17,559 --> 00:27:18,880 Yeah, absolutely, if it existed, you probably would have got that. 335 00:27:18,920 --> 00:27:23,680 I guess the question is the overhead Can I can I feel it when 336 00:27:23,680 --> 00:27:27,559 this thing's turned on as a number of requests per second go down just because 337 00:27:27,599 --> 00:27:32,319 you this is going to cost me something, it will cost you something, 338 00:27:32,839 --> 00:27:41,079 and that's something depends on the technology stock. So if you're using a stock 339 00:27:41,200 --> 00:27:45,599 like PhD, the challenge with PhD is that it's it's single process. Every 340 00:27:45,640 --> 00:27:51,039 request is processed in a single process, right, so we can't just you 341 00:27:51,079 --> 00:27:53,440 know, just do something behind the scenes, although we try and use a 342 00:27:53,480 --> 00:27:56,880 lot of different techniques. But if you're using phper used to pain anyway, 343 00:27:57,039 --> 00:28:07,519 right, But but you know the other technology stacks, you'd almost you would 344 00:28:07,559 --> 00:28:11,640 notice. It's like copying of request happens in nane a seconds, micro sequence 345 00:28:11,960 --> 00:28:18,519 max. The sending of the data. It's like if you have any luck, 346 00:28:18,559 --> 00:28:21,079 you know, any telemetry, it's basically the same. You know, 347 00:28:21,119 --> 00:28:25,759 it's just a synchronous maybe you get you spend a little bit more memory. 348 00:28:26,119 --> 00:28:30,279 I mean, I just like this from an observability perspective to see, Okay, 349 00:28:30,319 --> 00:28:33,079 well what do you see about what my APIs are up to? Like 350 00:28:33,160 --> 00:28:37,960 we've talked in the past about Azure API management, but that's mostly about you 351 00:28:37,960 --> 00:28:41,279 know, something that Carl mentioned earlier on which was going and yeah, can 352 00:28:41,319 --> 00:28:47,000 you deal with throttling when a runaway process is calling it fifty times a second? 353 00:28:47,400 --> 00:28:51,359 Certainly the most popular feature. So the unilease say that one bad actor 354 00:28:51,400 --> 00:28:56,039 can't knock everybody else down. Yeah, no, you guys are doing content 355 00:28:56,079 --> 00:28:59,759 analysis, which I think is a much trickier problem. Yeah, exactly. 356 00:29:00,039 --> 00:29:03,160 We can't go on an Arctic proxy unfortunately, so we can't we can't trottle 357 00:29:03,279 --> 00:29:08,839 anyone. That's a different thing. Use APR management for that. Yeah, 358 00:29:08,880 --> 00:29:12,960 exactly. Hey, guys, holding that thought for just a minute, while 359 00:29:14,000 --> 00:29:17,440 we take a moment for these very important messages. We'll be right back, 360 00:29:21,680 --> 00:29:23,799 and we're back. You're listening to dot net Rocks. I'm Carl Franklin. 361 00:29:23,839 --> 00:29:27,680 That's my friend Richard Campbell. Hey, and that's my new friend Anthony aler 362 00:29:27,759 --> 00:29:37,200 Rebe. And yeah, he's talking about APIs and observability and the AI part 363 00:29:37,319 --> 00:29:41,079 of this, which is really content analysis as he was just talking about. 364 00:29:41,759 --> 00:29:47,799 I'm really interested in how you use your medical training to write better software. 365 00:29:52,559 --> 00:29:57,640 That's that's an interesting question. Something that I that I have been thinking about 366 00:29:59,279 --> 00:30:03,839 that I think could learn from the medical world is s OPS or against We 367 00:30:04,039 --> 00:30:08,359 like to call it everyone's books in our industry, but in medicine, there 368 00:30:08,440 --> 00:30:15,599 is an ESPN s P Standard Operating prosidure, So there's a standard operating procedure 369 00:30:15,680 --> 00:30:18,880 for pretty much everything. You know, if you if your doctor is gonna 370 00:30:19,119 --> 00:30:22,559 open your hearts, he's not gonna be like, oh, I'm feeling crative 371 00:30:22,640 --> 00:30:33,079 today. Let's just go for it idea. Yeah right, I remember just 372 00:30:33,119 --> 00:30:37,200 opening from the down from from down this time, but no, let's go 373 00:30:37,319 --> 00:30:44,920 through the button transplant. Yeah. So, but it doesn't mean there is 374 00:30:44,960 --> 00:30:48,920 no creativity happening in the medical you know, there still is. But there 375 00:30:48,960 --> 00:30:56,240 is creativity happens with constraints. You know, when all the the common problems 376 00:30:56,279 --> 00:31:00,240 are solved, then you can like you know, you can get like small 377 00:31:00,279 --> 00:31:06,079 things at the time. And actually, like in software, we're always inventing 378 00:31:06,160 --> 00:31:11,039 the same pop the same things over and over and over. There's a lot 379 00:31:11,039 --> 00:31:15,200 of blog posts. If I want to create an up today, it might 380 00:31:15,240 --> 00:31:19,559 be different from how I created an up last morning. Well, and I 381 00:31:19,559 --> 00:31:23,559 do think we have that problem in software where no two projects are ever the 382 00:31:23,599 --> 00:31:27,480 same. One could be excused by, hey, the stacks are advancing, 383 00:31:27,519 --> 00:31:30,839 but the other is that we are prone to wanting to play with the new 384 00:31:30,880 --> 00:31:34,160 toys. So you know, the opportunity to go, oh, I want 385 00:31:34,160 --> 00:31:37,400 to play with this stack now, which not many. You know, that's 386 00:31:37,480 --> 00:31:41,880 kind of service to us rather than service to the customer. The rather to 387 00:31:41,960 --> 00:31:45,000 go with the stack. I already knew as well. Instrumented. You know, 388 00:31:45,079 --> 00:31:48,880 has the pipeline set up for just resists. It's okay to work within 389 00:31:48,880 --> 00:31:52,200 the same stack. There's innovations that can happen inside there. Well said, 390 00:31:53,160 --> 00:31:57,440 I think that is basically it that you know we have as engineers. We 391 00:31:57,480 --> 00:32:02,759 want to expect events, want to try new things. But if we can 392 00:32:04,119 --> 00:32:07,799 just agree on some standard I don't know, base layers, you could move 393 00:32:07,880 --> 00:32:14,640 that innovation step further. So maybe instead of everyone creating one more and plus 394 00:32:14,680 --> 00:32:19,240 one framework that is I don't know, almost the same as the other, 395 00:32:19,559 --> 00:32:22,960 we can start I don't know, innovating at I don't know, you know, 396 00:32:23,119 --> 00:32:27,799 some of the things that are actually different, like our business logics or 397 00:32:27,839 --> 00:32:31,319 the problems you really want to solve our performance. Maybe apps can finally be 398 00:32:31,400 --> 00:32:36,880 faster and we can only innovate on how to make things faster, et cetera. 399 00:32:37,759 --> 00:32:42,359 But it's something that I'm thinking about. Have you seen more maybe a 400 00:32:42,480 --> 00:32:47,640 question for you, like in your experience, how much have you seen one 401 00:32:47,680 --> 00:32:54,039 books being used in internal organizations? I mean more? Uh? And you 402 00:32:54,079 --> 00:32:59,680 know, and when I hear run books, I think chef right? Is 403 00:32:59,720 --> 00:33:04,400 there really the ones who popularized that net names? As I said, A 404 00:33:04,519 --> 00:33:08,319 cid CD pipelines. There's a bunch of different approaches, but it's automation. 405 00:33:08,759 --> 00:33:15,880 It's about don't open a word document, right, don't it's code. H 406 00:33:16,079 --> 00:33:20,279 You know, today we look at GitHub actions or as your DevOps, like 407 00:33:20,400 --> 00:33:24,160 in the in the dot net world, where often just checking the code in 408 00:33:24,319 --> 00:33:30,519 accepting the PR kicks off the process that begins all those integrations, decks, 409 00:33:30,519 --> 00:33:35,640 the rum book is activated, and I definitely see more and more developers. 410 00:33:35,720 --> 00:33:37,680 Was like, I don't want to think about this. I want to get 411 00:33:37,720 --> 00:33:40,160 as far as it worked on my machine, it passed my unit tests, 412 00:33:40,640 --> 00:33:45,480 I push it up as a PR, we go through the review process and 413 00:33:45,559 --> 00:33:49,880 the rest isn't my responsibility. The whole team took care of that. And 414 00:33:49,960 --> 00:33:53,759 I think there's a natural cycle that devs go through or dev teams where something 415 00:33:53,759 --> 00:33:58,960 new comes out and it promises to you know, be a little more complex 416 00:33:59,039 --> 00:34:01,200 upfront, but then save you time down the road, right, And this 417 00:34:01,279 --> 00:34:06,960 happens over and over again, and most devs resist that because they don't want 418 00:34:06,960 --> 00:34:10,599 to go through the initial pain of learning something new and changing moving your cheese 419 00:34:10,639 --> 00:34:15,400 and you know, changing their routine. But then you know, then they 420 00:34:15,480 --> 00:34:20,440 see somebody else, just like going at one thousand miles an hour and things 421 00:34:20,519 --> 00:34:23,320 are just happening, you know, Like the first time we interviewed Mark Rendel 422 00:34:23,719 --> 00:34:28,719 Richard and he told us all about like all these crazy processes that he had 423 00:34:28,760 --> 00:34:30,679 in automation that he was doing, and it was all manual and stuff. 424 00:34:31,039 --> 00:34:34,519 And the first my first thought was, oh my god, I'm never going 425 00:34:34,599 --> 00:34:37,159 to do that. But then I thought about, wow, the you know, 426 00:34:37,320 --> 00:34:45,000 from thought to implementation to publishing can just be instantaneous, and how powerful 427 00:34:45,039 --> 00:34:46,840 is that? Right? And then you know, after a while, these 428 00:34:46,880 --> 00:34:51,960 devs adopt these things, but it takes a while. I have customers that 429 00:34:52,159 --> 00:34:58,880 are still you know, living without source control. Oh man, can you 430 00:34:58,920 --> 00:35:05,239 believe that? And I have to educate them and how to use GitHub before 431 00:35:05,800 --> 00:35:09,239 before we can even start working. I found that interesting. Just like you 432 00:35:09,320 --> 00:35:15,880 said, I've seen that run books are most popular in c I CIT you 433 00:35:15,920 --> 00:35:19,760 know, or in deployments. You know, let's have a run book for 434 00:35:19,800 --> 00:35:24,880 how to roll out the new service. And because we have those run books, 435 00:35:25,400 --> 00:35:30,480 a lot of evolution happened, like like anty Ball and optio, you 436 00:35:30,519 --> 00:35:36,480 know, having Docker for example. Just the benefit we have in our industries 437 00:35:36,519 --> 00:35:38,440 that if a lot of if things are done over and over, it might 438 00:35:38,480 --> 00:35:43,920 be possible to just you know, box it up into Docker or into some 439 00:35:44,000 --> 00:35:50,840 library or something. But I I feel like it can be run books, 440 00:35:51,039 --> 00:35:54,920 or the idea of run books could be applied even further to more things, 441 00:35:55,440 --> 00:36:00,480 you know, And I mean, I'm not going to pretend can do the 442 00:36:00,559 --> 00:36:07,440 how. But assuming we had more maybe an industry wide agreed upon run books 443 00:36:07,960 --> 00:36:13,079 for a lot of things like maybe how do I beld X y Z or 444 00:36:13,079 --> 00:36:19,480 how do I you can onboard even the more junior developers and you can trust 445 00:36:19,519 --> 00:36:23,519 them to do quite a lot of task without being afraid because they're really just 446 00:36:23,559 --> 00:36:31,119 following a run book and then allowed the creativity just happen. Maybe in creating 447 00:36:31,159 --> 00:36:35,559 new run books for example, are we dancing around? The main thing here 448 00:36:35,639 --> 00:36:40,679 is which is part of that pipeline is testing, right in automated testing, 449 00:36:40,840 --> 00:36:45,320 so that the junior can't write a piece of code that's going to cause a 450 00:36:45,320 --> 00:36:50,159 problem because the test is going to kick it back. You're not you know, 451 00:36:50,320 --> 00:36:52,960 I think too many people think about the pipeline as just deploy this. 452 00:36:52,000 --> 00:36:58,519 For me, don't think right, it's really just the continuous deployment part and 453 00:36:58,559 --> 00:37:01,199 the only part of a continuous integration that's in there is compile it, which 454 00:37:01,519 --> 00:37:08,559 really real integration is is this code suitable to be integrated? I very much 455 00:37:08,639 --> 00:37:13,199 agree with that. Yeah, well, I think the tool your building here 456 00:37:13,360 --> 00:37:16,480 is about I have collected data for how this API has behaved in the past, 457 00:37:16,480 --> 00:37:20,280 so I can generate the set of tests for you that become part of 458 00:37:20,280 --> 00:37:24,199 the pipeline. So if your new code doesn't comply with what we know is 459 00:37:24,239 --> 00:37:30,320 the current way that the API behaves, it's not getting integrated. Yeah, 460 00:37:30,360 --> 00:37:34,760 exactly, it is exactly that you know that we can catch. And if 461 00:37:34,760 --> 00:37:38,360 you consider how as developers we always know the ideal, we know what we 462 00:37:38,400 --> 00:37:43,199 should be doing. We know we should all be writing tests. But every 463 00:37:43,199 --> 00:37:45,400 time I speak with a developer, I ask them, you know, like, 464 00:37:45,920 --> 00:37:50,679 how confident are you in all the tests you've written? Do you feel 465 00:37:50,719 --> 00:37:55,320 like the software you maintained right now is perfectly tested? It always feels like 466 00:37:57,719 --> 00:38:05,400 anybody who says yes is lying, right, like the the that's the old 467 00:38:05,480 --> 00:38:09,119 are you a good man? Anybody says you question, it's like the correct 468 00:38:09,119 --> 00:38:14,119 answer is I'm not sure, Like arguably it's not up to you, right, 469 00:38:14,480 --> 00:38:20,199 are you perfectly tested almost certainly. Not am I sufficiently tested? Perhaps, 470 00:38:20,800 --> 00:38:24,800 Like there's no absolutes here, just just for the just for as much 471 00:38:25,639 --> 00:38:30,039 for as I have. You know, at least I'm tested enough for the 472 00:38:30,119 --> 00:38:35,239 last incidents that we experienced. Yeah, I'm not likely to fail on the 473 00:38:35,239 --> 00:38:38,599 thing that knocked the system down last week. I'm not even saying certain about 474 00:38:38,639 --> 00:38:44,840 that. I'm not likely to fail on that. I was a betting man, 475 00:38:45,119 --> 00:38:47,559 Yeah, I kind of. I took it on the head for that 476 00:38:47,559 --> 00:38:51,679 one already. And I think I've written the test that that that would catch 477 00:38:51,760 --> 00:38:53,239 that before it gets deployed. Now, well, let's go to the next 478 00:38:53,239 --> 00:38:57,840 one. Let's see what happens again. I think the most important thing here 479 00:38:58,000 --> 00:39:02,960 is this idea that stated period of monitor of observability with this tool builds a 480 00:39:04,000 --> 00:39:07,960 test suite for you, because who likes writing tests? And then and here's 481 00:39:07,960 --> 00:39:12,639 a joke, like I write those tests, then we rev the API, 482 00:39:12,880 --> 00:39:15,039 so the tests are wrong and we got to go again. Right, So 483 00:39:15,079 --> 00:39:20,639 the idea that you could regenerate them after that is pretty compelling to me. 484 00:39:20,840 --> 00:39:22,960 Like that's a bunch of work nobody really wants to do in the first place. 485 00:39:23,159 --> 00:39:29,320 Yeah, Actually, this idea. Half of it came from something we 486 00:39:29,360 --> 00:39:32,920 did at Delivery Hero, which is we used to have this lot test that 487 00:39:34,000 --> 00:39:37,440 we would like write, you know with like case six, you know, 488 00:39:37,519 --> 00:39:43,880 write some general tests to show I don't know, a million requests a minute 489 00:39:43,880 --> 00:39:51,239 at the seven right, But they were always wrong because we had very good 490 00:39:51,559 --> 00:39:55,199 cushion, so pretty much all the steps would just get cushion and the great 491 00:39:55,639 --> 00:40:05,320 I don't just want. We got better results when we actually recorded, like 492 00:40:05,840 --> 00:40:13,000 there are lots of what reques production work exactly, and then just replayed that 493 00:40:13,159 --> 00:40:17,440 well, And that's because humans are weirder than testers. I mean, testers 494 00:40:17,480 --> 00:40:22,519 are pretty weird, but humans are weirder still, Like, yeah, they 495 00:40:22,559 --> 00:40:27,039 do stuff you couldn't even imagine to think of. Just think of little Bobby 496 00:40:27,079 --> 00:40:32,199 tables. Here you go. No, my favorite one was working on a 497 00:40:32,199 --> 00:40:37,639 site that was barely hanging on, like it was overwhelmed by the load and 498 00:40:37,679 --> 00:40:40,239 the reaction. And part of this problem was that the customers when the first 499 00:40:40,320 --> 00:40:44,440 page wind load, opened another window and tried again, and opened the window, 500 00:40:44,519 --> 00:40:47,360 tried again, and so it's like one ip looks like it's own DDoS 501 00:40:47,400 --> 00:40:52,360 attack and you get multiplied by ten thousand of those people and you you know, 502 00:40:52,440 --> 00:40:55,760 you designed your tests where each ip made one request, one page was 503 00:40:55,840 --> 00:41:04,480 and that is just not what was happening exactly us Just yeah, no, 504 00:41:05,440 --> 00:41:09,559 this this recording the workload and then being able to play, you know, 505 00:41:09,679 --> 00:41:20,039 a Saturday afternoon football tournament workload a bad one. And you would hope that 506 00:41:20,079 --> 00:41:23,480 whatever workload you run includes Scandinavian countries. I think that's gonna be important. 507 00:41:24,159 --> 00:41:30,800 Hopefully maybe that week all of Scandinavia. I was like, they were on 508 00:41:30,920 --> 00:41:35,519 vacation. You missed it. Yeah, they need to get their lutafisk on 509 00:41:35,679 --> 00:41:40,000 time. Can we talk about pricing here? I'm looking at pricing and there's 510 00:41:40,039 --> 00:41:45,159 a free uh tier, a plus tier for twenty dollars a month, I 511 00:41:45,199 --> 00:41:51,239 think yeah, and an enterprise for police call and so for free you get 512 00:41:51,400 --> 00:41:55,719 what twenty thousand requests per month, build monthly, one team member on the 513 00:41:55,800 --> 00:42:00,119 last fourteen days of data retained. So you can get aready checking this out 514 00:42:00,159 --> 00:42:07,440 without without you know, any pain. Why wouldn't you? So how do 515 00:42:07,519 --> 00:42:10,239 you hook this in? Like? Is this code I compile into my app 516 00:42:10,280 --> 00:42:14,559 and deploy it or is this a service? What does this look like? 517 00:42:15,519 --> 00:42:20,320 Exactly? It isn't middleware that you you know, you just added line in 518 00:42:20,360 --> 00:42:23,480 your code when we started out. You know, there's been a lot of 519 00:42:23,519 --> 00:42:29,360 movement in this e BPF space, and that was originally what I was thinking 520 00:42:29,360 --> 00:42:32,440 about, but then I realized that, you know, if you want to 521 00:42:32,480 --> 00:42:37,119 really help develop us, we want to give them as much context as possible. 522 00:42:37,239 --> 00:42:40,079 Sure, and that context includes things like, you know, when when 523 00:42:40,079 --> 00:42:45,480 this customer made this request and it failed, these were the exceptions that happened 524 00:42:45,920 --> 00:42:51,480 right in the code at that time. And when we do things with the 525 00:42:51,480 --> 00:42:53,159 e b p F for out, we don't have that context. And what's 526 00:42:53,280 --> 00:42:58,599 in the code, you know, we have just the HDTP Yeah, you 527 00:42:58,639 --> 00:43:02,000 want to be in that process, so you see all those exceptions occur exactly. 528 00:43:02,519 --> 00:43:06,679 So, so it's a middlewhere you log, you put it in the 529 00:43:06,719 --> 00:43:10,360 code, and it copies the request. If there are exceptions of the errors, 530 00:43:10,440 --> 00:43:16,800 it attaches the errors that happen to the to that particular users. I 531 00:43:16,840 --> 00:43:20,199 don't know log you know, so you can see, okay, this user 532 00:43:20,360 --> 00:43:24,239 with this IP address made this request and got this response, but there was 533 00:43:24,280 --> 00:43:30,639 an error with this tackt traits and whatever that happened in the meantime and you 534 00:43:30,719 --> 00:43:34,800 picked. So that line of code is then an invocation to a service you're 535 00:43:34,840 --> 00:43:39,519 hosting on their behalf or could they run it themselves internally, So that line 536 00:43:39,559 --> 00:43:43,880 of code is India code. It's in your application, you know, and 537 00:43:43,920 --> 00:43:50,639 you can so what is it calling to? Like, it's it's there's an 538 00:43:50,639 --> 00:43:55,960 open source SDK for most languages, which what the KID does is basically copy, 539 00:43:57,119 --> 00:44:00,519 you know, get all of this information for some cost them as they 540 00:44:00,559 --> 00:44:04,840 have sensitive data which they don't want to even live there save us in the 541 00:44:04,880 --> 00:44:09,079 first place. So it's able to redact that sensitive data and then just put 542 00:44:10,159 --> 00:44:15,800 put the data on, just send it to API to KID. For some 543 00:44:15,960 --> 00:44:21,320 enterprise customers, they also just run it on prem in that case, so 544 00:44:21,360 --> 00:44:23,400 there is an on prem option if you want to pay for it. Exactly 545 00:44:23,480 --> 00:44:27,239 in that case, it is just a ducker image you're configuring and it just 546 00:44:27,280 --> 00:44:31,679 sends it there. That's great, And exactly what I was thinking is like 547 00:44:32,079 --> 00:44:36,039 it's in many cases. In some cases like I want to say, they'll 548 00:44:36,079 --> 00:44:43,119 leave my data center or my assurance then so it's contained and it had to 549 00:44:43,159 --> 00:44:47,880 happen. It had to happen because I built this, because I worked in 550 00:44:47,880 --> 00:44:52,360 a bank in the digital bank and you know half of the problem came from 551 00:44:52,360 --> 00:44:58,800 there. But digital but are not gonna trust well, they're under regulatory requirements, 552 00:44:58,840 --> 00:45:06,719 like they'll lose their license exactly. So what's next for you? What 553 00:45:06,760 --> 00:45:10,440 are you working on? What I'm working on is basically this is just making 554 00:45:10,639 --> 00:45:16,400 especially the things around testing. I'm trying to see what context we can get. 555 00:45:16,519 --> 00:45:21,280 And you know, it's like you have all of this data. Every 556 00:45:21,280 --> 00:45:22,960 time you look at you you're like, oh, I could do this. 557 00:45:24,320 --> 00:45:29,280 So it's just you know, how can we make the life of developments better 558 00:45:29,360 --> 00:45:32,599 with the data we already get in anyway? Yeah, wow, if you 559 00:45:32,639 --> 00:45:36,519 can help me write tests for my API so I don't break it in the 560 00:45:36,559 --> 00:45:39,159 next rev you're my friend. Yeah, this is good stuff. Yeah. 561 00:45:39,679 --> 00:45:43,880 And to start with a free tier just to see does this make sense for 562 00:45:43,960 --> 00:45:46,679 me? And it's like, yeah, your first HiT's free, you'll get 563 00:45:46,719 --> 00:45:52,679 addicted. Yeah. And I think probably the takeaway message from this interview is 564 00:45:52,719 --> 00:45:58,639 that this is not an API management replacement. This is content. You're you're 565 00:45:58,719 --> 00:46:02,400 you're testing content and you're looking for that and using AI for that. So 566 00:46:04,119 --> 00:46:07,599 thanks Anthony, it's been great. This is a great product. Thanks. 567 00:46:07,679 --> 00:46:12,760 Thanks go. Thank you're welcome, thanks for us beinging this hour with us, 568 00:46:12,800 --> 00:46:36,840 and we'll see you next time on dot net Rocks. Than dot net 569 00:46:36,960 --> 00:46:40,519 Rocks is brought to you by Franklin's Net and produced by Pop Studios, a 570 00:46:40,679 --> 00:46:45,760 full service audio, video and post production facility located physically in New London, 571 00:46:45,760 --> 00:46:52,840 Connecticut, and of course in the cloud online at pwop dot com. Visit 572 00:46:52,880 --> 00:46:55,119 our website at d O T N E t R O c k S dot 573 00:46:55,159 --> 00:47:00,519 com for RSS feeds, downloads, mobile apps, comments, and access to 574 00:47:00,559 --> 00:47:06,239 the full archives going back to show number one, recorded in September two thousand 575 00:47:06,280 --> 00:47:08,719 and two. And make sure you check out our sponsors. They keep us 576 00:47:08,719 --> 00:47:14,159 in business. Now go write some code, see you next time. You 577 00:47:14,280 --> 00:47:22,840 got jud Middle vans Man