WEBVTT 1 00:00:04.679 --> 00:00:07.879 Hey, folks, welcome back to another Ruby Rogues. 2 00:00:08.279 --> 00:00:11.839 This week on our panel we have a Yushinwatya Hello. 3 00:00:11.880 --> 00:00:12.320 Hello. 4 00:00:13.480 --> 00:00:18.559 We also have Valentino Stole and now I'm Charles Maxwood 5 00:00:18.559 --> 00:00:20.960 from Top Endevs. And this week we have a special 6 00:00:21.000 --> 00:00:21.920 guest and that's Emil. 7 00:00:22.039 --> 00:00:24.839 Come hey, everybuddy. 8 00:00:25.879 --> 00:00:30.679 So Emil, I'm just gonna clear my throat because wow, okay, 9 00:00:30.839 --> 00:00:31.160 let me. 10 00:00:31.079 --> 00:00:34.280 Try that again. So Emil, I found you. 11 00:00:34.640 --> 00:00:40.399 I've been corresponding with people some on the discord for Kamal, 12 00:00:41.119 --> 00:00:47.759 which is the deployment technology that DH put out. I 13 00:00:47.759 --> 00:00:49.840 think he made kind of a deal about it at 14 00:00:50.280 --> 00:00:52.920 Rails World last year, and then a whole bunch of 15 00:00:52.920 --> 00:00:56.840 exciting stuff this year with Kamal Proxy and stuff like that. 16 00:00:56.920 --> 00:00:59.719 I mean, I switched over to and I switched over 17 00:00:59.759 --> 00:01:01.560 from Mall to the Mall too, with all much of 18 00:01:01.560 --> 00:01:06.079 help from you and others on there. And I have 19 00:01:06.159 --> 00:01:11.719 to say switching over was a little rough. Having switched over, 20 00:01:11.840 --> 00:01:18.280 it's a lot nicer. So anyway, so beyond kind of 21 00:01:18.280 --> 00:01:21.200 what I know about you from that, what should people 22 00:01:21.239 --> 00:01:21.680 know about you? 23 00:01:23.239 --> 00:01:28.719 Well? So I think, first off, I am a mountaineer, 24 00:01:28.920 --> 00:01:33.959 and in order to facilitate that I live in a 25 00:01:34.040 --> 00:01:36.000 very flat country, so I do a lot of running, 26 00:01:36.120 --> 00:01:39.439 and then I need to pay for my travels, so 27 00:01:39.560 --> 00:01:44.319 I do also engineering. And I've made a career of 28 00:01:44.959 --> 00:01:49.719 doing sort of critical infrastructure or things that have real 29 00:01:49.760 --> 00:01:57.439 world impact and real world consequences if something goes wrong, right, 30 00:01:57.480 --> 00:02:01.159 So both in terms of security and robust list and 31 00:02:01.200 --> 00:02:06.640 things like that. And that's everything from the fintech or 32 00:02:06.640 --> 00:02:12.479 the payment industry, to civil engineering to online voting. 33 00:02:15.000 --> 00:02:15.439 Wow. 34 00:02:16.560 --> 00:02:19.280 Yeah, it sounds like kind of what our topic is here, 35 00:02:19.280 --> 00:02:23.000 which is building critical infrastructure. So you've done some stuff 36 00:02:23.000 --> 00:02:26.719 that I kind of need to work, I guess. 37 00:02:28.240 --> 00:02:32.080 I mean, so a banking obviously needs to work when 38 00:02:32.199 --> 00:02:35.240 people get sort of pissed if they don't get their money. 39 00:02:36.520 --> 00:02:42.520 And then for civil engineering, I worked on tools that 40 00:02:43.520 --> 00:02:49.560 the engineers would use to build the airplanes, which and 41 00:02:49.960 --> 00:02:52.800 run tests on them. Right, so both test benches and 42 00:02:52.840 --> 00:02:55.919 design benches for airplanes. And I mean that has a 43 00:02:56.080 --> 00:02:59.919 real world consequence because they fall out of the sky, 44 00:03:00.120 --> 00:03:05.159 right if you mess it up. Yeah. And then yeah, 45 00:03:05.199 --> 00:03:09.280 online voting, that's the whole thing. And you guys in 46 00:03:09.319 --> 00:03:13.240 the US just went through one one of these messy 47 00:03:13.280 --> 00:03:19.080 events and they're messy everywhere, right, And recently I moved 48 00:03:19.240 --> 00:03:26.000 to two compliance and sort of making trying to make it, 49 00:03:27.439 --> 00:03:30.800 trying to make that easier and a better experience to 50 00:03:31.199 --> 00:03:36.159 become compliant and to obtain certifications. 51 00:03:36.840 --> 00:03:40.719 Right, makes sense, So let's just dive in. 52 00:03:40.800 --> 00:03:44.919 I mean, I'm not I'm trying to think if I've 53 00:03:44.960 --> 00:03:48.039 ever worked on anything that was you know, kind of 54 00:03:48.080 --> 00:03:50.080 at the level of, hey, if this breaks. 55 00:03:49.800 --> 00:03:52.560 Then you know, really bad things happen. 56 00:03:54.520 --> 00:03:56.919 I mean, if it breaks, I've had them come to 57 00:03:56.960 --> 00:03:59.960 me and be upset, like really bad things happened. But right, 58 00:04:00.400 --> 00:04:05.879 it doesn't affect society at large. What's the difference between 59 00:04:05.919 --> 00:04:08.120 working on those apps and working on just kind of 60 00:04:08.120 --> 00:04:11.199 you'r you know you mentioned not e commerce or not 61 00:04:12.520 --> 00:04:14.840 you know, hey I've got to manage my social media 62 00:04:15.120 --> 00:04:17.360 or you know, get podcasts out on time. 63 00:04:18.319 --> 00:04:21.000 Yeah, I mean, there's nothing just to be here, There's 64 00:04:21.120 --> 00:04:26.319 absolutely nothing wrong with that, right, That's just I started 65 00:04:26.360 --> 00:04:30.240 out in that area, as many developers do, right, building 66 00:04:31.759 --> 00:04:34.480 like some ad system when I was very young, and 67 00:04:34.519 --> 00:04:39.240 then moved to some you know, sort of off a platform, 68 00:04:39.319 --> 00:04:42.360 and then I sort of realized that it was it 69 00:04:42.439 --> 00:04:47.519 was just inconsequential in a way, right, I looked at people, 70 00:04:48.319 --> 00:04:50.439 you know, I had people around me. My father was 71 00:04:50.920 --> 00:04:53.439 is a civil engineer, right, and so I have people 72 00:04:53.639 --> 00:04:56.519 in my sort of my family, my near family, that 73 00:04:56.680 --> 00:04:58.920 was building things in the real world. And I was like, 74 00:04:59.519 --> 00:05:01.680 I want to do that, but I do like this 75 00:05:01.839 --> 00:05:04.439 computer stuff, So can we sort of maybe find a 76 00:05:04.439 --> 00:05:06.839 way where I can build things in the real world 77 00:05:07.079 --> 00:05:11.600 just using my computer. And so I slowly started to 78 00:05:11.720 --> 00:05:16.319 shift over into something that was founded more in civil 79 00:05:16.360 --> 00:05:22.399 engineering and sort of industry rather than commerce mm hmm. 80 00:05:23.959 --> 00:05:30.519 And then I started in a fintech company where we 81 00:05:30.519 --> 00:05:34.319 were where I was part of building, building out and 82 00:05:34.360 --> 00:05:37.199 continuing to build on a on a system where you 83 00:05:37.279 --> 00:05:46.680 can basically it's like direct debit bank transfers, but worldwide, okay, 84 00:05:48.240 --> 00:05:52.720 so and and there. It doesn't necessarily mean a lot 85 00:05:52.759 --> 00:05:56.000 to you and me. And it's the same with there's 86 00:05:56.040 --> 00:05:59.040 a bunch of them now, right, like these send money 87 00:05:59.079 --> 00:06:02.319 home types of things. It doesn't mean a lot to 88 00:06:02.360 --> 00:06:04.439 you and me, but it does mean a ton to 89 00:06:04.519 --> 00:06:06.720 those who is on the receiving end of that, right, 90 00:06:07.959 --> 00:06:10.600 because that's like half a year's pay or something that 91 00:06:10.720 --> 00:06:15.199 comes in, right, So it does matter to those people 92 00:06:17.360 --> 00:06:19.720 so so yeah, that was sort of where I ended up. 93 00:06:19.759 --> 00:06:23.839 And we are moving a lot of money, like it 94 00:06:23.839 --> 00:06:26.839 it seems it's one of those where little like it's 95 00:06:26.879 --> 00:06:28.920 it's a lot of tiny things that add up. Right, 96 00:06:28.959 --> 00:06:31.639 There's like millions and millions and millions of dollars being 97 00:06:31.680 --> 00:06:36.439 moved around for that purpose, and you don't want to 98 00:06:36.959 --> 00:06:41.120 miss handle those money. Right. So you are hooking into 99 00:06:41.160 --> 00:06:45.680 the to the banking infrastructure, into Swift and into the 100 00:06:45.720 --> 00:06:51.959 IBAND network and the whole sort of PC compliance and 101 00:06:52.040 --> 00:06:55.879 everything with that, right, so you need to make sure 102 00:06:55.920 --> 00:07:01.279 that the software is robust well tap that testing has 103 00:07:01.319 --> 00:07:04.120 been a main thing of mind for a long time 104 00:07:04.199 --> 00:07:09.360 because of that, and also documented well. Right, So that's 105 00:07:09.399 --> 00:07:13.240 where I sort of started out with being required to 106 00:07:13.360 --> 00:07:15.839 document what I was doing and how it was supposed 107 00:07:15.839 --> 00:07:19.959 to work. Rather than just shipping fast and breaking things. 108 00:07:20.480 --> 00:07:25.759 I was required to test and document, then test, then 109 00:07:26.240 --> 00:07:29.600 stage it, then test some more, then ship and not 110 00:07:29.680 --> 00:07:35.800 break things. So that was a different paradigm. Yeah, so 111 00:07:35.879 --> 00:07:38.399 that's sort of where I first met that, right, And 112 00:07:38.439 --> 00:07:42.319 that was a pretty great experience, and so I was like, oh, 113 00:07:42.480 --> 00:07:45.279 can I have more of this? Can I maybe even 114 00:07:45.319 --> 00:07:50.560 get even closer? Right? Yeah? So that's uh, that's where 115 00:07:50.600 --> 00:07:54.000 all that started. And then I was hooked on the 116 00:07:54.000 --> 00:07:58.120 whole can we test it? Can we build something that 117 00:07:58.240 --> 00:08:02.120 runs Ruby on rails but make a difference to people's 118 00:08:02.160 --> 00:08:06.680 lives like in real world? Right, So this was basically 119 00:08:07.040 --> 00:08:13.680 a payment situation, just building Ruby and rails, and back 120 00:08:13.720 --> 00:08:17.079 then many of the banks didn't have a proper API, 121 00:08:17.199 --> 00:08:19.800 so we ended up scraping a lot and that's you know, 122 00:08:20.199 --> 00:08:22.720 against terms of policy, in terms of service and all 123 00:08:22.759 --> 00:08:25.199 of that, and they were really mad and we went 124 00:08:25.240 --> 00:08:29.040 through that whole spang of getting you know, sort of 125 00:08:29.920 --> 00:08:36.519 scolded for that for a while. Yeah, So that it 126 00:08:36.639 --> 00:08:39.080 was a bit the wild West back then, but it's 127 00:08:39.120 --> 00:08:42.279 definitely gotten better. And I think there's a lot of 128 00:08:42.639 --> 00:08:45.600 again sort of compliance work and lew work that went 129 00:08:45.679 --> 00:08:50.960 into the EU did a lot to make sure that 130 00:08:51.120 --> 00:08:56.559 banks would have not standardized, standardized APIs, but a minimum 131 00:08:56.679 --> 00:09:00.519 level of information that you could retrieve with an A, 132 00:09:00.840 --> 00:09:03.039 which made all of that a lot easier. 133 00:09:03.120 --> 00:09:03.320 Right. 134 00:09:04.159 --> 00:09:07.399 It also means that I can technically build my own 135 00:09:07.440 --> 00:09:10.039 banking app for my own bank account because I have 136 00:09:10.159 --> 00:09:13.240 API access to it. It may not be obvious, it 137 00:09:13.279 --> 00:09:16.919 may be poorly documented, but it is there, and so 138 00:09:16.960 --> 00:09:18.000 I think that's pretty cool. 139 00:09:19.879 --> 00:09:26.399 Yeah, I'm curious, Valentine or I usually have experienced doing 140 00:09:26.440 --> 00:09:27.000 this stuff. 141 00:09:28.879 --> 00:09:30.159 Nothing that critical. 142 00:09:30.240 --> 00:09:30.320 Now. 143 00:09:30.399 --> 00:09:32.440 The closest I kind of came to anyone near that 144 00:09:32.519 --> 00:09:34.919 world was when I worked for transfer Wise, But I 145 00:09:34.960 --> 00:09:38.960 was an iOS developer, so I didn't really have to 146 00:09:39.440 --> 00:09:41.960 do any of the hard stuff. I just made API calls. 147 00:09:42.000 --> 00:09:44.440 But I know they were also doing quite a lot 148 00:09:44.440 --> 00:09:46.639 of the scraping and stuff. There's a lot of selenium 149 00:09:46.639 --> 00:09:49.519 going on behind the head to get into different bank accounts. 150 00:09:49.799 --> 00:09:54.759 It's uh, yeah, it's it's all a bit scary when 151 00:09:54.799 --> 00:09:55.759 you think about it. 152 00:09:58.000 --> 00:10:01.039 Yeah, yeah, it really is, because it's like your whole 153 00:10:01.120 --> 00:10:05.080 banking experience is tied up on some scraper working correctly 154 00:10:05.120 --> 00:10:09.679 and not messing around with the the thousands delimiters and 155 00:10:09.759 --> 00:10:14.120 the the scent comments are different on different regions, right, 156 00:10:14.120 --> 00:10:16.559 so you don't want to mess around with that. I 157 00:10:16.600 --> 00:10:21.480 did transfer some money incorrectly. Luckily, the backbone banking system 158 00:10:21.559 --> 00:10:24.639 is pretty robust as well, so we were able to 159 00:10:24.679 --> 00:10:28.639 basically not move the money anyways, right, but it I 160 00:10:28.679 --> 00:10:38.000 did move far too many money the wrong place. Yeah, 161 00:10:39.120 --> 00:10:40.519 that was a back breaking day. 162 00:10:41.440 --> 00:10:41.879 I'll bet. 163 00:10:42.879 --> 00:10:45.320 So I'm a little curious as we get into this, 164 00:10:48.720 --> 00:10:51.320 Like you mentioned testing, what what is your what does 165 00:10:51.360 --> 00:10:52.679 your testing set up look like? 166 00:10:52.720 --> 00:10:55.360 Because I mean, I I'm sorry. 167 00:10:55.360 --> 00:10:58.120 When I write my test, it's like, okay, it's good 168 00:10:58.159 --> 00:11:01.320 at Like I'm I'm reasonably confident, right, but I'm not 169 00:11:01.399 --> 00:11:05.200 like iron clad confident, like baking level confident that my 170 00:11:05.320 --> 00:11:05.879 code works. 171 00:11:05.960 --> 00:11:06.120 Right. 172 00:11:07.480 --> 00:11:13.720 Yeah, yeah, that's a good question. I tend to not 173 00:11:14.159 --> 00:11:16.480 be too dogmatic. I know there's a lot of like 174 00:11:16.600 --> 00:11:19.000 do this or do that paradigm or do test first 175 00:11:19.120 --> 00:11:25.960 or whatever. I tend to just play around with it 176 00:11:26.039 --> 00:11:31.080 and make myself certain first that this is gonna work 177 00:11:31.120 --> 00:11:35.399 whatever I'm sitting with, and then when I know that 178 00:11:35.519 --> 00:11:37.279 it you know, sort if I can see like okay, 179 00:11:37.320 --> 00:11:40.200 it works, it does what I needed to do, then 180 00:11:40.240 --> 00:11:45.919 it is worth testing, right. And then it's I try 181 00:11:45.960 --> 00:11:50.879 and test on the lowest possible level first, which means 182 00:11:50.919 --> 00:11:55.159 I will start with something like unit tests, and for 183 00:11:55.279 --> 00:11:59.879 that I want to permute everything, right. You want to 184 00:12:00.039 --> 00:12:03.759 make sure that every argument, So you basically start with 185 00:12:03.840 --> 00:12:06.120 the least amount of arguments for a method, and then 186 00:12:06.159 --> 00:12:09.480 you add an argument slowly. In I use aspects, so 187 00:12:10.240 --> 00:12:13.399 it's in every context block or in every described block. Right, 188 00:12:14.080 --> 00:12:17.679 so you're you're certain that every version of every argument 189 00:12:18.200 --> 00:12:25.799 is is tested and has been permuted. Yeah, so that everything, 190 00:12:26.240 --> 00:12:32.120 like every variable has been tested on its own, so 191 00:12:32.200 --> 00:12:35.039 you know how it behaves both with and without that 192 00:12:35.080 --> 00:12:38.799 particular variable, and you also know how it behaves if 193 00:12:38.840 --> 00:12:41.840 the variable has the wrong type. 194 00:12:41.960 --> 00:12:42.159 Right. 195 00:12:44.080 --> 00:12:48.639 So because Ruby isn't quite where we have type safety yet. 196 00:12:50.159 --> 00:12:55.320 There's a lot of work going on with that, but 197 00:12:55.600 --> 00:12:58.879 we aren't there yet. So I, you know, you want 198 00:12:58.879 --> 00:13:06.200 to make sure that we can handle strings, numbers, not strings, truths, boollians, whatever. 199 00:13:06.639 --> 00:13:10.039 There's like I have a little thing that basically says, 200 00:13:10.600 --> 00:13:14.679 take this and just you know, cycle it through all 201 00:13:14.720 --> 00:13:17.240 of the different types it can be, and just put 202 00:13:17.279 --> 00:13:21.519 in garbage so that I know it can not it 203 00:13:21.559 --> 00:13:24.480 doesn't break when a number is presented rather than a 204 00:13:24.519 --> 00:13:29.639 string or something stupid like that. Right, And then you 205 00:13:29.759 --> 00:13:32.200 sort of you slowly move up so you're confident that 206 00:13:32.279 --> 00:13:34.080 this method works, and then you move up to have 207 00:13:34.120 --> 00:13:37.519 recalled it, and then you I what I do is 208 00:13:37.600 --> 00:13:41.360 I then I go through and test that. But I 209 00:13:41.399 --> 00:13:48.960 think opposed to what is DHH preaches or or at 210 00:13:49.039 --> 00:13:51.960 least sort of he he has this idea that you 211 00:13:52.000 --> 00:13:54.279 would go through the entire stack, and I believe that 212 00:13:54.519 --> 00:13:57.519 they had their tests to do that. I tend to 213 00:13:59.080 --> 00:14:02.279 not to sort of stop testing at each level, right, 214 00:14:02.320 --> 00:14:05.879 So model tests are their own thing. And then when 215 00:14:05.919 --> 00:14:11.600 you test the controller, you basically verify that the model 216 00:14:12.080 --> 00:14:15.799 is being called with the expected parameters, because you know 217 00:14:15.879 --> 00:14:18.480 you've already dealt with the parameters on a lower level. 218 00:14:20.000 --> 00:14:23.200 So it's not mocking per se. It's just that I 219 00:14:24.360 --> 00:14:28.519 only need to test that the model was created or 220 00:14:28.759 --> 00:14:31.240 was called. I don't really need to test that it 221 00:14:31.320 --> 00:14:35.200 was the correct arguments, right, because I'm certain about the 222 00:14:35.240 --> 00:14:37.759 correctness of the arguments on a lower level than that. 223 00:14:39.159 --> 00:14:42.440 And then you saw slowly move up, which means that 224 00:14:43.480 --> 00:14:47.080 I find that that means that the individual testing levels. 225 00:14:48.240 --> 00:14:50.320 It gives you a little bit more freedom to sort 226 00:14:50.320 --> 00:14:54.879 of move a model around because it's coupled somewhat on 227 00:14:54.919 --> 00:14:59.159 a test level, but not tremendously, right. I only coupled 228 00:14:59.200 --> 00:15:02.320 the controller the model in the sense that I checked 229 00:15:02.360 --> 00:15:05.039 that the model was called, not that it was called 230 00:15:05.039 --> 00:15:07.799 with a particular set of arguments. So it makes it 231 00:15:07.840 --> 00:15:11.159 a little bit easier to also restructure your tests when 232 00:15:11.159 --> 00:15:15.720 your code change. But it's just a lot of work 233 00:15:15.759 --> 00:15:19.639 on lower level things. I'm a big fan of Spey, 234 00:15:19.879 --> 00:15:24.200 but it's not it's not quite there yet. I think 235 00:15:24.240 --> 00:15:28.240 it works reasonably well, but it's very hard for developers 236 00:15:28.279 --> 00:15:30.799 to use, and so I think we need to do 237 00:15:30.879 --> 00:15:33.679 there needs to be a lot of work done in 238 00:15:34.000 --> 00:15:37.200 terms of making it nice and sexy to use so 239 00:15:37.240 --> 00:15:39.320 that we can get people to use it right. 240 00:15:41.159 --> 00:15:43.320 I'd love to dig in a little bit more on 241 00:15:43.360 --> 00:15:48.720 the aspect because one of the biggest things I see 242 00:15:50.360 --> 00:15:54.919 miss is maybe like skipping over like the maybe some 243 00:15:54.960 --> 00:15:58.240 lower level implementations and then like a system update break 244 00:15:58.320 --> 00:16:01.360 something and it just wasn't covered right, Like how do 245 00:16:01.440 --> 00:16:05.120 you go about, like you know, waiting whether or not 246 00:16:05.159 --> 00:16:08.279 that it's worth it to like actually run like a 247 00:16:08.360 --> 00:16:12.320 system command or an external API call or something like that. 248 00:16:13.679 --> 00:16:19.879 Yeah, So system commands, I usually check just that the 249 00:16:20.000 --> 00:16:25.480 signature of the call is correct and probably that it 250 00:16:25.559 --> 00:16:27.879 returned the right thing right, so that if I hit 251 00:16:27.919 --> 00:16:31.480 the OS, I'm basically testing that I gave it these 252 00:16:31.519 --> 00:16:35.960 parameters and I expected something back like a string or 253 00:16:36.039 --> 00:16:39.919 whatever it is right, because it's so hard to know 254 00:16:40.000 --> 00:16:44.200 what it was supposed to be because sometimes the system 255 00:16:44.240 --> 00:16:49.320 commands will change their signature but not necessarily return value 256 00:16:49.399 --> 00:16:51.679 or the return value, but not the signature. So that's 257 00:16:51.840 --> 00:16:59.840 that's very hard. I try to not go there. I 258 00:17:00.080 --> 00:17:02.559 tried really hard not to go there because it is 259 00:17:02.960 --> 00:17:07.680 it's out of my control. Right on the other end, 260 00:17:07.720 --> 00:17:11.559 when I called other APIs, I definitely test that I 261 00:17:11.680 --> 00:17:14.480 use something like me CR or something. I capture all 262 00:17:14.559 --> 00:17:17.599 the network traffic, and I make sure to do the 263 00:17:17.759 --> 00:17:22.160 same kind of sort of making sure that if I 264 00:17:22.200 --> 00:17:26.000 call it with these arguments that I have control over, 265 00:17:26.240 --> 00:17:31.079 I get this particular response, and if I do something 266 00:17:31.079 --> 00:17:33.680 else then then it breaks right or it gives me 267 00:17:33.720 --> 00:17:39.079 the incorrect response. And I also found that, for example, 268 00:17:39.160 --> 00:17:43.759 with with banking, that one of the cool things we 269 00:17:43.799 --> 00:17:48.839 did was that we basically took all of the historical 270 00:17:49.880 --> 00:17:53.839 records right that we had for all the bank accounts, 271 00:17:53.920 --> 00:17:57.119 and we basically used it as a test bed because 272 00:17:57.160 --> 00:18:00.119 then you can train on that. You can basically say, 273 00:18:00.359 --> 00:18:03.559 if I feeded all of this information that we know happened, 274 00:18:03.839 --> 00:18:06.480 and it happened correctly because it happened, you know, but 275 00:18:07.000 --> 00:18:11.359 a year ago or two or whatever, it should result 276 00:18:11.519 --> 00:18:17.160 in the in in a particular set of of like 277 00:18:17.240 --> 00:18:21.799 it should result in a particular value on the account. Right, 278 00:18:22.319 --> 00:18:24.640 So you can actually use historical data as a test 279 00:18:24.680 --> 00:18:28.160 that and that makes that means that you have a 280 00:18:29.000 --> 00:18:34.279 factual way to check if it's correct or not. So yeah, 281 00:18:34.440 --> 00:18:36.640 I tend to use that as well. I also used 282 00:18:36.680 --> 00:18:41.880 that when when I did software for the for the 283 00:18:42.119 --> 00:18:45.400 for the sort of civil engineering and building flights and 284 00:18:45.480 --> 00:18:48.960 testing that. Right, we also used a lot of existing 285 00:18:49.000 --> 00:18:53.799 information to make sure that we had input output, so 286 00:18:53.839 --> 00:18:56.559 we understood that given these inputs, this is the output 287 00:18:56.599 --> 00:19:01.279 that we would expect. Right, We could basically take old 288 00:19:01.400 --> 00:19:03.799 designs and try and run them through and you can 289 00:19:03.839 --> 00:19:07.559 see if does it break it down like you would expect, 290 00:19:08.279 --> 00:19:12.000 does it run the does it sort of fit to 291 00:19:12.160 --> 00:19:17.440 the engineering standard design standard that we were implementing against. 292 00:19:18.039 --> 00:19:21.240 They also have test and value that you can use, right, 293 00:19:21.319 --> 00:19:24.519 or test data. So it's also about going out and 294 00:19:24.599 --> 00:19:29.720 finding big test data sets that you can use to 295 00:19:30.039 --> 00:19:33.799 also make a statistical sort of approach. Right, I'm not 296 00:19:34.200 --> 00:19:37.960 making any statistical claims, but I am running a lot 297 00:19:38.000 --> 00:19:40.799 of tests on a lot of data that you know, 298 00:19:41.079 --> 00:19:46.039 you can be reasonably certain that that if there were 299 00:19:46.880 --> 00:19:49.200 errors in your system. It would have been caught by 300 00:19:49.400 --> 00:19:54.240 this because it's so much effective effective production data. Right. 301 00:19:55.039 --> 00:19:58.279 Yeah, I imagine that working with like the public sector, 302 00:19:58.319 --> 00:20:01.440 that you would have a lot of access to that data, right, 303 00:20:02.799 --> 00:20:04.000 I guess depending where you live. 304 00:20:05.359 --> 00:20:07.559 Yeah, I mean, I mean interesting enough. None of this 305 00:20:07.720 --> 00:20:12.000 is for the public sector, which is a little bit weird. 306 00:20:12.039 --> 00:20:14.000 You would think that it was, but it actually isn't. 307 00:20:14.000 --> 00:20:19.480 Like the regulation of the aerospace is public sector, right, 308 00:20:19.519 --> 00:20:22.839 but the actual building of airplanes is a private commerce, 309 00:20:22.920 --> 00:20:28.440 ran private industry. And likewise, with banking, the the regulation 310 00:20:28.559 --> 00:20:32.200 of it is public, but the actual banks are private entities, 311 00:20:33.680 --> 00:20:35.240 and likewise. 312 00:20:35.880 --> 00:20:39.000 We find it difficult for like the data access, like 313 00:20:39.160 --> 00:20:43.000 is that data proprietary then for like doing large swaths 314 00:20:43.039 --> 00:20:44.640 of tests or finding it. 315 00:20:47.000 --> 00:20:50.160 Yes, but most of the times the companies are interested 316 00:20:50.200 --> 00:20:53.079 in providing it, right, Like if you if you go 317 00:20:53.119 --> 00:20:58.720 out to big to big airplane or aerospace company and 318 00:20:59.200 --> 00:21:02.039 your unique to design a test system for them or 319 00:21:02.839 --> 00:21:05.480 something to help them build, right, they are interested in 320 00:21:05.519 --> 00:21:07.960 giving you that kinds of dace. So it wasn't that 321 00:21:08.000 --> 00:21:10.359 it was hard. It's just that it is also quite 322 00:21:12.359 --> 00:21:15.599 narrow in scope, Like it's it's more more or less 323 00:21:15.759 --> 00:21:16.799 only for that one. 324 00:21:16.880 --> 00:21:17.039 Right. 325 00:21:17.119 --> 00:21:21.680 So in banking, it would be a lot of records, 326 00:21:21.720 --> 00:21:24.480 but only for one bank, so you would only know 327 00:21:24.519 --> 00:21:28.000 how it looked in one type of account system, right, 328 00:21:28.039 --> 00:21:29.799 and then you would run over your tests and that 329 00:21:30.759 --> 00:21:44.960 and now two seconds all right, yeah, there we go. Hello. 330 00:21:48.440 --> 00:21:53.319 I So, so it's not that it's hard to get, 331 00:21:53.359 --> 00:21:57.359 it's that it is it's quite narrow in terms of 332 00:21:58.480 --> 00:22:01.200 sort of like what, Yeah, it doesn't have a lot 333 00:22:01.240 --> 00:22:03.200 of breadth. You get it for like one bank or 334 00:22:03.279 --> 00:22:07.759 one airplane company, or you get it for one company 335 00:22:07.759 --> 00:22:12.119 who runs elections. Right. Public elections is a bit different 336 00:22:13.680 --> 00:22:16.720 by definition. You don't have any test data because all 337 00:22:16.759 --> 00:22:20.079 of the records are sort of you don't know who 338 00:22:20.519 --> 00:22:24.640 who voted, so you can't know how to test that, right, 339 00:22:24.680 --> 00:22:27.160 So you have to do a different testing paradigm there. 340 00:22:30.119 --> 00:22:30.960 Yeah, that makes sense. 341 00:22:30.960 --> 00:22:34.440 I mean here in the US we have a anonymous 342 00:22:34.440 --> 00:22:37.880 ballot as well, right, yeah, And I don't know how 343 00:22:37.880 --> 00:22:38.759 they do it in other states. 344 00:22:38.799 --> 00:22:40.960 I mean I'm involved in the process here in Utah. 345 00:22:41.000 --> 00:22:45.680 And yeah, you send in your envelope and they with 346 00:22:45.880 --> 00:22:48.519 your ballot in it, and they validate that your like 347 00:22:48.559 --> 00:22:49.920 your signature matches and stuff. 348 00:22:49.920 --> 00:22:50.839 Off the envelope. 349 00:22:51.200 --> 00:22:53.839 But then they pull your ballot out of the envelope 350 00:22:53.880 --> 00:22:58.880 and put it in the pile to be counted, and right, 351 00:22:58.920 --> 00:23:02.440 they don't they track who you voted for in that case, 352 00:23:02.559 --> 00:23:04.640 at least they're not supposed to correct. 353 00:23:04.880 --> 00:23:07.720 And that is also the case for the digital solutions. 354 00:23:09.640 --> 00:23:13.000 A friend of a colleague of mine, Stefan, who is 355 00:23:13.039 --> 00:23:17.440 a cryptographer. He or previous colleague of mine, he did 356 00:23:17.440 --> 00:23:21.599 a lot of work of sort of gathering together a 357 00:23:21.599 --> 00:23:24.160 lot of the academic work so that we could exactly 358 00:23:24.440 --> 00:23:30.680 do that in a cryptographically safe way but online m 359 00:23:31.480 --> 00:23:32.880 and you could also do it in a way that 360 00:23:32.920 --> 00:23:35.079 it was you know, you could track it, and you 361 00:23:35.119 --> 00:23:38.799 could ensure that this was in fact the case, Like 362 00:23:38.839 --> 00:23:41.920 you could give the the lock data and the trace 363 00:23:42.079 --> 00:23:46.480 data to mathematicians and they could verify that mathematically. It 364 00:23:46.559 --> 00:23:49.799 could only ever have worked if these rules were followed, 365 00:23:50.880 --> 00:23:53.680 which was like you know, you anonymized, and you did 366 00:23:54.039 --> 00:23:55.359 you did mixing, and you did a. 367 00:23:55.279 --> 00:23:59.279 Lot of these things, right, right, So you protect the 368 00:23:59.319 --> 00:24:02.119 identity of the voter, but you also ensure that only 369 00:24:02.160 --> 00:24:06.720 people who are legally permitted to vote vote and that 370 00:24:06.799 --> 00:24:08.799 they're only cast one ballot and things like. 371 00:24:08.759 --> 00:24:13.200 That, Yeah, exactly, And the test, the test sort of 372 00:24:13.240 --> 00:24:17.680 the build idea for that and the testing regime for 373 00:24:17.759 --> 00:24:21.039 that is significantly different, right because this isn't sure the 374 00:24:21.039 --> 00:24:24.680 public space to your point earlier, Valentino, this is a 375 00:24:24.720 --> 00:24:28.839 public domain, right, so you you are a bit more scrutinized. 376 00:24:28.880 --> 00:24:33.440 You're also publicly scrutinized by everybody, right because we all 377 00:24:33.440 --> 00:24:38.720 know how it went with the dominion voting machine case, right, 378 00:24:38.799 --> 00:24:41.680 that went real fast, Will Downhill. 379 00:24:42.119 --> 00:24:47.279 Yeah, so yeah, well and you bring up the dominion 380 00:24:47.640 --> 00:24:51.880 voting machine case and whether you believe that they worked 381 00:24:51.960 --> 00:24:57.119 or didn't. There if there's enough population that doesn't trust it, 382 00:24:57.200 --> 00:24:59.319 then it causes problems, right, Yeah. 383 00:25:00.960 --> 00:25:05.039 And that's that's the same everywhere, right, And that's I 384 00:25:05.119 --> 00:25:09.200 found that sort of on a on a completely different 385 00:25:09.279 --> 00:25:14.039 but somehow related topic. I really found that the aerospace 386 00:25:14.319 --> 00:25:17.640 or the airplane industry was was really good because the 387 00:25:19.480 --> 00:25:24.880 what's the American one, the TSB Transportation Safety Board? Is 388 00:25:24.920 --> 00:25:28.920 that tis be right? Yeah, those are the ones that 389 00:25:28.920 --> 00:25:31.920 that they do crash audit that sort of like go 390 00:25:32.039 --> 00:25:39.720 through both them and the European counterparts have a really 391 00:25:39.920 --> 00:25:44.519 really cool way of going about it. They don't ever 392 00:25:44.759 --> 00:25:48.799 go and say, oh, it was that person's fault, right, 393 00:25:49.359 --> 00:25:54.440 if if the person hadn't slept enough, if the person 394 00:25:54.759 --> 00:25:58.720 couldn't read the dials, it is the fault of their 395 00:25:58.759 --> 00:26:02.519 plan that that made him not have enough hours to sleep, 396 00:26:02.559 --> 00:26:05.039 and it is the fault of the dial for not 397 00:26:05.079 --> 00:26:09.480 being visible enough. Right, So it's not They don't ever 398 00:26:09.640 --> 00:26:13.519 cast personal brain or they very rarely cast personal Blane. 399 00:26:13.880 --> 00:26:16.640 And I really thought that was a super cool way 400 00:26:16.720 --> 00:26:19.599 of going about it also just from a sort of 401 00:26:19.640 --> 00:26:24.680