WEBVTT 1 00:00:00.120 --> 00:00:04.080 Imagine this. It's four point four to five pm on 2 00:00:04.120 --> 00:00:04.639 a Friday. 3 00:00:05.080 --> 00:00:06.599 No, I already know where this. 4 00:00:06.559 --> 00:00:09.080 Is going, right. The office is starting to empty out. 5 00:00:09.400 --> 00:00:12.599 You've just finished up a shiny new feature for your 6 00:00:12.599 --> 00:00:13.519 company's web app. 7 00:00:13.560 --> 00:00:15.240 You're ready to just pack it up and go home. 8 00:00:15.439 --> 00:00:18.879 Exactly. You take a sip of your lukewarm coffee, take 9 00:00:18.920 --> 00:00:21.519 a deep breath, and you click deploy. You're ready to 10 00:00:21.519 --> 00:00:24.640 head into the weekend. And then, I mean less than 11 00:00:24.679 --> 00:00:27.600 ten seconds later, your stomach just drops. 12 00:00:27.320 --> 00:00:28.920 The absolute worst feeling in the world. 13 00:00:29.079 --> 00:00:33.439 Your data dog dashboards turn crimson red. Users are suddenly 14 00:00:33.439 --> 00:00:37.039 getting five hundred internal server errors everywhere, and your phone 15 00:00:37.079 --> 00:00:40.159 just starts vibrating off the desk with frantic Slack messages 16 00:00:40.159 --> 00:00:44.280 from the support team. It is like the Universal Developer nightmare. 17 00:00:44.479 --> 00:00:47.399 It really is a visceral kind of panic. That feeling 18 00:00:47.479 --> 00:00:50.000 of instant regret when the deployment goes wrong on a 19 00:00:50.000 --> 00:00:53.280 Friday afternoon is something I mean, almost everyone in software 20 00:00:53.280 --> 00:00:56.840 has felt at some point, right, and it's usually because 21 00:00:56.880 --> 00:01:00.119 they trusted a process that was well just inharantly Friday. 22 00:01:00.159 --> 00:01:00.679 To begin with. 23 00:01:00.840 --> 00:01:03.479 And that fragility is exactly what we are going to 24 00:01:03.520 --> 00:01:06.920 dismantle today. Today's deep dive is into a book called 25 00:01:06.959 --> 00:01:10.040 Deploying PHP Applications by Nicholas Modis. 26 00:01:10.040 --> 00:01:11.480 It's a great read, it is. 27 00:01:11.680 --> 00:01:14.799 And our mission for you today is simple. We are 28 00:01:14.840 --> 00:01:19.560 going to transform deployment from this terrifying, you know, manual 29 00:01:19.719 --> 00:01:25.280 chore into a smooth, automated, completely invisible machine. 30 00:01:25.599 --> 00:01:26.159 That's the goal. 31 00:01:26.680 --> 00:01:29.159 Motus actually has this really funny analogy in the book. 32 00:01:29.680 --> 00:01:33.159 He references the American no Child Left Behind act and 33 00:01:33.200 --> 00:01:36.159 he spins it into the no Application Left Behind act. 34 00:01:36.200 --> 00:01:38.920 I love that part because honestly, no app should be 35 00:01:39.000 --> 00:01:41.519 left behind just because of a bad deployment pipeline. 36 00:01:41.519 --> 00:01:44.159 You know, it's a crucial shift in mindset really. I mean, 37 00:01:44.480 --> 00:01:47.519 deployment is too often treated as just a peripheral afterthought, 38 00:01:47.719 --> 00:01:48.400 like a chore. 39 00:01:48.560 --> 00:01:49.040 Exactly. 40 00:01:49.280 --> 00:01:51.640 It's seen as this administrative hurdle right at the very 41 00:01:51.760 --> 00:01:53.799 end of the sprint. But in reality, it is the 42 00:01:53.840 --> 00:01:57.480 critical bridge between writing code and actually delivering value to 43 00:01:57.480 --> 00:01:58.120 the user. 44 00:01:58.120 --> 00:01:59.920 Right because if they can't use it, what's the point. 45 00:02:00.159 --> 00:02:00.640 Exactly? 46 00:02:00.840 --> 00:02:03.640 If that bridge is rickety, it doesn't matter how elegant 47 00:02:03.640 --> 00:02:06.159 the architecture of the application is, it's going to crash. 48 00:02:06.400 --> 00:02:09.879 Okay, let's unpack this the book calls the origin of 49 00:02:09.919 --> 00:02:12.879 this ricketiness duct tape deployment. 50 00:02:13.039 --> 00:02:14.759 Duct tape deployment. Yeah, we've all seen it. 51 00:02:14.800 --> 00:02:17.879 We really have a project starts, it's moving incredibly fast, 52 00:02:18.280 --> 00:02:20.919 and one developer just needs to get the MVP out 53 00:02:20.960 --> 00:02:25.479 the door. So there are no tools, right, no CICD pipeline. 54 00:02:25.120 --> 00:02:26.479 This vibes and caffeine. 55 00:02:26.680 --> 00:02:31.879 Literally all the server pads, the configuration quirks, the SSH keys, 56 00:02:32.319 --> 00:02:36.080 they are stored entirely in that one single developer's head. 57 00:02:36.120 --> 00:02:37.199 Which is terrifying. 58 00:02:37.280 --> 00:02:39.360 It's like a restaurant where only one chef knows the 59 00:02:39.360 --> 00:02:42.000 secret recipe, right if they call in sick, the whole 60 00:02:42.039 --> 00:02:44.199 restaurant has to shut down for the day, or like 61 00:02:44.240 --> 00:02:47.039 a Jenga tower where only one person knows which blocks 62 00:02:47.039 --> 00:02:48.360 are actually bearing the load. 63 00:02:48.560 --> 00:02:51.840 It is a massive single point of failure. But you know, 64 00:02:52.000 --> 00:02:55.560 beyond the organizational risk of that one person being unavailable, 65 00:02:56.000 --> 00:02:59.120 the real technical danger of duct tape deployment is the 66 00:02:59.159 --> 00:03:00.840 reliance on manual steps. 67 00:03:01.159 --> 00:03:02.400 Right, because humans are humans. 68 00:03:02.520 --> 00:03:05.120 Exactly when humans are stressed or rushed like on a 69 00:03:05.159 --> 00:03:08.560 Friday afternoon, they make mistakes, mads. This actually highlights the 70 00:03:08.599 --> 00:03:12.120 specific danger of manual cash busting as a prime example 71 00:03:12.120 --> 00:03:12.360 of this. 72 00:03:12.680 --> 00:03:15.319 Oh yeah, this part was so relatable. So you update 73 00:03:15.360 --> 00:03:18.199 a CSS file right, and to make sure the user's 74 00:03:18.240 --> 00:03:21.639 browser doesn't serve up the old cached version of the styling, 75 00:03:22.039 --> 00:03:24.879 you have to manually increment the revision number on the 76 00:03:24.879 --> 00:03:27.000 filepath in the code. 77 00:03:26.960 --> 00:03:30.400 Changing style dot CS question mark v equals one to 78 00:03:30.439 --> 00:03:31.159 twenty three. 79 00:03:31.039 --> 00:03:33.800 To one four right right, Just bumping that number. 80 00:03:33.560 --> 00:03:38.039 Precisely, and in a manual deployment, changing that parameter requires 81 00:03:38.039 --> 00:03:41.240 a developer to actually remember to edit the code right 82 00:03:41.280 --> 00:03:42.000 before pushing to. 83 00:03:42.000 --> 00:03:43.840 Production, which is so easy to forget. 84 00:03:43.919 --> 00:03:46.240 Oh, it happens all the time. Yeah, if you forget 85 00:03:46.319 --> 00:03:49.080 that one minor step, the back end logic updates, but 86 00:03:49.120 --> 00:03:51.680 the front end layout breaks completely. For the user, it 87 00:03:51.759 --> 00:03:53.120 just looks like a garbled mess. 88 00:03:53.439 --> 00:03:57.280 It totally reminds me of the broken window theory in sociology, Like, 89 00:03:57.560 --> 00:04:00.719 if you accept one manual step like cash busting a 90 00:04:00.759 --> 00:04:03.800 file by hand, you're basically signaling to the whole team 91 00:04:03.800 --> 00:04:05.520 that the deployment process is lawless. 92 00:04:05.599 --> 00:04:07.280 Yeah, that the rules don't matter, right. 93 00:04:07.280 --> 00:04:09.719 And before you know it, people are manually running database 94 00:04:09.759 --> 00:04:13.439 scripts and production or FTPing files directly to the server. 95 00:04:13.560 --> 00:04:16.720 To fix bugs on the fly. Modest calls this shipping rot. 96 00:04:17.000 --> 00:04:21.040 Shipping rot. That is the exact mechanism of it. Once 97 00:04:21.079 --> 00:04:24.319 you normalize a manual workaround, you just start accumulating technical 98 00:04:24.319 --> 00:04:27.160 debt in your deployment pipeline. It just piles up, it does, 99 00:04:27.279 --> 00:04:30.680 the process rots until it's an unmanageable, terrifying mess that 100 00:04:30.759 --> 00:04:31.600 nobody wants to touch. 101 00:04:31.720 --> 00:04:34.120 But you know, locking down those manual steps is basically 102 00:04:34.199 --> 00:04:37.839 useless if the ground beneath the code keeps shifting. I mean, 103 00:04:37.839 --> 00:04:40.519 you can have a perfect deployment script, but if the 104 00:04:40.600 --> 00:04:44.759 servers themselves aren't identical, production is still going to crash. 105 00:04:44.800 --> 00:04:48.120 Absolutely. Yeah, that's why consistency has to happen at the 106 00:04:48.240 --> 00:04:50.920 environment level before you ever even write a deployment script. 107 00:04:51.680 --> 00:04:55.199 Predictability requires environments that are strictly repeatable. 108 00:04:55.319 --> 00:04:56.399 Strictly repeatable. 109 00:04:56.519 --> 00:05:00.160 Yeah, if your local development machine, your staging server, your 110 00:05:00.160 --> 00:05:04.399 production server have like slight variations between them, you are 111 00:05:04.480 --> 00:05:07.160 essentially playing Russian Roulette with your application. 112 00:05:07.360 --> 00:05:09.879 The source material actually has a great specific example of 113 00:05:09.920 --> 00:05:11.560 this regarding file system. 114 00:05:11.240 --> 00:05:12.879 Oh, the Mac versus Linux one. 115 00:05:13.120 --> 00:05:16.319 Yeah, So say you develop your app locally on a Mac, 116 00:05:16.560 --> 00:05:20.839 which generally treats filenames as case insensitive, but your production 117 00:05:20.920 --> 00:05:25.720 server runs CentOS, which treats filenames as strictly case sensitive. 118 00:05:25.920 --> 00:05:28.600 A classic trap, it really is. You could have a 119 00:05:28.680 --> 00:05:31.439 class file named with a capital letter that your local 120 00:05:31.480 --> 00:05:35.240 machine resolves perfectly, no errors, nothing right. But then you 121 00:05:35.319 --> 00:05:37.879 push it to production, the Cento filesystem can't find it, 122 00:05:37.959 --> 00:05:41.000 and the whole app throws a fatal air and goes down. 123 00:05:41.680 --> 00:05:46.040 It's those invisible infrastructure inconsistencies that cause the most maddening, 124 00:05:46.560 --> 00:05:48.480 you know, late stage deployment. 125 00:05:48.000 --> 00:05:50.120 Bugs, the ones that keep you up at night exactly. 126 00:05:50.600 --> 00:05:54.160 Maybe staging is running PHP eight point one, but production 127 00:05:54.279 --> 00:05:57.079 is somehow on eight point two. Or maybe the mice 128 00:05:57.120 --> 00:06:00.279 equal strict modes differ. Removing that uncertain is. 129 00:06:00.319 --> 00:06:02.560 Paramount, so staging has to be a clone. 130 00:06:02.800 --> 00:06:06.319 Yes, this means your staging environment must duplicate your production 131 00:06:06.439 --> 00:06:09.319 environment to the absolute, fullest extent possible exceptions. 132 00:06:09.519 --> 00:06:10.920 Right, If production. 133 00:06:10.680 --> 00:06:14.959 Utilizes a load balancer and a clustered database architecture, staging 134 00:06:15.040 --> 00:06:18.439 needs a load balancer and a clustered database architecture. You 135 00:06:18.480 --> 00:06:23.160 can't truly validated deployment if the stage doesn't match the theater. 136 00:06:23.360 --> 00:06:26.120 That makes total sense, and that identical setup extends to 137 00:06:26.319 --> 00:06:29.720 how we handle configuration right. The book leans really heavily 138 00:06:29.759 --> 00:06:32.399 into the twelve factor app methodology, a. 139 00:06:32.480 --> 00:06:34.319 Very famous set of principles. 140 00:06:33.839 --> 00:06:39.000 Specifically the strict separation of config from code using dot 141 00:06:39.120 --> 00:06:42.680 env files, usually via a tool like PHP. Don't v 142 00:06:43.319 --> 00:06:45.360 but I'm going to play Devil's advocate here for a second. 143 00:06:45.959 --> 00:06:48.279 I see the security angle of keeping passwords out of 144 00:06:48.279 --> 00:06:51.000 the codebase. I get it. But practically speaking, in like 145 00:06:51.040 --> 00:06:54.759 an early stage startup, juggling dot env files across three 146 00:06:54.800 --> 00:06:58.240 different developer laptops can feel like massive overkill. 147 00:06:58.360 --> 00:06:59.000 It can be annoying. 148 00:06:59.040 --> 00:07:01.240 Yeah, they constantly fall out of sync. So if I 149 00:07:01.360 --> 00:07:04.399 have a private GitHub repository that only my internal team 150 00:07:04.399 --> 00:07:07.560 can access, why not just hardcode the database pathword right 151 00:07:07.600 --> 00:07:09.879 there and it canfig file and save everyone the headache. 152 00:07:10.000 --> 00:07:12.279 Well it's a common temptation because you know, in the 153 00:07:12.279 --> 00:07:15.439 short term it really does save time, but it violates 154 00:07:15.439 --> 00:07:19.839 a foundational principle of systems architecture. Modis actually provides the 155 00:07:19.959 --> 00:07:22.120 ultimate litmus test for this exact scenario. 156 00:07:22.319 --> 00:07:23.079 Okay, what is it? 157 00:07:23.279 --> 00:07:26.959 Ask yourself this, Can you, at any given moment open 158 00:07:27.000 --> 00:07:30.839 source your application repository? Without compromising any credentials. 159 00:07:31.000 --> 00:07:36.120 Oh ah, Meaning if I flip the repo from private 160 00:07:36.120 --> 00:07:38.160 to public right now, what I have to scramble to 161 00:07:38.199 --> 00:07:41.199 revoke API keys and change passwords Exactly. 162 00:07:41.360 --> 00:07:44.199 If the answer is yes, your code and your configuration 163 00:07:44.360 --> 00:07:45.759 are dangerously entangled. 164 00:07:45.920 --> 00:07:47.720 Wow, that's a harsh reality check. 165 00:07:47.839 --> 00:07:48.160 It is. 166 00:07:49.040 --> 00:07:51.759 Relying on the repository's privacy setting is just a false 167 00:07:51.800 --> 00:07:55.000 sense of security. I mean team members leave laptops, get 168 00:07:55.040 --> 00:07:57.680 stolen access tokens, get accidentally logged, and slack. 169 00:07:57.800 --> 00:07:58.800 It happens all the time. 170 00:07:58.959 --> 00:08:01.839 Right, So, if you configureguration is baked into your code, 171 00:08:02.000 --> 00:08:03.800 a brooch of your code base is a breach of 172 00:08:03.839 --> 00:08:08.079 your entire infrastructure. Separating them means the codebase is purely logic. 173 00:08:08.480 --> 00:08:10.480 The environment itself dictates the secrets. 174 00:08:10.879 --> 00:08:14.160 It forces a really clean boundary. The code doesn't know 175 00:08:14.199 --> 00:08:16.959 where it lives. It just knows how to run based 176 00:08:17.000 --> 00:08:20.560 on the environment variables it's fed exactly. So, assuming we 177 00:08:20.639 --> 00:08:25.720 have identical environments and secure separated configurations, the foundation is solid. 178 00:08:25.959 --> 00:08:27.399 Here's where it gets really. 179 00:08:27.120 --> 00:08:28.560 Interesting, the fun stuff. 180 00:08:28.680 --> 00:08:31.959 Yeah, how do we actually push the new code to 181 00:08:32.000 --> 00:08:35.399 the users without them noticing the transition? How do we 182 00:08:35.440 --> 00:08:38.600 achieve what Modest calls the Holy grail atomic deploys. 183 00:08:39.120 --> 00:08:43.559 Atomicity is a fascinating systems concept. In the context of deployment. 184 00:08:44.000 --> 00:08:47.279 An atomic deploy means the transition from the old release 185 00:08:47.320 --> 00:08:50.759 of your application to the new release must appear instantaneous 186 00:08:50.840 --> 00:08:53.120 and entirely uninterrupted to the outside. 187 00:08:52.759 --> 00:08:54.320 World, meaning no maintenance pages. 188 00:08:54.399 --> 00:08:55.039 Ye exactly. 189 00:08:55.080 --> 00:08:56.879 It is a true zero downtime deployment. 190 00:08:56.960 --> 00:08:59.440 But the key word there is that it must appear 191 00:08:59.440 --> 00:09:03.120 instantane because the deployment process itself like pulling the code, 192 00:09:03.159 --> 00:09:06.720 installing dependencies, compiling assets, that actually takes time. 193 00:09:06.759 --> 00:09:08.440 You can take minutes, yeah. 194 00:09:08.080 --> 00:09:11.480 But the users shouldn't feel a millisecond of it. There's 195 00:09:11.519 --> 00:09:15.039 this cautionary tale in the book that beautifully illustrates the 196 00:09:15.120 --> 00:09:20.480 catastrophic difference between a fast deploy and an atomic deploy. 197 00:09:20.600 --> 00:09:22.639 It involves this email campaign service. 198 00:09:22.679 --> 00:09:25.240 Oh right, this is such a classic example. So this 199 00:09:25.320 --> 00:09:29.200 development team had built a robust service and users heavily 200 00:09:29.279 --> 00:09:32.679 utilized a feature that allowed them to import massive CSV 201 00:09:32.840 --> 00:09:34.840 files of subscribers. 202 00:09:34.240 --> 00:09:35.519 Like huge lists. 203 00:09:35.639 --> 00:09:38.279 Yeah, tens of thousands of rows. It was a background 204 00:09:38.360 --> 00:09:41.320 process that could take fifteen or twenty minutes to run. 205 00:09:41.600 --> 00:09:44.919 But suddenly the support team started getting these furious tickets 206 00:09:45.240 --> 00:09:46.399 every other Monday. 207 00:09:46.080 --> 00:09:47.200 Which is very specific. 208 00:09:47.480 --> 00:09:50.279 Very users were complaining that their massive data imports were 209 00:09:50.360 --> 00:09:52.840 just being abruptly aborted halfway through. 210 00:09:52.759 --> 00:09:55.639 Every other Monday. So the team dug into the server 211 00:09:55.759 --> 00:09:58.879 logs check memory loads, and they couldn't find any application 212 00:09:59.039 --> 00:10:01.639 errors at all until they mapped the failures to their 213 00:10:01.639 --> 00:10:02.519 deployment schedule. 214 00:10:02.600 --> 00:10:05.360 Yep, the deployment was the culprit as part of their 215 00:10:05.399 --> 00:10:08.840 automated deploy process. The final step was restarting a service 216 00:10:08.840 --> 00:10:13.679 called PHPFPM. That's the process manager that handles PHP requests. 217 00:10:13.960 --> 00:10:18.240 Now to the developer executing the deployment script, Restarting FPM 218 00:10:18.879 --> 00:10:22.559 took maybe fifty milliseconds. It felt lightning fast. 219 00:10:22.320 --> 00:10:26.879 But mechanically, when you issue a standard restart command to PHPFPM, 220 00:10:27.320 --> 00:10:30.000 it forcefully flushes the worker pools right exactly. 221 00:10:30.200 --> 00:10:32.799 It sends a kill signal to the child processes that 222 00:10:32.840 --> 00:10:36.200 are executing scripts in memory at that exact moment. 223 00:10:35.919 --> 00:10:38.799 Which is exactly what happened to the memory heavy PHP 224 00:10:38.960 --> 00:10:41.960 import script that's just chugging away in the background processing 225 00:10:42.000 --> 00:10:46.200 CSV rows. That fifty millisecond restart was a sudden execution dot. 226 00:10:46.240 --> 00:10:47.120 This has got wiped out. 227 00:10:47.240 --> 00:10:50.399 Yeah, the deplay was fast, but it interrupted the system state. 228 00:10:50.480 --> 00:10:52.840 It was not atomic, not at all. So how do 229 00:10:52.879 --> 00:10:55.879 we actually solve this at the filesystem level without restarting 230 00:10:55.879 --> 00:11:00.679 services or dropping connections? Modest details. A specific, highly elegant 231 00:11:00.720 --> 00:11:04.960 directory structure, relying on three components, builds, Shared, and Latest. 232 00:11:05.279 --> 00:11:07.639 Let's break down the mechanics of that, because it's brilliant. 233 00:11:08.120 --> 00:11:11.320 The Builds directory holds time stamped folders. 234 00:11:10.879 --> 00:11:12.480 Of your recent application. 235 00:11:12.000 --> 00:11:13.960 Builds, so they're all just sitting there, right. 236 00:11:14.360 --> 00:11:16.960 You might have the currently running version, plus say the 237 00:11:17.039 --> 00:11:20.120 last five historical versions sitting there passively on the disc. 238 00:11:20.279 --> 00:11:22.879 Yeah, just in case you need to execute a rapid rollback. 239 00:11:23.240 --> 00:11:27.440 Then there's the shared directory. This isolates everything that needs 240 00:11:27.480 --> 00:11:31.519 to persist across deployments, things like user uploaded media or 241 00:11:31.639 --> 00:11:34.440 file based session data, because. 242 00:11:34.120 --> 00:11:37.320 You definitely don't want active user sessions being wiped out 243 00:11:37.399 --> 00:11:39.600 just because you pushed a patch update exactly. 244 00:11:40.039 --> 00:11:43.000 And the orchestration relies on the third piece, which is 245 00:11:43.200 --> 00:11:44.279 the latest directory. 246 00:11:44.360 --> 00:11:46.279 But it's not actually a directory, right. 247 00:11:46.240 --> 00:11:48.720 It is a similink, a symbolic link at the OS 248 00:11:48.840 --> 00:11:51.879 level that simply points to the most recent time stamped 249 00:11:51.919 --> 00:11:55.159 folder in your builds directory, and your web server, whether 250 00:11:55.240 --> 00:11:58.960 it's in jinks or apatchee, is configured to exclusively route 251 00:11:59.000 --> 00:12:00.240 traffic to that latest. 252 00:12:00.559 --> 00:12:01.720 It's such a clever setup. 253 00:12:01.799 --> 00:12:04.440 Think of it like this. Imagine a magician pulling a 254 00:12:04.480 --> 00:12:06.679 tablecloth out from under a fully set dinner table. 255 00:12:06.720 --> 00:12:07.320 Okay, I like this. 256 00:12:07.840 --> 00:12:11.679 The plates, the glasses, the silverware. That's your user's experience, 257 00:12:11.919 --> 00:12:16.399 and the active HTTP requests. The tablecloth is the underlying 258 00:12:16.440 --> 00:12:20.440 server directory with a simlink. Your deployment script builds the 259 00:12:20.600 --> 00:12:23.200 entirely new version of the app in a new folder 260 00:12:23.240 --> 00:12:27.120 in the background. It downloads packages, builds assets, does everything 261 00:12:27.120 --> 00:12:28.360 it needs to do safely out. 262 00:12:28.279 --> 00:12:30.440 Of you, taking as much time as it needs. 263 00:12:30.159 --> 00:12:34.080 Exactly, and then at the very last microsecond, it updates 264 00:12:34.120 --> 00:12:36.279 the inode pointer of the simlink to point to the 265 00:12:36.279 --> 00:12:38.759 new build folder. It yanks the tablecloth. 266 00:12:38.799 --> 00:12:39.120 Wow. 267 00:12:39.360 --> 00:12:43.159 The OS updates the pointer in microseconds, meaning the very 268 00:12:43.200 --> 00:12:46.559 next HTTP request hitting the server gets routed to the 269 00:12:46.559 --> 00:12:49.159 new release directory before a human eye can even blink. 270 00:12:49.279 --> 00:12:50.360 The dishes don't move. 271 00:12:50.679 --> 00:12:53.120 It really is a flawless transition. The web server just 272 00:12:53.159 --> 00:12:56.960 continuously serves requests. One microsecond it's reading from release a 273 00:12:57.440 --> 00:12:59.360 The next microsecond it's reading from release B. 274 00:13:00.000 --> 00:13:01.279 When nothing gets killed. 275 00:13:01.200 --> 00:13:04.559 Right, no process managers are restarted, no background tasks are 276 00:13:04.600 --> 00:13:08.600 forcefully killed. It completely fulfills the definition of atymicity. 277 00:13:09.080 --> 00:13:13.399 It's basically magic, and there's always a but while we've 278 00:13:13.519 --> 00:13:17.120 magically swapped the tablecloth for our own code. Modern applications 279 00:13:17.120 --> 00:13:20.120 are deeply reliant on third party packages, all right, Absolutely, 280 00:13:20.159 --> 00:13:24.080 we pull in libraries for payment processing, routing, image manipulation. 281 00:13:24.679 --> 00:13:28.519 If one of those external packages updates unexpectedly right during 282 00:13:28.519 --> 00:13:31.240 our atomic swap, the whole application can shatter. 283 00:13:31.440 --> 00:13:34.399 This is the peril of dependency roulette. It really doesn't 284 00:13:34.440 --> 00:13:37.919 matter how tomic your deployment script is if the payload 285 00:13:37.960 --> 00:13:41.919 it's deploying is suddenly incompatible. Right to grasp how this happens, 286 00:13:42.360 --> 00:13:45.120 we have to look at the inherent flaw in relying 287 00:13:45.240 --> 00:13:47.440 solely on semantic versioning or. 288 00:13:47.440 --> 00:13:51.159 Semb okay, Soceember relies on the honor system. We trust 289 00:13:51.200 --> 00:13:55.759 package maintainers to adhere to the major minor patch numbering. 290 00:13:55.360 --> 00:13:58.320 Structure, right the extol why dot z format exactly. 291 00:13:58.360 --> 00:14:01.000 We trust that a patch update, say, going from version 292 00:14:01.039 --> 00:14:03.120 one point four point two to one point four point three, 293 00:14:03.320 --> 00:14:05.759 is purely a bug fix and completely safe to pull 294 00:14:05.799 --> 00:14:08.600 in automatically, but humans are fallible. 295 00:14:08.879 --> 00:14:12.360 What's fascinating here is the delicate and often really stressful 296 00:14:12.399 --> 00:14:16.720 balancing act maintainer's face between pushing crucial security updates and 297 00:14:16.840 --> 00:14:18.879 inadvertently causing downstream chaos. 298 00:14:18.960 --> 00:14:20.120 It's a tough spot to be in. 299 00:14:20.279 --> 00:14:23.399 It is Modis actually highlights a famous cautionary tail in 300 00:14:23.399 --> 00:14:26.559 the PHP ecosystem in leaving the larraval framework that illustrates 301 00:14:26.559 --> 00:14:27.159 this perfectly. 302 00:14:27.440 --> 00:14:31.159 Oh, the Laravell four point one point two five exploit. 303 00:14:32.080 --> 00:14:36.120 A critical security vulnerability was discovered involving how the framework 304 00:14:36.200 --> 00:14:37.759 handled remember me cookies. 305 00:14:38.000 --> 00:14:40.240 It allowed attackers to hijack user sessions. 306 00:14:40.320 --> 00:14:42.879 Yeah, it was severe, So the maintainer's rushed out a 307 00:14:42.919 --> 00:14:46.440 patch update, version four point one point two sex. 308 00:14:46.559 --> 00:14:49.919 Which was an urgent and totally necessary fix. Right to 309 00:14:50.000 --> 00:14:53.440 properly patch the security flaw, the maintainers had to alter 310 00:14:53.600 --> 00:14:54.799 how the tokens. 311 00:14:54.360 --> 00:14:56.600 Were stored, which required a database. 312 00:14:56.200 --> 00:14:59.440 Change exactly, which required a change to the database schema. 313 00:15:00.000 --> 00:15:02.480 Release it as a patch update, but it contained a 314 00:15:02.600 --> 00:15:05.679 non backwards compatible architectural requirement, and this. 315 00:15:05.600 --> 00:15:08.279 Is where the roulette wheel landed on black For thousands 316 00:15:08.320 --> 00:15:11.799 of developers, many teams can figure their dependency managers using 317 00:15:11.879 --> 00:15:16.320 wildcard constraints like four point one dot asterisk or using 318 00:15:16.360 --> 00:15:20.000 the carou operator, basically telling their system hey automatically grab 319 00:15:20.000 --> 00:15:22.360 any new patches because patches are inherently safe. 320 00:15:22.440 --> 00:15:25.720 So during their next deployment cycle, their automated systems dynamically 321 00:15:25.720 --> 00:15:29.200 pulled in four point one point twenty six, the framework updated. 322 00:15:29.600 --> 00:15:32.360 But because the developers hadn't written or executed a migration 323 00:15:32.440 --> 00:15:35.960 to update their production database schema to match this unexpected change, 324 00:15:36.120 --> 00:15:40.759 oh man, the application queried a column that literally didn't exist. 325 00:15:41.320 --> 00:15:45.679 The applications broke instantly, five hundred server errors across the board. 326 00:15:45.840 --> 00:15:49.200 It's terrifying because they did everything right on paper. They 327 00:15:49.200 --> 00:15:52.480 were keeping their dependencies up to date for security exactly. 328 00:15:52.559 --> 00:15:56.960 It proves that even patch versions aren't entirely safe. Now, 329 00:15:57.360 --> 00:15:59.720 if you're listening to this and groaning because you've spent 330 00:16:00.080 --> 00:16:04.559 hours resolving agonizing merge conflicts in a massive composer dot 331 00:16:04.600 --> 00:16:07.840 lock file, I get it. It's painful. We've all been there, 332 00:16:08.240 --> 00:16:11.559 But that pain is the exact premium you pay to 333 00:16:11.679 --> 00:16:13.200 avoid dependency roulette. 334 00:16:13.240 --> 00:16:16.639 The lock file is your absolute safeguard. The golden rule 335 00:16:16.720 --> 00:16:19.120 is that you never commit your actual dependency folders to 336 00:16:19.200 --> 00:16:22.120 version control, but you must strictly commit your locksile no 337 00:16:22.240 --> 00:16:25.279 exception right when you test your application locally and it 338 00:16:25.320 --> 00:16:29.120 functions perfectly. The lock file records the cryptographic hash and 339 00:16:29.200 --> 00:16:32.759 the exact specific version of every single dependency down to 340 00:16:32.799 --> 00:16:33.879 the final patch number. 341 00:16:34.120 --> 00:16:36.919 So when your deployment script runs in production, it doesn't 342 00:16:36.960 --> 00:16:39.320 look at the liberal constraints in your main canfig file. 343 00:16:39.600 --> 00:16:43.039 It reads the lock file and enforces those exact versions, 344 00:16:43.600 --> 00:16:48.240 no wildcards, no unexpected schema changes. If Larevell four point 345 00:16:48.240 --> 00:16:51.360 one point twenty six is released, your lock file keeps 346 00:16:51.399 --> 00:16:54.120 you securely tethered to four point one point two five 347 00:16:54.519 --> 00:16:57.360 until you intentionally decide to pull down the update, test 348 00:16:57.399 --> 00:17:00.600 the database changes locally and generate a new lock file. 349 00:17:00.960 --> 00:17:03.639 It basically guarantees that the code running on the developer's 350 00:17:03.679 --> 00:17:07.160 laptop is the exact same code executing on the production server. 351 00:17:07.240 --> 00:17:10.440 Okay, so let's recap. We have identical environments, we have 352 00:17:10.480 --> 00:17:14.440 atomic simlink swaps, and strictly locked dependencies. Now we have 353 00:17:14.480 --> 00:17:17.440 to orchestrate all of this to run securely with one click, 354 00:17:17.599 --> 00:17:17.960 and that. 355 00:17:17.880 --> 00:17:20.359 Brings us to the most volatile in phase of deployment. 356 00:17:20.440 --> 00:17:24.359 Database migrations. Right, migrations are essentially version control for your 357 00:17:24.400 --> 00:17:26.799 database schema using tools like thanks or are built in 358 00:17:26.839 --> 00:17:30.759 framework features. But we talk about migrations in an automated pipeline. 359 00:17:30.920 --> 00:17:33.599 The real danger isn't usually the upscript that adds a 360 00:17:33.640 --> 00:17:35.559 new table. It's the downscript exactly. 361 00:17:35.559 --> 00:17:37.119 It's the downscript during a rollback. 362 00:17:37.359 --> 00:17:40.319 Okay, say you push a feature that adds a column 363 00:17:40.400 --> 00:17:45.839 for subscription expiration date. Users start interacting with the app. 364 00:17:45.880 --> 00:17:49.160 They're generating real production data in that new column. But 365 00:17:49.240 --> 00:17:52.359 an hour later, you discover a critical logic bug in 366 00:17:52.440 --> 00:17:55.599 the code, and you trigger an automated rollback to revert 367 00:17:55.680 --> 00:17:56.319 the deployment. 368 00:17:56.759 --> 00:18:01.799 And if you're automated, rollback executes a micro downscript that 369 00:18:01.960 --> 00:18:03.240 drops that new column. 370 00:18:03.000 --> 00:18:06.359 To revert the database state. Oh no, you haven't just 371 00:18:06.440 --> 00:18:07.359 reverted a future. 372 00:18:07.960 --> 00:18:11.440 You have permanently destroyed live production data. The bug's gone, 373 00:18:11.799 --> 00:18:14.319 but so are the expiration dates for every single user 374 00:18:14.359 --> 00:18:15.799 who's subscribed in the last hour. 375 00:18:16.000 --> 00:18:18.960 It's catastrophic. You also have to navigate what Modus calls 376 00:18:19.000 --> 00:18:20.319 the batch rollback problem. 377 00:18:20.440 --> 00:18:21.680 Yeah, that's a tricky one, right. 378 00:18:21.720 --> 00:18:24.799 If your deployment pushes three separate migration files at once 379 00:18:24.839 --> 00:18:27.359 say A, B and C, and you trigger a rollback. 380 00:18:27.400 --> 00:18:29.759 Does your tooling revert just C or does it revert A, 381 00:18:29.960 --> 00:18:30.319 B and C. 382 00:18:30.680 --> 00:18:32.400 You better know the answer to that before you. 383 00:18:32.319 --> 00:18:36.000 Click the button exactly. If your reverted codebase still expects 384 00:18:36.000 --> 00:18:38.400 the tables from migrations and B to exist, but the 385 00:18:38.480 --> 00:18:41.480 database roll them back to your application is broken again. 386 00:18:41.720 --> 00:18:45.000 It's like diffusing a bomb in the dark, which is 387 00:18:45.039 --> 00:18:50.039 precisely why robust orchestration tooling is required. Modus explores tools 388 00:18:50.079 --> 00:18:55.960 like gethoks, Thing, Capistrano, and Rocketeer. Lots of options, yeah, 389 00:18:56.000 --> 00:18:59.319 but the specific syntax of the tool matters less than 390 00:18:59.359 --> 00:19:03.640 the overarch methodology. These tools exist to rigidly script this 391 00:19:03.799 --> 00:19:07.519 entire pipeline. They pull the code, validate the log file, 392 00:19:08.000 --> 00:19:12.000 execute the migrations, compile the new build directory, and finally, 393 00:19:12.279 --> 00:19:13.680 atomically flip the SIMB link. 394 00:19:13.799 --> 00:19:15.880 And before they flip that cimblink, they must pass the 395 00:19:15.960 --> 00:19:19.359 ultimate gatekeeper testing. Modest brings up a really great concept 396 00:19:19.400 --> 00:19:22.160 from Joel Spolsky called hallway usability testing. 397 00:19:22.240 --> 00:19:25.160 It's a classic heuristic. Spolsky argued that if you pull 398 00:19:25.160 --> 00:19:27.519 a random coworker from the hallway and just force them 399 00:19:27.519 --> 00:19:30.759 to navigate your new feature. They will uncover ninety five 400 00:19:30.799 --> 00:19:33.720 percent of the usability flaws immediately. 401 00:19:33.279 --> 00:19:34.640