WEBVTT 1 00:00:00.120 --> 00:00:03.799 Imagine having a conversation where like, every time you pause 2 00:00:03.839 --> 00:00:06.839 to take a breath, the person you're talking to completely 3 00:00:06.879 --> 00:00:08.359 deletes your formatting. 4 00:00:08.000 --> 00:00:11.279 Oh yeah, and just ignores half of what you actually meant, right, and. 5 00:00:11.240 --> 00:00:15.119 Then mangles your sentence into something totally unrecognizable. I mean, 6 00:00:15.320 --> 00:00:17.760 if you've ever typed a command into a computer terminal 7 00:00:17.800 --> 00:00:21.679 and watched it instantly fail, you know exactly what I mean. 8 00:00:21.800 --> 00:00:23.000 It's so frustrating. 9 00:00:23.079 --> 00:00:25.160 It really is, because we tend to think of our 10 00:00:25.199 --> 00:00:29.160 computers as these highly intuitive machines that you just want 11 00:00:29.199 --> 00:00:31.519 to help us out. But when you strip away the 12 00:00:31.559 --> 00:00:35.479 graphical interface, what you're left with is a blinking cursor 13 00:00:35.880 --> 00:00:37.640 that is ruthlessly. 14 00:00:37.159 --> 00:00:41.000 Literal, completely literal, and it's notoriously aggressive about how it 15 00:00:41.039 --> 00:00:43.560 interprets your words. To a lot of people see that 16 00:00:43.600 --> 00:00:47.200 blinking cursor the command line, and they just assume it's 17 00:00:47.200 --> 00:00:47.920 a relic. 18 00:00:47.719 --> 00:00:49.840 Like some archaic leftover from the nineteen. 19 00:00:49.600 --> 00:00:53.200 Seventies, exactly like something we only keep around for it technicians. 20 00:00:53.479 --> 00:00:56.640 But the command line is not a museum piece. It's 21 00:00:56.719 --> 00:01:01.640 a programmable, highly efficient conversation with your operating system. 22 00:01:01.240 --> 00:01:03.840 Which brings us to the core of our deep dive today. 23 00:01:04.040 --> 00:01:06.959 I'm your host, and today we're pilling some incredible insights 24 00:01:07.120 --> 00:01:10.480 from the Bash Cookbook, second edition by Carl Albing and 25 00:01:10.560 --> 00:01:11.359 JP Vawson. 26 00:01:11.439 --> 00:01:13.200 It's a fantastic resource. 27 00:01:12.840 --> 00:01:15.159 It really is. And our mission today is to crack 28 00:01:15.200 --> 00:01:19.879 open that dreaded cryptic command line interface. We're going to 29 00:01:19.920 --> 00:01:24.079 decode the strange punctuation, learn how to pipe data around 30 00:01:24.319 --> 00:01:26.200 like a well, like a master plumber. 31 00:01:26.239 --> 00:01:27.680 Really, I love that analogy. 32 00:01:27.799 --> 00:01:31.120 Thanks, and we'll discover why this text based tool is 33 00:01:31.159 --> 00:01:35.760 still the absolute fastest shortcut to getting complex work done. 34 00:01:35.519 --> 00:01:38.599 Because once you understand the grammar of this specific conversation, 35 00:01:38.719 --> 00:01:41.560 it changes everything. It remembers your commands, lets you edit 36 00:01:41.560 --> 00:01:44.760 them on the fly, and honestly, it can automate literally 37 00:01:44.799 --> 00:01:46.680 any repetitive task you throw at it. 38 00:01:46.799 --> 00:01:48.959 But before we start typing commands, we kind of need 39 00:01:48.959 --> 00:01:51.920 to address what this black box actually is, because the 40 00:01:51.959 --> 00:01:55.239 command line, or the shell as it's called, isn't actually 41 00:01:55.319 --> 00:01:59.239 the operating system itself, which I have to admit blew 42 00:01:59.280 --> 00:02:01.599 my mind a bit. I always assume the terminal window 43 00:02:01.719 --> 00:02:02.760 was the core of the computer. 44 00:02:03.120 --> 00:02:06.640 Oh, that's a really common misconception. But the separation between 45 00:02:06.680 --> 00:02:09.199 the two is actually a foundational part of the original 46 00:02:09.319 --> 00:02:13.639 Unix philosophy. How so well, in the earliest operating systems, 47 00:02:14.039 --> 00:02:18.000 the user interface, the part that actually accepted your typed commands, 48 00:02:18.439 --> 00:02:21.840 was hardwired straight into the operating system's core brain. 49 00:02:21.800 --> 00:02:24.280 So they were bundled together, right, So if. 50 00:02:24.120 --> 00:02:26.479 You wanted a better way to type commands for just 51 00:02:26.520 --> 00:02:29.759 a new feature, you had to rewrite the entire operating 52 00:02:29.759 --> 00:02:30.560 system to get. 53 00:02:30.439 --> 00:02:32.439 It, which sounds a bit like trying to upgrade your 54 00:02:32.520 --> 00:02:35.000 car's spearing wheel by replacing the entire engine. 55 00:02:35.080 --> 00:02:38.039 That is a perfect analogy. It was incredibly inefficient, so 56 00:02:38.080 --> 00:02:42.039 the creators of Unix separated them entirely. The operating system 57 00:02:42.080 --> 00:02:45.360 handles the hardware, the memory, the heavy lifting, the engine 58 00:02:45.400 --> 00:02:48.120 exactly the engine, and the shell is just a separate, 59 00:02:48.280 --> 00:02:51.719 replaceable dashboard. Its only job is to listen to you 60 00:02:51.919 --> 00:02:54.719 interpret your command and then pass it down to the engine. 61 00:02:55.080 --> 00:02:58.520 And because it was a separate program, developers could write new, 62 00:02:58.919 --> 00:03:01.639 better shells without ever touching the core system. 63 00:03:01.479 --> 00:03:03.599 So you really could just swap out the dashboard whenever 64 00:03:03.639 --> 00:03:05.639 you want. You could use shells with names like Crow 65 00:03:05.840 --> 00:03:07.439 or KSh or Soush yep. 66 00:03:07.560 --> 00:03:08.919 Absolutely, but today. 67 00:03:08.759 --> 00:03:12.120 Our sources are focused entirely on Bash So why did 68 00:03:12.199 --> 00:03:15.400 Bash become, you know, the universal rental car that everyone 69 00:03:15.439 --> 00:03:16.120 knows how to drive. 70 00:03:16.439 --> 00:03:19.039 It really comes down to a perfect storm of timing 71 00:03:19.080 --> 00:03:23.520 and utility. Bash stands for the GNU born again. 72 00:03:23.360 --> 00:03:26.159 Shell right, great pun by the way, Oh. 73 00:03:25.879 --> 00:03:29.439 Programmers love their puns. But back in the late nineteen eighties, 74 00:03:29.520 --> 00:03:32.719 the Gene project was trying to create a free, complete 75 00:03:32.719 --> 00:03:36.680 operating system that followed PO six standards and posx. 76 00:03:36.280 --> 00:03:39.479 Is essentially the universal rule book right for how Unix 77 00:03:39.479 --> 00:03:42.879 systems should behave so software can run anywhere exactly. 78 00:03:43.080 --> 00:03:45.919 They needed a standard shell for this free system, so 79 00:03:45.960 --> 00:03:49.199 they built Bash by taking the heavy duty programming features 80 00:03:49.199 --> 00:03:53.080 that developers required and combining them with the interactive conveniences 81 00:03:53.120 --> 00:03:55.120 that everyday users wanted. 82 00:03:54.919 --> 00:03:56.800 Things like just hitting the up arrow to see your 83 00:03:56.879 --> 00:03:57.439 last command. 84 00:03:57.560 --> 00:03:59.879 Yes, small things that made a huge difference. 85 00:04:00.159 --> 00:04:02.360 Well, it clearly won the war. I mean, it's the 86 00:04:02.400 --> 00:04:06.159 default on almost all Linux distributions historically, the default on 87 00:04:06.280 --> 00:04:09.520 mac os, and you can even install it on Windows 88 00:04:09.560 --> 00:04:12.800 through the Windows subsystem. For a Linux, it's everywhere. So 89 00:04:12.840 --> 00:04:15.439 you learn Bash and suddenly you have a passport to 90 00:04:15.479 --> 00:04:17.279 talk to almost any machine on the planet. 91 00:04:17.360 --> 00:04:20.279 You do, but to speak the language you first have 92 00:04:20.360 --> 00:04:23.560 to read the room. When you open that terminal, you're 93 00:04:23.639 --> 00:04:27.000 greeted by a prompt, and that prompt usually ends with 94 00:04:27.040 --> 00:04:30.800 a specific symbol, often a dollar sign or a hash symbol. Right, 95 00:04:30.959 --> 00:04:34.079 and that tiny character is actually screaming a warning at 96 00:04:34.120 --> 00:04:35.720 you if you know how to listen. 97 00:04:35.879 --> 00:04:38.759 So let's decode that for the listener. If my prompt 98 00:04:38.839 --> 00:04:41.720 ends in a dollar sign, the shell is basically saying 99 00:04:41.879 --> 00:04:45.279 you are a regular user, you have normal safe permissions exactly. 100 00:04:45.319 --> 00:04:46.800 But if it ends in a hash sign like the 101 00:04:46.839 --> 00:04:48.680 number symbol, it's saying you are. 102 00:04:48.680 --> 00:04:52.079 Root, the all powerful system administrator. As route, there are 103 00:04:52.120 --> 00:04:54.160 no safety rails whatsoever. You can do. 104 00:04:54.160 --> 00:04:57.199 Anything, which also means you can accidentally delete the entire 105 00:04:57.240 --> 00:04:58.680 hard drive with a single typo. 106 00:04:58.879 --> 00:05:01.480 Yep, seeing that hash sign is your visual queue to 107 00:05:01.600 --> 00:05:03.639 double check every single keystroke. 108 00:05:03.879 --> 00:05:06.519 Okay, So the prompt tells us who we are. It 109 00:05:06.560 --> 00:05:08.360 also usually tries and tell us where we are in 110 00:05:08.399 --> 00:05:11.800 the computer's folder structure. But the cookbook points out something 111 00:05:11.879 --> 00:05:13.480 incredibly sneaky. 112 00:05:13.040 --> 00:05:14.319 Here, oh about the paths. 113 00:05:14.560 --> 00:05:18.439 Yeah, the prompt sometimes lies to you, or at least 114 00:05:18.600 --> 00:05:20.959 it gives you an illusion like, wait, why would my 115 00:05:21.000 --> 00:05:22.920 computer lie to me about what? Folder I'm in. 116 00:05:23.040 --> 00:05:25.160 Well, it's not so much lying as it is keeping 117 00:05:25.240 --> 00:05:28.160 up a facade that you created. Okay, fair, Think about 118 00:05:28.199 --> 00:05:31.120 shortcuts on your desktop. In the Unix world, these are 119 00:05:31.160 --> 00:05:34.079 called symbolic links. You might have a folder on your 120 00:05:34.079 --> 00:05:37.639 desktop called photos, but the actual files are stored deep 121 00:05:37.680 --> 00:05:40.879 inside some external hard drive. If you navigate into that 122 00:05:40.920 --> 00:05:45.600 desktop folder and use the PW beauty command print working directory, 123 00:05:46.279 --> 00:05:49.240 the shell might just say you are on the desktop. 124 00:05:48.879 --> 00:05:50.480 Because that's the path I took to get there. 125 00:05:50.519 --> 00:05:54.120 Precisely, If you type pdbudd dash L, it gives you 126 00:05:54.120 --> 00:05:57.160 your logical path that's where you think you are based 127 00:05:57.160 --> 00:05:58.519 on the shortcuts you followed. 128 00:05:58.600 --> 00:05:58.920 Got it. 129 00:05:59.399 --> 00:06:01.759 But if if you need to know where those files 130 00:06:01.800 --> 00:06:05.920 physically reside on the spinning disc, you use PWD dash 131 00:06:05.959 --> 00:06:10.279 p P for physical Right. The dash P strips away 132 00:06:10.319 --> 00:06:13.800 all the illusions and reveals the true physical path. 133 00:06:14.079 --> 00:06:17.560 That distinction alone can save like hours of troubleshooting. Yeah, 134 00:06:17.600 --> 00:06:21.360 but here is where navigating gets truly dangerous. We all 135 00:06:21.399 --> 00:06:24.240 have hidden files on our computers right. Oh yes, and 136 00:06:24.279 --> 00:06:27.160 in Unix they're called dot files because their names literally 137 00:06:27.240 --> 00:06:29.480 just start with a period exactly. So if I want 138 00:06:29.480 --> 00:06:31.839 to see them, my instinct as a user is to 139 00:06:31.920 --> 00:06:35.040 use a wildcard. I know what asterisk means everything, so 140 00:06:35.120 --> 00:06:38.079 I figured I type the list command ls followed by 141 00:06:38.079 --> 00:06:40.279 dot asterisk. You have to see all my hidden dot 142 00:06:40.279 --> 00:06:43.959 file which seems totally logical, right, But the cookbook says 143 00:06:44.079 --> 00:06:47.959 this will cause a catastrophic wall of text. Why if 144 00:06:47.959 --> 00:06:51.800 the asterisk means everything, shouldn't dot asterisk just list my 145 00:06:51.879 --> 00:06:52.639 hidden files? 146 00:06:52.879 --> 00:06:55.439 This is such a classic trap, and it introduces a 147 00:06:55.480 --> 00:06:58.639 process called globbing, which is how the shell actually expands 148 00:06:58.680 --> 00:07:02.319 wildcards wlobbing. Okay, so the asterisk does mean any characters 149 00:07:02.439 --> 00:07:05.639 zero or more times. You're totally correct that dot asterisk 150 00:07:05.759 --> 00:07:07.399 matches any file starting with a dot. 151 00:07:07.480 --> 00:07:08.519 Okay, so what's the problem. 152 00:07:08.600 --> 00:07:11.240 The danger lies in how the Unix filesystem is built. 153 00:07:11.600 --> 00:07:15.480 In every single directory without exception, there are two invisible folders. 154 00:07:15.959 --> 00:07:18.480 One is named with a single dot, which is a 155 00:07:18.519 --> 00:07:21.879 shortcut representing the directory you are currently in, ah right, 156 00:07:21.959 --> 00:07:24.600 and the other is named with two dots dot dot, 157 00:07:24.879 --> 00:07:28.040 which is a shortcut representing the parent directory exactly one 158 00:07:28.120 --> 00:07:28.680 level above. 159 00:07:28.759 --> 00:07:29.720 Oh, I see where this is going. 160 00:07:29.839 --> 00:07:33.399 Yeah, when you type l's dot asterisk, the shell aggressively 161 00:07:33.439 --> 00:07:37.040 expands that wildcard before the l's command even runs. Oh no, 162 00:07:37.120 --> 00:07:39.959 it matches your hidden files, sure, but it also matches 163 00:07:40.120 --> 00:07:42.759 the single dot and the double dot. So the shell 164 00:07:42.879 --> 00:07:45.920 tells the l's command, please list the contents of the 165 00:07:46.000 --> 00:07:48.720 hidden files, and the contents of the current folder, and 166 00:07:48.800 --> 00:07:51.319 the entire contents of the folder above this. 167 00:07:51.240 --> 00:07:53.839 One, which could be my entire home direction exactly. It 168 00:07:53.839 --> 00:07:56.600 would just dump thousands of files onto my screen endlessly. 169 00:07:57.040 --> 00:08:00.240 So how do we actually isolate the hidden files without 170 00:08:00.279 --> 00:08:02.160 triggering an absolute avalanche? 171 00:08:02.240 --> 00:08:05.800 The cookbook provides a brilliantly precise piece of syntax. 172 00:08:05.879 --> 00:08:06.040 Or this. 173 00:08:06.519 --> 00:08:11.040 You type dot, dot, bracket, exclamation point, dot, close bracket 174 00:08:11.120 --> 00:08:11.680 as risk. 175 00:08:11.959 --> 00:08:14.680 Okay, let me translate that out of alphabet soup a dot, 176 00:08:15.079 --> 00:08:18.480 then an open bracket, an exclamation point, another dot, a 177 00:08:18.519 --> 00:08:21.959 closed bracket, and finally an asterisk. What is the logic 178 00:08:22.040 --> 00:08:22.519 behind that? 179 00:08:23.240 --> 00:08:27.199 So brackets let you define a specific pool of allowed characters. 180 00:08:27.639 --> 00:08:30.040 But when you put an exclamation point right inside the 181 00:08:30.079 --> 00:08:32.960 opening bracket, it acts as a negator, like saying not 182 00:08:33.159 --> 00:08:37.639 this exactly. It means absolutely not this character. So that 183 00:08:37.679 --> 00:08:40.879 whole sequence translates to find a file that starts with 184 00:08:40.960 --> 00:08:43.440 a dot. Now make sure the very next character is 185 00:08:43.559 --> 00:08:46.840 absolutely not a dot. After that, the asterisk says you 186 00:08:46.879 --> 00:08:50.360 can grab whatever is left. It perfectly targets your hidden 187 00:08:50.399 --> 00:08:54.000 files while explicitly ignoring the single and double dot directories. 188 00:08:54.080 --> 00:08:57.200 That is surgically precise, okay, and it brings us to 189 00:08:57.279 --> 00:09:00.840 a massive realization about Bash. How you format your text 190 00:09:01.000 --> 00:09:03.559 matters just as much as the command itself. Oh completely, 191 00:09:03.639 --> 00:09:07.159 Because that shell is an aggressive listener. It intercepts parses 192 00:09:07.200 --> 00:09:10.000 and modifies your words before it ever passes them to 193 00:09:10.039 --> 00:09:11.480 the actual program you're trying to run. 194 00:09:11.799 --> 00:09:14.960 That aggressive listening is exactly why spacing can drive new 195 00:09:15.039 --> 00:09:18.279 users crazy. The shell uses spaces purely to figure out 196 00:09:18.279 --> 00:09:20.320 where one argument ends and another begins. 197 00:09:20.720 --> 00:09:23.159 Right, So, if I use the echo command, which just 198 00:09:23.240 --> 00:09:25.720 prints whatever you type back to the screen, A type echo, 199 00:09:26.000 --> 00:09:28.559 then the word this, Then I hit the space bar 200 00:09:28.639 --> 00:09:31.559 ten times, then the word was, ten more spaces, and 201 00:09:31.600 --> 00:09:33.960 the word spaced. What comes out on my screen is 202 00:09:34.039 --> 00:09:36.440 just this was spaced with single spaces. 203 00:09:36.600 --> 00:09:38.200 It looks completely normal. 204 00:09:38.360 --> 00:09:41.320 Yeah, the shell just ate all my custom formatting. 205 00:09:40.960 --> 00:09:43.440 Because it didn't see ten spaces. It just saw boundaries. 206 00:09:43.759 --> 00:09:45.840 It grabbed the word this, the word was, and the 207 00:09:45.879 --> 00:09:49.159 word spaced, packaged them up and handed those three neat 208 00:09:49.200 --> 00:09:51.600 little boxes to the echo command. It threw all the 209 00:09:51.600 --> 00:09:53.559 extra white space straight in the trash. 210 00:09:53.759 --> 00:09:55.879 So if you want to protect your formatting from the 211 00:09:55.919 --> 00:09:59.639 shell's aggressive interpretation, you have to lock it away using quotes. 212 00:09:59.759 --> 00:10:01.679 Yes, quoting is essential, and. 213 00:10:01.679 --> 00:10:05.159 Here we enter the absolute tyranny of quoting in Bash 214 00:10:05.559 --> 00:10:09.320 because there is a massive fundamental difference between single quotes 215 00:10:09.320 --> 00:10:12.360 and double quotes. I like to think of single quotes 216 00:10:12.399 --> 00:10:15.679 like sealing your text inside a welded steel lock box. 217 00:10:16.080 --> 00:10:17.399 That's a good way to look at it, right. 218 00:10:17.639 --> 00:10:20.759 The shell physically cannot look inside. It just hands the 219 00:10:20.799 --> 00:10:24.240 heavy box directly to the program. But double quotes are 220 00:10:24.240 --> 00:10:28.039 like putting your text in a clear plastic bag. It 221 00:10:28.159 --> 00:10:30.919 keeps the spaces together, but the shell can still peer 222 00:10:30.919 --> 00:10:32.879 through the plastic and mess with the contents. 223 00:10:33.240 --> 00:10:37.600 That is an excellent visualization. Double quotes protect your white space, 224 00:10:38.080 --> 00:10:41.679 but they still allow the shell to perform substitutions. It'll 225 00:10:41.679 --> 00:10:45.639 look inside that clear plastic bag for special characters like 226 00:10:45.720 --> 00:10:49.559 dollar signs and exclamation points and actually try to execute them. 227 00:10:49.960 --> 00:10:52.639 The cookbook has this hilarious example of how badly this 228 00:10:52.679 --> 00:10:55.759 can go wrong. Yeah, imagine you want to print a simple, 229 00:10:55.840 --> 00:10:59.960 frustrated sentence to the screen. A coffee is five dollars 230 00:11:00.519 --> 00:11:01.480 with a dollar sign. 231 00:11:01.759 --> 00:11:03.039 Very relatable script. 232 00:11:02.879 --> 00:11:05.279 If you put that in double quotes, echo, double quote, 233 00:11:05.639 --> 00:11:08.960 A coffee is dollar sign five, question mark, exclamation point, 234 00:11:09.120 --> 00:11:11.559 double quote. It fails spectacularly. 235 00:11:11.759 --> 00:11:14.840 It fails on two separate fronts. Actually, first, the shell 236 00:11:14.919 --> 00:11:17.360 looks through the double quotes and spots the dollar sign 237 00:11:17.559 --> 00:11:20.679 in bash. A dollar sign means you were calling a variable, 238 00:11:20.919 --> 00:11:23.679 so it looks for a variable named five. Unless you 239 00:11:23.759 --> 00:11:27.360 specifically created one. That variable doesn't exist, So the shell 240 00:11:27.440 --> 00:11:29.519 just replaces dollar sign five with nothing. 241 00:11:29.799 --> 00:11:31.799 It just vanishes Rea coffee. 242 00:11:31.440 --> 00:11:35.440 Poof exactly, but the exclamation point makes it so much worse. 243 00:11:35.799 --> 00:11:38.600 An exclamation point triggers history substitution. 244 00:11:38.879 --> 00:11:40.919 Wait history, Yeah. 245 00:11:40.679 --> 00:11:43.320 The shell tries to look back in your command history 246 00:11:43.360 --> 00:11:47.679 for a previous command starting with whatever follows the exclamation mark. 247 00:11:48.159 --> 00:11:50.679 It attempts to pull that old command right into your 248 00:11:50.720 --> 00:11:54.200 current sentence, fails to find a match, and throws a 249 00:11:54.240 --> 00:11:57.799 confusing event not found error. Oh man, your script just 250 00:11:57.840 --> 00:11:59.000 completely crashes. 251 00:11:59.360 --> 00:12:03.159 So learning to quote properly is literally the difference between 252 00:12:03.159 --> 00:12:05.559 a working script and a script that makes you want 253 00:12:05.559 --> 00:12:06.840 to throw your laptop out. 254 00:12:06.679 --> 00:12:07.720 A window without a doubt. 255 00:12:07.799 --> 00:12:10.200 If you just wrap that exact same sentence in single 256 00:12:10.279 --> 00:12:13.159 quote steel lock box quotes, the shell wouldn't look for 257 00:12:13.240 --> 00:12:15.919 variables or history. It would just print the sentence. 258 00:12:16.440 --> 00:12:19.559 Understanding who is intercepting your commands is half the battle here, 259 00:12:19.799 --> 00:12:22.000 and that also applies to the commands themselves. What do 260 00:12:22.039 --> 00:12:24.519 you mean, Well, sometimes you type a command and you 261 00:12:24.639 --> 00:12:26.919 don't actually know if you're talking to a program installed 262 00:12:26.960 --> 00:12:29.600 in your hard drive or a feature built directly into 263 00:12:29.639 --> 00:12:30.720 the Bash shell itself. 264 00:12:30.879 --> 00:12:33.519 Ah like the difference between an external executable and a 265 00:12:33.559 --> 00:12:36.600 built in exactly. For instance, the CD command to change directories, 266 00:12:36.639 --> 00:12:39.559 that's a built in part of Bash, but a text 267 00:12:39.679 --> 00:12:43.639 processing tool like AC is an external program. So how 268 00:12:43.639 --> 00:12:45.480 do we reveal the truth about a command? 269 00:12:46.360 --> 00:12:49.919 You ask the shell directly using the type or which commands? Okay, 270 00:12:50.000 --> 00:12:53.200 if you type type CD, the shell will explicitly reply 271 00:12:53.480 --> 00:12:56.080 CD is a shell built in. But if you type 272 00:12:56.120 --> 00:12:59.120 type OC, it will give you the physical filepath on 273 00:12:59.159 --> 00:13:01.519 your hard drive, pointing to the OUC program. 274 00:13:01.720 --> 00:13:03.039 Why does that matter so much? 275 00:13:03.200 --> 00:13:06.919 This becomes crucial when writing scripts for other people. Built 276 00:13:06.960 --> 00:13:09.960 ins will always be there, but external programs might not 277 00:13:10.039 --> 00:13:11.960 be installed on your coworker's machine. 278 00:13:12.120 --> 00:13:15.080 Oh that makes total sense. Okay, So we've covered navigating 279 00:13:15.120 --> 00:13:18.039 the system and formatting our commands so the shell doesn't 280 00:13:18.080 --> 00:13:20.600 butcher them. Now let's talk about. 281 00:13:20.360 --> 00:13:21.639 The results the output. 282 00:13:21.799 --> 00:13:24.879 Right, we've essentially built a megaphone for our programs, but 283 00:13:24.919 --> 00:13:28.039 where do they shout their answers. This brings us to 284 00:13:28.080 --> 00:13:33.039 the fundamental magic of the Unix operating system IO redirection. 285 00:13:32.919 --> 00:13:37.120 Input output redirection. In Unix, everything looks like a file. 286 00:13:37.519 --> 00:13:41.399 A text document is a file, but your keyboard is 287 00:13:41.440 --> 00:13:44.159 also treated as a file. Your screen is treated as 288 00:13:44.159 --> 00:13:44.919 a file. 289 00:13:45.120 --> 00:13:47.799 Which is such a brilliant abstraction. I don't need to 290 00:13:47.840 --> 00:13:50.440 know how to write complex code to display pixels on 291 00:13:50.480 --> 00:13:53.559 a monitor, or write drivers to interface with a USB 292 00:13:53.679 --> 00:13:56.360 flash drive, not at all. The shell handles all that 293 00:13:56.440 --> 00:13:59.960 translation using the greater than symbol the right pointing bracket. 294 00:14:00.120 --> 00:14:03.480 Exactly, if you type echo hello greater than output dot txt, 295 00:14:03.960 --> 00:14:06.440 the shell intercepts the output before it ever reaches your 296 00:14:06.440 --> 00:14:09.720 screen and channels it directly into that text file instead. 297 00:14:10.039 --> 00:14:12.440 But if I run that exact command again, it wipes 298 00:14:12.480 --> 00:14:14.399 out whatever was already in that file and replaces it. 299 00:14:14.559 --> 00:14:16.200 Rate it does it overwrites it completely. 300 00:14:16.639 --> 00:14:18.480 So to safely add to the end of the file, 301 00:14:18.519 --> 00:14:21.039 I have to use two brackets greater than greater than, 302 00:14:21.519 --> 00:14:23.440 which tells the shell to append the data. 303 00:14:23.559 --> 00:14:26.519 Appending is safer, yes, but if you want a true 304 00:14:26.559 --> 00:14:30.159 safety net, Bash actually offers a brilliant feature called no. 305 00:14:30.240 --> 00:14:32.840 Clobber no clawber. Great name it is. 306 00:14:33.320 --> 00:14:36.960 By typing set dash no clawber, you instruct the shell 307 00:14:37.039 --> 00:14:41.240 to aggressively protect existing files. If you accidentally use a 308 00:14:41.240 --> 00:14:43.440 single grater then and point it out a file that 309 00:14:43.480 --> 00:14:46.080 already has data in it, the shell will refuse to 310 00:14:46.120 --> 00:14:48.039 overwrite it and throw an error instead. 311 00:14:48.399 --> 00:14:51.600 No clawber is amazing. It really saves you from yourself. 312 00:14:51.720 --> 00:14:53.879 But what if I really want to overwrite the file 313 00:14:54.000 --> 00:14:56.159 like I've got no clawber turned on, but I know 314 00:14:56.200 --> 00:14:59.159 what I'm doing and this specific file needs to be 315 00:14:59.200 --> 00:14:59.799 wiped clean. 316 00:15:00.000 --> 00:15:02.559 When you use the do it anyway operator the overrite right, 317 00:15:02.759 --> 00:15:05.279 you type the greater than symbol followed immediately by a 318 00:15:05.320 --> 00:15:08.840 vertical bar. That syntax tells the show I know the 319 00:15:08.879 --> 00:15:11.600 safety is on, but bypass no clubber for just this 320 00:15:11.720 --> 00:15:12.919 one specific command. 321 00:15:13.159 --> 00:15:16.399 It's the manual override switch. Now, speaking of redirection, there 322 00:15:16.480 --> 00:15:18.440 is a quicky behavior with the eel's command. That always 323 00:15:18.480 --> 00:15:18.960 confused me. 324 00:15:19.080 --> 00:15:20.240 Oh the column formatting. 325 00:15:20.519 --> 00:15:24.159 Yes, when I type l's in my terminal, it formats 326 00:15:24.200 --> 00:15:27.480 the files into a beautiful, neat grid across my screen. 327 00:15:28.320 --> 00:15:31.360 But the second I redirect it to a file, l's 328 00:15:31.600 --> 00:15:34.159 greater than save dot out and then open that file, 329 00:15:34.480 --> 00:15:38.399 the grid is gone. It's just one long, single column 330 00:15:38.480 --> 00:15:39.240 of file names. 331 00:15:40.039 --> 00:15:43.279 Why does the formating completely change just because I send 332 00:15:43.320 --> 00:15:43.879 it to a file. 333 00:15:44.039 --> 00:15:46.399 That's actually by design, and it shows how smart these 334 00:15:46.440 --> 00:15:50.399 tools are. Programs can actually detect the environment they're talking to. Really, Yeah, 335 00:15:50.559 --> 00:15:53.080 The cell's command checks to see if its output is 336 00:15:53.120 --> 00:15:55.519 going to a screen, a terminal, or if it is 337 00:15:55.519 --> 00:15:58.320 being redirected into a file. If it detects a screen, 338 00:15:58.639 --> 00:16:01.279 it assumes a human is reading it, so it formats 339 00:16:01.279 --> 00:16:03.279 it into columns to save visual space. 340 00:16:03.440 --> 00:16:03.919 Makes sense. 341 00:16:04.200 --> 00:16:06.600 But if it detects a file, it assumes another computer 342 00:16:06.679 --> 00:16:09.519 program might need to read that list later, and programs 343 00:16:09.600 --> 00:16:12.759 strongly prefer reading data one line at a time, so 344 00:16:12.799 --> 00:16:15.360 it automatically switches to a single column format. 345 00:16:15.440 --> 00:16:17.879 It literally changes its behavior based on the audience. 346 00:16:18.320 --> 00:16:21.960 That leads perfectly into a classic frustration every learner faces 347 00:16:21.960 --> 00:16:25.440 with redirection. I remember I tried to redirect my compiler 348 00:16:25.559 --> 00:16:28.759 errors to a file. I type my compile command then 349 00:16:28.840 --> 00:16:30.480 greater than errors dot txt. 350 00:16:30.840 --> 00:16:31.759 Let me guess yeap. 351 00:16:32.639 --> 00:16:34.960 When I hit enter, the error is still spammed all 352 00:16:34.960 --> 00:16:37.840 over my screen, and when I checked errors dot txt 353 00:16:38.360 --> 00:16:41.960 it was completely empty. Why did the shell catch the errors? 354 00:16:42.840 --> 00:16:46.200 This requires looking under the hood at file descriptors. When 355 00:16:46.200 --> 00:16:49.240 a program runs in Unix, it opens three invisible channels 356 00:16:49.240 --> 00:16:51.600 for data to flow through. They are numbered zero, one, 357 00:16:51.840 --> 00:16:52.200 and two. 358 00:16:52.360 --> 00:16:54.080 The channels zero, one and two okay. 359 00:16:54.200 --> 00:16:57.159 Channel zero is standard input where the program gets its data, 360 00:16:57.200 --> 00:17:00.519 typically your keyboard. Channel one is standard output. This is 361 00:17:00.600 --> 00:17:03.919 the normal expected output of a program right and it 362 00:17:04.000 --> 00:17:06.839 is buffered, meaning it saves up chunks of text and 363 00:17:06.880 --> 00:17:09.319 sends them all at once for efficiency and channel two. 364 00:17:09.440 --> 00:17:12.240 Channel two is standard error. This is a completely separate 365 00:17:12.240 --> 00:17:17.799 pipeline dedicated exclusively to error messages, and crucially, it is unbuffered. 366 00:17:17.400 --> 00:17:19.359 Meaning it doesn't wait exactly. 367 00:17:19.440 --> 00:17:23.119 It spits text out instantly, character by character. That way, 368 00:17:23.200 --> 00:17:26.160 if the program crashes dramatically and takes the system down 369 00:17:26.200 --> 00:17:28.640 with it, the error message still makes it to your 370 00:17:28.680 --> 00:17:30.079 screen before the lights go out. 371 00:17:30.279 --> 00:17:33.119 So if channel one is standardshipping via the loading dock. 372 00:17:33.279 --> 00:17:36.559 Channel two is like an emergency red phone line that 373 00:17:36.680 --> 00:17:40.400 bypasses the loading dock entirely, so the error gets out instantly. 374 00:17:40.599 --> 00:17:42.359 That's a great way to put it. When you used 375 00:17:42.359 --> 00:17:46.000 a simple greater than symbol, you are only redirecting channel one. 376 00:17:46.720 --> 00:17:49.400 Oh right, A single grater then is just shorthand for 377 00:17:49.440 --> 00:17:53.039 one greater. Then you successfully redirected the normal output to 378 00:17:53.079 --> 00:17:56.160 your file which was empty because your compiler failed. But 379 00:17:56.240 --> 00:17:59.319 the errors were traveling down channel two, the red phone line, 380 00:17:59.359 --> 00:18:01.160 which was still point it's straight at your screen. 381 00:18:01.319 --> 00:18:02.440 So how do I catch channel two? 382 00:18:02.599 --> 00:18:05.079 If you just want the errors, you pipe two greater 383 00:18:05.119 --> 00:18:09.079 than errors dot txt, but usually you want both to 384 00:18:09.160 --> 00:18:12.000 catch normal output A and D errors in the same file. 385 00:18:12.079 --> 00:18:16.279 You'll often see this classic slightly cryptic syntax greater than 386 00:18:16.359 --> 00:18:18.920 errors dot txt two greater than ampersan one. 387 00:18:19.160 --> 00:18:20.960 Okay, let's break that down because it looks like a 388 00:18:21.000 --> 00:18:25.799 typo to me. Greater than errors dot txt space two 389 00:18:26.640 --> 00:18:28.400 greater than ampersand one. 390 00:18:28.960 --> 00:18:32.680 Think of it like aiming two water hoses. The first part, 391 00:18:32.720 --> 00:18:36.400 greater than errors dot txt points hose one standard output 392 00:18:36.480 --> 00:18:39.200 into a bucket called errors dot txt. 393 00:18:39.680 --> 00:18:41.400 Okay, hose one is in the bucket. 394 00:18:41.440 --> 00:18:44.880 The second part two greater than ampersan one grabs hose 395 00:18:44.960 --> 00:18:47.880 to standard error the ampersand is the key. Here it 396 00:18:47.880 --> 00:18:50.680 acts as a reference pointer. It tells theshall, don't create 397 00:18:50.680 --> 00:18:53.640 a new file named one. Instead, point hose two to 398 00:18:53.720 --> 00:18:55.559 wherever hose one is currently aimed. 399 00:18:55.759 --> 00:18:58.920 Oh, you're bundling the two hoses together into the same bucket. 400 00:18:59.039 --> 00:18:59.400 Exact. 401 00:18:59.440 --> 00:19:01.519