Speaker 1: Imagine having a conversation where like, every time you pause
Speaker 1: to take a breath, the person you're talking to completely
Speaker 1: deletes your formatting.
Speaker 2: Oh yeah, and just ignores half of what you actually meant, right, and.
Speaker 1: Then mangles your sentence into something totally unrecognizable. I mean,
Speaker 1: if you've ever typed a command into a computer terminal
Speaker 1: and watched it instantly fail, you know exactly what I mean.
Speaker 2: It's so frustrating.
Speaker 1: It really is, because we tend to think of our
Speaker 1: computers as these highly intuitive machines that you just want
Speaker 1: to help us out. But when you strip away the
Speaker 1: graphical interface, what you're left with is a blinking cursor
Speaker 1: that is ruthlessly.
Speaker 2: Literal, completely literal, and it's notoriously aggressive about how it
Speaker 2: interprets your words. To a lot of people see that
Speaker 2: blinking cursor the command line, and they just assume it's
Speaker 2: a relic.
Speaker 1: Like some archaic leftover from the nineteen.
Speaker 2: Seventies, exactly like something we only keep around for it technicians.
Speaker 2: But the command line is not a museum piece. It's
Speaker 2: a programmable, highly efficient conversation with your operating system.
Speaker 1: Which brings us to the core of our deep dive today.
Speaker 1: I'm your host, and today we're pilling some incredible insights
Speaker 1: from the Bash Cookbook, second edition by Carl Albing and
Speaker 1: JP Vawson.
Speaker 2: It's a fantastic resource.
Speaker 1: It really is. And our mission today is to crack
Speaker 1: open that dreaded cryptic command line interface. We're going to
Speaker 1: decode the strange punctuation, learn how to pipe data around
Speaker 1: like a well, like a master plumber.
Speaker 2: Really, I love that analogy.
Speaker 1: Thanks, and we'll discover why this text based tool is
Speaker 1: still the absolute fastest shortcut to getting complex work done.
Speaker 2: Because once you understand the grammar of this specific conversation,
Speaker 2: it changes everything. It remembers your commands, lets you edit
Speaker 2: them on the fly, and honestly, it can automate literally
Speaker 2: any repetitive task you throw at it.
Speaker 1: But before we start typing commands, we kind of need
Speaker 1: to address what this black box actually is, because the
Speaker 1: command line, or the shell as it's called, isn't actually
Speaker 1: the operating system itself, which I have to admit blew
Speaker 1: my mind a bit. I always assume the terminal window
Speaker 1: was the core of the computer.
Speaker 2: Oh, that's a really common misconception. But the separation between
Speaker 2: the two is actually a foundational part of the original
Speaker 2: Unix philosophy. How so well, in the earliest operating systems,
Speaker 2: the user interface, the part that actually accepted your typed commands,
Speaker 2: was hardwired straight into the operating system's core brain.
Speaker 1: So they were bundled together, right, So if.
Speaker 2: You wanted a better way to type commands for just
Speaker 2: a new feature, you had to rewrite the entire operating
Speaker 2: system to get.
Speaker 1: It, which sounds a bit like trying to upgrade your
Speaker 1: car's spearing wheel by replacing the entire engine.
Speaker 2: That is a perfect analogy. It was incredibly inefficient, so
Speaker 2: the creators of Unix separated them entirely. The operating system
Speaker 2: handles the hardware, the memory, the heavy lifting, the engine
Speaker 2: exactly the engine, and the shell is just a separate,
Speaker 2: replaceable dashboard. Its only job is to listen to you
Speaker 2: interpret your command and then pass it down to the engine.
Speaker 2: And because it was a separate program, developers could write new,
Speaker 2: better shells without ever touching the core system.
Speaker 1: So you really could just swap out the dashboard whenever
Speaker 1: you want. You could use shells with names like Crow
Speaker 1: or KSh or Soush yep.
Speaker 2: Absolutely, but today.
Speaker 1: Our sources are focused entirely on Bash So why did
Speaker 1: Bash become, you know, the universal rental car that everyone
Speaker 1: knows how to drive.
Speaker 2: It really comes down to a perfect storm of timing
Speaker 2: and utility. Bash stands for the GNU born again.
Speaker 1: Shell right, great pun by the way, Oh.
Speaker 2: Programmers love their puns. But back in the late nineteen eighties,
Speaker 2: the Gene project was trying to create a free, complete
Speaker 2: operating system that followed PO six standards and posx.
Speaker 1: Is essentially the universal rule book right for how Unix
Speaker 1: systems should behave so software can run anywhere exactly.
Speaker 2: They needed a standard shell for this free system, so
Speaker 2: they built Bash by taking the heavy duty programming features
Speaker 2: that developers required and combining them with the interactive conveniences
Speaker 2: that everyday users wanted.
Speaker 1: Things like just hitting the up arrow to see your
Speaker 1: last command.
Speaker 2: Yes, small things that made a huge difference.
Speaker 1: Well, it clearly won the war. I mean, it's the
Speaker 1: default on almost all Linux distributions historically, the default on
Speaker 1: mac os, and you can even install it on Windows
Speaker 1: through the Windows subsystem. For a Linux, it's everywhere. So
Speaker 1: you learn Bash and suddenly you have a passport to
Speaker 1: talk to almost any machine on the planet.
Speaker 2: You do, but to speak the language you first have
Speaker 2: to read the room. When you open that terminal, you're
Speaker 2: greeted by a prompt, and that prompt usually ends with
Speaker 2: a specific symbol, often a dollar sign or a hash symbol. Right,
Speaker 2: and that tiny character is actually screaming a warning at
Speaker 2: you if you know how to listen.
Speaker 1: So let's decode that for the listener. If my prompt
Speaker 1: ends in a dollar sign, the shell is basically saying
Speaker 1: you are a regular user, you have normal safe permissions exactly.
Speaker 1: But if it ends in a hash sign like the
Speaker 1: number symbol, it's saying you are.
Speaker 2: Root, the all powerful system administrator. As route, there are
Speaker 2: no safety rails whatsoever. You can do.
Speaker 1: Anything, which also means you can accidentally delete the entire
Speaker 1: hard drive with a single typo.
Speaker 2: Yep, seeing that hash sign is your visual queue to
Speaker 2: double check every single keystroke.
Speaker 1: Okay, So the prompt tells us who we are. It
Speaker 1: also usually tries and tell us where we are in
Speaker 1: the computer's folder structure. But the cookbook points out something
Speaker 1: incredibly sneaky.
Speaker 2: Here, oh about the paths.
Speaker 1: Yeah, the prompt sometimes lies to you, or at least
Speaker 1: it gives you an illusion like, wait, why would my
Speaker 1: computer lie to me about what? Folder I'm in.
Speaker 2: Well, it's not so much lying as it is keeping
Speaker 2: up a facade that you created. Okay, fair, Think about
Speaker 2: shortcuts on your desktop. In the Unix world, these are
Speaker 2: called symbolic links. You might have a folder on your
Speaker 2: desktop called photos, but the actual files are stored deep
Speaker 2: inside some external hard drive. If you navigate into that
Speaker 2: desktop folder and use the PW beauty command print working directory,
Speaker 2: the shell might just say you are on the desktop.
Speaker 1: Because that's the path I took to get there.
Speaker 2: Precisely, If you type pdbudd dash L, it gives you
Speaker 2: your logical path that's where you think you are based
Speaker 2: on the shortcuts you followed.
Speaker 1: Got it.
Speaker 2: But if if you need to know where those files
Speaker 2: physically reside on the spinning disc, you use PWD dash
Speaker 2: p P for physical Right. The dash P strips away
Speaker 2: all the illusions and reveals the true physical path.
Speaker 1: That distinction alone can save like hours of troubleshooting. Yeah,
Speaker 1: but here is where navigating gets truly dangerous. We all
Speaker 1: have hidden files on our computers right. Oh yes, and
Speaker 1: in Unix they're called dot files because their names literally
Speaker 1: just start with a period exactly. So if I want
Speaker 1: to see them, my instinct as a user is to
Speaker 1: use a wildcard. I know what asterisk means everything, so
Speaker 1: I figured I type the list command ls followed by
Speaker 1: dot asterisk. You have to see all my hidden dot
Speaker 1: file which seems totally logical, right, But the cookbook says
Speaker 1: this will cause a catastrophic wall of text. Why if
Speaker 1: the asterisk means everything, shouldn't dot asterisk just list my
Speaker 1: hidden files?
Speaker 2: This is such a classic trap, and it introduces a
Speaker 2: process called globbing, which is how the shell actually expands
Speaker 2: wildcards wlobbing. Okay, so the asterisk does mean any characters
Speaker 2: zero or more times. You're totally correct that dot asterisk
Speaker 2: matches any file starting with a dot.
Speaker 1: Okay, so what's the problem.
Speaker 2: The danger lies in how the Unix filesystem is built.
Speaker 2: In every single directory without exception, there are two invisible folders.
Speaker 2: One is named with a single dot, which is a
Speaker 2: shortcut representing the directory you are currently in, ah right,
Speaker 2: and the other is named with two dots dot dot,
Speaker 2: which is a shortcut representing the parent directory exactly one
Speaker 2: level above.
Speaker 1: Oh, I see where this is going.
Speaker 2: Yeah, when you type l's dot asterisk, the shell aggressively
Speaker 2: expands that wildcard before the l's command even runs. Oh no,
Speaker 2: it matches your hidden files, sure, but it also matches
Speaker 2: the single dot and the double dot. So the shell
Speaker 2: tells the l's command, please list the contents of the
Speaker 2: hidden files, and the contents of the current folder, and
Speaker 2: the entire contents of the folder above this.
Speaker 1: One, which could be my entire home direction exactly. It
Speaker 1: would just dump thousands of files onto my screen endlessly.
Speaker 1: So how do we actually isolate the hidden files without
Speaker 1: triggering an absolute avalanche?
Speaker 2: The cookbook provides a brilliantly precise piece of syntax.
Speaker 1: Or this.
Speaker 2: You type dot, dot, bracket, exclamation point, dot, close bracket
Speaker 2: as risk.
Speaker 1: Okay, let me translate that out of alphabet soup a dot,
Speaker 1: then an open bracket, an exclamation point, another dot, a
Speaker 1: closed bracket, and finally an asterisk. What is the logic
Speaker 1: behind that?
Speaker 2: So brackets let you define a specific pool of allowed characters.
Speaker 2: But when you put an exclamation point right inside the
Speaker 2: opening bracket, it acts as a negator, like saying not
Speaker 2: this exactly. It means absolutely not this character. So that
Speaker 2: whole sequence translates to find a file that starts with
Speaker 2: a dot. Now make sure the very next character is
Speaker 2: absolutely not a dot. After that, the asterisk says you
Speaker 2: can grab whatever is left. It perfectly targets your hidden
Speaker 2: files while explicitly ignoring the single and double dot directories.
Speaker 1: That is surgically precise, okay, and it brings us to
Speaker 1: a massive realization about Bash. How you format your text
Speaker 1: matters just as much as the command itself. Oh completely,
Speaker 1: Because that shell is an aggressive listener. It intercepts parses
Speaker 1: and modifies your words before it ever passes them to
Speaker 1: the actual program you're trying to run.
Speaker 2: That aggressive listening is exactly why spacing can drive new
Speaker 2: users crazy. The shell uses spaces purely to figure out
Speaker 2: where one argument ends and another begins.
Speaker 1: Right, So, if I use the echo command, which just
Speaker 1: prints whatever you type back to the screen, A type echo,
Speaker 1: then the word this, Then I hit the space bar
Speaker 1: ten times, then the word was, ten more spaces, and
Speaker 1: the word spaced. What comes out on my screen is
Speaker 1: just this was spaced with single spaces.
Speaker 2: It looks completely normal.
Speaker 1: Yeah, the shell just ate all my custom formatting.
Speaker 2: Because it didn't see ten spaces. It just saw boundaries.
Speaker 2: It grabbed the word this, the word was, and the
Speaker 2: word spaced, packaged them up and handed those three neat
Speaker 2: little boxes to the echo command. It threw all the
Speaker 2: extra white space straight in the trash.
Speaker 1: So if you want to protect your formatting from the
Speaker 1: shell's aggressive interpretation, you have to lock it away using quotes.
Speaker 2: Yes, quoting is essential, and.
Speaker 1: Here we enter the absolute tyranny of quoting in Bash
Speaker 1: because there is a massive fundamental difference between single quotes
Speaker 1: and double quotes. I like to think of single quotes
Speaker 1: like sealing your text inside a welded steel lock box.
Speaker 2: That's a good way to look at it, right.
Speaker 1: The shell physically cannot look inside. It just hands the
Speaker 1: heavy box directly to the program. But double quotes are
Speaker 1: like putting your text in a clear plastic bag. It
Speaker 1: keeps the spaces together, but the shell can still peer
Speaker 1: through the plastic and mess with the contents.
Speaker 2: That is an excellent visualization. Double quotes protect your white space,
Speaker 2: but they still allow the shell to perform substitutions. It'll
Speaker 2: look inside that clear plastic bag for special characters like
Speaker 2: dollar signs and exclamation points and actually try to execute them.
Speaker 1: The cookbook has this hilarious example of how badly this
Speaker 1: can go wrong. Yeah, imagine you want to print a simple,
Speaker 1: frustrated sentence to the screen. A coffee is five dollars
Speaker 1: with a dollar sign.
Speaker 2: Very relatable script.
Speaker 1: If you put that in double quotes, echo, double quote,
Speaker 1: A coffee is dollar sign five, question mark, exclamation point,
Speaker 1: double quote. It fails spectacularly.
Speaker 2: It fails on two separate fronts. Actually, first, the shell
Speaker 2: looks through the double quotes and spots the dollar sign
Speaker 2: in bash. A dollar sign means you were calling a variable,
Speaker 2: so it looks for a variable named five. Unless you
Speaker 2: specifically created one. That variable doesn't exist, So the shell
Speaker 2: just replaces dollar sign five with nothing.
Speaker 1: It just vanishes Rea coffee.
Speaker 2: Poof exactly, but the exclamation point makes it so much worse.
Speaker 2: An exclamation point triggers history substitution.
Speaker 1: Wait history, Yeah.
Speaker 2: The shell tries to look back in your command history
Speaker 2: for a previous command starting with whatever follows the exclamation mark.
Speaker 2: It attempts to pull that old command right into your
Speaker 2: current sentence, fails to find a match, and throws a
Speaker 2: confusing event not found error. Oh man, your script just
Speaker 2: completely crashes.
Speaker 1: So learning to quote properly is literally the difference between
Speaker 1: a working script and a script that makes you want
Speaker 1: to throw your laptop out.
Speaker 2: A window without a doubt.
Speaker 1: If you just wrap that exact same sentence in single
Speaker 1: quote steel lock box quotes, the shell wouldn't look for
Speaker 1: variables or history. It would just print the sentence.
Speaker 2: Understanding who is intercepting your commands is half the battle here,
Speaker 2: and that also applies to the commands themselves. What do
Speaker 2: you mean, Well, sometimes you type a command and you
Speaker 2: don't actually know if you're talking to a program installed
Speaker 2: in your hard drive or a feature built directly into
Speaker 2: the Bash shell itself.
Speaker 1: Ah like the difference between an external executable and a
Speaker 1: built in exactly. For instance, the CD command to change directories,
Speaker 1: that's a built in part of Bash, but a text
Speaker 1: processing tool like AC is an external program. So how
Speaker 1: do we reveal the truth about a command?
Speaker 2: You ask the shell directly using the type or which commands? Okay,
Speaker 2: if you type type CD, the shell will explicitly reply
Speaker 2: CD is a shell built in. But if you type
Speaker 2: type OC, it will give you the physical filepath on
Speaker 2: your hard drive, pointing to the OUC program.
Speaker 1: Why does that matter so much?
Speaker 2: This becomes crucial when writing scripts for other people. Built
Speaker 2: ins will always be there, but external programs might not
Speaker 2: be installed on your coworker's machine.
Speaker 1: Oh that makes total sense. Okay, So we've covered navigating
Speaker 1: the system and formatting our commands so the shell doesn't
Speaker 1: butcher them. Now let's talk about.
Speaker 2: The results the output.
Speaker 1: Right, we've essentially built a megaphone for our programs, but
Speaker 1: where do they shout their answers. This brings us to
Speaker 1: the fundamental magic of the Unix operating system IO redirection.
Speaker 2: Input output redirection. In Unix, everything looks like a file.
Speaker 2: A text document is a file, but your keyboard is
Speaker 2: also treated as a file. Your screen is treated as
Speaker 2: a file.
Speaker 1: Which is such a brilliant abstraction. I don't need to
Speaker 1: know how to write complex code to display pixels on
Speaker 1: a monitor, or write drivers to interface with a USB
Speaker 1: flash drive, not at all. The shell handles all that
Speaker 1: translation using the greater than symbol the right pointing bracket.
Speaker 2: Exactly, if you type echo hello greater than output dot txt,
Speaker 2: the shell intercepts the output before it ever reaches your
Speaker 2: screen and channels it directly into that text file instead.
Speaker 1: But if I run that exact command again, it wipes
Speaker 1: out whatever was already in that file and replaces it.
Speaker 2: Rate it does it overwrites it completely.
Speaker 1: So to safely add to the end of the file,
Speaker 1: I have to use two brackets greater than greater than,
Speaker 1: which tells the shell to append the data.
Speaker 2: Appending is safer, yes, but if you want a true
Speaker 2: safety net, Bash actually offers a brilliant feature called no.
Speaker 1: Clobber no clawber. Great name it is.
Speaker 2: By typing set dash no clawber, you instruct the shell
Speaker 2: to aggressively protect existing files. If you accidentally use a
Speaker 2: single grater then and point it out a file that
Speaker 2: already has data in it, the shell will refuse to
Speaker 2: overwrite it and throw an error instead.
Speaker 1: No clawber is amazing. It really saves you from yourself.
Speaker 1: But what if I really want to overwrite the file
Speaker 1: like I've got no clawber turned on, but I know
Speaker 1: what I'm doing and this specific file needs to be
Speaker 1: wiped clean.
Speaker 2: When you use the do it anyway operator the overrite right,
Speaker 2: you type the greater than symbol followed immediately by a
Speaker 2: vertical bar. That syntax tells the show I know the
Speaker 2: safety is on, but bypass no clubber for just this
Speaker 2: one specific command.
Speaker 1: It's the manual override switch. Now, speaking of redirection, there
Speaker 1: is a quicky behavior with the eel's command. That always
Speaker 1: confused me.
Speaker 2: Oh the column formatting.
Speaker 1: Yes, when I type l's in my terminal, it formats
Speaker 1: the files into a beautiful, neat grid across my screen.
Speaker 1: But the second I redirect it to a file, l's
Speaker 1: greater than save dot out and then open that file,
Speaker 1: the grid is gone. It's just one long, single column
Speaker 1: of file names.
Speaker 2: Why does the formating completely change just because I send
Speaker 2: it to a file.
Speaker 1: That's actually by design, and it shows how smart these
Speaker 1: tools are. Programs can actually detect the environment they're talking to. Really, Yeah,
Speaker 1: The cell's command checks to see if its output is
Speaker 1: going to a screen, a terminal, or if it is
Speaker 1: being redirected into a file. If it detects a screen,
Speaker 1: it assumes a human is reading it, so it formats
Speaker 1: it into columns to save visual space.
Speaker 2: Makes sense.
Speaker 1: But if it detects a file, it assumes another computer
Speaker 1: program might need to read that list later, and programs
Speaker 1: strongly prefer reading data one line at a time, so
Speaker 1: it automatically switches to a single column format.
Speaker 2: It literally changes its behavior based on the audience.
Speaker 1: That leads perfectly into a classic frustration every learner faces
Speaker 1: with redirection. I remember I tried to redirect my compiler
Speaker 1: errors to a file. I type my compile command then
Speaker 1: greater than errors dot txt.
Speaker 2: Let me guess yeap.
Speaker 1: When I hit enter, the error is still spammed all
Speaker 1: over my screen, and when I checked errors dot txt
Speaker 1: it was completely empty. Why did the shell catch the errors?
Speaker 2: This requires looking under the hood at file descriptors. When
Speaker 2: a program runs in Unix, it opens three invisible channels
Speaker 2: for data to flow through. They are numbered zero, one,
Speaker 2: and two.
Speaker 1: The channels zero, one and two okay.
Speaker 2: Channel zero is standard input where the program gets its data,
Speaker 2: typically your keyboard. Channel one is standard output. This is
Speaker 2: the normal expected output of a program right and it
Speaker 2: is buffered, meaning it saves up chunks of text and
Speaker 2: sends them all at once for efficiency and channel two.
Speaker 2: Channel two is standard error. This is a completely separate
Speaker 2: pipeline dedicated exclusively to error messages, and crucially, it is unbuffered.
Speaker 1: Meaning it doesn't wait exactly.
Speaker 2: It spits text out instantly, character by character. That way,
Speaker 2: if the program crashes dramatically and takes the system down
Speaker 2: with it, the error message still makes it to your
Speaker 2: screen before the lights go out.
Speaker 1: So if channel one is standardshipping via the loading dock.
Speaker 1: Channel two is like an emergency red phone line that
Speaker 1: bypasses the loading dock entirely, so the error gets out instantly.
Speaker 2: That's a great way to put it. When you used
Speaker 2: a simple greater than symbol, you are only redirecting channel one.
Speaker 2: Oh right, A single grater then is just shorthand for
Speaker 2: one greater. Then you successfully redirected the normal output to
Speaker 2: your file which was empty because your compiler failed. But
Speaker 2: the errors were traveling down channel two, the red phone line,
Speaker 2: which was still point it's straight at your screen.
Speaker 1: So how do I catch channel two?
Speaker 2: If you just want the errors, you pipe two greater
Speaker 2: than errors dot txt, but usually you want both to
Speaker 2: catch normal output A and D errors in the same file.
Speaker 2: You'll often see this classic slightly cryptic syntax greater than
Speaker 2: errors dot txt two greater than ampersan one.
Speaker 1: Okay, let's break that down because it looks like a
Speaker 1: typo to me. Greater than errors dot txt space two
Speaker 1: greater than ampersand one.
Speaker 2: Think of it like aiming two water hoses. The first part,
Speaker 2: greater than errors dot txt points hose one standard output
Speaker 2: into a bucket called errors dot txt.
Speaker 1: Okay, hose one is in the bucket.
Speaker 2: The second part two greater than ampersan one grabs hose
Speaker 2: to standard error the ampersand is the key. Here it
Speaker 2: acts as a reference pointer. It tells theshall, don't create
Speaker 2: a new file named one. Instead, point hose two to
Speaker 2: wherever hose one is currently aimed.
Speaker 1: Oh, you're bundling the two hoses together into the same bucket.
Speaker 2: Exact.
Speaker 1: That makes so much much sense visually. Yeah. Now, if
Speaker 1: we can seamlessly funnel output into a bucket, the next
Speaker 1: logical leap is what if we funnel it straight into
Speaker 1: another program? Oh? Yes, This is the digital plumbing of Bash.
Speaker 2: The pipe, the pipe symbol. It's that vertical bar on
Speaker 2: your keyboard, usually above the ender key. It is arguably
Speaker 2: one of the most powerful concepts in all of computing.
Speaker 2: It takes the standard output of the program on the
Speaker 2: left and funnels it directly into the standard input of
Speaker 2: the program on the right.
Speaker 1: It's exactly like an assembly line conveyor belt. You have
Speaker 1: one small machine that just finds files. It plops them
Speaker 1: on the belt. The belt feeds them directly into a
Speaker 1: machine that sorts them alphabetically. Right, that machine drops them
Speaker 1: on a belt feeding into a machine that removes duplicates.
Speaker 1: You're building a custom complex factory out of simple single
Speaker 1: purpose tools.
Speaker 2: M Because it's a pipe, it's actually performing rudimentary parallel processing.
Speaker 2: The second program doesn't read for the first program to
Speaker 2: finish completely. As soon as the first program produces a
Speaker 2: single drop of data, it flows down the pie to
Speaker 2: the next program.
Speaker 1: But what if I'm building this complex pipeline and I
Speaker 1: want to save a copy of the data halfway down
Speaker 1: the assembly line without stopping the belt, Like I want
Speaker 1: to sort the data, save a copy to check my work,
Speaker 1: and then pass it to the duplicate remover.
Speaker 2: Then you use a tool literally named after physical plumbing,
Speaker 2: the T command tee. It acts exactly like a T
Speaker 2: joint under your sink. You place it in your pipeline
Speaker 2: for example, sort pipetcheck dot txt pipe unique. The T
Speaker 2: command takes the flowing data, pours a copy into check
Speaker 2: dot txt, and simultaneously lets the original stream continue down
Speaker 2: the pipe so the next program can consume it.
Speaker 1: It's perfect for debugging a long chain of commands. But
Speaker 1: and there is a catch with pipes, right, Pipes only
Speaker 1: pass data into a program standard input.
Speaker 2: That's true, and some.
Speaker 1: Programs stubbornly refuse to listen to standard input. They only
Speaker 1: take arguments provide it directly on the command line the
Speaker 1: ROM command. The remove command is a classic example. I
Speaker 1: can't pipe a list of files into ARM, So how
Speaker 1: do I use the output of a search command to deles.
Speaker 2: Use command substitution instead of a pipe. You wrap your
Speaker 2: search command in a dollar sign and parentheses like this
Speaker 2: dot rm dollar sign open parenthesis, fine dot dash name
Speaker 2: asterisk dot beck close parenthesis.
Speaker 1: So dollar sign open parenthesis, the fine command close parenthesis.
Speaker 1: How's that different from a pipe?
Speaker 2: The shell sees that dollar sign parenthesis syntax, and it
Speaker 2: essentially presses pause on your main command. It opens a
Speaker 2: completely separate subshell runs the inner fine command, gathers all
Speaker 2: the texts that command spits out, and literally paste that
Speaker 2: text back onto the command line right after the ARM wow.
Speaker 2: It substitutes the command with its own output, and then
Speaker 2: presses play on the main ARM command.
Speaker 1: That sounds incredibly powerful, but also incredibly dangerous.
Speaker 2: Oh, it is the definition of dangerous. RAM is unforgiving.
Speaker 2: There is no recycle bin on the command line. Right
Speaker 2: if you make a typo in that inner fine command
Speaker 2: and it locates more files than you intended. Say it
Speaker 2: accidentally lists every file in your home directory. The subshell
Speaker 2: will paste all of those file names directly into the
Speaker 2: arm command and they are gone forever ouch.
Speaker 1: So the rule is always test the inner command first.
Speaker 1: Always run the fine command by itself. Verify the list
Speaker 1: of files on your screen. If it is exactly what
Speaker 1: you want to delete, then you can wrap it in
Speaker 1: the dollar sign in parentheses and hand it to RAM.
Speaker 2: Precision is everything.
Speaker 1: Okay, so we've mastered building megaphones. We can pipe data out,
Speaker 1: redirect it, and substitute it. Let's pivot. What if a
Speaker 1: program needs us to feed it raw materials? First, how
Speaker 1: do we force feed data into a system. Because this
Speaker 1: is where shell scripts evolve from static lists of commands
Speaker 1: into truly interactive tools.
Speaker 2: Well, if output redirection uses the greater than signed to
Speaker 2: push data out, input redirection uses the less then signed
Speaker 2: to pull data in, it points the other way, right.
Speaker 2: If a program expects you to manually type data on
Speaker 2: the keyboard, you can just use less than data dot
Speaker 2: txt to pour the contents of that file directly into
Speaker 2: the program's input.
Speaker 1: But what if I am writing a script to share
Speaker 1: with my team, and I don't want to email them
Speaker 1: the script file on a separate data file. I want
Speaker 1: the data hard coded right inside the script itself.
Speaker 2: You use what's called a here document. The syntax is
Speaker 2: two less than signs followed by a marker word. Traditionally,
Speaker 2: programmers use eof, which stands for end of file.
Speaker 1: Wait, how does less than less than eof work? Conceptually?
Speaker 2: Normally, a script asks for a file on your hard
Speaker 2: drive to get data. The double less then tells the
Speaker 2: shell don't go looking on the hard drive. The file
Speaker 2: is right here glued to the bottom of this script.
Speaker 2: Keep reading the following lines and treat them as raw
Speaker 2: data until you hit this exact password eof on a
Speaker 2: line by itself.
Speaker 1: It's like an inline data stream. But the cookbook mentions
Speaker 1: a bizarre bug that happens with here documents.
Speaker 2: Oh, the dollar sign issue.
Speaker 1: Yeah. Let's say I write a script to log charitable donations.
Speaker 1: I use a here document to store the data. PETE
Speaker 1: gave one hundred dollars with a dollar sign. But when
Speaker 1: I run the script, it prints out PETE gave zero zero.
Speaker 1: What did one hundred bucks go?
Speaker 2: We are back to the aggressive listener and the tyranny
Speaker 2: of the dollar sign Even inside a here document, the
Speaker 2: shell is still scanning your text. No, it sees dollar
Speaker 2: sign one hundred. It assumes dollar sign one is a
Speaker 2: variable you are trying to call. That variable is empty,
Speaker 2: so it deletes the dollar sign one, leaving just the
Speaker 2: zero zero.
Speaker 1: So how do we stop it from scanning the data.
Speaker 2: We go back to our quoting rules. If you put
Speaker 2: single quotes around your marker word typing less than quote
Speaker 2: eof quote, you are essentially locking the entire here document
Speaker 2: in that steelbox. You're telling the shell turn off all
Speaker 2: special features. Do not expand variables. Just read this data
Speaker 2: exactly is written. Your one hundred dollars remains one hundred dollars.
Speaker 1: That is a subtle fix, but incredibly powerful. Though the
Speaker 1: cookbook also notes the eof marker itself is incredibly.
Speaker 2: Fragile, extremely fragile. The ending marker must be pristine. If
Speaker 2: you have even a single trailing blank space after the
Speaker 2: word eof at the end of your data, Bash will
Speaker 2: recognize it.
Speaker 1: It'll just keep going.
Speaker 2: Yes, it will think the data is still going. It
Speaker 2: will swallow the rest of your script, treating your actual
Speaker 2: commands as text data, and the whole thing will silently.
Speaker 1: Break No trailing space is allowed. Got it?
Speaker 2: Now? What if we don't want a hard code data.
Speaker 2: What if we want our script to actually pause and
Speaker 2: ask the human a question in real time?
Speaker 1: That is the job of the read command. It grabs
Speaker 1: input directly from the keyboard and stores it in a variable,
Speaker 1: and it has some fantastic flags for usability.
Speaker 2: Like what You can use read dash P to provide
Speaker 2: a custom prompt like read dash P enter your name.
Speaker 1: And my absolute favorite feature is read dash T, which
Speaker 1: sets a time limit. Yes, you could do read dash
Speaker 1: T three dash pe answer quickly. If the user doesn't
Speaker 1: type anything within three seconds, the read command times out
Speaker 1: and the script just moves on to the next line.
Speaker 2: It's brilliant for automation. It allows you to write scripts
Speaker 2: that run on their own with default behaviors, but still
Speaker 2: give you a three second window to interrupt them if
Speaker 2: you happen to be watching the screen. It's the mark
Speaker 2: of a truly professional script.
Speaker 1: So we started this deep dive looking at a blinking cursor,
Speaker 1: feeling completely intimidated by its aggressive, literal nature. But now
Speaker 1: we see it for what it is. The command line
Speaker 1: isn't a nostalgic textbox. It is the connective tissue of
Speaker 1: the operating system.
Speaker 2: It really is.
Speaker 1: By understanding the rules of quoting, knowing how filed descriptors
Speaker 1: route your data, and mastering the plumbing of pipes and redirects,
Speaker 1: you stop just using your computer and you start commanding it.
Speaker 2: And that is the true joy of Bash. It rewards
Speaker 2: precision and it vastly amplifies your creative.
Speaker 1: So we'll leave you with a challenge. Think about your
Speaker 1: daily digital life. The command line is basically just a
Speaker 1: highly efficient conversation using tiny, modular verbs. What complex, repetitive
Speaker 1: task do you do every single week? Resizing images, organizing downloads,
Speaker 1: searching for errors and logs that could be completely automated
Speaker 1: with just three or four words piped together. The cursor
Speaker 1: is blinking. It's waiting for you to start the conversation.