Speaker 1: So if you ask an artificial intelligence to write a
Speaker 1: Shakespearean saunet about I don't know a toaster.
Speaker 2: Right, it just does it in like three seconds.
Speaker 1: Flat, exactly. It's so incredibly fast and honestly so convincingly
Speaker 1: human that it's really easy to just throw our hands
Speaker 1: up and say, well, the computer is just thinking.
Speaker 2: Yeah, that's the illusion.
Speaker 1: But here is the secret that the tech world sort
Speaker 1: of you know, blides right past. The AI doesn't actually
Speaker 1: know what a toaster is.
Speaker 2: No, not at all.
Speaker 1: It doesn't know what a poem is. It's not experiencing
Speaker 1: this burst of creative genius underneath the hood. It is
Speaker 1: really just doing a massive amount of incredibly fast, very
Speaker 1: boring accounting.
Speaker 2: Which is exactly what we're going to get into.
Speaker 1: Right. So today, for you, our listener, we're opening that ledger.
Speaker 1: We are taking you on a custom tailored deep dive
Speaker 1: to totally demystify how artificial intelligence actually you know, learns.
Speaker 2: Yeah, no magic, no impenetrable labyrinths, just the raw mechanics.
Speaker 1: The mechanics.
Speaker 2: Because I mean, we appreciate the result of a neural network, right,
Speaker 2: we rarely understand the underlying chemistry of how it actually
Speaker 2: got there.
Speaker 1: It's totally a black box for most people exactly.
Speaker 2: So our guide for pulling back the curtain today is
Speaker 2: this fantastic book by Seth Widman. It's called deep Learning
Speaker 2: from Scratch, Building with Python from first principles A great resource,
Speaker 2: really is. And what we're going to do is take
Speaker 2: all that intimidating jargon, you know, the algorithms, the calculus,
Speaker 2: the hyper parameteris very stuff, all of it, and we're
Speaker 2: going to strip it all the way down to the
Speaker 2: foundational floorboards.
Speaker 1: So we're going to use simple math, visual diagrams, and
Speaker 1: some basic code to show you that deep learning is
Speaker 1: really just a highly scaled assembly line of very very
Speaker 1: simple mathematical factories.
Speaker 2: That's a great way to put it.
Speaker 1: But before we actually get to building that assembly line,
Speaker 1: we need to talk about why loving this stuff in
Speaker 1: the first place is normally such a complete nightmare. Oh
Speaker 1: it really is, because if you try to read a
Speaker 1: standard academic paper on neural networks, it often feels like
Speaker 1: you're trying to read ancient Greek. Why is the entry
Speaker 1: point so brutal?
Speaker 2: Well, Wideman tackles this right out of the gate. He
Speaker 2: uses that old parable of the blind men and the elephant.
Speaker 1: Oh right, sure.
Speaker 2: So you have a group of blind men who encounter
Speaker 2: an elephant for the first time. One touches the trunk
Speaker 2: and says, oh, an elephant is like a thick snake, right,
Speaker 2: Another touches the year and says, no, it's a fan.
Speaker 2: Another grabs the leg and says it's a tree trunk.
Speaker 1: And they're all kind of right but also completely wrong exactly.
Speaker 2: They are all describing a correct, isolated part, but none
Speaker 2: of them are describing the whole animal. And deep learning
Speaker 2: resources have historically done the exact same thing.
Speaker 1: Okay, that makes a lot of sense because, like, if
Speaker 1: you want to learn a standard computer science concept, say
Speaker 1: how a search algorithm works, the resources out there are
Speaker 1: usually holistic, Like a good textbook gives you a plain
Speaker 1: English explanation, then they give you a whiteboard diagram, then
Speaker 1: the math, and finally the pseudo code so you can
Speaker 1: actually build it. You get the whole elephant, right, you
Speaker 1: get the whole elephant. But AI resources fracture this, don't They.
Speaker 2: Completely The field sort of fractured into two really extreme camps.
Speaker 2: On one side, you have these highly conceptual, incredibly dense.
Speaker 1: Math textbooks, the ancient Greek exactly.
Speaker 2: Wideman points to Ian Goodfellow's famous Deep Learning book and look,
Speaker 2: it's an absolute masterpiece. Sure, but if you aren't already
Speaker 2: fluent in advanced calculus and linear algebra, you're going to
Speaker 2: hit a brook wall on like page ten, it's just
Speaker 2: a sea of abstract equations.
Speaker 1: So what's the other extreme then, Because if I don't
Speaker 1: want to drown in calculus, where do people usually go?
Speaker 2: Well, they go to the highly practical, code heavy tutorials.
Speaker 2: So you might look up the documentation for a modern
Speaker 2: library like PyTorch okay, and you just copy a block
Speaker 2: of Python code, you paste it, you run it, and
Speaker 2: you watch this number on your screen called the loss
Speaker 2: value start to go down.
Speaker 1: Which means it's working right.
Speaker 2: Technically, Yes, the network is learning, but the tutorial never
Speaker 2: actually stops to explain the why. It's like you're driving
Speaker 2: a sports car but you have zero clue how the
Speaker 2: internal combustion engine works.
Speaker 1: Which is I'm guessing where Widman's approach comes in. He argues,
Speaker 1: you have to merge these perspectives.
Speaker 2: Yes, exactly. His core thesis is that to truly understand
Speaker 2: neural networks, you have to hold multiple mental models in
Speaker 2: your head simultaneously.
Speaker 1: Okay, what does that look like?
Speaker 2: Well, you have to look at a neural network and
Speaker 2: see it as a mathematical function, but at the exact
Speaker 2: same time, you have to see it as a computational
Speaker 2: graph where data physically flows from left to right. Got it.
Speaker 2: You also have to see it as a series of
Speaker 2: layered neurons, and finally, you have to understand it conceptually
Speaker 2: as a universal function approximator.
Speaker 1: Wait, hold on, a universal function approximator. Yeah, that sounds
Speaker 1: like a fancy blender from a late night infomercial or something.
Speaker 1: What does that actually mean?
Speaker 2: I know it sounds super intimidating, but it just means
Speaker 2: a machine that can mold it self to approximate literally
Speaker 2: any pattern in the universe, provided it has enough parts.
Speaker 2: Any pattern, pretty much, whether the pattern is predicting tomorrow's weather,
Speaker 2: or recognizing a cat in a photo, or translating English
Speaker 2: to French. If there's a logical relationship between the input
Speaker 2: and the output, a neural network can approximate it. That's wild,
Speaker 2: it is, But you only realize how it does that
Speaker 2: if you force yourself to see the math, the diagram
Speaker 2: and the code side by side and.
Speaker 1: I guess that's why Widman forces the reader to build
Speaker 1: these networks from scratch in Python, using just like basic arrays.
Speaker 2: In numpis exactly.
Speaker 1: It's not because you're trying to build the fastest AI
Speaker 1: in the world. It's purely an exercise in solidifying your
Speaker 1: understanding of those models.
Speaker 2: Spot on.
Speaker 1: So let's start doing exactly that for the listener. Let's
Speaker 1: abandon the complex terminology. We need to start at the
Speaker 1: absolute foundation of all machine learning, the mathematical function and
Speaker 1: the derivative right.
Speaker 2: And usually when we learn about functions in high school,
Speaker 2: we use the Cartesian plane, you know, Rineiti Card's classic
Speaker 2: X and Y.
Speaker 1: Axes, the good old graph paper.
Speaker 2: Exactly, you plot some points, you draw a curved line
Speaker 2: through them, and that's fine for basic geometry, but it's
Speaker 2: actually a terrible mental model for deep learning.
Speaker 1: Yeah, drawing parabolas isn't going to help us build an AI. Instead,
Speaker 1: Widman tells us to visualize a function as a mini factory,
Speaker 1: just a physical box sitting on a table. Inputs go
Speaker 1: into the box on a conveyor belt. The factory has
Speaker 1: some internal strict rules that it applies to whatever comes in,
Speaker 1: and then a transformed output comes out the other side precisely.
Speaker 2: So let's say the factory is a square function. Okay,
Speaker 2: you send the number two into the factory. The factory's
Speaker 2: internal rule is to multiply the input by itself, so
Speaker 2: outcomes the number four. You send in a three outcomes
Speaker 2: of nine. It's just a simple predictable machine.
Speaker 1: Okay. So if the function is just a factory box,
Speaker 1: what is a derivative? Because just hearing the word derivative
Speaker 1: definitely triggers some traumatic math flashbacks for me.
Speaker 2: Oh for sure. But let's dick with our factory visualization. Yeah,
Speaker 2: imagine there is a physical string connecting the input of
Speaker 2: the factory to the output of the factory, a string.
Speaker 1: Okay.
Speaker 2: The derivative is simply asking a very practical question. If
Speaker 2: you pull on the input string by a very very
Speaker 2: small amount, a tiny nudge like point zero zero zero one,
Speaker 2: by what multiple does the output string move?
Speaker 1: Ah? Okay, So it's kind of like adjusting an analog
Speaker 1: volume knob on an old stereo. If I nudge the
Speaker 1: input dial just a tiny fraction of a millimeter, how
Speaker 1: much louder does the music actually get? Like? Does a
Speaker 1: tiny nudge on the input cause a massive blown speaker
Speaker 1: spike in the output, or does it barely move the
Speaker 1: needle at all?
Speaker 2: Exactly? You're measuring the rate of change.
Speaker 1: But okay, why is this tiny nudge so crucial? Like
Speaker 1: why does an artificial intelligence care so much about this
Speaker 1: little string?
Speaker 2: Because this rate of change? Knowing exactly how the input
Speaker 2: affects the output is the literal engine of machine learning. Yes,
Speaker 2: it is how the model knows how to correct its
Speaker 2: own errors. Think about it. If an AI makes a
Speaker 2: prediction and that prediction is wrong, it needs to know
Speaker 2: how to fix it right.
Speaker 1: It has to adjust.
Speaker 2: And if the AI knows exactly how a tiny nudge
Speaker 2: to its internal settings will affect the final outcome, it
Speaker 2: knows exactly which dials to turn and in which direction
Speaker 2: to get a better result next time. The derivative is
Speaker 2: basically the compass pointing toward the correct answer.
Speaker 1: I see. Okay, so we have a single mini factory.
Speaker 1: You nudge the input, you watch the output change, you
Speaker 1: adjust the dial. That makes sense, but predicting a housing
Speaker 1: price or writing a poem takes way more than one
Speaker 1: mathematical step. Real data doesn't just go through one simple rule.
Speaker 1: So how do these boxes actually talk? To each other
Speaker 1: without losing all the data.
Speaker 2: So this brings us to the concept of nested functions.
Speaker 2: In deep learning. You almost never have just one factory.
Speaker 1: You have a chain of them, an assembly line.
Speaker 2: Exactly an assembly line. The output conveyor belt of factory
Speaker 2: one feeds directly into the input conveyor belt a factory
Speaker 2: two one transforms the raw data, passes it to factory two,
Speaker 2: which transforms it again, and so on.
Speaker 1: Okay, but if I nudge the input at the very
Speaker 1: beginning of the assembly line, that ripple has to travel
Speaker 1: through every single factory to reach the end. How do
Speaker 1: we track that string across ten different boxes.
Speaker 2: We use what might be the single most important mathematical
Speaker 2: rule in all the deep learning, the chain rule from calculus,
Speaker 2: the chain rule. Yes and again, Wideman demystifies this beautifully
Speaker 2: using the factory boxes.
Speaker 1: Okay, let's trace the string. Then, let's say we have
Speaker 1: two boxes. We pull the string on the input to
Speaker 1: box one. We observe that its output changes by a
Speaker 1: factor of three, so a three in a multiplier. Right,
Speaker 1: That output is now the input for box two. And
Speaker 1: we already know that if we tweak the input of
Speaker 1: box two, its output changes by a factor of say
Speaker 1: migus two units.
Speaker 2: Perfect setup. So to find the total change across the
Speaker 2: entire chain from the very first input to the very
Speaker 2: last output, the chain rule says, we simply multiply those
Speaker 2: rates of change together. Just multiply them, just multiply them.
Speaker 2: One changes things by a factor three, box two changes
Speaker 2: things by a factor of niggas two. The total change
Speaker 2: across the whole chain is three multiplied by niggas two,
Speaker 2: which equals negative six.
Speaker 1: Oh wow, so a one unit nudge at the start
Speaker 1: creates a null six unit shift at the very end.
Speaker 2: Exactly.
Speaker 1: But wait, practically speaking, if I'm actually coding this assembly line,
Speaker 1: how does the system know those numbers? Like?
Speaker 2: Do?
Speaker 1: I have to run the data all the way forward
Speaker 1: to get an answer, and then somehow trace my steps
Speaker 1: all the way backward to figure out the chain rule math.
Speaker 2: That is exactly what you have to do. To code
Speaker 2: this from scratch. Your system has to make two distinct passes.
Speaker 2: First is the forward pass. Okay, forward, You feed your
Speaker 2: initial data into the first factory and you let it
Speaker 2: run all the way down the assembly line. But here's
Speaker 2: the catch. As the data moves forward, the system has
Speaker 2: to save all the intermediate quantities at every single step.
Speaker 2: It has to keep a meticulous record of what happened
Speaker 2: inside each box.
Speaker 1: Why doesn't need to save all that? If it reaches
Speaker 1: the end and gets an answer, hasn't it done its job?
Speaker 2: Because of the second step, the backward pass. Once the
Speaker 2: data reaches the end and then network spits out a prediction,
Speaker 2: you compare that prediction to the correct answer to see
Speaker 2: how wrong you were. Then you run backward down the
Speaker 2: assembly line. You use all those intermediate records you save
Speaker 2: during the forward pass to calculate the derivatives the strings.
Speaker 2: Going backward, you calculate box two string, then multiply it
Speaker 2: by box one string using the chain rule, all the
Speaker 2: way back to the start.
Speaker 1: I'm not going to lie. That sounds incredibly tedious to
Speaker 1: code by hand, keeping track of every single variable, saving
Speaker 1: it all in memory, running backward, multiplying the strings. It
Speaker 1: sounds like an absolute nightmare of bookkeeping.
Speaker 2: It is. It's a massive bookkeeping operation. Yeah, and this
Speaker 2: is exactly why modern deep learning libraries like PyTorch are
Speaker 2: so popular.
Speaker 1: Today because they do it for you exactly.
Speaker 2: They use something called automatic differentiation. They handle all that
Speaker 2: tedious forward and backward bookkeeping completely invisibly. You just define
Speaker 2: the factories and the library does the calculus for you.
Speaker 1: But Widman forces you to it by hand anyway, right,
Speaker 1: he does, because if you just rely on PyTorch, you're
Speaker 1: back to being a blind man touching the elephant. You
Speaker 1: don't see the whole process exactly.
Speaker 2: By coding the forward and backward passes from scratch in Python,
Speaker 2: you actually see the mechanics. You realize that learning isn't consciousness.
Speaker 2: It's literally just a series of multipliers passed backward down
Speaker 2: an assembly line.
Speaker 1: Okay, I'm with you on the strings in the assembly line,
Speaker 1: But single numbers are great for theory. Reality is messy.
Speaker 2: Very messy.
Speaker 1: If I want an AI to predict a housing price,
Speaker 1: I'm not just feeding it a single number. A house
Speaker 1: has dozens of features, square footage, number of bedrooms, age
Speaker 1: of the roof, proximity to a highway. So how do
Speaker 1: we pull a string on a massive spreadsheet of information?
Speaker 2: This is where we scale up to matrices and supervised
Speaker 2: learning sew of us. Learning is just finding relationships between
Speaker 2: characteristics that have already been measured, Okay, and to process
Speaker 2: all those characteristics, we can't use single numbers. We have
Speaker 2: to stack the data into grids, which in numb pie
Speaker 2: are called end arrays or n dimensional arrays.
Speaker 1: Right. So, if you visualize a spreadsheet, the columns are
Speaker 1: the features like bedrooms, square footage, and every specific house
Speaker 1: you are evaluating becomes a row.
Speaker 2: Yep.
Speaker 1: So a two x two grid might be two houses
Speaker 1: each with two features exactly.
Speaker 2: Now, when this grid of data enters the first factory,
Speaker 2: the model needs a way to evaluate it. It performs
Speaker 2: what's called a weighted sum.
Speaker 1: A weighted sum.
Speaker 2: Right. It looks at the features and decides how important
Speaker 2: each one is. Does the square footage matter more than
Speaker 2: the age of the roof? It assigns a mathematical weight
Speaker 2: to each feature.
Speaker 1: Okay, let me guess how this works mathematically. If I
Speaker 1: have a column for bedrooms in a way that says
Speaker 1: bedrooms are very important, is the factory just doing a
Speaker 1: dot product like matching them up?
Speaker 2: Yes. Think of a dot product as a matching game.
Speaker 2: The factory lines up the house's features in one hand
Speaker 2: and its internal priority weights in the other.
Speaker 1: Okay.
Speaker 2: It matches the bedrooms to the bedroom weight, multiplies them together,
Speaker 2: matches the square footage to the square footage weight, multiplies them.
Speaker 2: Then it throws all those paired results into one single
Speaker 2: bucket and adds them up.
Speaker 1: That's the sum, right, But if you keep multiplying features
Speaker 1: by weights, that bucket is going to overflow real fast.
Speaker 1: I mean, a three thousand square foot house multiplied by
Speaker 1: a heavyweight becomes a massive number. Do we just let
Speaker 1: the numbers get infinitely large?
Speaker 2: We can't, which is why we usually feed that bucket
Speaker 2: into another factory right afterward, typically something called a sigmoid function.
Speaker 1: A sigmoid function, we haven't covered that one. What's that?
Speaker 2: A sigmoid function is basically a squishing factory.
Speaker 1: A squishing factory.
Speaker 2: Yeah, it takes whatever wild massive number comes out of
Speaker 2: the weighted sum, and it brutally compresses it into a
Speaker 2: manageable decimal between zero and one.
Speaker 1: Oh.
Speaker 2: I see this is incredibly useful if you just want
Speaker 2: the network to give you a probability, like a point
Speaker 2: eight chance that the house is a goodbye, rather than
Speaker 2: outputting a raw score of four million.
Speaker 1: Okay, so our assembly line is now take the matrix
Speaker 1: of houses, match them with weights, sum them up, and
Speaker 1: then squish them through a sigmoid factory to get a probability.
Speaker 2: You got it.
Speaker 1: I get the forward pass. But here's where my brain
Speaker 1: completely breaks. To do the backward pass, we have to
Speaker 1: pull the string to correct the errors. How on earth
Speaker 1: do you track the derivative of a giant grid of
Speaker 1: interacting numbers. Every row and column is interacting with every weight.
Speaker 1: The calculus must just explode into absolute chaos, you would.
Speaker 2: Think so tracking every single string individually across a massive
Speaker 2: matrix would be impossible, right, But the math looks incredibly
Speaker 2: messy on a whiteboard, while the resulting code is brilliantly,
Speaker 2: shockingly clean. It's a magical property of linear algebra.
Speaker 1: WHOA, I would stop right there, time out. You literally
Speaker 1: cannot start this deep dive by promising no magic and
Speaker 1: then tell me the math relies on a magical property
Speaker 1: that is totally cheating. Explain it. Why does the matrix
Speaker 1: math clean up so nicely?
Speaker 2: Fair? Catch? Okay, you're right, no magic. It comes down
Speaker 2: to something called matrix transposition.
Speaker 1: Matrix transposition.
Speaker 2: Yes, when you need to compute the backward pass the
Speaker 2: gradient for a giant grid of weights, the chain rule
Speaker 2: dictates that you don't actually have to calculate a million
Speaker 2: individual strings. Instead, you take the input matrix, and you
Speaker 2: simply transcose it. You flip it on its side.
Speaker 1: Meaning the rows become columns and the columns become rows.
Speaker 2: Exactly, And why does this work mechanically? Think of the
Speaker 2: forward pass like a river flowing downstream, splitting into hundreds
Speaker 2: of tiny branches. Those are your data points interacting with weights.
Speaker 1: Okay, I picture it.
Speaker 2: If you want to send an error signal back up
Speaker 2: the river to the exact source that caused it, you
Speaker 2: just referse the map. Flipping the matrix on its side
Speaker 2: perfectly re routes the air signals backward along the exact
Speaker 2: same mathematical paths the data used to travel forward.
Speaker 1: Oh wow, So you aren't doing entirely new chaotic math
Speaker 1: to go backward, not at all. You're just taking the
Speaker 1: infrastructure you build going forward, turning it sideways, and letting
Speaker 1: the error flow back to the correct weight.
Speaker 2: Precisely because of how matrix transposes work out mathematically, this
Speaker 2: incredibly common plex web of interacting data collapses into a
Speaker 2: few incredibly simple lines of Python code. During the backward pass.
Speaker 2: It scales perfectly.
Speaker 1: That is wild. So it doesn't matter if I'm feeding
Speaker 1: the factory a single two x two grid or a
Speaker 1: massive matrix with a million rows representing every house in
Speaker 1: the country. The logic of the assembly line stays exactly
Speaker 1: the same.
Speaker 2: Exactly the same.
Speaker 1: The forward pass runs the matching game and squishes the numbers.
Speaker 1: The backward pass flips the map on its side to
Speaker 1: route the blame, runs the chain roll and updates the weights.
Speaker 2: And that is why we can have AI models today
Speaker 2: with billions or even trillions of parameters. The fundamental architecture,
Speaker 2: the mini factory, the chain roll, the matrix transposes. It's
Speaker 2: infinitely scalable. You just need more powerful computers to run
Speaker 2: the assembly line faster.
Speaker 1: Okay, let's bring this all together for you, the listener.
Speaker 1: We started this deep dive staring at a hidden circuitry
Speaker 1: that everyone just assumes is impenetrable, but by looking through
Speaker 1: the lens of Seth Weidman's work, we've stripped it down.
Speaker 1: We have deep learning is an assembly line of mini factories.
Speaker 1: We have inputs that flow forward through nested functions, matching
Speaker 1: features to weights. We save our math as we go.
Speaker 1: Then we compare our final answer, and we pull the
Speaker 1: strings backward, flipping the matrices to calculate exactly how to
Speaker 1: adjust our internal dials. It's not a brain, it's just
Speaker 1: very fast, very elegant bookkeeping. You now have the foundational
Speaker 1: mental model for how machines actually learn.
Speaker 2: It's a very empowering realization, honestly, to finally see the
Speaker 2: gears turning. But this actually raises a really important question,
Speaker 2: and it's the thought I want to leave you with today.
Speaker 2: It goes all the way back to the very first
Speaker 2: step of this entire process supervised learning.
Speaker 1: You mean, like setting up the grid of numbers in
Speaker 1: the first place.
Speaker 2: Right. Widman points out that to use these beautiful mathematical
Speaker 2: assembly lines, we have to translate the messy, ambiguous real
Speaker 2: world into precise numbers.
Speaker 1: Yeah.
Speaker 2: In our example, we chose price to perfectly represent a
Speaker 2: house's value. The market decides the price, so mathematically that works.
Speaker 2: But what happens when we try to force incredibly complex
Speaker 2: human concepts into a single numeric matrix just to make
Speaker 2: the math run.
Speaker 1: So wait, if we're building a hiring algorithm, we have
Speaker 1: to somehow turn a concept like desirability or work ethic
Speaker 1: into a column on a spreadsheet exactly.
Speaker 2: Or if we're building a loan approval model, we have
Speaker 2: to quantify something like reliability. We have to convert nuanced,
Speaker 2: deeply human ideas into cold, hard numbers so our minifactories
Speaker 2: can actually process them.
Speaker 1: So if we use something like say, zip codes to
Speaker 1: help predict loan defaults, the math might work perfectly on
Speaker 1: the assembly line, but we've accidentally built a machine that
Speaker 1: mathematically justifies redlining es. The bias isn't in the AI's brain.
Speaker 1: The bias is baked into the columns of the spreadsheet
Speaker 1: before the forward pass even starts.
Speaker 2: Precisely, if the factory only knows what we put on
Speaker 2: the conveyor belt, then the very first step of deep learning,
Speaker 2: choosing which numbers represent reality, might actually be its biggest vulnerability.
Speaker 1: That's terrible wifying.
Speaker 2: Actually, it's a huge issue. Are our models objectively learning
Speaker 2: the truth about the world or are they just efficiently
Speaker 2: mathematically learning the biases we hardcoded into the system the
Speaker 2: moment we decided what to measure.
Speaker 1: That completely flips the script. We spent this whole time
Speaker 1: demystifying the machinery inside the factory. We learned how the
Speaker 1: boxes work, how the chain rule connects them. But maybe
Speaker 1: the real question we should be asking isn't how the
Speaker 1: factory processes the materials.
Speaker 2: Who is deciding what materials are allowed on the conveyor
Speaker 2: belt in the first place.
Speaker 1: Exactly something for you to ponder until next time.