Speaker 1: Welcome to the Sentient Code, where intelligence is engineered, autonomy
Speaker 1: is emerging, and a line between human and machine grows thinner.
Speaker 1: Each episode, we decode the algorithms, explore the robotics, and
Speaker 1: examine the ideas shaping the future of artificial minds.
Speaker 2: So I want you to just take a second and
Speaker 2: look around the room you're in right now, or you
Speaker 2: know wherever you happen to be listening to this today. Yeah,
Speaker 2: just do a quick scan of your environment, right, and
Speaker 2: I want you to try and find a standard, maybe
Speaker 2: somewhat dim, incandescent.
Speaker 3: Light bulb, like the kind in an old desk clamp maybe.
Speaker 2: Exactly, or even just the little bulb inside your refrigerator.
Speaker 2: That specific bulb, it takes about twenty watts of power
Speaker 2: just to maintain its glow.
Speaker 3: Which is I mean, it's barely enough light to read.
Speaker 2: A book by, right, it's practically nothing. So I want
Speaker 2: you to hold on to that image, that tiny twenty
Speaker 2: watt energy budget.
Speaker 3: Because we're going to use that as the bits line
Speaker 3: for something pretty.
Speaker 2: Wild, completely mind bending. Actually yeah, because that exact same
Speaker 2: amount of energy, those twenty watts, is what your biological
Speaker 2: brain is using right this very second. To do well
Speaker 2: literally everything.
Speaker 3: It really forces a complete recalibration of how we view
Speaker 3: our own biology.
Speaker 2: Doesn't it. It totally does we just walk.
Speaker 3: Around completely oblivious to the sheer, I mean, the thermodynamic
Speaker 3: miracle happening inside our skulls every day.
Speaker 2: Yeah, think about what you, as the listener, are doing
Speaker 2: on just twenty wants Right now, you are processing the
Speaker 2: auditory signals of my voice.
Speaker 3: You're converting acoustic waves into electrical impulses.
Speaker 2: Exactly, and then you're parsing individual words out of that
Speaker 2: continuous stream of sound.
Speaker 3: And if you happen to be, say, walking down the
Speaker 3: street while listening, it's even crazier.
Speaker 2: Oh yeah, you're instantly recognizing familiar faces. You're navigating this
Speaker 2: complex three D environment, keeping your balance against.
Speaker 3: Gravity, regulating your core body temperature.
Speaker 2: Right, and you can learn a complete, letely new abstract
Speaker 2: concept from a single example and instantly integrate it with
Speaker 2: decades of stored memories.
Speaker 3: All of that high level computational work is happening simultaneously, And.
Speaker 2: While you're doing all of that, that identical twenty watt
Speaker 2: budget is sustaining the profound, completely unresolved mystery of human consciousness. Itself.
Speaker 2: It's just staggering all for the energy it takes to
Speaker 2: dimly light a closet.
Speaker 3: And you know the contrast becomes almost absurd when we
Speaker 3: look at our current technological landscape.
Speaker 2: Oh absolutely, let's talk about that.
Speaker 3: Like, consider the state of the art artificial intelligence models today.
Speaker 3: The systems that are performing complex pattern recognition or generating
Speaker 3: photorealistic images.
Speaker 2: Or simulating human likesuage right.
Speaker 3: To achieve even a tiny fraction of human cognitive capabilities,
Speaker 3: and usually only within incredibly narrow domains. Current AI requires
Speaker 3: physical infrastructure on a planetary scale.
Speaker 2: We are definitely not talking about twenty watts anymore.
Speaker 3: No, not at all. We're talking about sprawling, warehouse sized
Speaker 3: data centers, just packed floor to ceiling with specialized hardware.
Speaker 2: I mean, these facilities need dedicated industrial cooling towers just
Speaker 2: to prevent the silicon from literally melting.
Speaker 3: Exactly. They consume millions of watts.
Speaker 2: Millions of wants for a single model. It's basically the
Speaker 2: energy equivalent of a small dedicated power plan.
Speaker 3: Just to run a specialized software program that can write
Speaker 3: a passable email or identify a stop sign and a photograph.
Speaker 2: So the ratio of computational capability to energy consumption in
Speaker 2: the human brain isn't just like a little bit better
Speaker 2: than our.
Speaker 3: Ai, No, it is superior by multiple orders of magnitude.
Speaker 2: And the most critical realization here is that this isn't
Speaker 2: some temporary engineering hurdle, is it not?
Speaker 3: At all? This isn't a problem where we can simply
Speaker 3: wait for the next generation of silicon microchips and just
Speaker 3: assume the efficiency gap will organically close.
Speaker 2: Because the foundational architecture of how we build computers is
Speaker 2: fundamentally at odds with how biological brains operate physically at odds. Yeah,
Speaker 2: so we have this massive multi order of magnitude disparity
Speaker 2: and efficiency, and the traditional roadmap of computer science is
Speaker 2: essentially a dead end for solving it.
Speaker 3: Which brings us to the terrain we're exploring today.
Speaker 2: Yes, we are looking at a radical, incredibly ambitious field
Speaker 2: of engineering called neuromorphic computing.
Speaker 3: The name itself derives from the Greek roots for nerve
Speaker 3: and form.
Speaker 2: I love that. And just to be clear, this is
Speaker 2: not about writing clever software code to run on the
Speaker 2: exact same hardware we've been using since the nineteen eighties.
Speaker 3: No, this is the engineering quest to build actual physical hardware,
Speaker 3: new types of silicon chips that fundamentally think, physically operate,
Speaker 3: and are structurally organized exactly like a biological brain.
Speaker 2: It requires a complete paradigm shift, It really does, because
Speaker 2: the vast majority of artificial intelligence today operates by simulating
Speaker 2: a neural.
Speaker 3: Network in software, but then executing that simulation on conventional,
Speaker 3: non neural hardware.
Speaker 2: Right, It's just a similar running on a normal.
Speaker 3: Computer exactly, but neuromorphic engineering abandons the simulation. The goal
Speaker 3: is to instantiate the physical operating principles of biological neural
Speaker 3: computation directly into the atomic structure of the machine itself.
Speaker 2: Okay, So to understand why we need to completely reinvent
Speaker 2: the physical computer from the ground up, we first need
Speaker 2: to talk about the fatal flaw baked into the computers you.
Speaker 3: And I are already using, right, the von Neuman architecture.
Speaker 2: Yeah, if you're listening to this on the smartphone, that
Speaker 2: incredibly advanced device is constrained by a design choice made
Speaker 2: back in the nineteen forties.
Speaker 3: We have to go back to the mathematician John von
Speaker 3: Neuman set.
Speaker 2: The stage for us. What was happening in the forties.
Speaker 3: Well, during the era of those massive room sized vacuum
Speaker 3: tube computers like the Nis, von Neumann formalized a blueprint
Speaker 3: for computer architecture.
Speaker 2: And that blueprint became the standard for virtually every single
Speaker 2: digital device we manufacture today.
Speaker 3: Right exactly, and the defining characteristic of this von Noyman
Speaker 3: architecture is a strict physical set of duties.
Speaker 2: Okay, break that down for me.
Speaker 3: You have the processor which performs the actual mathematical computation
Speaker 3: and logic, and then physically separated from it, you have
Speaker 3: the memory unit which stores the data and the instructions.
Speaker 2: So they basically live in two completely different neighborhoods on the.
Speaker 3: Motherboard they do, and they're connected by a communication pathway
Speaker 3: known as a bus.
Speaker 2: A bus got it.
Speaker 3: Because they're physically separated. Every single time the processor needs
Speaker 3: to execute a task, whether that's adding two numbers or
Speaker 3: changing the color of a pixel on your screen, it
Speaker 3: can't do it alone.
Speaker 2: It has to ask for the data, right.
Speaker 3: It must send an electrical request across the bus to
Speaker 3: the memory. It has to wait for the data to
Speaker 3: be retrieved. The data is pushed back across the bus
Speaker 3: to the processor, the operation is performed, and then the
Speaker 3: result usually has to be sent back across the bus
Speaker 3: to be stored in memory. Again.
Speaker 2: Wow, if you really visualize that, it's just endless shuttling back.
Speaker 3: And forth, constant shuttling.
Speaker 2: Let me train an analogy here. Imagine a chef working
Speaker 2: in a high end restaurant kitchen. But there's a massive
Speaker 2: design flaw. The pantry isn't in the kitchen.
Speaker 3: Okay, where is it.
Speaker 2: The pantry is three blocks down the street.
Speaker 3: Oh, that sounds awful, right.
Speaker 2: So the chef is standing at the stove, water is boiling,
Speaker 2: and they realize they need a pinch of salt, so
Speaker 2: they have to leave the kitchen. They have to stop
Speaker 2: what they're doing, run out the door, sprint three blocks
Speaker 2: down the street to the pantry, grab the pitch of salt,
Speaker 2: run all the way back to the kitchen and sprinkle
Speaker 2: it in the pot.
Speaker 3: And then they probably need something else.
Speaker 2: Exactly Then they realize they need a chopped on you
Speaker 2: Boom out the door again, three blocks down the street,
Speaker 2: grab the onion, sprint back over and over millions of
Speaker 2: times a day, so by the end of the night.
Speaker 2: By the end of the night, the chef isn't tired
Speaker 2: from the actual act of cooking. The chef is completely
Speaker 2: exhausted from commuting.
Speaker 3: That is a perfect way to picture it, and the
Speaker 3: physical consequence of that commute is the crux of the problem.
Speaker 2: The von Neumann bottleneck exactly.
Speaker 3: Historically, in computer science it was viewed as a speed limit.
Speaker 3: The processor is incredibly fast, but it spends most of
Speaker 3: its time sitting idle waiting for data to travel back
Speaker 3: and forth over the bus.
Speaker 2: But nowadays it's not just about speed, is it No.
Speaker 3: In the context of modern artificial intelligence, the bottleneck is
Speaker 3: a massive energy crisis.
Speaker 2: Because pushing electrical signals back and forth across a physical
Speaker 2: copper wire billions of times a second that generates heat.
Speaker 3: Massive amounts of heat. Due to the physical capacitance of the.
Speaker 2: Wire, it actually takes physical energy to push electrons down
Speaker 2: a wire.
Speaker 3: It takes enormous energy. When we run complex AI neural
Speaker 3: networks on conventional hardware, we're shuffling massive matrices of data
Speaker 3: millions of parameters back and forth continuously.
Speaker 2: So the commute is basically bankrupting our energy budget.
Speaker 3: Precisely, the sheer thermodynamic cost of moving the data is
Speaker 3: astronomically higher than the energy cost of the actual mathematical
Speaker 3: operations being performed.
Speaker 2: We are burning our energy budget on the commute, not
Speaker 2: the computation.
Speaker 3: Which naturally leads us to ask a pretty important question.
Speaker 2: Right, If that's how our current devices arrange their kitchen,
Speaker 2: and it's clearly a disaster for energy efficiency, how does
Speaker 2: the human brain around its kitchen?
Speaker 3: How is your brain managing to do all of this
Speaker 3: high level processing on just twenty watts without completely melting down?
Speaker 2: Yeah? What's the biological solution?
Speaker 3: The biological solution entirely eliminates the commute?
Speaker 2: Wait, entirely.
Speaker 3: Yes. In the human brain, computation and memory are colocated.
Speaker 3: They occupy the exact same physical space.
Speaker 2: Okay, I need to understand how that actually works physically.
Speaker 3: Well, the brain contains roughly eighty six billion neurons, and
Speaker 3: they communicate with each other through microscopic junctions called synapses.
Speaker 2: And we have what trillions of those.
Speaker 3: Trillions of synactic connections. Yeah. In the biological paradigm, the
Speaker 3: synapse operate simultaneously as the processor and the hard drive.
Speaker 2: So the wiring itself is the memory.
Speaker 3: The connection itself is the physical medium through which the
Speaker 3: computation occurs, and it is also the physical substrate where
Speaker 3: learned information is stored. That is wild when you learn
Speaker 3: something new, say you're practicing a new language or memorizing
Speaker 3: the layout of a new neighborhood. The physical structure of
Speaker 3: your brain actually changes like.
Speaker 2: It physically alters. Its shape.
Speaker 3: The synaptic weight, which determines the strength of the electrical
Speaker 3: connection between specific neurons, physically alters.
Speaker 2: See if you open up a traditional computer, the memory
Speaker 2: is just a file saved in a discrete sector of
Speaker 2: a silicon ship. But in the brain, the memory is
Speaker 2: the physical shape, density, and chemical strength of the literal wiring.
Speaker 3: Computation and memory changed together. They are intrinsically linked in
Speaker 3: the same physical space.
Speaker 2: So when I see something what happens.
Speaker 3: When an electrical signal passes through a network of neurons
Speaker 3: in your visual cortex, It doesn't have to pause and
Speaker 3: fetch the memory of how to process a straight line
Speaker 3: from a different lobe of the brain.
Speaker 2: The instructions are just there.
Speaker 3: The processing instructions are built directly into the physical pathway
Speaker 3: the signal is currently traveling through. There is no shuttling
Speaker 3: of data.
Speaker 2: There's no bus, no bus at all.
Speaker 3: To use your analogy, the kitchen and the pantry are
Speaker 3: perfectly integrated at a microscopic level.
Speaker 2: And replicating this exact physical colocation in silicon hardware is
Speaker 2: the foundational premise of neuromorphic engineering exactly.
Speaker 3: That's the core of it.
Speaker 2: But you know, the physical layout of the von Normann
Speaker 2: bottleneck is really only half the problem, isn't it.
Speaker 3: That's true.
Speaker 2: It's not just that traditional computers separate the kitchen in
Speaker 2: the pantry, it's the very language they use to communicate internally,
Speaker 2: the fundamental way we treat the materials we build computers
Speaker 2: out of.
Speaker 3: Right, we need to look back at the origins of
Speaker 3: neuromorphic hardware to really understand this, which.
Speaker 2: Shakes us to Carver Mead right at Caltech.
Speaker 3: Yes, Carver Mead at the California Institute of Technology, in
Speaker 3: a landmark nineteen ninety paper, Mead, who was already a
Speaker 3: highly respected pioneer in integrated circuit design, actually coined the
Speaker 3: term neural morph.
Speaker 2: Okay, what was his big breakthrough?
Speaker 3: His most profound realization wasn't just about moving memory closer
Speaker 3: to the processor. It was about the fundamental physics of
Speaker 3: how we utilize silicon.
Speaker 2: Because right now the global tech industry uses silicon to
Speaker 2: mass produce transistors, and we treat those transistors in a
Speaker 2: highly rigid binary way.
Speaker 3: Yes, we treat them like tiny light switches.
Speaker 2: They're forced to be either entirely on, representing a one,
Speaker 2: or entirely off, representing a zero.
Speaker 3: Conventional computing is entirely digital and binary. It operates using discrete,
Speaker 3: strictly controlled, clocked operations.
Speaker 2: Everything is forced into that binary state of one or zero.
Speaker 3: Furthermore, millions or billions of times a second. A rigid
Speaker 3: internal clock forces the entire system to step forward synchronously.
Speaker 2: Just marching to a beat exactly.
Speaker 3: But med looking closely at biology, recognized that biological brains
Speaker 3: do not operate like binary calculators.
Speaker 2: They don't use ones and zeros.
Speaker 3: No, the brain operates in a fundament only analog, continuous
Speaker 3: time mode.
Speaker 2: So carver Meede looked at these silicon transistors which we're
Speaker 2: using to build these rigid digital calculators, and said, what
Speaker 2: if we stop forcing them to be strict on and
Speaker 2: off switches.
Speaker 3: He proposed operating the transistors in what physicists call the
Speaker 3: sub threshold regime.
Speaker 2: The sub threshold regime, what does that mean? In plain English?
Speaker 3: In standard digital logic, a transistor is hit with a
Speaker 3: relatively high voltage to snap it cleanly on, allowing current
Speaker 3: to flow freely, or the voltage is dropped to snap
Speaker 3: it cleanly off.
Speaker 2: It's a brute force approach.
Speaker 3: It is. But if you apply a very low voltage
Speaker 3: below that clean switching threshold, the transistor doesn't just go dead.
Speaker 3: Its behavior changes entirely.
Speaker 2: It does what happens.
Speaker 3: It responds in a graded, continuous analog way. The tiny
Speaker 3: trickle of leakage current that digital engineers usually try to
Speaker 3: eliminate the stuff they see as a bug, exactly the
Speaker 3: bug becomes the core feature. Mead realized that the analog
Speaker 3: physics of this sub threshold silicon, how electrical charge slowly accumulates,
Speaker 3: how capacitance builds, how ions.
Speaker 2: Diffuse, It acts like biology.
Speaker 3: It could naturally mimic the continuous physical dynamics of biological
Speaker 3: cell membranes and ion channels.
Speaker 2: So he wanted to use the raw physics of the
Speaker 2: material to just do the math organically, rather than forcing
Speaker 2: the material to act like a rigid mathematical abocus.
Speaker 3: That's a great way to put it. By utilizing the
Speaker 3: natural analog physics of the substrate, you bypass the massive
Speaker 3: thermo dynamic overhead of encoding every single piece of sensory
Speaker 3: information into long, complex strings of binary digits.
Speaker 2: And shoving them through that serial bottleneck we.
Speaker 3: Talked about, right, But this analog shift fundamentally changes how
Speaker 3: information is transmitted across the network.
Speaker 2: How so well?
Speaker 3: Modern artificial neural networks in software exchange precise, continuous numerical
Speaker 3: values floating point numbers like point seven four to three
Speaker 3: fro sheats of numbers basically yeah, But biological neurons do
Speaker 3: not exchange spreadsheets of numbers.
Speaker 2: They speak in spikes.
Speaker 3: They communicate through action potentials, commonly referred to as spikes.
Speaker 2: What exactly is a spike? Physically?
Speaker 3: These are brief, discrete, essentially identical electrical pulses that travel
Speaker 3: down the axon of a neuron and trigger a chemical
Speaker 3: or electrical response at the synaps.
Speaker 2: Okay, I want to make sure we truly grasp this
Speaker 2: because it's a massive departure from how we normally think
Speaker 2: about data.
Speaker 3: It is a huge mental leap.
Speaker 2: If the biological brain isn't sending specific numerical values to
Speaker 2: convey information, how does a literal zapp of electricity actually
Speaker 2: carry any complex meaning? I mean, a spike is just
Speaker 2: a spike, it's a great question.
Speaker 3: In biological neural systems, the information is not encoded in
Speaker 3: the size, the amplitude, or the shape of the signal.
Speaker 2: Because every spike is basically identical, right.
Speaker 3: Instead, the information is entirely encoded in the timing and
Speaker 3: the rate of those spikes.
Speaker 2: The timing and the rate.
Speaker 3: Okay, if a sensory neuron in your eye fires rapidly,
Speaker 3: producing a dense, high frequency cluster of spikes, it might
Speaker 3: be conveying a stronger stimulu, say a very bright light,
Speaker 3: compared to a neuron that fires rarely.
Speaker 2: That makes sense, more spikes brighter light.
Speaker 3: That is called rate coding. But more importantly, there is
Speaker 3: temporal code.
Speaker 2: Temporal coding like the exact microsecond.
Speaker 3: Yes, the precise microsecond timing of a single spike relative
Speaker 3: to the spikes of other surrounding neurons, and the network
Speaker 3: can carry incredibly complex, high dimensional information.
Speaker 2: Let's use an analogy to really visualize why this timing
Speaker 2: based spiking communication is such a monumental advantage for saving power.
Speaker 2: I'm all ears think about a traditional computer processor like
Speaker 2: a massive symphony orchestra, but the conductor is an absolute
Speaker 2: tyrant holding a metronome.
Speaker 3: A very rigid conductor, very rigid.
Speaker 2: The metronome is the computer's internal clock, ticking away billions
Speaker 2: of times a second. Every single musician in the orchestra
Speaker 2: is forced to play strictly to that beat. Right even
Speaker 2: if a musician, let's say the triangle player in the back,
Speaker 2: has absolutely no notes to play for a full twenty
Speaker 2: minutes of this infanty, they cannot relax.
Speaker 3: They have to stay engaged.
Speaker 2: The conductor forces them to stand at attention, physically tapping
Speaker 2: their foot rigidly keeping time on every single beat, exhausting
Speaker 2: themselves just to remain perfectly synchronized with the global metronome.
Speaker 3: They are burning massive amounts of energy just to actively
Speaker 3: do nothing.
Speaker 2: Exactly, and the physical reality of conventional digital chips mirrors
Speaker 2: that orchestra exactly doesn't it.
Speaker 3: It absolutely does. The global clock cycle forces electrical activity
Speaker 3: and power consumption across the entire chip, even.
Speaker 2: In sectors of the processor that are momentarily idle, even.
Speaker 3: When they have no new data to process. The wires
Speaker 3: are constantly charging and discharging just to maintain the rhythm.
Speaker 2: But a neuromorphic system, a spiking neural network, is fundamentally different.
Speaker 3: How would you describe it.
Speaker 2: It's like a cool late night jazz ensemble playing completely
Speaker 2: without a set tempo. There is no metronome.
Speaker 3: I like that.
Speaker 2: A musician in this ensemble will sit in total relaxed silence.
Speaker 2: They expend absolutely zero energy. They aren't tapping their foot,
Speaker 2: they are actively keeping time. They just wait. They just
Speaker 2: wait at rest until exactly the moment they were organically
Speaker 2: inspired to play a single note, a spike. They play
Speaker 2: their note, and then they immediately return to resting in
Speaker 2: total silence.
Speaker 3: And the energy savings of that jazz ensemble approach are
Speaker 3: staggering a bit, because when those artificial neurons are silent,
Speaker 3: the power draw of the chick drops precipitously. This is
Speaker 3: the essence of an event driven architecture.
Speaker 2: Event driven, so it only reacts when something happens.
Speaker 3: Spiking neurons are purely event driven. A neuromorphic chip consumes
Speaker 3: practically no power when there is no new data changing
Speaker 3: in its environment.
Speaker 2: It assists there.
Speaker 3: It sits in a quiescent state. Energy is expended only
Speaker 3: when and where information is actually flowing through the network.
Speaker 2: This means the overall energy cost to the system scales
Speaker 2: entirely with the activity of the network, right, not the physical.
Speaker 3: Size exactly, not the sheer physical size of the network.
Speaker 3: You could build a massive chip containing millions of artificial neurons,
Speaker 3: but if only one percent of them are actively spiking
Speaker 3: at any given microsecond, your power draw remains astonishingly low.
Speaker 2: Which perfectly explains how you get a human brain running
Speaker 2: on twenty wants.
Speaker 3: It makes perfect sense, now, doesn't it.
Speaker 2: It's a massive, dense network of eighty six billion neurons,
Speaker 2: but at any given moment, the vast majority of it
Speaker 2: is resting.
Speaker 3: Just whispering to itself in perfectly timed, sparse spikes.
Speaker 2: So the theory is beautiful and biology proves it works.
Speaker 2: But I want to shift gears from theory to cold
Speaker 2: hard silicon. Are we actually forging these jazz playing analog
Speaker 2: brains in the real world?
Speaker 3: We are. The engineering community has taken this challenge head on,
Speaker 3: and there are several major projects taking distinctly different architectural
Speaker 3: approaches to forging spiking neural networks in hardware.
Speaker 2: Okay, let's hear about them. Who's leading the charge.
Speaker 3: Let's start with one of the most prominent research platforms
Speaker 3: developed by one of the largest chip manuf facturers on
Speaker 3: the planet, Intel's lowy heat architecture.
Speaker 2: Intel. Really they are the undisputed giant of traditional von
Speaker 2: Neumann computing. How did they approach building a brain?
Speaker 3: Well? Intel released the first LOWI heat chip back in
Speaker 3: twenty seventeen and followed it up with a more advanced
Speaker 3: architecture LOWI Heat two and twenty twenty one.
Speaker 2: Okay, and the sheer.
Speaker 3: Scale of what they engineered is breathtaking. A single LOWI
Speaker 3: heat two chip contains over a million programmable artificial spiking
Speaker 3: neurons and over one hundred million synapses.
Speaker 2: A million neurons on one chip.
Speaker 3: But the true innovation is how it is structured. It
Speaker 3: is organized into a dense mesh of neuromorphic cores, and crucially,
Speaker 3: there is no global metronome, no clock, no clock. These
Speaker 3: cores communicate entirely through an asynchronous on chip spike routing network.
Speaker 2: How does that work without a clock?
Speaker 3: When a neuron spikes, it essentially packages that spike like
Speaker 3: a piece of mail and routes it through the mesh
Speaker 3: to its destination, independent of any global clock.
Speaker 2: A million neurons on a single chip, communicating like a
Speaker 2: postal system what does the power consumption actually look like
Speaker 2: when you run it?
Speaker 3: This is where the event driven philosophy really proves its worth.
Speaker 3: When the LOWI heat chip is actively computing processing complex inputs,
Speaker 3: it consumes power in the milliwat.
Speaker 2: Range milliwats and for context, a normal.
Speaker 3: Chip to put that in perspective, a standard GPU might
Speaker 3: consume hundreds of watts. Wow, But it gets better. When
Speaker 3: lowihi is at rest waiting for a sensor input, its
Speaker 3: power draw drops down to the microwat.
Speaker 2: Rank microwats, literally a millionth of a watt. You could
Speaker 2: run that off a watch battery for years.
Speaker 3: It represents a reduction in energy consumption of several orders
Speaker 3: of magnitude compared to running the exact same computational workload
Speaker 3: on a conventional CPU or GPU.
Speaker 2: That's incredible. So what are people actually doing with it?
Speaker 3: Intel provides this platform to a global community of researchers,
Speaker 3: and they're using it to demonstrate incredible edge applications like
Speaker 3: what things like real time gesture recognition, complex robotic control systems,
Speaker 3: and even artificial olfactory sensing, essentially giving machines the ability
Speaker 3: to process.
Speaker 2: Chemical smells the smelling computer.
Speaker 3: Yeah, because the architecture relies on spiking networks, it naturally
Speaker 3: excels at adaptive learning tasks where the system needs to
Speaker 3: learn and adjust to new patterns on the fly. In
Speaker 3: the real world.
Speaker 2: That's Intel's approach, but IBM has been a major player
Speaker 2: in this space as well. Right.
Speaker 3: Yes, IBM introduced their true North chip back in twenty fourteen,
Speaker 3: and it represents a slightly different philosophy within the neuromorphic
Speaker 3: design space. True North also featured a million artificial neurons,
Speaker 3: but paired them with two hundred and fifty six million synapses. However,
Speaker 3: True North was engineered with a highly regular, incredibly rigid,
Speaker 3: but extremely low power architecture.
Speaker 2: Rigid in what way.
Speaker 3: It was specifically optimized for deep spiking neural network inference inference.
Speaker 2: Okay, when you say inferns, you mean it was designed
Speaker 2: to take a neural network that has already in fully
Speaker 2: train somewhere else, loaded onto the chip and just run
Speaker 2: it as efficiently as physically possible.
Speaker 3: Exactly. The focus was on execution rather than on chip learning,
Speaker 3: got it. True North demonstrated that you could perform real
Speaker 3: time complex pattern recognition tasks like classifying multiple moving objects
Speaker 3: in a live video feed or processing spoken audio streams
Speaker 3: at power levels measured in just tens of milliwatts.
Speaker 2: Tens of milliwats for video processing.
Speaker 3: Yes, if you attempted to perform that exact same real time,
Speaker 3: high frame rate video processing on conventional hardware, you would
Speaker 3: easily be burning through hundreds of watts or even kilowatts
Speaker 3: for a dense enough network.
Speaker 2: That's a massive difference.
Speaker 3: True, North proved definitively over a decade ago that neuromorphic
Speaker 3: hardware could handle practically useful, commercial grade tasks at a
Speaker 3: dramatically lower energy cost.
Speaker 2: See I hear that, But I am struggling to see
Speaker 2: how this is financially practical. What do you mean You
Speaker 2: are telling me we need to build entirely new analog
Speaker 2: or highly specialized asyncre chifts from scratch. Yes, but the
Speaker 2: manufacturing pipelines for standard digital silicon transistors are worth trillions
Speaker 2: of dollars globally. Yes, we have perfected the art of
Speaker 2: making digital processors. You can't just throw that entire global
Speaker 2: infrastructure away.
Speaker 3: It's a very valid point.
Speaker 2: There has to be a middle ground, right, Yeah? Is
Speaker 2: there way to use the conventional digital chips we already
Speaker 2: mass produce? But somehow forced them to act like a brain.
Speaker 3: That is the exact pragmatic dilemma that birthed a completely
Speaker 3: different approach to neuromorphic engineering, the Spinnaker project. Spinnaker, Yes,
Speaker 3: it was developed at the University of Manchester, led by
Speaker 3: a researcher named Steve Ferber.
Speaker 2: Steve Ferber, why is that named? Legendary and computer science.
Speaker 3: Ferber is one of the original principal designers of the
Speaker 3: ARM processor architecture.
Speaker 2: The chips in our phones.
Speaker 3: Exactly, if you are listening to this on a mobile phone, tablet,
Speaker 3: or even many modern laptops, you are almost certainly using
Speaker 3: an ARM based processor.
Speaker 2: Wow. Okay, so what did he do?
Speaker 3: Ferber and his team took a fascinating hybrid route. Instead
Speaker 3: of designing highly experimental, custom analog neuron circuits from scratch,
Speaker 3: they took an array of small, standard conventional digital.
Speaker 2: ARM processors just off the shell processors, yes.
Speaker 3: But they fundamentally altered how they interact. They interconnected these
Speaker 3: standard processors using a completely custom, highly specialized asynchronous spike
Speaker 3: routing network.
Speaker 2: So the individual brain cells are just traditional digital processors,
Speaker 2: but the wiring connecting them forces them to speak the
Speaker 2: brain's language. Of spikes.
Speaker 3: You've got it. The individual processors run software models simulating
Speaker 3: the biological behavior of neurons, but the network architecture physically
Speaker 3: forces them to communicate via discrete spikes.
Speaker 2: That's a brilliant compromise, it is.
Speaker 3: This hybrid approach provides immense flexible programmability. Because they are
Speaker 3: standard processors, you can easily rewrite the software models of
Speaker 3: the neurons while still capturing the system level efficiency of
Speaker 3: an event driven spiking network.
Speaker 2: How big did they scale this?
Speaker 3: The scale is astounding. The second generation Spinnaker two, which
Speaker 3: is currently deployed at the Technical University of Dresden, scales
Speaker 3: this concept up to over a million interconnected arm processor cores.
Speaker 2: A supercomputer made of a million cores just to rut spikes.
Speaker 3: A network capable of routing billions of individual spike events
Speaker 3: per second in real time without traffic jams.
Speaker 2: That's incredible.
Speaker 3: Spinnaker serves as a vital bridge between computer engineering and
Speaker 3: computational neuroscience. It possesses the sheer computational power to run
Speaker 3: practical neuromorphic applications, but its primary design goal is to
Speaker 3: model massive biological neural.
Speaker 2: Systems, so scientists use it to study the brain exactly.
Speaker 3: It allows neuroscientists to simulate biologically realistic neural circuits at
Speaker 3: a scale that actually approaches the complexity of small regions
Speaker 3: of an actual mammalian brain.
Speaker 2: That is wild is using millions of digital brains to
Speaker 2: simulate an analog brain by enforcing a spiking language.
Speaker 3: That's one way to do it.
Speaker 2: But what if we want to abandon digital completely. What
Speaker 2: if a research team wants to go full analog physics,
Speaker 2: no digital processors, no software models at all.
Speaker 3: For that pure analog vision, we look to the Massive
Speaker 3: Human Brain project in Europe, specifically a system called brain
Speaker 3: scale S developed at Heidelberg University.
Speaker 2: Brain Scale brainscale S.
Speaker 3: Is a pure analog mixed signal neuromorphic platform. It does
Speaker 3: not use digital code to simulate biological neuron models.
Speaker 2: Go how does it work?
Speaker 3: Instead, it physically implements the differential equations of biological cell
Speaker 3: membranes directly into the electrical physics of its custom silicon circuits.
Speaker 2: So the physical electrical currents flowing through the chip are
Speaker 2: literally acting out the biology. The silicon is physically behaving
Speaker 2: like a cell wall.
Speaker 3: The capacitance and resistance of the physical circuits are tuned
Speaker 3: to perfectly mirror the ion channels of a neuron. And
Speaker 3: because it relies on pure analog physics happening at the
Speaker 3: speed of electronics rather than waiting for software to calculate
Speaker 3: mathematical equations, it operates at an astonishing speed fast. It
Speaker 3: does not run at biological real time. It runs neural
Speaker 3: dynamics set up to ten thousand times the speed of biological.
Speaker 2: Real time, ten thousand times faster than an actual living brain. Yes,
Speaker 2: if you're a listener, just pause and think about the
Speaker 2: implications of that for a second. If you're a scientist
Speaker 2: studying how a specific brain network learns, adapts, and evolves
Speaker 2: over say a full year of biological life, you don't
Speaker 2: have to sit in a lab and wait a year.
Speaker 3: No, you don't.
Speaker 2: You can run that entire year of complex physical brain
Speaker 2: evolution on the Brain scale S platform in about an hour.
Speaker 2: You're essentially putting physical brain dynamics on fast forward.
Speaker 3: The value proposition for basic science is immense.
Speaker 2: It has to be.
Speaker 3: It allows researchers to study the long term evolution of
Speaker 3: neural network behavior, to test complex plasticity rules over massive
Speaker 3: biological time scales, and to observe phenomena that would be
Speaker 3: impossible to track in a living organism, all in mere
Speaker 3: seconds or minutes of real time.
Speaker 2: But you know, all of these chips, whether it's Intel's
Speaker 2: digital mesh or a fervers million arm cores, or Heidelberg's
Speaker 2: accelerated analog physics, they face a massive wall.
Speaker 3: They do.
Speaker 2: You can build a billion artificial neurons, but a brain
Speaker 2: isn't a brain unless it can learn. It has to
Speaker 2: adapt to new information. How do you actually program or
Speaker 2: teach a billion silent spiking nodes when you don't have
Speaker 2: standard code to write.
Speaker 3: To fully grasp how a neuromorphic chip actually learns, we
Speaker 3: first have to understand why the dominant learning mechanism used
Speaker 3: by almost all conventional artificial intelligence today is completely fundamentally
Speaker 3: incompatible with biological brains.
Speaker 2: Okay, let's unpack that.
Speaker 3: Conventional deep learning, the technology behind large language models and
Speaker 3: image generators, trains its neural networks using a mathematical algorithm
Speaker 3: called backpropagation.
Speaker 2: Backpropagation. It is the undisputed engine of the modern AI boom.
Speaker 2: It is, how does it actually work? And why couldn't
Speaker 2: a biological brain just use it?
Speaker 3: Backpropagation is mathematic elegant, but biologically impossible. Let's walk through it.
Speaker 3: In a conventional AI network, data is fed forward through
Speaker 3: multiple layers of artificial neurons to produce an output. Let's
Speaker 3: say it's looking at an image and trying to guess
Speaker 3: if it's a handwritten number seven. It makes its guess.
Speaker 3: If the guess is wrong, the system calculates the exact
Speaker 3: mathematical error of that guess right backpropagation. Then takes that
Speaker 3: error value and mathematically propagates it backwards through every single
Speaker 3: layer of the entire network. It uses calculus to calculate
Speaker 3: an exact gradient, a specific adjustment for every single weight
Speaker 3: in the network, so that the network will be slightly
Speaker 3: more accurate the next time it sees that image.
Speaker 2: I want to visualize this. It sounds like an incredibly
Speaker 2: overbearing micromanager and a massive corporation.
Speaker 3: That's a very apt comparison.
Speaker 2: Like the company makes a mistake on a product, the
Speaker 2: micromanager immediately hits the pause button on the entire factory.
Speaker 2: They look at every single one of the million employees.
Speaker 3: Simultaneously, and they calculate exactly how much of the blame
Speaker 3: below belongs to each individual person.
Speaker 2: Right, they synchronously update every employee's instructions, and only then
Speaker 2: do they unpause the factory to try again.
Speaker 3: And the biological brain simply cannot function that way. The
Speaker 3: physical requirements of backpropagation are immense.
Speaker 2: Why can't the brain do it?
Speaker 3: First, it requires a global error signal, your micromanager that
Speaker 3: has an omniscient view of the entire network at once,
Speaker 3: which we don't have right. Second, it requires the system
Speaker 3: to temporarily store all the electrical states from the forward
Speaker 3: pass in a massive memory bank, so the backward pass
Speaker 3: can use them to calculate the.
Speaker 2: Blame, and we don't have that separated memory bank exactly.
Speaker 3: Third, it requires perfectly synchronous updates across the entire system.
Speaker 3: A biological brain simply does not possess the anatomical hardware
Speaker 3: for this.
Speaker 2: There's no biological manager neuron that can pause your brain,
Speaker 2: calculate a global calculus error, and physically adjust eighty six
Speaker 2: billion synapses simultaneously.
Speaker 3: No, that would be impossible.
Speaker 2: So if the brain can't do global backpropagation, how do
Speaker 2: we get smarter? How do we wire new memories?
Speaker 3: The brain relies entirely on local learning rules. It learns
Speaker 3: from the ground up, not from the top down local
Speaker 3: learning rules and the most heavily studied biological mechanism, which
Speaker 3: is now being directly implemented in neuromorphic hardware, is called
Speaker 3: spike timing dependent plasticity or STDP.
Speaker 2: Spike timing dependent plasticity. That's a mouthful. Let's break down
Speaker 2: the mechanics of it.
Speaker 3: It is entirely dependent on highly localized physical information, specifically
Speaker 3: the precise timing of spikes between two directly connected neurons.
Speaker 2: Okay, let's isolate just two neurons.
Speaker 3: We'll call the first one the presynaptic neuron, the one
Speaker 3: sending the signal. The second is the post synaptic neuron,
Speaker 3: the one receiving the signal.
Speaker 2: Got it pre impost.
Speaker 3: STDP dictates that the physical strength of the synapse connecting
Speaker 3: them changes based entirely on the exact relative timing of
Speaker 3: when they both spike.
Speaker 2: Just the timing, just the timing.
Speaker 3: If the presynaptic neuron fires spike just before the post
Speaker 3: synaptic neuron fires, the physical can between them strengthens. The
Speaker 3: biological logic is that the first neuron likely played a
Speaker 3: role in causing the second one to fire.
Speaker 2: It's recognizing cause and effect exactly.
Speaker 3: However, if the timing is reversed, if the post synaptic
Speaker 3: neuron fires first and then the pre synaptic neuron fire
Speaker 3: slightly later, the connection physically weakens.
Speaker 2: Why we caedday, the.
Speaker 3: Logic being that the first neuron clearly didn't contribute to
Speaker 3: the second one firing, so their association is meaningless and
Speaker 3: should be diminished to save energy.
Speaker 2: I want to frame this as a strict rule of
Speaker 2: cause and effect. For you listening, think about the relationship
Speaker 2: between lightning and thunder. That's a great real world example.
Speaker 3: If you're standing outside and see a brilliant flash of
Speaker 3: lightning and it is consistently followed right after by a
Speaker 3: massive crash of thunder, your brain notes that specific timing
Speaker 3: lightning first, then thunder right.
Speaker 2: Because the timing is consistent and strictly ordered, your brain
Speaker 2: physically wires those two concepts together. The synapse strengthens. You
Speaker 2: learn the rule causes thunder perfect. But imagine you hear
Speaker 2: a random crash of thunder, and then ten seconds later
Speaker 2: someone flashes a flashlight in your eyes. Your brain completely
Speaker 2: ignores the association.
Speaker 3: The timing is wrong, the order is reversed.
Speaker 2: STDP means the neuromorphic hardware is teaching itself based solely
Speaker 2: on the timing of highly localized events. It doesn't need
Speaker 2: a massive global manager telling the whole network it made
Speaker 2: a mistake.
Speaker 3: It learns autonomously synaps by sunaps based purely on what
Speaker 3: it experiences.
Speaker 2: That is just so elegant.
Speaker 3: This is the physical realization of heavy in learning, a
Speaker 3: concept proposed back in nineteen forty nine, often summarized by
Speaker 3: the famous neuroscience phrase neurons that fire together, wire together.
Speaker 2: I've heard that phrase.
Speaker 3: Modern neuromorphic chips like Intel's lowiha actually have highly configurable
Speaker 3: learning rules built directly into the silicon that replicate these forms.
Speaker 2: Of STDP, so they can learn locally.
Speaker 3: Yes, this enables the chip to learn continuously on the fly,
Speaker 3: directly from streaming sensory data in the real world, adapting
Speaker 3: to new patterns without ever needing to connect to a
Speaker 3: massive cloud server to run a backpropagation algorithm.
Speaker 2: But wait, if we follow this logic to its physical conclusion,
Speaker 2: we hit a roadblock. What's the roadblock if STDP requires
Speaker 2: the physical synapps to remember its own history to know
Speaker 2: if it fired before or after its neighbor how do
Speaker 2: you actually do that? In silicon? A standard digital transistor
Speaker 2: forgets everything the exact second the power drops.
Speaker 3: That is the core issue.
Speaker 2: Yes, you can't build a brain out of amnesiacs. Did
Speaker 2: Carver meat have an answer for that? Or did engineers
Speaker 2: have to invent a completely new material?
Speaker 3: That is the exact problem that pushes neuromorphic engineering into
Speaker 3: the realm of exotic materials science. To truly replicate a
Speaker 3: biological synapse, we need a physical component that naturally remembers
Speaker 3: its own electrical history.
Speaker 2: And what is that?
Speaker 3: And this brings us to what is widely considered the
Speaker 3: holy grail of neuromorphic computing, the memorister.
Speaker 2: Memorister, it sounds like a piece of alien technology. What
Speaker 2: exactly is it?
Speaker 3: The word itself is a portmanteau of memory and resistor.
Speaker 3: In standard electronics, a resistor is a basic component that
Speaker 3: simply resists the flow of electrical current by a fixed
Speaker 3: unchanging amount.
Speaker 2: Okay, like a bottleneck in a pipe, right.
Speaker 3: But a memristor is a two terminal electronic device whose
Speaker 3: physical resistance fundamentally changes based on the history of the
Speaker 3: current that has flowed through it in the past.
Speaker 2: The physical properties of the material permanently change based on
Speaker 2: what it has experienced.
Speaker 3: Yes, as current flows through it in one direction, its
Speaker 3: resistance might drop. If current flows in the opposite direction,
Speaker 3: its resistance might increase.
Speaker 2: Wow.
Speaker 3: And crucially, when you turn the power completely off, the
Speaker 3: memorister remembers its last state of resistance indefinitely.
Speaker 2: That is deeply profound.
Speaker 3: It is the literal physical embodiment of a biological synapse
Speaker 3: in a living brain. The weight of a synapse, how
Speaker 3: strongly it connects two neurons changes based on the history
Speaker 3: of ions passing through it.
Speaker 2: Memristor does the exact same.
Speaker 3: Thing electrically in a neuromorphic chip utilizing memoristers, The electrical
Speaker 3: resistance of the device directly encodes the strength of the connection.
Speaker 3: That resistance physically alters when current flows, effectively implementing a
Speaker 3: local learning rule directly in the atomic physics of the
Speaker 3: device itself.
Speaker 2: So we don't need a separate memory chip down the
Speaker 2: street to remember the synaptic weight, and we don't need
Speaker 2: a processor to calculate how it should change.
Speaker 3: No, you don't.
Speaker 2: The memorists are just is the memory in the processor simultaneously.
Speaker 3: And the engineering implications of that collocation are staggering, particularly
Speaker 3: when it comes to the core mathematics of artificial intelligence.
Speaker 2: The core mathematics, we're talking about matrix multiplication, right.
Speaker 3: Yes. To appreciate this, we need to understand the massive
Speaker 3: mathematical burden of AI.
Speaker 2: Okay, I need to understand this matrix math thing. Why
Speaker 2: is multiplying grids of numbers such a massive burden for
Speaker 2: normal computers?
Speaker 3: Think about how an artificial intelligence actually sees an emas image.
Speaker 3: Let's say we have a tiny image just one hundred
Speaker 3: pixels by one hundred pixels.
Speaker 2: That's ten thousand numbers representing brightness exactly.
Speaker 3: Now, feed that into a neural network layer with ten
Speaker 3: thousand artificial neurons. Every single neuron needs to connect to
Speaker 3: every single pixel. Oh wow, that creates a matrix of
Speaker 3: one hundred million distinct weights. To calculate the output, the
Speaker 3: computer must multiply the ten thousand pixel values by that
Speaker 3: one hundred million weights.
Speaker 2: So the processors basically doing hundreds of millions of tiny
Speaker 2: math homework problems one by one.
Speaker 3: And because of the von Neumann bottleneck, it is fetching
Speaker 3: the textbook from the library down the street for every
Speaker 3: single one of those problems.
Speaker 2: Just endlessly fetching and returning.
Speaker 3: It is fetching weights for memory, doing a digital multiplication,
Speaker 3: adding the result to a running total, and storing it back.
Speaker 3: Trillions of operations for a single frame of video.
Speaker 2: It's exhausting just thinking about it.
Speaker 3: Now, replace that conventional architecture with a massive grid of
Speaker 3: memoristers arranged in what we call a crossbar picture, a
Speaker 3: microscopic to tech toeboard.
Speaker 2: Okay, a dense grid of intersecting wires.
Speaker 3: At every single intersection where a vertical wire crosses a
Speaker 3: horizontal wire, there is a memorister connecting them. In this
Speaker 3: physical setup, we don't do digital math at all.
Speaker 2: Wait, no math.
Speaker 3: We map the input data the pixels of our image
Speaker 3: as electrical voltages sent down the vertical wires. The stored
Speaker 3: weights of our neural network are physically represented by the
Speaker 3: conductance of the memorisers at the.
Speaker 2: Intersections, conductance being the opposite of resistance. How easily they
Speaker 2: let the electricity through exactly.
Speaker 3: Now, basic high school physics takes over Alms law states
Speaker 3: that current equals voltage multiplied by conductance.
Speaker 2: Okay, remember that, so as the.
Speaker 3: Voltage flows into the memorister, the resulting electrical current coming
Speaker 3: out the other side is the exact literal mathematical product
Speaker 3: of the input and.
Speaker 2: The weight It multiplies it physically.
Speaker 3: Yes, and then Kershoff's current law states that currents flowing
Speaker 3: into a single node naturally add together. So the currents
Speaker 3: from all the memoristers in a column naturally sum up
Speaker 3: as they flow into the horizontal wire.
Speaker 2: Wait, the physics engine of the universe calculates the multiplication
Speaker 2: and the addition for us instantly, just by letting electricity
Speaker 2: flow through the grid.
Speaker 3: The entire matrix vector multiplication, the grueling operation that requires
Speaker 3: billions of digital steps and massive energy on a standard GPU,
Speaker 3: happens in a single instantaneous analog step.
Speaker 2: That is unbelievable.
Speaker 3: There is zero fetching from memory. The math happens naturally
Speaker 3: as the electricity flows through the material. The promise here
Speaker 3: is an improvement in energy efficiency for AI inference tasks,
Speaker 3: not by a factor of ten or twenty, but by
Speaker 3: several orders of magnitude.
Speaker 2: I have to play the skeptic here, though, Okay, go
Speaker 2: for it, because what you are describing sounds like pure magic.
Speaker 2: If we have a physical component that acts exactly like
Speaker 2: a biological sinn apse, eliminates the von Neumann bottleneck, entirely
Speaker 2: uses zero digital computation and saves massive amounts of power.
Speaker 3: Why isn't it in my smartphone? The fair question why
Speaker 3: isn't every tech giant completely abandoning traditional silicon and churning
Speaker 3: out memorisarch chips, Because the engineering reality of working with
Speaker 3: these novel materials is brutal. Bridging the gap from a
Speaker 3: beautiful theoretical physics concept to a mass produced, highly reliable
Speaker 3: commercial chip is an incredibly difficult materials science challenge.
Speaker 2: It's just hard to build them.
Speaker 3: Memoristive devices suffer from several severe practical physical hurdles that
Speaker 3: have slowed their widespread commercial deployment.
Speaker 2: What kind of hurdles are we talking about it?
Speaker 3: The first major obstacle is device variability variability. Yes, when
Speaker 3: a foundry manufactures a standard digital silicon transistor, the process
Speaker 3: is so perfected that the transistors are incredibly uniform. But
Speaker 3: when you attempt to create millions or billions of nanoscale
Speaker 3: memoristers in a dense array, individual devices tend to have
Speaker 3: slightly different.
Speaker 2: Physical characteristics, so they aren't identical.
Speaker 3: Their atomic structures aren't perfectly identical. They don't all react
Speaker 3: to electrical current in exactly the same way. In a
Speaker 3: digital system, minor variations don't matter because a strong signal
Speaker 3: is still read as a one, But in an analog system,
Speaker 3: the exact value matters.
Speaker 2: It's like having a world class orchestra where every single
Speaker 2: violin is tuned just slightly differently. It might still loosely
Speaker 2: sound like a sympathy, but the high level precision required
Speaker 2: for complex tasks is totally.
Speaker 3: Lost exactly the analog noise overwhelms the signal. The second
Speaker 3: major physical challenge is drift.
Speaker 2: Drift.
Speaker 3: What's that even when a memory star is not being
Speaker 3: deliberately programmed or changed, when it's just sitting there holding
Speaker 3: a memory its internal resistance the stored synaptic weight can
Speaker 3: slowly shift or degrade on its own overtime on its own.
Speaker 3: This is due to thermal fluctuations or the natural movement
Speaker 3: of atoms within the crystal lattice of the material. Imagine
Speaker 3: your computer's hard drive slowly randomly changing the text inside
Speaker 3: its own five while the power is turned off. It
Speaker 3: progressively degrades the accuracy of the neural network over time.
Speaker 2: That sounds like an absolute nightmare for reliability. If you
Speaker 2: train a self driving car in a memorister chip, you
Speaker 2: don't want it slowly forgetting what a pedestrian looks like
Speaker 2: over the course of a year.
Speaker 3: No, you definitely do not. And the third major hurdle
Speaker 3: is endurance. Endurance biological synapses are remarkably resilient. They can
Speaker 3: dynamically strengthen and weaken continuously for an entire human lifespan
Speaker 3: a century or more. Memorisers, however, rely on physically moving
Speaker 3: atoms or altering crystalline structures at the nanoscale every time
Speaker 3: they are reprogrammed, so they wear out. They eventually suffer
Speaker 3: from physical wear and tear. The number of times a
Speaker 3: memorister can be reliably programmed before its properties degrade irreversibly
Speaker 3: is limited. This endurance limit is a severe bottleneck for
Speaker 3: systems that are meant to learn continuously on the fly.
Speaker 2: So it's brilliant in theory and it works in the lab,
Speaker 2: but the physical materials keep breaking down, drifting, or forgetting
Speaker 2: things when we try to scale it up.
Speaker 3: It is an active, intensely competitive battleground in material science.
Speaker 3: Right now. Researchers worldwide are investigating numerous exotic candidate technologies
Speaker 3: fighting for supremacy to solve these.
Speaker 2: Exact issues, like what kind of technologies?
Speaker 3: We are looking at phase change memory, which uses heat
Speaker 3: to change the material from amorphos to crystalline. We are
Speaker 3: looking at resistive RAM, conductive bridging RAM, which physically builds
Speaker 3: and breaks tiny metallic wires. At the atomic level, we
Speaker 3: are looking at ferroelectric devices.
Speaker 2: That sounds like a lot of options.
Speaker 3: Each of these exotic materials offers a different trade off.
Speaker 3: One might have incredible endurance but terrible variability. Another might
Speaker 3: be highly stable but require too much energy to program.
Speaker 3: Progress is steady, but we haven't found the perfect Goldilocks
Speaker 3: material yet that can be seamlessly integrated into standard commercial
Speaker 3: silicon foundries.
Speaker 2: Okay, so the memborister revolution is still fighting its way
Speaker 2: out of the materials science labs. But despite these massive
Speaker 2: physical herds, neuromorphic chips, even the ones relying on standard
Speaker 2: silicon like Intel's LOWI heat, are already leaving the lab
Speaker 2: and entering the wild. They definitely are, and they are
Speaker 2: thriving in very specific environments, environments where traditional power hungry
Speaker 2: AI simply cannot survive.
Speaker 3: If you want to find the perfect natural habitat for
Speaker 3: neuromorphic technology. Today you look to a field called edge
Speaker 3: computing the edge.
Speaker 2: Edge computing refers to environments that are physically far removed
Speaker 2: from the cloud and those massive warehouse sized data centers
Speaker 2: we discussed earlier.
Speaker 3: So out in the real world.
Speaker 2: We are talking about devices operating out in the remote,
Speaker 2: chaotic physical world. These devices are constrained by strict physical limits.
Speaker 2: They're usually running on limited batteries, they cannot be plugged
Speaker 2: into a wall. Furthermore, they require instant, zero latency reactions
Speaker 2: to their environment. They simply cannot afford the time or
Speaker 2: the energy to beam sensory data back to a server farm,
Speaker 2: wait for a massive GPU to process it, and wait
Speaker 2: for the instructions to be beamed back.
Speaker 3: And a major part of operating effectively at the edge
Speaker 3: is how the machine actually perceives the physical world. We
Speaker 3: talked earlier about how traditional processors are forced to march
Speaker 3: to a rigid metronome clock cycle. Yes, well, it turns
Speaker 3: out our traditional sensors, the eyes of our machines, do
Speaker 3: the exact same thing, which brings us to the concept
Speaker 3: of event based cameras. Right. Traditional cameras operate on a
Speaker 3: rigid frame based paradigm. A standard digital camera takes a
Speaker 3: full complete picture, let's say, sixty complete frames every single.
Speaker 2: Second, regardless of what's happened exactly.
Speaker 3: It captures the light value of every single pixel, millions
Speaker 3: of pixels, over and over, completely regardless of what is
Speaker 3: actually happening in the scene in front of it.
Speaker 2: I love the analogy for this. A traditional frame based
Speaker 2: camera is like a terribly inefficient security guard sitting at
Speaker 2: a desk watching an empty hallway.
Speaker 3: A very bored security guard.
Speaker 2: The guard is forced by protocol to pick up the
Speaker 2: radio and call headquarters sixty two times a second to
Speaker 2: give a full report. The hallway is still empty. The
Speaker 2: hallway is still empty. The hallway is still empty.
Speaker 3: That sounds exhausting.
Speaker 2: The camera is generating massive amounts of entirely redundant data,
Speaker 2: and the processor downstream has to expand mass amounts of
Speaker 2: energy to process all those identical pictures just to mathematically
Speaker 2: confirm that nothing changed.
Speaker 3: And the consequence of that setup is a massive waste
Speaker 3: of power and computational bandwidth.
Speaker 2: So what's the alternative.
Speaker 3: An event based sensor, often called a dynamic vision sensor,
Speaker 3: is intrinsically tied to the neuromorphic architecture, and it fundamentally
Speaker 3: changes this paradigm. Instead of capturing full frames on a
Speaker 3: rigid clock, an event based camera only outputs a signal,
Speaker 3: a spike, when a specific individual pixel detects a meaningful
Speaker 3: change in brightness.
Speaker 2: Every pixel acts independently.
Speaker 3: Every single pixel acts independently.
Speaker 2: So our security guard completely goes to sleep. They use
Speaker 2: absolutely zero power. They only pick up the radio and
Speaker 2: call headquarters when someone actually opens the door at the
Speaker 2: end of the hallway.
Speaker 3: Exactly the camera produces a sparse, asynchronous stream of spikes
Speaker 3: corresponding only to movement or change in the visual field.
Speaker 3: If the scene is static, the camera outputs absolutely nothing
Speaker 3: and consumes practically no power.
Speaker 2: But if something moves, But.
Speaker 3: If an object moves rapidly, the individual pixels capture that
Speaker 3: movement with microsecond precision, far faster than a standard sixty
Speaker 3: frame per second camera could ever hope to catch. Furthermore,
Speaker 3: because each pixel manages its own exposure based only on
Speaker 3: local change, event cameras don't get blinded by looking directly
Speaker 3: at the sun the way normal cameras do. They have
Speaker 3: immense dynamic range.
Speaker 2: And because this sensory data is already in the form
Speaker 2: of discrete spikes. It forms a perfectly matched, seamless pipeline
Speaker 2: directly into a neual morphic chip.
Speaker 3: It speaks the same language.
Speaker 2: The chip doesn't have to translate a massive JPEG image
Speaker 2: into a neural network. It just receives the spikes and
Speaker 2: reacts instantly. Where's this hardware actually being deployed today?
Speaker 3: The most compelling and rap rapidly advancing application domain by
Speaker 3: far is robotics.
Speaker 2: Robotics that makes sense.
Speaker 3: Autonomous robots face the exact same fundamental survival problems that
Speaker 3: biological animals face in the wild. They have to process
Speaker 3: incredibly rich, noisy sensory data from a chaotic real world.
Speaker 2: They must adapt their physical behavior to unpredictable changing environments.
Speaker 3: They must make rapid life or death decisions without the
Speaker 3: latency of calling a cloud server. And they have to
Speaker 3: do it all while carrying their own limited energy source
Speaker 3: a heavy battery.
Speaker 2: Right picture an autonomous drone trying to navigate through a
Speaker 2: dense forest at forty miles per hour.
Speaker 3: That's a perfect example.
Speaker 2: It can't pause mid air for two seconds to upload
Speaker 2: a video frame to a server, wait for a neural
Speaker 2: network to calculate the depth of the forest and wait
Speaker 2: for instructions on how to dodge a branch. It will
Speaker 2: crash into a tree before the data even reaches the cloud.
Speaker 3: When researchers equip these robots with event based sensors directly
Speaker 3: wired into neuromorphic processors, they achieve extraordinary results.
Speaker 2: Oh like, what kind of results?
Speaker 3: These robotic systems can perform adaptive locomotion like a multi
Speaker 3: legged robot learning to walk over uneven shifting terrain on
Speaker 3: the fly, and high speed obstacle avoidance at power budgets
Speaker 3: that would absolutely crush a conventional computer.
Speaker 2: Because conventional computers are just too heavy and power hungry.
Speaker 3: If you tried to run a traditional state of the
Speaker 3: art computer vision model on a small flying drone, the
Speaker 3: battery would be entirely consumed just running the processor, leaving
Speaker 3: zero energy for the actual rotors to keep it in
Speaker 3: the air. Neuromorphic chips solve the fundamental power and weight
Speaker 3: constraint problems inherent in mobile robotics.
Speaker 2: It's essentially giving machines the survival instincts of an insect, fast,
Speaker 2: incredibly energy efficient, and fully self contained.
Speaker 3: But the applications are not limited merely to building better, faster, robots.
Speaker 3: There is a profound bidirectional relationship here. Building these physical
Speaker 3: neuromorphic chips is proving to be an invaluable tangible tool
Speaker 3: for the field of computational neuroscience itself.
Speaker 2: Wait, how so does building a silicon brain actually help
Speaker 2: us understand the wet biological brain better immensely.
Speaker 3: In theoretical neuroscience, researchers can sometimes get away with vague
Speaker 3: mathematical models or high level assumptions about how large populations
Speaker 3: of neurons.
Speaker 2: Interact because it's just theory.
Speaker 3: But when you are forced to build a physical piece
Speaker 3: of hardware in silicon you cannot be vague. Hardware requires explicit, rigorous,
Speaker 3: quantitative commitment.
Speaker 2: Right. You have to actually build the circuit.
Speaker 3: You must specify exactly how the voltage operates, exactly, how
Speaker 3: the timing windows work, exactly how the local learning rules
Speaker 3: are applied. The physical implementation of these theories forces neuroscientists
Speaker 3: to be absolutely rigorous.
Speaker 2: It acts as a strict physical reality check for their theories.
Speaker 3: Exactly when they run their biological models on physical platforms
Speaker 3: like the Heidelberg Brain Scale s system, the behavior of
Speaker 3: the hardware provides a tangible test of their theories about
Speaker 3: working memory, temporal coding, and sensory at adas adaptation.
Speaker 2: And what if the hardware doesn't act like the brain.
Speaker 3: If the silicon network does not behave the way the
Speaker 3: living animal does, it indicates that the biological theory is
Speaker 3: fundamentally flawed or incomplete. It creates a beautiful reciprocal feedback
Speaker 3: loop where engineering constraints inform neuroscience and biological discoveries inspire
Speaker 3: better engineering.
Speaker 2: We spend a lot of time celebrating the hardware. The
Speaker 2: analog architecture makes complete sense. The power savings are literally
Speaker 2: orders of magnitude better, the event based sensors are brilliant,
Speaker 2: and the edge robotics applications are undeniably the future. Yes,
Speaker 2: so I have to address the elephant in the room.
Speaker 3: The elephant in the room. Let's hear it.
Speaker 2: If this hardware is so perfectly designed, so biologically inspired,
Speaker 2: and so vastly efficient, why isn't the entire tech industry pivoting?
Speaker 2: Why is the artificial intelligence landscape still almost entirely dominated
Speaker 2: by massive power hungry GPUs churning through conventional deep.
Speaker 3: Learning Because of the software. The software we have hit
Speaker 3: what the industry calls the software wall. The harsh, pragmatic
Speaker 3: truth of neuromorphic computing today is that programming a spiking
Speaker 3: neuromorphic chip to do complex tasks is incredibly difficult.
Speaker 2: Why is it so hard? We have millions of brilliant
Speaker 2: software developers in the world. We build massive operating systems
Speaker 2: in complex video games.
Speaker 3: We do. But the entire global ecosystem of software development,
Speaker 3: the massive, highly polished libraries, the intuitive training frameworks like
Speaker 3: PyTorch or TensorFlow, the decades of accumulated mathematical knowledge is
Speaker 3: entirely built around conventional continuous digital math.
Speaker 2: Oh right, Because we're used to the von Neumann system.
Speaker 3: The software ecosystem for neuromorphic computing is by comparison, very immature.
Speaker 3: But the problem isn't just a lack of polished tools
Speaker 3: or developer familiarity. It's a fundamental mathematical mismatch at the
Speaker 3: core of the.
Speaker 2: Technology because of the spikes.
Speaker 3: Because of the spikes, we talked earlier about backpropagation, the
Speaker 3: calculus based algorithm used to train on almost all modern
Speaker 3: AI to high levels of.
Speaker 2: Accuracy, the micromanager.
Speaker 3: The micromanager backpropagation relies on calculating gradients, smooth, continuous mathematical
Speaker 3: slopes that tell the network exactly which direction to adjust
Speaker 3: its weights to reduce errors calculus. You can only use
Speaker 3: this calculus if the mathematical function you are analyzing is
Speaker 3: continuous and smooth. It must be what mathematicians call differentiable.
Speaker 2: Meaning you can calculate a precise slope at any given
Speaker 2: point on the curve.
Speaker 3: But spikes are not smooth or continuous. A spike is
Speaker 3: a discrete, binary event in time, it either happens or
Speaker 3: it doesn't. Mathematically, a spike is not a gentle hill.
Speaker 3: It is a sudden vertical sheer cliff face. You cannot
Speaker 3: calculate a smooth slope on a sheer cliff face. Therefore,
Speaker 3: spiking neural networks are fundamentally non differentiable.
Speaker 2: I think we need a grounded analogy for this. Imagine
Speaker 2: trying to train a dog okay to dog with traditional AI,
Speaker 2: where you have continuous math in calculus gradients, it's like
Speaker 2: training a dog using a leash. You have continuous control.
Speaker 2: You can gently pull the dog to the left or
Speaker 2: smoothly guide it to the right. It's a smooth, continuous correction.
Speaker 2: I like that, But a spiking network, it's like a
Speaker 2: light switch. It's either on or off. You can't gently
Speaker 2: flip a switch. You can't give a nuanced correction, you
Speaker 2: can't use the leash of calculus. How do you train
Speaker 2: the network You can't directly use the loosh. That is
Speaker 2: the central algorithmic challenge of the field. How do you
Speaker 2: train a network that speaks exclusively in discrete, non differential
Speaker 2: events to perform highly complex, nuanced tasks at the exact
Speaker 2: same level of accuracy as a conventional network trained with
Speaker 2: advanced calculus. So what is the compromise? How are researchers
Speaker 2: currently trying to bridge this mathematical gap.
Speaker 3: The current dominant approach relies heavily on techniques called surrogate
Speaker 3: gradient methods. Surrogate gradients essentially scientists mathematically cheat during the
Speaker 3: training phase in their computer simulations. They take the sheer,
Speaker 3: undifferentiable cliff face of a spike and replace it with
Speaker 3: a smooth, continuous curve a surrogate just for the purposes
Speaker 3: of doing the calculus.
Speaker 2: To use my analogy, they pretend the light switch is
Speaker 2: actually a smooth volume dial, just long enough to figure
Speaker 2: out which direction they need to turn it.
Speaker 3: That's a great way to think of it. They calculate
Speaker 3: the approximate gradients using backpropagation, update the weights, and then
Speaker 3: map those newly trained weights back onto the discrete spiking hardware.
Speaker 2: It's an approximation. You are forcing the analog hardware to
Speaker 2: learn using a digital math translation.
Speaker 3: It is an approximation. Then, while surrogate gradients have driven
Speaker 3: significant progress and allowed spiking networks to tackle much harder problems,
Speaker 3: it remains a mathematical.
Speaker 2: Compromise, so it's not perfect. No.
Speaker 3: Currently, spiking neural networks trained using these surrogate methods still
Speaker 3: generally lag behind conventional deep neural networks when tested on
Speaker 3: highly complex industry standard benchmarks like massive natural language processing
Speaker 3: or hyper detailed image generation. The raw accuracy isn't quite
Speaker 3: there yet.
Speaker 2: So we have built a physical hardware architecture that is
Speaker 2: theoretically perfect for energy efficiency, but we haven't cracked the
Speaker 2: code on how to teach it complex tasks as effectively
Speaker 2: as our flawed brute force power hungry systems.
Speaker 3: Closing the software performance gap, figuring out how to achieve
Speaker 3: state of the art accuracy natively on spiking hardware while
Speaker 3: maintaining that insane energy efficiency is arguably the defining engineering
Speaker 3: challenge of the next decade in this field.
Speaker 2: Let's step back and look at the massive journey we've
Speaker 2: just taken. We started by looking at the crippling heat
Speaker 2: and fundamental inefficiency of the von Neumann bottleneck, the processor
Speaker 2: and memory forever commuting back and forth burning power. We
Speaker 2: cover a lot of ground, we really did. We explored
Speaker 2: Carver Meade's radical vision of using analog physics to mirror biology,
Speaker 2: ditching the rigid digital metronome for this silent, event driven
Speaker 2: jazz ensemble of spiking.
Speaker 3: Networks, and we looked at the incredible silicon blains actually
Speaker 3: being forged.
Speaker 2: Today, from Intel's lowy heat to the massive semi relations
Speaker 2: of Spinnaker and the accelerated physics of brain scale less.
Speaker 2: We dove into the holy grail of membristers, physical devices
Speaker 2: that literally remember their own electrical history like a living synapse,
Speaker 2: allowing the laws of physics to do the math for us.
Speaker 3: And we saw how this technology is finding its true
Speaker 3: calling at the edge of the network, powering the survival
Speaker 3: instincts of next generation robotics.
Speaker 2: It is a vast, incredibly complex landscape.
Speaker 3: It is, but the overarching trajectory is clear, and I
Speaker 3: believe inevitable. Biological brains have already provided an existence proof.
Speaker 2: They've proven its possible.
Speaker 3: They have proven definitively that general, highly flexible, adaptable intelligence
Speaker 3: can run on just twenty watts of power. Replicating that
Speaker 3: precise physical operating system in silicon isn't just a quirky
Speaker 3: alternative to mainstream computing.
Speaker 2: It's essential.
Speaker 3: It is perhaps the most ambitious, fundamental and consequential project
Speaker 3: in the history of human engineering. The biological brain has
Speaker 3: solved the thermodynamic problem of intelligent computation in a way
Speaker 3: that our current machine simply have not.
Speaker 2: I want to leave you, the listener, with a completely
Speaker 2: new thought to mull over, something that builds on everything
Speaker 2: we've discussed about physically mirroring the brain and hardware. What's
Speaker 2: the thought if we truly succeed in making this hardware,
Speaker 2: if we overcome the massive materials science turdles with membisters,
Speaker 2: if we finally crack the software wall, If our machines
Speaker 2: start learning locally, adapting, dynamically, and physically rewriting their own
Speaker 2: electrical pathways exactly like our biological minds do, will these
Speaker 2: neuromorphic computers eventually inherit our biological flaws.
Speaker 3: Wow, that's a fascinating question.
Speaker 2: Think about it. We know human brains are highly susceptible
Speaker 2: to optical illusions, specifically because of how our neural pathways
Speaker 2: take rapid shortcuts to save energy.
Speaker 3: We suffer from cognitive fatigue when our synaptic neurotransmitters are depleted.
Speaker 2: Exactly, we are prone to irrationality and emotional biases because
Speaker 2: our physical wiring connects logic centers directly to keep survival instincts.
Speaker 3: That makes perfect sense.
Speaker 2: If we build an artificial brain that physically thinks exactly
Speaker 2: like us just to save power, what else does it
Speaker 2: start doing like us? Does a neuromorphic AI start seeing
Speaker 2: phantom patterns?
Speaker 3: Does an autonomous robot experience the silicon equivalent of fatigue
Speaker 3: or a loss of attention after processing too much sensory
Speaker 3: data without a reset.
Speaker 2: If we build the machine in our exact physical image,
Speaker 2: do we inevitably build our biological vulnerabilities into it as well?
Speaker 2: Something to ponder