Speaker 1: Okay, let's just jump right into it. We talk a
Speaker 1: lot about how fast technology is moving, right like a
Speaker 1: you see a new smartphone, or you read about a
Speaker 1: chatbot that can write a decent poem, and it feels
Speaker 1: like a big deal.
Speaker 2: Oh yeah, it always feels like the biggest thing ever.
Speaker 1: Right, But today we're looking at a shift that is honestly,
Speaker 1: it's arguably as significant as the invention of gunpowder.
Speaker 2: I'd go as far as nuclear weapons. It's really on
Speaker 2: that historical scale exactly.
Speaker 1: We are talking about the so called third revolution in warfare.
Speaker 1: We're steering right down the barrel of artificial intelligence and
Speaker 1: autonomous weapons systems. And look, before you think we're about
Speaker 1: to sit here and recap the plot of some sci
Speaker 1: fi movie or you know, talk about a certain famous
Speaker 1: cyborg franchise.
Speaker 2: I absolutely know you want to bring up.
Speaker 1: I mean, it's right there. I always want to bring
Speaker 1: it up. But no, this is in fiction. For this
Speaker 1: deep dive, we are exploring a very real and very
Speaker 1: current analysis. Our source material today is the book Cybersecurity
Speaker 1: in the Age of Artificial Intelligence and Autonomous Weapons, edited
Speaker 1: by memet emmint Erindor, And.
Speaker 2: That's a really crucial distinction to make right out of
Speaker 2: the gate here, this isn't just a conversation about big
Speaker 2: shiny robots marching down the street. It's about the intersection
Speaker 2: where cybersecurity actually meets the battlefield. We're talking about invisible risks,
Speaker 2: things like software bugs, hacking vulnerabilities, data poisoning.
Speaker 1: Because if you have a weapon that literally thinks for itself,
Speaker 1: the scariest thing isn't necessarily the physical weapon, it's the
Speaker 1: code running it.
Speaker 2: Precisely, the code is the weapon.
Speaker 1: So our mission for this deep dive is to explore
Speaker 1: this new frontier. We really need to understand what happens
Speaker 1: when we take the kill chain and just hand it
Speaker 1: over to algorithms.
Speaker 2: Yeah, and to really grasp the cybersecurity dilemma that comes
Speaker 2: along with that, because think about it, when software is
Speaker 2: what pulls the trigger, a simple bug isn't just your
Speaker 2: screen freezing, it's a massive catastrophe.
Speaker 1: Let's get into the first part of this and define
Speaker 1: this new battlefield. The source material makes a fascinating point
Speaker 1: right off the bat, which is that security is just
Speaker 1: about physical borders anymore.
Speaker 2: No, it hasn't been for quite a while. The text
Speaker 2: highlights that we've completely moved into a multidisciplinary field. It's
Speaker 2: no longer just placing soldiers to guard.
Speaker 1: A line on a map, right, It's so much broader.
Speaker 2: Exactly, It's about protecting cyberspace, devices, networks and the data
Speaker 2: flowing between them all. But the real game changer, the
Speaker 2: thing the book focuses on is the hardware that operates
Speaker 2: autonomously within that space.
Speaker 1: Well, let's clarify that because we keep saying autonomous weapons
Speaker 1: systems or AWS, we need to be specific about what
Speaker 1: the sources mean. Because my nephew has one of those
Speaker 1: little drones he flies around the park. But I'm assuming
Speaker 1: we aren't talking about that level of tech.
Speaker 2: We definitely are not. The book actually cites specific definitions
Speaker 2: from the UN and the International Committee of the Red
Speaker 2: Cross to clarify this exact point. The primary distinction is
Speaker 2: the human element. So your nephew's drone, he's standing there
Speaker 2: holding a remote controller. He decides where it flies, he
Speaker 2: pushes a button to take a photo or in a
Speaker 2: military context, to drop a payload.
Speaker 1: So there's always a human in the loop.
Speaker 2: Right, That is classified as a remotely controlled platform, but
Speaker 2: an autonomous weapons system in AWS, it uses its own
Speaker 2: sensors and algorithms to detect, identify, track, and engage targets
Speaker 2: completely without human intervention.
Speaker 1: Completely without Yes.
Speaker 2: Once you switch it on, the system makes the decisions.
Speaker 2: The human is totally out of the loop.
Speaker 1: That phrase without human invention is really doing the heavy
Speaker 1: lifting there, And the book points out this isn't some
Speaker 1: far off theoretical concept. We actually have a documented date
Speaker 1: for one. This might have crossed over into reality right.
Speaker 2: Now, so we do. March twenty twenty, the source material
Speaker 2: spends time discussing the Cargo two drone incident in Libya.
Speaker 1: Yeah, reading that part gave me absolute chills. Walk us
Speaker 1: through what happened there.
Speaker 2: So it was during a conflict involving the Haftar armed forces.
Speaker 2: A UN expert panel put out a report mentioning that
Speaker 2: these lethal autonomous weapons systems were programmed to attack targets
Speaker 2: without requiring any data connectivity between the operator and the
Speaker 2: new Wait.
Speaker 1: So the robot was essentially off the leash entirely effectively.
Speaker 2: Yes, it was reportedly hunting down retreating logistics convoys. It
Speaker 2: wasn't hovering there waiting for a pilot back at a
Speaker 2: base to save fire. It was identifying its own targets
Speaker 2: based purely on its programming and then engaging them. That
Speaker 2: is the exact threshold we are talking about crossing.
Speaker 1: Which brings us to the core motivation. Why are militaries
Speaker 1: and nations pursuing this so aggressively? Why remove the human
Speaker 1: from the loop. Is it just the appeal of having
Speaker 1: the most cutting edge tech?
Speaker 2: Hardly? According to the contributors in the book, it really
Speaker 2: comes down to speed and efficiency. A human brain can
Speaker 2: only process battlefield information so quickly, right, we have limits exactly,
Speaker 2: But an ai. An AI can process a chaotic battlefield
Speaker 2: of windspeed, thermal signature's, movement patterns of dozens of objects
Speaker 2: in milliseconds.
Speaker 1: So it acts as a force multiplier.
Speaker 2: Right, You significantly reduce the physical risk to your own
Speaker 2: soldiers because you just don't need them phys on the
Speaker 2: front line, and you reduce costs. Autonomous machines don't need
Speaker 2: pensions or long term medical care.
Speaker 1: But the text argues this creates that cybersecurity dilemma we
Speaker 1: mentioned earlier.
Speaker 2: Yes, and it's a huge point.
Speaker 1: How is that dilemma different from just a standard arms
Speaker 1: race like during the Cold War.
Speaker 2: It's related to an arms race, but it's more specific
Speaker 2: to the nature of the technology. It's this rapid escalation
Speaker 2: spiral driven by deep insecurity. If Nation A builds a
Speaker 2: swarm of autonomous drones that can react faster than human reflexes,
Speaker 2: Nation B feels they have absolutely no choice but to
Speaker 2: build their own swarm just to survive, just to keep
Speaker 2: up right. But by doing so, both nations end up
Speaker 2: feeling vastly less secure because the sheer speed of a
Speaker 2: potential conflict has now been accelerated completely beyond human control.
Speaker 1: It's like two people pointing guns at each other, but
Speaker 1: the triggers are these insane hair triggers that might just
Speaker 1: go off if the wind blows too hard.
Speaker 2: That is a highly accurate analogy, and that hair trigger
Speaker 2: reality brings us to the Achilles heel of this entire concept,
Speaker 2: because here is the profound irony the book continually comes
Speaker 2: back to. We are pouring billions into building these super advanced,
Speaker 2: terrifying weapons systems, but at their absolute core, they are
Speaker 2: just computers.
Speaker 1: And computers can be hacked. I mean, my email account
Speaker 1: gets hacked, and it's a massive headache. If an autonomous
Speaker 1: tank gets hacked, that is globally catastrophic.
Speaker 2: The source material references a concept called normal accidents theory.
Speaker 2: It's an academic framework, but it is absolutely critical for
Speaker 2: understanding this risk.
Speaker 1: Can you unpack that theory for us a bit?
Speaker 2: Sure? It's a theory used to describe complex technological systems
Speaker 2: that are what they call tightly coupled tightly cut.
Speaker 1: Okay.
Speaker 2: Think of a loosely coupled system first, like a line
Speaker 2: of human soldiers marching. If one soldier trips over a rock,
Speaker 2: the others can just stop, help him up, or simply
Speaker 2: walk around him. There is natural slack in the system,
Speaker 2: human buffer room basically exactly. But in a tightly coupled system,
Speaker 2: like a network of autonomous weapons, everything is digitally integrated
Speaker 2: and reactions happen instantaneously. If just one part fails, maybe
Speaker 2: a single sensor of glitches or one line of code
Speaker 2: has a logic error, that failure cascades through the entire
Speaker 2: network before a human operator can even blink, let alone intervene.
Speaker 1: So if I'm an adversary, I don't necessarily need to
Speaker 1: blow up your million dollar robot with a missile. I
Speaker 1: just need to mess with its brain.
Speaker 2: That's exactly it. You attack the software. The book details
Speaker 2: several specific attack vectors that are highly concerning. Let's walk
Speaker 2: through a few of those, starting with spoofing.
Speaker 1: Right. Spoofing, this is where you feed the system fake
Speaker 1: signal data to confuse it. The classic example, the text
Speaker 1: springs up is from twenty twelve with a research team
Speaker 1: from the University of Texas.
Speaker 2: Yes, Todd Humphreys and his research team, they successfully used
Speaker 2: fake GPS signals to completely hijack an unmanned aerial drone.
Speaker 1: And they didn't even hack it right.
Speaker 2: Nope, they didn't break into the drone's actual computer system.
Speaker 2: They didn't crack any encryption passwords. They literally just convinced
Speaker 2: the drones navigation system that it was somewhere else.
Speaker 1: They gas at the robot in a manner.
Speaker 2: Of speaking, Yes they did. They fed it false data,
Speaker 2: so the drone thought it was suddenly gaining too much altitude,
Speaker 2: so its automated systems tried to correct that by diving
Speaker 2: down right into the ground or in this specific test case,
Speaker 2: right into the flight path of researchers wanted.
Speaker 1: That is wild, and that experiment was over a decade ago.
Speaker 2: Correct, the tech has only gotten more sophisticated. Then. Another
Speaker 2: major vector is data poisoning and This one is much
Speaker 2: more insidious because it happens during the machine learning phase,
Speaker 2: long before the weapon is ever deployed to a battlefield.
Speaker 1: This is where you mess with the massive data sets
Speaker 1: the AI trains on right, because these vision systems learn
Speaker 1: by looking at millions and millions.
Speaker 2: Of pictures exactly. You feed it data saying this is
Speaker 2: a tank, this is a civilian vehicle, this is a tree.
Speaker 2: If a bad actor can infiltrate that data set early
Speaker 2: on and subtly manipulate.
Speaker 1: It, like telling the AI that a civilian bus is
Speaker 1: actually a hostile tank.
Speaker 2: Yes, or training it so a specific camouflage pattern is
Speaker 2: ignored as just background noise.
Speaker 1: Then the AI essentially learns the wrong lesson, and it.
Speaker 2: Learns it perfectly. When you finally deploy that system into
Speaker 2: a live combat zone, it honestly thinks it's functioning one
Speaker 2: hundred percent correctly when it targets that silly and bus.
Speaker 2: The internal logic is perfectly sound based on what it learned,
Speaker 2: but the foundational data was poisoned, and because deep learning
Speaker 2: models often operate as a black box, you might not
Speaker 2: even realize that data was poisoned until after the smoke
Speaker 2: clears and the tragedy has already happened.
Speaker 1: That is genuinely terrifying, because machine doesn't experience doubt, It
Speaker 1: doesn't know it's making a mistake. It is absolutely confident
Speaker 1: in its error.
Speaker 2: Highly confident. Yes, and then you have jamming. This is
Speaker 2: the more brute force method of cyber attack, simply cutting
Speaker 2: off the communication link between the weapon and the base.
Speaker 1: Now, wait a second. If these things are totally autonomous,
Speaker 1: why does jamming even matter. Isn't the whole selling point
Speaker 1: that they don't need a human signal to keep fighting.
Speaker 2: You've hit on the ultimate double edged sword there. Proponents
Speaker 2: of these systems argue exactly that. They say AWS is
Speaker 2: superior because it works even if the enemy jams all
Speaker 2: your signals. Yeah, you know, the enemy cut our calms,
Speaker 2: No problem, our robots will just keep executing the mission.
Speaker 1: But I'm guessing there's a huge downside.
Speaker 2: A massive one. The text raises a critical counterpoint here.
Speaker 2: If the communication link is jammed, you completely lose the
Speaker 2: kill switch. If the AI suddenly starts malfunctioning or targeting
Speaker 2: the wrong facilities, or in a true nightmare scenario, turns
Speaker 2: on your own friendly forces, the human operator is totally
Speaker 2: locked out, you cannot send the signal to shut it down.
Speaker 1: You've built a monster you can't leash, and that naturally
Speaker 1: leads to what the book categorizes as the ultimate nightmare scenario,
Speaker 1: which is repurposing.
Speaker 2: Yes, repurposing. This is where a hacker doesn't just want
Speaker 2: to disable the weapon or jam its calms, they want
Speaker 2: administrative control. Imagine a scenario where a swarm of autonomous
Speaker 2: drones is hacked mid flight and suddenly their targeting protocols
Speaker 2: are rewritten to attack their own launch site.
Speaker 1: You've essentially handed your enemy a highly advanced weapon system
Speaker 1: that they didn't even have to spend a dime to
Speaker 1: build exactly.
Speaker 2: It really highlights how the modern battlefield isn't just physical
Speaker 2: terrain anymore. It's lines of code. But let's actually shift
Speaker 2: gears a bit. We've talked a lot about the shooting aspect,
Speaker 2: but before a military can shoot, they have.
Speaker 1: To see right the intelligence side.
Speaker 2: Yes, the text outlines how AI is completely revolutionizing intelligence gathering.
Speaker 1: And this is where AI is arguably already having its
Speaker 1: most massive real world impact. We're moving from just automating
Speaker 1: the trigger to automating the eyes.
Speaker 2: It's the sheer unmanageable scale of the data. The source
Speaker 2: material throws out a staggering figure eight hundred million global
Speaker 2: connected devices plus thousands of satellite images, plus endless social
Speaker 2: media feeds.
Speaker 1: It's an ocean of nois No building full of human
Speaker 1: analysts could possibly read or watch all of that.
Speaker 2: It's the classic needle in a haystack problem. But the
Speaker 2: haystack is the size of the entire planet. Humans literally
Speaker 2: cannot do it, but AI can. The book discusses Project
Speaker 2: Maven as a prime example of.
Speaker 1: This shift right. Project Maven this was the US military
Speaker 1: initiative that caused quite a massive stir in the tech
Speaker 1: sector a few years ago.
Speaker 2: It certainly did. It involved the military using AI algorithms
Speaker 2: to analyze drone footage. Specifically, they were using it to
Speaker 2: identify objects and find radical terrorist targets in places like
Speaker 2: Syria and Iraq. It effectively automated the stair surveillance.
Speaker 1: So instead of a human intelligence analyst having to sit
Speaker 1: there and watch twelve hours of grainy video of a
Speaker 1: random dirt road just to see if one specific truck
Speaker 1: drives by, the AI just watches it exactly.
Speaker 2: The AI watches the full twenty four hours of footage
Speaker 2: and just flags the three seconds that actually matter for
Speaker 2: the human interview. It handles signal intelligence and geospatial intelligence
Speaker 2: at a scale we've never seen.
Speaker 1: And it's not even just classified military footage anymore. The
Speaker 1: text mentions a specific software called Orbital Insight. This one
Speaker 1: really fascinated me because it's a commercial product.
Speaker 2: Right, it is entirely commercial. Orbital Insight analyzes satellite imagery
Speaker 2: on a massive global scale. It literally counts cars in
Speaker 2: retail parking lots. It monitors global supply chains. It spots
Speaker 2: anomalies and construction sites halfway around the world.
Speaker 1: So wait, an algorithm can tell if a rival Nations
Speaker 1: factory is running at full capacity. Yeah, just by counting
Speaker 1: the employee cars in the parking.
Speaker 2: Lot from space exactly. And if you know the factory's output,
Speaker 2: you know the economic health of your potential enemy. The
Speaker 2: immense benefit here, as the experts of the book point out,
Speaker 2: is that it moves intelligence from simply reacting to things
Speaker 2: that happen to predicting things before they happen.
Speaker 1: Predictive analytics for global conflict.
Speaker 2: Right, if you see the supply chain abruptly shifting in
Speaker 2: a certain way, or troop transports gathering at rail yards,
Speaker 2: or oil reserves suddenly being topped up. The AI can
Speaker 2: predict the conflict before the first shot is ever fired.
Speaker 1: But and in this topic, there is always a bail.
Speaker 1: But this level of technology isn't exclusive to the traditional
Speaker 1: good guys. This brings us to the human element and
Speaker 1: the incredibly dark side of this tech, which the book
Speaker 1: explores regarding radicalization.
Speaker 2: This is a really crucial chapter. We generally tend to
Speaker 2: think of AI as the exclusive domain of major superpowers
Speaker 2: the US, China, Russia, but the barrier to entry is
Speaker 2: dropping rapidly. Extremist groups like ISIS or Hezbola are actually
Speaker 2: early adopters of this technology.
Speaker 1: How exactly are they using it? Because I assume they
Speaker 1: aren't manufacturing autonomous tank swarms and secret facilities.
Speaker 2: Not hardware, no, but they are absolutely weaponizing the software.
Speaker 2: The text explains how these groups use AI driven algorithms
Speaker 2: for mass propaganda. They use it to create highly tailored
Speaker 2: information bubbles to recruit people much more efficiently than ever.
Speaker 1: Before, like using social media algorithms against us.
Speaker 2: Exactly, they use these tools to find psychologically vulnerable people
Speaker 2: online and then feed them a constant stream of highly
Speaker 2: specific radicalizing content.
Speaker 1: So it's essentially hyper targeted marketing, but.
Speaker 2: For terrorism precisely, and deep fakes play a massive role
Speaker 2: in the strategy.
Speaker 1: Oh absolutely defikes her everywhere.
Speaker 2: Now they're using AI to create incredibly realistic fake videos
Speaker 2: of political leaders saying things they never actually said, or
Speaker 2: fabricating video evidence of war crimes to incite immediate anger
Speaker 2: and violence. It entirely blurs the line between truth and.
Speaker 1: Fiction, which is a weapon in itself.
Speaker 2: Exactly, if a population can't trust its own eyes, they
Speaker 2: can be easily manipulated into conflict.
Speaker 1: And this brings up a huge legal headache that the
Speaker 1: book calls the accountability gap, because if you think about it,
Speaker 1: let's say an autonomous weapon or even a radicalized AI
Speaker 1: bought commits a documented war crime. Who actually goes to
Speaker 1: jail for that?
Speaker 2: That is the multi billion dollar legal question of our era.
Speaker 2: Is it the software programmer who wrote the original code
Speaker 2: three years ago in an office, is it the military
Speaker 2: commander who authorized the deployment of the unit, or is
Speaker 2: it the machine itself?
Speaker 1: Well, you obviously can't put a machine in a jail cell.
Speaker 1: You can't court martial, you can't.
Speaker 2: And because of that, Black box problem we mentioned earlier,
Speaker 2: the fact that deep learning algorithms often cannot explain the
Speaker 2: why behind a specific decision. You might not even be
Speaker 2: able to legally prove negligence by the programmer or the commander.
Speaker 1: Wow.
Speaker 2: It creates this terrifying legal vacuum where absolute atrocities could
Speaker 2: happen on the battlefield without any single human being held
Speaker 2: legally responsible under current international law.
Speaker 1: The book also gets a little philosophical here, which I appreciated,
Speaker 1: referencing Karlvon Clausovits. I know, Class of Its is kind
Speaker 1: of the founding father of modern war theory, but how
Speaker 1: on earth does an eighteenth century Prussian general fit into
Speaker 1: the age of artificial intelligence.
Speaker 2: It's a great connection the authors make, so Clausibitz famously
Speaker 2: talked about the trinity of war. He said war is
Speaker 2: driven by three forces. The government, which provides the rational
Speaker 2: political objective, the military, which provides the operational skill and chance,
Speaker 2: and the people who provide the raw passion and hostility.
Speaker 1: Okay, so government, military, people right.
Speaker 2: The expert contributors in the book argue that autonomous weapon
Speaker 2: system fundamentally disrupt this trinity. AI removes the passion and
Speaker 2: it attempts to eliminate the chance war suddenly becomes this
Speaker 2: very cold, calculated, mathematical exercise.
Speaker 1: I mean, from a purely tactical standpoint, that almost sounds
Speaker 1: of it.
Speaker 2: YEA efficient maybe, but incredibly dangerous Strategically. If war becomes
Speaker 2: cold and calculated and a leader doesn't have to risk
Speaker 2: the lives of their own human citizens, they just send
Speaker 2: in the robots, it drastically lowers the political threshold for
Speaker 2: starting a conflict in the first place.
Speaker 1: Oh, I see, it makes going to war politically easier exactly.
Speaker 2: If the cost of waging a war is just measured
Speaker 2: in tax dollars and silicon ships rather than human blood
Speaker 2: and coffins coming home, leaders might be far quicker to
Speaker 2: pull the trigger on an invasion, and that is a
Speaker 2: deeply terrifying prospect for overall global stability.
Speaker 1: It's making war completely clean for the aggressor, while it
Speaker 1: obviously remains horrifically dirty for the victim. Exactly. Let's shift
Speaker 1: to our final major topic. Here' talked extensively about the
Speaker 1: software in the code, but at the end of the day,
Speaker 1: these autonomous machines are physical objects made of metal and silicon.
Speaker 1: We have to talk about the global supply chain.
Speaker 2: This is an incredibly critical and often completely overlooked vulnerability
Speaker 2: that the text highlights. An autonomous weapon, whether it's a
Speaker 2: drone or a submarine, is just a massive collection of microchips,
Speaker 2: sensors and processors. And where do all those physical components
Speaker 2: come from.
Speaker 1: They come from all over the world.
Speaker 2: They're manufactured globally, right, The global electronics supply chain is
Speaker 2: incredibly vast and complex. The book points out that malicious
Speaker 2: actors don't actually need to hack your robot while it's
Speaker 2: on the battlefield. They can just infiltrate the manufacturing process
Speaker 2: years earlier by tampering with the hardware itself exactly. They
Speaker 2: can introduce a compromised hardware component, a tiny back door
Speaker 2: baked right into the silicon of a processor before the
Speaker 2: weapon system is even fully assembled.
Speaker 1: So a military could spend millions buying the absolute latest,
Speaker 1: greatest autonomous tank, but its secretly has an off switch
Speaker 1: that an adversary installed at a factory.
Speaker 2: In another country, or it's a switch that quietly makes
Speaker 2: the tank broadcast its exact GPS coordinates constantly, or maybe
Speaker 2: a logic bomb hard coded into the chip that just
Speaker 2: permanently disables the weapon systems on a highly specific date.
Speaker 2: That's insidious, and we actually have a massive historical precedent
Speaker 2: proving that cyber weapons can physically destroy hardware. The book
Speaker 2: brings up Stucksnet.
Speaker 1: Stucks Net, right, that is the absolute classic example of this.
Speaker 2: It is the ultimate proof of concept. Stuck's Net was
Speaker 2: a highly advanced computer worm that targeted Iran's nuclear enrichment
Speaker 2: facilities back in the early twenty tens. But the key
Speaker 2: thing is it didn't just steal data or crash computers.
Speaker 2: It physically destroyed the actual heady machinery the centrifuges by
Speaker 2: manipulating their motors to spin entirely out of control, all.
Speaker 1: While actively feeding fake data to the monitoring software, so
Speaker 1: the human operators thought everything was running totally fine exactly.
Speaker 2: That is the normal accidents theory. In terrifying action, the
Speaker 2: system actively lied to the human operators while it tore
Speaker 2: itself apart. Now just imagine that exact capability applied not
Speaker 2: to a static centrifuge, but to a heavily armed autonomous
Speaker 2: robot oh Man, a malicious piece of code that doesn't
Speaker 2: just spin a motor but fires a live missile into
Speaker 2: a populated area, or intentionally crashes an entire drone swarm
Speaker 2: into a civilian city. The physical destructive capability of code
Speaker 2: is completely.
Speaker 1: Proven, and it seems like absolutely everyone is rushing to
Speaker 1: get this capability. It's not just a US project.
Speaker 2: No, not at all. The book extensively lists the major
Speaker 2: players investing heavily in this space. Obviously the the UK, Russia,
Speaker 2: and China, but also nations like Israel, South Korea and
Speaker 2: Turkey are making massive leaps. It is a truly global phenomenon.
Speaker 1: And because it's software driven at its core, it must
Speaker 1: be exponentially harder to regulate than say, tracking uranium shipments
Speaker 1: for nuclear weapons.
Speaker 2: Oh, it's nearly impossible. You can easily count physical nuclear warheads,
Speaker 2: you can send international inspectors to enrichment facilities. You absolutely
Speaker 2: cannot inspect a line of code at a border crossing.
Speaker 2: You can't put a naval blockade on an algorithm downloaded
Speaker 2: over the Internet.
Speaker 1: It really feels like we are collectively prying open Pandora's
Speaker 1: box here.
Speaker 2: I think the contributors to this book would heavily agree
Speaker 2: with that assessment. We are currently in the middle of
Speaker 2: a global arms race that is vastly outpacing our diplomatic
Speaker 2: ability to create treaties or safety protocols.
Speaker 1: So what does this actually mean for us? As we
Speaker 1: try to wrap our heads around this, We are essentially
Speaker 1: taking the military kill chain, the process to find fixed
Speaker 1: track target, engage in assess, and we are handing the
Speaker 1: keys over to algorithms.
Speaker 2: That is the ultimate summary. Yes, we are making a
Speaker 2: very deliberate trade off. We as a society are trading
Speaker 2: human control, ethical oversight, and moral responsibility to gain raw speed, precision,
Speaker 2: and mechanical efficiency.
Speaker 1: And by making that trade, we are actively opening ourselves
Speaker 1: up to these catastrophic cyber vulnerabilities. The exact autonomous weapon
Speaker 1: that is designed to protect your borders might be the
Speaker 1: very thing that turns against your own citizens if an
Speaker 1: adversary guesses the right password or subtly poisons the training data.
Speaker 2: It forces us to completely redefine what national security even
Speaker 2: means in the twenty first century. It's not just about
Speaker 2: armour plating and blast walls anymore. It's deply dependent on firewalls,
Speaker 2: encryption keys, and data integrity.
Speaker 1: It's an immense amount of information to process. As we
Speaker 1: close out this deep dive, what is one final thought
Speaker 1: you'd want to leave our listeners with today.
Speaker 2: I'd really leave you with this provocation, which strongly echoes
Speaker 2: the underlying anxieties throughout the text. We are moving toward
Speaker 2: a near future where autonomous weapons will operate entirely at
Speaker 2: the speed of silicon. They will be making life or
Speaker 2: death decisions to attack in literal nanoseconds in a world
Speaker 2: where those weapons can be instantly compromised by a single
Speaker 2: bad line of code. Does human diplomacy even have the
Speaker 2: time to function anymore?
Speaker 1: That is a staggering thought. If an AI suddenly decides
Speaker 1: to escalate a border skirmish into a full blown war
Speaker 1: because of a software bug or spoot sensor, the shooting
Speaker 1: starts before a president or a prime minister can even
Speaker 1: pick up a telephone to de escalate.
Speaker 2: Exactly will we even realize an error has occurred in
Speaker 2: time to somehow pull the plug, or will the war
Speaker 2: be fought and spiraled out of control before human leaders
Speaker 2: even though it started?
Speaker 1: But you know, to build on that that, there's something
Speaker 1: else for you listening to mull over. Even if we
Speaker 1: do keep a human in the loop, just sitting at
Speaker 1: a desk hitting a prove on whatever the AI suggest,
Speaker 1: how long until we suffer from absolute automation bias?
Speaker 2: Oh, that's a great point.
Speaker 1: Where the human operator just inherently trusts the machine's perfect
Speaker 1: math so completely that they never actually question it. At
Speaker 1: that point, we might think we're in control, but we've
Speaker 1: essentially just become a biological rubber stamp for an alien.
Speaker 2: Intelligence will become the slow part of the machine.
Speaker 1: Exactly speed kills indeed. Well, that is all for our
Speaker 1: deep dive into cybersecurity in the age of artificial intelligence
Speaker 1: and autonomous weapons. Thank you so much for exploring this
Speaker 1: with us today, and to everyone listening, keep questioning the code.
Speaker 2: Stay safe out there.