Could somebody like Russia hack into John Deere and turn off the tractors here in
the United States, And unfortunately the answer is probably not the answer you want
to hear. Welcome everyone to the Industrial Security Podcast. My name is Nate
Nelson. I'm here with Andrew Ginter, the vice president of Industrial Security at
Waterfall Security Solutions, who's going to introduce the subject and guest of our shows
today. Andrew, has it gone. I'm very well, Thank you,
Nate. Our guest today is Marcus Sucks. He is the deputy director for
Research at the McCrary Institute for Cyber and Critical Infrastructure Security at Auburn University,
and he's going to be talking about sort of his latest interest. He's been
in the field a long time. He's going to be talking about food.
Do everything from agriculture to food and beverage manufacturing all the way to restaurants.
All right, sounds interesting enough. Let's listen into your conversation with Marcus.
Hello, Mark, and welcome to the podcast. Before we get started,
can I ask you to tell our listeners a little bit about yourself and about
the good work that you're doing at Auburn University. Well, hello, Andrew,
and great to be with you today. I really appreciate taking the time
and to be able to talk to your audience. So I'm at Auburn University,
been here about three years, and we have two locations. There's the
main Auburn campus that's in east central Alabama, and then up here in Huntsville,
which is North Alabama, which is where I sit, and we're in
the shadow of the big rocket that we have here that sits alongside the highway,
part of the Redstone Arsenal and the space programs and things that go on
here. So Auburn's very interested in critical infrastructure protection from several angles. And
I'm a member of what's called the mccrarrie Institute for Cyber and Critical Infrastructure Security.
There are three of us up here in Huntsville, and of course we're
working with the people here mostly on applied research. So the things that we're
studying on campus, working with students and professors, we're on the theoretical side,
we can apply them here to the audience that's here in the Huntsville Redstone
area. There's also a team in Washington that works the policy side, so
they're very well integrated with the White House. And Defense Department, Congress and
others. So literally between those three like a three legged stool. Two organizations
in Alabama, one up in Washington. We cover the playing field pretty well
from policy theory and practical application. So what got me here. We were
living in the DC area. I was there for about twenty two years.
Decided to move to Alabama about four or five years ago, and I've had
an old relationship with Auburn. When they heard I moved here, they said,
hey, you need to come join us because what I was doing in
Washington for over two decades was critical infrastructure protection. So we had moved there
back in nineteen ninety eight. I was a military move. I was in
the Army at the time, and it was to create an organization to defend
the Defense Department's networks from ongoing intrusions coming from foreign countries, so as well
before there was a cyber command, and we began to develop the first concepts
of how you actually do this, how you would defend this big global network.
Along the way, Y two K happened, and of course September eleventh
happened, which was a physical attack versus a cyber attack, but lots of
lessons learned about the integration of the critical infrastructure. So after retiring from the
Army, I spent a couple of years at the White House working in the
National Security Council, and then went on to work for Stanford Research for a
while. Then did quite a long time, almost a decade at Verizon doing
national security work in the communications sector. Was the chief security officer for about
three years at NURK in the power world, so that's the North American power
grid including Canada, US and Mexico. And then did a little time with
some startups, just kind of playing in that world. And after moving here
to Alabama, and I've owned a farm for a number of years, and
when we got here to Alabama, the food and agricultural sector is one of
the sixteen critical infrastructure sectors. I began asking simple questions of what are we
doing to protect this sector. I knew all about comms, knew all about
power and it and things like that, and I really very few answers,
and so that got me inspired. And we've been doing a lot of work
now to try and figure out, Okay, what can we do, what
can the Meccrari Institute do to help push the needle a little bit to get
more security into the food and agricultural sector. And the more we began working
with dairy and beef and forestry, poultry and fisheries, all the things where
the food comes from, and then going up through the production of food in
say like the chicken poultry plants, or beef producers, or grainaries, canneries,
and then all the way to restaurants, grocery stores, commissaries and so
forth. It's a highly integrated world, very computerized, very industrialized, but
not a lot of understanding about the security threats from a computer or cyber world.
And of course lots of industrial systems, lots of control systems, lots
of the same words we see in regular production in other sectors are also there
in the food industry. So that's become a bit of a focus of mind,
along with continuing to work with power and comms in other areas. So
there there's a nice short answer as to what we're doing here at Auburn.
Wow. So that's a lot of responsibility over the years. Well, bluntly,
i'm impressed, good job, thank you for your leadership. But we're
here talking about your most recent interest the agricultural sector. You know the critical
infrastructure of producing food. And you know, thirty forty long time ago,
I was a young man. I was raised on a farm, and back
then, we had trucks that had analog controls. There were no computers in
the trucks back then. There you know, we had tractors where you know,
they steered because you moved the steering roll and there was a mechanical linkage.
You know, we we we did things in a sense. What what
I now understand is the old way. What does sort of the the big
end, the majority of food production look like today. Well, the good
news is those tractors are still there, so we haven't completely gone to a
robotic system where there's no farmer. So you can still ride around in your
Cabota or your whatever, your favorite tractor John Deere, et cetera. But
like with any technology, those mechanical devices, they've been evolving and as the
engines have gotten better, you know, we added electronics to them decades ago.
We've we've added computer systems to them. The big change, probably in
the last decade or so, is the connectivity of the tractor. We'll just
focus on a tractor for now. When the farmers up in his tractor.
Let's assume it's a nice and closed one that's heated into when it's cold or
air conditioned when it's hot. In front of you. Sits a whole control
system that, on the high end very well connected. It's it's got satellite
connectivity, or it could have groundcover four G, five G, you know,
LTE type stuff through your carriers. There could also be sensors across the
farm. The even bigger farms, the ones that are thousands of acres in
size, might have their own private grid of communications that the machines can talk
to. In addition to the commercial satellites and so forth. What comes through
there the machine itself might be getting information could be real time agricultural information from
overhead satellites, things about soil conditions, things about wind direction, if you're
spraying. Just about anything environmental can now be done real time versus estimating or
guessing. And in fact, the higher end machines. If you think about
a tractor that's pulling a sprayer or pulling some sort of applicator behind it,
we could be throwing out seeds, We could be throwing out fertilizer. We
could be throwing out a weedkiller. Each of those nozzles or each of those
little rows where the seeds are going into. They can now be finely controlled
so that we only apply the fertilizer where the fertilizer is needed, or we
only apply the weed killer where it's needed. Rather than just broadcasting uniformly,
it's being sprayed very precisely. And that's developed this term known as precision agriculture
or precision farming. And you'll hear that term precision used in lots of places
because that accurately determines or describes rather what's going on. And you can think
about the economic impact here. The cost of running a farm, the cost
of food production is brought down if you can be more precise with where you
put your seeds, where you put your fertilizer, where you put your weed
killers, and other things you might be applying. It also allows for better
input of data into the system. As you're collecting data about what's going on
at harvest time, the yield that's coming out of the field is much better,
not just estimated, but direct data collected. We can now track grains,
We can track things coming out of that farm far better as they go
into the food system. So you can see where the use of computer systems
and particularly connected computer systems, and I realized it'd be a whole other topic
talk about AI. But AI is also entering into this world to help the
farmer be better at farming. And mostly again we're talking economics here, lowering
the cost of farming, increasing the yields, and increasing the production and the
output through the use of all these connected systems you've mentioned the farming end of
it. They primary production. I've never worked in or with a you know,
sort of call it a food factory, a slaughterhouse, a place where
canned soups are produced. What what does that piece of it look like today?
So similar to any other manufacturing, whether it's automotive or steel production,
it's an assembly line, if you want to think of it. There's raw
ingredients to go in one end, there's a finished product that comes out the
other end, and there's some usually a mechanical system some sort, that's moving
along the the chickens, the beef, the cans, the fish, the
whatever it is. Let me just take poultry probably is an easy example.
Most people are familiar with the poultry industry. You get your chicken sandwich,
or your nuggets or your hot wings for football games that you know you're buying
those in a grocery store. But of course they start off as a chicken,
you know, a live bird grown in a chicken house, along with
thousands of other ones. They begin obviously as eggs produced by laying hens,
chicks are born, and all along the way here we've now got computer controls
that are monitoring temperature, humidity, air quality, food quality, beginning with
the laying hen through the egg to the chick, and then the chicks are
delivered to these growing houses, where of course they mature into adult chickens and
ultimately off to the processing plant. In an earlier era, the air conditioning
system, for example, in a chicken house all manually controlled. You'd have
thermostats that you could figure out what's going on, But today all computer controlled,
and in fact, in today's poultry world, most of these chicken houses
they're owned by a farmer, but the chickens inside are owned by the large
poultry company. The farmer's role is just to make sure that they're protected from
the elements that they get their food, that they're protected while they're growing,
but they don't actually belong to the chicken farmer the gathering of those birds.
When they're ready to be brought in and processed, a team will come in
and gather them up and they clean the house out income a whole bunch of
new chicks, and back again they go. So at the processing end you've
got big, large machines that live animals. You've got to consider this are
humanely killed. This is done in different ways, whether it's a bird,
whether it's a fish, whether it's a pig or a beef or whatever.
But then the animal has to be skinned and then separated into the different parts.
Historically this was all done by hand, and of course it would take
a while to slaughter a complete animal and produce the output. Now more and
more this is done by machines, and there still are humans there, of
course that are monitoring what's going on. But if you can imagine the amount
of work it takes to debone something to actually produce the meat that's coming off
of alpha an animal, there's a lot of human effort. If you can
let a machine do it, you cut back on the human effort. But
one of the issues they've had is you do create a lot of waste because
the human hands are still far better at this than a machine can be,
but we're getting better and again. Control systems, computers, learning AI is
all part of that. Inside one of these facilities, temperature has to be
regulated, they have to be sanitary, they have to be cleaned. There
of course, are inspected daily. The output coming out, the waste has
to be dealt with. In some cases the waste is actually reused, becomes
fertilizer, becomes other types of things. So there's very little actual product that
goes to a landfill because of the ways that we've been able to reuse virtually
all the parts of very animal that becomes food. So that's just on the
animal side. Of course, it's similar on the grain side if you're dealing
with corn or beans or things that you grow out of the ground. That
processing same thing, bringing in the raw materials, the grain, the peanuts,
the wheat, It goes through various stages and then what comes out on
the other end, depending on what it is that you're making, whether it's
flour, whether it's canned products, and so forth. Again, all assembly
line, if you want to think of it, large machines that are moving
very rapidly in an earlier era, it would have been more or less a
belt running along with humans working alongside that belt putting things together. Today it's
more of a rack type mechanism that's moving items, but it's largely electric controlled
solenoids, little DC motors that are turning little hands and paddles that might move
something on or off that assembly line depending on how they were determined in terms
of size or weight. We've got computers that can scan items, can determine
if the quality is proper, looking for metal pieces that might be inside X
ray machines, looking for that. So if you haven't been inside one of
these facilities in a while, it is absolutely amazing. What has this big
transformation that's occurred over the past several decades. And of course there's a lot
more coming as we get more and more computer control, more networked, more
of the analysis that can be done on the food as it's being produced.
So oftentimes for consumers, you go to the grocery store and you buy that
frozen dinner or the can of soup or a loaf of bread, and you
don't often think about everything it took to actually get there. And it is
amazing. And if you like this There are certainly plenty of videos that you
can go watch and get a better feel for what's going on, or go
take a tour. Sometimes they'll let the general public in to take a look
at how this process works. But it is absolutely fascinating. And again,
as I mentioned earlier, this is one of our sixteen critical infrastructures, and
it's probably the one that's been leased overlooked from in terms of what we're doing
with it and security and all the pieces that are behind it to protect that
food system. You know, there was a lot of stuff there that that
Marcus said. Let me chime in. Just I read a statistic the other
day. This is very big business, This is huge industry. I mean,
there's there's eight billion people on the planet. We've got to feed them
all. I saw statistic the other day. You know, pop quiz for
you Nate the you know, then look at the number of species on the
planet. You know, add up all of the humans on the planet,
all of the cows on the planet, all of the pigs on the planet,
all of the vertical rates which vertebrates species weighs the most? How many
mega tons? You know, what has the most mega tons of you know,
living population on the planet today. You want to guess it's good that
you specified vertebrates because I was going to choose ants. Alj probably wins,
Oh, yeah, that's a good point. It's a good point. I
can only sense that this is a trick question, so I'm not going to
say humans. No, it's got to be a farm animal. It turns
out to be chickens. Of all the vertebrates on the planet, the largest
number of megatons of living mass on the planet today is chickens, almost all
of which we're eating. You know, this is very big business, all
right, and that begs the question, how did we get to the point
where chickens so vastly outnumber everything else. I'm not an expert in the field,
but I read a book a couple of years ago. I recommend it.
It's called Enlightenment. Now. I think that was the book. I'm
pretty sure it was. It talked about sort of the grand scale of statistics,
and one of the points it made in the food industry in particular,
was that look in the late eighteen hundreds, coming up on the nineteen hundreds,
experts were making predictions that within a handful of decades, you know,
before the mid nineteen hundreds, you know, we were going to have a
billion and a half people on the planet. You know, at the time,
late eighteen hundreds, we had a little less than a billion people on
the planet. They're predicting a billion and a half by the middle of the
century, most of whom would be starving because you know, in the late
eighteen hundreds, they were turning enormous amounts of land from wilderness into agricultural land
and still were barely keeping up with population growth in terms of food production.
But today we have eight billion people on the planet and we have last I
heard, about twenty five percent more agricultural land in production than we had in
the late eighteen hundreds. So we increased the land used by twenty five percent
and the population by eight times. And today, you know, enlightenment now
argues we have more than enough food for everyone on the planet. The only
reason that people are hungry on the planet today is politics, war, you
know, nasty stuff like ethnic cleansing and so on, where where you know,
food isn't getting to people. But there's you know, there's more than
enough food in the planet. Something like thirty percent, one third of the
farmland in the United States that's used for corn is producing ethanol, so we
can burn it rather than corn to feed to animals or feed to humans.
There's more than enough land. And you know, to me, my sort
of my takeaway from enlightenment now was the biggest innovation in the twentieth century,
The biggest, single, most important invention was not the computer. It was
a cheap way to produce nitrogen fertilizer had to do with increasing the efficiency of
farms. And I assume that there are other factors involved, such as genetically
modified crops. And to the point of this podcast, you know, city
folk like me don't necessarily always understand, but the act, the practice of
farming is so computerized right now. Absolutely. I mean, you know,
back in the late eighteen hundreds, most of the world's population, in my
dim understanding, were farmers. They produced their own food that because you know,
that's what you did, that's how you produced food. Today, you
know, a handful of people in sort of a family or two getting together
can manage you know, five ten, twelve square miles. You know,
thousands, tens of thousands of acres of land in North America can can produce
food on that land, the number of the people involved in producing food has
dropped through the floor. And this is not by accident. It has to
do with with automation. You know, my sister and her husband ran a
dairy farm for twenty five years. You know, he went to school,
he did a four year degree on managing dairy farms. It was a very
modern operation. One of the the insights that I remember, you know,
hearing about sort of one of his policies was look andrew policy for us and
for all of our hired hands, never do anything by hand that a machine
can do for you. Automation is the way this is moving forward. And
you know, in the modern world, all modern automation involves computers, and
of course, putting more computers in everything puts more targets in everything. Hence,
you know the cybersecurity problems that now plague us in almost every industry that's
heavily automated, which is everything I do. Recall it. I took a
tour recently of a brewery and you know what, something you haven't really emphasized
that impressed me was the scale, the scale of you know, the factory,
the brewery. Well, since you mentioned beverages, that That's also another
piece is the heating and the cooling. So in food production, if you're
you know, you have a microbiological side of this where you know, you
can't let it get contaminated, you can't let bacteria grow. So oftentimes the
finished product it's not ready to be put directly in a grocery store. You
got to do something with it. You'll have these very large warehouses with temperatures
that are around freezing and so now that requires an enormous air conditioning system,
which requires large amounts of liquid whatever, ammonia, nitrogen, and so forth.
So there's a whole nother back into this that the size you're talking about,
the scale these big facilities, how do you cool something like that,
and how do you keep those temperatures regulated? That that's another industrial real control
type of area that is often overlooked but is absolutely necessary in food production.
That makes sense. So let me ask you, given the physical process,
given the degree of automation. You know, I'm going to ask in a
moment about what are we worried about? What kind of consequences? But you
know, let me add to start with, I'm guessing the obvious is there
if you shut down one of these large facilities. That's a lot of money
at stake, you know, beyond sort of shutdowns, what are we worried
about? What's the worst that can happen in this system? If the automation
goes awry in sort of the worst case, well, you can think of
it at a couple of levels. So in the growing world, let's say
somebody makes a mistake, there's a computer malfunctions, and we plant things wrong.
You could have a lower yield, so we don't get as much corn
out of that field as we should have, or not as many chickens come
out of the chicken house, et cetera. In the production world, you
could wind up with things that are not that big of a deal, like
the wrong measure. You're expecting sixteen ounces, you get sixteen and a half
or fifteen and a half, you know, that type of thing. But
as I mentioned earlier, there's a biology side to this, and that's probably
the worst side. Is that you get an undetected agent, some pathogen that
leaks into the system and you get contaminated food. And this is what consumers
mostly see as you get the food recalls. I'm sure you're very familiar with
this where it's determined that you know, a truckload of lettuce or whatever got
contaminated along the way, and now the grocery stores have to do product recalls
and consumers have to go check. But I guess the good side of this
is because the way they process and the way they can track, all the
batches are labeled and numbered, and so if you do wind up having some
bacteria, pathogen, some contamination of some sort because a machine failed you or
a trusted computer system failed you, we can at least track down where did
it come from, when was it produced, where did it go to.
Because of all the good tracking that we're doing now we can isolate where that
contaminated food is in the food supply system and go pull it back out.
Previously, we really were not good at doing that. We just kind of
had to guess said, yeah, we think it might have happened on this
day, and yeah, that output probably went to this grocery store chain.
It's not a lot of faith there, and we have a lot more precision
today and trying to determine if something goes wrong, who directly is impacted by
it, and then try to prevent consumption of that contaminated food, so nate
from my limited knowledge of the field, and you know, I've at one
point provided automation software to food manufacturers pharmaceutical manufacturers. One of the critical pieces
of automation in order and food factories, food and beverage is the batch record.
And this is usually a historian that is keeping track of everything that happens
to every batch. When you produce food, canned goods, you know,
peanut butter, whatever. When you produce food, it has been in my
understanding, by law it comes out in batches and the batch number has to
be identified on each each package that you produce, so that if there's a
problem that's found with a batch, you can recall the batch reliably. How
would you find a problem, Well, you know, you discover, I
don't know, the batch is contaminated. You go back to your batch records,
you say, what happened to this batch? You know it used this
input. Well, four other batches us that input. We think that's the
one that was contaminated. These are the four batches that have to be recalled
if you lose track of that information. Because a cyber attack comes in and
I don't know, encrypts your historian, you cannot get that information back.
You cannot sell the batch in modern you know, in the in the developed
world, it's only legal to sell a manufactured batch of goods in these large
plants. I mean, there's exceptions for smaller plants, but in the in
the big in the big iron, you can only sell the goods if you
have complete control of the batch record. If you've lost track of the batch,
you don't know. You know, if you discover a contaminant, if
dispatch was, you're not allowed to sell it. You have to throw it
out. So you know, this is this is sort of a peek into
the world of cyber to come here. Okay, So you know it's great
that those systems are in place. Can we talk about the threat? Who's
who's coming after us in this world of primary production or you know, food
and beverage or I don't know, even pharmaceutical I imagine manufacturing that's a human
consumable as well. Who's coming after us? And and you know what kind
of consequences are we seeing? How credible is the threat? Well, there's
two of them that we are concerned about here. As we've been working with
growers and doing research and even observations over the years. The first one that's
not quite well, yeah, not quite as obvious to most people. Let's
just talk about that unless you're in our world, and that's the threat of
intellectual property theft, either criminal theft or espionage style theft, and we've seen
a massive rise in that. The FBI has been doing some very good investigations
into Chinese theft, where they literally will come in to a field, they'll
dig up seeds, they'll steal plants, and they'll try and reverse engineer the
genetics of things that we're growing. You're probably very familiar with genetically modified products,
so we can get a better yield, a larger ear of corn,
greener leafy things by genetically modifying it. But of course those products are all
patented and protected by the companies that make them. Adversaries in other countries who
don't have the time, the money, the skill to genetically develop these food
sources, and they have large populations that they have to feed and care for,
will happily come over here and steal from us. So you can either
physically steal it, or you can come into the computer systems. If you
can get in and rate it that way, to take that intellectual property back.
So that's a competitive advantage problem. That's not going to poison the food,
that's not going to stop the machine from working, But economically, long
term, you can cause huge damage to the food supply system if you've got
competitors that are stealing information from you and then potentially selling cheaper into the market,
but then selling bad stuff. You're probably very familiar with the problems that
we've had with animal food, like dog food that's had to be recalled coming
out of China, coming out of other countries that don't have the same type
of food inspection that we have, and yet they're producing food cheaper than we
can produce it here because their labor rates are lower. But the way they're
doing that is from having stolen the information from us. The more obvious one,
of course, that people are concerned about is disruptions, and if we
were to ever get into just think national security, think big picture global like
what's happened with Russia and Ukraine, China and potentially Taiwan. Things that are
a foot down in Central and South America. Right now, We in the
United States, Canada, we tend to enjoy the bounty of our land.
We're well fed, we're well provided for. We've built a system that we're
all very comfortable with the rest of the world other than maybe Europe doesn't enjoy
that. And if we were ever to get into a large global conflict,
a World War three. We now have a very high tech, somewhat vulnerable
food supply system that you don't physically have to occupy North America to disrupt it.
You can disrupt it via cyber means, come in through the computer networks,
the trusted systems and potentially mess up our food supply. We have slack
in the system. It's not like you can do a disruption today and everybody
starts starving tomorrow. But depending on the type of food, the amount of
slack varies from a few days to a few weeks, So you could run
out of some products pretty quickly. You could have a few weeks worth on
hand in the system before you'd start running out. Our adversaries know that,
absolutely know that. So that threat of disruption, I think is something a
lot of people focus on. But the threat of intellectual property theft, that
hidden threat is something that we're not focusing on as much as we should,
or at least it's not as well known to those in the growing community.
They're producing community and so forth. That's a fair amount to worry about,
from intellectual property theft to sort of global conflicts. How are we doing,
what's the state of the press in this broad collection of industries. Let me
say, I think we're doing better than we've were. There has certainly been
a large amount of awareness that's been growing. One of the bad things that's
happening to everybody is ransomware. I'm sure you're very familiar with it. And
food industry companies can be victims of ransomware just like anybody else. And even
though we don't like ransomware, we'd like it to go away, the fact
that it exists has raised the awareness that we've got a problem here, that
there are ways to be disruptive via computer systems, connectivities to the Internet,
and so forth. And so that awareness has gone up because of the ransomware
problem. That and in a sense is good because it's caused the leadership of
companies, boards, CEOs, others beyond the security community to become aware that
this is an issue, and then they start asking the correct questions. There's
companies I've worked with that really didn't think about security, really didn't think that
they would ever be targeted. They're just happy to run their machines and plants
and whatever they're producing until they get hit with a ransomware attack and it shuts
them down. And then all of a sudden, as they come out on
the other side, they've got a brand new attitude and are investing heavily in
the security of the company, and not so much the financial business side,
but the actual production side, the side that would not have necessarily been impacted
by the ransomware, but they're very aware that, yeah, that could be
next, and we are certainly on somebody's radar somewhere, And so this,
in a weird sort of way, has helped to raise the awareness of the
problem that's out there. There's also the other good news is if you go
to various events where food safety officials are getting together, or if you read
Food Safety magazine or any of the other publications around there, over the past
few years, there's been a steady rise in the discussion of this problem.
So again the awareness is going up. Major problem though, I'm still seeing
is a lot of businesses look at security or cybersecurity as being an it problem.
It's not a safety problem, it's not a biology problem, it's not
a chemistry problems. Well that's it. You go talk to those fellows install
your PowerPoint and your email, and they need to worry about it, and
we've got to change that attitude. I'm afraid that's not just in the food
world, but probably in other manufacturing areas as well. Is that this type
of security, from disruptions to espionage to whatever, is really everybody's problem,
and from the low level employee all the way to the CEO and the board,
there has to be awareness, has to be engagement. It's very much
like environmental problems. We don't let just the environmental engineers worry about environmental harm.
We're getting everybody engaged and trying to work towards protecting the planet, protecting
the world around us. Safety works like this, where everybody is responsible for
safety, not just the safety engineer, and I think that's where we're going
with security, though we still have a long way to go. And of
course, culturally within the food world, there hasn't been a lot of exposure
because the computer systems are fairly new coming along, connectivity is fairly new.
This precision agriculture. Precision farming is fairly new as compared to say, communications
or power hospitals, you know, other critical infrastructure areas. So again,
I think the news is good that we're making progress, but there's still a
long way to go here before we're going to be as protected as some of
the other sectors. That makes sense to a degree. I mean, in
my own experience, the awareness of cybersecurity threats tends to be greatest in the
largest enterprises. And you know, you've talked about baker farms, you haven't
said million acre farms. Are we still dealing primarily with smaller businesses in this
industry where you know it's you know, they may not have an IT person
on staff, much less a cybersecurity person. In America North America, the
small farm tends to rule. There's still the family farmer or the smaller business,
the you know, thousand to a few thousand acre those big large things
we mentioned the millionacre farms, that would be more of the large, open
pasture raising types of things. I don't know if anybody that has an intact
million acres where they're growing wheat, that certainly could happen. But I think
that you're touching on a very interesting question, and that is at the growing
level, it is largely small businesses. And yes, there are some large
businesses that grow things, but the small to medium sized business is the backbone
here in North America. Those businesses don't have the funding, the knowledge,
the background to do security. And even as I've talked to many of them
that the influx of it their heart. They're highly dependent upon the manufacturer,
upon the vendor. You might put in a new system and everything is remotely
controlled by the vendor and the person on the ground is just making sure that
the power is on and they got connectivity. A great example of this is,
and this goes back to what we're talking about earlier, with tractors.
If I own a John Deere tractor, then that's just you know, a
popular brand. There are many other ones that's connected Most farmers instinctively, No,
I've got to have a satellite connection. I have to have, you
know, some way for this machine to communicate. But a couple of years
ago, if you recall when Russia went into Ukraine, the Ukrainians were using
their John Deere tractors to haul carcasses of Russian tank back over to the Ukrainian
side. Russians didn't like that. They hauled a bunch of Ukrainian tractors back
over to Russia. And when John Deere heard about this, they were able
to remotely turn off those tractors because they were the brand new, highly connected
ones. Well, you know, everybody applauds and cheers, Wow, this
is pretty cool. We can use technology to win the war. But it
didn't take too long for somebody to ask the two questions. One is,
well, if John Deere could do that to tractors hauled back to Russia,
can they also turn off a tractor here in the United States remotely? And
even worse, could somebody like Russia hack into John Deere and turn off the
tractors here the United States? Absolutely fair questions, and unfortunately the answer is
probably not the answer you want to hear, and it's yes to both.
And this couple of years ago began to raise a lot of questions from these
growers, the family small business types who feel like this has gotten a little
bit out of control. I've got this million dollar piece of machinery out here,
this big combine or tractor or whatever, and you're telling me that somebody
else can turn it off, somebody else can take control of it. That
may be a little more than what I had bargained for. So again,
the awareness is growing. Here at Auburn. We're a land grant university.
Every state has land grant schools. This goes back to the post Civil War
era. A lot of good history there. But what that means is we
have a mission to educate across the state, particularly in agricultural and mechanical areas,
because think about post Civil War, this was a transition from an agricultural
economy to an industrial economy, and we wanted to grow young engineers as well
as agricultural specialists to really get things going. And so these land grant schools
have extension services across the state. This is federally funded, state funded,
and really good outreach into the growing farming community. This is in all states
here in the United States, and I'm sure Canada has a similar type of
program. But through that extension system there's a lot of trust and one of
the areas we're looking at is, okay, can we use that to start
educating growers on all these new cybersecurity issues, all the new threats, the
new vulnerabilities. If you're going to build a mesh network across your farm,
and you're going to fully automate all of your machinery and put all this new
IT stuff in there. Do you understand the security side of it? And
we need to bring that in and you push into that rural broadband. You
know, there's a big growing effort to try and add fiber optic as far
and wide as we can, much like rural electrification was over one hundred years
ago. And as we bring small communities, small farms, others that previously
had not been connected to high speed internet, we now connect them up to
fiber optic. What are we doing to protect them and to protect those farms
and those businesses, those communities that had not really thought about the global threat
that you get from high speed connectivity. So this all plays into this whole
food and agg thing where those of us who live in cities, we kind
of understand how to protect ourselves for very street smart. Those that live in
the rural areas. They have a different mindset when it comes to personal protection,
but often it's local and you don't think about the big global threats that
could be brought in through all this connectivity. So there's a lot of work
here to be done. Again, there's a lot of good news in terms
of awareness, but I think we've still got a long way to go before
we get that entire food chain, everything from growing to the packaging to putting
it right in front of you on a plate in your favorite steakhouse, and
everything in between, to bring them up to the same level of security as
we have in other critical infrastructures. Marcus's point there about Ukraine and Russia and
the John Deere tractors is really interesting to me. Last year I had the
privilege of interviewing a gentleman who goes by sit codes. He's an independent hacker
who at Defcon demonstrated how he hacked into John Deere tractors and got them to
play doom on their little dashboard. And what he was explaining to me is
that, you know, ultimately these machines are made with the kinds of parts
that we are used to. There is a cyber attack surface that is as
well and good as any other machine you're talking about. And what does it
mean if a hacker were to break in, either locally or remotely and start
affecting the food supply. That's a good question and you know, to me,
yeah, there's there's a local risk. If you're standing there with a
you know, I don't know, your cell phone close to physically close to
the tractor and you're hacking into it, that's sort of a local threat that
you know, the farmers are likely to understand, who are you, what's
happening to my tractor? Stop this, get out with his you know,
his stick, and chase the off the property. To me, the scary
scenario is the cloud scenario, where in theory, you could hack into the
Jahn Deere cloud control system and send stop commands to every tractor on the continent.
This is not unique to the agricultural sector. There's lots of other sectors
that are talking about this cloud problem. You know. The one I'm most
familiar with is power generation. A lot of power generation is done with turbines,
steam drives, steam turbines, natural gas, you know, are gas
turbines are basically stationary jet engines. Water turbines are are in big hydroelectric dams.
These things are moving parts. They wear out. Friction is the enemy
of moving parts, and so the turbine vendors by and large are monitoring remotely
these turbines constantly, And the question is what happens if one of these vendors
remote monitoring sites is compromised and you can start sending instructions to cause damage to
the turbines. You could turn the lights out for a large part of the
nation by crippling these these turbines remotely. This is you know, lots of
lots of industries are talking about this problem, and you know, I take
your point. That's a very interesting thing that you bring up. But the
question that I'm more confused about is, you know, I understand why some
centralized entity in whether it be power generation or tractors would want them. It
would need to push software updates for example, but why Andrew do they need
that extra layer of control? Right? Like Microsoft sends me software updates for
my PC, but Microsoft doesn't control my PC. I know that, at
least with tractors, this is an issue for some farmers when it comes to,
for example, right of repair. Yeah, there's a there's a couple
of issues there. Let me touch on right of repair first. I'm thinking
back. There was an episode on the CANbus that we did a few months
ago. Doctor Ken Tindall was explaining how people hack into the CANbus in automobiles
and steal you know, expensive cars. You know, right of repair is
tied up in managing keys. The way to prevent people hacking into and stealing
cars or tractors in this example is to encrypt communications between automation within the tractor
and the vendor. Is the natural place to manage those keys. If you
pull in a random part from another sort of repair supplier, can't You can't
make it talk to the rest of the car without consulting the vendor and getting
the encryption key for that tractor. So that's an issue that I'm not sure
has been solved. More generally, lots of different industries are worried about connections
to the cloud, and you're connecting to the cloud, I mean the killer
app for the cloud right now is predictive maintenance. Whoever's created the machinery,
be it the tractor or the steam turbine or whatever, you know, the
stamping machine for refrigerators, whoever's created the machinery is the world's expert generally on
how it's supposed to work and diagnosing problems with it when something goes wrong.
And so most of these vendors are offering services where you know, in the
cloud, you're continuously monitoring these complex devices and from time to time issuing instructions
back to the devices, saying, change your mode of operation just a little
bit, because this piece of you is wearing out. We want to extend
your service life to the maximum. We want to you know, minimize service
costs. This is what predictive maintenance means. You're predicting what's going to be
needed maintenance wise, and you're adjusting the equipment to you know, to make
to make changes to make the equipment more efficient and and you know, require
less maintenance. So this kind of cloud connection a lot of people are worried
about. There are some solutions out there for sort of the really big iron
you know, uh, steam turbines, you know, people use uni directional
gateways and they do the corrections over the phone. For the smaller stuff,
you know, there aren't good solutions. You know, people are figuring this
out right now. In the long run, it looks like we have we
have nation state threats, we have you know, worries about you know,
food chain stability in terms in times of conflict, uh, coupled with you
know, in the today's world, smaller producers and a whole mix of of
you know, small and large manufacturers in the space in your mind. You
know, is there a solution here? Is there? You know these you've
described these as as critical infrastructures. I mean, should we be protecting every
farm out there as thoroughly as we protect a high speed passenger rail switching system
where you know, worst case consequence of compromises, mass casualty events. What
is the end goal here? What are you shooting for? I think these
are fair questions. And if you take a look at all of the infrastructures,
some of the analysis that we've been try and tease apart, what does
it mean? What are the different parts? There are some that are in
a time world. They're very time sensitive. So the power grid, for
example, is a good one. If it goes down within milliseconds, everybody's
impacted. The communications networks are like that. Hot hospitals, banks, maybe
a little less time or a little more time, maybe measured in minutes to
hours. Food system though that could be weeks or months before you actually see
an impact if there's some sort of an attack on a food system. So
when it comes to should I pay more attention to a transportation system like trains
or planes or something versus food. From a time sequencing, I would certainly
say more attention should be paid to the transportation side because the impact can be
fairly quick, whereas you have time working on your favor when it comes to
food. But all that being said, the unfortunate side of this is food
has largely been ignored from a safety perspective or a security perspective. Food safety,
if you put those words together, has largely been a biology or chemistry
type of conversation, and rightly so, because we want to make sure that
the food that we eat or that our pets consume or whatever is absolutely safe
and free of any toxins or any pathogens or anything that might harm or kill
us or cause cancer or all the other things. We absolutely need to focus
on that. But the disruptive side, or the spiona economics side, largely
has not been a focus. And I think now as we begin to understand
the long term impacts of what could happen inside this industry, yes we do
need to pay more attention to it. Who should pay attention is probably the
next question. Should this be a regulatory matter, should it be a business
matter? I think the jury's out on that. Many people are not huge
fans of additional regulation because that increases costs, that causes frustrations. On the
other hand, if businesses can't solve it themselves, then that's not a bad
use of government, you know, to come in and say, Okay,
if you can't, if the economics of it won't fix it, then maybe
a regulatory model would obviously beyond the scope of what we're talking about here,
but certainly a lot of room for the debate there in terms of how to
do that. But a third approach, and this has worked in lots of
places, and it's kind of an old cliche thing, and that has to
do with information sharing and in some sectors the flow of information between competing companies
is very normal when it comes to security, and so if you get a
company that gets data, breach, ransomware, whatever happens to them, they
will happily share the technical information with their competitors because both competitors are expected to
share that. That has nothing to do with competition. That has everything to
do with protecting that industry and making sure that they're safe and secure. In
the food world, Unfortunately, the culture is so resistant to any threat of
antitrust or anything that might even look like an anti competitive thing. That sharing
threat information has kind of fallen into that bucket, and it's a little frustrating
because information sharing about cyber threats is not done at the same level as information
sharing about biological threats. So if you have a salmonella issue, they'll share
that information all day long because they don't want that to impact But if you
have a cyber incident, a breach, or something else, trying to share
that information between competing companies is very hard to do. The lawyers don't want
you to do that. So I think that's that's an area we've got to
figure out how to fix it, and how do we allow these companies that
compete with each other in the food sector to encourage them and enable them to
be able to share threat data cyber threat data or other types of threat data
as readily as they're able to share biological threat data. And that'll help a
lot too. That that third area, that information sharing piece, it suggests
to me, you know, are you saying that in the agriculture and food
processing businesses, maybe what we need is a greater capability in terms of resilience
than necessarily in terms of absolute prevention. For example, if there's you know,
a refrigeration failure at a warehouse and we have to throw out the contents
of the warehouse, then do so and we're back. Or you know,
if you know, the the the combine or the tractor vendor is hacked by
a nation state and all of the tractors are bricked, that there's a way,
you know, possibly by regulation, by law saying there has to be
a way to turn these vehicles on again. Manufacturers make it happen so that
you can turn a physical key, or you can press a physical button,
or you can do something to reactivate the equipment. Is is this what I'm
hearing? So the question you're asking then is is there some way we can
anticipate failure and engineers a solution that either might prevent the failure that'd be great
or at least reduce the consequences of that failure. And what's your describing as
an initiative that's sort of new, been around a couple three years beginning to
gain some good traction called cyber informed engineering. The general idea being is that
if we're going to engineer something let's say we're going to build a new tractor,
We're going to build a new air conditioning system, whatever it is.
We anticipate a failure mode that could be caused by a trust problem. A
computer system that is no longer trustworthy. It's misbehaving. It either died or
it's producing bad information, or whatever it's controlling isn't being controlled properly. So
we want to engineer something that can then tolerate that loss of trust and can
survive it. Either have a good shutdown, you know, a proper control
shutdown, or you've got to build in workarounds. So your tractor example is
a perfect one like that, where I can anticipate that potentially my loss of
trust through the computer system with it might brick that tractor. I should be
able to reach under the hood, hit a little magic button, and now
it goes into completely manual mode. It's not dependent on the computer anymore,
and I can still fire up that tractor and run it. Yes, it's
less efficient, as it might cost more in terms of hours, but at
least it works and it runs. But I've deliberately engineered that into that tractor.
That's a great example of cyber informed engineering, or you can take it
into production where I'm going to depend on some cloud based service that's providing me
with AI enabled information, so my production is so much faster. But as
somebody attacks me, if I lose my network connection, if I lose trust
on my databases, I can still produce food. I just can't necessarily produce
it as efficiently as I could before, but I can at least continue production.
That's cyber informed engineering, and I think that's kind of a neat approach
here. I understand you've had guests in the past that they've spoken to this,
because what we're looking at is can we can we take the brains of
the engineers who are thinking about all sorts of different activities that are going on
with their engineered world that from the environment to humans, to others that they're
interacting with, and engineer systems that are resilient to these types of threats.
Well, we're now adding in cybers so we're not asking the engineers to become
cybersecurity experts. It's more of can you use your engineering skills to build an
engineered system that can survive a cyber attack or at least reduce the consequences of
a cyber attack down to some manageable level. So I'm glad you brought that
up because that is a great way to introduce that concept of CIE into the
food and at sector and how would it actually apply there? That makes sense.
Something else you mentioned a couple of minutes ago, you talked about lack
of information sharing. I mean, is there no you know, worldwide?
Is there no jurisdiction with a food and beverage or an agricultural iceac As far
as I know that, there's not a robust one. So if you look
at other sectors like the electricity I SACK or the financial services I sack,
healthcare is sack, they are very robust global sometimes in nature. There was
an attempt to create a food and egg I sack a number of years ago.
It lasted for a few years. They could not My understanding is they
couldn't attract enough companies that wanted to belong to it. And if you recall
what I said about the reluctance to share because of the fear of lawsuits,
antitrust lawsuits largely has derailed that. But what you wind up with is there's
still a desire to share, but there's a reluctance to share. And so,
as you mentioned earlier, there's this large companies, your well known brand
names that you see in the grocery stores. They probably are financially okay where
they can protect themselves. But below them the producers and growers of which most
people have never heard of because there's so many of them, the small how
businesses, the farmers and others. They don't have that luxury. And that's
probably where an ICEACK would have the most impact would be with those small communities,
the small farms, the small businesses because they just don't have their own
security team. And maybe through the extension system, maybe through some sort of
agricultural outreach Department of Agriculture, that could be done. There is an initiative
that occurred last year through the it I sack to create they had a special
interest group. Some of their members who belonged to the food sector created a
special interest group for that. They have labeled themselves as an Agricultural I sack,
but it's not the same as one that would be very inclusive of all
the small businesses and others. So it's still populated largely by large businesses,
but they're still hampered by the legal challenges of trying to share. So yeah,
I think that would make a big difference if we could do that.
The challenge is how do you do it and who pays for it? Because
again, if you want to bring in small businesses or medium sized businesses,
if they don't have the budget to have an IT staff or security staff,
would they have a budget to belong to an I sack? And what would
be the right membership rate or is this something that state government should just pay
for or agricultural grants much like we do with food stamps and other types of
nutrition programs. Maybe that's a way to finance it. So this is still
a great area of public policy to talk about. I don't think we have
a firm answer yet, but we do need to think very strongly about how
do we increase the amount of information sharing, the threat intelligence sharing, the
analysis and so forth that goes on, and just briefly you know, of
course, what we're doing at Auburn. One of the proposals has been to
create a consortium. Find other land grant schools across the States. Let's build
a consortium, and maybe that consortium together could become an I sack for small
farmers. Students in our universities could become the staff, become the analysts.
So they learned in their junior and senior years how to handle threats. They
learn about what's going on these earlier topics we were talking about, from nation
states to terrorists, to others that might impact the business they're going to work
for. So then when they graduate with their agricultural degree or engineering degree and
go off to their initial jobs, not only are they BookSmart on that area
that they study, but they're also well aware of the threats to their industry
and can start bringing in that level of knowledge. So that's an approach we've
been talking about that might work. Just we're open to trying lots of different
ideas because clearly the way we're doing it today is not working, and so
we've got to come up with some other approaches. That was a long answer.
Let me paraphrase just a minute. You know, in a traditional is
sac and I've sat in on some of these traditional is sacks, they have
weekly phone calls are maybe half an hour long. In every one of these
phone calls, one of the participating businesses, big oil companies, big manufacturers,
big power companies, you know, take the microphone and walk the rest
of the listeners through an attack scenario that they've observed recently or attacks that they've
defeated recently. And the bottom line is what's called actionable intel. Namely,
there are intrusion detection signatures that these people are providing. There's IP addresses that
you know, shouldn't be trusted anymore that these people are providing. They're providing
very technical information that the consumers. The people who are listening on the call,
are taking this information and putting it into their intrusion detection systems and putting
it into their SEMs so that if they detect this kind of activity in the
future, they know that it's malicious and they can activate their incident response teams.
So here's the problem with you know, small farms. We're talking you
know, one or two families operating you know, a handful of square miles
of farmland. There might be three people, there might be five people with
a couple of hired hands operating this this farm. Are they going to get
on a phone call once a week for a half hour or an hour listening
to IP addresses and you know, signatures and checksums. They don't have an
intrusion detection system. They don't have an incident response system. They need a
different kind of information, and you know what kind of information is that.
I don't know, but this is the work that Marcus is doing at you
know, at at the university there, so you know, it's it's it's
a good thing he is doing the work. I look forward to seeing as
results. Well, Mark, this has been tremendous. It's you know,
this is a field an industry that I know very little about, and I'm
grateful for the introduction. Thank you. Before we let you go, can
you sum up for us what should our listeners take away from this problem from
these solutions from this space. Thank you, Andrew, and I really do
appreciate you letting me take the time today to talk to you and talk to
your listeners about this. Probably the key thing, of course, is to
recognize that in these critical infrastructure sectors, they're all different, but they're all
interdependent. The food and eggs sector oftentimes is overlooked because it just works.
We have an abundance of food, we're not starving, but it, like
any other critical sector, is dependent on other sectors, and it's dependent more
and more now on the connected systems and the IT infrastructure and the internet and
cloud and AI and all these neat things that other sectors have embraced over the
decades is now being embraced by food and Egg. And I want to understand
that with all this new technology, while we're bringing in new efficiencies, we're
bringing in new vulnerabilities, new threats, potential consequences we've never thought of before.
And these are areas because it's changing so rapidly, we're challenged with how
best do you address it? And the solutions we've come up with with the
other critical infrastructure sectors may not work with food and agg We may need to
come up with different ways of delivering the message, different ways of handling these
threats, different ways of working with our regulators and working with the government,
the private sector, small medium, large businesses, and even working globally.
And how do you cooperate with Mexico and cooperate with Europe and China and so
forth. These are all areas of interest, areas of research here at Auburn
and other schools. This is great material for undergrads and grad students to dig
in if they've got to write a paper, if they need to work on
a degree program. These are perfect research areas. Otherwise we wind up just
making it up. We don't want to do that, and I think that's
one of the big benefits that the universities can bring in is that we can
do this type of research and we can come up with some pretty good proposals,
almost like a think tank might do it, but it's being done with
young minds. And the benefit, of course is those students upon graduation,
they're taking that knowledge with them right into industry and you know, helping industry
understand what these threats are because of what they've learned. One of the common
complaints we get from a lot of companies with and I'm sure every college professor's
heard this, is you know, you're turning out book smart students, but
they don't know anything about my industry and we have to start from scratch to
teach them. Well, maybe this is an opportunity here where we start cranking
out some book smart students, but they're also well aware of the threats,
vulnerabilities, consequences of all this new connectivity and all this new precision stuff that
we're beginning that we're bringing in and again focusing on food and egg, not
to leave out the other sectors, but that's where the focus is here.
So I think that would kind of be the big wrapper and where we might
want to go, you know, take take a look at what we're doing
here at Auburn, take a look at the McQuary Institute. We're fairly easy
to find online. And if you're interested in this, if it sounds fun,
contact us. Well. We would love to build out a consortium.
We'd love to get more engagement, build bigger partnerships. There's no way that
any one organization can own this problem. It has to be addressed and worked
on by multiple organizations and institutes and people that all would like to collaborate for
the common good. And there's a lot of room, you know, using
the farm analogy, it's a very big pasture. There's a lot of room
to spread out, so we don't all have to eat the same grass,
if that makes sense. So I think that's kind of what I'd like to
leave with. And if there's any other questions, Andrew open to answer those,
But thank thanks again for allowing me to have the time with you today.
And that just about does it for your interview with Marcus Andrew Yet another
episode where we talked about an industry we've somehow not managed to talk about in
all these years podcasting and presents unique and interesting new challenges to me. The
unique challenge here is, especially on the primary production side of the farms,
there's so many small operators. I mean, you know, in the electric
power grid, something like ninety percent of the world's power is produced by ten
percent of the world's power plants. We're talking about very big installations and the
small ones, in a sense, are noise here. Most of the world's
food seems to be produced by the smaller operators. And you know, those
operators are not just worried about you know, John Deere or some other tractor
vendor or equipment vendor shutting him down. You know, they are dependent on
fuel coming in on a regular basis so they can run the tractors. They're
dependent on electric power, They're dependent on communications facilities with satellites, with the
internet, with their with their cloud providers, and you know, so the
small operator is sort of a unique challenge here. The other thing I took
from the interview is that you know, I have to wonder if time is
not the key, because if a tractor is crippled for twenty four hours,
probably nobody much will notice. Everybody will be annoyed, but it's not even
going to impact the bottom line. If it's if attractor is crippled for a
week, we have the beginnings of a problem. If it's crippled for you
know, two months in planting season, this is a serious problem, especially
if it happens to a lot of tractors. So you know, time is
the key. If we can invent mechanisms so that if there's a cyber problem,
they can you know, that problem can be fixed or worked around promptly,
so that we can operate you know, the affected systems machinery in food
processing plants or tractors or whatever. We can operate these systems, maybe in
a degraded mode, maybe only you know, get ninety five percent of the
efficiency benefits out of it that we thought we were going to get. Then,
you know, then we've got a way forward. This seems absolutely doable.
But you know, as Marcus was saying, it hasn't yet been done.
There aren't those solutions and systems and knowledge in place. We need to
invent them. So, you know, good on Auburn University and you know
the other folks collaborating with them. Good on them. You know, let's
let's solve this problem. It seems eminently solvable. And with that, thank
you to Marcus Socks for this illuminating interview. And Andrew is always thank you
for speaking with me. It's always a pleasure. Thank you, Nan.
This has been the Industrial Security podcast from Waterfall. Thanks to everyone out there
who's listening. Seven days produced Davys spat