Speaker 1: How'd you like to listen to dot net rocks with
Speaker 1: no ads? Easy? Become a patron for just five dollars
Speaker 1: a month. You get access to a private RSS feed
Speaker 1: where all the shows have no ads. Twenty dollars a month,
Speaker 1: we'll get you that and a special dot net Rocks
Speaker 1: patron mug. Sign up now at Patreon dot dot NetRocks
Speaker 1: dot com. Hello and welcome back to dot net rocks.
Speaker 1: I'm Carl Franklin and I'm Richard Campbell, and we're here
Speaker 1: again for your dot net and all things, you know,
Speaker 1: geek pleasure. I think so, Uh, Richard, how's it going
Speaker 1: over there in Vancouver?
Speaker 2: Well, if we get our time shifting right and nothing
Speaker 2: weird happens and the creek don't rise, I should have
Speaker 2: published episode one thousand of run As Radio yesterday.
Speaker 1: Congratulations, thanks man, big milestone a lot of shows. Yeah
Speaker 1: it is. And do you want to talk about dev intersection?
Speaker 1: Can we talk about that now? Yeah?
Speaker 2: Yeah, we can talk about dev intersection and you know,
Speaker 2: all all the things come together well, and we're calling
Speaker 2: it the we're sort of bouncing between the names because
Speaker 2: there's a lot of emphasis on cloud development, so sort
Speaker 2: of Azure community development you know stuff as well as
Speaker 2: there's two other adjacent shows attached to it. But if
Speaker 2: you know dev inter Section, you know what to expect.
Speaker 2: The usual troublemakers including you and me, but also the
Speaker 2: next Gen AI show, so lots of co piloting and
Speaker 2: lots of real jenitor of AI, you know, smart folks,
Speaker 2: the science people so forth, building cool things. And our
Speaker 2: friend Michelle Busta Monte has started up a cybersecurity conference
Speaker 2: adjacent to the other two. So yeah, three shows in one.
Speaker 2: I got news about that one. Yeah, I'm doing two
Speaker 2: talks at Michelle's security conference. Oh wow.
Speaker 1: One of them is secure your Blazer Server Applications nice important,
Speaker 1: and the other one is a live security this week.
Speaker 1: Oh cool, I'm gonna do it on stage Channel show. Yeah,
Speaker 1: we don't actually have the time yet, but we think
Speaker 1: it might be in the evening, you know sometime.
Speaker 2: Yeah, it's a logical time to do it, right. Well,
Speaker 2: if you working your butt off that week is aren't
Speaker 2: you doing a session with Maddie as well on Yes
Speaker 2: Purifying Never Hanks.
Speaker 1: I'm really going to town this week. Crazy, I'm not
Speaker 1: only doing a Blazer workshop one day, but yeah, Maddie
Speaker 1: Montaguella and I and Jeff Fritz will be there too,
Speaker 1: are going to aspirify the dot NetRocks dot com website.
Speaker 2: Cool live in front.
Speaker 1: Of your face.
Speaker 2: There's nothing better than a Brownfield action man like actually
Speaker 2: taking existing code and applying new standards to it and
Speaker 2: not having a catch fire.
Speaker 1: And honest to god, like, I don't really know what
Speaker 1: to expect because this isn't something that's in my wheelhouse,
Speaker 1: you know, So.
Speaker 2: It's going to be a no, you're on the ride.
Speaker 1: Yeah, I'm on the ride. It's going to be a
Speaker 1: learning experience for me and hopefully, you know, the it
Speaker 1: will show.
Speaker 2: Learn experience for everybody who's there.
Speaker 1: It will show in the results. And then a couple
Speaker 1: of other talks to Yeah, I'm going to be working
Speaker 1: working hard. All right, let's start off with better no framework,
Speaker 1: roll the music, all right? What do you got? Do
Speaker 1: you know what lotty animations are?
Speaker 2: No?
Speaker 1: I didn't either until today, well until I found this, Yeah,
Speaker 1: lottifiles dot com. These are free, lightweight animations, but I
Speaker 1: guess they require a player, right, and so they look
Speaker 1: like animated gifts to me, and I don't really understand
Speaker 1: the difference yet. But However, I did find this Lotty
Speaker 1: player which is in a part of the mud Blazer
Speaker 1: extra package. So it's a Blazer Lotti player that makes
Speaker 1: it easy to integrate high quality Lotty animations into your
Speaker 1: Blazer applications. And it's a simple new get package and
Speaker 1: it's you know, very easy to do. I just don't
Speaker 1: know enough about Lottie to really care about it, but
Speaker 1: somebody does, because this is, you know, this is kind
Speaker 1: of an up and coming thing that's interesting. Yeah.
Speaker 2: Cool, another format.
Speaker 1: Another format, Yeah, exactly. So that's what I got today.
Speaker 1: Know it, learn it, love it. Who's talking to us?
Speaker 2: Richard grabbed a commenta of a show nineteen fifty five
Speaker 2: which you did at Build not that long ago, of course,
Speaker 2: published in June, somewhat after the show, where we talked
Speaker 2: to our friend Nicole Forrestkrenn, who hadn't seen in a while,
Speaker 2: one of the you know, Devop's goddesses. She ran Dora
Speaker 2: for the longest time and really was the person, I think,
Speaker 2: more than anyone, who put the numbers to what it
Speaker 2: meant to be a high performing team, like what the
Speaker 2: difference was when you organized into rapid iterating ways, integrated
Speaker 2: to and so Forth said, here's the performance different differences
Speaker 2: that these teams get across the large numbers of organizations.
Speaker 2: So she kind of took the you know, anecdotal side
Speaker 2: of DevOps away and gave it a real science rigger.
Speaker 2: And we've talked about this concept of frictionless development, about
Speaker 2: what happens when you can code quickly and have this
Speaker 2: infrastructure around you to know what's going on, and that
Speaker 2: included using the contemporary AI technologies, although that was you know,
Speaker 2: in May, and here we are in August September, where
Speaker 2: of course things have.
Speaker 1: Changed again light years away.
Speaker 2: Yeah, it keeps moving so fast. And so this comment
Speaker 2: comes from Neil Tiberwalla who says, thanks for the great episode.
Speaker 2: On one of the projects I'm working on, we are
Speaker 2: also struggling to see how llms fit into DevOps. However,
Speaker 2: are surprised to hear you all talking about multiple agents
Speaker 2: merge conflicts. It seems like this is really a non
Speaker 2: issue since the AI could just delete their work and
Speaker 2: start over and redo the whole task. It's just some
Speaker 2: computation not but otherwise not lost human programmer time. That
Speaker 2: is the motivation for a merged process. Yeah, there's also
Speaker 2: a cost in tokens. But I'm with you. Yeah, And
Speaker 2: as you said on the show, a new paradigm requires
Speaker 2: new ways of thinking. Definitely, software che all that is
Speaker 2: changing again, and I'm not unhappy about it. It's interesting
Speaker 2: to talk to folks and to experience all the different approaches.
Speaker 2: So Neil, thank you so much for your comment. And
Speaker 2: a copy of music Coby is on its way to you.
Speaker 2: And if you'd like a copy of music Koba, I
Speaker 2: write a comment on the website at don Arocks dot
Speaker 2: com or on the facebooks. We publish every show there,
Speaker 2: and if you comment there and I read it on
Speaker 2: the show, I'll send you a copy of music Goba.
Speaker 1: Music to code by I remember that, Yeah, that thing
Speaker 1: I started a long time ago to help you focus
Speaker 1: during development. Well now there's twenty two tracks, and if
Speaker 1: you don't want to send in a comment and get
Speaker 1: a free copy of it, you can go to music
Speaker 1: too coode by dot net and you can download the
Speaker 1: entire collection in MP three, flack or wave format.
Speaker 2: Nice.
Speaker 1: All right, So this being showed nineteen one hundred and sixty,
Speaker 1: let's talk about what happened in nineteen sixty six. There's
Speaker 1: you know this is one of those years, right that
Speaker 1: people remember space exploration, Soviet Lunar nine spacecraft, and I'm
Speaker 1: sure we'll talk more about that. The nineteen sixty six
Speaker 1: FIFA World Cup held in England. The host nation won
Speaker 1: its first championship by defeating West Germany in the final.
Speaker 1: The Cultural Revolution in China Mao Jeidong right, so significant
Speaker 1: social and political upheaval. The Vietnam War. The US announced
Speaker 1: a substantial increase in troops in Vietnam. Civil rights Sammy Young,
Speaker 1: junior civil rights activist, was murdered in Alabama, highlighting the
Speaker 1: ongoing struggle for civil rights. The New York City Transits Strike,
Speaker 1: a wildcat strike by public transportation workers, began January second
Speaker 1: and lasted until January thirteenth. But my favorite category, music
Speaker 1: and culture. The Beach Boys Pet Sounds was released, which
Speaker 1: totally reshaped pop music. John Lennon sparked outrage with his
Speaker 1: comment that the band was more popular than Jesus.
Speaker 2: Yeah not smart.
Speaker 1: I shouldn't have said it, and I'm sorry. And The
Speaker 1: flint Stones aired its series finale on April first, And
Speaker 1: there's of course a lot more of it. What do
Speaker 1: you want to tell us about technology?
Speaker 2: In space Richard, You know, ninety six six is not
Speaker 2: a huge here on the space side, just because it's
Speaker 2: the end of the Gemini program and just before the
Speaker 2: Apollo program really ramps up. Although there's lots of Cold
Speaker 2: War stuff going on, lots of reconnaissance satellites and so forth.
Speaker 2: But I would point out that Surveyor won launches and
Speaker 2: lands on the Moon successfully for the first time, one
Speaker 2: of the very first soft landings on the Moon ever.
Speaker 1: Wow.
Speaker 2: And also they start doing mapping of the Moon with
Speaker 2: the prospect for landing with an orbit called the lou
Speaker 2: Or orbiter. And it's a time before CCDs and so forth,
Speaker 2: so taking pictures and sending them back is not a
Speaker 2: simple thing in nineteen sixty six, So yeah, it's not
Speaker 2: the most exciting year sings. They'll get hairy from here
Speaker 2: as the Apollo program really gets rocking along. But at
Speaker 2: this but on that year they still did a few
Speaker 2: important things and it wouldn't be You're not wrong that
Speaker 2: the Soviets had the Luna twelve mission, which also took
Speaker 2: photos of the Moon, although admittedly after the Americans got them.
Speaker 2: But let's talk about important stuff, like nineteen sixty six
Speaker 2: is the year the cool whip is invented, all right,
Speaker 2: nothing like an edible oil product.
Speaker 1: Yeah, edible oil. Cardiovascular dream come true.
Speaker 2: Listen, palm oil is well, let's be clear, not your friend.
Speaker 1: Not your friend.
Speaker 2: On the computer side, Helid Packard's very first computer, the
Speaker 2: HP twenty one sixteen A, which is a funny number
Speaker 2: for your first one. It was actually an acquired computer.
Speaker 2: They bought a comp called Data Systems who made a
Speaker 2: thing called the DSi one thousand, although they had never shipped.
Speaker 2: They bought the company back in sixty four finished it up.
Speaker 2: It was all integrated circuits, which is very novel of
Speaker 2: the time, and magnetic core memory because they're still a
Speaker 2: couple of years away from digital ram being a thing.
Speaker 2: This is a sixteen bit mini computer with a ten
Speaker 2: megahertz clock. It comes with four K words sixteen bit words,
Speaker 2: but you could externally expand it that's more coarse up
Speaker 2: to eight k and if you wanted to go to
Speaker 2: a big old sixteen you need an external box for it.
Speaker 1: Wow.
Speaker 2: Weighed about two hundred and thirty pounds and cost about
Speaker 2: twenty thousand dollars when it first shipped in nineteen sixty six.
Speaker 2: And finally, yeah, very relevant for the time. In nineteen
Speaker 2: sixty six, a scientist be the name of Joseph Wisenbaum
Speaker 2: working at MIT, released a piece of software called Eliza.
Speaker 1: That was so fascinating to me.
Speaker 2: There Rogerian chatter bought, as they called it at the time,
Speaker 2: ran on an IBM seventy foot ninety four. Whisenbaum talked
Speaker 2: about it doing language transformations, which is interesting to think
Speaker 2: about sixty years later.
Speaker 1: All right, so the real story is that the term
Speaker 1: artificial intelligence was being bounced around, and he knew that
Speaker 1: these were just programs, and to prove it, he wanted
Speaker 1: to say, you know, this might seem like a human,
Speaker 1: but it's just looking for keywords and spinning out wrote
Speaker 1: answers and actually not even answers, answering questions with.
Speaker 2: Questions questions, the Rogerian approach.
Speaker 1: Yeah, yeah, so if you say, you know, I hate
Speaker 1: my brother or whatever, it would say, tell me more
Speaker 1: about your family. But he did it as a goof really, yes,
Speaker 1: he did. He did it to show how ridiculous it was,
Speaker 1: how ridiculous.
Speaker 2: And people spent hours with it.
Speaker 1: It's true. Yeah, I have witnessed it.
Speaker 2: Yeah, they did run it through a Turing test. And
Speaker 2: it failed. Of course, not that the Turing test is
Speaker 2: particularly meaningful, but this is in the early days of it,
Speaker 2: but it was one of the they later on to
Speaker 2: come on to define this idea called the Eliza effect,
Speaker 2: which is this a tendency for humans to overvalue language
Speaker 2: interactions as especially intelligent answer promatively. You know, language is
Speaker 2: a pretty profound thing in this sort of sentient space, right,
Speaker 2: so we're kind of wired to respond to that for
Speaker 2: better or worse. But hi, golly, aren't we battling these
Speaker 2: same problems right now?
Speaker 1: Oh, one hundred thousand times more.
Speaker 2: Yeah. Now we call it chat GPT psychosis.
Speaker 1: It really is too, isn't it. I mean, it's very serious.
Speaker 1: I'm not going to say who it is, but somebody
Speaker 1: that you and I both know what. You know. He
Speaker 1: was trying to talk to his mother about AI or whatever,
Speaker 1: and to impress her he basically spit out a document.
Speaker 1: He went to chat GPT and spit out a document,
Speaker 1: you know, in minutes, and she was amazed by it.
Speaker 1: And I said to him, I said, did you check
Speaker 1: the facts? Heause, no, no, I didn't. It was actually
Speaker 1: an application. It was like something that you would send
Speaker 1: to somebody to apply for something, right, and it had
Speaker 1: all sorts of facts and did you check it? Because
Speaker 1: you know, the thing does get stuff wrong. He goes
Speaker 1: fabricate stuff whole cloth.
Speaker 2: Yeah, and he.
Speaker 1: Didn't even think about it.
Speaker 2: Right.
Speaker 1: It's like, you know, and this is an intelligent because
Speaker 1: the computer is always right, right. This is an intelligent
Speaker 1: person that you and I both know, and you know,
Speaker 1: just forgetting to fact check. Yeah, yeah, yep, scary all right.
Speaker 1: End of Soapbox nineteen sixty six, you know, sixty years ago.
Speaker 1: All right, Shall we introduce our guest, Michael Levan. Michael
Speaker 1: translates technical complexity into practical value. He's a seasoned engineer, consultant, trainer,
Speaker 1: and content creator in the Kubernetes, security, DevOps and platform
Speaker 1: engineering space, spending his time working with startups and enterprises
Speaker 1: around the globe. Michael is also a Microsoft MVP in
Speaker 1: the Azure space, in AWS community builder three times, a
Speaker 1: four times published author, a podcast host, an international public speaker,
Speaker 1: CNCF ambassador, and he was part of the Kubernetes version
Speaker 1: one point two eight and version one point three to
Speaker 1: one release team. Welcome to the.
Speaker 3: Show, Michael, Thank you so much for having me now,
Speaker 3: I have the friendly reminder that I need to cut
Speaker 3: down my bio a little bit.
Speaker 1: Actually it's quite impressive and I wouldn't leave anything out.
Speaker 2: And he's knocked out a couple of run answers with
Speaker 2: me over the years too.
Speaker 1: Yep. Yeah, but your first time on this show.
Speaker 3: Yeah, yeah, thank you guys so much for having me
Speaker 3: really appreciate it and worries.
Speaker 1: Well, you know, part of DevOps is dev so oddly enough,
Speaker 1: you do you do have your hand in that space
Speaker 1: as well?
Speaker 3: Yeah, no, absolutely, I mean, ironically enough, I feel like
Speaker 3: I kind of live in two worlds. I do a
Speaker 3: lot of you know, the devopsy cloud based stuff, but
Speaker 3: I would say probably as a consultant, half of my
Speaker 3: a year or so is spent doing a lot of
Speaker 3: back end software engineering. So I've been doing a lot
Speaker 3: of Python lately. Before that, I was doing a lot
Speaker 3: of Go in my free time, you know, outside of
Speaker 3: client work. I'm diving deeper and deeper into Rust. So
Speaker 3: I'm definitely in all the worlds ironically enough, so.
Speaker 2: All the worlds. Yeah, it's good to be a polyglot, right,
Speaker 2: to exercise the different languages and sort of see how
Speaker 2: the various sides live, although I just went through a
Speaker 2: process where we took a sample ap of an MVP
Speaker 2: and regenerated in six languages in a matter of days.
Speaker 2: It's like, here's what it looks like in Swift, Here's
Speaker 2: what it looks like in Rust. Like, I mean, you're
Speaker 2: don't even wonder what language even means anymore.
Speaker 3: The irony of it is, I've been thinking about this
Speaker 3: a lot, is that I wonder if we'll reach a
Speaker 3: point where like natural language will be the language.
Speaker 1: Yeah, you know.
Speaker 3: I wonder if we'll hit a time where it's like, yeah,
Speaker 3: you know, I want to write a memory, you know,
Speaker 3: sensitive application, and then you have two choices. You say, well,
Speaker 3: you can go with you know, this language, which is
Speaker 3: maybe Go, or you go with this language, which is
Speaker 3: maybe you Russ. And then you say, okay, this is
Speaker 3: what I wanted to do, and then it goes and
Speaker 3: it does it. I don't know, maybe we reach a
Speaker 3: point where it's like, you know, everything's just bytecode and
Speaker 3: we just kind of don't see it. It's all natural language.
Speaker 2: Yeah, well, if you never look at the code, what
Speaker 2: do you care what it was generated?
Speaker 1: Unless it's job ASCRIPT, then I would protest, Yeah, do care? Hardily.
Speaker 2: You know, mostly it's about understanding what's going on. I
Speaker 2: was thinking about cucumber the other day. Remember this testing
Speaker 2: lie where you're supposed to sort of write natural language tests,
Speaker 2: and it's kind of you know, we've always tried, you know,
Speaker 2: what is specifications or requirements, but natural language that describes code,
Speaker 2: that is ultimately our measure for how we provided a
Speaker 2: solution which it seemed to be moving further up there.
Speaker 2: And it's not like any of this stuff doesn't take
Speaker 2: a lot of skill to do.
Speaker 1: Well.
Speaker 2: Yeah, you know, it's you know, this is not a
Speaker 2: trivial practice, but certainly evolving without it.
Speaker 1: So Michael has DevOps I don't know changed in the
Speaker 1: last three years. For you, what's been the most significant
Speaker 1: change in DevOps in the last few years.
Speaker 3: I think, you know, the obvious answer is AI, right,
Speaker 3: But I think conceptually, like I always look at it
Speaker 3: in two different ways. I look at it in the
Speaker 3: positions and like what you're doing in your day to
Speaker 3: day at your job. And then I also look at
Speaker 3: it from a like underlying technology perspective. And I always
Speaker 3: say this to everybody, like, if you have good fundamental knowledge,
Speaker 3: like you know, if you have a decent fundamental knowledge
Speaker 3: and you know, the computer science and stuff, right, so
Speaker 3: like data structures, standard algorithms, all that fun stuff. And
Speaker 3: then if you have a good base of you know,
Speaker 3: standard IT networking systems, how all that works. The irony
Speaker 3: is ninety percent of what you're doing is kind of
Speaker 3: staying the same, and you're just sprinkling different abstractions and
Speaker 3: different tools on top of it. I often think about
Speaker 3: like service measures as well in the realm of kubernetes.
Speaker 3: You know, they bring your network observability, your traffic routing,
Speaker 3: your MTLs, encryption between service and service, all that stuff
Speaker 3: we had already. We just now have this layer of
Speaker 3: abstraction that this program that sits on top of a
Speaker 3: service versus embedding you know, TLS code within your application code.
Speaker 3: So point being is I think a lot of it
Speaker 3: is still the same. I think what's just changing is
Speaker 3: like the different tools that we're putting top on top
Speaker 3: of Okay, cloud based environment systems all that.
Speaker 1: Certainly the human aspect of DevOps, which is you know, people,
Speaker 1: processes and tools. I guess the thing that Richard likes
Speaker 1: to talk about all the time, the people stuff. Has
Speaker 1: that changed because of AI or are the people skills
Speaker 1: still there and required.
Speaker 3: Oh one hundred percent still required? Totally, totally. I think
Speaker 3: that with bringing an AI into DevOps and anything right
Speaker 3: like software engineering, maybe we could say security as well,
Speaker 3: but I don't think it's it's there yet, but a
Speaker 3: lot of it is around, like DevOps and software engineering.
Speaker 3: And in my opinion, I think I need to work
Speaker 3: harder now using an LLM uh than I did before,
Speaker 3: and I think that the level of effort is just
Speaker 3: changed a little bit as the human operator. So like before,
Speaker 3: for example, maybe I'm sitting there looking at a line
Speaker 3: of code for an hour saying to myself, like what
Speaker 3: is going on here? And then I finally found it
Speaker 3: and I say, hooray, I have fixed all my life's problems.
Speaker 3: Now now I'm.
Speaker 2: Where can I get me some of that?
Speaker 3: Now I'm generating, you know, this part of my application
Speaker 3: with an LM. But now I'm looking through the whole
Speaker 3: thing and I'm saying, this isn't right. I'm going to
Speaker 3: change this, this isn't right. I had a scenario the
Speaker 3: other day where I wanted to build on an MVP
Speaker 3: of something and I built it and I used claud
Speaker 3: code and I was running it, and I'm like, why
Speaker 3: am I result? Why are my results the same across
Speaker 3: every system that I test this on? And I looked
Speaker 3: at it turns out Claude cod I didn't ask it
Speaker 3: to do this. By the way, claud Code created a
Speaker 3: bunch of mock data within it and with a bunch
Speaker 3: of print statements, so like I literally had to go
Speaker 3: in and refactor everything because it was just all fake
Speaker 3: data that was getting printed out to me on the terminal.
Speaker 3: So like I had to spend probably more time fixing
Speaker 3: that than I would if I just wrote it myself.
Speaker 3: So yes, I think the Humanator, the Humanator, the human.
Speaker 1: That was my nickname in high school.
Speaker 2: It's nice.
Speaker 3: I think that piece is actually probably more important now
Speaker 3: than ever, to be honest.
Speaker 1: Yeah, And unfortunately this is another one of those shows
Speaker 1: where we're talking about AI. But you know that we
Speaker 1: catch some flak for that and our listeners like it
Speaker 1: when we just get back to real development. But but
Speaker 1: you can't deny that. You know things are changing, and
Speaker 1: this is one of those things we've been saying this
Speaker 1: a bunch that you have to you now have to
Speaker 1: think of yourself if you're going to embrace these tools.
Speaker 1: You have to think of yourself as sort of a
Speaker 1: manager of lower level engineers and fact checker, right or
Speaker 1: or code checker, you know, code reviewer. If that's the
Speaker 1: role that you're taking, you know, and hearing we have
Speaker 1: a situation where it's even taking longer because of that
Speaker 1: than if you did it yourself.
Speaker 3: Yeah, I mean, and we aren't you know, like having
Speaker 3: these layer of layers of abstraction isn't new, right, Like
Speaker 3: in the beginning, if you wanted to test something out,
Speaker 3: you had to write your own compiler and you had
Speaker 3: to write your own editor just to run this thing.
Speaker 3: And now I pop open VS code and you know,
Speaker 3: I run my high level programming language and I and
Speaker 3: it's good to go. So like there was a time
Speaker 3: I'm sure where everybody was like, oh, we're gonna humans
Speaker 3: are going to go away, and we're going to lose
Speaker 3: all of our knowledge because you know, we're not writing
Speaker 3: our own editors to run this application anymore. So it's
Speaker 3: we've seen this countless at times, you know, through the
Speaker 3: last sixty five years of software engineer.
Speaker 1: Also, I think that there's a an expertise that gradually
Speaker 1: grows when you're interacting with these things so that you
Speaker 1: know how to interact with them. The problem is that
Speaker 1: they change so quickly and change so often that it's
Speaker 1: hard to know. You know. We we live with a
Speaker 1: single compiler, a single ide for years and we finally
Speaker 1: get it figured out, and then they make a switch,
Speaker 1: and then we have to figure something out. But it
Speaker 1: takes years. Now these things, who knows, they could have
Speaker 1: changed in the back while you were taking a nap
Speaker 1: or drinking coffee, and and the way that you interact
Speaker 1: with it to be most productive has changed. Totally awesome.
Speaker 2: Yeah, yeah, but you know, it's all part of the
Speaker 2: evolution of things too.
Speaker 1: Right.
Speaker 2: I'm I got concerns. I do have problems with the
Speaker 2: hype cycle. But what I like about DevOps is it
Speaker 2: is in the product, right. It is a set of
Speaker 2: disciplines that make a better team. It's another one of
Speaker 2: those terms that's kind of fading away in the sense
Speaker 2: that this is just what a high functioning team looks like.
Speaker 2: That everybody's pulling towards the same goal of delivering solutions
Speaker 2: to customers that we care about, both its creation and
Speaker 2: its deployment and its operations and the telemetry and how
Speaker 2: that feeds back to making a better product. The fact
Speaker 2: that the tooling shuffles around underneath us par for the course, right,
Speaker 2: you could hope these things are only going to get better.
Speaker 3: Yeah, And I think you know to that point, a
Speaker 3: couple of years ago, five six years ago, maybe if
Speaker 3: you went and you wanted to do a talk on
Speaker 3: DevOps at a conference, I mean, that was the thing, right,
Speaker 3: like every conference shows, right, Yeah, that was it.
Speaker 1: That was it.
Speaker 3: Now you know it's different because to your point, Richard,
Speaker 3: it's just all kind of there in the background. It's
Speaker 3: just everybody's just kind of doing the thing.
Speaker 1: Uh.
Speaker 3: You know, we we're we're seeing it very similar with Kubernetes.
Speaker 3: Like Kubernetes from a container orchestration perspective, that's just the
Speaker 3: thing that you run it on. It's not it's not
Speaker 3: the hot sexy topic anymore, but the stuff that you're
Speaker 3: building on top of it is.
Speaker 1: Uh.
Speaker 3: And I'm sure you know I will go through that
Speaker 3: same thing. The one thing that I will say though,
Speaker 3: is when you're not programming day to day, you get
Speaker 3: rusty quick.
Speaker 2: Oh sure.
Speaker 3: So if you're constantly using you know, whatever your favorite
Speaker 3: m is to generate your code for you, you may
Speaker 3: be able to like, of course, still look at it,
Speaker 3: you know, figure figure out this is breaking here. I
Speaker 3: got to put some air handling over there. Blah blah blah.
Speaker 3: But like if you're just staring at a blank ide,
Speaker 3: you may have some trouble after a while, like getting
Speaker 3: something out on paper, so to speak.
Speaker 2: Sure humans are better at editing than they are to
Speaker 2: creating out a whole cloth anyway, yea, And Heaven help
Speaker 2: you if you try and build a CICD pipeline from
Speaker 2: scratch and YAML on a blank screen. I'm sorry. I
Speaker 2: cut and paste it all and then I go through
Speaker 2: it and try and understand what I just paste it
Speaker 2: in and make the tweaks accordingly, like we build on
Speaker 2: the shoulders of giants.
Speaker 1: It's the same for writing pros or poetry. Is that
Speaker 1: totally You don't write, you brainstorm and then edit. Writing
Speaker 1: isn't about writing, it's about editing.
Speaker 2: Yea, and all these things are the same. I'm just
Speaker 2: realizing we talked about in the earlier versions of Visual
Speaker 2: Studio how hard it was to build out the tooling,
Speaker 2: all the bits and pieces you need to actually automate
Speaker 2: the deployment pipeline. Today it's almost like we have an
Speaker 2: abundance of riches, like you're gonna get hub action this
Speaker 2: as you're devopsing this, yes or terrorforming this, like you
Speaker 2: have a lot of choices.
Speaker 1: I remember when customers would hire app Phenex to create
Speaker 1: a CICD pipeline as a you know, a consultancy because
Speaker 1: they didn't really know how to do it. And now
Speaker 1: it's just click click click boom boom.
Speaker 2: Well, and it was kind of a one off thing too,
Speaker 2: Like why would you want to get good at this?
Speaker 2: Although let's be clear you should, although today, like you said,
Speaker 2: it's just a set of checkboxes, Like the real crazy
Speaker 2: part is that you didn't. I still feel like the
Speaker 2: automated tests aren't great, although I gotta say looking at
Speaker 2: the prompt magicians, the guys that are really good at
Speaker 2: using these tools, they they are generating tests as part
Speaker 2: of the code and iterating on the test, passing with
Speaker 2: the code using the tools. Like arguably, if you do
Speaker 2: a good job with this, like you're doing that, you're
Speaker 2: you're insisting that these tools do the ideal case you
Speaker 2: never even measured up to yourself.
Speaker 3: It is very true.
Speaker 2: There's nothing like nothing like one hundred percent code coverage
Speaker 2: when you don't have to do.
Speaker 3: It and it's all about the templates, right. It's like,
Speaker 3: you know, do I have to write my one hundred
Speaker 3: thousandth unit test or mock test or integration test or
Speaker 3: functional test. No, it would be really great if I
Speaker 3: could just offload that to my AI friend, whichever one
Speaker 3: I decided to go with.
Speaker 2: You know, the one that I really struggle with is
Speaker 2: good telemetry. It's easy to collect a lot of data,
Speaker 2: it's hard to collect the data that really tells you
Speaker 2: what's going right and what's going wrong m hm.
Speaker 1: And particularly know how to make sense of it.
Speaker 2: Yeah, we're clearly getting some sense of messages coming back here.
Speaker 2: We have reams and reams of data coming in and
Speaker 2: we're paying for all that. But are we learning anything
Speaker 2: about how people are using our app or are they
Speaker 2: what their frustrations are?
Speaker 1: Like?
Speaker 2: I think this takes a lot of thought.
Speaker 1: Yeah. Are you still seeing the same problems in twenty
Speaker 1: twenty five with DevOps that you were three or four
Speaker 1: years ago? Yeah?
Speaker 3: I mean I feel like I'm seeing the same problems
Speaker 3: that I saw, you know, fifteen years ago. I think
Speaker 3: so many of the the fundamental problems don't change all
Speaker 3: that much because the fundamental problems are honestly typically people problems,
Speaker 3: right It's you know, I think that's just how it
Speaker 3: always is, which is why I you know, a lot
Speaker 3: of people are nervous right now because you know, they think, hey,
Speaker 3: I'm going to take jobs in this now, blah blah blah.
Speaker 3: But that fundamental thing doesn't change, you know, until the
Speaker 3: terminator like universe is fully in play here. I think
Speaker 3: we're all all right. But you know, in terms of
Speaker 3: like DevOps as a whole, I think the biggest things
Speaker 3: honestly right now, phinops is a huge one that everybody's
Speaker 3: trying to figure out, like the whole idea around cost
Speaker 3: optimization and with that performance optimization and resource optimization. Things
Speaker 3: are moving, I mean, and I know we always say this, right,
Speaker 3: but it feels more so now that things are moving,
Speaker 3: you know, quote faster than ever. And with that, there's
Speaker 3: just four more resources being used. And now we're seeing
Speaker 3: a lot of of like actual like cost optimization or
Speaker 3: finops engineers or whatever the title is getting hired in
Speaker 3: organizations specifically to do this job. And that's very interesting.
Speaker 3: So I'm definitely seeing a lot of that. But in
Speaker 3: terms of you know, the general what DevOps is doing.
Speaker 3: I don't think much of it has changed recently. I
Speaker 3: think the I don't know if you would call this
Speaker 3: a DevOps change per se, but the whole idea around
Speaker 3: platform engineering, I think really what they're trying to do
Speaker 3: with that is, say, I have engineers that are managing
Speaker 3: and building and maintaining this particular tool that we're utilizing
Speaker 3: internally like a product. Right, So they're putting their customer
Speaker 3: service hats on because you know, their internal engineers are
Speaker 3: the customers of the platform. You know, they're building it
Speaker 3: in a way where they're thinking more about architecture than
Speaker 3: they are about let me go write the code, and
Speaker 3: then they're building this thing for use to be able
Speaker 3: to move faster inside. Now, we've always kind of had
Speaker 3: tools like that, or you know, rather engineers that would
Speaker 3: build stuff like that. You know, when I had a
Speaker 3: senior Principal Infrastructure engineer title years ago, when I was
Speaker 3: building out little automation tools in Python, which would maybe
Speaker 3: look like you know, platform engineering today. But I would
Speaker 3: say that's probably the biggest shift that I've seen recently
Speaker 3: in this space overall.
Speaker 1: Sure, it seems like a good place to take a break,
Speaker 1: So we'll be right back after these very important messages
Speaker 1: stick around. Did you know that you can work with
Speaker 1: AWS directly from your ide AWS provides toolkits for visual studio,
Speaker 1: visual Studio, code, and jet brains rider Learn more at
Speaker 1: AWS dot Amazon dot com, slash net slash tools. Now
Speaker 1: we're back. It's dot NetRocks. I'm Carl Franklin. That's my
Speaker 1: friend Richard Campbell, Hey, and this is our friend Michael Levan,
Speaker 1: And we're talking about DevOps in twenty twenty five. Because
Speaker 1: you know, some things have changed, but some things have not.
Speaker 2: Uh. Security, I'm just going to drop that bomb out there,
Speaker 2: because yeah, you know it's the battle a. It's way
Speaker 2: more difficult these days. The black hat's gotten way smarter.
Speaker 2: Oh god, it's got to be part of the equation.
Speaker 2: A lot of this generated software people are not putting
Speaker 2: enough consideration in. They're much less. Not that they did
Speaker 2: but when they were writing it all by hand. But
Speaker 2: how do you see a fitting in the pipeline, Like
Speaker 2: what's the culture like when to make this work? Well,
Speaker 2: I like to.
Speaker 3: Look at it from the software side and then from
Speaker 3: the networking side. So from a software from an application perspective,
Speaker 3: how you're securing your application, whether you're going with you know,
Speaker 3: your standard app set stuff like your SaaS stuff and
Speaker 3: your SCA stuff, what libraries you're using, you know, are
Speaker 3: you encrypting traffic? What does the code quality actually look like?
Speaker 3: So that's one piece of it. But from a DevOps perspective,
Speaker 3: ironically enough, I feel like the majority of security issues
Speaker 3: are around networking. So you know, if there are any
Speaker 3: network admins or network engineers listening, you know, the whole
Speaker 3: idea around what does ingress look like, what does egress
Speaker 3: look like? How are services talking to each other? So like,
Speaker 3: for example, in the realm of Kubernetes, are you using
Speaker 3: a service mesh? Are you encrypting the traffic going back
Speaker 3: and forth at the L seven layer? Are you doing
Speaker 3: pod to pod encryption? Are you, you know, encrypting at
Speaker 3: the L three and the L four layer within your cluster? So,
Speaker 3: and that's just an example, of course, but like I
Speaker 3: think that's honestly, in my opinion, the biggest part of
Speaker 3: it is what traffic is coming in, what traffic is leaving,
Speaker 3: and perhaps even most importantly in the world of containerization,
Speaker 3: how are these containers, are these pods talking to each other,
Speaker 3: and how does the traffic look. Is this pods supposed
Speaker 3: to be talking to that pod or that container? Are
Speaker 3: these services supposed to be calling out to this database?
Speaker 3: And I would say, honestly, that's a majority of the
Speaker 3: security issues in the realm of debops and platform engineering.
Speaker 3: The other big thing is and this is I think
Speaker 3: just across any realm of it. I think one of
Speaker 3: the red hat reports that I read maybe last year
Speaker 3: or the year before, it was like seventy four seventy
Speaker 3: six percent of security issues are due to misconfigurations. Yea,
Speaker 3: and that's just everywhere.
Speaker 2: And then I'm patch servers, you know, throwing on my
Speaker 2: run ass hat, like for honest the goodness, I think
Speaker 2: we're actually making headway on on SQL injection these days,
Speaker 2: so that it's I think it's a third now. It
Speaker 2: used to be first forever. But it's like, you know, misconfiguration.
Speaker 2: You haven't secured the thing properly in the first place,
Speaker 2: and you didn't get the patch donet a time. Like
Speaker 2: we've had huge discussions on run ass about what's higher risk,
Speaker 2: deploying a bad patch or not deploying the patch as
Speaker 2: quickly as possible. And it seems like that sort of
Speaker 2: flip that I'd rather the outage because I deployed the
Speaker 2: patch fat than the exploit that comes from not getting
Speaker 2: that patch out there quickly, which is crazy, but it
Speaker 2: seems to be the new reality.
Speaker 1: So it seems to me that you've got two sort
Speaker 1: of realms of security when it comes to DevOps. You
Speaker 1: have the configuration of the topography itself, and once that's
Speaker 1: done and everything is secure and working, now you have
Speaker 1: a continuous job, which is you need a software bill
Speaker 1: of materials. You need to know what devices you have,
Speaker 1: what their versions are, and update them every chance you get.
Speaker 1: So you need to keep on top of that, and
Speaker 1: on top of that the software, the libraries and dependencies
Speaker 1: that you're on. You need the graph of dependencies so
Speaker 1: that and you have to watch the news. So if
Speaker 1: there's an exploit in one of those things, you need
Speaker 1: to know that, and you need to you know, if
Speaker 1: it's hardware, you either have to patch it right away.
Speaker 1: Sometimes the only thing you can do is take it
Speaker 1: off the network because there's no patch available and you're vulnerable.
Speaker 2: Worst case scenario. Yeah yeah, but hopefully you're going to
Speaker 2: security Microsoft and you're watching the CVE stream, like exactly,
Speaker 2: that's probably the first place to look, Michael, are you
Speaker 2: big on things like the actual API management? Like that
Speaker 2: saved my bacon a couple of times now just using
Speaker 2: those tools, Yeah, to show I know who uses my APIs,
Speaker 2: I can set thresholds per user. So when somebody got
Speaker 2: exploited and started doing mass extraction of data, the API
Speaker 2: limits kicked in and kicked up a bunch of warnings.
Speaker 2: You know, they had legit credentials. Everything looked fun. It's
Speaker 2: just the traffic was out of shape because it was
Speaker 2: an exploit.
Speaker 3: Yeah, one hundred percent. And I think tools like that
Speaker 3: we all need to be looking at constantly because the
Speaker 3: problem is, well it's not even a problem, it's just
Speaker 3: something that maybe we all still have to get used to,
Speaker 3: is that there are way more endpoints now and way
Speaker 3: more layers to networking than there were before. You know,
Speaker 3: to take because we're talking about DevOps, right, take the
Speaker 3: Kubernetes example again, Like there was a time where you
Speaker 3: just had this monolithic app and like there was a
Speaker 3: server and there was one entry point, right, and everything
Speaker 3: was there and it was all good. Now they're just
Speaker 3: multiple layers, like you know, you're looking at the Kubernetes cluster.
Speaker 3: You have your host networking layers like your virtual machines
Speaker 3: and stuff and everything running it. And then you have
Speaker 3: the container networks, the pod networks. That's another layer that
Speaker 3: you have to manage. Then you have your Kubernetes services,
Speaker 3: which are another networking layer, and that's where you know
Speaker 3: your security centric cni's and your service match like histio
Speaker 3: and stuff will come.
Speaker 2: To Gee, I wonder why we have configurations problem, Michael,
Speaker 2: I'm confused. I'm on my seventeenth layer and I feel
Speaker 2: really good.
Speaker 3: Well, and that's that's honestly the Again, it's not a problem.
Speaker 3: It's just something that we all have to really get
Speaker 3: used to because as where you can call it decoupled
Speaker 3: applications or your micro services or whatever whatever phrasing you
Speaker 3: want to call it, everything is split up now and
Speaker 3: everything is taught to each other. It's not just in
Speaker 3: one box anymore. And because of that, you now have
Speaker 3: not only multiple layers, but a lot of east west traffic,
Speaker 3: a lot of north south traffic. Like everything is talking
Speaker 3: to everything inside of a cluster, but then also outside.
Speaker 3: Maybe it's hitting a public endpoint, maybe it's hitting a database.
Speaker 3: There's a lot of traffic going back and forth, and
Speaker 3: that's why I think, you know, outside of the software
Speaker 3: engineering security stuff, I certainly hope AppSec gets more and
Speaker 3: more popular, especially with AI right now. It needs to.
Speaker 3: But aside from that, from a purely DevOps perspective, and
Speaker 3: I think the majority of it is networking. Like you
Speaker 3: have your patches, of course, and you want to be
Speaker 3: able to update your APIs like from a you know,
Speaker 3: let's say like you're utilizing a specific API within an
Speaker 3: application running. You're not managing that application, but you want
Speaker 3: to be able to upgrade those versions. That's one thing.
Speaker 3: But yeah, I mean I'm going off on a tangent here,
Speaker 3: but a big part of it is networking, honestly.
Speaker 2: Yeah, I arties like why haven't I set up an
Speaker 2: agent that's watching the CVE stream from Microsoft and just
Speaker 2: evaluating against every project that I've got they exist, Yeah,
Speaker 2: I appresume they do. It's this is non dirigional line idea.
Speaker 3: Well, the MCP stuff, but it's.
Speaker 1: A great idea. M M Yeah, you don't necessarily want
Speaker 1: to give it the ability to update your firmware, but no,
Speaker 1: but certainly I just want to heads up right a
Speaker 1: heads up because would be good. Yeah, just reading all
Speaker 1: those things like reading a CVE is painful. What right,
Speaker 1: They're not fun to read. But before you can get
Speaker 1: that list effectively, you need to have software building.
Speaker 2: Materials are at the other side of this, right, Yeah.
Speaker 1: So are you seeing s BOMs slowly creeping into the
Speaker 1: culture or still no? I mean last time I checked
Speaker 1: that there's not not everybody's doing those. Yeah.
Speaker 3: I don't think as much like it. It hasn't grown
Speaker 3: in the same way as like your general security practices,
Speaker 3: like for example, like policy as code, you know, making
Speaker 3: sure that you're following all the best practices and you're
Speaker 3: using something like open Policy Agent or or you know,
Speaker 3: Kaiburno or one of the other policy as code tools.
Speaker 3: I think stuff like that we're seeing more so, but
Speaker 3: things like s BOM, things like you know, your your
Speaker 3: overarching cybersecurity pieces. I don't you know what it is.
Speaker 3: I just don't think it's like it's not like a
Speaker 3: hot topic, you know what I mean.
Speaker 2: It is it's very preventative, but like it.
Speaker 3: Doesn't sound good, you know what I mean, And people
Speaker 3: are like, ah, you know, we don't have to worry
Speaker 3: about that. Because it's not you know, a top five
Speaker 3: thing in the realm of cloud native or whatever, and
Speaker 3: that's just that those are the hype cycles and all
Speaker 3: that fun stuff.
Speaker 2: So well, sbom got more love when log for Jay
Speaker 2: got exploited. How bit exposed are we? Right? Like how
Speaker 2: many of these things only get dealt with after something's
Speaker 2: on fire?
Speaker 3: Right, And that's the general security unfortunately, is that it's
Speaker 3: really important when things go bad and richer to your
Speaker 3: point of like you know, hey, I want to set
Speaker 3: up an agent to be able to listen for, you know,
Speaker 3: any TVs that are coming in or whatever. Then you
Speaker 3: then you got another security issue, right, is what the
Speaker 3: agent telling you accurate?
Speaker 1: Right?
Speaker 2: Yeah? Even correct?
Speaker 1: And then.
Speaker 2: Yeah, around and around. Well, but this is you know,
Speaker 2: who puts on the tinfoil hat in your organization?
Speaker 1: Right?
Speaker 2: Like where We're lucky, there's somebody wears it all the
Speaker 2: time and they like it, which is rare. Uh, And
Speaker 2: that's a strange person. And I you know, I'm very
Speaker 2: aware that there in a few places like Okay, one
Speaker 2: some month, I'm going to put the hat on and
Speaker 2: like today I'm the security guy, I look a little
Speaker 2: more nervous and I'm angry all the time, you know,
Speaker 2: but you know, and you put your foot often you
Speaker 2: find yourself focusing on preventative work, right, and maybe that's
Speaker 2: the time when you build that thing.
Speaker 1: The tinfoil hat guys are generally focused on conspiracies, but
Speaker 1: fortunately for them, you know, hacks and exploits are conspiracies
Speaker 1: against you.
Speaker 2: Is it still a conspiracy if it's really happening, Because
Speaker 2: you're looking at the logs and just watching them hammer
Speaker 2: away at you, it's like, is this a conspiracy or
Speaker 2: it's just a log file?
Speaker 1: Like, I'm pretty sure the tinfoil hat guys think people
Speaker 1: are inherently evil and then they get proof of it
Speaker 1: and it makes them happy to reinforces their tinfoil hatness.
Speaker 2: Yeah, it just but if nobody's looking, then you find
Speaker 2: out the hard way.
Speaker 1: Yeah sure, yeah, Like I'm greeing with you, Richard. I
Speaker 1: think you got to put those guys to work.
Speaker 2: And I've included a link to the gethup repository of
Speaker 2: the s bomb tools that Microsoft put together.
Speaker 1: Cool.
Speaker 2: Oddly enough, I think it was around the time of CrowdStrike.
Speaker 2: I don't know one of those you know, conspiracy another
Speaker 2: one gees, how could that have happened. It was earlier
Speaker 2: than that. Maybe it was it was one of the
Speaker 2: other like supply chain acts, like there's been a few.
Speaker 2: It's just you know, it's frightening. But it's of course
Speaker 2: this is not actually a security show. This is just
Speaker 2: how do we include this in our overall life cycle
Speaker 2: of making software that it does get going And you know,
Speaker 2: again it's in some ways it's like we're going to
Speaker 2: be the better version of ourselves because we're writing this
Speaker 2: as a prompt for tools, for these tools to actually
Speaker 2: try and implement it right up until it gets hard
Speaker 2: and it's like, no, you actually now have to work
Speaker 2: on this just because you found a CV that might
Speaker 2: be relevant and it relates to a UNK code that
Speaker 2: is deployed in your organization. Now you got to figure
Speaker 2: out what to do.
Speaker 3: Yeah, And I think a big thing is it's really
Speaker 3: hard for organizations, specifically management to like specify an ROI
Speaker 3: on we're going to take the week or whatever it'll
Speaker 3: take to make this application a little bit more secure,
Speaker 3: and no, sorry, we have something something deadlined. Sorry, something
Speaker 3: something needs to be out this week at the end
Speaker 3: of the quarter.
Speaker 2: When you're also avoiding the big thing, which is that
Speaker 2: the consequences to the company getting exploited never seem to
Speaker 2: be that significant either. You know, you if you engage
Speaker 2: your PR crisis team, you do your make help us
Speaker 2: on TV if you if you're a large enough scover
Speaker 2: that thing. All of the customers that got exploited, you
Speaker 2: sign them up to some kind of monitor during service
Speaker 2: and say sorry, and then you go on with your day.
Speaker 1: Yeah.
Speaker 3: And I've even heard, like quite literally, I've heard we
Speaker 3: don't want to know.
Speaker 2: Yeah, yeah, because he represents liability.
Speaker 1: Yeah. If I don't know, if you knew, you know,
Speaker 1: it's gonna blame me. Yeah.
Speaker 2: Yeah, it's so. And I'm not a big on willful ignorance,
Speaker 2: but holy man, right, like come on, yeah, and again
Speaker 2: it so the side effective the run last time of
Speaker 2: dealing with real security guys where quit a job because
Speaker 2: they refused to deal with it and then when it
Speaker 2: went down big time, they got sa panted.
Speaker 3: Right right, yep. Yeah, it's not a uh, it's actually
Speaker 3: not an uncommon thing. I mean it's very Yeah. Security
Speaker 3: is securing your application in general, securing your your DevOps
Speaker 3: piece of the puzzle. I mean, it's for for everybody listening.
Speaker 3: If you're wondering like, oh, do I have to go
Speaker 3: and dive into security full time? Or you know, what
Speaker 3: do I gotta do here? Really, it's just when you're
Speaker 3: designing your your pipeline, or you're designing your Kubernetus cluster,
Speaker 3: you're designing your application stack, whatever it is, just take
Speaker 3: the extra five to ten minutes to think it through,
Speaker 3: you know, and a lot of it again, it's it's
Speaker 3: so many just simple misconfigurations. Oopsies. I left root access
Speaker 3: on these r back permissions when I was testing something
Speaker 3: and I forgot they were there. I mean that's probably
Speaker 3: like in the top three things that happened, right, you know, Yeah.
Speaker 2: Yeah, deployed with it with the dbug still on, and
Speaker 2: people can stap in back black hat stap in it
Speaker 2: and take advantage of it. Liketh, it's it, Yeah, well
Speaker 2: were I'm not clever enough to think of these things.
Speaker 2: All I have is the case studies.
Speaker 3: Yeah, it's really, honestly, so much of it, especially especially
Speaker 3: from a DevOps perspective, is just yeah, take the extra
Speaker 3: you know, breathing through your nose and out through your
Speaker 3: mouth and just say okay, does does the look okay?
Speaker 3: And honestly, if you if you just take that couple
Speaker 3: of minutes, I can guarantee that you're gonna find some stuff.
Speaker 3: And you're like, oh, this would have been bad.
Speaker 1: Yeah, well that's the standard cycle of development too. You know,
Speaker 1: when I'm developing, I always when I'm done, okay, there,
Speaker 1: I get up and I walk away for fifteen to
Speaker 1: twenty minutes, and I come back and I go through
Speaker 1: it again and just you know, take a deep breath
Speaker 1: to think. But you know, now now you have to
Speaker 1: think about it from the perspective of security, you know,
Speaker 1: and everything else, not just development.
Speaker 3: And that's that's tough for a lot of teams, right,
Speaker 3: especially for DevOps teams, because honestly, I think that the
Speaker 3: biggest problem is I've spoken to a lot of people,
Speaker 3: and a lot of people are like, yeah, I wish
Speaker 3: we could do this, but we don't have the time.
Speaker 3: Yeah I wish we could do this, but we got
Speaker 3: to get this done tomorrow. Yeh, I wish we could.
Speaker 3: And that that's it's always what it is. So sometimes
Speaker 3: you have to manage up as well. Again, and it's
Speaker 3: it's never a technology problem. It's it's always a people thing,
Speaker 3: not even a problem, right, It's just you need to
Speaker 3: be able to and this is going to be different
Speaker 3: across every organization. You need to be able to think
Speaker 3: about what your manager, how your manager receives information, and
Speaker 3: mold what you want to tell them in a way
Speaker 3: that gets them to understand, ooh, this is important, we
Speaker 3: need the extra day because usually a manager can find
Speaker 3: the extra day or the extra couple of hours as
Speaker 3: long as you present the information in a way that
Speaker 3: they can digest it and it's important to them. If
Speaker 3: you're frantically running around and saying, oh, this thing needs
Speaker 3: to be done because of this and that and technical
Speaker 3: jargon here and technical jargon there, chances are a manager's
Speaker 3: going to say, yep, just another person that you know
Speaker 3: just wants extra time to do this thing and to put,
Speaker 3: you know, a couple of extra sprinkles on top. I've
Speaker 3: seen this a thousand times in my career. But if
Speaker 3: you're the person that goes to your manager and explains
Speaker 3: it in a way that makes them realize just how
Speaker 3: important it is, and hopefully you know your manager well
Speaker 3: enough to know how to do that, If you don't learn,
Speaker 3: then your outcome is going to be very different. So really,
Speaker 3: I mean implementing all the security implementations can be as
Speaker 3: simple as just understanding what the other person needs to
Speaker 3: be able to hear to make it an effective decision.
Speaker 1: You know, one thing we haven't talked about is social
Speaker 1: engineering from outside forces into you know, the staff on
Speaker 1: the edge of the of the organization, you know, clicking
Speaker 1: on emails and all of that stuff, and then somebody
Speaker 1: getting control of your network inside. I mean, the way
Speaker 1: to prepare for that isn't necessarily with technology, but education
Speaker 1: of your staff. And I wonder how how often that
Speaker 1: creeps up in your line of work.
Speaker 3: I would say not as much as like, if you're
Speaker 3: on the help desk or you're you know, in the
Speaker 3: cisadmin space or something right now, you're probably going to
Speaker 3: see that a lot more. The only time that I
Speaker 3: personally see it is like if I accidentally click on
Speaker 3: a phishing scan, which I'm human like, sometimes things look legit,
Speaker 3: you know, and I'm like, oh, this was one of
Speaker 3: those no before tests or something like that.
Speaker 1: But I guess, like the standard backup practices protect against ransomware, right,
Speaker 1: so if something does happen, what do you do? And
Speaker 1: you could spend an ordinate amount of money on protecting
Speaker 1: yourself with multiple backups and off site and all of
Speaker 1: this stuff. How do you how do you prepare for
Speaker 1: the or do you even prepare for ransomware? Are you
Speaker 1: just like, now, if that happens, we're screud.
Speaker 3: Should you prepare for it? Absolutely? Like taking the backup example.
Speaker 3: Last week, I actually with a client, I did a
Speaker 3: dr strategy test where we were on the call for
Speaker 3: about three hours and we blew a database away and
Speaker 3: we tested to make sure that things were going to
Speaker 3: work as expected to luckily you know we you know,
Speaker 3: they test recoveries. They don't just back things up and
Speaker 3: hope for the best.
Speaker 1: Now, you don't have backups unless you test.
Speaker 3: Them exactly, that's it. So things like that, I mean, yeah,
Speaker 3: like you can absolutely sit there and prepare for But again,
Speaker 3: it's one of those things where you want to do it,
Speaker 3: and you can do it from a technology perspective. It
Speaker 3: is there and it's readily available, but now you have
Speaker 3: to go in convince your manager that you need to
Speaker 3: take to three to four hours to go and test
Speaker 3: this thing and confirm that it's going to work as expected.
Speaker 3: Ironically enough, in the realm of DevOps and in the
Speaker 3: realm of a lot of engineering. When you get to
Speaker 3: the senior at to principal level and you're the one
Speaker 3: that's supposed to be making these decisions and calling these
Speaker 3: things out, your job is a lot more sales than
Speaker 3: it is engineering. You have to go and sell this
Speaker 3: idea and you can do it, and the technology is
Speaker 3: there and there is trust me, there are twenty vendors
Speaker 3: as we speak that want to sell you their solution
Speaker 3: that can do this thing, and they can do it
Speaker 3: very very well. Now you've got to go sell that
Speaker 3: idea to upper management.
Speaker 1: Well, the best way to do that is put a
Speaker 1: dollar figure around it, Like if we got ransomwared, what
Speaker 1: would it be worth to you to get all this
Speaker 1: data back? You know? And they're going to know that.
Speaker 1: So ten million dollars, twenty million, thirty, one hundred million
Speaker 1: dollars like those aren't out of the out of the
Speaker 1: realm of the ransoms that we've seen paid totally. And
Speaker 1: so you convince the management with dollar figures, what is
Speaker 1: the risk if we don't do this? Yeah?
Speaker 3: A thousand percent?
Speaker 2: Always yeah, And it's always good to bring this numbers
Speaker 2: to the CFO and bring scary headlines to the CEO.
Speaker 1: That's it.
Speaker 3: Yep, yep, yeah, and you just you really just got
Speaker 3: to be able to sell the idea of like what
Speaker 3: can happen. And for your manager it could be those
Speaker 3: dollar figures. For another manager, it could be, well, if
Speaker 3: this happens, we're going to have to hire a consulting
Speaker 3: firm and we're going to pay them five hundred thousand
Speaker 3: dollars to do this, Or could be we need well,
Speaker 3: we're going to have compliance issues, right, and we're gonna
Speaker 3: you know, we're going to mess up and we're not
Speaker 3: going to be able to get our High Trust serve
Speaker 3: or our SoC two compliance or whatever it is. So
Speaker 3: you just got to figure out what's important.
Speaker 2: Could end up in an audit, you know, if your
Speaker 2: financial services he gets hairy.
Speaker 1: And it's something as simple as how much would it
Speaker 1: cost us per day to not have our data exactly right?
Speaker 1: To be offline in one day would cost x amount? Right,
Speaker 1: And now you're so you're more likely to pay the
Speaker 1: ransom to get your data back.
Speaker 2: Accept it. Often when you pay the ransom, you don't
Speaker 2: get your data back.
Speaker 1: Yeah, that's true. You know, there's no guaranteed failure rate is.
Speaker 2: Failure rate's pretty high you know, we have taken that
Speaker 2: story where we're a certain security person actually had to
Speaker 2: fix the decryptor when decrypt files larger than two gigs. Yeah,
Speaker 2: and the reality and we've done this show and run
Speaker 2: as as well. It's a year minimum. You know, you're
Speaker 2: going to have people involved, You're going to be cleaning
Speaker 2: up messes like it's just nothing's fast. It takes a
Speaker 2: long time. And it takes the same amount of time
Speaker 2: where you paid, where you paid the ransom or you did.
Speaker 1: I've heard of companies doing testing their staff by sending
Speaker 1: out phishing emails, you know, and seeing if they actually
Speaker 1: click on them, and you know, when they get there,
Speaker 1: you say, congratulations, you just ransom.
Speaker 2: You just signed up for additional training.
Speaker 1: Yeah, exactly, And a lot of them.
Speaker 3: Are actually pretty good. Like you know, the old method,
Speaker 3: for example, used to be, you know, hover over the
Speaker 3: email and make sure that the domain doesn't have like
Speaker 3: a zero.
Speaker 1: Instead of an ozer exactly.
Speaker 3: And now it's you know, everything's legit and you're like, yep,
Speaker 3: this is a legit email and you click it and
Speaker 3: it's like oop, sorry, so.
Speaker 2: Yeah now and the well, now you get into the
Speaker 2: other out level of this, which is that be more
Speaker 2: careful next time? Is not a strategy now that we
Speaker 2: actually have got a.
Speaker 1: Lot Another big one is you know, here, copy this
Speaker 1: file off my USB stick right oops. So in the
Speaker 1: studio here, I have customers that come and go with
Speaker 1: their own data all the time, and I have a
Speaker 1: machine that is not on the Internet, that that has
Speaker 1: you know, malware bytes and all that stuff on it.
Speaker 1: So I get the latest updates, I unplug it from
Speaker 1: the Internet, stick the thing and run a scan on it,
Speaker 1: and if it's okay, I feel comfortable enough that I
Speaker 1: can put it in my real computer, right right. Yeah.
Speaker 1: So cynical, Yeah, I wish I didn't have to be,
Speaker 1: but yeah yeah.
Speaker 2: But is this the pain point these days? Really just
Speaker 2: think it's the security side of things when it comes
Speaker 2: to software, Like I feel like a lot of the
Speaker 2: a lot of stuff has been well automated deployments automated
Speaker 2: now like that part works, Continuous deployment like that, that
Speaker 2: seems to work pretty well. If you put in the time,
Speaker 2: you can get your results. Total testing still seems to
Speaker 2: be a strugg but that's evolving. But actually having a
Speaker 2: sense of the security quality of your app, I don't
Speaker 2: have good measures man like that. You only find that
Speaker 2: out in the field, it seems.
Speaker 3: Yeah, And I think to that point, the biggest thing
Speaker 3: right now, going back to what I was saying before,
Speaker 3: is the networking aspect, like where are my packets going?
Speaker 3: There's so many different layers now, yeah, east west traffic,
Speaker 3: north south traffic, so many different layers that it's like,
Speaker 3: where is this going? And do I know where it's
Speaker 3: going and what it's doing and who can hit it?
Speaker 2: Did it really need to go there right right doing
Speaker 2: unnecessary things?
Speaker 3: And I guarantee, I guarantee that can fix ninety percent
Speaker 3: of the security issues that you see in like a
Speaker 3: devop slash platform engineering environment right now where there's a
Speaker 3: lot of you know, Kubernetes and there's a lot of cloud.
Speaker 2: But it speaks to whitelisting, allow no traffic except the
Speaker 2: paths I've specified, and I don't see very many folks working.
Speaker 3: That way totally the defense and depth, but you know,
Speaker 3: it's it's all that stuff. It's least privilege right, like
Speaker 3: block everything and then open up as necessary. The problem
Speaker 3: is again that's that's that takes a lot of time
Speaker 3: and people are like, hey, I got you know something
Speaker 3: something deadline, that's it. That's it, yep, yep, yeah, and
Speaker 3: it's it's it's wild to see even even like let's
Speaker 3: say you're setting up different metrics toolings, right like maybe
Speaker 3: you're you're setting up Jaeger, you're setting up Prometheus, and
Speaker 3: you're you're collecting you know, end to end app metrics,
Speaker 3: you're collecting standard server metrics. Whatever you're collecting. Go look
Speaker 3: at those sometime and see what's talking to what from
Speaker 3: a networking perspective, and your mind is going to be blown.
Speaker 2: You know, it's not not even controlling it, just monitoring it.
Speaker 3: You just look at it.
Speaker 2: You'll be surprised even.
Speaker 3: Like every once in a while, you know, I'll I'll
Speaker 3: open up this this fancy new tool called wire shark.
Speaker 3: Yeah crazy, and I'll look at what's talking to what,
Speaker 3: and I'm like, I don't even know.
Speaker 1: What this is.
Speaker 2: Although I'm minutely wire sharks better at marking updata now
Speaker 2: than it's ever been before. The problem with wire sharks
Speaker 2: you turned that on one. Okay, there's a lot of stuff.
Speaker 2: I have no idea what it means. It's overwhelming.
Speaker 3: Yeah, yeah, it's all fun.
Speaker 1: Until somebody loses an eye exactly, sure enough. So Michael,
Speaker 1: what's in your inbox? What's coming up for you? What's next?
Speaker 3: So really still focused in the Kubernetes realm, I'm going
Speaker 3: to be focusing more and more on the networking aspects
Speaker 3: of things, Securing networks, observing networks, making sure that proper
Speaker 3: traffic routing is going where it's supposed to go. Is
Speaker 3: this thing is supposed to be talking to this thing?
Speaker 3: And then also in the realm of AI, you know,
Speaker 3: looking at various agents, how are these agents talking to
Speaker 3: one another? How are you talking to agents? Really everything
Speaker 3: around the Kubernetes networking security realm, and then of course
Speaker 3: just overall platform engineering, making sure that what tools are
Speaker 3: implemented are necessary, which I call in my head just
Speaker 3: proper architecture. I think what's next for me is just
Speaker 3: proper architecture.
Speaker 1: That's good, good, Okay, Well, thanks for hanging out with
Speaker 1: us for an hour.
Speaker 2: We learned a lot.
Speaker 1: I know I did anyway, and thanks again.
Speaker 3: Thank you so much for having me.
Speaker 1: You bet, and we'll talk to you next time on
Speaker 1: dot net rocks. Dot net Rocks is brought to you
Speaker 1: by Franklin's Net and produced by Pop Studios Service audio,
Speaker 1: video and post production facility located physically in New London, Connecticut,
Speaker 1: and of course in the cloud online at PWOP dot com.
Speaker 1: Visit our website at d O T N E t
Speaker 1: R O c k S dot com for RSS feeds, downloads,
Speaker 1: mobile apps, comments, and access to the full archives going
Speaker 1: back to show number one, recorded in September two thousand
Speaker 1: and two. And make sure you check out our sponsors.
Speaker 1: They keep us in business. Now go write some code.
Speaker 1: See you next time. You got javans
Speaker 2: And