Speaker 1: There we go. What's up everyone, Welcome to another episode
of Adventures in DevOps Warren. Thanks for joining me again.
Speaker 2: Yeah, you know, I think you don't need to thank me.
Speaker 3: Every time I decided to show up, I always said
that we were going to have like a maybe a
cool fact at the beginning of every episode, and I
try to come up with one, although I do believe
last time Jillian committed to bringing the fact and I don't.
Maybe it's a coincidence that she's just not here today,
so I will have to step in and share something.
Speaker 2: I forgot the pin for my backup phone.
Speaker 3: And I didn't remember how many digits it was and
on Android, apparently you can keep going forever.
Speaker 2: So I was on one hundred and fifty attempts.
Speaker 3: I would try to guess for this PINNs and it
wasn't like, no, I don't care about factory resetting it,
but you know, I just wanted to see, like can
I get it? And actually one hundred and fifty one
I finally got it. I had up the pin size.
Speaker 2: To twelve digits and like that was like a real
for me.
Speaker 3: So you know, if you forget your Android, Penn, just
keep trying there's there's no there's no time. I mean,
there's a thirty second time out for up per attempt,
but other than that, you just keep going.
Speaker 1: So is that like manual brute force attack? Is that
the category this would fall into?
Speaker 4: Right on?
Speaker 1: Cool, I'm excited about today's episode. Joining us today, we
have Matt Lee from Schematical and from cloud wargames dot com.
Looking at your background here, Matt, co founder of drawn
by dot ai, author of ship or Get Off Thepot
dot com, and then gymnastics coach for eighteen years and
out of Madison, Wisconsin. Welcome Matt, Thanks thanks for having me.
Speaker 5: Yeah, the shipper dot com I'm not doing anymore, but
uh the I actually did write a book on coaching
gymnastics called Hacking Here Strategies how to help high performing
police through mental issues.
Speaker 4: So fun fays right on.
Speaker 1: My youngest son did gymnastics from I think from when
he was about ten all the way up until his
senior year and high school and his senior year. I
mean you're probably well aware of this, Like the number
of collegiate positions for men gymnastics is very, very small,
and so he knew that that wasn't going to get
him anywhere, and he wanted to participate in some kind
of activity at school because for his entire education, he
would get out of school, his mom would be waiting
in the parking lot and she would drive him an
hour to the gymnastics facility and he would work out
for four hours, doing his homework in the car, and
that was his life growing up. So his senior year
he wanted to do something that was school related and
that was the end of his gymnastics career. But it's
a commitment and there's so much mental to itbsolutely.
Speaker 5: Yeah, it's not an on count of a story unfortunately,
men gymnastics it's just.
Speaker 4: Not doing as well these days. Too many competitors this
sport are cooler in.
Speaker 1: Other things, right for sure. Yeah, And I think the
commitment to it just kind of kind of just like
self filters, you know, so that when you do get
to those people that are there, you've got some very
committed individuals.
Speaker 4: Yeah you don't.
Speaker 5: You played football, You play soccer. No one plays gymnastics.
Speaker 1: Right A point. Cool, So tell us a little bit
about actually, first you it looks like you've been in
Wisconsin for a long time. Are you Wisconsin native?
Speaker 5: Yeah, born and raised, traveled the world that I keep
coming back here, and about four years settled down a
bout a house out in the countryside a little north
of Madison, and.
Speaker 3: And love right on Madison is fantastic. I lived there
for two years, and I will say that I love
being there in the summer. Only qualifier that.
Speaker 5: Summer and fall. Yeah, you would get this time of
year or not not as nice.
Speaker 1: What's your current temperature today?
Speaker 4: Oh, it's actually really nice. I was outside earlier with
the dogs out, and.
Speaker 5: I do some jumping jacks to wake up myself pretty
for the podcast and be around ten degrees.
Speaker 4: So it's actually very pleasant compared to Yeah, funny we
were close to a week ago.
Speaker 1: Yeah, that's balmy.
Speaker 4: Right.
Speaker 1: So Cloud I'm gonna mess up with the name. Dammit,
sorry about that. Cloud wargames dot Com tell us a
little bit about that.
Speaker 5: Don't feel bad about messing up the games. I keep
calling it cloud watch games because the cloudwashed all day
long if you had signers on web services and uh,
you know with cloud war games. Yeah, so this goes
back a little bit, if you don't mind me going back.
Speaker 4: I had a team I was running.
Speaker 5: I think I had like eleven people on staff, and
I needed to come up with the training idea of.
Speaker 4: Project for that. So I've come up with a couple of.
Speaker 5: Different games over the years and different tools to help
out my devs. And so one of them was the
predecessor to cloud board games, which is where I would
take awayboard and draw network mathlets like you know, and
so I would I would do with this Dungeons and Dragon.
Speaker 4: Style game with them, and what.
Speaker 5: I did size just straight and down all these different
points as I had hundreds of scenarios of things that
had gone horribly wrong over the years that I would
then throw at them, and I'd say something like, yeah,
give them a brief, which would be like all of
a sudden, your website's timing out, you know, was that,
and they'd have to walk through and tell me how
they wanted to solve it. And so this one, in
this case a security group switch. Somebody removed a security
group which is basically a firewall rule from one of
the various instances or resources running there and started to
time out. So I've got all these other ones incident
cannot deploy, and so it's you know, you don't get
a lot of information.
Speaker 4: A lot of times when people come to you saying there's.
Speaker 5: A problem at the time as customer service person saying,
our guys, you know, our shopping cart is disappearing randomly
for our e commerce store, like people all of a sudden
the shopping cars empty. Uh you know, So using that
and just their wits and and we would just walked
through it as if I was a dungeon master, and
it was a pretty good training tool. You could really
figure out who was, you know, thinking outside the box,
who was like really analytical or who just kind of I.
Speaker 4: Don't know, you know. It turned to be a great
sales tool as well. You go off the client through this,
They're like.
Speaker 5: That can go wrong, That's like, like that can happen.
Speaker 4: You're like, yeah, I can. What would you do?
Speaker 5: I don't know. I guess I'd call you all right.
So that is how it started.
Speaker 4: Then. Uh literally the last Thursday.
Speaker 5: I was on a Mastermind group and I was talking
about some goals I got. I was going to launch
a community for people to want to learn at US.
And I was telling them about my idea for war games,
which is where we spin up actual infrastructure, a database
infrastructure because we got infrastructures code.
Speaker 4: It's not like I have to mangle you.
Speaker 5: It's like to spin it up and then I basically
knock down a part of it, or fill up the database,
or remove the security groups, you know something or gid
us attack myself at the front of it. And then
people will the participants that hot seat participants, which people
with their cameras on ideally will be sitting there and
they'll be like, hey, they'll they'll they they'll try and
fix it, you know. And there's a couple of different variables.
I'm gonna I've got a beginner, intermediate, and advance. Like
the beginner one maybe they would just talk to me
and I'll fix it on my computer screen. The advanced ones,
I'll give them the actual AWUS keys and say here.
Speaker 4: You go, go ahead and fix it.
Speaker 5: And so there's just a yeah, I'm trying to make
it available to all ages. So I'm getting creative with it.
But yeah, that was the of oh yeah, it's Friday.
I bought the domain cloudmoregame dot com and then I
set it up and sent out a mailing list to
it by easily underd subscribers, and I had the biggest
reaction I've ever seen. I've got people signing up coming
out of the look where I had no idea who
they are, and then I you know, you guys, and
then you guys are like you want to be on
so here I am five days later and uh, this
is it's going.
Speaker 3: Well, Well, I'm glad it's going, you know, so quickly
for you, you know, to have a hundred different scenarios
that must have taken quite a long time. Like I'm
just thinking back throughout my career and like, I don't
know if I could come up with like one hundred
things that had gone wrong and have the whole I'm like,
for sure more than one hundred things have gone wrong
in my career. It's just you know, having remembered them
and where I've written them down like that must have
taken some effort on your part.
Speaker 5: I am a compulsive I mean I showed you guys
in the previam I am a compulsive note taker and
journaler and whatnot.
Speaker 4: I actually had multiple journals.
Speaker 5: And a journalists side on that journal here, journal there,
business notebooks.
Speaker 4: So I piece it together.
Speaker 5: And you know, if I'm not if I write it
down another scenario, if something goes wrong, I also make
a comic about it.
Speaker 4: Sometimes, so.
Speaker 1: Right, Yeah, you shared some of your comics with us
when we were emailing back and forth. Is that are
theose something you publish regularly?
Speaker 4: I try and do them.
Speaker 5: I was doing them twice a week, but now that
my content schedule has changed, I'm doing more video content
publicly again, I'm probably only.
Speaker 4: Going to do once a week. But we'll see.
Speaker 3: That's still a lot. That's you know, fifty two a year.
That's still a lot of content.
Speaker 4: I'll be honest.
Speaker 5: They're not all bangers, but some of them take Yeah,
I printed out this was the first one that broke
a hundred thousand.
Speaker 4: Views on Reddit, soo I was. I was pretty happy
about that. This one was a banger too. But don't
get the coffee cut yet. I got to work out
the details and those.
Speaker 3: R Do you find that through through your I mean
capting journals of all of the different scenarios and what
had happened, what went wrong, and how it was fixed, Like,
did that have a beneficial impact for your career? Did
you just find that you are able to consume that
information now to make the cloud war games?
Speaker 5: I mean I had thought and this this idea. I
thought about coming back to you for a long time. So
I have been keeping a lot of times now. I mean,
I will just write a deal to time deal with emailing.
Listen if there's something that goes wrong or something I
hear about. There's one that I want to really run
a center through the S three ransomware attackers out there.
Right now, I'm gonna I'm gonna attack myself with that
and that should be fun. But yeah, I write it
down and and nail it out, you know people. So now, yeah,
I guess collecting all that data has been helpful. I
mean the last year I've been better. I hired a
business coas and have been better at sharing that information,
you know, before I write it down just because I
pecically take notes. And then now it's like I'm actually
finding and helping other people get up based on Amazon,
you get up there, you know, on things I'm an
expert and I enjoy doing so. Yes, I think it's
kind of beneficial impact that I didn't really capitalize on
that and self care of that recently.
Speaker 3: Yeah, I mean the fact you just mentioned this, like,
I can't believe I haven't been keeping track of every
single bug that I run into that as some sort
of you know, owt as it turns out, you know,
and then a story because I feel like there is
some novel, interesting thing there that is worth sharing. And
now I'm starting to think, like would I give this
advice to even experience senior engineers, Like, oh, you know what,
like start keeping a personal journal of these sorts of things.
I mean I usually talk about in business context, like
it's great to keep track of what you're doing from
like a brag standpoint, so that you can report on
it during your hopefully quarterly performance evaluation. Like you have
a list already of everything you've done, and I think
there's a lot to be said there. But I think
there's another level of like the interesting technical challenges that
you had to face and how you sort of tackle them,
because like you've already found a new opportunity based off
of what you've written down to you know, turn into
a business.
Speaker 4: Yeah.
Speaker 5: I mean, it's a good habit to do after action reports,
especially when you're training in a junior or something like that.
You know they make mistake, you know, a fairly expensive mistake.
Speaker 4: Maybe you know they.
Speaker 5: Delete something they shouldn't, and then you've got two options.
You could slap on the risk and say you know
you're in trouble, get it, or you say.
Speaker 4: What did you learn from this?
Speaker 5: We just you know, say I got a client, you know,
their website goes down, and say it costs some one
hundred thousand dollars an hour. Junior takes down that website
for a while and they can look at it like,
oh my god, we've lost all this money. Or you
just say, oh, we invested this much into our junior
running important lesson. Let's have them present that in and
after action.
Speaker 4: Report to the whole team. So we've got you know,
there's a positive from it.
Speaker 5: Here's you're writing the SOPs for the business standard operating
tenders for the business.
Speaker 4: I don't know what not to do.
Speaker 5: You know, sm I try and push towards my developers.
No fail forward faster, no where to fail at where
you can make mistage at, you know, fail on stage,
nuke staging out of the water and slow down our
dev for a day, pretty our identity for a day.
It still costs us less than if you knew production
for even a few minutes.
Speaker 4: You know, So learn where to fail, learn how to fail.
And so this is one of those ranks that you're
working out.
Speaker 1: I want to come back to something. There's been a
couple of things you've said that have stood out to me,
and I want to come back to those, even though
they have nothing to do with Cloudwardgames dot Com. You
mentioned that you're in a Mastermind and then you also
talked about your journaling, and those are two of like
the consistent things that like, if you study like high impact,
high performing people, like those two things come up over
and over again. So how did you get into a
Mastermind and what does that look like for you?
Speaker 5: I'm actually in several and we can do a whole
other podcasts and consulting side of things, instruction in your business,
which I'm gonna save for another time, but there's one
where I am a group coaching where they do live
q and a's and stuff like that aimed at solo consultants,
expertise people who want to put themselves as expert. And
I would never call myself the expert, but I've been
doing this a long time, so that's a good one.
I had direct access to the main community guy there.
And the other one is the room of around Robin Approach,
where I found that one, I think through another community,
a third community where it's actually got very limited interaction,
but they do things and I can give you.
Speaker 4: As the name of some of the stuff if you
want later.
Speaker 5: But I'm actually starting, I'm hoping to start another Mastermind group,
but I kind of deprioritize.
Speaker 4: That for CTO for Cloud of Organcy. Now I don't
even remember the name of because that's that's got a
lot of trash right now.
Speaker 5: But I would strongly advise that, especially if you're going
out there and starting on your own and you're not
part of the big team, you're starting your own consulting,
you need.
Speaker 4: That sounding board.
Speaker 5: I'm very fortunate to have a informal business partner that
we have just we are always bouncing ideas of each other,
because it could be, you know, in the vacuum when
you're trying to get clients, it can be you know,
very daunting, seem very dark.
Speaker 1: Yeah, for sure, I belong to a Mastermind that I've
been in for well over ten years now, and it's
been it's been the difference in my career because, like
our sole purpose in that group is to call each
other out on our bullshit and hold each other accountable
and it's made a difference.
Speaker 4: Yeah.
Speaker 3: We actually we actually do a full episode dedicated to
Will being called out on as bullshit.
Speaker 1: I mean, because there's a lot of it. Like those
guys in My Mastermind, they have a full time job
calling me out on my bullshit.
Speaker 5: Yeah. My most valuable people are the people.
Speaker 4: That have blunts us with me, you know, willing to
be like stop building developer tools, you're fool.
Speaker 3: And giving them away for free. I think that's the
important caveat there.
Speaker 1: Yeah, so for sure. And then on journaling, is that
a daily exercise for you?
Speaker 4: Yeah? Let the dogs out and then it's pretty much
straight to journaling.
Speaker 5: And then I've recently been trying to do more journaling
in the evening actually.
Speaker 4: The close of the thought.
Speaker 5: I have a lot of trouble sleeping, and I wake
up in the night and I'm like, all these things
I should do, so right forward to bed. I try
and write down anything that's that's open ended here and
on theece of paper, and I got sticky nuts inside
of the bed. They're like right up to do to
do to do it that way, like okay tomorrow and Matt,
we'll get to this. I don't have to be thinking
about it. Let's me sleeps. It's a very strong tool
for me.
Speaker 4: Yeah.
Speaker 3: I think there were a non trivial number of times
where in order to sleep well at night, I had
to write up a whole email regarding a situation and
then just not send it. That's the important factor there. Yeah,
for sure, when when you're when when you're frustrated, definitely
don't click said.
Speaker 2: But you know, it's because those those thoughts are.
Speaker 3: Going around in your head, and it's like I'm constantly
focused on whatever that is, whatever problem it is, whether
or not I did the right thing in that situation,
and sometimes just I mean I know from a fact
now I have to terminate those threads, whatever the thread
is that I picked up and I started just getting
it done, whether it's writing an email.
Speaker 2: I use like every single kind.
Speaker 3: Of scheduling tool available, Like I will delay emails to myself,
I will snooze them, I will send myself future emails.
I'll use calendar events, anything to get stuff off of
my radar and into some sort of temporal delay so
that can come back to it later when I like
at the right time.
Speaker 4: Yeah, that's a big one.
Speaker 5: I talk about you frustrated and you write that email
when I get frustrated with something or possibly even a
Kylie or something like that right under stick, you.
Speaker 4: Know, and a month later it's a comic. So I
just you know, that's how I.
Speaker 5: Don't send it to them necessarily unless it's funny.
Speaker 2: Yeah, that's great.
Speaker 4: Right on.
Speaker 1: So cloud We'll come back to cloud Board Games. You've
got people signing up for it. Is it like, uh,
are you operating it like scheduled head to head matches
or is it a single player experience? What's the interface
look like?
Speaker 5: I like to do it with small teams, So I'm
thinking right now, we've got four.
Speaker 4: People in the hot seat.
Speaker 5: We're going to experiment with our initial match. So that
means they've got two events. Four people could be on camera.
You know, MIC's hot and well hopefully anybody else that
wants to be in the weightlist or whatever the gallery.
They can chat quietly. But like I said, there's I'm
going to do three tiers of it, and in the
future I might do four people sign up for Big Dinner,
four people sign up for Intermediate, four people sign up for.
Speaker 4: Advance or something.
Speaker 5: But you can kind of just hang around if you
did the beginner one, and you can listen.
Speaker 4: To the intermediate and the advance, but.
Speaker 5: I don't want the advanced people solving the intermediate one
instantly there. And so the way I do it is
there'll be a pre kind of let's just say it's
an hour long session.
Speaker 4: First fifteen minutes. You can get in there, you get your pairings,
you can I'll.
Speaker 5: Send out network diagrams using my pixelert network diagram software
that I use for my YouTube videos and ahead of time,
and I'll set up the terraform ahead of time, so
then people can get a feel for that.
Speaker 4: But then we get in there, maybe you set up
your dashboards.
Speaker 5: You know, you figure out your firefighting tools for the
first fifteen minutes, and then I do the brief and
I say, Customer service just called and said, you know
that we just sent out this is actually this is
probably my first one, so preview for those of you guys,
I don't know when this is coming out, but pre
view for the first advanced one. Customer Service shows up
and says, we just sent out a lot of emails
from our primary email address that people will get and
it's for a pharmaceutical add of some nature it's.
Speaker 4: Like that, and we don't do that. So what just happened?
Speaker 5: So and that happened to me? That did happen to me,
and it also has account, so you know. And then
after that, let's just say fifteen minutes to half an
hour diagnosing, So you got to do diagnosis first.
Speaker 4: And then from there they got to come up with
a solution.
Speaker 5: And I'm going to give re access to my aw's
account to anybody on there, but I'm not going to
give you a right.
Speaker 4: Access, So you get to tell me the solution. I'm
not never hearing anybody else right access to that.
Speaker 5: And then from there, last fifteen minutes after action report,
you know, I kind of get them peeped by the scene.
Speaker 4: What did I do?
Speaker 5: And how could you have improved the speed at which.
Speaker 4: You diagnosed the situation?
Speaker 1: So, oh, that's cool, that's cool. Are you recording these
for like a dish, like creating a YouTube channel where
people can watch it after the fact.
Speaker 5: Yes, I mean, one thing I thought about is that
one part of the needless thing is that there are
so many AI generated resumes out there. It's tough for
people software engineers, DevOps people to stand out and that's
that resumes.
Speaker 4: Also, so many people using AI codd.
Speaker 5: And I'm not against that, but you know, the coding
standard coding tests don't work so well. So this I
want them to record it, and I want them to
share that, be able to share this with perspective hiring people.
On the other side of this, if you're hiring people,
do you know that they're actually able to problem solve?
Speaker 4: Don't you want to kind of get a feel for it.
Speaker 5: Don't you want to see if they are collaborative in
nature or they're more like I'm going to just go
heads down and not tell anybody.
Speaker 4: Who I'm thinking, you know, you you you want to
get a feel for that.
Speaker 5: Maybe there's you know, there's you want one to the other.
Speaker 4: But this could be a really powerful hire tool.
Speaker 1: I think, oh yeah for sure. Yeah yeah, because then
you get like a little insight into what this what
it's like to work with this person under pressure, which
is everything that you want to know before hiring a person,
but have no way to test it.
Speaker 4: Well.
Speaker 5: Yeah, and then the flip side, it's a great way
to inoculate yourself to stress. I see so many developers
and the you know, the the CEO comes running in saying.
Speaker 4: We're losing money.
Speaker 5: We're losing money, and they're screaming and everything, and they're
just like, oh, they lock up, you know, And this
evacuated from the stress.
Speaker 4: You know we talked about.
Speaker 5: I mean, back to my gymastics career. I'm pretty good
at dealing with this type of stress because in my
gymnastics career, if you're spotting somebody and you mess up
that you could significantly injure or harm an athlete. You
know this, some rich guy loses a bunch of money,
but no one dies. And so if you put that
in that perspective, it makes it easier to handle the
stressing situation.
Speaker 3: I mean, sure, there's not a lot of opportunity to
practice emergency situations in yeah, attack right, and so the
only time we're in that situation is when we really
mess up. I mean, some companies do, like, there is
this idea in the industry, oh, you should, you know,
practice your run books, you know, databases being down, genal failures,
et cetera. But I think that a majority of companies,
the ones who need to practice at the most because
they're too much in fire fighting modes, are exactly the
ones who don't practice it. Yeah, so I mean this
this could be more than just a hiring tool. Realistically,
this could be like out of band practice for engineers
at any company to sign up and be like, hey,
you know, go through this equivalent you know, live training, right,
continue education on how to do these things. Uh, you know,
how are you going to respond in that situation and
then potentially be graded or get opportunity like feedback on
how they can improve.
Speaker 4: Yeah, I mean I could.
Speaker 5: I'd see myself being bringing this into a large company
and then looking at their infrastructure, setting up a drill
similar to their infrastructure and knocking it over and doing
trainings internally. I mean there's there's all sources of options.
I mean back to the kind of individual side. I'd
love to have a leader board and championships eventually. I've
got ideas for all sorts of fun stuff to do.
But yeah, there's I think there's a lot it can
be done.
Speaker 1: It could be like the next next iteration of like
e gaming, like the professional gamers.
Speaker 5: Yeah, you're actually not just I am a gamer obviously,
but you know, a little more productive than just beforetnite. Yeah,
so I think I saw will you're you run across.
But I can see the Crossted games for you know,
opspeakle for infrastructure people.
Speaker 1: Oh yeah, for sure.
Speaker 4: But let it.
Speaker 3: I'm sure no one ever heard me say this before.
Speaker 2: But you know, what I'm done with work.
Speaker 3: What I want to do is go home and play
another game, Like play a game that's exactly the same
as what I do for work every day, you know,
with with with high higher stakes potentially than what I'm
doing at work.
Speaker 5: Yeah, it's I'm hoping people do.
Speaker 4: That so far. I mean, like people are interested.
Speaker 3: Yeah, it's to be great.
Speaker 1: Yeah, And I think it's like, well, I'm going to
make that a little stronger statement. I don't think. I
know that there are a lot of people who are
early in their career and DevOps is one of those
industries that it's so hard to get into because you
can't get into DevOps without having DevOps experience, and you
can't get DevOps experience without working in DevOps. And this
actually gives you a way to solve that problem.
Speaker 5: As this Thursday's comic by the way, you have to
have five years of experience for the century level job
or whatever.
Speaker 4: How do you chick that egg?
Speaker 5: This might solve that problem though.
Speaker 3: You know, yeah, I mean, I think that's sort of
the same thing though, where like people don't leave the
university and go directly into executive role where realistically part
of I think one of the most important aspects of
DevOps and sort of understanding the connection of the business
and what you're trying to execute and running it and
how that impacts the technology and what it actually means
for uptime and providing that as a value add.
Speaker 2: To whoever your users or customers are.
Speaker 3: And I think, you know, one of the huge challenges
there is that's just not really something you're taught in
any level of education system. Like you don't go to
your chemistry class or your physics or you know, calculus
and be taught about how that provides value to end
users somewhere and how critical that is. I mean, because
if you were, then you could sort of maybe make
the jump there. But I feel like that's one of
the biggest things, the lack of experience, but also the
lack of attention to how to utilize those skills.
Speaker 2: In a real world environment.
Speaker 3: And I think that like the email sent out that
has a business impact starts to get like scratch that
it's not just that there's an emergency situation. It could
just be about a debugging experience, like how did this happen?
But understanding that there is a real impact to real
custom somewhere that someone will care and money.
Speaker 2: Is getting lost.
Speaker 3: I think that's the sort of thing that a lot
of people are insulated throughout their academic career.
Speaker 5: Yeah, that's the I see engineers making a lot of
decisions based on ones and zeros and not dollars and cents.
I like that tagline. Yeah, I see it happen, and
as a business owner, I have to take them into
account the bottom line. Recently, I had a major outage
for one of my clients right before the holiday season,
and we had a triage. We actually had to decide
to have a bunch of different domains. Kind of it's
not Copra, PEPs or anything, but think about how like
PEPSI owns Mountain Dew and it owns KFC or whatever,
it owns Taco Bebble, one of those mister PIB was down,
you know, was causing this to the traffics. Mister pip
was causing every other site to have problems that rippled
across the whole system, and we had the triage that say, listen,
we don't sell that much, mister pid shut it down.
We don't care, you know, so you've got to It's
it's interesting they'd do that in real time.
Speaker 4: While firefighting that that was a pretty exciting day.
Speaker 5: But same type of concept I see with the open
source software where versus paying Amazon. If you're a big company,
you've got thousands of people and you can save millions
of dollars, it makes sense to host your own cluster
of xyz. But if you're a smaller company and you
forget the engineers forgetting that, oh yeah, my man hours
are being wasted maintaining this thing and we're only getting
an ROI of one third of what my man hours
are spent. You know what they're paying me to maintain
this cluster, then what are we doing it for you?
So that's that's a very important skill that Yeah, you're
a don't teach in school.
Speaker 1: Yeah, for sure. I think it's something I've said on
this podcast that at least a billion times, like no
company cares about how tight and compact and efficient doctor
file is, and because they only care about what value
it's providing to the customers who are paying money to
that business. And I think that's the biggest career advice
anyone in this industry can learn, is your technical skills
are a tool used by the business. They're not the product.
Speaker 5: And to be clear, I'm not saying there's one right
answer for the other. You've got to take into account
the bigger picture and say, you know, we're a company
this size.
Speaker 4: You know, how much time should I be spending on
this or is that moving the needle for the company?
Speaker 5: You know, because you always got to pick what you
know it is important there So sorry I didn't interrupt you.
Speaker 4: No, I was done. That was it.
Speaker 2: I mean, I think this goes in another direction as well.
Speaker 3: And I'm sort of curious your perspective, Matt, is that
right now? I know there's a allegedly the job economy
is not great for knowledge workers, and I am I'm
curious if you see particular opportunities in some way or
really like, how do you stand out? What what are
there particular things that do help engineers in this area
or throughout the industry stand out compared to you know,
we talked about AI generated resumes or you know, other
aspects that they could really lay on any any thoughts
on that.
Speaker 5: Yes, tons, The first thing is that with I'm actively
talking to several companies that are creating large language model
lllms to basically replace meat and my designs so they
can just put in words and then it's spits up
terra form and it goes There's they're not there yet,
They're a long way from it be able to maintain it,
but they'll get there. So that means I'm out. That's
my whole job there. But just like elevator operators back
in the day, when's the last time you saw an
elevator operator?
Speaker 4: I think quite some time?
Speaker 5: You know, So that who wants to stand in a
hot box people's you know, inclosed space like that, especially
after COVID and everything.
Speaker 4: Oh my gosh.
Speaker 5: You know, so you no nowhere elevator operators, but they
need engineers to maintain those things.
Speaker 4: You know, you don't those things fall apart. That could
be real bad for you.
Speaker 5: So you've got to evolve. That's something that I can't
emphasize enough is to evolve and change your skills, you know,
holding on to that position like Okay, I'm going to
be there one typing in the code at the CSS
or pixel push and you're like, no, you want to
be a step above that and you want to find
that next step. So things I do to stand out,
I mean, I make sure they see my face. I
do a ton of content on stuff I try, and
you know, basically, I mean, I'll be honest with my content.
Sometimes I'm just reading in the news and that way
they see my face and they could recognize me there.
But that's something I personally had to get over. I'm
an introvert. Oddly enough, I probably don't see it, but
I introvert by nature, extrovert by necessity. So you know,
don't be afraid to make videos about stuff. Some of
my videos, one of my videos from As Always Bugs
with Me, I did a video where I was trilaborating
Wi Fi signals in between projects to track down printers
and different things, just because I was bored out of
my mind, and I put it on YouTube and it's
still one of my most popular videos. And I don't
do that and I have no cap so stuff like that,
you know, the communication, that's a huge skill.
Speaker 4: You need it.
Speaker 5: I mean, even if you're typing into an LLM, some
guy in a suit is gonna come up and say hey,
we need this bigger picture thing, and then you can
sure go ahead type of army of lms and have
it generated. But they can't do that prompt. There's values
to you doing that. So find a way to bridge
that gap between human and machine. It's just we're just
changing the way that works, you know. So find ways
to add value to other people. And that's one thing
I obsess over my clients. It's like, I'll help my
clients like source other developers. Now that this tool here,
my winners, I'm going to send rades to my clients
like you need a guy.
Speaker 4: You know, I'm a girl that need a person. Whatever.
Speaker 5: Here you go, this is this one just won, This
one solved my puzzle and two second and flat.
Speaker 4: So yeah, that's what I would do.
Speaker 5: Communicate, that's it, you know, you give them a wise
service there and don't communicate the same way everybody else's
There's a reason I use pixelart everywhere.
Speaker 4: People remember, I love my client's death.
Speaker 5: They gained as kids, you know, we're the perfectations of
the games of kids. They remember, you know, Mario pros
and so they see that and they remember, so standout
and and and communicate quite a bit.
Speaker 3: I love the perspective on spend some time creating content
because you know, it really does resonate with the there's
only so much you can do of practicing lead code
interviews or learning a new programming language, but you know,
really take that next step of doing literally anything else.
You know, if everyone's focusing on tech, focus on creating
content or making some artwork or you know, using l
MS or whatnot, or hand drawn or you know, making comics,
because that is some way that you can really stand
out you can put on your resume. Oh yeah, and
also all of these are me, right, you know, this
is something about me that really stands out that's unique
for who I am compared to everyone else who's trying
to compete for the same job. I think right now
it's it's I think this is always true. It's always
now is the best time to say, even start your
own company. I see like a lot of resumes are
like practice projects by engineers who are interested, like, oh
I made a recommendation engine or I made this other thing.
And the most important question I always ask is why
did you do that? Like what was your end goal here?
And it's like, oh, I just wanted to do a project.
Speaker 2: I'm like, Okay, well that doesn't really mean anything to me.
Speaker 3: You had no bounds, no restrictions, no constraints on what
you were doing. You just went wherever you felt like it.
What's way more interesting is, oh, I was actually trying
to make a business, or I was actually trying to
automate my my pictures being uploaded to LinkedIn or whatever,
and like I had a real problem that I actually
needed to solve, and this is how I went about
solving it. That's a way more interesting thing to hear
as a potential interviewer. So you know, the content's great,
the webcomics are great. You know, anything that really makes
you stand out as an individual.
Speaker 5: Yeah, Another thing that pops into mind is he here
the t skill set.
Speaker 4: You know, you don't want to be a generalist. You know,
if you can.
Speaker 5: Find a niche, a niche that's in demand, you know,
then the volks and I think my aws.
Speaker 4: Niche for my consulting businesses too wide.
Speaker 5: I'm trying to figure out how to like zero that
and like maybe I'm just going to do whaff you know,
or now they've got these trainings, I'm probably going to
focus on that.
Speaker 4: But you know, I was thinking.
Speaker 5: About just being the WHAFT command or web application firewall.
Speaker 4: Can I fight d DOS attacks all day long?
Speaker 5: And I could do a whole course on that, and
Weld teach a whole course on that. So but yeah,
find a niche that's in demand and dive in. And
I've got a discord full of hotepodge of brand developers
and they had these really cool projects, but they that
are niche and they just don't showcase them.
Speaker 4: They tell you about it in Texas like.
Speaker 5: Yeah, cameras everywhere your computer, you can grow the screen
just screen recorded and show this cool project, you know,
off as much as you can.
Speaker 4: Yeah.
Speaker 1: Yeah, it's definitely we're moving from like a technical marketplace
to a creator marketplace. And it's I think what both
of you are saying really highlights that. It's that the
technical skills are the foundation, but then as the creator,
you have to to showcase and and market those to
make you stand out and then identify like here's the
thing that I do. Like on the te like you've
got the top line of the tea here where everyone
has the same skill set, and then here's where I
go deep.
Speaker 5: M Yeah, I'm not saying don't you know, don't learn
things outside your purview, but don't make that your public
facing persona.
Speaker 4: You know, you I know how to do a lot
of things. I could do.
Speaker 5: Word press anybody ever, but you know, stay with I publicly.
I have very particular thing I talk about, you know,
And you've got a very short amount of time to
get people's attention right now with you know, all the
spam and garbage out there. So you've got to figure
out a way to do that very quickly, and and
find people that have your problem, not everybody. You know,
I solve a million dollar problems for some of my clients.
Not everybody has those problems. You've got to find the
people that have the that you are the ass into
their headche you know. So so yeah, be very aware
and don't just cast a wide and that and try
and pitch to everybody. Find the businesses that are suffering
from your stuff. I mean, I've got criteria that I
use for lead gien where they are the running at
as great. You know, I have a very specific size requirement.
I don't want to be too big. I've got an
HR department. You're probably bigger than you don't need me.
And then if you've gone down recently, then I'm talking
to because you've you've got the pain you've like if
your site's slowing down because you're growing too fast, I
want to talk to you right away. You want to
talk to me. So if you have that criteria, then
it makes it so you're focus. You're not just you know,
get being disheartened because you sent you know, a thousand
resumes out to Glamo Joe. Is that you know it
doesn't have you know, doesn't have the need that you
can scratch.
Speaker 4: You want to be like, this.
Speaker 5: Is your problem, this is my this is my expertise.
This is how I can solve that problem. Is how
I've solved it. Hunderd times case site. It's a big
thing I'm working on. Issue is getting more cases on
my website. Show how you how you solve that exact problem, right,
put a coin and the I'm gonna keep going.
Speaker 3: Yeah, definitely, I know. I'm Now I'm gonna ask you
another question, uh, like the sort of issues that you
see come up for each of your customers and from
your experience, do you feel like they are sort of
repetitive in any way, like, oh, you've seen this problem
before or do you see that some of these problems
like they are actually nuanced and new in some way
that you haven't experienced.
Speaker 4: Yes, and no.
Speaker 5: The the people problems I see, Uh, that pops up
point a bit. So I don't do a lot of
hands on work for my clients. I do advisory overside
quite a bit. So they paired me with the project
manager and they say, we've got this problem we want
we want to scale up to this thing. I do
a design and then I work with PM to hand
that off to other people to do so I trained
other people. I see a lot of people problems that
are repetitive. The technology problem, yes, for nine times out
of ten is.
Speaker 4: A security group problem. That's true.
Speaker 5: But with that said, past that, there's always new technologies.
We're getting detossed in ways I've never seen before. It's
it's quite fascinating. Actually, I got this, if you guys
will indulge me, I've got one kind of more prop
like the game that.
Speaker 4: You guys might find interesting. Real quickly.
Speaker 5: Sorry, I'm going to digitize this. I don't know the spinners.
I've lost the spinner. Oh well, so you said you
talk about the repetitive stuff. When I had some mid
level junior devs. A lot of times they'd get stuck
on things and they'd be like, I don't know what
to do.
Speaker 4: I don't do they come up to me and and
a lot of times it became repetitive.
Speaker 5: So so I took the top like ten or twenty things
that I don't know.
Speaker 4: I mean, you think that.
Speaker 5: Happens, including chef the security group, you know. And I
put a spinner on this thing, and I said, you can't.
You can't bug me until you spun it at least,
you know, I think I said three three times to
try three different things on here, Like you got to
tell me you've already checked the logs, you've already rt FM.
I'm not going to explain that abbreviation, but yeah, add
more logging, you know. So this is actually coming soon
to h to cloud. At war games, they get stuck
spin for the beginners take us away from the advanced people.
Speaker 2: So you've got to free spind you know, spend Yeah, exactly.
Speaker 5: A phone a friend as to ask your peer before
you come asking me, like another junior might actually have
this lead to work together collaborate.
Speaker 3: People and not just in the same company, you know,
I find that, especially above the senior level. A lot
of engineers haven't realized that they can go outside the
company to solicit assistance, Like you don't need to pay someone,
but there are communities out there that you can go to.
And it's so ridiculous to me that at like SMBs,
like small medium businesses, like not giant modelists, you know,
the fang companies, et cetera. You know, at smaller sizes,
you just don't have anyone that's more experienced than you
and the company. You have no one to mentor you
no one to sort of bounce ideas off of. And
they're like, well what do I do? And I'm like, well,
how many how many discord servers are you in? Like
I'm like in thirty or forty or something like. There
are those out there where people you know, are willing
to provide assistance if you just ask the question publicly.
And the idea that they're just not even they don't
even see this as an opportunity is like, like really
great additional source of valuable information that you're missing out on.
Speaker 5: Yeah, I mean, as part of the reason I am
starting a community for my niche is that there's there,
absolutely people are you know, where do I go?
Speaker 4: Where do I go?
Speaker 5: I love that I've got these communities I can go
to about my business and all the other stuff. But
if you if you're an engineer and you know, for
a smaller company at least, and you're running on Amazon, like,
there's probably no one else that knows more than you
and you need answers, and it's like, yeah, you can
take a course, you know, or take a bunch of
tests and get certified, but it's almost better to have
that mastermind group, that community you can you bounce off.
There's there's some additional value to it. So that's with
all the AI going on, you can ask chat, youpt
all day, you know, to have somebody else that actually
breathes and has a pulse that understands your pain. Even
sometimes if you just event to about like ah, it's
the security group for one hundred times, why did I
make that?
Speaker 4: I say again, you know, so I completely agree with you.
Speaker 5: I think that's a huge value matter if you if
you're not in one of those, you know, I would
strongly recommend it.
Speaker 4: Yeah.
Speaker 1: Yeah, And it seems like cloud Ward Games is going
to be like a foundation for helping you identify those
groups because you get in, you start working with people
that you don't know, and then you just start chatting
from there. It looks like those are just going to
naturally form as a result of this.
Speaker 4: I would love that.
Speaker 5: I hope, I hope there's some because you get some
good relationships that are formed in foxholes.
Speaker 4: So to say you're in the fire there, it's like
you know, if if someone.
Speaker 5: Cracks in the pressure, you know, but I've got some
guys that I'm pretty tight with because we've splitted it out,
you know, in bad times and you know that there's
there's be amazed how quickly you forged those relationships.
Speaker 4: Sorry, hopefully the audio is still good.
Speaker 3: Well, I mean, the the worse the situation is the
more interesting and honestly the thing that you can sort
of look back at later to sort of enjoy it.
The better the story it is that you can, you know,
walk through like, oh, there was these decisions that were
made and there were problems there and then and then
you're able to tell that story later, Like it's a
much better story if everything just worked right all the time,
Like your job would be boring and you would leave
after when you retired, be like you didn't do anything,
but the fact you can look back and be like, oh,
like there was one time the website was down because
someone did this and it took forever, Like that's a
great story that you can tell.
Speaker 4: Yeah, yeah, I tell my parents' stuff.
Speaker 5: My mom was like, you know, I don't know how
you can handle this.
Speaker 4: This sounds so bad, Like.
Speaker 5: Well, if this stuff didn't go down occasionally, we didn't
get tired, we didn't get attacked like this, then I
wouldn't have a job. So you know, it's kind of
like that's there's Yeah, I find these kind of outliers.
The problem is you design a system that's so solid
doesn't go down, so then they don't need you anymore.
But luckily the internet is full of evil people attacking us,
so it gives me a job.
Speaker 4: Sorry, that sounds weird to say that.
Speaker 1: It's always opportunity somewhere. Yeah, so you've been you're literally
just a week indo cloudboard games, right.
Speaker 4: Less than a week.
Speaker 1: But yeah, yeah, what was the trigger that made you
decide to to start this?
Speaker 5: So I had thought of it as a side par
for my kind of community I was creating.
Speaker 4: I wanted to help.
Speaker 5: I'm trying to my mission try and finds many people
that I can add value to their lives. And you know,
some of my big clients, you know, that's that's great.
But what if a mom and pop, tiny little shop
comes to me and needs something.
Speaker 4: I can't offer them as much one on one time.
Speaker 5: So I was like, I'll build a community and then
they can, you know, share information just like we've been
talking about, and I can help more.
Speaker 4: People and it works out for all of us.
Speaker 5: But then I was in the Mastermind group and I
was like, one of the events we're going to do
is the wargaming and they're like, no, no, no, that's
the front and center thing, like that's I can see
companies hire you just go in and do war game
through all these other things. And then they got me
hyped up and I'm like, I'll have it launched. I'll
had it launched by tomorrow. I said, I'll have it.
I'll have a post, I'll have a video, and I'll
have a landing page. And you know, I tied my
hands because I in the same way with I was like,
no coding, okay, you can't, no over engineering. Luckily, I'm
a orrible designer, so I was like, I'm not going
to design, I'll just use a template, and I set
up just I'm convert kit dot com this landing page
and I used calendarly to schedule the first couple and
I just wrote some emails there so there's no coding,
because I do have a tendency to over engineer if
I allow myself. And so here we are Monday morning,
and I had.
Speaker 4: Five people signed up, and now it's doubled already.
Speaker 5: It's overdoubled since then, so it's moving and now I'm
on this podcast.
Speaker 4: So yeah, it was just a masterminder. They focused me.
It really was going to cut away the other stuff
and focus.
Speaker 1: Yeah, for sure. That's like another benefit of that is
having people who will well, like you said, just focus you,
you know, because we tend to over engineer it. And
you think, oh, man, I've got to have a website,
and I need a database to store the people's information
and get their profiles, and I need authentication. And but honestly,
by having someone hold you accountable to it, you have
the VP in less than five days without having to
do any of that stuff that's not actually gonna test
your idea or not.
Speaker 5: Yeah, and I will say there's also some other factors here.
It's a hot dog business. Being as I had all
these other ingredients just sitting around, Like, I already have
open source terra form scripts I manage, and I used
to run my entire infrastructure and some of my client stuff,
so I already had that. Next so I'm like, oh,
I could spin up a domain that we can knock down.
Actually that box explodeme dot com yesterday, which is gonna
be attack, so I can spin that up. I didn't mean,
it's not quite ready yet, but I can have it
up in a few hours. And then all I gotta
do is take some of these scenarios that I've got
written down, make them more formal, you know, figure out, okay,
this is what I'm gonna do phase one, phase two,
and I will put some hints in there and stuff.
So I already just had these pieces of the puzzle
ready to go, and that they were just gathering dust.
I'm like, all right, let's kind of.
Speaker 4: Like hot dogs.
Speaker 5: There's a bunch of other pieces of meat that you got,
the little leavings that you put them together, and now
you got hot dogs in Bologne or whatever else. So
that was uniquely that I was lucky enough to have that, fortunate.
Speaker 4: Enough to have that.
Speaker 1: So right on, how long have you been in the
Mastermind groups? How long have you been doing that?
Speaker 4: Yet?
Speaker 5: On and off throughout the years, But I over the
last two years I really.
Speaker 4: Went edged down.
Speaker 5: I hired a bitch business coach and I think it
was twenty twenty two. They didn't quite work out, but
then I found this solo expertise one in twenty twenty three.
Speaker 4: At the beginning of that that was great.
Speaker 5: I actually ended up engaging the founder of that for
a one on one coaching and we met Yesterday's phenomenon.
Speaker 4: And then I threw a parallel I found a parallel group.
Speaker 5: That was more hands off, small best dot com that
then I found another person that was run another Mastermind
group that was more hands on, and I jumped in
with them, and I think I almost mentor them as
much as they mentored me.
Speaker 4: Luckily at this but I mastered my group. Is So
that's that's been.
Speaker 5: That was about nine months ago there, and I'm looking
for more and I'm probably gonna start another one in
the next few weeks, well maybe.
Speaker 4: Months, I just say next few months. I'm gona finish.
Speaker 1: Work right, cool? And then you're your cartoons that you
release Where do you release those at for anyone who
wants to see those?
Speaker 5: I have my mailing list, my socials, and then I
push them. If they're relative to terrorform, I push them
into terrorform.
Speaker 4: Groups here and there, our slash terraform, they're relevant.
Speaker 5: Our slash programmer humor. That's where I launched a bunch
of them. Those are the ones where I get the
most views.
Speaker 6: Yeah, yeah, And there was one week there was Yeah,
there was, there was one that was relevant to something
we said earlier that that just came out last week.
Speaker 5: But I don't think I can connected reads anymore.
Speaker 1: So the one you sent that cracked me up was
the Y two K one because I like, I know
people that in nineteen ninety nine they went into bunkers
and I've never seen them since, and so pretty frequently
I wonder like, wow, did they because some of them
were going to stay in their bunkers for like ten
or twenty years, and I was like, did they stay?
Like are they still alive? What happened when they came out?
And I don't know where any of them are?
Speaker 4: Yeah, I got to think and I was this could
be an annual one.
Speaker 5: Because this is actually I released on the thirty first
of December.
Speaker 4: I got to think of what's the dest thing that
you could go back into a bunker and live your
life and hiding for JavaScript everywhere?
Speaker 7: What they had a little bit of the controversy to it,
because you know, of course, as like half the internet
hates JavaScript and half the internet's like JavaScript, let's put
it in everything right.
Speaker 4: That helped it definitely helped it go up the numbers
for sure.
Speaker 3: I mean, if people I know about this as a controversy,
and I just I wonder if the people who are
against JavaScript will be more in favor or against whatever
LM generated programming language the whole world ends up utilizing
in the future. I mean, we'll be forced by our
robot overlords to use the one true programming language, whatever
that is. And I wonder if they'll be okay with that,
or whether or not they'll have wished to go back
to the days of JavaScript and everything.
Speaker 4: You just give me an idea for a camera.
Speaker 5: We're arguing about JavaScript versus not JavaScript, but what about
we can all agree that people that use no code
solutions are fools.
Speaker 1: I think I probably get pinned in the anti JavaScript
crowd a lot, and I'm not anti JavaScript. Like the
point I try to drill in there is the JavaScript's
not always the right answer, you know, Like you don't
hire a carpenter based on the type of hammer that
they use. You hire them based on their ability to
build a garage or a house or a fence or
whatever you are. And I think JavaScript, to me is
a tool, and so JavaScript's not always the right answer.
So it's not that I'm against JavaScript. I'm against saying
JavaScript is always the answer.
Speaker 3: Yeah, I mean, I was definitely in my life on
the like they're just all the program languages are tools.
But you know, I've definitely come more around to the
fact that every single one of these tools has some
specific quirks which change the utility in the solution space
that we're in, and so it actually is more important
that we evaluate whether or not this tool, this language,
what it implies, how it works, who the maintainers are,
what their mentality for the long term is, for the
type of solution that we're building. And I feel like
we're still very much stuck in the fact of whether
or not people sort of like the tool itself, Like
I like cameras, I don't like cameras, rather than the
company that manufactures the hammers is who we should be evaluating.
Speaker 5: Yeah, yeah, from from a my thoughts on jobascript from
a business standpoint are.
Speaker 4: It's probably a little data now.
Speaker 5: But back in let's just say the mid twenty tens,
I can high if you're gonna if you're gonna write
PHP and have a website, you know you're gonna need
to have JavaScript in the browser. Right, that's just you
can't run PHP in the browser. Same thing with Java,
the same thing with a lot of things. So a
lot of these developers that I was finding had some
amount of jabascript background, and I found that you can
repurpose you know, that little bit of JavaScript in the
browser to run server side if you wanted to, much easier.
So it was more of a cost efficient thing. I
could a PHP dev that's full stat and nike them
a no JS dev without breaking the bank on that one.
With that said, now, with AI being a big thing,
it would be silly to try and use to run
their own.
Speaker 4: Networks in JavaScript. I know because I've tried it. Probably
not a good.
Speaker 5: Idea, So but no yeah, you can you So you're right,
you got to pick the right tool for the job.
Speaker 4: But it was strictly on like a who could I
you know, we can.
Speaker 5: Cast a wider net and hire people that have JavaScript experience,
and you know, so we can get a wider range
of people coming in, better applicants and ideally a better outcome.
Speaker 4: So that was that's that business. You know, numbers, you
got to kind of do that calculus as well. Not
the right tool for AI, not the right tool for
a lot of things.
Speaker 1: Yeah, but it might be the right tool for your
business because you just need a versatile programmer who can
get shit done.
Speaker 5: Yeah, if I'm hiring for the top Python dev, a
lot harder to figure out how to hire those guys.
Speaker 4: Than just you know, top job of death for sure.
Speaker 1: So for people who want to go check out Cloud
war Games, cloud wargames dot com.
Speaker 4: Cloud wargames dot com. Yeh, that's that's where you find it.
You can find me the time arm cloths. That's kind
of tells you all the different.
Speaker 1: Things I was trying to I was trying to tee
it up to let you plug the website, but then
I just plugged the website for you and stole the
answer out from under you.
Speaker 4: I sure did.
Speaker 3: For for the release of the episode on the website
adventure indevops dot com, there'll be a bunch of blurbs
along with whatever links you want us to post there.
So if you want to just respond to us after
the podcast, you know, all of the things that we've
talked about in this episode.
Speaker 2: For anyone who's interested, well, I'll be listed there.
Speaker 5: Sure, okay, well yes, Cloud wargames dot com is where
you can find that stemmatical dot com that's the worst
schematical al at the end is where you can find
most of my writings. Matical dot com and soon. I
haven't announced this publicly. It is the first time announcing it.
I actually just in the last three months of my
life building a video course for O'Reilly's the publishing company,
an on demand video course that very they wanted me
to say on demand on themand video of course for
o'riiley Publishing, and so that will be live hopefully by
the next two weeks or so, may go at the
time this comes out. So if you want to learn
zero to Hero and a w security and animated guide
to security in the cloud, then we'll be there.
Speaker 4: So if I'm overplugging this, feel free to cut it.
Speaker 1: But now I'm super interested in this. So now this
is an animated course on a w security.
Speaker 5: Yes, yes it is. Actually if if you want, I've
got a diagram here that I queued up. Okay, we
can cut this if you want. But if you guys want,
I'm actually we can play a micro round of cloud war.
Speaker 4: Game right now.
Speaker 1: Let's do it.
Speaker 5: Can I present here? Share a screen? Share screen this
right here? You guys tell me if you could see it?
Speaker 4: We can.
Speaker 1: Now anyone listening to the podcast can, so I'm gonna
narrate it for them.
Speaker 5: There, Okay, there we go. All right, we have our
infrastructure here. This is a network map, all right, and
we've got to use it right here. So this I'm
gonna go as fast as I can because I know
we're I want to be respectful.
Speaker 4: The timement. A request come in.
Speaker 5: It's the application load balance, or it's the it's the
application server. Then you're shopping cart, so I should set
the tone. E commerce store. You run an e commerce
store here, they're shopping cart stored and reddis so they say, hey,
I want to buy this thing. It goes into reddis
it goes back to the application layer, and now they've
added it to their car.
Speaker 4: And let's just say that they actually go.
Speaker 5: To make a purchase. It's the application server. We're doing
a vendor of an architecture. Maybe it's not the right one.
All right, We're gonna fgure about the animation for now.
The vendor and architecture pumps a kinesis that updates the
database it purchases made and updates the inventory management software
and updates the product worker which charges them and updates
the nail service and all that stuff. That All right,
I'm getting to all the details here.
Speaker 4: Here's the scenario.
Speaker 5: Okay, Customer service comes to you and says, we are
getting reports that people are missing stuff in their cart.
They they're adding stuff to their cart, and all of
a sudden, when they go to check out, it's not there.
Speaker 4: They go to another page, it's not there. All right,
what do we do?
Speaker 1: So I'm gonna highlight this real quick. We're looking at
an animated diagram of an AWS environment, right and is
this part of the Cloud Wars games interface?
Speaker 4: It will be, it's not.
Speaker 5: Actually this is running locally right now, because I've never
bothered to put it on a website, but it'll be
up eventually.
Speaker 4: I'll use it locally if I have to.
Speaker 1: For now, right on. Okay, so there's stuff's missing from
their cart. I'm assuming that the cart is persisted somewhere
in some data store somewhere.
Speaker 4: That is correct, It is in reddits right on?
Speaker 1: So do we have a way to check the logs
on reddus or check the whatever's writing logs to reddus.
Speaker 4: It's running out elastic cash. Elastic cash does have events.
Speaker 5: I don't know if you get directly to the logs there,
but you can log into reddis and run some stuff.
You've got metrics too, don't forget you've got you do
have five watch metrics.
Speaker 4: Good, I guess.
Speaker 3: The first question is, like, is the data in the
database where it's supposed to be if we're using reddus
like Reddit commander or something to tell us whether or
not the items are actually in the shopping carts that so.
Speaker 5: You go in there, you've got's call it one hundred
thousand keys and you look at it.
Speaker 4: There is data in there.
Speaker 5: You can I wouldn't do this step for you normally,
but I'm going to do it just because I'm speak
things long you go in there and you try and
add thinks to your car. Of course, you add things
to your car, you check yourself key against it, and
your stuff is.
Speaker 4: In the cart.
Speaker 3: There does the cart for the customers complaining? Uh, are
you able to have a session for that that you
can track the check to see if there are actual
items for that session in the cart.
Speaker 5: So this is this is where I want to go,
like a five ceented diet. You have a technical minded
customer or not?
Speaker 4: Okay, you just rolled. You have a technical minded.
Speaker 5: Customer, they can open up their browser and copy the
cookie for the session and send it to and so yes,
have their session.
Speaker 3: I mean I always assume they're not a technical customer,
and so you try to get them to purchase some
fake item in the cart that only only their session
will will have access to. This is a magic r
L right that allows That's brilliant attracted.
Speaker 5: Yeah, yeah, I love that if yeah, I was just
being nice because we don't want to be for the
second time.
Speaker 4: But yes, I love that solution. This is the type
of solution I didn't I wouldn't have thought of that.
I would have thought this is the type of stuff
you can get with these seven games.
Speaker 3: Well, we we gotta we gotta debug this stuff for
real all the time. At our company, I mean we're
not e commerce, but uh, you know someone we we
do log in and access control, and so it's always
a point of contention like why didn't this user have
access to this resource at this particular moment, and so
figuring that out, you know, requires a bunch of things.
So we have ways to give our customers information that
they can inject into their environment when they're performing the
sort of investigation that we can see on our side,
so that we can highlight exactly which traces make sense. Otherwise,
you know, you're looking at all of the data for
every customer in order to narrow that down.
Speaker 4: Yeah.
Speaker 3: So I mean you said technical customer or some magic solution,
and are we getting their items in the cart?
Speaker 2: Their session busted?
Speaker 5: You checked the session against the reddest statvasion. You have
the session key, there, items You're gone, not there there,
there's nothing, There's nothing but that.
Speaker 4: Y key's gone.
Speaker 1: So you mentioned elastic cash. I'm assuming that wasn't just
randomly throwing out words where's that at in this ie pline.
Speaker 5: Running on a US Is the lastic cash just don't
have the last cash service that runs value key stores. Okay,
I got a couple of different ones, but I picked
reddis this is my go to.
Speaker 1: Gotcha interested? So they put something through the browser in
the cart, never made it up, never made it into reddish.
Who's responsible for sending it to reddis to clarify?
Speaker 4: So just again, I'm giving you way too much here.
Speaker 5: I never said it didn't make it into reddis in
the first place.
Speaker 3: Just got deleted instantaneously, Like you know, by the time
it gets to the cart, by the time you look
at it, it's no longer there. So there could be
some sort of TTL on the items that are going
into the cart, or some other process that's eliminating them
from redus, or you know, of course not getting there
in the first place.
Speaker 4: M yeah.
Speaker 5: So yeah, So just to speed us along, I'm gonna
drop what I gives a hint uh that you do.
Speaker 4: Have cloudwashed logs for uh for reddas, so you could
ask me what they are saying.
Speaker 1: Well, what are the logs saying?
Speaker 5: Number of items is pretty number of items went up
the flat lined. Okay, see if usage fine, number of
evictions is up quite a bit.
Speaker 1: Evictions, Oh, so they they got evicted.
Speaker 5: Yep, and I guess the usual memory drop. So again
I'm gonna simulate this. Yeah, I wasted one of my
real juicy ones, but I guess it's probably good here.
Speaker 4: So yeah, so what what's the what's the solution there?
Speaker 1: So I don't use reddus a whole lot. So obviously
it seems like reddis evicted for some reason because it
either has like a max number of key values that
it can store, or it ran out of space.
Speaker 4: Yep, it is correct, it just runs out of dat.
Speaker 5: You can set aviction policy, but in this case, I
think the default is to just start evicting the older ones.
And so that's what happens, is they filled up. Maybe
it was a big shopping day, you know, only you
didn't expect this much. This many people put stuff in
the cart and so it filled up. Or I mean
it's again I'm gonna just kind of speed through this.
But possibly you could look at it and some change
made the payload of each key too big, you know,
So we added something and now we're cashing like this
whole structure instead of just the product ideas and quantities
were catching the metadata and the products.
Speaker 4: For some reason.
Speaker 5: Somebody actually messed up and put that in there, so
and now it's just bloated.
Speaker 4: So the fix would I mean, I get you.
Speaker 5: I'll let you guys present the fix if you guys
have anything there and to the problem.
Speaker 1: I think the emergency fixes increase the size of reddish.
Speaker 5: So that's that's probably what I would do, and then
the long term fix would need to fix the bug
that Adam.
Speaker 4: Catch all that so actually happened once actually happened. They
had a cash a bunch of HTML in there.
Speaker 5: That was like, I'm like, what this is adjacent objects
with HTML and it like what what are you doing?
Speaker 4: Your stop cashing that stuff in this thing. But it happens.
Speaker 5: So yeah, all right, congratulations after action report and so.
Speaker 4: On and so forth.
Speaker 3: So well, yeah, I mean this is great. I'm actually
sort of curious on the scenario. Is using reddis as
a temporary store for the cart common pattern in e
commerce stores.
Speaker 4: At early stage. Yeah.
Speaker 5: So a lot of my clients, you know, I love
the death clients, understand they you know, they start off
with something simple like WordPress and they start slapping more things.
Speaker 4: That not the way I would do it. Okay, not
the way I would do is. I mean, there's there's
a lot of data you can extract from carts. It's
not ephemeral data. You want to know.
Speaker 5: Who's adding what when, where's you can send a reminder, emails,
all stuff.
Speaker 4: But a lot of times with.
Speaker 5: Dev's, a lot of times it's the stuff everything in
the session and then prey.
Speaker 3: Yeah, I mean I get I get the perspective of
knowing that it's temporary data. It's not the order hasn't
been created yet, so storing it in a data store
that may not be persistent. I also get the idea
that people believe that reddis is this non ephemeral data
store and you know, go ahead and use it as
your source of truth database. But you know, in the
mature businesses, as you said, like you you actually don't
even want to throw away that session data.
Speaker 2: As soon as you have it, you might as well be.
Speaker 3: Persisting it for for permanency so that you can evaluate,
you know, why didn't that convert to an actual order?
Speaker 2: You know, why didn't that person come back?
Speaker 3: And then once they do actually click by h and
payment gets processed, you don't need to convert it from
your temporary data store into into a permanent one.
Speaker 2: You already have it there. The payment is sort of separate.
Speaker 4: I'm trying. I'm actually trying to get you to exactly
what you're describing for my the video I did on
Amazon Glue and.
Speaker 8: You know, putting those that data in a long term
coal source, basically telling the people who difference between source
of truth ephemeral and long term term you know, red
shift or big data analytics tools, data warehouse, data leg
type of situation.
Speaker 3: So this is one of your videos that's going on
on demand on O'Reilly or no, that's.
Speaker 4: My YouTube videos.
Speaker 5: The Riley videos are specific to the security course, so
you can see if you have a bunch.
Speaker 4: Of that in here. But I see, all right, I'm
gonna stop sharing sometimes.
Speaker 5: No, I didn't prepare that one.
Speaker 2: Will has gone Will has gone dark for me.
Speaker 3: He's got no camera on's he's hiding from the from
the public and anonymous. Yeah, I mean, I think, you know,
I think it's a good point to maybe wrap up
the episode, unless to anything last word you want to
share about cloud war games or your consultancy.
Speaker 5: No, I want to thank you both for being the
first participants, and I mean's somewhat of an unofficial. It's
somewhat an unofficial capacity there, but thank you both for
being the first participants. I mean, we kind of played
a micro around there, but I invite you once I
get the thing moving, I invite you guys to come
and participate in. If you guys are game, maybe I
come back and run a full length game with you
guys some of that nature.
Speaker 4: So it's s throwing that out there.
Speaker 1: If you're interesting, that'd be a blast and I would
love to have you back on at any time for that.
And there's a bunch of other topics we we like
touched on here that I think would be cool to
dig deeper on.
Speaker 4: Yeah, for sure, be my pleasure.
Speaker 5: You let me know what and I'll you know, another
guest drops out, you come find me.
Speaker 4: With that.
Speaker 3: Should we move on to pickswell we should lop.
Speaker 2: Ah man words.
Speaker 1: Yeah right. Just like to take this opportunity to point
out to everyone that English is my native language and
I still screw it up.
Speaker 5: You should see me editing my videos. Love swearing.
Speaker 1: Well, for sure. I used to do quite a few
YouTube videos and did some video courses and stuff, and
my wife would always give me a hard time. She
would say I can always tell how it's going by
how many cuss words I can hear coming out of
your office. But at one point I made like a
blooper reel where I just took all the cuss words
that I cut out of all my videos and it
was like an eight minute stream of just some of
the most offensive language that has ever been recorded. Yeah,
all right, anyway, on the picks, Warren, you brought it up,
so I'll go first. I'm picking. A couple of weeks ago,
I picked Kunk on Earth, the Netflix special, and I
think that was when we had aj Funk on and
he said that there was a new one called Kunk
on Life, and so I watched that, and I thought
Kunk on Earth was so great that there would be
no way a follow up to it could be as entertaining.
And I'm happy to tell you that I was wrong
on so many levels. So Kunk on Life. Go check
that out on Netflix. The just the sense of humor
there and the dryness of that sense of humor had
me just rolling on the floor laughing, just absolutely amazing,
and they accidentally made it educational as well. So go
check that out.
Speaker 4: Well.
Speaker 3: I definitely will, because I liked all of the other
series by Filamina. Yeah, I guess that's her her stage
name and not her not.
Speaker 2: The actress's name.
Speaker 3: Okay, mine is I guess I also have a show.
So I read the book's Three Body Problem by Leo Sheen,
and it's actually one of the scenarios where I actually
like the series better. I think the series is absolutely fantastic.
I think some of the books are good, but the
series like absolutely really well done.
Speaker 2: I really look.
Speaker 3: Forward to the fallow Up seasons. If you haven't seen
it or read the books, I highly recommend a lot
of good philosophical ideas that.
Speaker 4: Are brought up.
Speaker 1: Okay, give me the high level of what it's about,
because I'm just assuming from the title three Body Problem,
this is how to get away with murdering someone, but
I may be wrong.
Speaker 3: No, it actually has to do with the unsolvability of
finding out without the initial conditions, what the path and
orientation of a solar system with three masses in it is.
Speaker 1: Okay, that's not going to be as useful to me
as I thought it was at first.
Speaker 3: See yeah, without without without spoiling, without without spoiling really
that much. There is one of the very close solar
systems in our galaxy to the milk in the Milky
Way to our solar system is one with three stars
in it, and so there is a relation's science fiction,
really good philosophical dilemmas, absolutely fantastic, well done.
Speaker 1: Is that the Alpha Centauri system that you're referring to?
Speaker 4: Yeah? Yeah, because they.
Speaker 1: Originally thought it was one star and then found out
that it was two binary stars in orbit with another
star close by.
Speaker 3: I think, yeah, yeah, yeah, it's four light years away right.
Speaker 1: Just across the street.
Speaker 2: Yeah, So, Matt, what do you got for us?
Speaker 5: Yes, you got all the science science stuff.
Speaker 4: I did not come prepared for that.
Speaker 5: I got the comic book Scott Pilgrim Versus of the World,
a lot of Yeah, they do a lot of pixel
aart in there, and I've been doing and wrapping on
my desk, and I've been doing a lot of comic
book stuff, so I'm like, Okay, this would be a
good thing to pick up. And I'm trying to look
at screens less before bed. So I've been uh, just
reading through this lately. Yeah, not not quite as not
quite as big as as interesting.
Speaker 4: As as the three body problem, but it helps me
go to sleep.
Speaker 2: What was what was that last thing that you held up?
Speaker 4: Scott Philbrin versus, The world was just uh that's volume one.
Speaker 3: Vollime, I say, I didn't know it was a written
work of art.
Speaker 4: Yeah, I believe that's how it started. And then I
think they made the video game and then the movie
so interesting.
Speaker 9: But the video game, it was the guy's the dark
for that guy named Paul Robinson, I think the probas
G or whatever, and he's just a phenomenal pixlartist, and
I just I.
Speaker 4: My work doesn't come close to it. And I got
a style that I can do.
Speaker 5: A two seconds, but he spends like years building these
amazing animations.
Speaker 4: It's phenomenal.
Speaker 1: So right on. Cool well, Matt, thanks again for coming
on the show. This has been a blast and we
will definitely have you back on. For everyone else, check
out cloud war games dot Com and anywhere else that
you hang out, you hang out on like ex LinkedIn,
any of that kind of stuff.
Speaker 5: What you can find me, you find me most of
these places schematic Okay, I'll try and grab it anywhere
I can.
Speaker 1: So okay, Yeah, so check it out on schematical and
thanks for listening. And we'll see everyone next week.