Speaker 1: Ah, Warren, you're looking kind of sharp there in that jacket.
Speaker 1: You look kind of like a NASCAR driver with all
Speaker 1: those corporate logos.
Speaker 2: What's going on?
Speaker 3: Man's fun It's funny you should mention that, because I
Speaker 3: have something interesting for today's episode. One of the recurring
Speaker 3: themes on our podcast I feel like is incident management.
Speaker 3: It's something like lots of people want to talk about,
Speaker 3: and quite a few guests that come on to discuss
Speaker 3: their stressful, traumatic experiences with on call and whatnot. One
Speaker 3: of these guests has stepped up and wanted to be
Speaker 3: today's sponsor for the episode, and that's a pager Duty.
Speaker 3: So I've actually been a fan of page Duty in
Speaker 3: the past when I've reached for an incident management tool.
Speaker 3: And what's nice is, compared to the competitors that we've
Speaker 3: heard from, it's clear that they're actually listening to feedback,
Speaker 3: unlike you know, other enterprise companies utilizing your internal messaging
Speaker 3: platform like Slack to interact with incidents. Especially for US,
Speaker 3: I feel like it's like a baseline requirement for communication
Speaker 3: and collaboration, and they've actually opened up their Slack integration
Speaker 3: to everyone, not just the customers who have shelled out
Speaker 3: money for their enterprise plan, so it's really nice to
Speaker 3: see compared to competitors where it feels like you need
Speaker 3: to pay an enterprise tax even if you aren't an
Speaker 3: enterprise company. I'm particularly like their automatic channel creation when
Speaker 3: there is an incident. You know, if you have like
Speaker 3: lots of incidents, like one every single day, that gets
Speaker 3: pretty tedious. So, uh, thank you Page Duty for sponsoring
Speaker 3: this episode.
Speaker 1: Yeah, that's super cool. Definitely thank you to Page of Duty.
Speaker 1: And I'm a big Page of Duty fan. It's one
Speaker 1: of those tools which I think there's not a lot
Speaker 1: in this category. I could be wrong, but it's just
Speaker 1: one of those like when you are trying to do
Speaker 1: a certain thing, you know, incident response, like Page of
Speaker 1: Duty is just the name that comes to mind.
Speaker 3: You a sake of you know, people thinking like it's
Speaker 3: really easy to do, and then they go out and
Speaker 3: do it themselves. And like I've been in a bunch
Speaker 3: of companies where this has been a pattern of trying
Speaker 3: to do it yourself. And the best part is when
Speaker 3: you hook up your incident management or on call reporting
Speaker 3: you know, monitoring solution to your production systems and you're
Speaker 3: running on the same infrastructure, and then you have a
Speaker 3: production incident. Well you know what else is down at
Speaker 3: that exact same moment, So I can I can highly
Speaker 3: recommend not building this yourself. That's my own sort of
Speaker 3: traumatic experience.
Speaker 1: I think that's a rite of passage, building your own
Speaker 1: monitoring system. And then you do it and you learn
Speaker 1: that lesson that you just brought up and like, wow,
Speaker 1: that was stupid, and then you go to patre Duty's
Speaker 1: website and move on with your life.
Speaker 3: So I've I've shilled out for the episode. So maybe
Speaker 3: we'll talk about something interesting now right on.
Speaker 1: Well, yeah, I think today's guest might have some interesting topics.
Speaker 2: O Mair, how are you, Bud?
Speaker 4: Yes? Good, how are you? Thank you for having me here?
Speaker 2: Dude, welcome back.
Speaker 1: I can't believe we determined it was three years ago,
Speaker 1: right that you were last time?
Speaker 4: Yeah? Yeah, that's just nuts.
Speaker 1: Yeah, Like I don't even want to have this conversation
Speaker 1: with myself about what I've done in those last three
Speaker 1: years that I also don't remember.
Speaker 3: So the listeners are are missing out on that pre
Speaker 3: recorded conversations that were having about you know, how long
Speaker 3: how long has it been? Actually? Because Will you were
Speaker 3: telling us, you know, last three years may or may
Speaker 3: not have happened for you.
Speaker 1: Yeah, And I did mention before we started recording that
Speaker 1: I bucket things into events that happened between before nineteen
Speaker 1: ninety and things that happened between nineteen ninety and yesterday.
Speaker 1: And I really can't get any more granular than that.
Speaker 1: So Omer was just here yesterday, I know exactly.
Speaker 2: And you know what's weird.
Speaker 1: I noticed that you and I are wearing the same
Speaker 1: T shirt today, and I've got to go back and
Speaker 1: look further recording because I think we were in the
Speaker 1: same T shirt last time you were on as well.
Speaker 4: Here's a fun fact. I only have one kind of
Speaker 4: T shirt. I have like thirty of these differing colors.
Speaker 4: It's the same T shirt. That's very possible.
Speaker 1: Yeah, my wardrobe is very much like that as well.
Speaker 1: My wife bought me one of these T shirts once
Speaker 1: and I was like, oh, that's super cool.
Speaker 2: And now that's all I.
Speaker 3: Have, just that one T shirt.
Speaker 4: Just the one one copy.
Speaker 2: Yeah.
Speaker 1: I mean Omar's over there flex and saying that he
Speaker 1: has like them in different colors and multiple shirts. I
Speaker 1: have just this one.
Speaker 3: Every day is laundry day for you.
Speaker 2: Yeah, sure, yeah, yeah, yeah, yeah, we'll go with cool.
Speaker 1: So we were going to talk about Kubernetes and ll ms, right, yeah,
Speaker 1: give us a rundown on that.
Speaker 4: Over the past years things have happened for me as well.
Speaker 4: So I'm an already architect a zest, which means that
Speaker 4: I lead our Kubernetes products. So we're building stuff to
Speaker 4: help you optimize kubernets clusters. And recently it seems like,
Speaker 4: well everybody have noticed the one hype that drags the
Speaker 4: world to direction. It's all around AI now and people
Speaker 4: are starting to focus more on kubernets and AI at
Speaker 4: the same time, which is something I never expected. I
Speaker 4: actually thought the word goes specifically for DevOps. I thought
Speaker 4: it was always going towards serverless and kind of not
Speaker 4: worrying about infrastructure and managing your own stuff or not
Speaker 4: caring about resources, which in some way did happen. And
Speaker 4: you did see like different platforms like both serverless from Awus,
Speaker 4: azure GCPAA, but also platforms like versaill, Heroku, fly Io,
Speaker 4: things like that helped you just deploy your app and
Speaker 4: move on with your life. But then that's some I
Speaker 4: don't know what changed the tide but it seems like
Speaker 4: companies are pushing towards Kubernetes. The last time last Cubicon,
Speaker 4: I think it was in London, they said seventy something
Speaker 4: between seventy and seventy five percent of corporates in the
Speaker 4: world are either already using kubernets or migrating. So that
Speaker 4: was mind blowing to me. I never saw that coming.
Speaker 4: And over time we've started seeing companies naturally go to AI,
Speaker 4: either using AI throughout their products or trying to train lms,
Speaker 4: or actually adopting them after they've been trained, but trying
Speaker 4: to deploy them on their own. This is kind of
Speaker 4: things that we can focus on. We can also speak
Speaker 4: about the unrecorded part that has to do with AI
Speaker 4: hype and how people are vibe going their way to production.
Speaker 3: I think we're definitely going to get there. But you
Speaker 3: mentioned a lot of interesting things because I was on
Speaker 3: the same path for you, like Servilis for me started
Speaker 3: like twenty twenty fourteen, even like before Kubernetes was a thing,
Speaker 3: and there was ideas of this coming into a lot
Speaker 3: of companies, And I feel like there is this question
Speaker 3: of like why did not the better technology become more popular,
Speaker 3: the more extreme like push things aside, focus only on
Speaker 3: the business value. And you basically you just said it
Speaker 3: really got me thinking like why did And I'm sure
Speaker 3: I'm going to get some angry emails for this. Why
Speaker 3: did a worse solution become more popular? Why did you
Speaker 3: get better adopted? And I think it's because there's this
Speaker 3: natural tendency for humanity to take step changes for things
Speaker 3: rather than giant leaps. And there's actually a core concept
Speaker 3: for this in mathematics. It's like you found a local
Speaker 3: optima and you're just making small little jumps, and in
Speaker 3: order to find a larger jump out larger maximal, you
Speaker 3: have to make giant leaps. And in Japanese it's called
Speaker 3: kaikaku rather than kaisen if you're familiar with manufacturing lean terms.
Speaker 3: And I feel like it's really uncomfortable for people to
Speaker 3: throw away everything they have and make a huge leap,
Speaker 3: and I feel like serverless is a huge leap, and
Speaker 3: Kubernetes is people can just keep doing what they're doing
Speaker 3: today and delude themselves into thinking they're making a real change.
Speaker 4: Do you feel like it's a step back.
Speaker 3: I don't think it's better than what the Open container
Speaker 3: initiative Docker containers could have been, but given the companies
Speaker 3: that were backing it, which was pretty much Docker, I
Speaker 3: think that really answered the question of why it didn't
Speaker 3: get further.
Speaker 4: My opinion of this is that the one great thing
Speaker 4: that Kubernetes did is growing insane, like an insane community
Speaker 4: around it, which goes to open source projects under C
Speaker 4: and CF, but also companies either building on top of
Speaker 4: these open source projects or just pushing themselves into C
Speaker 4: and CF, which is incredible because from a very raw
Speaker 4: product that you had to do so much to just
Speaker 4: get to production, you can now get your well you
Speaker 4: deploy Kubernetes. You can either do it on your own,
Speaker 4: but you probably won't. You use a service, but then
Speaker 4: things like helm and Customize and other things around it
Speaker 4: can just help you deploy things. And then companies started
Speaker 4: building on top of that operators, so you can get
Speaker 4: you know, elastic logging, monitoring databases, cash instances, you can
Speaker 4: put whatever you want. With an operator, you just help
Speaker 4: install something and then you have everything you need. It's
Speaker 4: not from the start, but it kind of gives you
Speaker 4: control over everything. And then people said, okay, then now
Speaker 4: you have to manage infrastructure, have to manage notes and
Speaker 4: what's going to be with auto skating and things like that.
Speaker 4: But then you have projects like Carpenter, which is AWS
Speaker 4: is pushing, but Azure is jumping on that wagon. So
Speaker 4: you can kind of have the best of both throws, right,
Speaker 4: You keep your control, you don't pay as much, which
Speaker 4: is debate of but we can talk about that. It's Kubernetes.
Speaker 4: It's something that deploys containers. That's it.
Speaker 1: I think there's a nerd aspect of it as well,
Speaker 1: because it's something that's just fun to totally nerd out on.
Speaker 1: It's so configurable and so flexible. I know quite a
Speaker 1: few people who are part of the Kubernetes at Home
Speaker 1: project or the Kubernetes Home Lab project, and the amount
Speaker 1: of money and level of work these people have put
Speaker 1: into their home Kubernetes lab for doing who knows what,
Speaker 1: you know, like telling the refrigerator when it's time to
Speaker 1: defrost or whatever, and and they just get completely passionate
Speaker 1: about it. And I think that's a big allure to
Speaker 1: Kubernetes because in our industry were people who like to
Speaker 1: just nerd out on stuff and tinker with stuff, and
Speaker 1: with servilists, you don't get that option. You can deploy
Speaker 1: your container and it works, and if it doesn't work,
Speaker 1: you can deploy your container again and then it'll work.
Speaker 4: Yeah.
Speaker 3: I mean, I think the whole lad thing is interesting.
Speaker 3: You got experience doing something that you liked more than
Speaker 3: whatever your company was doing because your company was doing
Speaker 3: something horrific, and you can go down that route. I mean,
Speaker 3: I do see people who are even running stuff at
Speaker 3: home that would prefer servile lists, but they feel like
Speaker 3: it's too much of a burden to convince their organization
Speaker 3: to make the switch. Fundamentally, it does feel like it
Speaker 3: has to be a switch, you know, And I think
Speaker 3: that cognitive burden or political burden is just too much
Speaker 3: for people to deal with a lot of ways. What
Speaker 3: I'm actually interested in is something you said earlier, and
Speaker 3: that's you're surprised there's a marrying between AI and Kubernetes because,
Speaker 3: as we'll pointed out, running locally in a home lab,
Speaker 3: like what are you going to run? Like, you're not
Speaker 3: going to run dockers Form and you're definitely not going
Speaker 3: to run Nomad After everything that Hashi Corpus done, so
Speaker 3: like what are you left with? You're you open open
Speaker 3: stack or coro os, like you're talking about spinning up
Speaker 3: operating systems everywhere. I do feel like Kubernetes is an
Speaker 3: answer there. But for AI it also is like one
Speaker 3: of the few things that I have actually recommended, especially
Speaker 3: if you need to spin up lots of models or
Speaker 3: configure the parameters for running those models, for doing infernts
Speaker 3: that are different per user or per customer. Having independent
Speaker 3: models scales well with different namespaces in Kubernetes, whereas the
Speaker 3: other container orchestrators don't really have this concept. You just
Speaker 3: spin up the same container over and over again with
Speaker 3: the same parameters, and it's not really fundamentally controllable. So
Speaker 3: I feel like the models a little bit different there.
Speaker 4: The beautiful thing about Kuberneties is that it's very extendable.
Speaker 4: You can basically do whatever you want. Right It's a
Speaker 4: set of APIs that you have on the basic binary.
Speaker 4: But if you want to build on top of that,
Speaker 4: we mentioned operators earlier, we're building operators at ST. You
Speaker 4: just make up your own APIs, you deploy them to
Speaker 4: the cluster. They work, so you can kind of change
Speaker 4: whatever you want. You mentioned name spaces, you can if
Speaker 4: you want it. You can build your own notion of
Speaker 4: namespaces that would fit whatever you're trying to do, and
Speaker 4: it would just work. You would have to install it
Speaker 4: with other applications as a service provider or as a
Speaker 4: stand alone project, but anything is possible. That's how they
Speaker 4: This is something I really appreciate about kubernets. First of all,
Speaker 4: I totally agree with Will that it's totally something to
Speaker 4: nerd on about and just see things moving around and
Speaker 4: having your containers shift around different notes. It's cool, it's
Speaker 4: fun to play with. Here's a hot take. It created
Speaker 4: job security in a sense, so people are kind of motivated.
Speaker 4: You mentioned home labs, right, both of you people were deployed.
Speaker 4: Who would in their right mind with deploy Kubernets in
Speaker 4: their home lab. It's just it's it's so many layers
Speaker 4: of complexity. You can solve it with a fifteen other
Speaker 4: open source project, but you're going to kubernatis because you
Speaker 4: already know the beast and you feel like you've tamed this.
Speaker 4: So now you're going to deploy everywhere, including your own home,
Speaker 4: and that's how you want to see things progress. Because
Speaker 4: you moved people would literally only only answer to recruiting
Speaker 4: emails that have Kubernetis in this job description, right, So
Speaker 4: that's part of it, and I think it created like
Speaker 4: this ecosystem of companies are building for Kubernetes. People only
Speaker 4: want to work with Kubernetis. It's be they build a
Speaker 4: fomo around it. Everybody wants to be there. It's that
Speaker 4: somehow keeps being the next cool technology thing that always
Speaker 4: builds up and progresses for partially for a good reason.
Speaker 4: You ask what's good to use in your home lab. Well,
Speaker 4: you can use Kubernetes. There's k three S, which is
Speaker 4: the smaller sibling, the lighter weight thing Kubernetes. There's also
Speaker 4: we mentioned Fly Versail, all these commercial companies, there are
Speaker 4: open source alternatives to all of these, like Qualify is
Speaker 4: one that comes to mind. This is an open source Versail,
Speaker 4: slash Heroku, slash Fly that you can deploy in your
Speaker 4: home lab, which integrates you with anything. So if you
Speaker 4: wanted LMS, for example, you can. It has a list
Speaker 4: of you can make up your own plugins. It's just containers.
Speaker 4: But you can't choose an LLLM or speak to open
Speaker 4: AI from your application, or put your own LM there
Speaker 4: to run next to your application. And it works.
Speaker 1: So I'm curious about the LLLM aspect of it. You know,
Speaker 1: using lllms with Kubernetes, what kind of what do you
Speaker 1: actually gain from that?
Speaker 4: Right? So, I think there's a lot of use cases.
Speaker 4: The two main ones are either I'm an AI company
Speaker 4: and there's you know who is not an AI company
Speaker 4: these days.
Speaker 2: So great anyone who wants funding.
Speaker 4: Is every company's name has changed over the past twelve
Speaker 4: months to something AI. First of all, there are these
Speaker 4: companies that are either they own the LLM or they
Speaker 4: have to train it right, so they need a fleet
Speaker 4: of large and powerful machines to train their lms. They
Speaker 4: need big discs, YadA ya. And then the other part
Speaker 4: of it is these services are starting to cost a
Speaker 4: ton of money, especially if that's your core business. You're
Speaker 4: starting to pay like your cloud bill is now the
Speaker 4: second problem in the organization, not the first. So some
Speaker 4: companies find it that running your own l M or
Speaker 4: you know, there's a ton of open source lms. You
Speaker 4: can go to hugging face and get whatever you want,
Speaker 4: maybe lighter ones, smaller llms that can work even quicker,
Speaker 4: maybe tailor made to whatever you're doing, and then you
Speaker 4: can run it alongside your application, which means reduced latency
Speaker 4: not as high costs. But then this brings the complexity
Speaker 4: complexity story of infrastructure right because you have to right
Speaker 4: size things. Speaking about Kubernetes, Kubernets lets you dictate how
Speaker 4: much memory and CPU you're going to use. But once
Speaker 4: that set, and we'll talk about the last upgrade of kubernets.
Speaker 4: But once that's set, it's set. That's it. It's there.
Speaker 4: Over time, you probably want to change these requests and
Speaker 4: limits to fit whatever the application is doing. Either it's
Speaker 4: consuming too or it requested too much and it's not
Speaker 4: actually utilizing everything, or it needs more and now there's
Speaker 4: no more memory to serve, so you want to change
Speaker 4: these things. There's a few ways to do that, but
Speaker 4: it's really really hard to do automatically. That's one aspect
Speaker 4: of things. The other is these llms are usually just large,
Speaker 4: large language models. They also consume They consume a lot
Speaker 4: of disk, and with disk space it's exactly the same.
Speaker 4: You can create a PVC on kubernets, but once it's there,
Speaker 4: it's really hard to change. You can extend it. Sometimes
Speaker 4: if you're working with the right cloud provider, the right
Speaker 4: c sign. I'm not sure if you're aware. Kubernet Is
Speaker 4: one point three to three was just released a week
Speaker 4: or two weeks ago, and you can now go to
Speaker 4: your requests on a specific pod and change them and
Speaker 4: you don't have to restart the pod. They can just
Speaker 4: change and it will inflate inside the node to whatever
Speaker 4: it needs to consume, which is a really really cool
Speaker 4: change and that helps us a lot. You don't have
Speaker 4: to restart pods anymore if you want to scale up
Speaker 4: the resources that you consume. So that's a really big
Speaker 4: release and that's only been out for a week or two.
Speaker 3: I mean, I hear things like that and I think, wait,
Speaker 3: why wasn't it doing that all along? That seems like
Speaker 3: a required fundamental piece of the infrastructure. And you mentioned
Speaker 3: earlier the APIs have been figured out. You build a
Speaker 3: lot of things around the APIs to make adjustment. So
Speaker 3: my question is, like you actually like the api is
Speaker 3: that Kubernetes provides, because if you're building operators all the
Speaker 3: time to sort of adjust and change the abstraction layer
Speaker 3: to interact with your provider containerization level schedule, or your orchestrator,
Speaker 3: then I feel like you're getting closer to exactly the
Speaker 3: promise that Servilus has been offering all along.
Speaker 4: That's a philosophical question right there. Oh, definitely, that's what
Speaker 4: you're trying to do, right You're trying to abstract things
Speaker 4: by building them yourself because you don't want to mess
Speaker 4: it with infra, but you want to control it, which
Speaker 4: is kind of the conflict we all have because sometimes
Speaker 4: a lot of times we need the control. Right, we
Speaker 4: as a company have to have access to the notes,
Speaker 4: so we can't work with serverllus. We literally have to
Speaker 4: make changes on Linux. By the way, the recent change
Speaker 4: why would why can't you change a container? A running container?
Speaker 4: Why can't you change? The resource had to be developed
Speaker 4: in order for us to get the functionality. So we're
Speaker 4: always trying to make changes so that it's easier to
Speaker 4: deploy things. Production is more stable, our life are easier.
Speaker 4: But we still want the overall control, and there's still
Speaker 4: the ability to make changes to APIs and new operators
Speaker 4: and you know, change this beast. However we want to
Speaker 4: sell new products.
Speaker 3: I'm gonna keep going there, Like, how many companies actually
Speaker 3: need this level of control?
Speaker 4: It depends what you're calling this level. I mean, not
Speaker 4: everyone needs access to the noes, but some do in
Speaker 4: a way, not every You mentioned the scheduler earlier, By
Speaker 4: the way, big thing now in Kubernetis, don't know if
Speaker 4: you know, you can't really access the scheduler, which is
Speaker 4: just another open source component within kubernets. You cannot use
Speaker 4: it in most cloud providers as it was intended to.
Speaker 4: Meaning since when you're using Kubernettis through a cloud provider
Speaker 4: again Aws, Azure, GCP, or any other flavor, the scheduler
Speaker 4: is part of the control plane, you cannot access it.
Speaker 4: You can use it if you're instantiated a new pod
Speaker 4: the scheduler, we'll take that pod and then we'll schedule
Speaker 4: it wherever is available. But if you want to extend
Speaker 4: the scheduler, which is something you can do in kubernets,
Speaker 4: for example, you can plug in your own extender and
Speaker 4: create custom logic scheduling, you can't do it. So that's
Speaker 4: another place where Kubernetis was built to serve everyone so
Speaker 4: that they can do anything. But then the cloud providers
Speaker 4: take bits of it and say, okay, no, no, no, that's serverless.
Speaker 4: Now you're not touching that. We take care of that,
Speaker 4: which is okay until it's not. I don't know. It's
Speaker 4: the ever going conflict of who manages what and who
Speaker 4: has access to it. By the way, AWS can run
Speaker 4: you know, fargate on AWS. Probably other services other cloud
Speaker 4: providers have the same, But fargate you can run Kubernetes
Speaker 4: without nodes. I think to some extent it works. But
Speaker 4: what happens when you do need the access I don't know.
Speaker 4: I think most most companies actually don't use it.
Speaker 3: Oh, I mean, so fargate it's sort of a special
Speaker 3: part of AWS that gives you the serveralless aspects of
Speaker 3: container management without having to go deep into understanding the
Speaker 3: complexities of the node management or scheduling. And it was
Speaker 3: like I'm going to say it was recent, but it
Speaker 3: may have been three years ago still that you couldn't
Speaker 3: actually run fargate effectively with an EKS on AWS due
Speaker 3: to some of the limitations that came along with it.
Speaker 3: Like you could do it, but then for whatever reason,
Speaker 3: you wouldn't be able to get Internet access or i
Speaker 3: AM wasn't working correctly, like you know, permissions didn't really
Speaker 3: work out of the box for accessing other services, and
Speaker 3: so you know, people obviously were trying to do that,
Speaker 3: but it was sort of a joke from that standpoint.
Speaker 3: If you want Kubenetes, you're only going to get the
Speaker 3: horr mode version of it. I think now some of
Speaker 3: those have been or most of them have been fixed,
Speaker 3: so there's, you know, back to very little excuses not
Speaker 3: to use that. Although the canonical reason not to use
Speaker 3: our gate or server list has been access to say GPU,
Speaker 3: the if you are building models or doing any sort
Speaker 3: of video rendering, et cetera, et cetera, you aren't going
Speaker 3: to be able to use fargate. I want to say,
Speaker 3: it's been a while since I looked at it, but
Speaker 3: it used to be the case that you need to
Speaker 3: actually get virtual machines that have access to GPUs or
Speaker 3: GPU optimized machines in order to run your cluster.
Speaker 4: Yeah, that's a great point, but if you don't have
Speaker 4: that requirement, I would urge you to test what was
Speaker 4: farget was built originally for ECS, which is the alternative,
Speaker 4: a WS alternative to Kubernetes, which works great, really a
Speaker 4: great orchestrator. If you don't have anything complex, you don't
Speaker 4: need operators, things like that. You are going to be
Speaker 4: married to a WS naturally, but you get everything out
Speaker 4: of the box, just as you would expect with any
Speaker 4: other platform. Right, you get autoscaling and routing and the
Speaker 4: firewalls and naturally cloud which connects to for every monitoring
Speaker 4: need you have. Again, you're going to have to pay
Speaker 4: something for AWS to run all of that, but it's
Speaker 4: going to be usually it's going to be cheaper than
Speaker 4: running Kubernetes, something I really like doing. Running ECS with
Speaker 4: Farge is great.
Speaker 3: Yeah, no, it's one of the best things. Ever, I'm
Speaker 3: surprised that more companies don't find opportunities to utilize that. Like,
Speaker 3: if you're I usually start out to con station of that.
Speaker 3: You need to prove why you can't use that as
Speaker 3: a solution before you decide to just hop over to
Speaker 3: EKS or running kubernets on top of your own easy too.
Speaker 4: And we talked about the community right around kubernets. There
Speaker 4: was an ever going rumor I think there still is
Speaker 4: that AWS are going to ditch ECS in favor of
Speaker 4: EKS forever for like a decade really, and every time
Speaker 4: you ask them they would officially unofficially that we tell
Speaker 4: you no, we're building it. It's core business. Said that.
Speaker 4: I think Netflix was running a large part of it once,
Speaker 4: it's not going anywhere, but there still is a rumor.
Speaker 4: So many manages to maintain that rumor going on and
Speaker 4: just you know, convince people to ditch whatever they're using
Speaker 4: and go over to kubernets.
Speaker 1: It's probably started by the Amazon eks team.
Speaker 4: Kudos to them.
Speaker 2: Right cool.
Speaker 1: So back to your point earlier, Warren about who needs
Speaker 1: this level of control, I think there's a it's like
Speaker 1: one of those right time.
Speaker 2: Right place things.
Speaker 1: You know, you hit a certain level of scale where
Speaker 1: if you're going to manage your costs, you do need
Speaker 1: that level of granularity or at least visibility into it.
Speaker 1: And with a lot of the serverless type providers, you know,
Speaker 1: you just get this huge bill because it provisioned whatever
Speaker 1: you told it to and then your finance team is like, hey, dude,
Speaker 1: you need to figure something out here. And that's whenever
Speaker 1: you start wanting to get more control over it. But
Speaker 1: I think it's it's not something that most people initially need.
Speaker 1: It's something that you find you need after you've got
Speaker 1: everything else up and running, you know.
Speaker 3: I mean it's interesting you bring that up because I
Speaker 3: always like big the model contrarian here. We don't use Scubernetes,
Speaker 3: and not only do we try to embrace over lest
Speaker 3: wherever possible, we actually try to use edge workers. So
Speaker 3: in out front that's lambdad edge or cloud from Functions,
Speaker 3: or in cloud Flare it's web workers. I don't actually
Speaker 3: know if Azure and GCP have something, which is the
Speaker 3: fact they never heard of. It encourages me to say, yeah,
Speaker 3: they don't have something, but I'm not going to be
Speaker 3: caught recorded saying that's you know, on the record for sure,
Speaker 3: which is really interesting. And honestly, even with that, even
Speaker 3: if you multiply our compute costs two or three times more,
Speaker 3: it's still nowhere near the top of the biggest cost
Speaker 3: concern in our organization or even in the cloud.
Speaker 4: I have a question to you. Maybe that would be
Speaker 4: a segue to the other point we talked about earlier
Speaker 4: with vibe coding and how everything is going around AI.
Speaker 4: But it feels, it feels, at least from scrolling doom
Speaker 4: scrolling LinkedIn that everyone's building something right. Everyone can now
Speaker 4: spend a weekend and build an MVP or even a
Speaker 4: product and push that production. And I wonder where they're
Speaker 4: starting something, starting a narrative where people on the Internet
Speaker 4: can just build their own products mainly with AI. By
Speaker 4: vibe coding not vibe quoding whatever, you can pretty quickly
Speaker 4: get to something working and then deploy that to one
Speaker 4: of the platforms we mentioned earlier, which is mostly completely serverless. Right,
Speaker 4: you pay a monthly subscription based on how much you use,
Speaker 4: and you don't have to worry about anything. So do
Speaker 4: you think that would change the tide a little bit
Speaker 4: in how many organizations are using Kubernetes and how many
Speaker 4: people most of the people we know, I think, are
Speaker 4: not actually running businesses on their own. They're part of
Speaker 4: a team in a company, a large company that uses
Speaker 4: whatever one of the cloud platforms in Kubernetes. Do you
Speaker 4: think that will shift something? Because so many people that
Speaker 4: are building products and trying to build their own businesses
Speaker 4: and using so much serverless because they don't code, and
Speaker 4: they don't know how to manage infra or at least
Speaker 4: that's not their core business and that's not what they
Speaker 4: want to deal with.
Speaker 3: I mean, that's I think what I heard there is
Speaker 3: technically lms are generat like when we use vibe coding,
Speaker 3: the result is actually a serverless solution. So everyone who
Speaker 3: uses LMS, the vibe code or generate solutions using AI
Speaker 3: in anyway is actually saying Kubernetes is wrong. Serverless is
Speaker 3: the right answer. Yeah, I mean sounds good. I like
Speaker 3: that argument.
Speaker 4: I mean you're on you're left with that option only right,
Speaker 4: because it generates code. Most people who I think most
Speaker 4: people who use it don't actually tell it here's the
Speaker 4: architecture we're going to use. This is how you're going
Speaker 4: to separate things. It's not working like that. They actually
Speaker 4: they describe the business logic or how they want things
Speaker 4: to look and work, and that's all they care about.
Speaker 4: They don't care about how it's built, whether it's the
Speaker 4: most efficient solution ever. They just care about something that
Speaker 4: does the business logic they care about and for it
Speaker 4: to be accessible to other people on the internet, right,
Speaker 4: which basically means serverless.
Speaker 3: Like you said, so, I think from my research and
Speaker 3: what I've read through like door reports, and we actually
Speaker 3: interviewed a whole bunch of product managers, what we did
Speaker 3: find is that quality goes down, but the throughput on
Speaker 3: delivering solutions goes up. Speed for development in a way,
Speaker 3: when you're using vibe coding or ELMS in any way,
Speaker 3: so I think the question is, you know, are you
Speaker 3: willing to make the trade off of less quality for
Speaker 3: delivering a solution faster. Then the ones that were most
Speaker 3: effective in this mode were the ones that could basically
Speaker 3: give an LM a spec of their solution, so not
Speaker 3: just the architecture, but literally how it's supposed to work,
Speaker 3: and those interactions followed from what product managers do in
Speaker 3: some way. Now, if product managers today are giving specs
Speaker 3: to their development teams on what they should be building,
Speaker 3: I mean they're probably not doing a very good job
Speaker 3: because I don't know any humans that like taking you know,
Speaker 3: hard coded specifications unless you're a consulting company or contracting
Speaker 3: company who's doing software development doing value based work getting
Speaker 3: paid by the spec But most most teams are not,
Speaker 3: and so you transition their abilities to working with LMS,
Speaker 3: and they're very effective with churning stuff out. Tim goes
Speaker 3: with like deep challenging hard tech. If you're an engineer
Speaker 3: and you're working with some specification released by standards body,
Speaker 3: converting that into actually something working using an LM is
Speaker 3: way more effective because it is very much consume this
Speaker 3: data and transform it. Transformations are very effective. So I
Speaker 3: have a little bit of my own hot take here,
Speaker 3: which is that my theory is that the more engineers
Speaker 3: nerd out about a topic, the less value it offers
Speaker 3: the organization.
Speaker 4: I tend to agree.
Speaker 2: Yeah, that one's going to be hard to argue against.
Speaker 4: I wanted to ask you something. You started by saying,
Speaker 4: based on what you measured, that quality goes down, like
Speaker 4: throughput goes out. Yeah. And what I hear between then
Speaker 4: and correct me if I'm wrong here, But what I
Speaker 4: hear between the lines is we moved the problem to
Speaker 4: our future. Set right, because it's out there. You see there,
Speaker 4: you go, it's in production, you can see and use it.
Speaker 4: The functionality is there. However, the moment things need to
Speaker 4: scale or you know, stability. If quality goes down, it's
Speaker 4: not as stable. There's more bugs to fix. These things
Speaker 4: tend to grow exponentially. So don't you feel that's just
Speaker 4: pushing the problem either elsewhere or to the future.
Speaker 3: Yeah, and in a critical way. I think this is
Speaker 3: one of the pasts that will cause that, Like the
Speaker 3: downfall of humanity doesn't come from you know, robotic AI
Speaker 3: terminators that are that are you know, impacting us. It
Speaker 3: comes from very subtle things that we've already accepted. I
Speaker 3: was reading some paper and I don't remember what it was,
Speaker 3: and I may have a link later, but it was said,
Speaker 3: we're comparing often humans capabilities, like how well we do
Speaker 3: versus how well LMS can do, And what we should
Speaker 3: be comparing is our weaknesses versus their strengths. It's very
Speaker 3: part of war some sup perspective here, what problems are
Speaker 3: we causing for ourselves that lms are you know, falling
Speaker 3: into and are going to cause this problems in the future.
Speaker 3: So yeah, for sure, it's a huge issue in a way. However,
Speaker 3: I think this goes into the perspective of like, what
Speaker 3: do you actually need in your company? You can sacrifice
Speaker 3: quality in some way and deliver your product because your
Speaker 3: end users don't care about it, then yeah, for sure.
Speaker 3: You know, increasing throughput on delivery, increasing your delivery rate,
Speaker 3: you know, is a thing that you should do. But
Speaker 3: if you care about performance and reliability and architecture, you know,
Speaker 3: something that my company cares about. I think a lot
Speaker 3: of companies secretly care about this. If you look longer term,
Speaker 3: you can't be using LMS in this way to be
Speaker 3: long term effective. It's going to be a critical problem
Speaker 3: for your company, sooner rather than later.
Speaker 4: I think it goes even beyond that what I've seen
Speaker 4: in one of my projects and one of two three
Speaker 4: more developers. But it's basically just me and I figured
Speaker 4: using so much it's curser now. But I've used a
Speaker 4: bunch of them. Sometimes it would build the feature and
Speaker 4: the feature works, and then when I code review, it
Speaker 4: removed a bunch of other lines totally irrelevant to what
Speaker 4: it was trying to do, which is I could not
Speaker 4: figure out why or how. And this made me think,
Speaker 4: if I deploy this to production, my throughput goes up.
Speaker 4: Right if it's a GERRA ticket, that Gerra ticket is
Speaker 4: now done. I've finished my task. I can move on.
Speaker 4: But if I don't have automated QA or the right
Speaker 4: CI pipeline, nobody knows about this thing until someone needs
Speaker 4: this feature a month from now, which again begs the
Speaker 4: question is throughput goes up does not mean that everything
Speaker 4: is done correctly, And if people don't actually code review
Speaker 4: what's going on? Is it real? In a way? And
Speaker 4: that made me think that maybe developers are kind of
Speaker 4: moving away from being the ones that write most of
Speaker 4: the code to the ones that have to review most
Speaker 4: of the code.
Speaker 3: So we know that that's going to be a failure
Speaker 3: though too, because in order to effectively review stuff, you
Speaker 3: need to be able to have the whole context of
Speaker 3: what's going on. And for a human you're like, I
Speaker 3: already forget things, you know, I'm sure everyone forgets things
Speaker 3: in a solution that has millions and millions of them
Speaker 3: lines of code, and so especially code that you wrote yourself.
Speaker 3: I think there's tons of jokes out there is like,
Speaker 3: who is the idiot that programmed this?
Speaker 4: Oh? Oh that was me?
Speaker 3: Actually so and now that idiot is is going to
Speaker 3: be an LM and also produced ten times as much
Speaker 3: or one hundred times as much code that you've never
Speaker 3: seen before. And so that's not a realistic solution to
Speaker 3: expect people to actually review that. They're gonna, you know,
Speaker 3: looks good to me and approve it. And the counter
Speaker 3: argument has been for a while, oh well, they'll just
Speaker 3: also create automated tests and you'll review those for the
Speaker 3: business cases and validate your solution against it. However, the
Speaker 3: problem is that the context window, the context window is
Speaker 3: fixed sized, so the thing that the input tokens into
Speaker 3: every LM will never go to infinity will never be
Speaker 3: able to contain all of the relevant information that is
Speaker 3: necessary because it just it's a computational model. Even if
Speaker 3: it gets more and more, you still have to provide
Speaker 3: it that context way. Maybe you hope that providing it
Speaker 3: all of the source code on MPM, if you're using JavaScript,
Speaker 3: and also your source code, and also your GEO tickets,
Speaker 3: hopefully you're not using Gira, using linear or something else,
Speaker 3: and you know your GitHub or get lab repositories and
Speaker 3: every email that was ever sent in your company, and
Speaker 3: every Slack or discord message or some better chat tool,
Speaker 3: like everything the company has ever done. Maybe you have
Speaker 3: enough context there, maybe you'll get to that point. The
Speaker 3: interesting thing though, is that humans aren't computational models, So
Speaker 3: the value we're providing into the system includes some non
Speaker 3: computable box black box that is an input to this
Speaker 3: buffwer development into the business development, and where if we're
Speaker 3: taking humans out of the loop there, we're actually by
Speaker 3: nature removing something that an LM will never be able
Speaker 3: to replace.
Speaker 4: That's super interesting in my Again, I might be it
Speaker 4: might be disrespectful to elms, but I feel when I'm
Speaker 4: working on a project for a few months or a
Speaker 4: few years. I have a deep sense of familiarity, and
Speaker 4: like you said, maybe one day this context window grows
Speaker 4: enough to replace me. But at the moment, it feels
Speaker 4: like even if things work and it's not deleting lines,
Speaker 4: that it shouldn't they work because it's just created a
Speaker 4: bunch of additional code that is already there. And maybe
Speaker 4: it's doing things. I have a ready sketch instance that
Speaker 4: works for my application, and it's just built another layer
Speaker 4: of interaction with Reddish, just because it wanted to extract something,
Speaker 4: and there's a library specifically for that called Reddish. It
Speaker 4: just couldn't find it and just did something on its own.
Speaker 3: There's a great paper that compares the Linux operating system
Speaker 3: versus I think it's ECOALI and how the DNA structure
Speaker 3: represents the source code and how these two things compare
Speaker 3: to each other, and you find that Linux is sort
Speaker 3: of this upside down pyramid where there are some root
Speaker 3: modules that are fundamentally critical and used by everything, and
Speaker 3: then there's leaf nodes that you know, depend on composite
Speaker 3: things that end up depending on the route, an upside
Speaker 3: down binary tree, if you will, and whereas e COLI
Speaker 3: is like a right side up pyramid. The most critical
Speaker 3: functions are highly replicated throughout the DNA, because if one
Speaker 3: of them becomes corrupted, you don't end up with a
Speaker 3: single point of failure or catastrophic failure for the organism.
Speaker 3: It can still continue on and you know, not leak
Speaker 3: all your customers data to the internet. I mean not,
Speaker 3: you know, just go through apoptosis and die as an organism.
Speaker 4: So you're saying it's a good thing the way it operates, well.
Speaker 3: Yes, and no. I think there's an intentionality behind the
Speaker 3: evolution where you can say, well, for reliability, it needs
Speaker 3: to be this way. But the LM isn't doing it
Speaker 3: based off reliability to preventing its mutations, right, I mean
Speaker 3: it's not going in that direction.
Speaker 4: The other thing it's not it doesn't care about is maintainability. Right.
Speaker 4: If you'd have a feature to build and it's just
Speaker 4: added a bunch of additional code, it might not be
Speaker 4: all that critical to anyone, not even the resources. Fine,
Speaker 4: a few more lines of codes, just a few more
Speaker 4: bites that are stored, especially if you're come finding it.
Speaker 4: But it's not mainteenable, and that begs the question should
Speaker 4: it or if we're taking if elms are going to
Speaker 4: take over everything, it should not really be maintainable. However,
Speaker 4: something that's not maintainable is not really reviewable. Right If
Speaker 4: there's one thousand lines of codes added to everything, every
Speaker 4: little feature you develop, because just how lms work, it's
Speaker 4: not really maintainable, scalable, reviewable. You just kind of shift
Speaker 4: humans away out of the process.
Speaker 3: I think there's a huge mistake where we're generating things
Speaker 3: from l lambs and committing that as the relevant artifact
Speaker 3: or other humans to review. So in the case of
Speaker 3: generating source code, having humans review that, or even writing
Speaker 3: tests and then reviewing that, or you know, generating I
Speaker 3: think emails or blog posts written by l lambs and
Speaker 3: outputting that, Like, that's not the value. Isn't the output
Speaker 3: In this case, the value was the prompt. It was
Speaker 3: the human input here or what however you generate flipping
Speaker 3: or flipping a coin or asking the l lump to generate, Like,
Speaker 3: it doesn't really matter. You have a prompt, that's the
Speaker 3: thing which was valuable. It's like when someone says, hey,
Speaker 3: I used an LM to completely generate this blog post,
Speaker 3: I'm like, cancel. Just tell me what prompt you use
Speaker 3: to generate the blog post, because then I can do
Speaker 3: it myself and interrogate the result. I don't need your
Speaker 3: blog post. You didn't apply any original thought there. You
Speaker 3: just copied what someone else created. If you use claude,
Speaker 3: you copied what anthropic thought. If you use CHGPT, you
Speaker 3: just copied what OpenAI has for data. So just get
Speaker 3: get rid of all that. And from a source code standpoint,
Speaker 3: it means committing these prompts and trusting the underlying models
Speaker 3: in some way, or doing some sort of model validation separately,
Speaker 3: and then using the prompts as the mechanism and so on.
Speaker 3: Every build of your project's solution architecture, you rerun all
Speaker 3: the prompts against the model, generate a new output validated
Speaker 3: against some historical data from what your users use, for instance,
Speaker 3: and go from there. And I think that's a much
Speaker 3: more mature understanding of how lms can be effective.
Speaker 4: So I heard someone doing that. But in order for
Speaker 4: just throwing out prompts and expecting some results and then
Speaker 4: doing something, his prompt is always I'm going to ask
Speaker 4: for something. Don't do anything yet, just give me the
Speaker 4: plan and build it in the best best practices in mind,
Speaker 4: blob about mcp another buzzword we can throw in there,
Speaker 4: but if you have the right mcps, you can actually
Speaker 4: grab the best practices from whatever you're building and then
Speaker 4: give me the plan. Let's talk about it, let's go
Speaker 4: every over everything, and then once we're done and I approve,
Speaker 4: then you start building, which he says reduces the number
Speaker 4: of errors by like fifty percent and he can maintain
Speaker 4: the same output but without so many errors.
Speaker 2: I think that's.
Speaker 1: There's a video I was just watching yesterday from Anthropic
Speaker 1: mastering cloud code in thirty minutes, and it's from the
Speaker 1: guy I don't know his name, the guy who created
Speaker 1: the claud cli, and that was that was his fundamental
Speaker 1: approach in the talk is first is just have a
Speaker 1: chat with the AI about what you're trying to do,
Speaker 1: have it, throw out some suggestions on ways to approach it,
Speaker 1: and then talk those through and really just having a
Speaker 1: much more interactive conversation with it before you ever let
Speaker 1: it start doing anything. And then I think to touch
Speaker 1: back on something you guys brought up a few minutes ago, like,
Speaker 1: I think that's the real role, the long term role
Speaker 1: of software engineers with AI. It's not reviewing the code,
Speaker 1: and it's not having AI replace you. It's about giving
Speaker 1: AI a clear set of instructions and scope so that
Speaker 1: when it goes off to do a task that it
Speaker 1: doesn't you know, build a completely new library instead of
Speaker 1: using the one that's already there. Or one of the
Speaker 1: cases I had early on, I asked it to write
Speaker 1: some tests and then went and checked the word work
Speaker 1: that it did, and it it was trying to like
Speaker 1: install Postgress inside of my doctor container so that it
Speaker 1: had a database to to use during the tests, And
Speaker 1: I'm like, no, no, we're.
Speaker 2: Not doing that.
Speaker 1: How about you just mock the database? Call okay, can
Speaker 1: we do that? But it comes down to like to like,
Speaker 1: you know, giving a clear set of instructions. You know,
Speaker 1: had I told it up front, that it would have
Speaker 1: gotten to the result faster. And so I think that's
Speaker 1: probably the downfall of like vibe coding and letting it
Speaker 1: take on large chunks of work, is it's going to
Speaker 1: make bad decisions and then you end up with the
Speaker 1: problems that we've talked about already, with something that's not maintainable,
Speaker 1: largely inaccurate. And then tying back to the original conversation,
Speaker 1: it's probably going to be a cost hog when you
Speaker 1: try to run it on some serverlist platform.
Speaker 3: I feel like this is the the quintessentially example of
Speaker 3: our user experience. Right, it's as a user, don't do
Speaker 3: the thing you want, instead, you need to be trained
Speaker 3: to use the tool. And I feel like I'm dystopian
Speaker 3: perspective at this moment where it's like we're being trained
Speaker 3: on how to interact with the robots that we've created,
Speaker 3: rather than changing the models in a way to respond
Speaker 3: to how individually we work. And I mean I say
Speaker 3: that it's like sort of really ridiculous, but you know,
Speaker 3: we're now in a way beholden to our AI overlords
Speaker 3: who have already decided what's right and wrong, and only
Speaker 3: if we interact with them in the correct way where
Speaker 3: we actually get a valid response and we're what we're
Speaker 3: looking for.
Speaker 1: So do you think it's a valid analogy then to
Speaker 1: say I shouldn't have to learn how to drive a car.
Speaker 1: I just want to get in it and go.
Speaker 3: Yeah, And I think that has improved cars over time. Right,
Speaker 3: automatic seat belts, automatic braking, automatic air bags, cruise control. Right,
Speaker 3: I mean these things are like, right, We're bad at
Speaker 3: all of these things we should provide capabilities and improvement.
Speaker 3: And you see, like maybe UI products for companies that
Speaker 3: do care about the user experience are improving them. And
Speaker 3: I know you meant that as a joke and I
Speaker 3: took it too serious.
Speaker 1: No, No, it was a serious question because I wanted
Speaker 1: to see how the analogy compared.
Speaker 4: It's actually a great analogy, right, It's something that we
Speaker 4: use technology to. It's people's work, right, driving cars, driving taxes,
Speaker 4: driving whatever. If you move that over to ropaths, then
Speaker 4: these people need to change their line of work, which
Speaker 4: I'm trying to think of in my line of work.
Speaker 4: Is it going to be redundant? Is it going to
Speaker 4: be able to be done by AI solely? You don't
Speaker 4: need to review anything, you don't need to All you
Speaker 4: have to do is prompt the right things and it works.
Speaker 4: And I wonder if this is going to happen and
Speaker 4: when and in what way? Because throughout history, every time
Speaker 4: there was a technological advancement, people thought, Okay, that's the
Speaker 4: end of the world. Everyone is everyone's going to be
Speaker 4: out of work, and the opposite happen. Instead of it
Speaker 4: improving our lives and making us work less hours. Is
Speaker 4: just the other way around. Okay, great, more profit like
Speaker 4: more throughput more profit, work more, produce more.
Speaker 3: I like that you brought up this example because I
Speaker 3: actually feel like it's a counter example to the argument
Speaker 3: if we if you look at books like Sapiens, which
Speaker 3: was released not too long ago, we see that the
Speaker 3: goal for improving or automation has never been to improve
Speaker 3: individuals lives.
Speaker 4: That's exactly where I put this from.
Speaker 3: By the way, allow allows society to support additional humans,
Speaker 3: even if it means subjugating even larger portion of those
Speaker 3: humans or entities organisms to you know, a below poverty line,
Speaker 3: or you know, sacrificing even more for them. So it's
Speaker 3: not that those technologies exist to make humanity better, it's
Speaker 3: humanity exists to be able to make the technologies better
Speaker 3: so that, you know, other we can increase our population size.
Speaker 3: So you know, there's a question.
Speaker 4: Of who's controlling who's controlling who?
Speaker 3: Yeah, and how many humans can be supported on this planet.
Speaker 4: But the argument means that we as humans support the technology, right,
Speaker 4: so we produce technology to improve the chology. That's basically
Speaker 4: what we're doing. That then reproducing that one.
Speaker 3: Deep AI may take away our capability to reproduce in
Speaker 3: the future. That's that's where you wanted us to get too, well.
Speaker 4: Right, I heard an interesting argument that said that humans
Speaker 4: are the sex organs of machine it's going to take over,
Speaker 4: but where it's we're the reproduction capability.
Speaker 3: I mean obviously not taken to be literal, right, I
Speaker 3: don't know.
Speaker 1: No, there's there's got to be a movie or a
Speaker 1: book that's that that covers that topic. Like that's that's
Speaker 1: some pure sci fi gold, right there.
Speaker 3: I mean, in a way, it's sort of a how
Speaker 3: viruses work, like AI in a way is a virus
Speaker 3: and we and viruses work by getting into your cells
Speaker 3: and if there RNA viruses, replacing your DNA or inserting
Speaker 3: in your DNA their own set of DNA sequences for
Speaker 3: those amino acids, so that your body, you're the cells
Speaker 3: automatically produce the virus itself.
Speaker 4: And it wants to reproduce and infect others, right, which
Speaker 4: is great at we're using everybody's using My parents use
Speaker 4: AI now to ask whatever they want and it works
Speaker 4: and then they tell their friends and that infects someone else.
Speaker 4: So it's exactly like a very good virus, to be.
Speaker 3: Honest, effective virus effective.
Speaker 4: Yes, a virus.
Speaker 3: Virus is virus is canonically by scientists all over the
Speaker 3: world have said to not be alive. So I like
Speaker 3: this analogy. If humans are the cancer, then the virus
Speaker 3: the AI is definitely the virus.
Speaker 4: This lives me with some thoughts, Well, bring them on.
Speaker 1: We got to we gotta keep going with this episode
Speaker 1: till we all get canceled.
Speaker 4: I don't know how to bring this back to devovs
Speaker 4: and code about it feels like we're installing viruses.
Speaker 3: And I think it's useful for people to take a
Speaker 3: deeper look at the technology that they're utilizing and how
Speaker 3: it's being deployed within their company and what changes that
Speaker 3: they're making over time to make that more effective but
Speaker 3: for them and both their future jobs but also the
Speaker 3: long term in the company.
Speaker 1: Wow, well done, Warren, getting us right back on topic.
Speaker 2: That was a pro move.
Speaker 3: Thank you.
Speaker 2: Yeah, and I got nothing to follow up.
Speaker 4: I have so many thoughts I need to process.
Speaker 3: What's the What's the most important one for you? I
Speaker 3: think would be the question. So you know, you brought
Speaker 3: up the topic of not just Kubernetes and how ZESSI
Speaker 3: is utilizing it, but also the impact of building it
Speaker 3: to support lllms, both internally and from third party companies.
Speaker 3: Have you seen hands on specific challenges other than what
Speaker 3: we've already talked about, not really.
Speaker 4: If I'm trying to connect that into both the philosophical
Speaker 4: aspect of it and the technical parts. It's mainly focusing
Speaker 4: on improving, right, which is everything we talked about is
Speaker 4: improving either the technology or whatever drives the technology. So
Speaker 4: people are trying to companies are trying to improve the
Speaker 4: way they run lms, the lms themselves, the infrastructure that
Speaker 4: surrounds it, and honestly their cloud bill at the end
Speaker 4: of the month, which has to do with everything. And
Speaker 4: it's also a funny aspect of AI because it consumes
Speaker 4: so much energy resources not in the form of computers,
Speaker 4: and chifts in the form of energy, right, and the
Speaker 4: costs are to other professionalism. What they're saying is that
Speaker 4: it's not maintainable and it's not scalable, and it's going
Speaker 4: to hit a wall at some point, and I'm wondering
Speaker 4: whether that wall is going to be more companies trying
Speaker 4: to run more tailor made lean llms that only serve
Speaker 4: one purpose or general purpose solutions like that are going
Speaker 4: to be more consumed by AI companies, much like we're
Speaker 4: consuming cloud resources from cloud providers and shifting away. So
Speaker 4: I don't know exactly where it's going, but I seems
Speaker 4: to be a very very expensive resource at the moment,
Speaker 4: So I don't know.
Speaker 3: You've pulled out the optimistic perspective. There's a good principle
Speaker 3: on this, and I just I the name eludes me
Speaker 3: at the moment. But even if you make it cheaper,
Speaker 3: you'll end up with this actually contradiction where the result
Speaker 3: is more usage, not less. And I think the biggest
Speaker 3: problem is that we're already seeing we talked about this
Speaker 3: a little bit in one of the previous episodes, that
Speaker 3: companies will just continue to use additional energy and rather
Speaker 3: than care about trying to make it cheaper, will keep
Speaker 3: on trying to figure out how to build more and
Speaker 3: create more energy. Yeah, and so this means open reopening
Speaker 3: coal and gas mines. Now, the question could be, do
Speaker 3: you think there's an opportunity for good here where there
Speaker 3: will be companies will start trying to invest in figuring
Speaker 3: out how to get fusion reactors so that we can
Speaker 3: get a step up in energy creation over because we're
Speaker 3: never going to get there with sore or wind or water.
Speaker 3: I mean, we have seen some situations where I believe
Speaker 3: it's trying to trying to beam energy from outside the
Speaker 3: atmosphere back down to Earth.
Speaker 4: You know how in Google flights, when you're searching for
Speaker 4: a flight, it would tell you how much how much
Speaker 4: pollution it creates or whatever in terms of forgot always
Speaker 4: the measurement. So at one point we were trying to
Speaker 4: do the same because we're in the business of making
Speaker 4: infrastructure more efficient and effective and in a way allows
Speaker 4: you to reduce both your costs but how much infrastructure
Speaker 4: you're using, which you can follow up to aws not
Speaker 4: using as much. No, so maybe we're you know, supporting
Speaker 4: the environment by reducing the energy companies use, and at
Speaker 4: some point we're trying to give you how much you've saved,
Speaker 4: but also how much you've helped the environment by saving
Speaker 4: on resources, which is an interesting analogy. I don't know
Speaker 4: if it would work, but okay.
Speaker 3: So I'll be pressed by you know, the contrarian perspective.
Speaker 3: So here it is, it's the only way this could
Speaker 3: be effective is if we paid companies for having a
Speaker 3: low spend but compared to what how do you actually
Speaker 3: do this. We know carbon credits didn't work for reducing
Speaker 3: carbon dioxide emissions into the atmosphere, so yeah, so I
Speaker 3: mean that fundamentally is the problem. But there's actually another
Speaker 3: issue here. If reducing your spend or using comparative technologies,
Speaker 3: let's say the cloud providers using AWO said, you know,
Speaker 3: here's A and here's B use B, we'll actually reduce
Speaker 3: the cost more. You know, this is actually cheaper, not
Speaker 3: because it's technology technologically cheaper or requires less energy, but
Speaker 3: because it is better for the environment. The problem is
Speaker 3: that you'll start to see companies pop up that abuse
Speaker 3: option B and reselling that, you know, at a cheaper
Speaker 3: way to other companies, so like competing with aws but
Speaker 3: increasing the price and then taking a cut of it.
Speaker 3: So there are companies out there that just deal with
Speaker 3: carbon credit resells. They buy credits to resell them, or
Speaker 3: even the worst case, the worst polluters in the world
Speaker 3: just buy tons of credits from these intermediaries, and so
Speaker 3: it doesn't help at all in any way, and you're
Speaker 3: just making another company rich in the process who is
Speaker 3: just abusing that gamification.
Speaker 4: Model can fight human nature, you can with AI.
Speaker 1: Maybe no GCP has that there's different different regions you
Speaker 1: can choose, and some of them like display their carbon
Speaker 1: emissions or carbon offsets that you gain by using resources
Speaker 1: in that particular region.
Speaker 3: Yeah, I mean, I guess you'd have to be fined
Speaker 3: for how much you're utilizing. But you know, from a
Speaker 3: human perspective, if you get a fine, then you say, oh, no,
Speaker 3: it's okay that I'm doing this. The government is, you know,
Speaker 3: extracting their reward for their their you know, return for that.
Speaker 3: So I don't know how you can really think about
Speaker 3: this in a way that makes sense, Like maybe there's
Speaker 3: some way. I just haven't thought about it enough. But
Speaker 3: it seems like there's not a lot of good options
Speaker 3: like how much should it be? How many? How many
Speaker 3: carbon credits like or usage should I have as a company?
Speaker 3: Like one five is five a lot. I don't know
Speaker 3: what the appropriate numbers are. I think it's like one
Speaker 3: one kilogram per like international flight. I think it is
Speaker 3: the number. I don't know if that's right.
Speaker 4: I'm just gonna go with that kilogram of carbon dioxide
Speaker 4: per international flight.
Speaker 3: Yeah, I think it's something like that. I don't know
Speaker 3: if that's right.
Speaker 4: And what does not even mean, okay, let's say that
Speaker 4: that's what does it mean in terms of pollution in
Speaker 4: terms of that effect on the atmosphere, well.
Speaker 3: I mean actually figuring out what the direct effect is
Speaker 3: is an impossible problem to solve. So you know, you
Speaker 3: just take what the pollutant is and you measure it, right,
Speaker 3: like the amount of whatever poison that you jump into
Speaker 3: the river. How much poison is it matters for human beings? Well,
Speaker 3: that's sort of hard to describe. There's like a huge
Speaker 3: problem right now with the forever chemicals. Not the not
Speaker 3: like the teflon on your on your hand, but the
Speaker 3: products that care hate the teflon on your pant are
Speaker 3: dumped in the water by companies. And how much of
Speaker 3: that is bad? Well we can say that you know,
Speaker 3: one is work is worth like two is worse than one,
Speaker 3: But is how bad is two? How bad is one?
Speaker 1: Like?
Speaker 3: That's that's a really hard problem to answer. So I
Speaker 3: don't know what it is for carbon credits. I do
Speaker 3: know there are a bunch of companies out there that
Speaker 3: are investing in trying to expose as information and somehow
Speaker 3: utilize it. And there's lots of countries with grants available
Speaker 3: to create green products or projects, but they don't usually
Speaker 3: focus on like the carbon credits because it's like it's
Speaker 3: such a challenging thing to go off of. So instead
Speaker 3: they invest in things that they believe are sustainable for
Speaker 3: whatever definition is sustainable.
Speaker 2: You have like Kubernetes.
Speaker 4: This actually always made me think whether everybody hates the
Speaker 4: cloud platforms? Right? Everybody wants to. I mean there's a
Speaker 4: people want to manage their own infrastructure. They don't usually,
Speaker 4: but they like to hate on them. And I always thought,
Speaker 4: do I by using a cloud provide like AWS, RGCP
Speaker 4: or azure, is that better for the environment? Solely on
Speaker 4: that perspective, Is it better to use something central that
Speaker 4: has a lot of resources that are actually shared resources
Speaker 4: in a lot of ways, which means it's more efficient
Speaker 4: in a global level as opposed to a company or
Speaker 4: me just putting a server act here, which which could
Speaker 4: naturally consume a lot more than it actually should because
Speaker 4: you're you're buying in order to scale, So companies who
Speaker 4: do that probably buy lots more than what they actually need.
Speaker 4: Is it better for us to use? Again, solely on
Speaker 4: the perspective of efficiency utilization and how it affects the environment,
Speaker 4: is it better to to use a cloud provider than
Speaker 4: to run your own infra?
Speaker 3: Yeah, I mean that's sort of difficult. I think there's
Speaker 3: a couple of different parts to that equation. The first
Speaker 3: one is how bad is it for what you're doing
Speaker 3: if you're running on PREMI I think that the recycling
Speaker 3: of electronics waste is like one of the biggest waste
Speaker 3: recycling problems in the world, and it's like only getting
Speaker 3: worse by a huge fact. There's a magnitude in order
Speaker 3: to do the waste processing. It actually consumes a ton
Speaker 3: of energy, both humans and like physical energy, and no
Speaker 3: one plant can take care of it all. Usually it's like, well,
Speaker 3: we remove the plastic parts and then ship all the
Speaker 3: other parts to other people, and then we we take
Speaker 3: out the gold and then ship the rest of it,
Speaker 3: and then we take out the silver and ship the
Speaker 3: ret like we can't do anything else, and then at
Speaker 3: the end of the day it's all in the ocean. Yeah,
Speaker 3: that's right. So that's sort of a hard answer. I think,
Speaker 3: how long as an individual, both as a company and
Speaker 3: a individual, are you good at handling your technology waste?
Speaker 3: You get a new iPhone every every year, you know,
Speaker 3: that's probably bad. Although you know, here's the flip side.
Speaker 3: You know, what would you use the money for? Are
Speaker 3: you taking the money that your company saves by hypothetically
Speaker 3: not using the cloud provider and doing you know, using
Speaker 3: it for green purposes? You know, are the products you're
Speaker 3: creating and making the world better? Well, yes, okay, it's
Speaker 3: that's an absolute. Really, you have to compare it to
Speaker 3: the company you gave the money to. So is Amazon
Speaker 3: on if using aws taking your money and building green
Speaker 3: product projects to improve the world or what they're doing
Speaker 3: worse than what you would have done with the money
Speaker 3: that you had as a company. So sometimes paying less
Speaker 3: could be worse for the environment. Other times paying less
Speaker 3: is better because now you have more cash to do
Speaker 3: things in a better way. But that doesn't mean that
Speaker 3: you will.
Speaker 4: I always like to think that being in the optimization business,
Speaker 4: it's reducing waste at the end of the day, regardless
Speaker 4: of how it's reducing waste on the application level translates
Speaker 4: to the resource level, translates to energy papapa. But thinking
Speaker 4: about it further, that might not always be the case,
Speaker 4: which is super interesting to me.
Speaker 3: We optimize things because we like to.
Speaker 2: Builds.
Speaker 4: Might very well be a local optima that doesn't affect
Speaker 4: the chain. Right. Another thought, I'll be left with the night.
Speaker 1: It's busy work with a dopamine hit. Well, I feel
Speaker 1: like we've thoroughly covered the topic. We think should we
Speaker 1: do some picks?
Speaker 3: I think I think it's time.
Speaker 1: Where Yeah, we've been here before. Would you bring for
Speaker 1: a pick this time?
Speaker 4: Two things? One, if you're watching series TV series, I
Speaker 4: really liked Mobland. Did you hear about that? It's guy
Speaker 4: Ritchie Tom Hardy. It's really cool Mobland. That's one right
Speaker 4: and completely relevant. And the other one actually is a
Speaker 4: little bit relevant. Is a few. I think it's a few.
Speaker 4: It's probably one software developer from Google on their spare
Speaker 4: time start building kind of an alternative to GET, which
Speaker 4: is not really an alternative because they can work together.
Speaker 4: So I started using it in one of my projects.
Speaker 4: It's called jj jiu Jitsu. It's a really cool open
Speaker 4: source project that kind of lets you work with change management,
Speaker 4: but not as much hassel as GIT. So it's just
Speaker 4: at a chain of changes that you can just change
Speaker 4: at any time, move through history like it was nothing,
Speaker 4: where GIT makes everything a little bit more complicated. That's it.
Speaker 4: These are the two.
Speaker 3: It's like editing the object model graph asd or or
Speaker 3: GET as your direct mechanism. So I feel like, you know,
Speaker 3: people that want to spend more time with their source
Speaker 3: of control revision system but feel better about it. This
Speaker 3: sounds like the perfect tool.
Speaker 4: Yeah, yeah, it's it's really nice and they work together. Again,
Speaker 4: if you're working on a Git project and everybody else
Speaker 4: is working on GitHub with the git locally, you can
Speaker 4: still run jjly on your own machine, but then when
Speaker 4: you're done, kind of wrap it in a commit and
Speaker 4: then push it to a different branch which opens a
Speaker 4: PR and everything, so you can enjoy both worlds. I
Speaker 4: really liked it.
Speaker 2: Right, all right, Warren, what do you got?
Speaker 3: So I have a very controversial pick this time for
Speaker 3: our listeners. I'm going to say doing surveys that that's
Speaker 3: going to be my pick, now hear me out. I'm
Speaker 3: not talking about like doing surveys that like from to
Speaker 3: pay you money, because those are a waste of time.
Speaker 3: Although I did start like that as a person who
Speaker 3: thought that you could make some money doing that, I
Speaker 3: never did. What I'll say is that surveys are like
Speaker 3: I see as my opportunity to change the world in
Speaker 3: my favor. And by doing them and giving feedback means
Speaker 3: that I can change these companies how they're thinking, hopefully
Speaker 3: so that they start and actually listen to that and
Speaker 3: then make some changes. And so if I withhold my opinions,
Speaker 3: that means I basically say, I love the world the
Speaker 3: way it is right now, but I also don't care
Speaker 3: if it changes, but it definitely couldn't be better, and
Speaker 3: I don't think that's true. I like complaining about things.
Speaker 3: So now, lots of companies we know just completely ignore
Speaker 3: the surveys after they're done. You know, if you've ever
Speaker 3: taken like an EMPs survey to ask you if you
Speaker 3: love working for that company, we all know your executive
Speaker 3: team is completely ignoring whatever you wrote there. I'm sorry
Speaker 3: to tell you that. But on the flip side, if
Speaker 3: you fill out the survey for the Adventures and DevOps podcasts,
Speaker 3: I can guarantee you that you'll be entered into one
Speaker 3: of the four remaining twenty dollars AWS credits left that
Speaker 3: we have in store, So you definitely want to do that.
Speaker 4: I'll sign up for that. I just have to say,
Speaker 4: I'm I have a searching in the gym. Every time
Speaker 4: I come back home, I get an email with a
Speaker 4: survey and I filled it like one, two, three times.
Speaker 4: Nothing happened. It was it was I put in a
Speaker 4: lot of time. They didn't even they didn't even reply.
Speaker 4: So it feels like they are throwing it away. But
Speaker 4: I'll definitely do yours.
Speaker 3: The only reason that The only thing worse than doing
Speaker 3: the survey and feeling and not getting replied, I feel
Speaker 3: like they throw it away is doing the survey that's
Speaker 3: really long, getting to the end, and when you click submit,
Speaker 3: it says like, oops.
Speaker 4: It crashed, right, you have fifteen minutes.
Speaker 3: Yeah, please don't if you make a survey, please don't
Speaker 3: do that.
Speaker 4: That's what happens when you have vipe quarders building that oops.
Speaker 1: The other thing I don't like about surveys is when
Speaker 1: you get the email that says like how do we
Speaker 1: do and it's like a smiley or a frowni emoji,
Speaker 1: and so you just click the smiley emoji like cool,
Speaker 1: we're done. Oh wait, no, you're asking follow up questions. Okay,
Speaker 1: I'll do the follow up question, and then then there's
Speaker 1: another follow up question and then it's like.
Speaker 2: No, screw you. I'm not doing this. I was trying
Speaker 2: to be nice, but now f off.
Speaker 3: So everyone that's listening. If you're building a survey, remember
Speaker 3: that Will says that you can get him both with
Speaker 3: the email and then one more question after that, and
Speaker 3: maybe one more if you promised him something. That's that's
Speaker 3: that's the threshold. I mean, there is a there's like
Speaker 3: the sun cost fallacy.
Speaker 4: Right, exactly what I wanted to say.
Speaker 3: Yeah, so you know you've already committed to submitting your
Speaker 3: feedback still more.
Speaker 2: Before you got the reward, right, for sure, you got to.
Speaker 1: You gotta feel like you're unlocking something in each step
Speaker 1: of the survey.
Speaker 3: Yeah, it should definitely have a like anyone who build
Speaker 3: a survey platform s, I should definitely see every survey
Speaker 3: that I submitted, all the feedback so I can be like,
Speaker 3: I got five more points for this for filling out
Speaker 3: this feedback.
Speaker 2: I don't know what.
Speaker 4: That's absolutely nothing.
Speaker 3: I mean Google does it and it makes me feel
Speaker 3: good about leaving reviews for restaurants and other places.
Speaker 4: You have to getting your green Garden githubh.
Speaker 2: Oh yeah for sure?
Speaker 1: Right, yeah, so you just need to tie the survey
Speaker 1: to fake Internet points and everybody will be fighting to
Speaker 1: fill it out.
Speaker 3: Yeah, well, what'd you bring cross?
Speaker 1: My pick has to do with some changes for me.
Speaker 1: I have become the engineering manager for a new company
Speaker 1: called Katana, and so my pick is going to be Katana.
Speaker 1: If you want to go check out a website, it's
Speaker 1: Katana dot Network.
Speaker 2: It is a layer two blockchain.
Speaker 1: That specializes as a DeFi platform. So we're like really
Speaker 1: rethinking the way that decentralized finance works.
Speaker 2: And how to make it.
Speaker 1: More financially rewarding, but also a lower barrier to enter,
Speaker 1: so that if you've ever played with DeFi in the past,
Speaker 1: you know that you had to go and get like
Speaker 1: buy ethereum and then find someplace to convert that to
Speaker 1: wrapt ethereum and then find a bridge that would let
Speaker 1: you swap it into what you were really trying to
Speaker 1: invest in, and like every step of the way, you're
Speaker 1: going deeper and deeper into rabbit hole and not really
Speaker 1: sure if like this place that you're interacting with is
Speaker 1: just fixing to steal everything in your wallet or if
Speaker 1: that really is the right pass. So we're trying to
Speaker 1: eliminate all of that.
Speaker 2: So well, that's my pick.
Speaker 3: That's quite interesting. Honestly, I see you a get a
Speaker 3: new new opportunity there. I know whenever I try to
Speaker 3: do anything with crypto, I always ask an l ALM
Speaker 3: to decrypt my my wallet for me.
Speaker 4: Now there's a way to use energy efficiently.
Speaker 2: Well, I don't think.
Speaker 3: We're ready to get started on in the next podcast. Yeah,
Speaker 3: I just.
Speaker 2: Had chat GPT remember my seed phrase for me? Cool?
Speaker 1: Yeah, so there you go. If you're interested in that,
Speaker 1: checkout Katana dot Network. It's it's been pretty cool, like
Speaker 1: there's some smart dudes working on it and I'm excited.
Speaker 3: So we'll have a link below the podcast. But is
Speaker 3: there like something you're specifically looking for at the moment,
Speaker 3: like are you looking to hire? Are you looking for
Speaker 3: customers or users? Like what's what's the breakdown?
Speaker 1: So I am hiring a full stack role. So if
Speaker 1: you're interested in that, hit me up. But other than that,
Speaker 1: if you're just interested in DeFi check it out and
Speaker 1: I would love to have your feedback to see what
Speaker 1: we're getting right, what we're getting wrong.
Speaker 2: Yeah, do that survey right, It's so there's no survey.
Speaker 1: Just hit me up on X or email and say, dude,
Speaker 1: this is cool or dude, this sucks and that's the
Speaker 1: end of the survey.
Speaker 3: Uh oh, so you're gonna plaster your email over the
Speaker 3: all of the internet so people can respond to you.
Speaker 2: Well, it's not hard to find.
Speaker 1: I mean yeah, like, based on the number of recruiting
Speaker 1: emails I get, my email cannot be hard to find.
Speaker 2: Cool.
Speaker 1: Oh ma, thanks man, it's been fun having you back
Speaker 1: on the show.
Speaker 4: Yeah, thank you for having me. Good to see a
Speaker 4: boat Warren.
Speaker 1: As always, thank you appreciate everything you do here and
Speaker 1: we'll see here when next week