Speaker 1: And we're live with another episode of Adventures in DevOps.
Speaker 2: Almost blew the podcast name right there, Warren, Well, you know, thanks.
Speaker 1: Catching, I've only done I've villain done a few hundred
Speaker 1: of these at this point. It's it's still a learning curve.
Speaker 1: So how are you, Warren, to bring us a security
Speaker 1: fact today?
Speaker 2: Well, you know, I think again. I had brought one
Speaker 2: last time, and I thought, you know, this time for
Speaker 2: sure our fellow co host was going to be here
Speaker 2: at share, the long awaited one, Matt. I had been
Speaker 2: promised at it, you know, coincidence not here, so I
Speaker 2: don't have anything today.
Speaker 1: She's pausing for dramatic effect, like a really really dramatic effect.
Speaker 2: It's gonna be the best fact ever. I swear it.
Speaker 1: When it actually drops, it is it is right well.
Speaker 1: Also joining us Daniel Loretto from Jedifhy. Daniel, Welcome to
Speaker 1: the show man.
Speaker 3: Thank you so much. Will happy to be here.
Speaker 1: Dude, I'm excited to have you here. So you are
Speaker 1: with Jedify. Now tell us a little bit about your background.
Speaker 3: Yes, I'm a software engineered by trade.
Speaker 4: You know, I've worked at a lot of different tech
Speaker 4: companies of all different sizes, so I can talk about,
Speaker 4: you know, different experiences. But I worked at Google, Airbnb, Twitter,
Speaker 4: I worked at you know, tiny startups that you've never
Speaker 4: heard of. I worked at Verta Health, which is a
Speaker 4: telemedicine company for the ABDIS, where I ran engineering, data science,
Speaker 4: it info SEC kind of like all the tech stuff.
Speaker 4: And now I'm founder and CEO of Jedify. We're a
Speaker 4: company kind of trying to make it easier and faster
Speaker 4: to turn your ideas into working software. And anyways, today
Speaker 4: we've been playing with AI and agents and how they
Speaker 4: can help with the software develop process.
Speaker 2: I mean, you you've got that opening so well nailed down.
Speaker 2: I can tell this isn't the first time you've.
Speaker 3: Had yourself think you.
Speaker 1: Got the elevator pitch nailed well.
Speaker 2: I mean, it's sort of funny because like, I meet
Speaker 2: with a lot of people, and a lot of them
Speaker 2: feel the need to sort of introduce themselves and say
Speaker 2: a lot about their background, even if it has nothing
Speaker 2: to do with the conversation at all, And some of
Speaker 2: them just go on and on for minutes, and I'm like,
Speaker 2: you're mentioning other people's names and I don't know who
Speaker 2: any of those people are, and you're talking like they're
Speaker 2: the most important people in the world, and like I
Speaker 2: for sure, like I don't care. I mean, at least
Speaker 2: you start to company name, so like, you know, that's
Speaker 2: great on you. And I think a lot of people
Speaker 2: know most of those companies. I think most of them
Speaker 2: still exist by their original name, so you know, that's
Speaker 2: what's important.
Speaker 1: I was raised in a Midwest Protestant community, just really
Speaker 1: deep for it's just not saying anything. You're gonna mute me,
Speaker 1: aren't you.
Speaker 2: I got, I got. I actually got sucked for a
Speaker 2: second because I spent not short period of time in
Speaker 2: the Midwest after I graduated from the university because it
Speaker 2: was the only place that I could get hired as
Speaker 2: a job. I wasn't actually a software engineer by trade,
Speaker 2: and so I ended up there and I was like,
Speaker 2: do I have a connection for this, Like can I
Speaker 2: talk about the religious values? I'm like, not really. I
Speaker 2: actually I actually don't know a lot there, Like I've
Speaker 2: never really been a huge into the religious scene, so
Speaker 2: I can't say anything. And so I just was like,
Speaker 2: I'm just gonna keep my mouth shut.
Speaker 1: I was trying. I was thinking like in the second
Speaker 1: Austin Powers movie, whenever doctor Evil goes to therapy with
Speaker 1: his son Scott and they're like, tell us about yourself,
Speaker 1: and doctor Evil gets up and he does like this
Speaker 1: long rant about how he was raised by his father
Speaker 1: and it goes really really deep. But I couldn't remember
Speaker 1: enough of it to try to work it into it
Speaker 1: as a joke.
Speaker 2: Here, So don't don't worry. You'll have plenty of opportunity
Speaker 2: to go look up that for for next time.
Speaker 1: So today chick for the next S episode.
Speaker 2: I don't spoil it now. I really like the idea
Speaker 2: of getting back on topic of dev boxes, environment machines,
Speaker 2: Like I feel like this is a huge challenge that
Speaker 2: still exists. And you know, it's interesting because I work
Speaker 2: personally and like over the years and a lot of
Speaker 2: different ideas and a lot of different software languages, and
Speaker 2: I've always hated it, Like there was never a time
Speaker 2: where I'm like, you know what, working in this language,
Speaker 2: this is the best. And I feel like, you know,
Speaker 2: if you're in the space, you've got some horror stories.
Speaker 2: You've got some like like how did you even get
Speaker 2: into this?
Speaker 3: You know?
Speaker 2: Is why is this where you went?
Speaker 3: Yeah?
Speaker 4: So man, So I mean I guess a lot of
Speaker 4: these stories start with kind of scratching your own edge
Speaker 4: and trying to solve a problem that you have. So,
Speaker 4: you know, just having worked at different software companies, I've
Speaker 4: seen those stories where things work on your computer and
Speaker 4: they don't work on your teammates computer, or there's a
Speaker 4: bug in production where you can't reproduce it locally, and
Speaker 4: often that's tied to the environment. And so when we
Speaker 4: started edifying, we were like, you know what, from the beginning,
Speaker 4: let's just set up some reproducible the environments. And we
Speaker 4: decided to set up some Docker files for every kind
Speaker 4: of piece of software that we were working on. But
Speaker 4: very quickly we run into some pains because you know,
Speaker 4: Docker uses a lot of resources on your computer, and
Speaker 4: so when you're developing locally, like shipping to production great,
Speaker 4: but developing locally like I don't know, the fan spinning
Speaker 4: on your laptop, you do a little change and you
Speaker 4: want to see the results, and oh, it was the
Speaker 4: wrong line in the doctor file, so you busted the
Speaker 4: entire cash and now you're like rebuilding the entire container.
Speaker 4: And so all of that felt really, really painful. And
Speaker 4: so we had heard about this technological nicks. We wanted
Speaker 4: to try it. We kind of felt two things at once.
Speaker 4: One it was like, this is awesome.
Speaker 3: It kind of creates reproducible environments.
Speaker 4: And then we were like, oh my god, the developer
Speaker 4: experience and UX for this thing is not what we
Speaker 4: wanted to be. So anyways, we ended up building an
Speaker 4: open source tool called debox that kind of wraps nicks
Speaker 4: and lets you create those environments locally. And you know,
Speaker 4: I just posted it on Hacker News before.
Speaker 3: We knew it.
Speaker 4: It was like number one on Hacker News that day,
Speaker 4: and so kind of told those oh, people are having
Speaker 4: this problem that they want some solution, and that's yeah,
Speaker 4: that's how we started with the developer environment work.
Speaker 2: I love the story of like pain and suffering through
Speaker 2: development process because it's such an unusual I mean, it's
Speaker 2: such a common strategy for innovation, right, Like I as
Speaker 2: a software engineer, you can fix your own tools in
Speaker 2: a way, and so realistically, like when that happens, it's like, well,
Speaker 2: I'm going to make something better that doesn't it isn't
Speaker 2: as bad as the thing we're using. It's interesting you
Speaker 2: brought up docer. One of the things I've seen a
Speaker 2: lot is engineering teams attempting to get the development experience
Speaker 2: on their local machine working through Docker. And this is
Speaker 2: something I've never understood, and maybe this is something you
Speaker 2: could jump into if you have some understanding about it,
Speaker 2: Like why do that? Like, if I have the source
Speaker 2: code on my machine that I've cloned from get repository somewhere,
Speaker 2: why why not just install the couple of tools I
Speaker 2: need and you know, hit F five and run it.
Speaker 2: What does doctor really get me there?
Speaker 3: Yeah?
Speaker 4: So I mean I think the issue forget doctor for
Speaker 4: a second, like why isn't it enough, you know doctor
Speaker 4: or no doctor? Why isn't it enough just to check
Speaker 4: out the code, install the tools, and color to day.
Speaker 4: What happens is that in many cases, you don't just
Speaker 4: need the source code. You need the specific versions of
Speaker 4: the tools that you're working with. So it's not like
Speaker 4: I need I don't know Go or Python? Is I
Speaker 4: need Python two point seven point eleven?
Speaker 3: Right?
Speaker 2: Hopefully not anymore?
Speaker 3: Yeah?
Speaker 1: Ussion but just well but but see, I mean you
Speaker 1: say that.
Speaker 4: But like what happens a lot is is you're working
Speaker 4: at a large company, uh, and there's this project that
Speaker 4: everybody depends on, but it's not staffed well enough because
Speaker 4: all the other things became.
Speaker 3: More important, and you got to keep that thing running.
Speaker 1: Right every day everybody depends on it, and nobody's willing
Speaker 1: to update exactly there is.
Speaker 2: I think we should just pause for a moment here,
Speaker 2: because I know a non trivial amount of our audience
Speaker 2: members are like suffering through some sort of PTSD withdrawal
Speaker 2: right now. Having said that, like, or you know, you're
Speaker 2: working in some environment and you know some code that
Speaker 2: hasn't been touched in a while is not working, Like
Speaker 2: I feel like that's like ten percent of your job,
Speaker 2: whatever it is. You know, you have no idea what's
Speaker 2: going on there, and it's not working, and you didn't
Speaker 2: write the code. You may be lucky if you knew
Speaker 2: the person who did write it. Because so one of
Speaker 2: one of my first jobs I had, there was an
Speaker 2: engineer who if I saw his name on a commit message,
Speaker 2: I know that there was gonna be a bug with
Speaker 2: that code that was going to cause me to lose
Speaker 2: the rest of my best I like, if I'm debugging
Speaker 2: around and I see a line of code by bye
Speaker 2: by by Craig, uh, that's it, Like I'm sorry, Like
Speaker 2: I know it's gone.
Speaker 3: So but but anyway.
Speaker 4: I mean, yeah, that's the point, Like you often have
Speaker 4: to work with these projects and they often need specific
Speaker 4: versions of software, and if you don't use those versions
Speaker 4: of those tools, things.
Speaker 3: Bring, right.
Speaker 4: And so I guess in a in a project where
Speaker 4: you haven't set up too much around developer environments, you
Speaker 4: might just have some read me file saying install Python. Uh,
Speaker 4: and so everybody is installing Python, but somebody install Python.
Speaker 4: You know a year ago, somebody installed Python six months ago,
Speaker 4: somebody in Stoll python today. They all got different versions
Speaker 4: of Python, right, and you compound that by all the
Speaker 4: tools you might need in a project, right, so it's
Speaker 4: not just Python. But maybe let's say you're using protocol buffers,
Speaker 4: you have a compiler, you know, and so on, and so.
Speaker 3: Yeah, I think that it's rude, like that's.
Speaker 4: That mismatch of versions is kind of what people are
Speaker 4: trying to solve that is not solved by just to
Speaker 4: read missaying install these tools and start working on the project.
Speaker 2: Yeah, I definitely agree. I mean there's a huge aspect here,
Speaker 2: which I think has to do with the languages themselves
Speaker 2: and their package managers are part of that. I got
Speaker 2: it working once. Let's not go back and touch it
Speaker 2: by even the language maintainers themselves. You know, it's very
Speaker 2: limited experience there, and so they go off they you know,
Speaker 2: the packaging manager for whatever language you're using, like it's working,
Speaker 2: and so there are some that are definitely not as
Speaker 2: great as others. And then you just compile that on
Speaker 2: and put that usage as a dependency for it literally
Speaker 2: everyone who uses that language and framework. And I still
Speaker 2: think there are languages today that no one is happy
Speaker 2: with the package manager for that language, and yet somehow
Speaker 2: they're getting over it.
Speaker 4: And I mean there's languages where you have five package managers.
Speaker 2: So I have a feeling that you are throwing the
Speaker 2: JavaScript environment under the bus there a little bit. But
Speaker 2: I have to say that JavaScript is one of no
Speaker 2: JS specifically, is one of the best development communities compared
Speaker 2: to the other ones. Like yes, there's en package managers,
Speaker 2: but all of them like work at least ninety nine
Speaker 2: percent of the way, And there are problems with them
Speaker 2: for sure, individually, you know, they're not so totally perfectly robust,
Speaker 2: but you know, compared to some other ones, like I
Speaker 2: don't I don't struggle to pull out a new jobascript
Speaker 2: package manager. I do struggle to figure out how to
Speaker 2: make Maven work, even though I've gotten it to work
Speaker 2: like a hundred times before, and.
Speaker 4: I haven't touched the GEMA world in such a long time.
Speaker 4: I have no comments.
Speaker 1: Yeah, uh, just had a little bile come up in
Speaker 1: the back road.
Speaker 3: You know.
Speaker 2: The thing is like I remember seeing sharp like new
Speaker 2: Get was not a good package repository. The tool wasn't great.
Speaker 2: Even dot Net improved a lot of things there, but
Speaker 2: they had a diamond dependency resolution problem where like you
Speaker 2: depend on two dependencies who require different versions, and and
Speaker 2: no j has like figured out how to get around that.
Speaker 2: NPM solved it, yarn't solved it, PMPM, they all have
Speaker 2: solutions there. So yeah, I mean the individual tools aren't great,
Speaker 2: and it's not great that there's lots of them, but
Speaker 2: at least I feel like they're not getting in my way.
Speaker 2: But yeah, I like, if I'm programming in C plus
Speaker 2: plus or something old school with it for like a
Speaker 2: large company, then not only are there dependencies that are
Speaker 2: probably commit to a repository, like they're not in some
Speaker 2: package manager somewhere, they're like the binary dependencies are like
Speaker 2: in the source code and they require some Microsoft reddistributable
Speaker 2: C plus plus library from twenty years ago that I
Speaker 2: have to go and download from the internet from some
Speaker 2: suspicious website in order to even you know, build my
Speaker 2: project to get it to run somewhere.
Speaker 3: Yeah, agreed, So yeah.
Speaker 4: Anyways, I think it's all those paper cards that just
Speaker 4: slow you down in your development process, and all of
Speaker 4: a sudden, you are, I don't know, you wanted to
Speaker 4: work on a feature, right, you wanted to work on
Speaker 4: something related to the project that you're interested in, and
Speaker 4: you find yourself independency hell and just going deep into
Speaker 4: package managers when all you wanted to do was your
Speaker 4: working on an actual feature.
Speaker 1: Well, I think that's I think another aspect of that
Speaker 1: is onboarding new developers on your team, right, Like, how
Speaker 1: long does it take to get them where they're actually
Speaker 1: working on the code that you hired them to work
Speaker 1: on versus setting up an environment to actually see what
Speaker 1: they're actually supposed to be building.
Speaker 2: That sounds like a question to air our dirty laundry
Speaker 2: on the show, So I guess I'll go first. You know,
Speaker 2: it's like a couple of weeks to a month, and
Speaker 2: then I think we know that the long tail turnaround
Speaker 2: is like ninety days to get to peak performance and
Speaker 2: then it's like six months to do effective reviews at
Speaker 2: that point. Before that, I think there is a very
Speaker 2: long time where there's still a lot of training and
Speaker 2: education that goes in not but for us, it's mostly
Speaker 2: domain specific, like if we don't hire a domain expert
Speaker 2: for our company, we deal a lot in the authentication
Speaker 2: authorization space, So like how Samuel works, how oh off
Speaker 2: two works, how open id works, how SCIM works, how
Speaker 2: audit trails work. Like, there's a lot of things that
Speaker 2: you don't want to get wrong in a way, and
Speaker 2: so there are some tools. So but it's a long
Speaker 2: period of time. Will you're smiling there, so I feel like,
Speaker 2: you know you're going to tell me that you know
Speaker 2: your new hires that I believe you were interested in
Speaker 2: interviewing and getting into your team. They're like a couple
Speaker 2: of weeks max, and they're already a performance.
Speaker 1: So this is definitely an outlier. So I won't say
Speaker 1: that this is the state of our environment by any means.
Speaker 1: But the last guy I hired about a month ago,
Speaker 1: he was pushing production commits by the end of the week,
Speaker 1: which is good or bad, right, Okay, he was pushing
Speaker 1: functional production commits that actually did beneficial things.
Speaker 2: Wow, that's amazing.
Speaker 1: Yeah, but it was like it was a very unique situation,
Speaker 1: you know what. We hired him and he came in
Speaker 1: with a very specific set of skills that we needed.
Speaker 1: So he looked at our environment and was like, oh, yeah,
Speaker 1: here's where it's wrong and immediately started contributing to that.
Speaker 1: And then there's like the rest of the company, like
Speaker 1: the Web three space that we live in, where he
Speaker 1: still has a lot of onboarding to do before he
Speaker 1: fully understands that. So it was cool to see and
Speaker 1: I paraded it around the company just because that's the
Speaker 1: kind of person I am. But you know, and like
Speaker 1: the big picture of things, you know, he couldn't have
Speaker 1: done a domain specific commit in that first week.
Speaker 2: Yeah, are you using something like some sort of thermal
Speaker 2: development environment or like how was he able to get
Speaker 2: acclimated so quickly?
Speaker 1: Yeah, it was over. It was on our Kubernetes clusters,
Speaker 1: and so he had a deep Kubernetes experience and just
Speaker 1: looked at the way that ours were configured and was like, oh, yeah,
Speaker 1: here's here's something that's going to help you out. And
Speaker 1: then he also had a lot of OI d C experience,
Speaker 1: so he removed we were maintaining a lot of permissions
Speaker 1: and roles via pull requests and get repository and so
Speaker 1: he refactored that to use OI DC so that you
Speaker 1: just get it inherited when you authenticate.
Speaker 3: Yeah.
Speaker 4: I mean, I think the okay, you guys presented two
Speaker 4: different like onboarding experiences. But I think regardless for me,
Speaker 4: you want to spend the time creating the main specific knowledge.
Speaker 4: You want to spend the time kind of teaching and
Speaker 4: mentoring and having your your employees grow. I think you
Speaker 4: want to take away there. I'm just fighting with the
Speaker 4: tools and this should be a solve problem for sure. Yeah,
Speaker 4: I still have to deal with this now for I
Speaker 4: don't know, to days, just to get my environment set
Speaker 4: up before I can even start, you know, doing a
Speaker 4: little co change and testing it.
Speaker 1: Yeah, I totally agree. Like, are none of our customers
Speaker 1: benefit from the time and effort we spend into working
Speaker 1: on Kubernetes. They only benefit from the time and effort
Speaker 1: we spend building a better product.
Speaker 2: That they that they use.
Speaker 1: And I think that's what you're alluding to and warned
Speaker 1: to catch on a point you made earlier there, you know,
Speaker 1: dealing with ephemeral environments. We recently refactored a bunch of
Speaker 1: ours because we had this one VM instance with I
Speaker 1: don't know, it had hundreds of gigs of RAM on it,
Speaker 1: and it was a bunch of Doctor composed files and
Speaker 1: when you needed a new integration environment, you would copy
Speaker 1: and paste one of the existing Doctor composed directories and
Speaker 1: then go modify the stuff that you needed in there.
Speaker 1: And so we refactored that to use Helm charts. So
Speaker 1: now the launching of a new environment, you just add
Speaker 1: a new Helm Helme values file for the environment that
Speaker 1: you want and then let Helm build that out for you.
Speaker 2: Yeah. So I will say that NIXT is something that's
Speaker 2: always scared me personally, Like I feel like I never
Speaker 2: wasn't an opportunity where it was like, you know what,
Speaker 2: I'm going to reach for that new tool. So like,
Speaker 2: you know, congratulations for you know, seeing that as an
Speaker 2: opportunity and going at it and then building the UI
Speaker 2: on top of it to make it happen. What do
Speaker 2: you feel like the turnaround time is for getting new
Speaker 2: engineers acclimated to using the dev boxes through through NICKS,
Speaker 2: so they come into a team, they have no Knicks experience,
Speaker 2: no experience with maybe Thermerald dev environments. You know, they're
Speaker 2: all doing software development on their local machine. What's the
Speaker 2: turnaround time for them understanding and being able to execute effectively?
Speaker 4: Yeah, I mean, assuming we're talking about an environment where
Speaker 4: the tools needed are kind of available in the NIXT
Speaker 4: package store and kind of the versions are well represented,
Speaker 4: I think it's very very fast because I mean dev Box,
Speaker 4: the tool we built, it presents an interface that's very
Speaker 4: similar to package managers from different languages that you're already
Speaker 4: familiar with. Right, So you know, in you know we're
Speaker 4: talking about JavaScript earlier, you have that package the adjacent file.
Speaker 4: Well in that book, you have that books the adjacent file,
Speaker 4: and it lists a bunch of packages that you need
Speaker 4: for your project. And you might just say, you know
Speaker 4: Python and the version, and it might say the product
Speaker 4: of filer and the version. And it has a command
Speaker 4: to add a new package, to search for a new package,
Speaker 4: to remove a new package. And so what we try
Speaker 4: to do is kind of present that Nick's functionality but
Speaker 4: under aux that people are already familiar with, and because
Speaker 4: people are already familiar with it from other tools.
Speaker 3: I think it's a quick learning curve.
Speaker 2: How does this affect sort of software development cycle? So
Speaker 2: I make a change in my code, we're talking about
Speaker 2: impacting the Docker built cash and forcing it to rebuild.
Speaker 2: Like what's the corollary for dev blox? Do I do
Speaker 2: the software development locally? And then it uses some sort
Speaker 2: of remote server somewhere where the dev blox is being
Speaker 2: spin up?
Speaker 4: Like you know, yeah, so that box is our open
Speaker 4: source project. It lets you declare your environment in this
Speaker 4: Jason file and then the idea is that you can
Speaker 4: take the adjacent file and reproduce that environment. In many places,
Speaker 4: we we offer kind of on our paid solutions, ways
Speaker 4: to spin those environments on the cloud. But just to quarify,
Speaker 4: like that box is just going to be the open
Speaker 4: source tool and let you kind of do it locally.
Speaker 4: And the way it works is it is, like I said,
Speaker 4: it uses next behind the scenes. In practice, there's no
Speaker 4: container locally. I mean you can you can choose to
Speaker 4: use that box inside a container if you're building a container,
Speaker 4: but you can also choose to do it outside of
Speaker 4: a container if you don't need one. And what's really
Speaker 4: happening is that Nick's kind of installs all the versions
Speaker 4: of all the software that you need, and what's known
Speaker 4: as the next store. Think of this as some path
Speaker 4: in your local drive where every binary or piece of
Speaker 4: software that's installed is stored under a content hash, so
Speaker 4: that the path for it is unique. So even if
Speaker 4: you start installed five versions of Python, they will each
Speaker 4: have its own content hash and therefore will live in
Speaker 4: a different place in that store. And so anyways like that,
Speaker 4: it's able to install all sorts of versions of software,
Speaker 4: even if they would be technically conflicting. And then when
Speaker 4: you start an environment, it kind of sets the paths
Speaker 4: to point to the appropriate versions of things.
Speaker 3: And now you're working locally.
Speaker 2: So is it changing the environment variables and the alternatives
Speaker 2: that are set on my operating system to make the
Speaker 2: environment match what's necessary for that?
Speaker 3: Yes?
Speaker 4: Yes, so it will change, for example, change your path
Speaker 4: environment variable. And so it might you might say, Okay,
Speaker 4: in this project, we need in this version of Python,
Speaker 4: this version of the product of compiler. Oh, I know
Speaker 4: those live over here in the next store. Let me
Speaker 4: set up the path such that you're going to have
Speaker 4: access to those binaries and not the other ones.
Speaker 2: Okay, sure that makes sense.
Speaker 1: Is that done like system wide or is that done
Speaker 1: on like a project per pect.
Speaker 3: Just one a project basis.
Speaker 4: So so the analogy I make if we want to
Speaker 4: your reference existing tools for particular languages, it's kind of
Speaker 4: like on Python you have.
Speaker 3: Virtual type of environments. Imagine that, but now kind of like.
Speaker 4: At the system level for the tools and compilers that
Speaker 4: you're using for a project.
Speaker 2: So done correctly is what I'm hearing.
Speaker 3: We think, So for in your own opinion.
Speaker 2: I mean, I think virtual m is like a quintessential
Speaker 2: example of it doesn't get much worse than that. You know,
Speaker 2: maybe maybe someone can point me in the right direction there,
Speaker 2: but I don't remember. Like usually things automatically get committed
Speaker 2: to memory for me if they're easy to understand, you know,
Speaker 2: like I think everyone has this. You know, you see something,
Speaker 2: it's like all I can remember what that is. There's
Speaker 2: a le of things.
Speaker 3: Three months later you come back and you're.
Speaker 2: Right, that's the thing. Is like, I don't touch it enough.
Speaker 2: So the only time I actually touch virtual end at
Speaker 2: all is when I'm generating new h cert box certificates
Speaker 2: from let's encrypt on one of my remote servers, And
Speaker 2: that happens once every ninety days and every ninety days
Speaker 2: I cannot remember how to do that. So you know
Speaker 2: there's Celsius something is wrong with that technology for sure.
Speaker 2: So you know it's great when there's an easy way
Speaker 2: that makes sense to set it up and exit the machine.
Speaker 2: And that of course, it has to be processed specific
Speaker 2: and local so that you can potentially run multiple the
Speaker 2: same time, so that you can run multiple pieces of
Speaker 2: like different services that have to communicate with each.
Speaker 3: Other, right, right, makes sense?
Speaker 2: How do I how do I know that? Early on
Speaker 2: in my career it was always like, I don't need
Speaker 2: that garbage. I'm just going to do it myself. And
Speaker 2: later in my career I've definitely gotten to the point
Speaker 2: of I should definitely listen to those things and not
Speaker 2: think that I could do it with myself. So like,
Speaker 2: what's the advice here? You know, how do you know
Speaker 2: you're in a situation where whatever you're doing needs to
Speaker 2: be upgraded to using next through dev box like Quintessential.
Speaker 4: Yeah, I mean, I think let me maybe start with
Speaker 4: the cases where I don't think you necessarily need it.
Speaker 4: I think those tend to be you you're working on
Speaker 4: a team with this pretty standardized stack, probably have one
Speaker 4: particular programming language that you're focused on, and for the
Speaker 4: most part, all you need are the tools for that library,
Speaker 4: and you're not bringing into many external tools, right. I
Speaker 4: think as soon as you start having more kind of
Speaker 4: system level dependencies, additional tools that you're bringing in, maybe
Speaker 4: multiple languages across the company, then tools, you know, things
Speaker 4: to control the environment start making more sense. You just
Speaker 4: to grab an example. Let's see you're coding in Python.
Speaker 4: If you've standardized a version of Python and you're building
Speaker 4: a pure Python libraries, I don't know, you probably don't
Speaker 4: need the tool like this, but as the range al
Speaker 4: these days, and you decide to do some mL stuff, right,
Speaker 4: and you need some Python libraries that really depend on
Speaker 4: native libraries, and so now all those native libraries are
Speaker 4: going to get pulled and you're doing some image processing
Speaker 4: and so you need like image magic installed in the container,
Speaker 4: and so it depends on a particular version of image Magic.
Speaker 4: And so now your environment shifting from kind of pure
Speaker 4: Python to kind of like these extra dependencies. And I
Speaker 4: think that's where this makes sense, because you can now
Speaker 4: treat those extra dependencies. That's something that you track and
Speaker 4: version and pin as opposed to I don't know, just
Speaker 4: will West and you know, pull whatever is available today,
Speaker 4: and yeah it might be different yesterday from tomorrow.
Speaker 3: I give you.
Speaker 4: You know, some examples I give earlier were only on
Speaker 4: the development phase. But like imagine you you're doing image
Speaker 4: processing in production and the container requires a particular version
Speaker 4: of image Magic installed. Like now we're talking about production
Speaker 4: environments too, that if you're not careful, are just kind
Speaker 4: of changing nearly willing How does.
Speaker 2: The conversion from using next through dev box to get
Speaker 2: your local development environment. I mean, I feel like you
Speaker 2: open the can of worms here, so uh, when you
Speaker 2: do that locally, how does that invert to using getting
Speaker 2: those same dependencies ready for deployment production?
Speaker 4: Yes, to be fair, I do think that's an area
Speaker 4: we're continuing to improved of early days on the shift
Speaker 4: from using the box from development to production. But high level,
Speaker 4: I mean, you can craft a doctor file and a
Speaker 4: container in many different ways and really just kind of
Speaker 4: running any command you want right and setting up the
Speaker 4: environment inside the container.
Speaker 3: So what we think is that.
Speaker 4: In general, you can have one definition of the next
Speaker 4: packages that you need, and that one definition can be
Speaker 4: used to set up your developer environment. But it can
Speaker 4: also be used to install those same packages in the
Speaker 4: production container.
Speaker 3: Uh.
Speaker 4: Of course, you have to differentiate between things that are
Speaker 4: needed in both production and development versus things that are
Speaker 4: needed just for development.
Speaker 3: Now, I said it is work in progress.
Speaker 4: I think something that I feel Knicks packages haven't been
Speaker 4: optimized too much for in the past, and I think
Speaker 4: it is one of the challenges with what I'm saying.
Speaker 3: Is that.
Speaker 4: I don't think Knicks packages are optimized for size, and
Speaker 4: so sometimes they pull in a lot of things into
Speaker 4: the container and the container grows kind of bigger than
Speaker 4: you want it to be.
Speaker 3: So I think that's something that.
Speaker 4: Still needs to be improved within, you know, in terms
Speaker 4: of using Knicks for creating these types of production containers.
Speaker 2: Yeah, but I mean, I think it's a huge step
Speaker 2: in the right direction because the dependency management through Docker
Speaker 2: has always been oh yeah, just install like use Pseudo
Speaker 2: and then install a huge list of packages that are
Speaker 2: required to run here, and like what version those are,
Speaker 2: whether or not they work with the actual Linux os
Speaker 2: that you're actually running there compared to the one on
Speaker 2: your development machine or in your you know, in your
Speaker 2: depth box or whatever could be totally different or what's
Speaker 2: available production because you're not actually running the exact container.
Speaker 2: There are some differences there. And then there's also your
Speaker 2: CICD pipeline which probably have a different version of Linux on.
Speaker 3: Right, And this is exactly this is.
Speaker 2: Just if you're using Linux, right, I mean, then you
Speaker 2: throw in ARM and other operating systems and it's it's
Speaker 2: for sure a huge mass. So I mean, I definitely
Speaker 2: see that. Persistence is another aspect like Nix doesn't. You're
Speaker 2: focused on using it for dev boxes in sort of
Speaker 2: ephemeral way, right, Like you're not making permanent changes to
Speaker 2: the operating system, but in production you almost want the
Speaker 2: permanence there. So there is this sort of disconnect, and
Speaker 2: you're achieving the connection there through using doctor or plugging
Speaker 2: into persisting it in the doctoring container build file assume.
Speaker 4: Yeah, exactly, exactly right, Yeah, And I mean I think
Speaker 4: you touched on this, but just to emphasize it, I
Speaker 4: think sometimes people don't think about it. A doctor file
Speaker 4: is not necessarily reproducible. In fact, I argue it's probably
Speaker 4: not reproducible. People sometimes think of doctor as reproduce in
Speaker 4: the sense that if you previously built a particular image
Speaker 4: and you reuse that image, then you know you're getting
Speaker 4: you're getting what you already packaged before.
Speaker 3: Right.
Speaker 4: But if you have a doctor file with instructions and
Speaker 4: you build it six months ago and you build it today,
Speaker 4: the output, the resulting image will likely be different because again,
Speaker 4: I mean it's the same scenarios talking about with with
Speaker 4: your engineers just installing the environment. Probably your doctor file
Speaker 4: just says install Python, and six months ago the version
Speaker 4: of Python, you know, was X.
Speaker 3: Today the version of Python is Y.
Speaker 4: And so you build a container six months ago versus today,
Speaker 4: you get different images as a result.
Speaker 2: I think code dependencies definitely fall. Code dependencies fall in
Speaker 2: this abyss here right there. They're not quite infrastructure as code,
Speaker 2: which has been solved as far as deploying what we
Speaker 2: have to our cloud environment, and they're not the source code,
Speaker 2: which is stored so elegantly in there is this middle
Speaker 2: area where I need to configure that container or heaven forbid,
Speaker 2: you're using a virtual machine directly or bear metal in
Speaker 2: a very specific way, and it's like, well it's not
Speaker 2: I see, and it's not my source code, and then
Speaker 2: what I'm end up with is some sort of script
Speaker 2: in Bash. A lot of times to explain how to
Speaker 2: actually install that thing. And so it's like, well, why
Speaker 2: aren't we doing declarative programming here like we are everywhere else.
Speaker 2: It's like the white area that still is procedural and
Speaker 2: still prone to a lot of issues, And like, that's
Speaker 2: what I'm hearing is like, Okay, this is totally solved now.
Speaker 2: You know, if you need more than one dependency, how
Speaker 2: can you not be using this solution?
Speaker 3: Right? Yeah, exactly.
Speaker 4: I think you said in a way that I agree,
Speaker 4: Like it'd be great to just have your developer environment
Speaker 4: defined in a declarative way. And then that declaration, you know,
Speaker 4: we can use to build the environment correctly, and you
Speaker 4: can use it to build the environment in many different places.
Speaker 4: You can build it locally, you can build it inside
Speaker 4: a container, you can build it on the cloud, right,
Speaker 4: and yeah, take the environment with you where you need it.
Speaker 1: Will's processing all of this, I can say, I am,
Speaker 1: but I think I failed to press the turbo button
Speaker 1: on the CPU this morning because it's processing slow. I'm
Speaker 1: just trying to slow this morning.
Speaker 2: That's okay, it happens to all of us, right.
Speaker 1: So with the when it comes back to the production dependencies,
Speaker 1: is that declarative in the manifest file this dependency is
Speaker 1: a production one or in this one's a development dependency.
Speaker 4: Right now, if you want to talk about like the
Speaker 4: implementation we have in the in the open source school,
Speaker 4: you would have like two different books adjacent files, one
Speaker 4: for production, one for development. But we have talked about
Speaker 4: you know, potentially UNI find that in one file.
Speaker 3: Actually we were making.
Speaker 4: The analogy to package the jacent before and in the
Speaker 4: package that Jason, you have the dependencies and dependencies that
Speaker 4: are for both. So yeah, we're we're considering cannot unifying that,
Speaker 4: but yeah, I mean put aside the exact implementation we
Speaker 4: have today, the exact version. I think the idea is,
Speaker 4: can we declare our environments by specifying the packages we
Speaker 4: want on the versions that we want, and then specifying
Speaker 4: which packages belong in both development and production and which
Speaker 4: ones belong only in development, and then take away that
Speaker 4: very procedural like Bash scripts that you're doing to install
Speaker 4: every little thing, and then let a system that can
Speaker 4: reproducibly create the environment.
Speaker 3: From that declaration you do the work.
Speaker 2: I think you hit on really well the scenario where
Speaker 2: you may not need to do something like this, like
Speaker 2: you don't need the I hesitate to call it complexity
Speaker 2: of using NEXT based configuration, but it's just it's one
Speaker 2: more tool, and so.
Speaker 3: No I would call it like, I mean, it is
Speaker 3: an extra tool. Yeah, I agree with you, Like.
Speaker 2: Yeah, you know, compared to I mean, swap that in
Speaker 2: for something else. I mean, it's pretty much a very
Speaker 2: simple bootstrap and clear configuration files that are very understandable
Speaker 2: of what they're doing. I'm curious if you see a
Speaker 2: particular vertical or industry market segment where it becomes super critical. Still,
Speaker 2: Like maybe if we say web servers, web development, maybe
Speaker 2: that's really simple. It's not too complicated here. But if
Speaker 2: you're building a game, I'm just making stuff up now,
Speaker 2: like you know, this is where you can jump in
Speaker 2: and say, you know, this is exactly it, like game
Speaker 2: or web three. Like, maybe there's a lot more value
Speaker 2: in or doing something with LMMEL, a lot more value
Speaker 2: in next configurations.
Speaker 4: Yeah, I think, I mean, I think one of them
Speaker 4: is when you have a lot of when you need
Speaker 4: to use native dependencies that are not native to the
Speaker 4: language that you're using. Right, So I think Python and
Speaker 4: mL is a good example. But you know, it's not
Speaker 4: just the male. Like you know, there's several cases where
Speaker 4: you might need to install Python libraries that depend on
Speaker 4: kind of underlying C library.
Speaker 3: I think those tend.
Speaker 4: To not work as well if you don't have a
Speaker 4: tool like like that book.
Speaker 3: I think the other.
Speaker 4: Case is when you rely i mean made similar but
Speaker 4: maybe it's not a library in the scenario, but you're
Speaker 4: you're relying on a binary that needs to be present
Speaker 4: in the environment because you want to call, you know,
Speaker 4: exect that binary or do something with it.
Speaker 2: How much of a mess is mL development from a
Speaker 2: dependency management today?
Speaker 3: Oh say no more think I think it can get complicated.
Speaker 4: Yeah, because and then yeah, but by the way, you know,
Speaker 4: I'm talking about that box and mL, but there's even
Speaker 4: cases where we are not able to help fully in
Speaker 4: that vertical because then kind of drivers come in, uh,
Speaker 4: you know, for the different GPUs and whatnot, and and
Speaker 4: there's some things that NICKS can't fully control there unless
Speaker 4: you were using next os. Right, But we're remember we're
Speaker 4: doing this in a scenario where you keep your current computer,
Speaker 4: current operating system and just use that as added virtual environment.
Speaker 3: But yeah, you can you can get real.
Speaker 4: Messy, not just with the library dependencies, but kind of
Speaker 4: like all the related drivers and how they match the
Speaker 4: corresponding architecture that you're developing.
Speaker 2: Again, so you wouldn't recommend doing mL development today quite yet.
Speaker 2: I mean we wait a couple more years though.
Speaker 4: Yeah, yeah, I mean I think it's unimproved. Maybe I'm
Speaker 4: saying I think it's unimprovement, but it's not all the
Speaker 4: way then.
Speaker 2: Well, I do remember, and I'll date myself here when
Speaker 2: I was trying to extract bitcoins from the bitcoin faucet
Speaker 2: when I was in the university, and that included having
Speaker 2: to basically getting my operating system to a state where
Speaker 2: I would just ended up with a black screen with
Speaker 2: a terminal, you know, and it's like, Okay, you know,
Speaker 2: I really mess something up here, And like that was
Speaker 2: the state of the software at the time to just
Speaker 2: do anything related in the cryptocurrency space in cryptography in general,
Speaker 2: really good luck using those tools. So like that's definitely improved.
Speaker 2: Now you can use Python or JavaScript to interact with
Speaker 2: any of the chains that are available out there ethereum
Speaker 2: and all the derivatives of it without a problem for
Speaker 2: the most part. I mean, there are some educases for sure.
Speaker 2: If you want a stake or mine, you're going to
Speaker 2: have a problem. And I think that's where IML is right.
Speaker 2: It used me much worse, and I think it's slightly better.
Speaker 2: But I do feel like maybe we're going so quickly
Speaker 2: and we've monetized whatever little amount of value we've been
Speaker 2: able to create as a society through model development, that
Speaker 2: we haven't really took the opportunity to innovate with the
Speaker 2: development pipeline.
Speaker 3: Yeah, and you know, I'm sure this.
Speaker 4: I do think there's probably plenty of people working in
Speaker 4: solving this, even you know, the AI wave and the
Speaker 4: whole interest in.
Speaker 1: You.
Speaker 2: I mean, it's I think it's I think it's optimistic,
Speaker 2: but I also I also think that may be a
Speaker 2: little bit realistic because with the deep state model that
Speaker 2: had come out, I mean deep seek right, Uh that.
Speaker 1: It I thought we were in a different podcast.
Speaker 2: Yeah, I mean it's a joke that you know, I
Speaker 2: frequently go around with some of my colleagues about you
Speaker 2: know the impact of these models. But the fact that
Speaker 2: you are there's now competition requires doing things more efficiently.
Speaker 2: You can't bring a new engineer on board and have
Speaker 2: them waste months and months not on on tool set,
Speaker 2: like not even on the domain. So that's what drives
Speaker 2: innovation there a lot when you're in competition to succeed.
Speaker 2: And I think that's going to be a really important
Speaker 2: qualifier going forward in the mL space that we continue
Speaker 2: to have sufficient competition to drive innovation there.
Speaker 1: So this is an area where I haven't spent any
Speaker 1: time or effort, but Daniel, you have, right because you
Speaker 1: have been working on some mL work to do QAT
Speaker 1: in testing. Right, So what's the what's the barrier to
Speaker 1: entry to get into that space?
Speaker 3: I meaning, like, what's the barrier for.
Speaker 4: Engineers to start creating agents themselves and kind of what
Speaker 4: knowledge and tools might they need?
Speaker 2: Yeah? For sure?
Speaker 3: Yeah.
Speaker 4: So yeah, I mean, like, like you said, we are
Speaker 4: developing an agent that we call test Pilot. It's meant
Speaker 4: to be an ai QA engineer take away all that boring,
Speaker 4: repetitive manual clicking on UIs that people often do.
Speaker 3: Yeah, and we've been learning a lot.
Speaker 4: I mean, I think I think agents are relatively like
Speaker 4: big investments in agents are relatively new, and so I
Speaker 4: think the barrier of entry is something you can catch
Speaker 4: up on, Like, like I would actually encourage if you're
Speaker 4: interested in creating agents. I think it's a good time.
Speaker 4: I think you can learn. I think you can do it,
Speaker 4: I would say, I mean to me, one of the
Speaker 4: most important things you need to do with developing AI
Speaker 4: applications is set up a.
Speaker 3: Evaluation framework for the work that you're doing.
Speaker 4: So in non AI non you know, kind of traditional
Speaker 4: software development lets, you say, you can often write a
Speaker 4: function that you can reason about fairly well, and then
Speaker 4: you can write tests that are fairly deterministic to know
Speaker 4: whether your the behavior is what you expect right as
Speaker 4: soon as you involve the LLLM. Well, now you have
Speaker 4: non probabilistic like not sorry, non deterministic behavior, right, And
Speaker 4: so you might call it once and it might tell
Speaker 4: you something. You might call it again in a second,
Speaker 4: and i I'll tell you a completely different thing. And
Speaker 4: often because you're dealing with very general cases, you might
Speaker 4: be feeding in lots of different examples and you don't
Speaker 4: fully grasp how it's behaving.
Speaker 3: Across all those different examples.
Speaker 4: So to me, kind of like one of the first
Speaker 4: things you need to do is set up a bunch
Speaker 4: of examples with kind of like the expected behavior and
Speaker 4: then be able to measure kind of the changes that
Speaker 4: you're doing and whether they're improving that evaluation.
Speaker 3: That you have in mind.
Speaker 4: All right, and in this case, it's going to be
Speaker 4: not purely past fail. Is going to be like, Okay,
Speaker 4: today you're at fifty percent of being able to handle
Speaker 4: the cases that you want, and then you do a
Speaker 4: few changes and okay, now you're at fifty five percent,
Speaker 4: and then you're at sixty percent and so on.
Speaker 3: So yeah, anyways, I don't know, I can probably think
Speaker 3: of more.
Speaker 4: I mean best practices, but that's the first one that
Speaker 4: comes to mind.
Speaker 2: I would even maybe take a step back before that.
Speaker 2: I mean, that's really great insight, like someone who hasn't
Speaker 2: spent a lot of time in it. Is there a
Speaker 2: particular site or foundational model that is where you would
Speaker 2: suggest people start going for first? I mean probably, I
Speaker 2: think from my own experience, I know that building models
Speaker 2: is super expensive, so they're probably not going to do that.
Speaker 2: But maybe there's a SCE out there that you say,
Speaker 2: you know, go pull this model and try to fine
Speaker 2: tune it or use it out of the box. Like
Speaker 2: any recommendations on that.
Speaker 4: Yeah, I mean I do think you need to start
Speaker 4: with the with the pre existing in general model, I
Speaker 4: recommend using probably a kind of multimodel provider, something like
Speaker 4: open router that allows you to easily swap models, because
Speaker 4: what's happening is I mean you guys see on the
Speaker 4: on the tech news all the time, right, every other month,
Speaker 4: each AI vendor like outdoes each other, and it's like, oh,
Speaker 4: we have this new new model and it performs better
Speaker 4: on this benchmark. And so you don't want to tie
Speaker 4: yourself so strongly to one model that when those innovations
Speaker 4: are happening, you can't test the other one, right, So
Speaker 4: I would pick some provider like, like I said, open
Speaker 4: Router is one of them. I think Rock. I think
Speaker 4: cloud Flaer has something these days too, But anyways, a
Speaker 4: provider that lets you spat which model you want to
Speaker 4: use and then kind of build the agent on top
Speaker 4: of that. I think the other part of building an
Speaker 4: agent is not just the underlying model that helps you reason,
Speaker 4: but there's kind of how you prompt it, right, and
Speaker 4: kind of how you tell it to do what you
Speaker 4: want it to do.
Speaker 3: And then the last.
Speaker 4: Piece is kind of like the tools that you give
Speaker 4: it so that it can take action.
Speaker 3: Right.
Speaker 4: So a high level in my mind, like an agent
Speaker 4: is you're grabbing the LLLM for the reasoning. But you
Speaker 4: need to kind of give it hands and eyes, if
Speaker 4: you want to think about it that way, so that
Speaker 4: it can manipulate an environment that you're giving it control of.
Speaker 3: Right.
Speaker 1: Is that something that you've got a significant portion of
Speaker 1: your team dedicated to or is it something that you
Speaker 1: can iterate on quickly with just a few people, like
Speaker 1: how big of a how big of a man power?
Speaker 4: Yeah, I mean most of our team is working on
Speaker 4: on test pilots.
Speaker 3: I mean.
Speaker 4: Something we've found is so we're a Go shop. Like
Speaker 4: we use Go as our primary programming language. We've been
Speaker 4: using it for for a while and so we wanted
Speaker 4: to continue using it. And what we're seeing is that
Speaker 4: a lot of the tools that are readily available in
Speaker 4: this space are Python based, and you know, we can
Speaker 4: use Python for stuff that's not production, but for our
Speaker 4: production stuff we want to continue and go and so
Speaker 4: we're having to build some of them ourselves and go.
Speaker 3: We do plan to open source them later on and
Speaker 3: kind of.
Speaker 4: Give back to the community and hopefully kind of people
Speaker 4: that like Go, you know, let let them have some
Speaker 4: readily available tools for building AI and agents. But yeah,
Speaker 4: that stuff that's getting built right now, and often we
Speaker 4: look for it and it's like, oh, I guess nobody
Speaker 4: has you know, done this type of library yet in
Speaker 4: goal for AI, So it is early days.
Speaker 2: Yeah, I mean I think that's a really good point actually,
Speaker 2: Like it's usually a mistake to rely on Python for
Speaker 2: high reliability software development and production, and so if the
Speaker 2: tools aren't there to support your production workloads in a
Speaker 2: different language, then that really tells you how early it is,
Speaker 2: because the people that are utilizing those tools don't care
Speaker 2: about like sustainable code or reliable code because it doesn't
Speaker 2: offer the core features to guarantee safety or quality of
Speaker 2: your value of your service that you're building.
Speaker 4: Yeah, like I I mean there's many different ways of
Speaker 4: building a company, but just I've had to deal with
Speaker 4: like Groovy production systems, Python production systems, all that stuff,
Speaker 4: and I'm kind of like in this case where we're
Speaker 4: making different language and kind of moving forward with that.
Speaker 2: Yeah. No, I'm totally with you. I used to be
Speaker 2: like early on in my career. I'm just like every
Speaker 2: language is the same, right, you know, there's you can
Speaker 2: swap between one or another if you've had a sufficient
Speaker 2: years of experience. But then over time you start to
Speaker 2: realize the fringe benefits of individual languages or negatives that
Speaker 2: are associated with them, So things like Go or other
Speaker 2: well typed languages like c sharp, the community or the
Speaker 2: packages that are available to help drive forward. Having the
Speaker 2: right execution of the right stuff in production is like
Speaker 2: super important, not which is when you're building something quick
Speaker 2: and hacky, but when you actually care about it working continuously,
Speaker 2: and like if you're at a company and one of
Speaker 2: the core things you're building is reliable builds, I feel
Speaker 2: like you know that goes hand in hand or using
Speaker 2: other I mean, it's sort of like a different mindset, right,
Speaker 2: Like if you're hiring an engineer and they care about
Speaker 2: using scrappy tools that ones that potentially break from day
Speaker 2: to day, the nightly build version of that tool is
Speaker 2: broken by default, they're probably not in the right mindset
Speaker 2: to be working on something where you need high reliability
Speaker 2: or reproducible builds or you know these other technologies. It's
Speaker 2: just a complete other side of the spectrum, right.
Speaker 3: Yeah, I totally agree.
Speaker 4: And yeah, I mean what we've been doing maybe where
Speaker 4: we kind of allow Python to come in is you know,
Speaker 4: I mentioned we have this kind of evail process. We're
Speaker 4: able to try ideas, and so I'm like, okay, if
Speaker 4: we're prototyping a direction and there happens to be kind
Speaker 4: of like a Python library that does that off the bat,
Speaker 4: and we just want to check if that's gonna improve
Speaker 4: our performance, you know, the performance of our agent. Like
Speaker 4: you prototype something quickly with Python, pull that library in,
Speaker 4: run it against our evil framework. Let's see if this
Speaker 4: direction has legs. Oh, it does have lengths. It is promising. Okay,
Speaker 4: how are we going to build it now for production usage?
Speaker 4: And you handle the low we want to handle, and
Speaker 4: take care of all the edge cases we want to
Speaker 4: take care of and so on, and then we at
Speaker 4: that point in time, then we spend the time developing
Speaker 4: the the go software and infrastructure.
Speaker 2: Makes a lot of sense.
Speaker 1: So that's one of the things I want to ask you.
Speaker 1: You know, you talked about open sourcing some of the
Speaker 1: tools that you're making and contributing back to the community.
Speaker 1: That's a huge commitment and it's a it's a conscious
Speaker 1: decision by companies because a lot of companies are like, yeah,
Speaker 1: we invested this, it's a core part of our business.
Speaker 1: We're using it for the business. But then you're saying,
Speaker 1: I want to I want to give it back to
Speaker 1: the community. How do you how do you tie that
Speaker 1: back into your business model and and like justify or
Speaker 1: account for the time that and the salaries that you're
Speaker 1: spending on people working on an open source product.
Speaker 4: Yeah, it's a great question, and you know, I'm sure
Speaker 4: everybody will kind of make different judgments there, But for us,
Speaker 4: the way we think about it is, Okay, we're developing tests.
Speaker 3: I love this kind of q A A I q
Speaker 3: A engineer.
Speaker 4: You can think of the software that makes up test
Speaker 4: Pilot as being into different buckets list in my hand. Right,
Speaker 4: one is like the truly domain specific logic and kind
Speaker 4: of proprietary aspects of it that make it behave like
Speaker 4: a QA engineer, right, and that we're not open sourcing.
Speaker 4: But then there's the tools that we wish we had
Speaker 4: had that that we had to develop for ourselves that
Speaker 4: are applicable to developing all sorts of AI agents and
Speaker 4: all sorts of kind of.
Speaker 3: AI related products.
Speaker 4: And you know, we're a startup, We are trying to
Speaker 4: get more eyeballs. We're trying to kind of become more prominent,
Speaker 4: get people to hear about us, and so I actually
Speaker 4: think of open sourcing almost as a marketing uh exercise. Uh,
Speaker 4: we don't have marketers, we're all you know, we're very
Speaker 4: technical teams basically engineers. But sometimes I think you can
Speaker 4: look as open sources as a way to kind of
Speaker 4: bring in that audience. And because our audience are developers, right,
Speaker 4: I think if we're in a different vertical, you're serving
Speaker 4: different customers, maybe open sourcing doesn't work that way. But
Speaker 4: our audience is developers, and if we can create a
Speaker 4: reputation of Wow, those guys that genify really know what
Speaker 4: they're doing in terms of AI agents and have you
Speaker 4: checked you know, have you seen the tools they've made
Speaker 4: and the libraries they've made. I think that all just
Speaker 4: kind of comes back to the type of business that
Speaker 4: we're building.
Speaker 2: Right, I totally agree. I mean we went down the
Speaker 2: same path a while ago with some of the things
Speaker 2: that we've opened source. It is like unfortunate to say
Speaker 2: it is like a huge marketing story. It's marketing story
Speaker 2: for brand awareness. It's a marketing story for hiring engineers,
Speaker 2: so they know about your company or what you do
Speaker 2: before that you know, they show up the end interview.
Speaker 2: They can research it and see that you have something
Speaker 2: technical rather than a website with a couple of links
Speaker 2: on it and a knowledge base like that's not a lot, right,
Speaker 2: you know, this gives them more ability to have insight
Speaker 2: in what you're doing. I mean, there is a pessimistic
Speaker 2: side of me that says, like, no one cares what
Speaker 2: you put on the internet, Like, no one's that special, right,
Speaker 2: your thing doesn't matter. I remember like a bunch of
Speaker 2: times like Netflix went and open source a bunch of
Speaker 2: huge technologies, and I'm just like, I don't care about
Speaker 2: those things. But now some people maybe are using that
Speaker 2: in production. I guess could be potentially because I think
Speaker 2: they wrote them for Kubernetes or something, and you know,
Speaker 2: so that's great, you know, you get that experience. I
Speaker 2: think the one thing we try to be really careful
Speaker 2: about that I think is important here is that the
Speaker 2: thing that you're building needs to be really close to
Speaker 2: your value, your actual product, because even if it's like
Speaker 2: one or two hops away, then it's the people that
Speaker 2: are utilizing it. They don't care what that thing is
Speaker 2: that you're actually doing, which is super unfortunate. But I
Speaker 2: will say that rather than a marketing tool. There's a
Speaker 2: second step here, which is if you're going to open
Speaker 2: source something, I feel like you feel like it's more
Speaker 2: important to do it well. And things that are done
Speaker 2: well will live longer and people will see and start
Speaker 2: to utilize more. And that give brings us sort of
Speaker 2: like a your own usage levels it up because you
Speaker 2: actually invested in doing that thing well. And so like,
Speaker 2: even if you weren't a deb box, wasn't sort of
Speaker 2: out there as a cloud thing that you could utilize
Speaker 2: and pay for the fact that you've opened source it
Speaker 2: means that that tool is now serving you, hopefully better
Speaker 2: for your mL development, for your own agent.
Speaker 4: Right, yeah, absolutely, And I actually we're about like different
Speaker 4: benefits that open source my brain. I'll give you a
Speaker 4: third one, at least in my view, which is, you know,
Speaker 4: when you're a startup, I think one of the most
Speaker 4: important things is be talking to users, right, be talking
Speaker 4: to users and learning from them. And we've found that
Speaker 4: when we open source things, we often get to talk
Speaker 4: to more users, but we also take get to talk
Speaker 4: to users that normally would not have talked to us.
Speaker 4: So you take that box. I don't know if we
Speaker 4: can share the specific companies. But there's some big ish
Speaker 4: companies using that box that I think if we had
Speaker 4: just kind of pure proprietary solution and or this little startup,
Speaker 4: they never would have looked at it. I think what
Speaker 4: gave them confidence to look at it is like, oh,
Speaker 4: there is this open source project kind of you know,
Speaker 4: worst case scenario, the startup goes nowhere, we can rely
Speaker 4: on that open source project and we can kind of
Speaker 4: continue improving it and whatnot. And so that's given us
Speaker 4: at least a channel to communicate with those types of
Speaker 4: potential customers and users, and it brings in a ton
Speaker 4: of learning for us in terms of Okay, what features
Speaker 4: are important, what is that class of user or customer
Speaker 4: asking us about and how should that I affect our
Speaker 4: product roadmap and whatnot.
Speaker 1: Oh that's huge, like in my opinion, like one of
Speaker 1: the like the primary objective of every startup is to
Speaker 1: figure out the difference between what you built and what
Speaker 1: you what your customers thought you were building. And so
Speaker 1: open sourcing gives you a way to have that conversation
Speaker 1: without trying to force them down through a sales pipeline, right,
Speaker 1: exactly right.
Speaker 2: It's scary though, because you know when you put it
Speaker 2: out there, it's like, well, people start will start to
Speaker 2: see my actual bad code, or it is scary my
Speaker 2: commitment messages you know that maybe I should have worded differently.
Speaker 4: And I mean it is tricky too, and like, you know,
Speaker 4: I gave you our kind of like ar heuristic for
Speaker 4: figuring out what parts of the spilot are open source
Speaker 4: and which ones aren't. But sometimes you're in some grade
Speaker 4: lines and you're kind of like should I open source
Speaker 4: this or not? Like is it Do I get more
Speaker 4: benefit from opening it up to the community and getting
Speaker 4: the audience and maybe getting some faster adoption, or do
Speaker 4: I get more benefit from keeping this kind of like
Speaker 4: the secret sounds of the company. And you know, there's
Speaker 4: all those stories like Elastic Search and AUS and.
Speaker 3: Everybody gets big and starts changing their licenses.
Speaker 2: So like, son, how I don't know how right I
Speaker 2: am here. We've done some initial research on the topic,
Speaker 2: and it does actually seem like customers that fall into
Speaker 2: a particular bucket never leave that bucket and go to
Speaker 2: a different one. Like a free user never converts to
Speaker 2: your lowest tier paid plan, and the average business plan
Speaker 2: doesn't escalate to enterprise. It just doesn't happen. People self
Speaker 2: identify with whatever bucket they're in, and so whether or
Speaker 2: not you have an open source solution really only functions
Speaker 2: as advertisement. The people who are using it aren't going
Speaker 2: to be more likely to get to like start to
Speaker 2: pay you. And one of the things we recognize, which
Speaker 2: is one of the reasons why our product, authoress is
Speaker 2: not open source for ninety percent of it, is because
Speaker 2: those free users actually generate ninety nine percent of the
Speaker 2: support cases that you need to deal with. So there's
Speaker 2: like a huge cost investment that goes into it, and
Speaker 2: you don't get any reward coming back from that. I mean,
Speaker 2: of course there's some gray areas where there's highly coupled
Speaker 2: technology and where it's being utilized effectively, but often the
Speaker 2: one the customers that are going to pay you the
Speaker 2: most money or be your best customers, like, they don't
Speaker 2: particularly care about running it themselves in most cases, unless
Speaker 2: they're like a government entity or you know, they have
Speaker 2: some regulations they have to deal with.
Speaker 4: Yeah, I mean usually if they really carry some compliance requirement,
Speaker 4: but that might just be pointing to like you need
Speaker 4: to solve that compliance requirement as well, right.
Speaker 2: Like, oh yeah, I mean, compliance doesn't mean security, so
Speaker 2: like that's a completely different story.
Speaker 3: Yeah.
Speaker 1: Absolutely, it's an additional chex right on. That was that
Speaker 1: was intense.
Speaker 3: A good way.
Speaker 1: Yeah, yeah, yeah, definitely in a good way. Definitely a
Speaker 1: good way.
Speaker 2: I mean, I know we jumped through a lot of
Speaker 2: different topics, Daniel. If you feel like there's something, you know,
Speaker 2: one last thing you want to share regarding either dev
Speaker 2: box or the QA copilot test pilot, you know.
Speaker 1: Have about it, let me think famous last words, go.
Speaker 3: Oh man, a lot of pressure. I mean, I don't know.
Speaker 3: I guess I'll just said maybe this.
Speaker 4: Is less about the specific products and just kind of
Speaker 4: things I'm excited about. But I am bullish on on
Speaker 4: AI agents. I mean, obviously I'm building one, but I
Speaker 4: just think there's more and more parts of software development
Speaker 4: that could be taking care of through agents, and I
Speaker 4: think it can be done in a way where we
Speaker 4: take away that kind of repetitive work and we still
Speaker 4: leave a lot of the fun and creativity to the humans.
Speaker 4: I know there's more doomsday scenarios that people talk about
Speaker 4: right where it's like, oh, we're all out of jobs
Speaker 4: and whatnot, But I'm actually hoping it's more like, now
Speaker 4: we can all achieve more because those less interesting parts
Speaker 4: of the cycle are taking care of automatically, and now
Speaker 4: you can focus more on system level thinking, on product
Speaker 4: level thinking and so on. So yeah, maybe I am
Speaker 4: an optimist like you guys were saying before.
Speaker 1: I'm just gonna say to that, no comment, I'm gonna
Speaker 1: agree with you, Daniel, like, regardless regardless of what technology does.
Speaker 1: I know, there's always going to be problems, and I've
Speaker 1: I've made a career out of solving problems, so I'm
Speaker 1: not too worried about it.
Speaker 2: Okay, I'm super worried about it. Should I share?
Speaker 1: Yeah?
Speaker 2: Absolutely, So you.
Speaker 4: Can worried about it, but like, let's make the world
Speaker 4: we want.
Speaker 3: Well.
Speaker 2: I feel like it's not a technology that's right for
Speaker 2: being democratized, for access for everyone to utilize equivalently. The
Speaker 2: people with the most money and the most power will
Speaker 2: have the most powerful agents at their disposal, and that
Speaker 2: just entrenches their own power and prevents them from being ousted,
Speaker 2: and it doesn't leave a lot of opportunity for those
Speaker 2: that are already being disadvantaged to rise up and fight
Speaker 2: against that oppression.
Speaker 1: There's a documentary on this called Terminator check it out.
Speaker 1: I think I've heard it.
Speaker 2: I think it has a well known narrator, right, yeah, yeah, No,
Speaker 2: I mean that's that for me, is that's optimism, that
Speaker 2: documentary scenario.
Speaker 1: All right, Now that we've all made our way onto
Speaker 1: an FBI watch list, let's move on to picks.
Speaker 2: Yeah, sure, so I'll go first today. My pick is
Speaker 2: going to be The Martian, the book by Andy Weir.
Speaker 2: He has a whole bunch of books. I don't know
Speaker 2: if I'll say this one is the best, but it's
Speaker 2: definitely really good. They're all science fiction adjacent. I mean,
Speaker 2: they're not the reality played out in a hypothetical future
Speaker 2: where well it's new scenario, and they're great. I think
Speaker 2: it's really well written. One of the first things I
Speaker 2: ever read from Andy where was actually something called The Egg,
Speaker 2: which is a very short story which I find also
Speaker 2: really good. And I and many many years later, I
Speaker 2: read The Martian and I didn't realize that it was
Speaker 2: the same author. And it's quite interesting to have read
Speaker 2: two pieces by the same author and be like, wait,
Speaker 2: it was this other really good thing that was written
Speaker 2: in a similar way, and like going back and actually
Speaker 2: seeing it's like I actually read that thing. So if
Speaker 2: you haven't read it, I haven't seen the movie yet actually,
Speaker 2: but if you haven't read the book, I highly recommend it.
Speaker 1: Right, and I have done both. I've read the book
Speaker 1: and seen the movie. But I'm happy to hear you
Speaker 1: say that that his other books are better, because that's yeah, yeah.
Speaker 2: So I really liked the short story. He has a
Speaker 2: drawn comic with like an overlap between Alice in Wonderland
Speaker 2: and Dorothy from Wizard of Oz, and I forget who
Speaker 2: the third character is. That's okay. The Martians great. There's
Speaker 2: two other books. I think there's a fourth one coming out.
Speaker 2: They're not related at all. The one Luna I think
Speaker 2: for the Moon. It's not as good but still interesting.
Speaker 2: And then there is one with a Deep Space, which
Speaker 2: is also really fantastic.
Speaker 1: Right on? Cool, Daniel, what'd you bring for pick?
Speaker 3: Awesome?
Speaker 4: So I have a book as well. It's Wild Robot.
Speaker 4: So I think I mentioned earlier. I have two young kids,
Speaker 4: an eight year old and a five year old, and
Speaker 4: so I do a lot of bedtime routine and reading
Speaker 4: with them, and so while Robot is one of the
Speaker 4: books that I'm reading with one of my kids by
Speaker 4: Peter Brown, there's a movie as well, kind of similar
Speaker 4: to the example you gave, But yeah, I just find
Speaker 4: it really interesting to read with my kids, your high level.
Speaker 4: It's about a robot that founds itself in an island,
Speaker 4: doesn't know where it came from, and eventually needs to
Speaker 4: take care of an animal, and it kind of makes
Speaker 4: you reflect on all these questions, like, you know, where
Speaker 4: his parenthood is this robot taking care of this animal?
Speaker 4: You know, is that a parent? What's the relationship between
Speaker 4: that animal and the robot?
Speaker 3: And I don't know. I guess I get a little
Speaker 3: sentimental because I'm reading that.
Speaker 4: With my kids and I have just I've just found
Speaker 4: it like really interesting for both of us to go
Speaker 4: through the story.
Speaker 1: That's cool, that's super cool. All right, Well, we're gonna
Speaker 1: make it a three peat. I'm changing my pick and
Speaker 1: I'm going with a book as well, because Warren you
Speaker 1: said something about, you know, recognizing the writing style. A
Speaker 1: while back, I read a book and really enjoyed it,
Speaker 1: and I was like, Wow, this guy writes so much
Speaker 1: like Stephen King. It's just insane. And the author's name
Speaker 1: is Joe Hill. Turns out it's Stephen King's son, so oh, wow.
Speaker 3: Yeah.
Speaker 1: Yeah. The book the book I read was Heart Shaped Box.
Speaker 1: And if you like Stephen King's writing style, Joe Hill
Speaker 1: just seems to have picked up on that style and
Speaker 1: continued running along with it.
Speaker 2: Is it also like thriller suspense stuff or is it
Speaker 2: a different.
Speaker 1: Genre, same same stuff, same like sick twisted thing where
Speaker 1: you're like, oh wow, I did not see that one coming. Yeah,
Speaker 1: good stuff.
Speaker 2: Do you ever find it a challenge to like pick
Speaker 2: up a book that you know is going to be
Speaker 2: like that and go through it and be like, I'm
Speaker 2: I don't know if I can keep going, like I'm
Speaker 2: really afraid of what I'm going to read next.
Speaker 1: No, I don't. I don't think so.
Speaker 2: Yeah, He's like, I love it. That's what I look
Speaker 2: forward to.
Speaker 1: Right, Yeah, I mean that. I guess. I guess the
Speaker 1: rest of that conversation is for me and my therapist.
Speaker 2: But well, don't worry, we'll shut the camera off here
Speaker 2: in a moment and conversation.
Speaker 1: Or it'll be like no, no, I promise I stopped recording.
Speaker 1: Tell us all about it.
Speaker 2: Well, you see this little red light on the top
Speaker 2: left hand corner.
Speaker 1: It's just a U I glitch. Don't worry about it,
Speaker 1: and thank you so much for joining us. This has
Speaker 1: been a blast, that's fantastic.
Speaker 3: I've enjoyed as well. Thank you guys for having me.
Speaker 1: All right, and to all of our listeners, hopefully you
Speaker 1: enjoyed it as well. Thank you for listening, and we'll
Speaker 1: see everyone next week.