Speaker 1: And we're live with another episode of Adventures in DevOps.
Speaker 2: Almost blew the podcast name right there, Warren, Well, you know, thanks.
Speaker 1: Catching, I've only done I've villain done a few hundred
of these at this point. It's it's still a learning curve.
So how are you, Warren, to bring us a security
fact today?
Speaker 2: Well, you know, I think again. I had brought one
last time, and I thought, you know, this time for
sure our fellow co host was going to be here
at share, the long awaited one, Matt. I had been
promised at it, you know, coincidence not here, so I
don't have anything today.
Speaker 1: She's pausing for dramatic effect, like a really really dramatic effect.
Speaker 2: It's gonna be the best fact ever. I swear it.
Speaker 1: When it actually drops, it is it is right well.
Also joining us Daniel Loretto from Jedifhy. Daniel, Welcome to
the show man.
Speaker 3: Thank you so much. Will happy to be here.
Speaker 1: Dude, I'm excited to have you here. So you are
with Jedify. Now tell us a little bit about your background.
Speaker 3: Yes, I'm a software engineered by trade.
Speaker 4: You know, I've worked at a lot of different tech
companies of all different sizes, so I can talk about,
you know, different experiences. But I worked at Google, Airbnb, Twitter,
I worked at you know, tiny startups that you've never
heard of. I worked at Verta Health, which is a
telemedicine company for the ABDIS, where I ran engineering, data science,
it info SEC kind of like all the tech stuff.
And now I'm founder and CEO of Jedify. We're a
company kind of trying to make it easier and faster
to turn your ideas into working software. And anyways, today
we've been playing with AI and agents and how they
can help with the software develop process.
Speaker 2: I mean, you you've got that opening so well nailed down.
I can tell this isn't the first time you've.
Speaker 3: Had yourself think you.
Speaker 1: Got the elevator pitch nailed well.
Speaker 2: I mean, it's sort of funny because like, I meet
with a lot of people, and a lot of them
feel the need to sort of introduce themselves and say
a lot about their background, even if it has nothing
to do with the conversation at all, And some of
them just go on and on for minutes, and I'm like,
you're mentioning other people's names and I don't know who
any of those people are, and you're talking like they're
the most important people in the world, and like I
for sure, like I don't care. I mean, at least
you start to company name, so like, you know, that's
great on you. And I think a lot of people
know most of those companies. I think most of them
still exist by their original name, so you know, that's
what's important.
Speaker 1: I was raised in a Midwest Protestant community, just really
deep for it's just not saying anything. You're gonna mute me,
aren't you.
Speaker 2: I got, I got. I actually got sucked for a
second because I spent not short period of time in
the Midwest after I graduated from the university because it
was the only place that I could get hired as
a job. I wasn't actually a software engineer by trade,
and so I ended up there and I was like,
do I have a connection for this, Like can I
talk about the religious values? I'm like, not really. I
actually I actually don't know a lot there, Like I've
never really been a huge into the religious scene, so
I can't say anything. And so I just was like,
I'm just gonna keep my mouth shut.
Speaker 1: I was trying. I was thinking like in the second
Austin Powers movie, whenever doctor Evil goes to therapy with
his son Scott and they're like, tell us about yourself,
and doctor Evil gets up and he does like this
long rant about how he was raised by his father
and it goes really really deep. But I couldn't remember
enough of it to try to work it into it
as a joke.
Speaker 2: Here, So don't don't worry. You'll have plenty of opportunity
to go look up that for for next time.
Speaker 1: So today chick for the next S episode.
Speaker 2: I don't spoil it now. I really like the idea
of getting back on topic of dev boxes, environment machines,
Like I feel like this is a huge challenge that
still exists. And you know, it's interesting because I work
personally and like over the years and a lot of
different ideas and a lot of different software languages, and
I've always hated it, Like there was never a time
where I'm like, you know what, working in this language,
this is the best. And I feel like, you know,
if you're in the space, you've got some horror stories.
You've got some like like how did you even get
into this?
Speaker 3: You know?
Speaker 2: Is why is this where you went?
Speaker 3: Yeah?
Speaker 4: So man, So I mean I guess a lot of
these stories start with kind of scratching your own edge
and trying to solve a problem that you have. So,
you know, just having worked at different software companies, I've
seen those stories where things work on your computer and
they don't work on your teammates computer, or there's a
bug in production where you can't reproduce it locally, and
often that's tied to the environment. And so when we
started edifying, we were like, you know what, from the beginning,
let's just set up some reproducible the environments. And we
decided to set up some Docker files for every kind
of piece of software that we were working on. But
very quickly we run into some pains because you know,
Docker uses a lot of resources on your computer, and
so when you're developing locally, like shipping to production great,
but developing locally like I don't know, the fan spinning
on your laptop, you do a little change and you
want to see the results, and oh, it was the
wrong line in the doctor file, so you busted the
entire cash and now you're like rebuilding the entire container.
And so all of that felt really, really painful. And
so we had heard about this technological nicks. We wanted
to try it. We kind of felt two things at once.
One it was like, this is awesome.
Speaker 3: It kind of creates reproducible environments.
Speaker 4: And then we were like, oh my god, the developer
experience and UX for this thing is not what we
wanted to be. So anyways, we ended up building an
open source tool called debox that kind of wraps nicks
and lets you create those environments locally. And you know,
I just posted it on Hacker News before.
Speaker 3: We knew it.
Speaker 4: It was like number one on Hacker News that day,
and so kind of told those oh, people are having
this problem that they want some solution, and that's yeah,
that's how we started with the developer environment work.
Speaker 2: I love the story of like pain and suffering through
development process because it's such an unusual I mean, it's
such a common strategy for innovation, right, Like I as
a software engineer, you can fix your own tools in
a way, and so realistically, like when that happens, it's like, well,
I'm going to make something better that doesn't it isn't
as bad as the thing we're using. It's interesting you
brought up docer. One of the things I've seen a
lot is engineering teams attempting to get the development experience
on their local machine working through Docker. And this is
something I've never understood, and maybe this is something you
could jump into if you have some understanding about it,
Like why do that? Like, if I have the source
code on my machine that I've cloned from get repository somewhere,
why why not just install the couple of tools I
need and you know, hit F five and run it.
What does doctor really get me there?
Speaker 3: Yeah?
Speaker 4: So I mean I think the issue forget doctor for
a second, like why isn't it enough, you know doctor
or no doctor? Why isn't it enough just to check
out the code, install the tools, and color to day.
What happens is that in many cases, you don't just
need the source code. You need the specific versions of
the tools that you're working with. So it's not like
I need I don't know Go or Python? Is I
need Python two point seven point eleven?
Speaker 3: Right?
Speaker 2: Hopefully not anymore?
Speaker 3: Yeah?
Speaker 1: Ussion but just well but but see, I mean you
say that.
Speaker 4: But like what happens a lot is is you're working
at a large company, uh, and there's this project that
everybody depends on, but it's not staffed well enough because
all the other things became.
Speaker 3: More important, and you got to keep that thing running.
Speaker 1: Right every day everybody depends on it, and nobody's willing
to update exactly there is.
Speaker 2: I think we should just pause for a moment here,
because I know a non trivial amount of our audience
members are like suffering through some sort of PTSD withdrawal
right now. Having said that, like, or you know, you're
working in some environment and you know some code that
hasn't been touched in a while is not working, Like
I feel like that's like ten percent of your job,
whatever it is. You know, you have no idea what's
going on there, and it's not working, and you didn't
write the code. You may be lucky if you knew
the person who did write it. Because so one of
one of my first jobs I had, there was an
engineer who if I saw his name on a commit message,
I know that there was gonna be a bug with
that code that was going to cause me to lose
the rest of my best I like, if I'm debugging
around and I see a line of code by bye
by by Craig, uh, that's it, Like I'm sorry, Like
I know it's gone.
Speaker 3: So but but anyway.
Speaker 4: I mean, yeah, that's the point, Like you often have
to work with these projects and they often need specific
versions of software, and if you don't use those versions
of those tools, things.
Speaker 3: Bring, right.
Speaker 4: And so I guess in a in a project where
you haven't set up too much around developer environments, you
might just have some read me file saying install Python. Uh,
and so everybody is installing Python, but somebody install Python.
You know a year ago, somebody installed Python six months ago,
somebody in Stoll python today. They all got different versions
of Python, right, and you compound that by all the
tools you might need in a project, right, so it's
not just Python. But maybe let's say you're using protocol buffers,
you have a compiler, you know, and so on, and so.
Speaker 3: Yeah, I think that it's rude, like that's.
Speaker 4: That mismatch of versions is kind of what people are
trying to solve that is not solved by just to
read missaying install these tools and start working on the project.
Speaker 2: Yeah, I definitely agree. I mean there's a huge aspect here,
which I think has to do with the languages themselves
and their package managers are part of that. I got
it working once. Let's not go back and touch it
by even the language maintainers themselves. You know, it's very
limited experience there, and so they go off they you know,
the packaging manager for whatever language you're using, like it's working,
and so there are some that are definitely not as
great as others. And then you just compile that on
and put that usage as a dependency for it literally
everyone who uses that language and framework. And I still
think there are languages today that no one is happy
with the package manager for that language, and yet somehow
they're getting over it.
Speaker 4: And I mean there's languages where you have five package managers.
Speaker 2: So I have a feeling that you are throwing the
JavaScript environment under the bus there a little bit. But
I have to say that JavaScript is one of no
JS specifically, is one of the best development communities compared
to the other ones. Like yes, there's en package managers,
but all of them like work at least ninety nine
percent of the way, And there are problems with them
for sure, individually, you know, they're not so totally perfectly robust,
but you know, compared to some other ones, like I
don't I don't struggle to pull out a new jobascript
package manager. I do struggle to figure out how to
make Maven work, even though I've gotten it to work
like a hundred times before, and.
Speaker 4: I haven't touched the GEMA world in such a long time.
I have no comments.
Speaker 1: Yeah, uh, just had a little bile come up in
the back road.
Speaker 3: You know.
Speaker 2: The thing is like I remember seeing sharp like new
Get was not a good package repository. The tool wasn't great.
Even dot Net improved a lot of things there, but
they had a diamond dependency resolution problem where like you
depend on two dependencies who require different versions, and and
no j has like figured out how to get around that.
NPM solved it, yarn't solved it, PMPM, they all have
solutions there. So yeah, I mean the individual tools aren't great,
and it's not great that there's lots of them, but
at least I feel like they're not getting in my way.
But yeah, I like, if I'm programming in C plus
plus or something old school with it for like a
large company, then not only are there dependencies that are
probably commit to a repository, like they're not in some
package manager somewhere, they're like the binary dependencies are like
in the source code and they require some Microsoft reddistributable
C plus plus library from twenty years ago that I
have to go and download from the internet from some
suspicious website in order to even you know, build my
project to get it to run somewhere.
Speaker 3: Yeah, agreed, So yeah.
Speaker 4: Anyways, I think it's all those paper cards that just
slow you down in your development process, and all of
a sudden, you are, I don't know, you wanted to
work on a feature, right, you wanted to work on
something related to the project that you're interested in, and
you find yourself independency hell and just going deep into
package managers when all you wanted to do was your
working on an actual feature.
Speaker 1: Well, I think that's I think another aspect of that
is onboarding new developers on your team, right, Like, how
long does it take to get them where they're actually
working on the code that you hired them to work
on versus setting up an environment to actually see what
they're actually supposed to be building.
Speaker 2: That sounds like a question to air our dirty laundry
on the show, So I guess I'll go first. You know,
it's like a couple of weeks to a month, and
then I think we know that the long tail turnaround
is like ninety days to get to peak performance and
then it's like six months to do effective reviews at
that point. Before that, I think there is a very
long time where there's still a lot of training and
education that goes in not but for us, it's mostly
domain specific, like if we don't hire a domain expert
for our company, we deal a lot in the authentication
authorization space, So like how Samuel works, how oh off
two works, how open id works, how SCIM works, how
audit trails work. Like, there's a lot of things that
you don't want to get wrong in a way, and
so there are some tools. So but it's a long
period of time. Will you're smiling there, so I feel like,
you know you're going to tell me that you know
your new hires that I believe you were interested in
interviewing and getting into your team. They're like a couple
of weeks max, and they're already a performance.
Speaker 1: So this is definitely an outlier. So I won't say
that this is the state of our environment by any means.
But the last guy I hired about a month ago,
he was pushing production commits by the end of the week,
which is good or bad, right, Okay, he was pushing
functional production commits that actually did beneficial things.
Speaker 2: Wow, that's amazing.
Speaker 1: Yeah, but it was like it was a very unique situation,
you know what. We hired him and he came in
with a very specific set of skills that we needed.
So he looked at our environment and was like, oh, yeah,
here's where it's wrong and immediately started contributing to that.
And then there's like the rest of the company, like
the Web three space that we live in, where he
still has a lot of onboarding to do before he
fully understands that. So it was cool to see and
I paraded it around the company just because that's the
kind of person I am. But you know, and like
the big picture of things, you know, he couldn't have
done a domain specific commit in that first week.
Speaker 2: Yeah, are you using something like some sort of thermal
development environment or like how was he able to get
acclimated so quickly?
Speaker 1: Yeah, it was over. It was on our Kubernetes clusters,
and so he had a deep Kubernetes experience and just
looked at the way that ours were configured and was like, oh, yeah,
here's here's something that's going to help you out. And
then he also had a lot of OI d C experience,
so he removed we were maintaining a lot of permissions
and roles via pull requests and get repository and so
he refactored that to use OI DC so that you
just get it inherited when you authenticate.
Speaker 3: Yeah.
Speaker 4: I mean, I think the okay, you guys presented two
different like onboarding experiences. But I think regardless for me,
you want to spend the time creating the main specific knowledge.
You want to spend the time kind of teaching and
mentoring and having your your employees grow. I think you
want to take away there. I'm just fighting with the
tools and this should be a solve problem for sure. Yeah,
I still have to deal with this now for I
don't know, to days, just to get my environment set
up before I can even start, you know, doing a
little co change and testing it.
Speaker 1: Yeah, I totally agree. Like, are none of our customers
benefit from the time and effort we spend into working
on Kubernetes. They only benefit from the time and effort
we spend building a better product.
Speaker 2: That they that they use.
Speaker 1: And I think that's what you're alluding to and warned
to catch on a point you made earlier there, you know,
dealing with ephemeral environments. We recently refactored a bunch of
ours because we had this one VM instance with I
don't know, it had hundreds of gigs of RAM on it,
and it was a bunch of Doctor composed files and
when you needed a new integration environment, you would copy
and paste one of the existing Doctor composed directories and
then go modify the stuff that you needed in there.
And so we refactored that to use Helm charts. So
now the launching of a new environment, you just add
a new Helm Helme values file for the environment that
you want and then let Helm build that out for you.
Speaker 2: Yeah. So I will say that NIXT is something that's
always scared me personally, Like I feel like I never
wasn't an opportunity where it was like, you know what,
I'm going to reach for that new tool. So like,
you know, congratulations for you know, seeing that as an
opportunity and going at it and then building the UI
on top of it to make it happen. What do
you feel like the turnaround time is for getting new
engineers acclimated to using the dev boxes through through NICKS,
so they come into a team, they have no Knicks experience,
no experience with maybe Thermerald dev environments. You know, they're
all doing software development on their local machine. What's the
turnaround time for them understanding and being able to execute effectively?
Speaker 4: Yeah, I mean, assuming we're talking about an environment where
the tools needed are kind of available in the NIXT
package store and kind of the versions are well represented,
I think it's very very fast because I mean dev Box,
the tool we built, it presents an interface that's very
similar to package managers from different languages that you're already
familiar with. Right, So you know, in you know we're
talking about JavaScript earlier, you have that package the adjacent file.
Well in that book, you have that books the adjacent file,
and it lists a bunch of packages that you need
for your project. And you might just say, you know
Python and the version, and it might say the product
of filer and the version. And it has a command
to add a new package, to search for a new package,
to remove a new package. And so what we try
to do is kind of present that Nick's functionality but
under aux that people are already familiar with, and because
people are already familiar with it from other tools.
Speaker 3: I think it's a quick learning curve.
Speaker 2: How does this affect sort of software development cycle? So
I make a change in my code, we're talking about
impacting the Docker built cash and forcing it to rebuild.
Like what's the corollary for dev blox? Do I do
the software development locally? And then it uses some sort
of remote server somewhere where the dev blox is being
spin up?
Speaker 4: Like you know, yeah, so that box is our open
source project. It lets you declare your environment in this
Jason file and then the idea is that you can
take the adjacent file and reproduce that environment. In many places,
we we offer kind of on our paid solutions, ways
to spin those environments on the cloud. But just to quarify,
like that box is just going to be the open
source tool and let you kind of do it locally.
And the way it works is it is, like I said,
it uses next behind the scenes. In practice, there's no
container locally. I mean you can you can choose to
use that box inside a container if you're building a container,
but you can also choose to do it outside of
a container if you don't need one. And what's really
happening is that Nick's kind of installs all the versions
of all the software that you need, and what's known
as the next store. Think of this as some path
in your local drive where every binary or piece of
software that's installed is stored under a content hash, so
that the path for it is unique. So even if
you start installed five versions of Python, they will each
have its own content hash and therefore will live in
a different place in that store. And so anyways like that,
it's able to install all sorts of versions of software,
even if they would be technically conflicting. And then when
you start an environment, it kind of sets the paths
to point to the appropriate versions of things.
Speaker 3: And now you're working locally.
Speaker 2: So is it changing the environment variables and the alternatives
that are set on my operating system to make the
environment match what's necessary for that?
Speaker 3: Yes?
Speaker 4: Yes, so it will change, for example, change your path
environment variable. And so it might you might say, Okay,
in this project, we need in this version of Python,
this version of the product of compiler. Oh, I know
those live over here in the next store. Let me
set up the path such that you're going to have
access to those binaries and not the other ones.
Speaker 2: Okay, sure that makes sense.
Speaker 1: Is that done like system wide or is that done
on like a project per pect.
Speaker 3: Just one a project basis.
Speaker 4: So so the analogy I make if we want to
your reference existing tools for particular languages, it's kind of
like on Python you have.
Speaker 3: Virtual type of environments. Imagine that, but now kind of like.
Speaker 4: At the system level for the tools and compilers that
you're using for a project.
Speaker 2: So done correctly is what I'm hearing.
Speaker 3: We think, So for in your own opinion.
Speaker 2: I mean, I think virtual m is like a quintessential
example of it doesn't get much worse than that. You know,
maybe maybe someone can point me in the right direction there,
but I don't remember. Like usually things automatically get committed
to memory for me if they're easy to understand, you know,
like I think everyone has this. You know, you see something,
it's like all I can remember what that is. There's
a le of things.
Speaker 3: Three months later you come back and you're.
Speaker 2: Right, that's the thing. Is like, I don't touch it enough.
So the only time I actually touch virtual end at
all is when I'm generating new h cert box certificates
from let's encrypt on one of my remote servers, And
that happens once every ninety days and every ninety days
I cannot remember how to do that. So you know
there's Celsius something is wrong with that technology for sure.
So you know it's great when there's an easy way
that makes sense to set it up and exit the machine.
And that of course, it has to be processed specific
and local so that you can potentially run multiple the
same time, so that you can run multiple pieces of
like different services that have to communicate with each.
Speaker 3: Other, right, right, makes sense?
Speaker 2: How do I how do I know that? Early on
in my career it was always like, I don't need
that garbage. I'm just going to do it myself. And
later in my career I've definitely gotten to the point
of I should definitely listen to those things and not
think that I could do it with myself. So like,
what's the advice here? You know, how do you know
you're in a situation where whatever you're doing needs to
be upgraded to using next through dev box like Quintessential.
Speaker 4: Yeah, I mean, I think let me maybe start with
the cases where I don't think you necessarily need it.
I think those tend to be you you're working on
a team with this pretty standardized stack, probably have one
particular programming language that you're focused on, and for the
most part, all you need are the tools for that library,
and you're not bringing into many external tools, right. I
think as soon as you start having more kind of
system level dependencies, additional tools that you're bringing in, maybe
multiple languages across the company, then tools, you know, things
to control the environment start making more sense. You just
to grab an example. Let's see you're coding in Python.
If you've standardized a version of Python and you're building
a pure Python libraries, I don't know, you probably don't
need the tool like this, but as the range al
these days, and you decide to do some mL stuff, right,
and you need some Python libraries that really depend on
native libraries, and so now all those native libraries are
going to get pulled and you're doing some image processing
and so you need like image magic installed in the container,
and so it depends on a particular version of image Magic.
And so now your environment shifting from kind of pure
Python to kind of like these extra dependencies. And I
think that's where this makes sense, because you can now
treat those extra dependencies. That's something that you track and
version and pin as opposed to I don't know, just
will West and you know, pull whatever is available today,
and yeah it might be different yesterday from tomorrow.
Speaker 3: I give you.
Speaker 4: You know, some examples I give earlier were only on
the development phase. But like imagine you you're doing image
processing in production and the container requires a particular version
of image Magic installed. Like now we're talking about production
environments too, that if you're not careful, are just kind
of changing nearly willing How does.
Speaker 2: The conversion from using next through dev box to get
your local development environment. I mean, I feel like you
open the can of worms here, so uh, when you
do that locally, how does that invert to using getting
those same dependencies ready for deployment production?
Speaker 4: Yes, to be fair, I do think that's an area
we're continuing to improved of early days on the shift
from using the box from development to production. But high level,
I mean, you can craft a doctor file and a
container in many different ways and really just kind of
running any command you want right and setting up the
environment inside the container.
Speaker 3: So what we think is that.
Speaker 4: In general, you can have one definition of the next
packages that you need, and that one definition can be
used to set up your developer environment. But it can
also be used to install those same packages in the
production container.
Speaker 3: Uh.
Speaker 4: Of course, you have to differentiate between things that are
needed in both production and development versus things that are
needed just for development.
Speaker 3: Now, I said it is work in progress.
Speaker 4: I think something that I feel Knicks packages haven't been
optimized too much for in the past, and I think
it is one of the challenges with what I'm saying.
Speaker 3: Is that.
Speaker 4: I don't think Knicks packages are optimized for size, and
so sometimes they pull in a lot of things into
the container and the container grows kind of bigger than
you want it to be.
Speaker 3: So I think that's something that.
Speaker 4: Still needs to be improved within, you know, in terms
of using Knicks for creating these types of production containers.
Speaker 2: Yeah, but I mean, I think it's a huge step
in the right direction because the dependency management through Docker
has always been oh yeah, just install like use Pseudo
and then install a huge list of packages that are
required to run here, and like what version those are,
whether or not they work with the actual Linux os
that you're actually running there compared to the one on
your development machine or in your you know, in your
depth box or whatever could be totally different or what's
available production because you're not actually running the exact container.
There are some differences there. And then there's also your
CICD pipeline which probably have a different version of Linux on.
Speaker 3: Right, And this is exactly this is.
Speaker 2: Just if you're using Linux, right, I mean, then you
throw in ARM and other operating systems and it's it's
for sure a huge mass. So I mean, I definitely
see that. Persistence is another aspect like Nix doesn't. You're
focused on using it for dev boxes in sort of
ephemeral way, right, Like you're not making permanent changes to
the operating system, but in production you almost want the
permanence there. So there is this sort of disconnect, and
you're achieving the connection there through using doctor or plugging
into persisting it in the doctoring container build file assume.
Speaker 4: Yeah, exactly, exactly right, Yeah, And I mean I think
you touched on this, but just to emphasize it, I
think sometimes people don't think about it. A doctor file
is not necessarily reproducible. In fact, I argue it's probably
not reproducible. People sometimes think of doctor as reproduce in
the sense that if you previously built a particular image
and you reuse that image, then you know you're getting
you're getting what you already packaged before.
Speaker 3: Right.
Speaker 4: But if you have a doctor file with instructions and
you build it six months ago and you build it today,
the output, the resulting image will likely be different because again,
I mean it's the same scenarios talking about with with
your engineers just installing the environment. Probably your doctor file
just says install Python, and six months ago the version
of Python, you know, was X.
Speaker 3: Today the version of Python is Y.
Speaker 4: And so you build a container six months ago versus today,
you get different images as a result.
Speaker 2: I think code dependencies definitely fall. Code dependencies fall in
this abyss here right there. They're not quite infrastructure as code,
which has been solved as far as deploying what we
have to our cloud environment, and they're not the source code,
which is stored so elegantly in there is this middle
area where I need to configure that container or heaven forbid,
you're using a virtual machine directly or bear metal in
a very specific way, and it's like, well it's not
I see, and it's not my source code, and then
what I'm end up with is some sort of script
in Bash. A lot of times to explain how to
actually install that thing. And so it's like, well, why
aren't we doing declarative programming here like we are everywhere else.
It's like the white area that still is procedural and
still prone to a lot of issues, And like, that's
what I'm hearing is like, Okay, this is totally solved now.
You know, if you need more than one dependency, how
can you not be using this solution?
Speaker 3: Right? Yeah, exactly.
Speaker 4: I think you said in a way that I agree,
Like it'd be great to just have your developer environment
defined in a declarative way. And then that declaration, you know,
we can use to build the environment correctly, and you
can use it to build the environment in many different places.
You can build it locally, you can build it inside
a container, you can build it on the cloud, right,
and yeah, take the environment with you where you need it.
Speaker 1: Will's processing all of this, I can say, I am,
but I think I failed to press the turbo button
on the CPU this morning because it's processing slow. I'm
just trying to slow this morning.
Speaker 2: That's okay, it happens to all of us, right.
Speaker 1: So with the when it comes back to the production dependencies,
is that declarative in the manifest file this dependency is
a production one or in this one's a development dependency.
Speaker 4: Right now, if you want to talk about like the
implementation we have in the in the open source school,
you would have like two different books adjacent files, one
for production, one for development. But we have talked about
you know, potentially UNI find that in one file.
Speaker 3: Actually we were making.
Speaker 4: The analogy to package the jacent before and in the
package that Jason, you have the dependencies and dependencies that
are for both. So yeah, we're we're considering cannot unifying that,
but yeah, I mean put aside the exact implementation we
have today, the exact version. I think the idea is,
can we declare our environments by specifying the packages we
want on the versions that we want, and then specifying
which packages belong in both development and production and which
ones belong only in development, and then take away that
very procedural like Bash scripts that you're doing to install
every little thing, and then let a system that can
reproducibly create the environment.
Speaker 3: From that declaration you do the work.
Speaker 2: I think you hit on really well the scenario where
you may not need to do something like this, like
you don't need the I hesitate to call it complexity
of using NEXT based configuration, but it's just it's one
more tool, and so.
Speaker 3: No I would call it like, I mean, it is
an extra tool. Yeah, I agree with you, Like.
Speaker 2: Yeah, you know, compared to I mean, swap that in
for something else. I mean, it's pretty much a very
simple bootstrap and clear configuration files that are very understandable
of what they're doing. I'm curious if you see a
particular vertical or industry market segment where it becomes super critical. Still,
Like maybe if we say web servers, web development, maybe
that's really simple. It's not too complicated here. But if
you're building a game, I'm just making stuff up now,
like you know, this is where you can jump in
and say, you know, this is exactly it, like game
or web three. Like, maybe there's a lot more value
in or doing something with LMMEL, a lot more value
in next configurations.
Speaker 4: Yeah, I think, I mean, I think one of them
is when you have a lot of when you need
to use native dependencies that are not native to the
language that you're using. Right, So I think Python and
mL is a good example. But you know, it's not
just the male. Like you know, there's several cases where
you might need to install Python libraries that depend on
kind of underlying C library.
Speaker 3: I think those tend.
Speaker 4: To not work as well if you don't have a
tool like like that book.
Speaker 3: I think the other.
Speaker 4: Case is when you rely i mean made similar but
maybe it's not a library in the scenario, but you're
you're relying on a binary that needs to be present
in the environment because you want to call, you know,
exect that binary or do something with it.
Speaker 2: How much of a mess is mL development from a
dependency management today?
Speaker 3: Oh say no more think I think it can get complicated.
Speaker 4: Yeah, because and then yeah, but by the way, you know,
I'm talking about that box and mL, but there's even
cases where we are not able to help fully in
that vertical because then kind of drivers come in, uh,
you know, for the different GPUs and whatnot, and and
there's some things that NICKS can't fully control there unless
you were using next os. Right, But we're remember we're
doing this in a scenario where you keep your current computer,
current operating system and just use that as added virtual environment.
Speaker 3: But yeah, you can you can get real.
Speaker 4: Messy, not just with the library dependencies, but kind of
like all the related drivers and how they match the
corresponding architecture that you're developing.
Speaker 2: Again, so you wouldn't recommend doing mL development today quite yet.
I mean we wait a couple more years though.
Speaker 4: Yeah, yeah, I mean I think it's unimproved. Maybe I'm
saying I think it's unimprovement, but it's not all the
way then.
Speaker 2: Well, I do remember, and I'll date myself here when
I was trying to extract bitcoins from the bitcoin faucet
when I was in the university, and that included having
to basically getting my operating system to a state where
I would just ended up with a black screen with
a terminal, you know, and it's like, Okay, you know,
I really mess something up here, And like that was
the state of the software at the time to just
do anything related in the cryptocurrency space in cryptography in general,
really good luck using those tools. So like that's definitely improved.
Now you can use Python or JavaScript to interact with
any of the chains that are available out there ethereum
and all the derivatives of it without a problem for
the most part. I mean, there are some educases for sure.
If you want a stake or mine, you're going to
have a problem. And I think that's where IML is right.
It used me much worse, and I think it's slightly better.
But I do feel like maybe we're going so quickly
and we've monetized whatever little amount of value we've been
able to create as a society through model development, that
we haven't really took the opportunity to innovate with the
development pipeline.
Speaker 3: Yeah, and you know, I'm sure this.
Speaker 4: I do think there's probably plenty of people working in
solving this, even you know, the AI wave and the
whole interest in.
Speaker 1: You.
Speaker 2: I mean, it's I think it's I think it's optimistic,
but I also I also think that may be a
little bit realistic because with the deep state model that
had come out, I mean deep seek right, Uh that.
Speaker 1: It I thought we were in a different podcast.
Speaker 2: Yeah, I mean it's a joke that you know, I
frequently go around with some of my colleagues about you
know the impact of these models. But the fact that
you are there's now competition requires doing things more efficiently.
You can't bring a new engineer on board and have
them waste months and months not on on tool set,
like not even on the domain. So that's what drives
innovation there a lot when you're in competition to succeed.
And I think that's going to be a really important
qualifier going forward in the mL space that we continue
to have sufficient competition to drive innovation there.
Speaker 1: So this is an area where I haven't spent any
time or effort, but Daniel, you have, right because you
have been working on some mL work to do QAT
in testing. Right, So what's the what's the barrier to
entry to get into that space?
Speaker 3: I meaning, like, what's the barrier for.
Speaker 4: Engineers to start creating agents themselves and kind of what
knowledge and tools might they need?
Speaker 2: Yeah? For sure?
Speaker 3: Yeah.
Speaker 4: So yeah, I mean, like, like you said, we are
developing an agent that we call test Pilot. It's meant
to be an ai QA engineer take away all that boring,
repetitive manual clicking on UIs that people often do.
Speaker 3: Yeah, and we've been learning a lot.
Speaker 4: I mean, I think I think agents are relatively like
big investments in agents are relatively new, and so I
think the barrier of entry is something you can catch
up on, Like, like I would actually encourage if you're
interested in creating agents. I think it's a good time.
I think you can learn. I think you can do it,
I would say, I mean to me, one of the
most important things you need to do with developing AI
applications is set up a.
Speaker 3: Evaluation framework for the work that you're doing.
Speaker 4: So in non AI non you know, kind of traditional
software development lets, you say, you can often write a
function that you can reason about fairly well, and then
you can write tests that are fairly deterministic to know
whether your the behavior is what you expect right as
soon as you involve the LLLM. Well, now you have
non probabilistic like not sorry, non deterministic behavior, right, And
so you might call it once and it might tell
you something. You might call it again in a second,
and i I'll tell you a completely different thing. And
often because you're dealing with very general cases, you might
be feeding in lots of different examples and you don't
fully grasp how it's behaving.
Speaker 3: Across all those different examples.
Speaker 4: So to me, kind of like one of the first
things you need to do is set up a bunch
of examples with kind of like the expected behavior and
then be able to measure kind of the changes that
you're doing and whether they're improving that evaluation.
Speaker 3: That you have in mind.
Speaker 4: All right, and in this case, it's going to be
not purely past fail. Is going to be like, Okay,
today you're at fifty percent of being able to handle
the cases that you want, and then you do a
few changes and okay, now you're at fifty five percent,
and then you're at sixty percent and so on.
Speaker 3: So yeah, anyways, I don't know, I can probably think
of more.
Speaker 4: I mean best practices, but that's the first one that
comes to mind.
Speaker 2: I would even maybe take a step back before that.
I mean, that's really great insight, like someone who hasn't
spent a lot of time in it. Is there a
particular site or foundational model that is where you would
suggest people start going for first? I mean probably, I
think from my own experience, I know that building models
is super expensive, so they're probably not going to do that.
But maybe there's a SCE out there that you say,
you know, go pull this model and try to fine
tune it or use it out of the box. Like
any recommendations on that.
Speaker 4: Yeah, I mean I do think you need to start
with the with the pre existing in general model, I
recommend using probably a kind of multimodel provider, something like
open router that allows you to easily swap models, because
what's happening is I mean you guys see on the
on the tech news all the time, right, every other month,
each AI vendor like outdoes each other, and it's like, oh,
we have this new new model and it performs better
on this benchmark. And so you don't want to tie
yourself so strongly to one model that when those innovations
are happening, you can't test the other one, right, So
I would pick some provider like, like I said, open
Router is one of them. I think Rock. I think
cloud Flaer has something these days too, But anyways, a
provider that lets you spat which model you want to
use and then kind of build the agent on top
of that. I think the other part of building an
agent is not just the underlying model that helps you reason,
but there's kind of how you prompt it, right, and
kind of how you tell it to do what you
want it to do.
Speaker 3: And then the last.
Speaker 4: Piece is kind of like the tools that you give
it so that it can take action.
Speaker 3: Right.
Speaker 4: So a high level in my mind, like an agent
is you're grabbing the LLLM for the reasoning. But you
need to kind of give it hands and eyes, if
you want to think about it that way, so that
it can manipulate an environment that you're giving it control of.
Speaker 3: Right.
Speaker 1: Is that something that you've got a significant portion of
your team dedicated to or is it something that you
can iterate on quickly with just a few people, like
how big of a how big of a man power?
Speaker 4: Yeah, I mean most of our team is working on
on test pilots.
Speaker 3: I mean.
Speaker 4: Something we've found is so we're a Go shop. Like
we use Go as our primary programming language. We've been
using it for for a while and so we wanted
to continue using it. And what we're seeing is that
a lot of the tools that are readily available in
this space are Python based, and you know, we can
use Python for stuff that's not production, but for our
production stuff we want to continue and go and so
we're having to build some of them ourselves and go.
Speaker 3: We do plan to open source them later on and
kind of.
Speaker 4: Give back to the community and hopefully kind of people
that like Go, you know, let let them have some
readily available tools for building AI and agents. But yeah,
that stuff that's getting built right now, and often we
look for it and it's like, oh, I guess nobody
has you know, done this type of library yet in
goal for AI, So it is early days.
Speaker 2: Yeah, I mean I think that's a really good point actually,
Like it's usually a mistake to rely on Python for
high reliability software development and production, and so if the
tools aren't there to support your production workloads in a
different language, then that really tells you how early it is,
because the people that are utilizing those tools don't care
about like sustainable code or reliable code because it doesn't
offer the core features to guarantee safety or quality of
your value of your service that you're building.
Speaker 4: Yeah, like I I mean there's many different ways of
building a company, but just I've had to deal with
like Groovy production systems, Python production systems, all that stuff,
and I'm kind of like in this case where we're
making different language and kind of moving forward with that.
Speaker 2: Yeah. No, I'm totally with you. I used to be
like early on in my career. I'm just like every
language is the same, right, you know, there's you can
swap between one or another if you've had a sufficient
years of experience. But then over time you start to
realize the fringe benefits of individual languages or negatives that
are associated with them, So things like Go or other
well typed languages like c sharp, the community or the
packages that are available to help drive forward. Having the
right execution of the right stuff in production is like
super important, not which is when you're building something quick
and hacky, but when you actually care about it working continuously,
and like if you're at a company and one of
the core things you're building is reliable builds, I feel
like you know that goes hand in hand or using
other I mean, it's sort of like a different mindset, right,
Like if you're hiring an engineer and they care about
using scrappy tools that ones that potentially break from day
to day, the nightly build version of that tool is
broken by default, they're probably not in the right mindset
to be working on something where you need high reliability
or reproducible builds or you know these other technologies. It's
just a complete other side of the spectrum, right.
Speaker 3: Yeah, I totally agree.
Speaker 4: And yeah, I mean what we've been doing maybe where
we kind of allow Python to come in is you know,
I mentioned we have this kind of evail process. We're
able to try ideas, and so I'm like, okay, if
we're prototyping a direction and there happens to be kind
of like a Python library that does that off the bat,
and we just want to check if that's gonna improve
our performance, you know, the performance of our agent. Like
you prototype something quickly with Python, pull that library in,
run it against our evil framework. Let's see if this
direction has legs. Oh, it does have lengths. It is promising. Okay,
how are we going to build it now for production usage?
And you handle the low we want to handle, and
take care of all the edge cases we want to
take care of and so on, and then we at
that point in time, then we spend the time developing
the the go software and infrastructure.
Speaker 2: Makes a lot of sense.
Speaker 1: So that's one of the things I want to ask you.
You know, you talked about open sourcing some of the
tools that you're making and contributing back to the community.
That's a huge commitment and it's a it's a conscious
decision by companies because a lot of companies are like, yeah,
we invested this, it's a core part of our business.
We're using it for the business. But then you're saying,
I want to I want to give it back to
the community. How do you how do you tie that
back into your business model and and like justify or
account for the time that and the salaries that you're
spending on people working on an open source product.
Speaker 4: Yeah, it's a great question, and you know, I'm sure
everybody will kind of make different judgments there, But for us,
the way we think about it is, Okay, we're developing tests.
Speaker 3: I love this kind of q A A I q
A engineer.
Speaker 4: You can think of the software that makes up test
Pilot as being into different buckets list in my hand. Right,
one is like the truly domain specific logic and kind
of proprietary aspects of it that make it behave like
a QA engineer, right, and that we're not open sourcing.
But then there's the tools that we wish we had
had that that we had to develop for ourselves that
are applicable to developing all sorts of AI agents and
all sorts of kind of.
Speaker 3: AI related products.
Speaker 4: And you know, we're a startup, We are trying to
get more eyeballs. We're trying to kind of become more prominent,
get people to hear about us, and so I actually
think of open sourcing almost as a marketing uh exercise. Uh,
we don't have marketers, we're all you know, we're very
technical teams basically engineers. But sometimes I think you can
look as open sources as a way to kind of
bring in that audience. And because our audience are developers, right,
I think if we're in a different vertical, you're serving
different customers, maybe open sourcing doesn't work that way. But
our audience is developers, and if we can create a
reputation of Wow, those guys that genify really know what
they're doing in terms of AI agents and have you
checked you know, have you seen the tools they've made
and the libraries they've made. I think that all just
kind of comes back to the type of business that
we're building.
Speaker 2: Right, I totally agree. I mean we went down the
same path a while ago with some of the things
that we've opened source. It is like unfortunate to say
it is like a huge marketing story. It's marketing story
for brand awareness. It's a marketing story for hiring engineers,
so they know about your company or what you do
before that you know, they show up the end interview.
They can research it and see that you have something
technical rather than a website with a couple of links
on it and a knowledge base like that's not a lot, right,
you know, this gives them more ability to have insight
in what you're doing. I mean, there is a pessimistic
side of me that says, like, no one cares what
you put on the internet, Like, no one's that special, right,
your thing doesn't matter. I remember like a bunch of
times like Netflix went and open source a bunch of
huge technologies, and I'm just like, I don't care about
those things. But now some people maybe are using that
in production. I guess could be potentially because I think
they wrote them for Kubernetes or something, and you know,
so that's great, you know, you get that experience. I
think the one thing we try to be really careful
about that I think is important here is that the
thing that you're building needs to be really close to
your value, your actual product, because even if it's like
one or two hops away, then it's the people that
are utilizing it. They don't care what that thing is
that you're actually doing, which is super unfortunate. But I
will say that rather than a marketing tool. There's a
second step here, which is if you're going to open
source something, I feel like you feel like it's more
important to do it well. And things that are done
well will live longer and people will see and start
to utilize more. And that give brings us sort of
like a your own usage levels it up because you
actually invested in doing that thing well. And so like,
even if you weren't a deb box, wasn't sort of
out there as a cloud thing that you could utilize
and pay for the fact that you've opened source it
means that that tool is now serving you, hopefully better
for your mL development, for your own agent.
Speaker 4: Right, yeah, absolutely, And I actually we're about like different
benefits that open source my brain. I'll give you a
third one, at least in my view, which is, you know,
when you're a startup, I think one of the most
important things is be talking to users, right, be talking
to users and learning from them. And we've found that
when we open source things, we often get to talk
to more users, but we also take get to talk
to users that normally would not have talked to us.
So you take that box. I don't know if we
can share the specific companies. But there's some big ish
companies using that box that I think if we had
just kind of pure proprietary solution and or this little startup,
they never would have looked at it. I think what
gave them confidence to look at it is like, oh,
there is this open source project kind of you know,
worst case scenario, the startup goes nowhere, we can rely
on that open source project and we can kind of
continue improving it and whatnot. And so that's given us
at least a channel to communicate with those types of
potential customers and users, and it brings in a ton
of learning for us in terms of Okay, what features
are important, what is that class of user or customer
asking us about and how should that I affect our
product roadmap and whatnot.
Speaker 1: Oh that's huge, like in my opinion, like one of
the like the primary objective of every startup is to
figure out the difference between what you built and what
you what your customers thought you were building. And so
open sourcing gives you a way to have that conversation
without trying to force them down through a sales pipeline, right,
exactly right.
Speaker 2: It's scary though, because you know when you put it
out there, it's like, well, people start will start to
see my actual bad code, or it is scary my
commitment messages you know that maybe I should have worded differently.
Speaker 4: And I mean it is tricky too, and like, you know,
I gave you our kind of like ar heuristic for
figuring out what parts of the spilot are open source
and which ones aren't. But sometimes you're in some grade
lines and you're kind of like should I open source
this or not? Like is it Do I get more
benefit from opening it up to the community and getting
the audience and maybe getting some faster adoption, or do
I get more benefit from keeping this kind of like
the secret sounds of the company. And you know, there's
all those stories like Elastic Search and AUS and.
Speaker 3: Everybody gets big and starts changing their licenses.
Speaker 2: So like, son, how I don't know how right I
am here. We've done some initial research on the topic,
and it does actually seem like customers that fall into
a particular bucket never leave that bucket and go to
a different one. Like a free user never converts to
your lowest tier paid plan, and the average business plan
doesn't escalate to enterprise. It just doesn't happen. People self
identify with whatever bucket they're in, and so whether or
not you have an open source solution really only functions
as advertisement. The people who are using it aren't going
to be more likely to get to like start to
pay you. And one of the things we recognize, which
is one of the reasons why our product, authoress is
not open source for ninety percent of it, is because
those free users actually generate ninety nine percent of the
support cases that you need to deal with. So there's
like a huge cost investment that goes into it, and
you don't get any reward coming back from that. I mean,
of course there's some gray areas where there's highly coupled
technology and where it's being utilized effectively, but often the
one the customers that are going to pay you the
most money or be your best customers, like, they don't
particularly care about running it themselves in most cases, unless
they're like a government entity or you know, they have
some regulations they have to deal with.
Speaker 4: Yeah, I mean usually if they really carry some compliance requirement,
but that might just be pointing to like you need
to solve that compliance requirement as well, right.
Speaker 2: Like, oh yeah, I mean, compliance doesn't mean security, so
like that's a completely different story.
Speaker 3: Yeah.
Speaker 1: Absolutely, it's an additional chex right on. That was that
was intense.
Speaker 3: A good way.
Speaker 1: Yeah, yeah, yeah, definitely in a good way. Definitely a
good way.
Speaker 2: I mean, I know we jumped through a lot of
different topics, Daniel. If you feel like there's something, you know,
one last thing you want to share regarding either dev
box or the QA copilot test pilot, you know.
Speaker 1: Have about it, let me think famous last words, go.
Speaker 3: Oh man, a lot of pressure. I mean, I don't know.
I guess I'll just said maybe this.
Speaker 4: Is less about the specific products and just kind of
things I'm excited about. But I am bullish on on
AI agents. I mean, obviously I'm building one, but I
just think there's more and more parts of software development
that could be taking care of through agents, and I
think it can be done in a way where we
take away that kind of repetitive work and we still
leave a lot of the fun and creativity to the humans.
I know there's more doomsday scenarios that people talk about
right where it's like, oh, we're all out of jobs
and whatnot, But I'm actually hoping it's more like, now
we can all achieve more because those less interesting parts
of the cycle are taking care of automatically, and now
you can focus more on system level thinking, on product
level thinking and so on. So yeah, maybe I am
an optimist like you guys were saying before.
Speaker 1: I'm just gonna say to that, no comment, I'm gonna
agree with you, Daniel, like, regardless regardless of what technology does.
I know, there's always going to be problems, and I've
I've made a career out of solving problems, so I'm
not too worried about it.
Speaker 2: Okay, I'm super worried about it. Should I share?
Speaker 1: Yeah?
Speaker 2: Absolutely, So you.
Speaker 4: Can worried about it, but like, let's make the world
we want.
Speaker 3: Well.
Speaker 2: I feel like it's not a technology that's right for
being democratized, for access for everyone to utilize equivalently. The
people with the most money and the most power will
have the most powerful agents at their disposal, and that
just entrenches their own power and prevents them from being ousted,
and it doesn't leave a lot of opportunity for those
that are already being disadvantaged to rise up and fight
against that oppression.
Speaker 1: There's a documentary on this called Terminator check it out.
I think I've heard it.
Speaker 2: I think it has a well known narrator, right, yeah, yeah, No,
I mean that's that for me, is that's optimism, that
documentary scenario.
Speaker 1: All right, Now that we've all made our way onto
an FBI watch list, let's move on to picks.
Speaker 2: Yeah, sure, so I'll go first today. My pick is
going to be The Martian, the book by Andy Weir.
He has a whole bunch of books. I don't know
if I'll say this one is the best, but it's
definitely really good. They're all science fiction adjacent. I mean,
they're not the reality played out in a hypothetical future
where well it's new scenario, and they're great. I think
it's really well written. One of the first things I
ever read from Andy where was actually something called The Egg,
which is a very short story which I find also
really good. And I and many many years later, I
read The Martian and I didn't realize that it was
the same author. And it's quite interesting to have read
two pieces by the same author and be like, wait,
it was this other really good thing that was written
in a similar way, and like going back and actually
seeing it's like I actually read that thing. So if
you haven't read it, I haven't seen the movie yet actually,
but if you haven't read the book, I highly recommend it.
Speaker 1: Right, and I have done both. I've read the book
and seen the movie. But I'm happy to hear you
say that that his other books are better, because that's yeah, yeah.
Speaker 2: So I really liked the short story. He has a
drawn comic with like an overlap between Alice in Wonderland
and Dorothy from Wizard of Oz, and I forget who
the third character is. That's okay. The Martians great. There's
two other books. I think there's a fourth one coming out.
They're not related at all. The one Luna I think
for the Moon. It's not as good but still interesting.
And then there is one with a Deep Space, which
is also really fantastic.
Speaker 1: Right on? Cool, Daniel, what'd you bring for pick?
Speaker 3: Awesome?
Speaker 4: So I have a book as well. It's Wild Robot.
So I think I mentioned earlier. I have two young kids,
an eight year old and a five year old, and
so I do a lot of bedtime routine and reading
with them, and so while Robot is one of the
books that I'm reading with one of my kids by
Peter Brown, there's a movie as well, kind of similar
to the example you gave, But yeah, I just find
it really interesting to read with my kids, your high level.
It's about a robot that founds itself in an island,
doesn't know where it came from, and eventually needs to
take care of an animal, and it kind of makes
you reflect on all these questions, like, you know, where
his parenthood is this robot taking care of this animal?
You know, is that a parent? What's the relationship between
that animal and the robot?
Speaker 3: And I don't know. I guess I get a little
sentimental because I'm reading that.
Speaker 4: With my kids and I have just I've just found
it like really interesting for both of us to go
through the story.
Speaker 1: That's cool, that's super cool. All right, Well, we're gonna
make it a three peat. I'm changing my pick and
I'm going with a book as well, because Warren you
said something about, you know, recognizing the writing style. A
while back, I read a book and really enjoyed it,
and I was like, Wow, this guy writes so much
like Stephen King. It's just insane. And the author's name
is Joe Hill. Turns out it's Stephen King's son, so oh, wow.
Speaker 3: Yeah.
Speaker 1: Yeah. The book the book I read was Heart Shaped Box.
And if you like Stephen King's writing style, Joe Hill
just seems to have picked up on that style and
continued running along with it.
Speaker 2: Is it also like thriller suspense stuff or is it
a different.
Speaker 1: Genre, same same stuff, same like sick twisted thing where
you're like, oh wow, I did not see that one coming. Yeah,
good stuff.
Speaker 2: Do you ever find it a challenge to like pick
up a book that you know is going to be
like that and go through it and be like, I'm
I don't know if I can keep going, like I'm
really afraid of what I'm going to read next.
Speaker 1: No, I don't. I don't think so.
Speaker 2: Yeah, He's like, I love it. That's what I look
forward to.
Speaker 1: Right, Yeah, I mean that. I guess. I guess the
rest of that conversation is for me and my therapist.
Speaker 2: But well, don't worry, we'll shut the camera off here
in a moment and conversation.
Speaker 1: Or it'll be like no, no, I promise I stopped recording.
Tell us all about it.
Speaker 2: Well, you see this little red light on the top
left hand corner.
Speaker 1: It's just a U I glitch. Don't worry about it,
and thank you so much for joining us. This has
been a blast, that's fantastic.
Speaker 3: I've enjoyed as well. Thank you guys for having me.
Speaker 1: All right, and to all of our listeners, hopefully you
enjoyed it as well. Thank you for listening, and we'll
see everyone next week.