How'd you like to listen to dot NetRocks with no ads? Easy? Become
a patron For just five dollars a month, you get access to a private
RSS feed where all the shows have no ads. Twenty dollars a month will
get you that and a special dot NetRocks patron mug. Sign up now at
Patreon dot dot NetRocks dot com. Hey, Carl and Richard here with your
twenty twenty four NDC schedule. Will be at as many NDC conferences as possible
this year, and you should consider it tending no matter what. Ndcoslow is
happening June tenth through the fourteenth. Get your tickets at ndcoslow dot com.
The Copenhagen Developers Festival happens August twenty sixth through the thirtieth. Early bird discount
ends April twenty sixth. Tickets at Cphdevfest dot com. Ndcporto is happening October
fourteenth through the eighteenth. The early bird discount ends June fourteenth. Tickets at
Ndcporto dot com. And we'll see you there, we hope. Hey,
guess what it's not need rocks. I'm Carl Franklin and I'm Richard Campbell and
Man. Next week we're going to be a build. Yes we will.
Yeah, Actually I think this show comes out while we're at build. Okay,
so we're a build now I'm shifting is hard? Yeah, how's your
build going? It's great? Gosh, I loved I had real problems with
travel. Yeah, recording amazing web shows. Yeah, everything's fine. And
while we're here at build. Next week, we're going to be interviewing Scott
Hanseman for our nineteen hundredth episode. Right, can you believe it? What
could go wrong? It's gonna be awesome. It's going to be like the
last one was Sean Wildermere. Three old white guys tired to talk about code.
Who knows Well, he's a VP now, so the conversation will be
a little different. Well, he's got, you know, get off my
lawn stories. I'm sure, so we're probably gonna hear it. Oh oh
yeah, maybe a few. All right, Well, anyway, let's get
right into a better no framework. What do you got? Well, if
you remember the comment you read last week, the person commenting used the word
Gubbins. Gubbins. First time I heard Gubbins was interviewing John Skeet for DNR
TV, and it turns out a tweet by Skeet is my better no Framework
today awesome, So he says, I've been bitten by the Raspberry Pie bug.
Now. The photo below is an x Touch Mini, which is a
Barringer mixer USB mixer. It's about one hundred bucks with a Raspberry Pie zero
two w hacked into the case. Oh man, he opened the case and
he put this little thing inside just supply power and it controls whatever digital mixer
is configured and over Wi Fi, no separate device required. So he has
basically modified this thing and giving it some smarts, and he put in to
it an app that he wrote called digit Mixer onto the Raspberry Pie. So
if you look in the links, so we have a link to his post
that's the better no Framework eighteen ninety nine dot pop dot E. And then
we also have a link to the Barringer z Touch right ninety nine bucks,
and that has a USB interface that he's now just looping into the Raspberry Pie
that's in the case. He's powering the Raspberry Pie with that and the Raspberry
Pie. If you look at the link on Amazon, it's twenty bucks.
Yeah, and it's a little bored. Those little pie zeros. Yeah,
yeah, they're impressive. I haven't done anything with the Raspberry pie in a
long time. And wow, they're getting powerful and small. They got a
lot of horsepower. Yeah. I run a pair of them as my pie
is my DNS sync for all things AD just sounds wrong, but let me
tell you, Like, it's one thing to run an AD blocker on your
browser, Yeah, it's another thing to run a pie hole in your house
because that means that the stuff that my television gets gets blocked. All that
stuff gets blocked automatically. Like you have so much more control over what data
is going out from your home. This is really circling back to the very
first interview I dated with you, Richard Campbell on dot Ahrock's show sixty nine
before you were co host and you were talking. We were talking about the
TVO and you say, yeah, TVA doesn't work and replay TV Canada because
we don't get metadata up here. Yeah, and apparently that's still a problem.
Well no, I mean, so what did I do? I ran
an instance of Linux as a DNS server to lie to the replay TV to
run my loader of metadata. So that I had replay D be populated with
the right stuff because it's just some digital recorder. But you know, this
is the joys of being a programmer for yourself. Yep, totally. Yeah.
Well ah that doesn't work. Let me write in a little after that.
I'll just make a thing with the stuff, and then we're done.
Make a thing with this stuff and you're done. I you know, I'll
tell you another story just as an aside before we get into this, because
of course, now that we've moved up to the coast, I'm moving like
all of my services. So I met my new optometrists this week, okay,
and I'm chatting with the front desk lady that the the lady actually runs
the show, and we end up in a conversation about her husband's hearing problems
and how he's on the list for a calcular implant, which you know takes
a little time to get a certain don stort. It's like, that's this
is not hearing aid bad, this is you cannot hear. You can't hear.
And I showed her live transcribe on Android because she just didn't know.
Wow, right, it's like right now she's like, I'm frustrated trying to
talk to him because I have to shout at him, and it says,
well, that automatically changes your mood, and you know you're frustrated. And
I literally just fired it up on the phone and put it in front of
her and we start kept talking, like look down, everything we're saying is
there, Like, let me put us on your phone when you go home
tonight. Just put it in front of him and start talking and see if
you don't figure it out right away. That's so cool. So I got
a call from her yesterday. It's like you've changed everything. That is awesome.
I'm also really aware like cochlear implants are only so good, Like they're
expecting hations for what an implant's going to be. Are you know me?
You see a path ahead of someone where it's going to be months of sadness.
It's like, look, there are other ways to address this. Yeah,
Like if there's anything we've got going for us living in this technology world,
it's take that knowledge to folks who don't have it. You're so right,
she owned the phone, she didn't know the product existed, and it's
free. Yeah, amazing, that's a great story, Richard. Yeah,
well, anyway, that's our better note framework who's talking to us today.
Mister Campbell grabbed the comment of the show eighteen ninety two. The one we
did with Michelle Duke once was Michelle Mannering but then you know, got married,
changed your name, all that good stuff. Who works for Gethub and
we talked about gethub copilot, which we'd done before with her, so it
was fun to get an update on that. And Joshua Hillarup, who's been
on the show as well, you know guests and regular commenter, said,
I'm with Richard on the future of AI. I can't say what will happen
hundreds of years from now in this space, but I think we before we
would get a business person giving all the requirements to an ass generative program,
we would get AI largely replacing that business person is dead good line. I
mean, let's be clear, how much time do you spend getting business requirements
from a domain expert like It takes a while, so that's not a trivial
thing. But yeah, there's many changes coming in space. And I know
we're going to talk to Aaron today and he's certainly working with companies that have
a role to play in generative AI, so that I thought I'd grab that
little story and think, you know, you know, don't don't take the
obvious answers here. Every time we have a new automated automation tool in our
industry or in any industry, that industry evolves and generally results in more work,
not less. One, Joshua, I know you already have a copy
of music code By, but I wrote the comment anyway because I really appreciate
your insights and you can reach out to me anytime if you like it.
If you'd like a copy of music code By, I write a colm on
the website at don at Rocks dot com or on the facebooks. We publish
every show there, and if you comment there and ever reading the show,
I'll send you a copy of music Coba and you can follow us on Twitter
or x or y or z or whatever the hell it is. Today.
But the cool kids are hanging out at mastadon. I'm Carl Franklin at techhub
dot social, and I'm Rich Campbell at Macedon dot social. Aaron Ericson the
last time he was here was two thousand and nine. He leads a team
building autonomous agents at Nvidio maybe a little company might have heard of. As
I said, his first book, Nomadic Developer, dropped in two thousand and
nine, and that's the last time we talked to him. And since then,
he spent nine years at thought Works, moved to San Francisco, built
internal developer platforms at Salesforce, became a VPE at New Relic, then lost
his mind and decided to become a startup founder, where he attempt to San
Francisco, where he attempted to I just want to know where mushrooms involved.
Uh. Yeah, So he attempted to build a company there at the startup
that used AI to automate your company reorg. Wow. After that startup didn't
work, he joined Nvidia, where he now builds AI agents that optimized the
usage of GPU resources across the GPU fleet for Nvidia. So basically he's a
slacker. Yeah, just later on, Just dude, no wonder we haven't
talked to him in fifteen years. Didn't have anything, he wasn't doing anything.
Oh my god, what kind of bio is that? How are you
doing? Aaron? And welcome back? I feel fantastic. I mean,
you know, what kind of life charmed life? Do you live? Where
you fail at a startup, and they say, hey, why don't you
just do even crazier stuff with AI than automating reorcs, which already sounds,
I don't know, a little bit audacious. You could say, yeah,
that's a nice word. Yeah, I don't know, dude. That seems
like we already have a problem with too much remoteness as it is, right,
Like nobody wants to be laid off by text message. It's just the
phrase that gives me shells. I'm sure we could humanize it it certainly.
The biggest problem I have with reorcs is often people don't go through all of
the steps to take care of folks properly. And so if I had a
digital checklist that's making sure I'm doing the right things, maybe I can actually
do it better. But that's a tough elevator pitch, it is. Yeah,
the funny was so funny enough. You know where we landed with this.
So we're working on a problem which was I think a little bit more
virtuous when we started, which was how do we help engineering leaders reorganize our
teams in a better way, in a way that kind of has multiple displaynering
teams. I think it started out that way, and then, as what
happened to a lot of startups in twenty twenty three or late twenty one two,
chech gt comes out and everybody's like, well, how the hell do
I raise another round? And how do I like it if I don't have
this? Yeah, how do I plant dot ai at the end of whatever
I'm doing and make it moderately relevant? And so our dumb idea was,
well, we already have this engine that allows you to kind of do your
change your organization in a draft mode and then have your friends approve it or
in your org and then go and do it. Why don't we just put
a chech tee wrapper around it and say we're going to write that reorger email
that everybody hates writing It already sounds robotic. I mean, who has ever
gotten an email from HR that didn't look like it came from chet gipt in
the first place. It certainly infused with HR legal ees, yes and double
speak. I mean, it's to me, it's the classical case. I
mean, there's a lot of things you do not want Jenner Todai to do,
you know. I look at the music that you know, Carl that
you're into, and I look at a lot of really creative pursuits, and
I think creativity is going to be you know, despite all the noise,
right, it's going to be one of the last bestiges where jenner Ai is
really useful robotic things from HR. I think it is extraordinarily useful in that
because at least we can get rid of the pretense that anybody really cares about
that communication, right, because mostly they care about it for risk management.
They don't like, how do we make it sound as anodyne as possible?
That that's the whole point. So why not have can we soften the blow?
Ye? Yeah, well and not expose ourselves to legal issues, right,
you know, I mean that therein lies the bigger thing. Like I
know, I know folks in HR who are remarkably compassionate people. They're just
not allowed to be most of the time because you slam into labor laws and
and anything you say can be used. You know, you're almost automatic living
in Miranda Rights Land, just by talking about HR related issues. Yeah,
I mean I used to do this thing at conferences where I'd say I'd write
the memo, but then you could put at the end and write it in
the form of a Homrian epic or sometimes a limerick. Yes, and you
have been let go. But you know what, it's kind of dumb.
But it was like I would really appreciate that if I was being let go
to actually read it, you know, in Iambic pentaminter for example, here
you go rhyming couples. You know, at least it would make me laugh.
Man. I mean, it's kind of crazy that it's been fifteen years
since Nomadic Developer. I mean, spending nine years at Thought Works, is
that really nomadding You kind of put down roots, I mean, except for
the fact that you're going from company to company to company and with a banner
over your head. Yeah. Yeah, and my personal stuff, I mean
I was I went from engineered a bunch of other roles, you know.
I think it was like, hey, be a product manager, be you
this, be you that, And you know, at the end of the
day, it's like, decide what you want to do for a living.
Maybe you are you an engineer? Are you not? I'm still thinking about
it. I don't know. Yeah, yeah, I don't know what.
I don't know what I want to be when I grow up. That same
thing, but again, it's still is the nomadding mindset, the idea of
what's always the next thing. And you know what most people just say was
a managed, well managed career. You kind of put the nomad's been on
at least that part of it. I think a lot of it for me,
A lot of it, despite the randomness of it, was intentional.
And it was intentional because I wanted to, you know, if you want
to be, you know, somebody leading something. I think it's good to
know a little bit of sales. It's good to know a little bit of
product. It makes you a better engineer. If you know a little bit
of product. You know a little bit about why, why are we building
this? Who does it matter to have you done customer research? You know,
you become a better engineer if you ride along totally right something. We
did it strangely, we started rotating devs into Strangely, we were building network
appliances, and so it turned out that the lynchpin of the whole company were
our installers, our system integrators. Because you're touching, you're getting, you're
putting stuff into people's networks. Everybody's network has the ugliness. This is the
question of where you know where it is right and the Sis and they are
all geniuses, like just brilliant, brilliant people inevitably had to fix something in
a network to have the appliance work, like it's just not there was never
not that. And when we started sticking devs with them, you do a
six week rotation with the SIS as a dev just be on the call see
some of the things. Every single time they came back, they're like,
holy man, those guys' job is hard and our product got better for it
every time because they just saw, how do I provide his ability to the
behavior? How do I show you know this kind of flow? How do
you how do I give them switches to turn features off and on quickly so
they can figure out what's going on with that customer. It's this cross discipline
thing is it's insanely valuable. I think you need a certain amount of depth
to pull it off. But boy, I don't know how you become good
without doing it. Yeah, I mean, I mean there's lots of people
get at their niches. I wouldn't, you know, downplay the person that's
been doing engineering and a very specific thing in a very specific way. There's
very good people doing that. But I would say, if you aspire to
run a company someday, or if you aspire to be a really truly great
engineering leader, having those perspectives will be a lay up on average. As
a way, I would say, yeah, yeah, And every time you
can get a chance to dip into the other spaces, see how decisions are
made, Learn that language, you know how those folks speak and think,
just makes you more capable to helping more people into being better at your own
job. I find it really interesting that you went to Nvidia. I mean,
in Vidia makes the infrastructure that makes a possible. And obviously you can
see the success of Envidia by looking at their stock price over the last few
years, it has gone through the roof the trillion dollar club. Oh my
god. And so what I'm really interested in is you know where Nvidia lies
in the software side of AI, which is what you're doing there. How
does that happen? Well, the real advantage of Nvidia is in fact software,
It's not just hardware, right. Kuda is the moat or one of
the main motes of why in Vidia is successful. It's kind of like how
Windows helped make Microsoft successful because they had backwards compatibility for many years. Well,
it turns out if you wrote your software for Kuda in two thousand and
seven, it still works on every Nvidia device you buy, on every Nvidia
GPU. So explain Kuda for those who don't know. Kuda is a it's
the low level AKA there that allows you to write code and C right or
and there's Python libraries to do it too. But if you need to write
soft it's basically kind of like the assembly layer plus the C layer for every
GPU if you're trying to if you're trying to do multi parallel processing of you
know, primarily graphics. That's why it's called the GPU. But it turned
out, and this is almost kind of they put it out there, and
Jensen Longer CEO, tells the story a lot better than I could possibly tell
it. But you think about how AI researchers found out. It also seems
to be seems to do matrix multiplication very very well well, which is core
to how generative AI works in most AI works in general. So it turns
out that that was in the in the sense of accelerated computing, which is
what we talk about now, which AI is one type of accelerated computing.
We don't have More's law anymore quite the same way we did, right,
you know, I mean it's level by the new More's law. I mean,
it's this kind of massive rise of the ability for multiple parallelism for a
great you know, scale to accelerate computing. So that's kind of the story
of how a thirty year company that kind of you know, most companies don't
become two trillion dollar companies in their thirtieth year. They've become that maybe in
their tenth year, and it wasn't. Yeah. Well, I mean,
and I remember I have a friend who's one of those low level developer types
who's moved from job to job, staying in that And one of the gigs
he did was in the middle Oughts, and he was working for an astronomical
society and he was building software to utilize GPUs to do analytics for identifying asteroids.
So you're literally looking at two pictures of saying what moved kind of thing
rights complicated stuff, and he was hand coding all this stuff because Andrew's that
kind of guy. And Kuda came out and he literally spent a weekend re
implemented with using Kudah. And when I've been wasting my time. This is
just drue and we're talking to v one stuff of kudo. Like, it's
not like they were thinking about Jenner of AI in two thousand and seven.
That's not what they were thinking about, right, It's just astonished. It's
astonishing to think that that library is continued to evolve to become one of the
great underpinnings of this multi parallel processing space we're now running in as hardware people.
Well, I remember talking to Steve Sanderson in at NDC in twenty thirteen.
I think about his project using WebGL to take advantage of GPU processing inside
a browser in twenty thirteen. Yeah, that's the HTML five revolution, right,
the CAMAS compute and all of that sort of thing. I've certainly been
banging on the drum about Moore's law ending for some time now. Was TMSC
talking about the three nanometer process like, folks, you're running out of atoms
just cannot get much smaller than that, right center. Well, and also
that it's not just that we went parallel, it's that we're also doing specialized
compute. So I mean for the long we you know, go back a
few years and we had math coprocessors for our CPUs, and then but when
you move up the stack. Obviously, the GPU was the first real alternative
to the CPU for major workloads. Now they're talking about MPUs, right,
they start a neural processing unit, although I personally don't I mean, I
haven't dug far enough it is to say how different is an MPU from a
GPU. Really, I think if you really kind of break it down,
there's going to be a lot of different kinds of we'll just call it asterix
PU, right, you know. And there's as we've gotten more advanceds of
technology. I mean, I don't know if it's widely known, but we
use a lot of AI to develop chips. Sure it's SOLF, right,
and a lot of that's going to come up with lots of different designs.
Like if you can imagine if you can compile your program down to something that
actually runs on hardware where you know your normal instruction set is xor and yeah,
you know your normal kind of assembly language instructions you might have learned early
in your career. What if it's the software, right, What if you
had a special chip for every kind of little bit of software that you were
going to run be it generated bay, I be it whatever kind of future
architecture around neural processing or other things, because the cost to build specialized chips
is going down as well, right, so so much so, there's like,
I think in New York there's a there's a you know, somebody has
like a food cart, but instead of a food cart, it will make
a die. You can actually literally change the software on the die and make
it build like a string copy function that runs purely in hardware or something.
Right, you know, I like going out for chips, but this is
a different kind of chip. Do not get this one in mayonnaise? That
is wrong. My first album with my brother Franklin Brothers nineteen ninety nine,
we program I mostly me programmed drums, you know, drum samples and stuff
for all the drums, and we gave him album credits. Chip Franklin,
I didn't know you had a third brother. I don't. I don't.
Well, depends on your definition of brother. I guess. You know the
count Ai. You know it's maybe a family member, I guess. But
it's a little wild, certainly. I mean, I think Gethub nailed it
with the term co pilot, just like it's an assistant. Right, it's
an automation tool. You're still the pilot. It's your fault when things go
wrong, And that co pilot has become a buzzword for any kind of a
I've stopped trying to drink every time they say it because I'm out of the
keynote in five minutes, right, like you've done. I have a provocative
question, what if you become the co pilot? Yeah? I don't think
it's all that prodocative because I haven't haven't seen a piece of software that can
that has direction yet. Yeah, but you know, so like a lot
of the things that we're doing now these days are so I have a project.
So we talked about autonomous observability agents. We want to go there?
Yeah there, All right, what does that do? Because a lot of
people are like that sounds like a cool title. What does that actually mean?
Well, one of the things we discovered and so in my day job,
which is we do GPU allocations. So we got to figure out all
the internal external groups that are going to use GPUs and take the scarce resource
and allocate them in a manner that is not just good for a video but
good for a customer base. All that stuff right, all the good things.
It turns out we have more data sources than we can possibly just kind
of automatically do engineer or like human do data engineering to understand as quickly as
we need to. And so it came to us, what if we built
a system that could do texts where you can say words in human in English
or whatever human language you want, convert that into either you know, druid
squol or elastic search queries or any other thing. Right, So we're not
trying to be dev in the software developer per se. We're not trying to
write and try our programs. It turns out AI is extraordinarily good at writing
short programs that all the well known patterns. And there's a lot of training
data on how to write SQ well in the corpus of models today, and
you can do a little bit of tuning to make it right. You know,
code that runs against rest endpoints, code that runs against well known things
like elastic search. You know we have in fact, you know, Jensen
talked about it in his latest keynote, doing text to a back and SAP.
So if you want to just ask questions of your database, ask questions
of your large ERP system right. Being able to do that. What if
I can ask questions about my GPU telemetry? What if I can get answers
back about my GPU telemetry? Wouldn't it be great? And I had this?
The story I can tell is, you know, one of my frustrations
when I was, you know, VP of engineering at New Relic was I'd
be asked to do things like, hey, what happened with that incident last
night? A lot of times it wouldn't be as accurate as I would like
because I had to go at eight am, I had to call up a
bunch of directors and ask, hey, what have last night? And they're
tired, right, they're they're they're exhausted. They they spent the night fighting
the fire, right, And so you know, you think about AI's a
hallucinating Well, nobody hallucinates like somebody that's not gotten very much sleep, right,
So so you've hit you have this dynamic. And so I walk into
these meetings half the time right, half the time completely off base. What
I wanted was a system where I can ask questions, what the F happened
last night? Right, and add out the F if you want, no,
No, I don't think and then I get an answer back, and
I can then say, oh great, that took three seconds or even took
thirty seconds because it thought a lot about the question. We allowed it to
ruminate a little bit and confer with other experts in the system about the answer,
and then I don't give me a considered answer and then be able to
help me ask the next question. You see this in perplexity today. By
the way, you see in perplexity you have this capability where you ask a
question and gives you the next five good questions as a consequence of that,
I want that against observability back. Then if I could go back in time,
right, so I could spend that two hours interrogating the system rather than
waiting for everybody to call with bad information. Right, I mean, I
mean part of me wonders, you know, is in my my time in
that role is like you had to root cause analysis problem. These folks were
fighting the fire and they weren't getting to the summary phase before you needed to
talk to leadership about it, like you need you needed to finish root cause
analysis and you didn't have time or or you know, there needed to be
more automation around it. The number times that the IRC log for strange loops
save my bacon. You know that the you know, the firefight actually happened
over I R C because we were all remote, and that you could you
want to know what happened? Read it. Read how we spent four hours
getting to knowing what went wrong and fifteen minutes fixing it in back the day
that I would the first thing I would do when I wake up every morning
was look at the backscroll in slack one of any incident mansion rooms, and
you would get some you'd get some ideas about it, but it still wasn't
like great. Sometimes you didn't resolve to root cause you just resolved to how
do we fix a thing? You've got to be up again? And then
you went to bed. Yeah, he didn't actually get your root cause.
But and I think these gen AI tools are really good at summarizing those long
logs like that seems to be one of the strengths are at although it would
also argue it's very hard to test if they're bad at it. Yeah,
Like you ask it to summarize forty pages of notes and it gives you a
paragraph back. How do you know if it's a good paragraph or a bad
paragraph. Hm, good point, right, because you didn't read the forty
pages that was the point. So how would you know if it was bad?
And to me, the biggest battle I have with all of the neuralnet
stuff right now is testability. How do you know it's correct or what to
what degree it is correct? Well, this is called you build a robust
evil set in any of these models that you're doing. Be It and all
the big generative AI companies. I mean there's leader boards like how much they
test work in the email. This gets really tricky and this is a very
new area, right because I think there's maybe a small number of people in
the world right now that are even attempting to do text to SQL or text
to other computer language where you're having to do the same thing summarization, but
instead of doing kind of traditional RAG or having it in the model, you're
pulling new data in every time, right, You're asking what happened last night
from systems of record that would be able to tell you from the telemetry what's
actually happening. And so it's it's a new area. We're we're developing it
as we go, but a lot of what we do, a lot of
our work, and our team is less programming more finding a good evail set
and validating, so we have confidence in the systems that are telling us what
we think is happening. And I think that's one of the things that folks
who really struggling with as from the development perspective, is recognizing how data driven
all this actually is that a good evail set is a set of data you
can test effectively against that without Without that, no mon of code is going
to save you. And gentleman, I need to interrupt for one moment for
this very important message and we're back. It's Dot a Rock So I'm Richard
Campbell. That's Carl Franklin. Hey, hey, hey, talking to our
friend Aaron Erickson, who has been no madding and has found himself landed at
Nvidia, which is very cool, dude, Like, congratulations, what an
amazing company. And it was it was cool before jen a I took off
and you had to run the whole place. But why are you folks building
autonomous agents? Like aren't you happy selling shovels during the gold Rush? Because
because we can no we so we have a we have a need to increase
utilization of our GPU fleet. We have a need to be able to monitor
our own GPU environment. The reason I got so are doing this is because
we had a need in that category and it trued out. You know,
if you work for an AI company and you find AI to be useful in
solving a problem, you should probably do that. And so we are doing
that well. And I mean you had before the break that little lead in.
It's like what if you were what if you were the copilot? Like
to me, the autonomous agent is still the copilot. It's just got a
better instruction set. It's not it's not a gopher, you know. It's
not ask a question, get a response. Ask a question, get a
response. It's more of a here's a problem space that needs to work on,
report back routinely. Like it's almost a if you were equating it to
a junior employee. And I loathe to do that because I do everything I
can to avoid anthro memorphizing software. Now that it's become a real problem,
that's the only way we can think of it, though, Yeah, except
that it makes human wrong right like, it makes us think. It makes
us think that this thing has intent. It gives we as soon as you
give it an agent, then it's then its response has credibility more than it
deserves. Like you. But do you think that us being the copilot means
the AI is driving the train? Well? Is that the implication? That's
the idea? So and you and you assist the AI when it needs help
from somebody when it doesn't have the answer. Is that what you're talking about?
Correct? Correct? So what you do, and I stated this publicly,
you create an org structure of agents. So there is a director agent.
The director agent has a goal. It has a goal of a metric,
almost like giving it an OKR in an organization structure, and its job
is to analyze data. Work with analyst agents that might understand a domain might
be I understand GPS need to run in this temperature range. I understand financial
transactions need to operate in a certain manner. You know, you can apply
to a lot of different domains. And this is a pattern. I'm not
really talking about anything secret here. And then you have worker agents. Their
job is to take human language and convert it into database queries or rest calls
or whatever else, so that confer with themselves. Right, maybe human assistant
it might call you back to say can you clarify this yes or no?
Is this true? Right? It might actually ask you questions and then figure
out an action plan as a result of the analysis. And so the action
plan might be realoicate these GPUs from here to here. It might be hey,
human go take a look at this data. Tell me what you think
of it. Right, there's all sorts of ways you can think about this,
But what it starts to do is once you reach a certain threshold of
capability, it becomes more of its driving things. It's driving the details and
you're kind of monitoring it right with the real data about how is this actually
improving the system? Now you've said like you've given an emission and it's it's
essentially going down the checklist of the right way to execute on that mission and
responding to the impadences that he gets back to it from it. And I
would hope you said monitoring is the key, right. I mean, there's
got to be a CEO that's watching it and not allowing it to make decisions
until they're understood and passed off by human io. So have you been to
San Francisco lately? No, but I suspect you have. So I have
an app on my phone, I can call a way Mo self driving car
with no driver in it. Yeah. Yeah, And occasionally, if it
gets into a jam, it will notify somebody at a real, real human
Hey, I'm in a traffic jam or I'm stuck in some unusual condition and
they take over. Yes, somebody put a traffic cone on my hood.
Yes. Yes. And so if you've seen these videos, I'm sure you
probably have. Not everybody has. I mean, there's people that have never
seen this and haven't heard this is actually happening. But it's this kind of
scaling of self driving cars. Like we started out with cruise control, we
moved to smarter cruise control, we move to kind of full self driving.
Is like the biggest manifestation of this, But there's this kind of face shift
that happens between hey, it helps me till now we're in San Francisco,
it's I help it when it calls me. And it's just kind of an
evolution of how these things happen. And so it's a lot of investment,
a lot of evals, a lot of monitoring on the way there. But
once you get to that threshold where you're monitoring the thing and it's right ninety
nine percent of the time, especially in domains where it being wrong isn't catastrophic.
Like you may not do this with financial transactions tomorrow, but you might
do this with certain other maybe kind of lower risk things, right, And
this is where it starts, and then you start to get more and more
of this stuff. Yeah, I don't know if we're building on a house
of cards here, but I certainly think that there are routine tasks in throughout
business that can be off greated from a set of rules. You know,
we've made fact box long ago, uh Tier one tech support. More and
more automation happening on that because there is this set of did you turn it
off and on again? Did you check this? That? You know,
like it's a checklist of things as soon as you've got a person flipping through
a book, you know, going back and forth on the standard set of
questions like that could be software. It just seems like layers of automation.
I've just always concerns me when when we get a sense of consdering more intent
than that that it is still a set of instructions, although often those sets
of instructions are based on a parametric response to data coming in. You know,
there's a reason why we turn the we turn the lights red when it's
past a certain threshold, right, right, But there's so there's a good
job or a book called forgive Me. I'm going to quote a book name.
It's not a curse word, but it's called bulkt jobs of the book.
And it turns out there's a lot of roles that I think maybe aren't
that intellectually challenging, but for whatever reason, we still keep around where you
know, I call these kind of like low level intellectual robots. And if
you can kind of automate a lot of the low level intellectual roboticness out of
jobs, and you think of like what happens at the DMV, I don't
know that there's a lot of creativity going on at the DMV. And I
don't know that you want a lot of creativity going out on the DMV.
I don't know that you want a lot of creativity in your finance department.
In some domains, there's some kinds of software development that aren't terribly creative.
I don't ever want to write another get rest point, get you know,
rest endpoint again that gets data from a simple thing. I know that can
be automated for the most part, So we use copilot for that today.
Right, We're going to find more and more things that we just use copilot
for a lot. That's that's low level of intellectual labor, right, and
we will start t all to make that more well and and well. Every
time we've ever done this, it frees up resources for doing more creative work.
Right. The reality, a lot of travel agents lost their jobs as
the airline industry and the hospitality industry moved online, but travel exploded far more
people traveled, all those industries got bigger, and a net more jobs were
made than were lost from travel agents. You know that that pattern consists.
It happens over and over again. So the theory being these you know,
menial jobs, these menial tasks that can be automated, should be automated because
that person is capable of more. Yeah, it's the only way you can
look at it, because you can be mad that this is happening, and
lots of people are and I have a lot of you know, but reality,
it's just how we manage disruption is the important of this, right,
Like, yeah, we should be able to manage this disruption compassionately. But
it doesn't mean, you know, the alter of his not having disruption,
isn't there. I referenced the the Luddite movement. Yeah, you know,
and for those who don't mean we've heard the term. But originally, when
the first automated the first steam powered looms were deployed in Scotland, the folks
that were used to weaving by hand freaked out and smashed them. And here's
something that people don't know about that movement. But the Lunites weren't anti technology.
In fact, they were the technologists of the day, right. They
were the high tech. They were using high tech machinery. They understood it,
but it was it was really about you know, jobs, right,
yeah, And the reality, of course was that was a communication between the
employer and employees. The result was that ultimately rebuilt the machines and had all
those people running them. Right. The bigger thing that happened as they automated
the weaving of cloth is that people bought more clothes because the price of them
fell dramatically. And not only that all of those people keep their jobs,
but a whole lot of more people got imployed because they made a lot more
cloth. And you know, the industry exploded. The idea of owning different
garments for every day, not just a good set of clothes and a working
set of clothes, came out of being able to automate cloth weaving. So
you want to know one of the most ironic things about what's happening right now,
I'm sure you do, and that is so now engineers are automating engineering.
And you see this with some of the responses that Devin the software developer.
I found that fascinating because who here in software development hasn't automated the living
crap out of somebody else's jobs, a lot of other people's jobs. We've
been doing that for the entire what do you think CICD pipelines are, right,
We've been doing this for the entire history of computing. Okay, normally
there is nobody. I'm sorry this might make some people mad. I'm getting
older, so I kind of don't care, as we all are, right,
we all age one. But the reality is if you're mad that,
like programmers are automating programmers jobs, well, sorry, I guess that's just
kind of like what we've been doing, and I guess it's coming back and
it's kind of funny. But like when people get mad at this, like
they're like, oh my god, Devin, it's like, yeah, sorry,
we did it to ourselves. By the way. By the way,
you have so much opportunity now if you can learn how to construct these systems
together, use this. So you know, we built this thing that in
video inference microservice just a pattern just being video, it could be an inference
microservice. So now instead of building a micro service that knows one domain well
and you build it in code, you build ALM that knows one domain well,
that takes in requests in human and generates data for you in whatever form
that you need. But it's the same idea, right, it's taking some
of these same archeittral patterns that we've known as engineers and using it to build
bigger, smarter, more interesting systems that then let us be more creative.
Yeah, why did you think that the methods for using machine learning tools weren't
going to improve? Right? Really, come on, like, that's of
course they are the Especially in a young in technology like this, it evolves
even faster than the more mature technologies, where a lot of the stuff's one
point zero and just becoming to two is a big jump. Maybe three is
kind of where we're talking about these first few versions of how we build this
stuff. Of course is going to be massively transforming. Well, but it's
funny the way it's happened in AI, where it's kind of like it's very
quiet for periods of time, right, you know, you have researchers doing
things. It kind of feels like it's it's quiet, it's quiet, and
then boom, something like tchike happens and then massive, massive change. I
think ken Beck had a quote saying, I don't know how this is going
to change, but I know it's going to change a lot about Yeah,
a lot of things how I write code, and I think the best response
to a lot of this is wonder yeah, you know, and and just
kind of being even as you get into your later years in your career,
you know, I'm kind of rounding third base in my career, I think
at this point, and I still every day I wake up, I'm like,
what can we do with this stuff? How can we build on these
with these tools that we have to do things that are incredible for our customers,
that are incredible for new Like I look at what Nvidiot does. I'm
sorry if I'm if I sound like a fanboy, I am, you know,
guilty. So it's a lot. But things we're doing around protein discovery,
things that our customers are doing around protein discovery, that we're never even
possible pre transformers, pre the transformer model. There's going to be diseases,
care there's going to be technologies built that make humanity better in every way possible.
There's also going to be, let's be real technologies that make things terrible
in a lot of ways. And you know, I don't. The thing
is, you can't stop it though. You know, there's no world where
you can suddenly, oh, GPU's got to throw them away like they're here.
And even even if technology doesn't move at all from what we have now,
so only started to use these tools in various ways, and you open
the door to philosophy. So let's go here. Yeah. One of the
questions that is always on my mind is, Okay, there are a lot
of skilled labors in the workforce who can make the adjustment. There are way
more unskilled laborers that just need a job, and so how do you see
the future playing out for them? I wish I knew the answer that question,
but I don't know the answer that question. If I'm honest, I
can speculate. I can speculate that I think you know. The answer is
sometimes everybody's going to talk about UBI. That sounds kind of terrible. Universal
even if you had a good UBI. Yes, universal basic can come like
Okay, maybe they don't do anything. I think we have to make the
world more prosperous so that we can give people the opportunities to be creative.
Yeah, would be the way. If I could be the dictator of the
world, right, how I would arrange that. Yeah, that's the goal.
Yeah, for Aaron to be the dictator of the world. Sure,
yes, that's exactly what I meant. Richard, Thank you. I have
a better idea. Why don't we just have a super smart AI be the
president. I know it sounds a little ridiculous, but you know, so
are the presidents regardless of you're more ridiculous. Yeah, ridiculous. As a
service, we admittedly are at a dark phase in some of this stuff,
but I'm pretty sure I wish to will want people to be the compassionate ones
that are helping set our values and are setting the rules for the software to
do their things. Yeah, I know I'm such a party pooper. Fine,
I can't have president GPT. I mean, you know, I would
love I would love somebody to do an experiment at least elect g In fact,
that was one of the first things I asked when auto GPT came out,
which is like one of the first agent systems that somebody built. Almost
immediately when CP four was live, They're like, what if we put this
in a loop and what if we So I asked it, how do you
elect yourself mayor of San Francisco? Because you know, it just kind of
sucks here in that manner, and it immediately knew, oh, I'm going
to have to get a straw candidate because AIS can't get elected. But if
I can control a candidate and they just trust me with the decisions, I
don't know, it still sounds ystopian. I mean maybe we should stop watching
and being like, I wonder this would be great products. Yeah, here's
the real issue, Eron, is that we decided to attach the term artificial
intelligence to this technology. Yeah, because that has had seventy years of science
fiction, you know, perverting working against it. You know, the first
time the public hears the phrase artificial intelligence is in two thousand and one a
Space Odyssey and Hal tries to kill everybody, like we have set a path,
and now you want to hang that moniker on this software. Yeah,
like it was. It's a mistake because it's hard enough to introduce new automation
into society, but to introduce new automation to society with decades of telling me
it's going to kill everybody, like, it's kind of a dumb thing to
have done. I'm going to take it up with the branding department of AI
and see if we can come over with a better moniker. Good luck.
Unfortunately, it's probably stuck. I think. I think once it's like out
there, it's kind of just gonna be a thing. But Genie's out of
the bottom. It's it's a little late now, right. And the joke,
of course, is that Minski came up with the term to convince the
US Army to give him money. It has always been a marketing term to
raise money. That's the sort of reality of it. Now, every so
often you build some software and it's happened several times. You can call a
Mai Winters if you want, but the iteration is the same. Science works
on the problem for a while until it gets to a certain level where the
engineers take over and apply it, and once the applications are done, it's
just software, right, right, are they? The joke term I've used
for this has been artificial intelligence is what you call a piece of technology when
it doesn't work, right, Because as soon as it does work, it
gets a new name. It becomes planning and optimization software, it becomes vision
image recognition, it becomes large language model. Right as long as there you
know, I do a lot of investing. If you call it AI technology,
you just told me you have stuff that doesn't work. When you call
it something else, then okay, let's take a look at it. That's
a great point because I think a lot of people think they just got to
slap an AI, you know, peanut butter in my product and it gets
better. Well, I have a failed startup to say that does not,
yeah, work that way, And I think the best products they use AI
don't mention AI at all. They just solve a problem. No, because
I think it's plumbing. I think it's just going to be part of the
software. The ones that do It's questionable whether what we think of as software
developers is AI, where's involved at all? It's just algorithmic programming, right.
I mean there's some they all want to slap the AI monicor on it
when there may may not be any you know, any machine learning, any
that kind of thing. It could be a series of statements. But listen,
without a doubt, there is a software technology using probabilistic language analytics that
is getting effective results. And it's and that's it right Like now, there's
an interesting area to think about about how we revise UX's and how we can
put software in more places like this feeds into the other stories of ubiquitous computing,
and you know the fact that we're making them in a small form factic
we want to put on our shirt and so forth. It speaks to you,
I want an even more personal compute device than the slab of black glass
that Apple gave me. You know, we are ripe for change in this
particular space. But I think part of the problem we're having right now is
this Ultron versus Jarvis kind of science fiction crap that's infected our thinking around an
interesting piece of software that we can do some good with well, and the
thing that cheching be tenailed wasn't the fact that it's AI. Yeah, it's
great. What chech tenailed was what people want is I want to ask questions
to a box and get moderately or ideally perfectly accurate answers to my questions,
regardless of what they are. I debate the irebade the accurate part. They
wanted answers. They wanted responses that we're spoken confidently right, like we're and
apparently many people are happy with the answers even if they are not right.
That was sufficient. Now I when I'm doing these AI talks, I bring
up Alpha fold two because I think it's one of the most important pieces of
machine learning software that almost nobody knows about. Is this the Wolfram thing?
No, this is al fold exactly comes from deep mind. These are the
guys who solved go okay, but then they reapplied the technology to protein folding.
And again it's like, this is pretty esoteric stuff. But there's a
few things said. I find it important and only a it's doing protein.
It's computing protein fold results that are taking years to even validate, Like that's
so we're not even sure if it's all right. But everything that we've been
able to prove so far, it's getting far better results than anything else we've
ever done. But also like the competition, the CASP competition, which is
very much like the ImageNet competition that kicked off vision recognition in the twenty tens,
The CASP competitions were going on for a few years to what four years
ago was the first time alpha fole jumped in on it, and they open
source the whole frickin' thing. The next two year one it was like half
alpha fol, but this year's a CASP event. It's all alpha fol.
Everybody's using it, Like every scientist looked at this and said this is a
better way to work on protein problems, and they've all adopted it. Like
this is where we're seeing automation and new technology at its best. There's nobody
trying to make up billion dollars off of this yet. Right when products start
to appear from it, then things might get ugly. But right now,
in the research phase, everything has been shared that this non deterministic algorithm for
computing behavior at the very low level of biology is open to every scientist who's
interested in this space, and they're all testing each other, they're all competing
with each other to make it better. We went from media remember what you
know it was the XKCD where it's like identify a burden this picture. It's
like, I need a huge research budget too. Now it's a trivial thing
you can do on your phone. We may be up against that for understanding
the fundamentals of biology and complex biological organisms with alpha fold Like, it's so
profound and it's not being I love that it's not being celebrated in some respects
because it's people too busy working. You know, this current detonation around LMS
is about the scientists moving on right. The Hinton's Suskevards of the world are
largely saying, hey, jen Ai has gone as far as it go and
that's like, Okay, we figured out a concrete Now you engineers can play
with it, and we as engineers now get to change the world with it.
Sam Malton in a recent talk I literally just listened to this morning,
but it was a really good talk. I think he was an MIT or
something. He talks about how great we're going to get GP five. But
what we want to be able to distill is the ability to figure out the
reasoning part separate from the architecture that does the kind of like large language more
data. Somehow, somehow we knew we added more data and we got something
that looks like reasoning, and we're not entirely sure it's real reasoning or what
it really is. But if we start to understand the architecture of reasoning in
a much more nuanced way, I think that's where the next big breakthrough is
going to come from. At least that's based on IPI today. I mean,
it can change tomorrow with whatever papers. I mean, but once we
get better kind of synthetic reasoning and we understand that architecture, that's going to
be probably the next I certainly look at the smell perspective and say, are
we still just making it bigger so because we don't actually understand it, or
I'm hoping that jat that the GPT five is much more reasoned, that it's
we're not automatically bigger, that we're more focused in areas that we're starting to
do more discrimination, you know, effectively understanding testing your language in deeper layers
before you present it. I don't know if we're there yet. There is
a set of code smells that you see in every rapidly evolving piece of technology,
and one of them is right sizing, and you are starting to see
movement towards right sizing. But what you're describing then is, rather than wait
for the next research breakthrough from the scientists, that engineers iteratee and find incremental
improvements in iteration that occasionally can lead to another breakthrough. Well, and a
lot of this. You talked about anthromorphizing earlier in the episode, and I
want to call out there's a great researcher Ethan Mallock, I think professor Wertn
or something like that, and he wrote a great book called co Intelligence,
and he talks about actually the virtuousness of using a limited form of anthromopore that
I can't even say the word right, but talking in human and optimizing your
prompts that way. So we saw that even just asking it to think step
by step, if you do a chech fee question and say and think step
by step, people have demonstrated to get better results. What I really look
forward to is the application of industrial organizational psychology to using LMS and seeing a
different techniques such as how people talk to each other, how organizations work,
and do organizations and lms, when you have multiple agents working with each other,
share some of those same properties, which on a conjecture basis, we
haven't proven this, but hypothetically they should because lms are trained on people,
and they're also trained on people's interactions, and that's the important part. It's
not that's speaking to it more like a human makes it more human. It
was trained on human data. Yeah, so obviously it's going to work more
effectively if you work with the data set as it is. And again I'm
always balancing this what came from us, and it is the reason that things
work the way they do, versus imbibing any of that capability on the software
itself. It's software. It's a reflection, which is why you always think.
This is why you always think chet GPT and ask talk to it nicely,
because studies are shown when you ask somebody nicely for something, you get
a better result than being a jerk to it. And there's the most anthropomorphy.
Does this mean my wife should stop telling sch Alexa to shut up instead
of saying stop she does that? And I don't think there's an LM and
Alexa yet, So you don't have nothing to worry about I forgot what Alexa
was. I thought you were actually talking about a child. Okay, now
no Alexa, Yeah, no, no, no, I have I'm starting
a new meme where if you put in front of anything that you don't like,
that's a diminutive. So you know this coffee, Oh my god.
Well, plus insaying the app word because folks are sometimes playing this in open
air shire's their devices, which makes them sad. And that's been a running
joke in dotam Rocks too. So you got a lot of mileoget of that.
Sounds like you're working on some amazing stuff, Aeron. I can't wait
to see some of it in the field. Yeah, like the emergence of
autonomous agents. I hear the noise in a lot of different places. Not
the product yet, but obviously it's a goal. And I would categorize the
show as a geek out. I mean, we really went a lot of
places that wasn't real, you know, practical day to day programming, and
these are great shows to do from time to time. So thanks Erin.
I'm so happy to be here. It's been fifteen years. I feel like
maybe I've got a few more wrinkles. Yeah, maybe, y'all do too,
but I feel like maybe a little bit more Risdom as well, or
maybe not, but you know, I'm excited to see y'all's It's great to
be on the show. And good luck in the future and that means everybody,
and thank you for listening to dot Rocks and we'll see it next time.
But dot net Rocks is brought to you by Franklin's Net and produced by
Pop Studios, a full service audio, video and post production facility located physically
in New London, Connecticut, and of course in the cloud online at pwop
dot com. Visit our website at d O T N E, t R
O c k S dot com for RSS feeds, downloads, mobile apps,
comments, and access to the full archives going back to show number one,
recorded in September two thousand and two. And make sure you check out our
sponsors. They keep us in business. Now go write some code. See
you next time. You got Jvan in summer time that means home. Then
my Texas a lie present Hall