Hey, welcome back to another episode of JavaScript Jabber. This week, on
our panel, we have Dan Shapire, Hey from a nice, nice weather
in television. I'm Charles max Wood from Top End Devs. It actually snowed
here over the weekend, so we finally have a few inches of snow outside
just here. No, we're wearing a T shirt. Yeah, well we
go skiing. But yeah, anyway, we have a special guest this week.
It's He's May. I didn't get your last name on here, but
oh here we go. Oh that's a hard one. You're gonna have a
hard time with. Well, us say, ed, that is perfect.
That is perfect. So he's May. Do you want to introduce yourself real
quick? Let people know who you are and why we're excited to have you
here. Sure? Sure, So, as Charles said, my name is
Ismael. I am the co founder of an open source framework called Superagent.
Superagent is a framework which allows any developer, regardless of their skill set,
to create and integrate AI assistance into whatever type of application, stack or environment
that they are using to build their apps. So, as you guys know,
AI can be pretty complex and there aren't that many machine learning engineers out
there there are actually very few, and Superagent allows any developer to basically become
a machine learning engineer and leverage all of the fantastic technology that's being developed every
day without having to actually know that much. So we abstract the way all
of the tiny machine learning pieces that need to fit together in order to create
a accurate, production ready application, so that the developers can, you know,
focus on their users, focus on creating amazing user experiences, and you
know, tap into the vast possibilities of AI. That's so cool. And
I have to say that a few years ago we started a show on machine
learning called Adventures in Machine Learning, which is much more on the engineering side
of building models and stuff like that, and I thought, oh, this
is stuff that I really want to pick up. But I figured out that
I am much more interested in building things with the engines that other people built.
There. I am in, oh, I've got to wrangle this data
too and somehow make it jive with this other data so that I can get
a model that gives me the answer I want. So anyway, this is
cool stuff, and this is something I wanted to talk about for a while.
Cool I mean, and I see that I came from Originally, I
came from an open source background, so I've been a contributor to many of
you know, the machine learning frameworks out there that people use. And one
of the things that frustrated frustrated me a lot was that, you know,
regular developers were having a really hard time using these frameworks because it required so
much you know, knowledge of you know, deploying this stuff and running it
in production, and how to get it accurate, how to get the responses
you want, how to attach you know, data to your AI model,
how to have that model interact with you know, third party ABIs, everything
like that. It was just a mess. So I just decided one day,
I think it was in May, and my co founder was in Vegas
playing poker and I had a week off, and I just said, fuck
it, man, let's sit down and do this. And so I took
a week and I just coded everything, like the first version in a week.
And when it was ready, I was like, this is this this
this feels good. You know, perhaps other people would want to use this.
So I just decided to slap an MIT license on top of that and
open sources. And you know, the feedback I got was amazing, and
it just blew up and now we are taking it basically to the next level.
So that's how it started. That was I wasn't it wasn't meant to
be, but it was meant to be, so to speak. I love
those weekend projects, so the best things in life, you know. I'm
trying to decide if we should start with the chatbot side or the AI engine
LM or whatever side. I think the best thing to start with actually would
First of all, I think it's worth while mentioning that before, before we
started recording, you had a bit of interesting news to tell us how about
the project, and I think it's worthwhile to highlight that up front. Yeah,
let's do that. So I'm happy to announce, you know that Superagent
will be a part of the Y Combinator Winter twenty four batch, which is
like an amazing thing, one could say a dream come true. I think
it was like twenty four thousand applicants or something, and you know, we
were chosen as one of I don't know how many, but not that many.
So it's an amazing thing. And I'm situated in Sweden. I've lived
here almost all my life and it's a lot of snow here as well as
you might imagine. And I will be relocating to the States and San Francisco
on Thursday. So I'm packing. I'm packing, and I just found out
that my passport is going to expire in two months, so I have to
fix that as well prior to But but but that's that's on my side.
Uh So, really excited about that, and really excited. You know.
One of the great things with y Combinator is that you can focus a lot
on building something that's actually viable and can become something big and viable in the
future. And I'm really excited about that. You know, sitting here all
the way, you know, across the Atlantic basically and only chatting with people
on our discord channel or having like the zoom calls is great. But you
know, I think that being where, you know, in the mosh pit
of AI in person would be even more great. And I could you know,
achieve more and have better you know, talk to people who do this
stuff all of the day, and all of the all of the devilopers and
machine learning engineers and stuff that are working on this and and and basically you
know, building the future. That would be that That's my dream, you
know, to talk to those guys and learn and adapt and try to incorporate
whatever works into our framework. So oh yeah, so obviously congratulations and kind
those Uh And I wanted to add that one of the great things about being
part of something like by Combinator is that it's definitely not as smart, it's
not that money. It's as smart as you can get in that regard.
So it opens a whole lot of doors. And and it's it's not just
getting you know, the funding, it's also being part of the program,
uh, you know, really pushing your your project forward in terms of recognition
and adoption and stuff like that. So I'm really excited for you guys.
And as I said, kudos. And the second and the second thing that
I wanted that I think it's a good starting point is to kind of to
understand what what super agent might be used. Four. So if you can
give a concrete example of one or two or maybe even three things that you
know that super Agent makes easy to do that would otherwise be you know,
much more challenging. Yeah, sure, I'll give you two examples. I'll
give you a personal example and then I'll give you like an enterprise example as
well. So personally, I use Superagent every day. And as you as
any like open source maintainer will tell you, the community is everything. When
you're building an open source project, the community is everything. And that's that's
one of the main reasons why you open. So you want feedback, you
want contributions, you want a bunch of stuff from the target audience that you're
trying to, you know, build a product or a framework or whatever it
is for. And so when you come to when you get to a certain
stage and when you have a critical amount of contributions being made, it's like
having fifty employees. It takes a lot of time to go through contributions,
to talk to users to see what developers want, how we should build whatever
they want, and what ideas they have, and what code they've written,
and so all of that is really time consuming. And one of the most
time consuming things is doing code reviews, so reviewing other people's code that are
not actually a part of your team, they're just open source contributors. And
so if I would estimate that takes me around ten to fifteen hours a week
to only do that to review other people's code. So what I did was
that I set up a assistant, as we call it. An assistant is
basically an agent or a language model that it has a bunch of tools connected
to it and can be trained on different data. So I trained my assistant,
which I call Shuri Can. It's a Japanese name because Superagent has this
Ninja thing and a Japanese aura around it. Everything we do we call it,
you know, Japanese names basically, So Shuri Can. She's an assistant
that I built using Superagent, and her sole purpose is to do code reviews
for contributions that we get from open source you know people, and she does
that so well that even the contributors don't know that it's an assistant. It's
an AI assistant. So it has passed the touring test multiple times, I
would say. And the way we do that is that basically it's super simple.
You take with a couple of clicks in our UI or with code in
our SDK, you can set up an assistant and attach as many files or
data sources that you want to that assistant, and that assistant will learn all
of that data and you can instruct it to do specific tasks. So in
this case, I've taught her everything there is to know about our code base,
and so when a contribution comes in, she knows exactly how our codebase
is done. She knows how we like to write code, she knows all
of our rules, all of our internals set up, and she can give
feedback to contributors on what they could do better or what they should change in
their code review, and then the developers do that and then I can go
in and merge that in. That's just one simple example of what you can
build. So if you would generalize that, that's like building a team member.
So to answer your question down, one of the most exciting use cases
I see is to augment your team and increase productivity of your team. That's
one of the main I think, you know, unique selling points that AI
has, which which I believe will be a huge thing in the future,
even more prominent than it might be today. So that's one yeah, yeah,
So before we move to the other one, a few questions about that.
So first of all, what model do you use for sure can so
in the Shuriken case, we use an open AI model, the GPT three
point five model that is fine tuned on our code base basically, so that's
that's a proprietary model that we use an open Ai proprietory model for that specifically,
So the starting point was open Ai three point five, and then you
further trained it on your own codebase. That's correct, Yes, fine tuned
it on our on our own codebase. And because this is definitely not my
area of expertise, is there, like I would imagine there's some sort of
a lower limit about how you know, how small your code base could be
before you would actually get some value after training on that specific codebase. Now,
so yes and no. Uh, you know, generally I can say
that the more the more data you have doesn't necessarily amount to a better performing
model. Uh, it's the quality of the data that's important. So you
can actually have quite small training data sets, but if the quality is high,
and the quality is of course depending on what it is the assistant or
language model is supposed to solve. But depending on the task and depending on
the data quality, you can actually get away with just a couple of thousand
you know, rows in a spreadsheet to train your model to be very effective
on a specific task. So I would say quality is you know, a
better thing to focus on when it comes to data sets than quantity, and
that has also been improven by open source model developers that have you know,
basically generated synthetic data, small synthetic data sets that have a really high quality
and have been able to train models that are as equally good as GPT three
point five with less data, less compute, less money. So quality is
super important when you do that, and there are a bunch of you know,
bunch of stuff that you need to learn before you can train your model.
Of course, how should the data, what should the data look like,
what type of data for what type of task, et cetera, et
cetera. That's all of the questions that regular developers ask themselves every day,
but they don't have any you know, they don't have any prior experience working
in this field. So that's also one of the reasons why we decided to
build a framework for the mainstream regular developer and abstract a way all of all
of this stuff that we are discussing now, just abstract that away and make
it as easy as uploading a file like Okay, you want you want your
assistant to be trained on ten PDFs, Cool, upload them and we'll do
the training in the background. You won't notice we'll send you an email when
it's done and you can use your assistant. That's it your case. So
in your case, you kind of pointed it at GitHub rep and that's how
you trained it exactly to multiple repositories. We have multiple repositories, so pointing
it to those multiple repositories, and then we have a bunch of technology that
you know, extracts the necessary data chunks, it splits it and fine tunes
a model for that specific use case. So one thing that I'm curious about
with this is I see a lot of people add what you're talking about,
at least to me, sounds a lot like continuous integration or you know,
some of the steps that people put their into their GitHub actions where it basically
says, you know, all the test pass you pass through the linter,
and hey, you know your style matches our style, or hey will you
accept you know the automatic cleanups that come out of the linter and stuff like
that. So I'm a little curious as to what sure it can gives you
that that you don't get from you know, just kind of a standard what
do you call it? A standard CI set up? Yeah, So CI
usually looks at syntax, and you know code conventions like Lindin. You know,
it's it's basically a code convention. You might write sloppy code, but
it will pass the linter. So that's the thing with sury Can. A
linter will catch bugs. It will catch you know, could be type errors
or something like that, you know, but it won't catch sloppy code.
If it works, even if you build it, it won't catch that because
it will build. It won't throw any errors. So what sure Can does
is that she actually looks at the code itself and and you know, gives
feedback on the actual code, like you should not write this function like this,
write it like this, or use this package instead of that package.
You know, that kind of feedback, the ocular feedback that a code reviewer
usually gives his or her employees. That's what sukin can do in our repository.
So I'll actually I actually push on that. Can you give a really
super concrete example of the type of feedback that you might get, like I
can. I can show you if you want to, if that's possible,
well, and you can. And that's nice, But we are primarily audio.
Most of our listen, most of our audience is you know, just
listening. So it would be better if you just describe it as much as
best you can. Yeah. Sure. So Let's say that you are writing
a piece of code that should, you know, do a specific thing,
and the code works. The code that you wrote works, but it and
it follows the code conventions and it's passing all of the tests and all of
that stuff that we you know, run on the code base. But it
might not be optimized, it might be really slow, it might be poorly
written, so to speak, and that happens all of the time. So
what Suriken does is that she makes sure that the code is not only you
know, follows our code conventions and the linters and all of that stuff,
but she actually makes sure that the code is as effective and you know,
fast and proficient as it could be by giving you small tips, hints,
pointers on what you could do better when writing that specific piece of code.
Let's say it's a function for logging in. You can write log in functions,
you know, in a hundred different ways, but usually there's only one
way that's efficient, effective and proven to be like the way to do it.
She will if you haven't written it in that way, she can you
know, give you feedback that that's the way you should write it, please
make this necessary change, and then you know, the developer goes in and
does that, and then I can feel confident that, Okay, someone has
looked at this and I can merge it into the main branch. Basically,
so very specific feedback on the actual code, which is something a human being
does today, is done by an AI assistant that I built in like three
minutes in our repository. And for me, you know, when I look
at it, I'm amazed that this actually works. I'm amazed by it.
I can sometimes, you know, I get emails when sure It can mentions
me in a comment on GitHub, and I get blown away, like this
is not actually a human being. This is just AI that's trained to do
a specific thing. But in that context, in that small, you know,
narrow vertical context, she does an amazing job as well as any human
might be able to do it. Is there a GitHub repo for a sure
again that we can look at or yeah, so super the super Agent propository
that you posted here in the chat if you go to the issues tab and
or sorry, the pull requests tab, which is where people you know contribute
code and you open up a pull request, you will see that there is
an contributor that's named sure it can Bites and she basically gives comments on you
know, different type of depending on what type of code you have writtain,
she'll give you different type of comments if it's if she doesn't find anything weird,
she'll just say, good job, you know, good job Charles,
thank you for the contribution, thumbs up love and that's that. Gotcha.
But what I'm wondering about is, you know, is there like a walkthrough
to set up something like this or yeah, so we have a I have
a YouTube channel actually that has like all of the I have a bunch of
like videos and stuff. When I set up a bunch of different type of
assistants and she's one of them. And actually the code is open source,
so anyone can grab your it can make her their own code reviewer if they
want to. Does she also like, okay, have you seen her occasionally
block full requests like not approveful requests? No. I trained her specifically to
only give feedback and not block anything, because you know, I wanted to
try it out. I didn't want it to be so definitive. So she
only gives feedback. You know, it's up to the user to do the
change or the country viewterr and up to me to make sure that the code
you know, works still when I merge it in. But but she only
she's she tries to be good vibes. I can, I can say as
much as that, So no blocking, No, you know, this is
bad. So she's trained to be humble, nice and give you know,
feedback, So it would be amusing to train her on Linostova's responses. Yeah,
yeah, yeah, yeah, And that's actually a good you know,
that's actually a good use case. If you want the language model to act
as a specific individual, if you have data on that specific individual, Uh,
you know, you could train the model to to try to resemble the
way like Linus would would answer a code review. I'm guessing it wouldn't be
as much of a good vibes that I have sued. So basically what you're
saying is if you use the entire Linux kernel code based as the training data
set, you will get very snippy comments whenever anybody tries to do a pull
request, you could get that or yeah, you could absolutely do that.
And I know that there are companies out there doing exactly this, training models
to resemble you know, Arnold Schwarzenegger or whatever it might be. Right,
So, right, you go pull Elon Musk's tweets and say this is how
you're supposed to respond. Right, That would just that would just be so
awesome. What is interesting to see? Yeah, to see stylistically, like,
okay, go pull this person's tweets, right, as long as it's
not like a corporately handled right, they don't have pr people running their account,
right, It's it's them straight into the phone. Yeah. I recall
like a few years ago, even before the whole machine learning craze, I
think Microsoft or somebody try and release some sort of a chat bolt into Twitter,
and she became a pornographic neo nazi within like two days or something like
that. Yeah, and they had to pull her, They had to pull
her off. They got so much heat that it basically killed the whole AI
think due to the heat they got for that. So that's the downside of
it, could be could be the downside. But I'm a positive guy.
I don't like you know, Yeah, I don't like to be like a
doomer on this kind of stuff. I believe that it's good for humanity.
I believe that you know, it's a must. It's a must. Well,
and I think you can intelligently because I don't know what parameters they put
onto that Twitter bot from Microsoft. They kind of said, well, we
just turned it on all of Twitter, right, And yeah, I think
it probably fed off of a bunch of other bots, right that in some
corner of Twitter that I just never see. But I don't know. But
this is interesting because you you can specifically pick the data set that you want
to emulate and then do it that way. Right. So you said that
you have two examples. You gave us one, right, So the second
one is is more of a you know, enterprise use case. And so
if you think of a regular company, let's say a you know, a
producing company or a company that is not a tech company, just a regular
company, you know, And so they have a bunch of data. You
know, they have a bunch of data in their management systems. They have
a bunch of files, they have a bunch of PDF excel sheets. They
have a sales or CRM system, they have a help desk system, they
have a bunch of data about their users. They have a bunch of other
type of data, and there is a lot of people working with extracting knowledge
from this data. So if you take like a marketing manager as an example,
one of the one of the key tasks of that marketing management manager is
to look at different data and try to figure out what works what doesn't work.
That's one of the jobs that they have to do in order to be
successful. So the second use case is that you can take like in my
case, I took our GitHub repositories, but in an enterprise use case,
you could take all of your enterprise data and do the exact same thing.
You can feed it to the model. You can train the model on that
data, and you can start asking questions about your data. You can ask
questions like which you know, what are my top five best customers, and
it will give you an answer within seconds. You can ask it to plot
charts on sales and it will plot a chart on your sales for the you
know, for whatever time period that you might have in mind. Usually how
this works is that you have a person at a company wants some data.
They go to another person at a company that has is tasked with like extracting
that data. A so called analyst. And so what you can do is
that you can increase productivity for all your employees by just training a model on
your data and making it available to whoever should have access to that data,
so they can ask questions without having to go to a analyst or without having
to wait one week to get an answer. You can just ask and the
model will answer that question. And if you think about how much data a
company of thousand employees of that size, which is not a big company like
a medium sized company, how much data they have and how much knowledge they
can extract using this technology by just training a model on it. So that's
the second use case that we see a lot of. Basically, that's the
use case that people use it for super agent that is Yeah, So two
questions about that really. Question number one, as we've seen with general machine
learning models chag EPT and whatnot, occasionally they quote unquote lie, they provide
misinformation, and they do it in a way that seems really reasonable and self
assured, which can really lead people down the wrong path. So my first
question in that context is when you, let's say, train the the agent
on your internal corporate data, how likely is it that when you ask the
question, you'll potentially get misinformation. Yeah, so it is likely, and
we call that hallucination. So the model can hallucinate, you know, and
give a false false answer, false answer to any query or question. Now
the question is how do you solve that problem? Because if you think about
it, if I ask you something, Dan, if we were working at
the same company, I would ask you a question, how do I know
that you are correct? The only the only way to know that you are
correct is to see the sources from which you derived the answer to whatever question
I ask you. If I ask you how many customers do you do we
have today and you just come up with a number, you know that could
be wrong. So usually what you do is that you make a PowerPoint presentation
and you show the actual data and then you you you answer that question by
showing the underlying data. And so what Superagent does is the exact same thing.
Instead of just you know, blindly uh accepting a text that is generated
by a model, we also uh you know, uh make the underlying data
transparent. So we actually show if you ask a question about a contract,
not only do we answer that question, we also show you that specific contract.
Not only the specific contract but also the section that was used to answer
the question. So you can ocularly look at the source, look at the
answer, and you know, make up your mind if this is you know,
makes sense or not. If it does make sense, you have the
possibility to rate the response. So if it's a good response, you give
it a thumbs up, If it's a bad response, you give it a
thumbs down. Every time you give it a signal, or engine fine tunes
the model on those signals so that the responses get better and better over time.
But I think the main thing is we are all used to using like
I am, at least using chat schipt. One of the issues with chatchipt
is that it doesn't actually show you the underlying source. So if you could
visualize that, you know, with the charts, with data, with tables,
with deep links to documents, then that's a completely different game. And
that's the way humans are used to communicating data and stuff to each other.
Like if you go on Twitter right now, somebody writes something, how do
you know it's true? You don't until you actually check out the data or
the source. And that's what we are trying to visualize, not only like
the answer from the model, but also the underlying data that is used to
answer that question by the model. Cool. That's that's really insightful. And
my second question in this context is how do you provide or what safeguard do
you provide against inadvertently leaking private or secure data. I mean, if I
train something on a model, and it's quite possible that somebody asks a question
that will reveal data that you know I don't want revealed, right, and
so this is a big question. And this is the primary reason to why
Superagent is open source. So when it's open source, it allows you to
do two things. The first thing is that you can deploy Superagent to your
own infrastructure, completely isolated in your own environment. Nobody else has access to
your data or anything like that. The second thing is that we allow you
to run Superagent with your own language models. Usually these language models are open
source models like Lama too that you've deployed to your own environment, or Mistrol,
which is a new model which is very effective, which you have deployed
to your own model. I myself am not a believer of proprietary models at
all. I don't believe that. I don't believe that a company if the
end game is to get enterprises to adopt this technology. I don't believe that
they can adopt it if it's black boxed. I don't believe that that can
happen. And I don't see any other examples actually where enterprises give away their
day to a black box and don't have any control over that data. That
very rarely happens. And me, as a developer, if I'm developing some
kind of app, it's very you know, it's very rare that the underlying
core technology is outsourced to some third party black box company. It almost never
happens. Almost never happens. One example is like Google Maps. I don't
know if you guys remember, but initially when the iPhone was launched, we
only had Google Maps. There were no Apple Maps, there was no app.
What happened, Well, Apple found out like like any same company would
do, like, we can't give away all of the you know, traffic
and data to Google on our platform. So what are we supposed to do?
The only way to solve that is to build your own maps you know
app and deploy that as an alternative. So Superagent allows anyone to take the
platform. It's open source, it's free. The boy to your own environment,
use your own model to run it without having to leak any data to
any external party, including US. So that's like the way to mitigate that
issue, which is a big issue. And I believe that you can build
you know, small hobby projects on top of open AI and GPTs which they
have just released. But I don't think that an enterprise grade like healthcare company
could leverage that. I don't think a legal firm or law firm could leverage
that due to the nature of it being black box and proprietory. So that's
why I believe in open source. So to clarification or follow up questions on
that, first of all, when you're saying you use your own data or
your own model, what you're actually from my ownunderstanding, what you're actually saying
is that you start with a general model, but then you refine it with
your own data, and that refined model stays within your organization and never leaves.
Is that is my understanding? Correct? Yes? And you also probably
make a distinction between which data you use for internal versus external services. So
for example, if if you know, in our cases next insurance, let's
say we might train I work at an insurance company. You know, we
do a lot of stuff with machine learning and whatnot. That's one of the
things about us that you do a lot of stuff online and you know,
use MARC learning. So there would be a difference. You know, you
might use that internal data about the policies that people have for the internal operation
of the company, but you won't expose that to to external users, whereas
you might train a model for external use based on you know, the questions
and that you might ask uh and various you know, general terms that are
pub in a public domain about you know, types of insurance and stuff like
that. That that would that would be externally available, right, And the
way we we we do that is that you know, every agent or assistant
that you create with superagents gets its own little brain, its own little memory,
its own data. It's completely decoupled from other assistants. So you can
actually deploy hundreds of these assistants with that are trained on different data sets that
some of them might be for internal use, some of them might be for
external use. A good example is we are working with the ispeing in the
UK and they have one assistant that's trained on internal data that they use to
you know, educate employees on different things inside of their company, and then
they have a customer support assistant which is only meant to help their users or
customers with general queries like how do I reset my router, how do I
do this? How do I do that? And that's only trained on you
know, public information that they have on their website. So the answer to
that is that it's very easy to deploy a lot of these assistants. And
I would say, like, in average, each user has around five different
assistants that they run on our platform. So if my understanding is correct,
you basically deliver three main things, if I find the same correctly. One
is the actual agent itself, like clone our repo, run our agent on
wherever you want to, you know, run as many instances of it as
you like. The second thing that you provide is the way to is the
ability to take an existing quote unquote standard the open source language model, and
then refine it using your own data. And the final thing that I understand
that you're giving is the way to attach those agents to various API or input
output sources. Is my understanding correct that that is the stuff that you provide.
That is exactly three things So the three things we usually, you know,
we have a different way of explaining them, but I would say it's
perception. That's the first thing. That's the model itself, the brain,
or sorry, this is the brain. The second thing is the perception,
which is the data that you feed to it. So we have the brain,
you have the perception. And the third thing is what we call tools.
And a tool could be an existing API, it could be a third
party service such as you know, Salesforce, but it could also be code
that you want the assistant to run, so you could build you know,
automation workflows with assistants that run code, predefined functions, or whatever it might
be. We call them tools. So it's the brain, the perception,
and then the tools. These are the three main things that we the three
main pillars that Superagent is built on, and we make it easy for any
developer to orchestrate them in order to create an assistant that can do basically anything
that you would wanted to do. So if I wanted to build an assistant,
I guess the things that I'm thinking about here are you know one it
sounds like, yeah, I can hook it up to open AI, or
I can you know, pull in my own model. That's that's probably you
know, beyond having it set up and being able to access it through standard
methodologies, you know, you handle all of the stuff as far as like
providing it more data or providing it specific data and then extracting the responses.
I guess the other part of a chatbot, though, is or an assistant,
is the delivery, right, So whether it's in some kind of user
interface. I'm thinking like code assistance, right where you have them plugged into
like visual studio code and it'll highlight code and give you feedback on the code
and things like that. But I'm also thinking like chatbots, right, so
maybe a discord bot or a you know, an embedded bot on your website
that people can ask questions of and things like that. So how do you
interface with the delivery of these systems? So we do that in two ways.
We do that in a so called no code way, which is basically,
you create your assistant and we'll give you a embeddable chat or a embeddable
you know, user interface which allows your users to interact with this assistant.
Similar very similar to how chat GPT is formed. Basically super simple but still
powerful. So that's the first way. That's for developers that are you know,
just starting out trying to prototype something, trying to get a feel for
how how their assistant you know, actually works and how it how it how
accurate it is, and you know, prototyping. The second way, which
is the most used way, is that we give you three set of SDKs
and a rest API, so you can use either of these SDKs or rest
APIs to orchestrate your own assistance and then build whatever type of UI you want
for the delivery part. So you know, the thing with I think that
something that people are missing, you know, is when you look at chat
GPT and when I look at it, and I come from a design background.
I've designed and built you know, front and type of applications all my
life basically, And the one strange thing is that when chat GPT launched,
it was like all of the other UI components that have been refined for you
know, a century just got thrown away and got replaced by a chat markdown
box. And you know, that's in my you know, when I think
about that, that's really like, that's really not the way the Internet usually
works. You know, when you consume a software, it has a bunch
of different components which makes it feel valuable where you can extract more value than
just text. You know. One good example of this is a company that's
called Perplexity AI. I don't know if you guys know about them, but
they have built basically the new Google Search. And it's not just the chat.
It has a bunch of other type of UI components that makes that user
experience ten times better than what Google Search is today. So I believe that
if you are going to deploy a chat you know, an AI assistant chat
plot, you need to use existing you know UI components that people are used
to, not only chat. It's very limiting in what you can accomplish with
only chat, and it doesn't work well for all use cases. It might
work well if you are trying to chat with an aim then it works well,
but if you're trying to extract knowledge, then chat might not be the
best way to do that alone. Stand alone. There might be other components
that you would want to visualize for the user in order for them to be
able to quickly extract information. Simple example would be a chart you know that
is that a user can interact with, which is very common in any other
software. Right, all dashboards have charts, but chatchipt doesn't, you know,
And so I believe that in the future we will see you know,
if you think about all of the UI component libraries on MPM, like the
registry for no JS, there's a bunch of awesome UI components out there.
I believe that AI and these UI components will merge eventually so that you get
the power of AI in those use user facing UI components. So having an
AI dynamically generate the user interface needed for the user to absorb or extract information
that they are looking to extract from the assistant, I believe that's the future.
That's something that we are working on actively as well, making it more
dynamic and making it more rich the user experience, not just only chat.
I believe that's a big thing for adoption, especially if you're looking at enterprises
that are trying to adopt this inform you know technology. By the way,
what is superagent implemented in which TIM members back in back in this Python uh
concurrent Python so asinc which was quite a mess but eventually worked out, and
it runs on fast API, which is an open source framework for running concurrent
Python threaded Python basically, and then we have some services that are building Rust
specifically for memory. So you know, if you just take this is interesting,
but if you just take a language model off the shelf and try to
chat with it, it won't actually remember your previous conversation, and it's like
talking to someone that doesn't remember stuff, you know, and that's just not
feasible. So you have to build memory. And that memory usually is some
kind of key value store, some kind of readiest database, you know,
and then you need some way of integrating your model into that key value store.
So that part of our service is built in Rust. The memory,
it's forth a short term memory, but also a longer term memory, so
you can ask questions about stuff you chatted about, you know, a month
ago. So that's built in RUSS. And then the UI is built on
next jas, which is a framework, open source React based framework typescript and
yeah, so that's that's the staff. Basically. Infrastructure wise, we you
know, rely heavily on GCP and AWS of course, and so that's where
the infrastructure is at this point. Okay, so I mid missed this because
I came in late. But how is this different from Aluma how is the
different plophically and how's it different technically? So technically LAMA runs locally on your
machine. Uh super, any where you put it. Uh no, it
doesn't actually because you need to put the model somewhere, right, So it
runs the model locally but right right, But what I mean is wherever you
put it, right, Like I can put it on Digital Ocean. If
I want to pay a billion dollars a month, I can put it on
AWS. But like, yeah, yes, yes, yeah, So it's
similar to a LAMA in the sense that you can run different type of models.
Its open source and that kind of stuff. What differs is that we
focus on a specific type of agent, the knowledge assistant as we call it.
So we feed your model and fine tune your model on the data that
you want the model to have access to. We both fine tune the language
model. But something that people miss a lot is that there is actually another
type of model involved in, you know, fetching data from third party sources.
That model is called the retrieval or encoder model. We also fine tune
that model on your data. There's actually two models in play. Usually when
you use chat, youpt or anything else, and you upload a file to
that. So these models we fine tune on your specific data, making them
super accurate for the specific use case that you have for your assistant. And
so it's an orchestration layer. Is that what's referred to as long chain Because
I was looking into this because I wanted to figure out how to do with
a LAMA. Yeah, and what was coming up was long chain And it's
like, it's not part of a LAMA, but it's like you there's an
API you feed in more data. Is that right? Is that what that
is? Right? Yeah? So so lang chain is an open source framework
which allows lang Chain is an open source framework that allows you to I'm actually
a contributor there. That's where I started off contributing to that framework that allows
you to build these type of assistance, any type of assistant. It has
similarities to superagents, and we actually utilized lang chain in parts of our infrastructure
as well. The main difference is that lang chain is built for machine learning
engineers, people that know what the heck they're doing, people who know how
to accurately fine tune a model, how to accurately orchestrate the whole assistant,
you know. Superagent documentation seemed like they were using very very technical terms that
once I understood it seemed like things that could have been explained in a sentence.
Yeah. So Superagent is like, you know, the version of lang
chain that's meant for the mainstream developer people that don't have any skill set or
any knowledge or background in this type of technology. In the short, you
know, if you want to explain Superagent, it's like Stripe for payments,
but it's for building AI assistance instead. Think you know, prior to Stripe,
you know how hard it was to you know, set up payment,
recurring payments, subscriptions on your whatever, you know service that you were building.
It was a pain in the butt. Uh. Most of these open
source frameworks are you know, give you the building blocks to build whatever you
want, but you need to know what you're doing. In the Superagent case,
you don't need to know anything. You just instructed to do specific things
with texts. Give it a prompt, as we call it, You feed
it with data, and we take care of the heavy lifting on our side
so as to make sure it's actually use a chatbot to generate the other personas
uh, yeah, we use we use language models to generate training data and
you know, generate all of the stuff that needs to go into the model
in order for it to be accurate for the specific use case that you have,
and you don't have to think about any of that as a developer.
That's on our side. That's like the value we bring to you as a
developer. You know, if you talk to a Superagent user and you ask
them why do you use Superagent? The number one thing they tell us is
that Superagent allows me to focus on my product. I don't need to become
some other kind of engineer and learn something new. I can focus on the
stuff I'm building, like an iPhone app, and I can just integrate all
of this wonderful technology with a simple SDK like in twenty lines of code.
So that's the value prop that we bring to developers. Cool cool, All
right, well that's nice, thank you. I don't want to shut down
the conversation, but I do have another podcast scheduled in like eighteen minutes.
You're living a busy life, chuck, Yeah, So I'll just I'll just
ask one last final, really quick question. Yeah, you usually run models
locally or in the cloud in most cases one hundred percent cloud, one percent
cloud. That's so expensive though, how can you but it isn't No,
it isn't It isn't expensive. It doesn't have to be on our cloud.
It could be on your cloud. You know. That's the thing. If
you have if you can deploy your model to your own cloud, you can
deploy your model to these serverless like infrod providers that are out there, and
you can get a model running for you know, the price of running a
model. It's going to go to zero in a very short time, that's
the thing. So it is that with the cloud yet so far, so
far, everything that introduces more expensive than the previous thing. Prices have not
gone down in a decade. They've up. Yeah. But but if we
if you think about you know, uh, it might not be zero now,
but if you think about the trend, even open Ai, you know,
they slash their prices with like yeah, if you're talking about open Ai,
yes, yes, yes, because they are going to optimize it and
they may eventually be able to get the price down, you know, not
that they're not having to pay what you would have to pay for Azure right
and getting they're getting a very different rate. Yeah. And and that's the
thing that the other providers as youre a w S. They are right now
as we speak, deploying technology and this has already happened where you can basically
host your model in a serverleust environment and only pay you know, the couple
of zero points zero zero two five cents per token that you would pay open
ai. So this transition is already happening. It's a requirement otherwise nobody will
be able to be able to run this technology in production. You know.
So if the price is high, no business can run it. You cannot
have a chatbot that you know costs thousand, ten thousand dollars a month and
three people are using it. So the whole industry is pushing this for the
prices to go down, and we already see it now. Even the prices
of the hardware is going down GPUs. So if I if I want to
run Superagent I you said it's open source, like, so I could just
pay a flat forty dollars a month, get eight cores eight VCPUs and run
it on that and that would be good enough for you know a few people
using it at a time. Yesaight, and completely free. You don't have
to pay us anything since it's open source. No what I mean is like
I'd have to host it somewhere. Yeah, you have to pay it.
It's like forty bucks a month for you know, and then you don't have
to pay per token. You just yeah, Okay, I would love I
would love to learn how to host this because I've got some people that are
I was actually going to build something with a Lama for some people, but
yeah, I and I think let's do that. And I think Llama is
great. And the reason why I think it's great is because it's completely open
source. It allows you, as a developer, to have complete control over
what you're doing, where you're deploying it, what models you're using. It's
up to you. It's not up to some you know company to decide where
your data is, how you can extract the data, and all of that
stuff that comes with like proprietary black box models. I don't. I don't.
As I said initially, I am strongly opposed to that type of model,
and I don't think it will work. I don't. I don't think
it will work. How do How do I pay you for support and to
make sure that superagent is still around? Is like the number one reason I
pay for things is like, I want to pay because I want the project
that I'm paying for the product, I want it to be successful so that
when I wake up tomorrow, the web page is still there and the download
button is still there, and the support email is still there. Because they
don't need your money, They've got why compiny their money? But yeah,
I agree with Yeah, the bill comes due and we we don't want to
live off of VC money. That's not the plan we want to We want
to create and we talked about this initially. The thing with open source is
that it's great to build a community. You can get a lot of users,
but it's very hard to create a business out of that. A good
example of that is, you know, let's take Django, the Python framework.
How many businesses have been built on Jangle A lot of them. How
many people for Jangle zero? Same thing with fast Api, it's one of
the best Python frameworks out there. How many people pay for it zero?
How many people use it a lot? So it's really hard to take something
open source and make a commercial business out of it. That's what we're working
on now to be able to do that. There are examples of people who
have succeeded with that. By offering you know, support plans and stuff like
that. There are ways of doing it, but it is the solution is
hosting. I'm not I don't think that. I don't think that that will
help because the whole premise here is that you would want to host it yourself.
You don't want to give your data to us, to ISMAEL. You
don't want me to have all your financial data. You want to have all
your financial data. So hosting might work for the absolute, you know,
smallest companies, but if you're working with a healthcare company, they won't host
with us at this point. So what we are doing there is to give
them services such as AJ mentioned support packages. You know, we we we
can run instances for them on their cloud. There are a bunch of different
business models you can run, but it is hairy and it isn't you know,
as straightforward as just having a B to B SaaS company that is proprietory,
you charge per month, people trust you, et cetera, et cetera.
So that was what I initially said, is that it is a hard
thing to monetize and a lot of people have failed, I mean, the
greatst have failed at it. So I'm just hoping I a buy button on
it, and I'm going to buy this thing in whatever form because I have
are I have to build something. I'm going to be using some mix of
either Olama or open Ai. There's still a little bit of you know what
am I going to do? Your solution. I can choose to switch between
Hugging Face, which is where I get the Alama models, and open Ai,
which is where our prototype is deployed. So like to me, this
provides a layer of abstraction that for the things that we want to do for
this project, this is the right tool. So you put the buy button
there, I'm going to buy it. It doesn't even really matter what you
charge. I'm going to pay it because i have a time crunch and I've
got to get a storry. Sorry but yeah, no, I you know,
in a lot of ways I agree with AJ and this reminds me of
something that I'm going to throw out my picks. But yeah, let's go
ahead and do picks, and uh yeah, I've got like ten minutes and
then I've got to roll into the other show. Dan, what are your
picks? I'll make it short and sweet this time. So my first pick
is going to be Prometheus, the monitoring solution. I've been using it a
lot to as you people who've been listeners to this podcast. No I do
stuff related to performance, to analyzing how applications execute, and Prometheus has been
an amazing tool for this purpose. So collecting all sorts of performance data and
execution profiles, getting them in there, building dashboards using Grafana, running all
sorts of from QL queries. I'm now actually looking to try to solve a
really hairy challenge within our organization. We've got something like fifteen micro services,
some of them having hundreds of endpoints, and I want to be able to
analyze the entire system to catch performance degradations without having to manually configure and specify
you know, limits manually for each and every endpoint and dependencies and whatnot.
And it'll be really interesting to see if I can get something like that up
and running just based on the capabilities that are built into Prometheus. But you
know, so far, so good, so we will say, but I
have to shout this out as a monitoring tool. And by the way,
I also contributed back into the prom ql no JS client, which was really
cool, and I have some additional ideas of some additional contributions that I want
to do back to that project. We probably should have the owner of that
project on our show. It's a really cool project, so that, yeah,
for sure, So that would be my first pick. My second pick
is if you've listened to our past episode, you know that we did it
around a bunch of polls that I ran off my x or Twitter account.
I've recently run another one which is getting a lot of votes even as we
speak, which is basically trying to see which framework is the unreact, Like
if you think about React, what's the opposite of React in the in for
in front and frameworks? And the options that I gave were, well,
you know X only allows like four options like max, so that that's kind
of limiting. But the ones that I gave our HTMX swelt solid, slash,
quick, and other. And ask the people who you know who answer
other to like specify what they meant, and I got some really interesting responses.
So I'll probably share the link to that tweet, like people like people
like Jack Harrington jumped on, Ryan Corneato, Carson Grove himself and others.
It's it's becoming a really interesting discussion. I'll just throw it out there that
Ryan's Corneato's choice for the most unreact framework out there. You know what it
was. Can you guess no, React is the most unreact one of course?
Yeah. That he basically make like made a distinction between the current react
and when where react was ten years ago and said that current react is like
the most unreact compared to React of ten years ago and vice versa. Uh
So it was a really interesting choice. Anyway, It's it's a fun poll.
We'll see how it comes out in the end. I won't spill the
beans about Too's in the lead, although you might guess. Uh, and
those would be my picks for today. Awesome, what are your picks?
Well, first and foremost, I got a his in Herbi dat Well,
I think all the days are his and Herbi days. I think they all
have the female button as well. But it was only thirty bucks. I
don't know why I didn't get one years ago. I guess I thought it
was going to be complicated or something. The only thing complicated about it was
the people who built my house. Of course, couldn't spend the extra two
dollars to put the adjustable pipe on there, so I had to go down
the street to Low's and grab a four dollars and sixty nine cent connector piece
to put it on, Like it came with everything in the box that it
should have come within the box. It is. The toilet was installed in
a way that it was permanent, and I had to replace a fixed length
tube that could not be moved in any way because it was solid rather than
flexible. But anyway, yeah, so now instead of being like an eighty
American and spending one hundred billion dollars on toilet paper to do the job that
the good Lord figured out how to do for us millions of years ago.
Yeah, yeah, that's that's the exciting news. But I think, like
warm water everywhere, like like going on, it's just cold water. It's
it's not fancy. It's just like a cheap thirty dollars thing. It seems
to work perfectly. It's not leaking as far as I can tell, at
least not. Yeah, it means day one, but it's it's the reason
I was late because it was gonna be I had an hour. It's a
ten minute project. It's like, undo this one thing, put this thing
in place of it. You're done. I mean it gets a little cramp
depending on your bathroom side trying to reach around. But then because the whole
thing with the tube, and it's like, oh crap, I can't put
this back together. I have to make a run to Low's. I have
to wait and oh crap being the operative word here, yeah, yeah,
yeah, and then I and then I had a nice crap after it all
and was able to confirm that it works. So anyway, Yeah, don't
be a dumb American, be like the like the people in the rest of
the civilized world and and use the same thing that people have been using for
thousands of years to wipe your heinee. Okay, get some water in there.
Goodness knows, and then I'll uh, there's there's a couple other things,
but I'll try to just keep this one short for today's purposes and later
huh, Charlotte got to go in four minutes, so fast, right.
So the other thing I'll pick is a LAMA, because I have used Olama
and I've gotten value out of Alama, and it's super easy to get some
of the models that automatically downloads them from hugging face for you. You don't
even have to know about hugging face. But I've used misterl and I've used
code up, and I've used a couple of other models, And depending on
the model you pick, they're better than Chad gpt is for a specific because
they're the benefit of Alama is that rather than getting a you know, billion,
billion, billion data point model of everything, you're getting models that are
more fine tuned for specific things. And then there are ways to lang chain
to load stuff in. It automatically does the history for you between the different
requests and whatnot. And so I'm just gonna put the web install dot dev
slash Olama link there because that's where the installer is. That makes it dead
simple so you don't have to think about it. And then uh yeah,
I will just mention that I I started using Home Assistant and I now have
a thermostat that is connected to a Google calendar like calendar, and I'm gonna
I'm gonna put this in my wall soon and I'll pick that next time.
So that's all all right. I'm gonna jump in here with a couple of
things. Now, I always do a board game or a card game.
This one is one that my wife got us for Christmas. I think technically
Santa brought it for Christmas, but my eight year old's not here to split
hairs on that. And yeah, so Disney Chronology, it's a really simple
game. I think we play it in like fifteen minutes twenty minutes, especially
if my daughter goes before my wife, because my daughter can't get them right,
so my wife steals them. So essentially, you pull the card.
It has a year, and it has like Disney released steamboat Willie to blah
blah blah blah, right, and then the answers like nineteen twenty eight or
whatever, right, and it has the month of the year. And so
you have three of those cards in front of you and you pick either before
all the cards in between too, you know, any two of the cards
that are next to each other, or after your last card, right,
And so yeah, if you don't get it right, then the person next
around the circle gets a chance to you know, put it into their front
and steal it. That's the whole game. But if you're kind of a
Disney nut like my wife is, then it's kind of fun to see if
you can guess them. And if you've got the fourteen year old going before
you, that can't get them right, then she wins because she steals them
all. Anyway, it was fun, a lot of fun. So I'm
gonna pick that it's a super simple game, and yeah, it's it's fun
because Disney, not because the game, because the game's idiotically simple. Another
pick that I'm going to throw out is, so a lot of you know
that I spend a lot of my time writing Ruby on Rails, and so
I followed David Heinemeier Hanson or DHH and he and Jason Freed are doing something
a little bit different going forward in kind of the SaaS space. I'm just
going to put the link in for once dot com. The idea is is,
way back in the day, you used to buy software and then you
could install the software wherever you want it and use it however you wanted.
And the way you guys were talking about super Agent was kind of the same
idea in some ways, where it's, hey, you can take this code
and you can run it wherever you want and just own it and love it
and you know whatever. And so they're pushing forward this idea out there on
the Internet where they deliver effectively what would be SaaS apps, right so you
can install as many instances of base Camp. I don't know if they're doing
it with base Camp, but as many instances of whatever as you want.
Because you now own the code, you own the version of the code you
bought. So I think that's cool and I think it'll be interesting to see
how much of a difference it makes out there in the market. I think.
I think it could be disruptive. I think in other ways it may
take a while for people to catch on and go, oh, this is
a one time payment and this is a good deal. But yeah, I
really love the idea, So I'm going to pick that next thing. You
know, people will want to purchase and own music and movies. No,
some of us still do. Yeah, yeah, yeah, I think if
that one, people just have to you have to unlearn what you've learned.
Because the whole thing is people think that that running a command to start a
service on Linux is like something that requires a four year degree. It's like
it's a five minute thing. If that, Yeah, I think the other
thing that you have to run that people are going to run into is they
will have purchased movies and had them on some streaming service that they you know,
they got access to them on right, so you enter the code or
whatever, and then it's not going to be licensed to whatever service that is,
and they'll lose access to it. And then they're going to go what
the heck? And then right, and then they'll start looking at Okay,
how do I own this again? I think flex servers might take off for
some of that. I've also scene another and I'll see how I can find
it because I've been wanting to play with it. But I've seen a service
out there that will attach to your audible account and download all of your audio
books in open audible open Audible. It's like twenty bucks. It's the best
thing ever. I love it. I have to I have to cut in.
I actually have to drop off guys. So off is. It was
great speaking with you. I learned a whole bunch best of black. Thank
you, thank you, and and it's an awesome thing that that you're doing.
And I'm really loving the approach so so again, and it was great
having you on and by appreciate it. I'm I'm going to send an email
to uh Ruby Roads real quick and just let them know I'll be a few
minutes late. But he's my, what are your picks? Uh? So,
first off TV show started rewatching Fargo. I don't know if you guys
have watched it, but it's an amazing show. It's like five seasons and
they are not interconnected in any way, so you can jump into any season,
pick the season you want and the actors you like the most, and
it's just amazing. So that's on the show side, the fun side,
the dead side. I have this project, open source project that I would
like to highlight, and you should bring that guy on. His project is
called outlines dot dev. He's a French guy called Remy, and what he
does is that he allows you to create models that can answer in other formats
than just text without having to write a bunch of prompts and stuff. So
it's a technique that you use to you know, basically an extension you plug
into your language model which allows you to control its output, which is because
it gives you ideas of what other type of extensions you might want to plug
in there that would do other things. So that's really interesting. It's state
of the art stuff. Outlines dot dev. I would I would pick that
one. All right, cool, all right, well let's go ahead and
wrap it up. Thanks for having East Mail, Thank you for having me
guys, it was a pleasure. My Twitter is My Twitter is home man
P so h O m A n P. Thanks bye,