Speaker 1: Hey, everybody, welcome back to another episode of JavaScript Jabber.
Speaker 1: This week, on our panel we have Dan shapiir.
Speaker 2: Hey from the still very hot in Monkey tel Aviv.
Speaker 1: I'm Charles Maxwood from Top End Devs. Yeah it. Yesterday
Speaker 1: we had a high of ninety five, which was nice
Speaker 1: and cool compared to what it's been so yeah, I
Speaker 1: feel that. We also have Steve Sewell from builder Io. Steve,
Speaker 1: do you want to say hello and remind people who
Speaker 1: you are?
Speaker 3: Yes, everybody, I'm Steve. I'm our co founder and CEO
Speaker 3: at Builder. We make cool AI, designed to code, designed
Speaker 3: to live. You know, website cm my stuff which I'm
Speaker 3: sure going to And it is only fifty eight degrees
Speaker 3: farrenheit and San Francisco. It's not hot at all. I
Speaker 3: wish it was a little hard to be honest.
Speaker 1: Wow, is that why you're sitting in the server room?
Speaker 3: Get a little a little warm? Yeah? Exactly?
Speaker 1: Yeah, good deal. Well we got you on. I've seen
Speaker 1: a whole bunch of videos from you about AI. I
Speaker 1: know you, you know, run builder dot io with with
Speaker 1: Mishko and a bunch of other folks. And I guess
Speaker 1: what I'm wondering and where maybe we should start is okay,
Speaker 1: So what is the CEO of this? You know, hey,
Speaker 1: build your website with us. You know, why do you
Speaker 1: care about AI? Like, how does that fit into the
Speaker 1: life that you're running? And you know, how will that
Speaker 1: fit into the life of somebody who's going, hey, I'm
Speaker 1: a JavaScript developer.
Speaker 3: Yeah, no, it's a it's a great question. So, you know,
Speaker 3: I actually, as words it sounds, I'm a little bit embarrassed.
Speaker 3: In retrospect, I think it's probably still the right idea.
Speaker 3: I was excited by l M progress when I was
Speaker 3: seeing it. You know, GBT three was interesting but very
Speaker 3: hard to get good results from. Three point five was
Speaker 3: a big breakthrough, Like, ohater, it was better. It was
Speaker 3: noticeably better and easier to work with. You could talk
Speaker 3: to like a human, not like you'd weird. You know.
Speaker 3: I remember GPT three somewhere I would say something even
Speaker 3: as symbol as ending a your prompt to get a
Speaker 3: completion off of with like a new line or something,
Speaker 3: it would like break the whole thing. So they'd be like,
Speaker 3: just don't do that, And like three point five was like,
Speaker 3: let's assume humans are humans and we will you know,
Speaker 3: make sure it works with whatever you give it. That
Speaker 3: was pretty huge breakthrough. But even then you saw like
Speaker 3: all these companies rushing to add eight out of the product,
Speaker 3: or like we even have vcs who invested in us,
Speaker 3: Like every CEO is talking about redoing their whole roadmap
Speaker 3: to be focused on AI. People are rebranding their companies AI,
Speaker 3: and I was like, you know, I'm thinking of like
Speaker 3: crypto and all this stuff, and I'm like, no, we're
Speaker 3: we are not doing that at all. We are dabbling,
Speaker 3: We're not changing significant plans. As we dabbled further, the potential,
Speaker 3: hypothetical potential became really clear. You know, if you work
Speaker 3: in that space of like you know, you've got a
Speaker 3: design and a program like FIGM on one end, and
Speaker 3: you've got a website or app that you probably have
Speaker 3: a mix of developers coding on because we focus kind
Speaker 3: of larger businesses generally, so you've got developers. You're not
Speaker 3: like some small mom and pop shop or something. So
Speaker 3: you've got developers writing code, and you have people who
Speaker 3: are not developers trying to put out pages or update
Speaker 3: pages or something through a CMS. Where can AI help most?
Speaker 3: You know, we've seen all these cool demos and this
Speaker 3: is where things get confusing too, is there's a lot
Speaker 3: of demos that are not representative of the average users experience.
Speaker 3: So you could, for instance, go into chat to BT
Speaker 3: and have it summarize a long piece of text, and
Speaker 3: every demo will do well and every user's experience will
Speaker 3: probably be pretty good at that. That's something that the
Speaker 3: elms are good at. Take a large amount of information
Speaker 3: and condense it down. And if you don't like the
Speaker 3: style of how it condensed it, you know, like the
Speaker 3: language it used. Like when I paste a huge amount
Speaker 3: of stuff and I tell it like turn this into
Speaker 3: an email, it assumes that I'm like this corporate person
Speaker 3: emailing of a million people. No, no, we're startup for what
Speaker 3: are we now? Seventy people? I use a pretty chill
Speaker 3: vibe when I talk. Here's some examples of how I
Speaker 3: usually talk. Emulate that. GPT is still pretty bad at that,
Speaker 3: but Claude as much better in my experience. But anyway,
Speaker 3: those are good. But then you see these other demos
Speaker 3: of like you know, these one off cherry pick things
Speaker 3: of like hey, I built me this whole program. That's awesome.
Speaker 3: In fact, Claude with artifacts is great at generating like
Speaker 3: a snake game or something. But that gives you a
Speaker 3: look into like what people would like to have happened.
Speaker 3: You know, people would like to If I'm a developer
Speaker 3: that has a Figma design coming from a design system
Speaker 3: and it's got behavior implicit in it. This is a
Speaker 3: new dashboard mock up with we we've got APIs for
Speaker 3: this data, and we've got components for these charts, and
Speaker 3: it's you know, it's got a layout and Figma. If
Speaker 3: I can just turn that into like almost finished code
Speaker 3: to just start me off, there's always nuances I got
Speaker 3: to do. But if you can connect it to APIs,
Speaker 3: assemble the components, create the layout and tail and whatever
Speaker 3: I'm using, and then let me work on it from there,
Speaker 3: that's pretty cool, especially when you know those things can
Speaker 3: actually work fairly reliably. That becomes like a why not
Speaker 3: type of thing? Why am I writing this all by hand?
Speaker 3: If the I can do that pretty effectively. On the
Speaker 3: flip side, if you're because part of what we are
Speaker 3: is headless CMS. So if you're a user trying to
Speaker 3: create new pages within a next JS app or whatever,
Speaker 3: same thing, you've got this mock up of a page.
Speaker 3: If I could just click a button and make that
Speaker 3: become real and then maybe use natural language to say,
Speaker 3: actually move the button over there, or when I click
Speaker 3: the button and should trigger the off flow or whatever,
Speaker 3: rather than learn this complicated tool of like, yes we
Speaker 3: have the off components registered in the tool, and yes
Speaker 3: we have our APIs connected over here, and if I
Speaker 3: know how to click one hundred buttons, I could do it.
Speaker 3: But if I could just say it and have it happen,
Speaker 3: that's the pipe dream. That's the obvious reason to care.
Speaker 3: The question obviously is just how well can it do that?
Speaker 3: And then more importantly, how can we make a reliable
Speaker 3: path to make as many of those dreams come true
Speaker 3: as possible without being full of foot guns. And that's
Speaker 3: been kind of our focus in research and development of
Speaker 3: the last year or two.
Speaker 2: So it's basically the domain that the space that you're
Speaker 2: working in is essentially Figma to code.
Speaker 3: That's a way to think about it. It's not every
Speaker 3: customer's use case. You don't as you can imagine if
Speaker 3: you could turn prompts into real life stuff using the
Speaker 3: React components you have and stuff you don't need Figma,
Speaker 3: you can just tell it to make me a thing
Speaker 3: and it can make you the thing. But Figma is
Speaker 3: probably one of the most common ways you represent in
Speaker 3: great detail what you want to happen before it happens.
Speaker 3: So you can think of us as like the if
Speaker 3: you saw TL draws old make or not old, Like
Speaker 3: they're make real demos, you know they diagram you hit
Speaker 3: make real becomes real. You can think of us as
Speaker 3: like an entire make real application or platform. Most of
Speaker 3: the time those are in Figma designs already. Sometimes they're
Speaker 3: just in a gyro ticket or a slack conversation. We
Speaker 3: we imagine a future world is not too far where
Speaker 3: we could have like a slack bot where you tag
Speaker 3: the builder slack bot. It looks at a thread, it
Speaker 3: summarizes your idea, implements, it sends you back a link.
Speaker 3: How does this look? Turn that to code, sink it
Speaker 3: to your code base, or just hit publish. Run it
Speaker 3: as a five percent test and see how well that does.
Speaker 3: Stuff like that, you know.
Speaker 2: Can be without the Figma.
Speaker 3: Yeah, with or without the Figma, that's one way to
Speaker 3: put it totally right.
Speaker 2: To be fair, though you're not, it's not really green field.
Speaker 2: I mean Figma themselves, as I recall, are looking at
Speaker 2: ways of transforming their designs into code m that's correct.
Speaker 3: Yeah, I think the green fields of l ms are
Speaker 3: what you know. I think the the hype is real.
Speaker 3: I was very against making any deviations to our plans
Speaker 3: or marketing or anything just because AI looks cool. I
Speaker 3: feel like there's a lot of startups who just at school,
Speaker 3: we want to be cool. Let's just do that. We
Speaker 3: put a lot more thought into it before investing behind it.
Speaker 3: And to be honest, in most cases where I saw
Speaker 3: a startups just say AI, the school is do AI
Speaker 3: for lack of better terms. Some would see spikes and
Speaker 3: sign ups, some would do kind of cool things. But
Speaker 3: where I saw many fail completely were when they were
Speaker 3: adding AI for AI's sake, as opposed to solving major
Speaker 3: problems that only large language model school solve. So there
Speaker 3: are of ideas of things we could do with AI
Speaker 3: for the sake of doing it. But one of the
Speaker 3: biggest problems that Pigma has in their code generation that
Speaker 3: we have and explorin this too, that site builders have
Speaker 3: is that people have a lot more ideas then they
Speaker 3: can get into code in a high quality, and any
Speaker 3: tool has tried to turn any figma idea into code
Speaker 3: in an automated way is bad. It's the simplest way
Speaker 3: to put it. They do a very very bad job.
Speaker 3: And we found a few techniques that work pretty well
Speaker 3: that can only be done with llms. We found that
Speaker 3: we at the end of the day, like it's funny
Speaker 3: because I don't want to overhype lllms. I think they're
Speaker 3: a critical missing piece, but they are not the full solutions.
Speaker 3: What I mean by that is, you know, I've got
Speaker 3: some videos about like use AI as little as possible,
Speaker 3: and I still firmly believe in that most of our
Speaker 3: AI solutions that just look like, for instance, a Figma
Speaker 3: design is the input or a prompt and the output
Speaker 3: is code. It looks like we just fed that into
Speaker 3: the LM and we got that out the other side
Speaker 3: a more code than that, and that's good because we
Speaker 3: have control over that. It makes the product differentiated. There's
Speaker 3: not just an enormous amount of code pre processing the
Speaker 3: heck out of everything, post processing the heck out everything.
Speaker 3: But also we've trained our own models for specific parts too,
Speaker 3: and so you can take You could take various approaches here,
Speaker 3: but the one that's worked well for us is solve
Speaker 3: everything without AI as much as possible, break it down
Speaker 3: into the Tinese problems that are just not really solvable
Speaker 3: with just typical you know, conditions in code. For us,
Speaker 3: that was those were much smaller than you think, especially
Speaker 3: if you're really grind out the problem. It's really small.
Speaker 3: But you can't do without AI. But there's probably gonna
Speaker 3: be some point, like if you're turning designs to code
Speaker 3: or prompts to code or whatever. That what you can't
Speaker 3: decipher with most models, you could train yourself. You know,
Speaker 3: you could do training as simple as things like decision trees,
Speaker 3: which can work great if you just don't know what
Speaker 3: value to put into a box or set of values decision,
Speaker 3: try training with like XG boosts or something. It could
Speaker 3: be phenomenal to wipe out a bunch of crappy conditional
Speaker 3: code into one. Essentially, the AI takes example datas and
Speaker 3: kind of writes conditional code for you in a sense.
Speaker 3: Fantastic We use that for certain things. You can do
Speaker 3: fancy stuff like random forest. You know, a bunch of
Speaker 3: decision trees helping make a decision awesome. But what nothing
Speaker 3: can really do well outside of an LM is understand
Speaker 3: meaning of things and that's where again the LLM in
Speaker 3: our experience should not be treated as this opaque box
Speaker 3: where very raw inputs get out and very finished outputs
Speaker 3: get out the other side. But when you start identifying,
Speaker 3: here's a basic example. You've got a responsive design and
Speaker 3: it's got desktop end mobile, and on desktop you've got
Speaker 3: a nav you know, with all the links horizontal, and
Speaker 3: on mobile we've got a Hamburger menu. There's no world
Speaker 3: where we could solve that without an LLM. But again,
Speaker 3: we're not passing the designs in and saying figure it out.
Speaker 3: We're not doing screenshot to code, which is kind of
Speaker 3: awesome sometimes but always not quite right. People want you
Speaker 3: know you designed the position, you use the specific design
Speaker 3: tokens of Figma components, so those map to design tokens
Speaker 3: compotes in your code. When it's a screenshot, that's the
Speaker 3: conduit you lose all that information, and that's try not
Speaker 3: to lose that unless you're like, you know, again, you're
Speaker 3: just some random you're building some random project on the side.
Speaker 3: Maybe you don't care, it's it's just beginning. But if
Speaker 3: you're doing Burker Company, you need a lot more than
Speaker 3: that for it to be useful. You don't want to
Speaker 3: have to get code and rewrite it all, and so
Speaker 3: you need to use some type of LLM to understand
Speaker 3: the meaning of that and say, oh, this should become
Speaker 3: an interactive Hamburger menu. It did not have to figure
Speaker 3: out all those other things, how components map, how design
Speaker 3: tokens are map, all this stuff. We figured that out
Speaker 3: in advance, and we've found a format to pass that
Speaker 3: into the LM heavily preprocessed. You know, when all we're
Speaker 3: saying is here's baseline code that's almost done. It's just ugly.
Speaker 3: It's poorly named, it's poorly structured, and its might have
Speaker 3: some semantic misses, like the accessibility might not be quite
Speaker 3: right because we programmatically generated this with old school code
Speaker 3: and old school models, which are fast and reliable and
Speaker 3: best to use when you can. But all you want
Speaker 3: to do is do some clean up of the code,
Speaker 3: refactor this, name the components better, give it some props,
Speaker 3: some better class names, and then anything that's meaningfully wrong,
Speaker 3: such as the way we converted the horizontal. Now maybe
Speaker 3: our default logic turns it just vertical, because that's kind
Speaker 3: of a rule that works pretty well. Responsive design stuff
Speaker 3: side by side becomes vertical on the you know, narrow
Speaker 3: screen when that clearly didn't work well and you actually
Speaker 3: wanted a HAMBURGERM menu. Ho, the LM take a pass
Speaker 3: at that. Maybe that's the area that's least accurate, but
Speaker 3: still pretty much there, and then you're not having the
Speaker 3: developer at the end of the day have to tweak everything.
Speaker 3: They may just have to make some small tweaks to
Speaker 3: that Hambergerman. That's a bit rambly.
Speaker 2: That's maybe an example.
Speaker 1: Yeah, I kind of want to back up a little
Speaker 1: bit one thing here that I'm just going to throw
Speaker 1: out there. We talked to Obie Fernandez on Ruby Rogues
Speaker 1: and he's got a book about, you know, building and
Speaker 1: working with the LMS, and you know it talks to
Speaker 1: APIs and you know it does He's he's building chat assistance, right,
Speaker 1: So we're kind of talking about a different problem at
Speaker 1: but I think a lot of the ideas are the same.
Speaker 1: Where he basically breaks it down and says, yeah, so
Speaker 1: you have a like a gethub gethub ai chatbot, right,
Speaker 1: and so it knows about it's APIs and it's special
Speaker 1: things that it can do. And then you might have
Speaker 1: some other bot that knows kind of the next level
Speaker 1: up that orchestrates things, right, So he he was advocating too,
Speaker 1: to like break things down into really really granular things
Speaker 1: and have it come together that way. Of course, his
Speaker 1: stack is kind of a stack of AIS and you
Speaker 1: kind of figure it out and set up context and
Speaker 1: things like that. But I imagine a lot of people
Speaker 1: and there are some comments to this effect on here too,
Speaker 1: and I think, I think this is kind of where
Speaker 1: I want to start with. What you've talked about is
Speaker 1: you know, Jack Harrington on Twitter said what are the
Speaker 1: wrong ways to integrate AI and your application? And what
Speaker 1: are some of the right ways, And you kind of
Speaker 1: got into, you know, breaking the problem up and things
Speaker 1: like that. But then, yeah, Charles G said, AI is
Speaker 1: only reliable for prototyping and search. It feels like vaporware
Speaker 1: for most other stuff. It's that use models, you know,
Speaker 1: gotten off the hype train, and I think some of
Speaker 1: the stuff is going to get better. But yeah, so
Speaker 1: so you you talked about these specific instances, but what
Speaker 1: what are the problems that it solves? Well, like before
Speaker 1: do I look at my stack and go okay, I
Speaker 1: want I wanted to hit here, and you know, maybe
Speaker 1: I'm going to hit you know, this and this and
Speaker 1: this and this and this and this, and I'll have
Speaker 1: different models or different l ms that I hit or whatever.
Speaker 1: But how do I know that this is a good fit?
Speaker 1: And then how do I start putting that in there?
Speaker 1: Because it does feel like some things are moving this
Speaker 1: way other things. Yeah, it's not there yet.
Speaker 2: But so before Steve answers, and I'll definitely let Steve answer,
Speaker 2: I just wanted to mention that one of the catalysts
Speaker 2: for this entire conversation is an excellent blog post that
Speaker 2: Steve ro vote and posted on the Builder blog, which
Speaker 2: is titled how to build AI Products that Don't Flop.
Speaker 2: So I think that in a lot of ways, what Steve,
Speaker 2: what I assume you'll be saying and and kind of
Speaker 2: also addresses a lot of the issues that were brought
Speaker 2: up the stuff that you actually wrote in that blog post.
Speaker 2: So first of all, I I you know, we will
Speaker 2: post that blog post here in the chat, so and
Speaker 2: I highly recommend for people to go and check that out.
Speaker 2: And and it also have to say that it also
Speaker 2: resonated a lot with me because of stuff that we
Speaker 2: are doing at the size sense, which is the company
Speaker 2: that I recently joined, which turns out has a very
Speaker 2: similar philosophy and how we're using AI in our own products. Uh,
Speaker 2: And maybe I'll touch on that after you kind of
Speaker 2: answer the questions.
Speaker 3: Yes, no, this is a great question, and so you know,
Speaker 3: at the end of the day, maybe it starts by
Speaker 3: covering like what are the AIS bad at and why,
Speaker 3: and that can help us distill down what they're good
Speaker 3: at as an alternative and why. So what they are
Speaker 3: bad at is the biggest problem that lms have is
Speaker 3: what people call hallucinations. I don't love that term, but
Speaker 3: it's the term people use where they just make things up,
Speaker 3: and they make it up confidently. And it's probably a
Speaker 3: result of how they're trained. They're just trained on lots
Speaker 3: of information of people saying you know, they're just saying things.
Speaker 3: You know, they're saying things as if they're.
Speaker 2: Factually kids say that. It's the same approach that I
Speaker 2: take with most everything.
Speaker 3: Just say what I think as if it's certain.
Speaker 2: I say what they think, and I say it very confidently, exactly.
Speaker 3: Yes, I know many people in my life I probably
Speaker 3: are in one as well, who will do that as well.
Speaker 3: Whatever you think, whatever kind of sounds right. I got
Speaker 3: complete confidence in it when's saying it out.
Speaker 1: I've never done that everything.
Speaker 3: So you think about it, what's the training data? It
Speaker 3: is an AI trained on emulating the training data and
Speaker 3: training data as humans saying things if they know everything.
Speaker 3: So the AI just says things as if it knows everything.
Speaker 3: In the however many billion parameters they use, they can't
Speaker 3: store all the information of the world. And it seems
Speaker 3: like two date, people have still not figured out a
Speaker 3: liable way of having LMSA. Sorry, I don't know the
Speaker 3: answer to that. They will just make up dates and
Speaker 3: times and answers and stuff like that. It's very annoying.
Speaker 1: Well, I just want to chime in here because this
Speaker 1: has always been a problem with AIS. Right is you
Speaker 1: have a certain probability of not getting a right answer.
Speaker 1: The difference is is that the answers we're looking for
Speaker 1: now are a fully written out email or code or
Speaker 1: things like that right where you can be mostly okay
Speaker 1: except for these couple of things, where in the past
Speaker 1: it was generally something like AI vision or something like that.
Speaker 1: And so if it if it didn't always identify the
Speaker 1: dog as a dog, people would just kind of you know,
Speaker 1: as long as it was generally accurate, it was useful.
Speaker 1: And now it's it's problematic because it's generally accurate, but
Speaker 1: that's not good.
Speaker 3: Enough exactly, or especially if you think about it. Yeah,
Speaker 3: a good example is we have like one of those
Speaker 3: apps that lets us know when an animal goes in
Speaker 3: front of the camera at home, so we think it's
Speaker 3: our dog and turns out it was just a shadow.
Speaker 3: But it's like, who cares? Like whatever, the AI is wrong?
Speaker 3: Who cares? When it's not a who cares is when
Speaker 3: it's an essential part of a product. The product has
Speaker 3: a flow from start to finish, and the LM is
Speaker 3: wrong five percent of the time, that's a huge problem,
Speaker 3: especially when you use lms maybe for multiple steps.
Speaker 2: Or if it's strong one hundred percent of the time,
Speaker 2: but to a five percent extent.
Speaker 3: Yeah, exactly exactly, that's a problem. Those add up if
Speaker 3: you think about everybody loves the idea of AI agent's
Speaker 3: like a complete multi step tasks and to end, well,
Speaker 3: those little errors compound to become big problems. And if
Speaker 3: you've never used something like autogpt, it derails and it's
Speaker 3: a big problem. It's our generator a mess of data
Speaker 3: has no clue it's been off the rails for the
Speaker 3: last hour. It just makes it worse. And that's a
Speaker 3: huge issue with with this kind of future we want.
Speaker 3: But there are solutions to this. So actually there's two
Speaker 3: solutions that we found to be extremely effective. One has
Speaker 3: to do with actually that blog post you mentioned, Dan,
Speaker 3: and one has to do with some like microagent techniques.
Speaker 3: We've been really investing behind both in an open source
Speaker 3: project we've recently open sourced and some work we're doing
Speaker 3: internally in the product. So solution one that works fantastic.
Speaker 3: Let me give an example. Builder has lots of docks.
Speaker 3: People don't know the answers to the questions, and it's
Speaker 3: tedious to try and come through all the docks to
Speaker 3: find your answers. No matter how we try and restructure
Speaker 3: the docks or surface the right information the right time,
Speaker 3: it's never good enough. And I know that when I
Speaker 3: use other people's products too.
Speaker 1: You're not the only ones exactly.
Speaker 3: It's everybody's problem. It's it's it's it's very difficult. And
Speaker 3: so we're like, okay, let's feed all of the information
Speaker 3: of our docks into you know, the context window for
Speaker 3: an ll M and that's the big change that's happened recently.
Speaker 3: The context one is Windows have gotten freaking massive, and
Speaker 3: that's huge. I've even seen papers on using large the
Speaker 3: large context windows is more effective than fine tunings. Rather
Speaker 3: than fine tuning with hundreds or thousands of examples, just
Speaker 3: fit ten examples in the large context and you'll outperform.
Speaker 3: But they didn't even mention, or maybe they did and
Speaker 3: I missed it.
Speaker 1: The biggest thing is just one of the things talked
Speaker 1: about in the beautiful too.
Speaker 3: Yeah, and so what's what's practical? Because assuming we're talking
Speaker 3: jobs script devs, we're talking practitioners. They want to use
Speaker 3: the AI. The practical benefit of that is you don't
Speaker 3: need to assemble those thousands informations. You don't have to
Speaker 3: make a separate fine tune model instance for every use case.
Speaker 3: You can assemble things on the fly or make changes
Speaker 3: in experiment at a faster rate. It's it's really opens
Speaker 3: up a lot of benefits. And so what didn't work well?
Speaker 3: So here's if you go to chat ChiPT and ask it,
Speaker 3: ask it a question like how do I add a
Speaker 3: user in builder do io via API? It'll tell you
Speaker 3: go to build a dio slash API, slash users send
Speaker 3: a post that API doesn't exist. There's not an API
Speaker 3: to add users to your account and builders through APIs,
Speaker 3: though it will tell you that it exists. That's a
Speaker 3: huge problem. So then if you then try and augments
Speaker 3: this with okay, well we're going to take chatchipt or
Speaker 3: anthropicalaud APIs and we're going to supply our API documentation
Speaker 3: into it. Maybe we'll do some fancy based on embeddings
Speaker 3: and semantic search to find the right docs to include
Speaker 3: in that context and send it. It still will it'll
Speaker 3: get better, but it still will tell you to add
Speaker 3: a user. Go to API v one slash users and
Speaker 3: so the thing that works extremely well. So what the
Speaker 3: elms are really good at in my experience is condensing
Speaker 3: information down. So if you tell it very clearly, here's
Speaker 3: a set of information. You are only allowed to answer
Speaker 3: questions using this information, and if that information has the answer,
Speaker 3: condense it down and provide the answer. If it does not,
Speaker 3: say I don't have an answer to this, it probably
Speaker 3: doesn't exist. That works wildly well in our experience. Whether
Speaker 3: that's here's a lot of code, simplify the code, whether
Speaker 3: that's here's a transcript, tell me the key points discussed,
Speaker 3: whether that's here's the API documentation. Answer questions about the APIs.
Speaker 3: As long as you firmly say you could only use
Speaker 3: this information and nothing else, it works. I mean almost
Speaker 3: one hundred percent, and I mean almost in terms of
Speaker 3: like LM still can surprise you from time to time,
Speaker 3: use a good one, So like don't use I hope
Speaker 3: peop aren't using GPT three or three point five anymore.
Speaker 3: For most things there's better options. But if you're using
Speaker 3: something like anthropoclout three point five's on it and you're
Speaker 3: giving it lots of information more than a needs, I'm
Speaker 3: telling it to reduce and only use that information. It
Speaker 3: works great. Another example was so we have this assistant
Speaker 3: in our docks. It answers your questions and it works
Speaker 3: a lot better. Another thing that started doing, though, is
Speaker 3: making up links, like oh I added to the instructions
Speaker 3: at one point, like include links as much as possible,
Speaker 3: because a lot of what you're doing, or a lot
Speaker 3: of what we hope AI can help with the navigating
Speaker 3: docs is just knowing what for my use case, what
Speaker 3: docs I need to read, and so I told it
Speaker 3: will the answer with links. Maybe just a way to
Speaker 3: quickly find the right links is a good, good example.
Speaker 3: But I started making up links. So what did I do?
Speaker 3: I added to the prompt. I took our site map
Speaker 3: essentially and some additional contexts on each link, and I
Speaker 3: fed into the prompt and said use links only these links,
Speaker 3: nothing else. And then fantastic. It always links to things
Speaker 3: that are relevant. The more you do that, the more
Speaker 3: you tell it to do nothing more than cadets information
Speaker 3: and use nothing more than information provided, and just add
Speaker 3: a copious amount of information. Because these context windows are massive,
Speaker 3: it can work phenomenally well. So that's a use case
Speaker 3: that can be great. Another example of that is how
Speaker 3: our AI designed to code works is we have old
Speaker 3: school code, and old school models generate fairly accurate code
Speaker 3: for a design. It's just verbose and ugly. It's just
Speaker 3: too much. It's one massive component of div soup and
Speaker 3: the classes are named like div one v two to
Speaker 3: three to four. If you take that and pass it
Speaker 3: to an LM and say, just reorganize this into multiple
Speaker 3: files and components, well named renamed classes according et cetera. Again,
Speaker 3: give it a model like the latest two point five
Speaker 3: son it or GPT four is pretty good, but song's
Speaker 3: really good. It's just a fantastic job. And so we've
Speaker 3: spent an enormous at the time in the past trying
Speaker 3: to take generated code and try and make up class names.
Speaker 3: How the heck you're going to do that. That's back
Speaker 3: into that bucket of meaning. We don't know meaning in
Speaker 3: any type of code or model before at LLM really
Speaker 3: for this type of use case. But if you can
Speaker 3: distill down that way, it works fantastically well. And then
Speaker 3: the last piece, the last piece we've learned here, and
Speaker 3: i'd love to touch on the agent piece because there's
Speaker 3: one other technique that's great. The last piece we've learned
Speaker 3: is the user interface matters a lot too, and there's
Speaker 3: a user interface pattern that is just almost always wrong,
Speaker 3: yet we see almost everyone jump to all the time,
Speaker 3: including us, which is you want to assume that the
Speaker 3: AI is never perfect, and especially because when we're talking
Speaker 3: about meaning and summarization and stuff like that, it will
Speaker 3: not get everything right the first time. Even if it's accurate,
Speaker 3: it may not be quite exactly what you want. So
Speaker 3: what you don't want is a UI like we used
Speaker 3: to have, which is like if I wanted to get
Speaker 3: builder to update my content in some way. It used
Speaker 3: to be like click a button, then you get a box,
Speaker 3: then you type in it and hit some of it,
Speaker 3: and the box goes away and it makes your update
Speaker 3: and it acts like you're done. Most likely you're not done.
Speaker 3: Most likely it got you closer, but you're not quite
Speaker 3: there at least some portion of the time we've actually
Speaker 3: been landing on. A chat interface is almost always the
Speaker 3: right interface for an LM, even if your use case
Speaker 3: is not chat. So if your use case is imperfect
Speaker 3: but designed to code, and here's code, you should have
Speaker 3: a chat interface next to it saying what would you
Speaker 3: like to change about the code? Oh, I forgot to mention,
Speaker 3: I'm using tailwinds update it. There's too many components or
Speaker 3: there's not enough components. You should be able to constantly iterate,
Speaker 3: and the chat format lets you maintain that context and
Speaker 3: never assume it has to be done the first time
Speaker 3: or what's coming soon for us is importing Figma designs
Speaker 3: and then like a Pigma prototype where you have like
Speaker 3: you know, you have this mockup of like clicking this
Speaker 3: launches this modal and does this thing and updates this data. Well,
Speaker 3: we're going to suck that in and make real but
Speaker 3: not every You know, sigma is not a spec it
Speaker 3: is a suggestion, right, and so we need to treat
Speaker 3: it that way. And it's a description, So we're going
Speaker 3: to treat it that way. We're going to make some
Speaker 3: assumptions and then when actually you want clicking this to
Speaker 3: do that, you should be able to say it and
Speaker 3: see that happen. So that's one big bucket of learnings.
Speaker 3: The other big bucket of learnings is if you want
Speaker 3: to use lllms on a loop, you know, you could
Speaker 3: describe an AI agents just an LLLM on a loop.
Speaker 3: Do a thing, analyze a thing, do the next thing,
Speaker 3: decide when you need to stop.
Speaker 1: Right. The only thing I would add is that whatever
Speaker 1: you're telling it gets added to the context window and
Speaker 1: things like that, so that it knows what you've already
Speaker 1: done and what you've already told it correct.
Speaker 3: And you could also add that it's taking actions at
Speaker 3: each step, so the prior thing feeds into the next thing,
Speaker 3: the next context, and it takes another action and it continuous.
Speaker 3: Those are I think people firmly understand how we will
Speaker 3: be able to do even more magn wild things of
Speaker 3: AI at some point, you know, like I mentioned, we
Speaker 3: envision a world where another example is like assign a
Speaker 3: Gerra ticket to Builder and have it just implement that thing,
Speaker 3: and then you take a look at that or the
Speaker 3: Slack example too. Here's our idea. Implement it because Builder
Speaker 3: has an API and could hook right up to your
Speaker 3: live app. You could have it. If it powers your
Speaker 3: homepage like it does for lots of customers like j
Speaker 3: Crew or Zapy or whatever, you could just tell it
Speaker 3: like hey, up to the homepage with this and they'll
Speaker 3: just go do it. Maybe it to be safe. Ruends
Speaker 3: it as a one percent test shows you in a
Speaker 3: day that the data is good, the ab test is
Speaker 3: winning some you know metric, and then you great scale
Speaker 3: it up. You know stuff like that. Awesome, Just be
Speaker 3: be my homide that helps me do TV stuff that
Speaker 3: I didn't feel like doing. The problem though with agents
Speaker 3: that that have to take a sequence of actions is
Speaker 3: how they derail. Those small errors compound, so you need
Speaker 3: some type of mechanism to bring them back and analyze
Speaker 3: are they off track or not. And so this is
Speaker 3: a technique we've been using. We call it like a
Speaker 3: microagent technique because it's about specialized and I think this
Speaker 3: is a topic that has to do with product development
Speaker 3: in general, which is, don't try and boil the ocean,
Speaker 3: like don't try and build Devin the world's do everything
Speaker 3: software engineer. Then suddenly you're solving everyone's problems simultaneously. It's
Speaker 3: an impossible task. It's not a good idea. Rather, if
Speaker 3: you're an agent for one specific type of thing, starting
Speaker 3: there and then building up over time through feedback, through iteration,
Speaker 3: through all that stuff. That's how I always believe in
Speaker 3: building products. It's work better for me. So we have
Speaker 3: this open source project called Microagent, where the technique we've
Speaker 3: we realized works really well is if on each step
Speaker 3: you can have something that's not AI. We're exploring if
Speaker 3: AI can be the step, but I haven't had good
Speaker 3: results yet. But something that's not AI can essentially test
Speaker 3: if that step was successful and if it was not,
Speaker 3: feed feedback in and then let the LM run again,
Speaker 3: so it has the feedback from the last iteration.
Speaker 2: So for example, if you said that you had the
Speaker 2: LM output explanations that were link heavy, and you said
Speaker 2: that you verified the links by limiting it to your
Speaker 2: site map, but Another thing you could do is obviously
Speaker 2: just look at the links at the response, test them
Speaker 2: out and if you get I don't know, five hundred
Speaker 2: or something, then you can feed that back and say okay,
Speaker 2: that link is broken. Don't provide that as an answer
Speaker 2: or something like that.
Speaker 3: Spot on. No, that's a perfect example, and that's what
Speaker 3: we've seen. So I'll give you a couple more use
Speaker 3: cases for this. So one how the microagent project works
Speaker 3: as a CLI and instead of going to like chat
Speaker 3: ept and saying giving me a give me some code
Speaker 3: that converts marked down to HTML, you instead well run
Speaker 3: the cli microagent and describe that convert marked on cod
Speaker 3: to htmail instead of just giving you code and it's
Speaker 3: your problem if it doesn't work. Instead, it generates a
Speaker 3: test first, and it'll have all these input output examples
Speaker 3: of marked down to htmail and then it'll say it
Speaker 3: doesn't look good. You can give feedback on the test,
Speaker 3: like this is wrong, or add more or less or whatever,
Speaker 3: or just say okay. Then it'll it'll right the test
Speaker 3: and then it'll write code, and every time it writes code,
Speaker 3: it'll run the test and any testing id feedback. It's
Speaker 3: a I TDD and it works really really well, especially
Speaker 3: for certain use cases that AI generally wasn't good at before,
Speaker 3: And so you get this sort of guarantee that with
Speaker 3: AI spitting out code, you don't have to any longer
Speaker 3: hope that that code works because it'll look good and
Speaker 3: it might work on one example but not others. The
Speaker 3: test will ensure it works on all the examples. And
Speaker 3: when I run microagent and have it generate big complex
Speaker 3: things for me, I rest easy that if it passed
Speaker 3: all the tests and the tests are pretty thorough that
Speaker 3: it generates it works, and then if we find any
Speaker 3: issue subsequently, we'll update the tests accordingly. So it actually
Speaker 3: is a much more like high confidence solution. One other
Speaker 3: example is Figma has this concept called component sets, where
Speaker 3: it's it's like how it represents a button with all
Speaker 3: kinds of variations, you know, like color primary. It's a
Speaker 3: design system feature exactly. It's how you have something similar
Speaker 3: to React opponents and Figma, so you can place the
Speaker 3: button and change the color from primary secondary, you know,
Speaker 3: aero state, true, you know whatever, it's cool. It's weird
Speaker 3: as hell how it works in Figma and Figma you
Speaker 3: actually design so if you have like three different props
Speaker 3: with three different options each. You have to make nine designs.
Speaker 3: You have to manually code up or design every possible combination.
Speaker 3: It does us weird, it's funky, but it's it's effective.
Speaker 3: So how you translate that to code. What we do
Speaker 3: is we generate baseline code for every single variant and
Speaker 3: then we tell the LM consolidate down this all down
Speaker 3: into one piece of code with props you know, and react.
Speaker 3: And what we do is to verify the LM did
Speaker 3: that correctly, we will run it through tests. So because
Speaker 3: we know what the end state of every combination of
Speaker 3: prop should look like, we then take what the LM
Speaker 3: provide it and test it against every end state, give
Speaker 3: it every combination of props and make sure that the
Speaker 3: essentially the snapshot is correct to the original spec and
Speaker 3: if it's not, we feed the feedback in and if
Speaker 3: we've actually found you can play with it. You can
Speaker 3: either feed this to a slow high quality model like
Speaker 3: before three point five sonnet with anthropic there was claud Opus,
Speaker 3: which it was not obvious. I've had way a better
Speaker 3: experience with anthropic models in general than open A models.
Speaker 3: As becoming more popular now and more discussed but it
Speaker 3: was a little bit of a hot take in the
Speaker 3: past or less common knowledge. But anyway, you can play
Speaker 3: with the knobs. You can either say we're going to
Speaker 3: do the big, expensive model and probably takes less iterations,
Speaker 3: or you can scale down to like we actually had
Speaker 3: good results with Anthropic Haiku their smallest model, and it
Speaker 3: would take four or five iterations, but that would run
Speaker 3: faster than the two and three on opus and you're
Speaker 3: still guaranteed an accurate result at the end. So if
Speaker 3: you have that automated check that can feed into the AI,
Speaker 3: it could work really well. And that's kind of my
Speaker 3: point again about hopefully this illustrates good examples of a
Speaker 3: I can't be the entirety the brain of your product,
Speaker 3: but you can isolate these different techniques and use it
Speaker 3: to accomplish things that just would have been kind of
Speaker 3: impossible previously.
Speaker 2: So to speak, related to what you just said, one
Speaker 2: of the most salient points that you made in the
Speaker 2: blog post, and it's kind of related to what you
Speaker 2: just said right now, is that in all cases, it's
Speaker 2: not about building your product to be Holy Eye centric
Speaker 2: with a thin layer around it. It's taking some existing
Speaker 2: service product that solves a real problem and then sprinkling
Speaker 2: in AI to make it better. And so it's the
Speaker 2: reverse of what the VCS might have liked, but it's
Speaker 2: much more grounded and down to earth. And like I said,
Speaker 2: where I'm seeing very similar thing. At Sizense, we kind
Speaker 2: of build dashboards and stuff for BI and so in
Speaker 2: the past it was like, you know, you basically need
Speaker 2: to either best case scenario or dragon drop do you
Speaker 2: build your dashboard or you might you know, do some
Speaker 2: coding to build your dashboard. Well it's now we're working on,
Speaker 2: like you said, the sort of a chat thing where
Speaker 2: you can describe the dashboard that you that you like
Speaker 2: and you get it. But at the end of the day,
Speaker 2: it uses all the components and know how in capabilities
Speaker 2: that we've already built for constructing dashboards. So it's so
Speaker 2: it's it's adding AI to existing infrastructure to make it
Speaker 2: better and to make it more approachable, I might say,
Speaker 2: and easier to configure and without you know, specialized knowledge.
Speaker 1: It what you're talking about, Dan, And it sounds like
Speaker 1: it's a different level of interactivity.
Speaker 2: Right Look at the end of the day, I you know,
Speaker 2: even like then you when Google came out, I say,
Speaker 2: what was it? It's twenty something years ago. I said,
Speaker 2: why aren't are you? Are you eyes like that? Instead
Speaker 2: of pulled down menus and stuff, just give me a
Speaker 2: text box where I can say what I want and
Speaker 2: the software does it. Well, we're finally kind of getting
Speaker 2: that in a way.
Speaker 1: I think I have a question related to this because
Speaker 1: you know, Dan says, we're finally starting to get to this,
Speaker 1: and it seems like, you know, we've talked about, Hey,
Speaker 1: we had GPT three, which was hmm, we had GPT
Speaker 1: three point five, which was okay, GPT four is pretty good.
Speaker 1: I mean, are we going to continue to see this
Speaker 1: kind of thing too, where you know, we we get
Speaker 1: more of the text boxer? I mean the demos and
Speaker 1: the demos kind of made me upset because you know,
Speaker 1: then you'd see people trying to do the demo where
Speaker 1: they were talking to GPT for and it wasn't working
Speaker 1: for them like the demo work. But you know, we're
Speaker 1: a lot closer. You know, how does this continue to advance?
Speaker 3: Yeah? I mean the assumption we're making is that it's
Speaker 3: hard to perfectly quantify this, but the malls will get
Speaker 3: something like ten better year reyear they'll get you know,
Speaker 3: some percent less hallucinations or weird hiccups or weird problems.
Speaker 3: If you have a ninety five percent success right, maybe
Speaker 3: next year you have a ninety six. Maybe next year
Speaker 3: of a ninety seven. And these things tend to slow down,
Speaker 3: though they also tend to be little s curves as well,
Speaker 3: Like I think one one not counter argument, but counterpoints
Speaker 3: to the idea of it slowing down completely is things
Speaker 3: like I don't know if you've used rock g RRO
Speaker 3: and I'm forgetting what spelling one is Elion mustling, but
Speaker 3: one is like a hardware company. A hardware company, I
Speaker 3: think it's jer Okay is super interesting because they can
Speaker 3: run Lama three at insane speeds. You type in their box,
Speaker 3: you just get tons of text quickly. And they were
Speaker 3: one of the first to realize at a full ability
Speaker 3: to commercialized level, you can make hardware optimized for LMS
Speaker 3: and it's not just like a small percent better, it's
Speaker 3: like ten times faster and cheaper, which is crazy. And
Speaker 3: so we are not currently using Rock because we need
Speaker 3: larger context windows and LAMA three and all the others
Speaker 3: don't support the huge context windows that like Opening Eye
Speaker 3: and anthropic have I'm hoping, I know, medicinal training LAMAI
Speaker 3: three four hundred B. I think it is. And so that,
Speaker 3: you know, maybe a magic there is a larger context window.
Speaker 3: I don't know that could be huge, but breakthrough innovations
Speaker 3: like that could accelerate things. One example of that is, like, hey,
Speaker 3: people love the idea of you know, as an example,
Speaker 3: because build has an API and SDK, it can dynamically
Speaker 3: render your reaction app and components. When you drag and
Speaker 3: drop or use AI to modify it, you can see
Speaker 3: it all in real time, so it feels like Figma,
Speaker 3: but it's actually your react or quick or view or
Speaker 3: whatever app. So that's cool. And then changes can turn
Speaker 3: the code or published in their live to your users,
Speaker 3: so you can get people who are not even developers
Speaker 3: making changes, pushing updates, marketing pages or whatever you want.
Speaker 3: But what people really love the idea of is well,
Speaker 3: why are we hard coding apps at all? Can't we
Speaker 3: make components and understand things about our users? And when
Speaker 3: I jump into you know, JCrew dot com, can it
Speaker 3: just dynamically produce an experience that's one to one, personalized
Speaker 3: and fitted for me. Now that's like Wow, that's cool.
Speaker 3: That's no, we're not even close to that. I've watched
Speaker 3: the code generate slowly enough times to say we're not
Speaker 3: even remotely there. But then when things like Groc come out,
Speaker 3: which is probably the first commercially successful commerciuccessfulse and like
Speaker 3: I see people using it, I'm using it myself from
Speaker 3: time to time. Breakthrough an LM performance that was a
Speaker 3: big step towards that. You know, you see that paper
Speaker 3: of like what is it one point five bit YadA
Speaker 3: YadA for LMS, and it's like, hey, you might take
Speaker 3: another le forward as well. I don't think Rock is
Speaker 3: using an architecture think used to something differently, so that
Speaker 3: might be a subsequent innovation to add on. Now it's like, okay,
Speaker 3: if we can do these in real time, maybe that
Speaker 3: is possible only for certain parts of your app. It's
Speaker 3: not you know, it's not like it's not like you
Speaker 3: deploy an empty repo and suddenly applications build themselves in
Speaker 3: front of users in real time. But like you can
Speaker 3: start small, Like we were working with this very large
Speaker 3: company on the use case where they want to just
Speaker 3: be able to type in a very common use case.
Speaker 3: They want to type in a query and fetch the
Speaker 3: data associated to whatever that is. So like, I just
Speaker 3: want to see this data, and it can pull the
Speaker 3: data and visualize it for you. And the way they're
Speaker 3: doing visualization is code generation. But just like any tool,
Speaker 3: this is very common, or they're using anthropic claude artifacts
Speaker 3: or tail draws make real, you always get just like funky,
Speaker 3: plain old raw HTML REACT. It's not like using your
Speaker 3: APIs your components, You're you're using tail whatever it's it's
Speaker 3: not using your it's not code. It's gonna go to production.
Speaker 3: It's all throw away code. But if you could instead
Speaker 3: be able to in real time, either offline or online,
Speaker 3: assemble what you have your pieces. Offline use case would
Speaker 3: be like you know, I'm just gonna import a page
Speaker 3: to Figma and then I'll hit publish and then I'll
Speaker 3: go online when I'm done editing. So that's ultimately the
Speaker 3: offline use case. The online use case would be showing
Speaker 3: me these things on demand. So this large company that
Speaker 3: wants to surface your data, wants to generate code effectively
Speaker 3: with your components, Well, what can do that? We already
Speaker 3: have these SDKs where you if they're aware of your components.
Speaker 3: They can dynamically render out in real time your stuff
Speaker 3: we're exploring, like, hey, yeah, you queer your stuff and
Speaker 3: you've it's aware of your just the components you want
Speaker 3: it to be for this use case charts, diagrams, pipe
Speaker 3: chart table, et cetera. I can just dynamically produce u
Speaker 3: eys based on your specific query in pretty real time
Speaker 3: using LAMA three on rock. That actually works pretty well.
Speaker 3: So that's sort of like your first like online on
Speaker 3: demands generation and that makes sense for loged users maybe
Speaker 3: paying a certain amount per month, you know, that could
Speaker 3: be justified by the cost of serving that. But if
Speaker 3: we keep making these innovations on cost and speed, et cetera,
Speaker 3: you could get to a world where online generation of
Speaker 3: parts or larger parts. I mean, let's take if you're
Speaker 3: Jay Crew, maybe you want to manually merchandise that hero.
Speaker 3: You want to promote this new product lines as the
Speaker 3: hero is this thing, but maybe down below whatever Amazon
Speaker 3: highly personalizes the products you see. What if you could
Speaker 3: just throw that at the LM and the whole UI
Speaker 3: is based on what you'd be interested in seeing. Those
Speaker 3: are kind of interesting. I can't remember if there was
Speaker 3: a question here or what I was answering. But I
Speaker 3: want to throw that in because I think it's interesting
Speaker 3: direction that we might get to.
Speaker 2: Maybe we'll also get to a world where it's not
Speaker 2: just what's their name? The only company actually making money
Speaker 2: off of AI is Uh, I'm blanking out Microsoft.
Speaker 1: I was gonna say Microsoft.
Speaker 2: No, who makes the hardware? Oh yeah, and VideA and Vidia,
Speaker 2: And VideA is the only company actually making money off
Speaker 2: of the off of the AI revolution.
Speaker 3: Their stock price definitely yeah.
Speaker 2: Yeah, yeah, because you raise money as a startup and
Speaker 2: then spend all that money paying in VideA.
Speaker 1: You know this this kind of gets into one of
Speaker 1: the other questions I had, and you mentioned this. The
Speaker 1: previous question that you were answering was you know, you know,
Speaker 1: how does this continue to advance? And I think you
Speaker 1: pretty well answered that. Beyond that, you kind of got
Speaker 1: into more of the arena of cost and speed right
Speaker 1: as opposed to capability, And of course, costs and speed
Speaker 1: kind of play into capability, right because if it has
Speaker 1: infinite costs, then I can't provide it to my customer
Speaker 1: unless they have an infinite bank account. And similarly, you know,
Speaker 1: if it's not fast enough, then again, you know, it
Speaker 1: lowers the utility. So one of the things that I've
Speaker 1: talked to some people who are beginning to adopt AI
Speaker 1: features into their stuff is yeah, up to a certain point,
Speaker 1: it you know, it's it's great, and then it gets expensive. Right,
Speaker 1: So so how do you start to how do you
Speaker 1: start to manage some of that.
Speaker 3: That's a great question. That's something we've looked a lot
Speaker 3: at in a few different kind of findings of ours.
Speaker 3: First one, I could tell you, just from a consumer
Speaker 3: point of view, I hate the idea of all these
Speaker 3: different applications trying to charge me another twenty dollars per
Speaker 3: month per user for their AI features. Hey, right, you
Speaker 3: know part of me.
Speaker 1: Is like you've solved my problem, thank you.
Speaker 3: Yeah, I don't want to do for it exactly. And
Speaker 3: it's like the part that kills me is and I
Speaker 3: know no product should ever do this. It just doesn't
Speaker 3: make sense. But I'm like, I know you charge me
Speaker 3: that amount because the LM is expensive. Can I just
Speaker 3: supply everyone my open AI or anthropic key and they
Speaker 3: can just build me based on usage and you can
Speaker 3: just make sure the feet that the way it works
Speaker 3: with your products really well. I don't imagine anybody doing that. Obviously,
Speaker 3: when we make open source AR projects, we have several Yes,
Speaker 3: you just supply your key and then it works that way.
Speaker 3: But yeah, those things really add up, and in a
Speaker 3: lot of cases, I think the companies are just trying
Speaker 3: to make sure that their bottom line is taken care of.
Speaker 3: They know if they don't charge twenty dollars a month,
Speaker 3: if you use this heavily, they might be underwater on you,
Speaker 3: and that's a big problem. And a SaaS company usually
Speaker 3: wants to have eighty percent margins, So if you're going
Speaker 3: to cost.
Speaker 2: Them and we're back to the we're back to the
Speaker 2: uber model, like raise a whole lot of money and
Speaker 2: then basically subsidize your users.
Speaker 3: That's how a co pilot started out. They were losing
Speaker 3: money on a per user basis, but aren't anymore, at
Speaker 3: least that's what was reported, And so yeah, I think
Speaker 3: we're definitely seeing that across the board. Also a lot
Speaker 3: of startups losing money on the training costs and all
Speaker 3: the other as well. You know from VC money, but
Speaker 3: you know, you could probably put a value on it
Speaker 3: doesn't even take that much math or mental exercise to
Speaker 3: put a value on various things. For instance, we just
Speaker 3: have our AI in our docs for free for anyone
Speaker 3: anyway can prompt and use it. In a lot of cases,
Speaker 3: you can one just ship it and see what it costs.
Speaker 3: Doesn't cost a ton, and two we actually know you know.
Speaker 3: Our platform is probably in the bucket of it's not
Speaker 3: trying to be like a fort seal, which is like
Speaker 3: it's one button and you never do anything. You never
Speaker 3: have to know anything. It's definitely there are things to
Speaker 3: learn and more power to get as you learn.
Speaker 2: Also, your model is different. I mean, at the end
Speaker 2: of the day, your model is is a customer every
Speaker 2: once in a while needs a I in order to
Speaker 2: translate their Figma designs into code. I assume they don't
Speaker 2: do that every day. Whereas firstyle is you're not even
Speaker 2: a paying customer yet. Here we'll show you how we
Speaker 2: take your whatever and use AI to turn it into whatever.
Speaker 2: It's a totally different model in terms in terms of
Speaker 2: the finances exactly.
Speaker 3: And that's where you need to make sure. You just
Speaker 3: have to work make sure that the value you're providing
Speaker 3: in the rate that the user needs AI to get
Speaker 3: that value is less than you're charging them and an
Speaker 3: ideal world. You know what we've done in a lot
Speaker 3: of cases is it's kind of back to that point
Speaker 3: of making sure you do as much as possible without
Speaker 3: the LM. It's going to be faster, cheaper, better for
Speaker 3: ninety nine percent of the flow between whatever input whatever
Speaker 3: output is needed. And so that's how we cut down
Speaker 3: on cost traumatically as well use our own trained models
Speaker 3: when we can. If you are in a world where
Speaker 3: you know, Charles, you were mentioning the example of like
Speaker 3: you might use multiple llms, and we've done that in
Speaker 3: some cases. Like there's various ways you can do this,
Speaker 3: but there's certain use cases where you might want an
Speaker 3: LM to plan the work first and then another LM
Speaker 3: to execute on the plan. That can be a really
Speaker 3: good one. It can be not expensive if you don't
Speaker 3: need to feed too much context in multiple times. The
Speaker 3: output of a plan is usually not a lot of tokens,
Speaker 3: but it can be. Let's say you needed a whole
Speaker 3: chain of lllms, or let's say you had a situation
Speaker 3: where LM one takes lots of context and provides a plan,
Speaker 3: and then you have different LMS executly each step with
Speaker 3: a plan. But they each need the large amount of context.
Speaker 3: Then it can start adding up and it's like, ah,
Speaker 3: you know, can we do that without LMS or can
Speaker 3: we do other things? In the design to code flow,
Speaker 3: there's a lot of different steps and for us, some
Speaker 3: of the steps that can't be done with code I
Speaker 3: mentioned we train our own models. Those are it's hard
Speaker 3: to underscore how drastically faster, cheap, and more reliable training
Speaker 3: your own models than an LLM if it's a fit
Speaker 3: use case, and the fitting use cases might be more
Speaker 3: out outside of the box than you might think. Like, Yes,
Speaker 3: one of our use cases is image detection. When you
Speaker 3: have a Pigma design, certain things that are like one
Speaker 3: hundred vectors should actually be one image when it gets
Speaker 3: to a web or app, and then the text around
Speaker 3: it should stay text and UIs and so that is
Speaker 3: a good sufficient use case for like an object detection
Speaker 3: model which uses a convolutional neural net very common. It's
Speaker 3: hot dog, not hot dog. It's identifying dog, but it's
Speaker 3: identifying an image from training data, which in our case,
Speaker 3: you can generate that training data from the web. Scrape
Speaker 3: web pages, see what are images? Screenshot it, give it
Speaker 3: the screenshot plus the bounding boxes where the images work to.
Speaker 3: Now you've got infinite training data. You can train your models.
Speaker 3: But when it comes to things like decision trees, I
Speaker 3: think are way more interesting than people realize. You can
Speaker 3: break a lot of problems down to decision trees, where
Speaker 3: you basically provide for a very specific problem. In our case,
Speaker 3: it can be things like here's a whole mess of
Speaker 3: layers and figma, which of these should be considered groups together,
Speaker 3: like you know, into a flex row or column. Things
Speaker 3: like that. You can actually generate decision trees and be
Speaker 3: able to figure out or rain a forest, or you
Speaker 3: can get you can start simple like conditions and code,
Speaker 3: upgrade to desisionery if you need to upgrade to a
Speaker 3: rain of forest, if you need to upgrade to a
Speaker 3: neural network, if you need to upgrade to an LM
Speaker 3: at that point, and you can play the cost knob
Speaker 3: as much as you want. And there's definitely cases of
Speaker 3: products I see sometimes online where I'm like, the economics
Speaker 3: are just not gonna work on that product. I know
Speaker 3: you're charging twenty month now, but it's not gonna last.
Speaker 3: You have to find one that will.
Speaker 2: So I have to ask, I mean, I assume that
Speaker 2: what is it? When did you found the builder io?
Speaker 3: Oh goodness, twenty nineteen. I think end of your twenty
Speaker 3: nineteen something like that.
Speaker 2: Oh, five years ago? Yeah, well four or five years ago.
Speaker 2: I assume that four or five years ago, a lot
Speaker 2: of the stuff that you're talking about today you didn't know.
Speaker 3: Correct.
Speaker 2: Yes, so a lot of what we're talking about today
Speaker 2: are things that you learned during the past four years,
Speaker 2: maybe even one or two years. How did you go
Speaker 2: about learning all this stuff?
Speaker 3: A great question?
Speaker 1: I do have an end the Internet into his brain model.
Speaker 3: I'm sorry, no, it's it's more of an agentic approach.
Speaker 3: I hate that word. By the way people keep saying agentic.
Speaker 3: It's like, I don't know if that's a real word.
Speaker 3: I hate it so much. It means agents, like it's
Speaker 3: like referring to like an approach, like an agent agentic.
Speaker 3: It just feels like a VC talk term. But whatever,
Speaker 3: it's the morning agents on.
Speaker 1: Some shades and use code words.
Speaker 2: I mean, because he AI was this academic thing a
Speaker 2: couple of just a few years ago, and all of
Speaker 2: a sudden, it feels like you need to be an
Speaker 2: expert and otherwise you're kind of potentially left behind. So
Speaker 2: it's a whole lot of stuff like that you need
Speaker 2: to learn very quickly.
Speaker 3: Well, so, in my opinion, you know a lot of
Speaker 3: people naturally say I want to get good with AI,
Speaker 3: so I need to learn how AI works. I need
Speaker 3: to get a book on machine learning. I need to
Speaker 3: take a course on machine learning, like how the neural
Speaker 3: networks are trained, or how transformers work under the hood,
Speaker 3: et cetera.
Speaker 1: I've done. It's hard.
Speaker 3: Hard. In my opinion, it's like trying to learn physics
Speaker 3: to answer questions about biology or psychology. It's not usually
Speaker 3: a good idea. It's not going to give you that
Speaker 3: high of ROI and so, and I'll give you an example.
Speaker 3: I majored in cognitive science in college, which is like
Speaker 3: the closest thing to an AI specific major that existed
Speaker 3: at Berkeley, and I dropped out up to two years
Speaker 3: because it was actually horribly impractical. It was a cool information,
Speaker 3: but it wasn't as practical as I just want to
Speaker 3: get hands on and build stuff. And still today, in
Speaker 3: my opinion, the way you'll learn more about how to
Speaker 3: build effective AI products with LLLMS, et cetera, is to
Speaker 3: build air practic with LMS and run into all the issues.
Speaker 3: I don't think the academics could tell me all the
Speaker 3: strengths and weaknesses for our type of customer, you case
Speaker 3: that an LLM. Willer once have if they haven't actually
Speaker 3: gone through it all. Now, if you look at like
Speaker 3: the research papers coming out, there actually are different categories
Speaker 3: of research papers. Some are actually useful, some are literally
Speaker 3: sort of like the analysis of the usage of these things,
Speaker 3: like the one I mentioned comparing fine tuning to context models.
Speaker 3: That's actually useful, but that's not necessarily big brain stuff.
Speaker 3: Even if they have some weird calculus equations. You can
Speaker 3: feed it into. You could upload a chat to BT
Speaker 3: and ask what you want to ask and it'll tell
Speaker 3: you what you're looking for from the paper, which is nice.
Speaker 3: But all the ways I have learned anything is just
Speaker 3: rapid iteration. So that applies to learning AI. That applies
Speaker 3: to building a company in general, building a product. That
Speaker 3: applies to learning how to market. I didn't know how
Speaker 3: to market anything. We made a ton of mistakes in
Speaker 3: the past. We still make mistakes now selling It's just
Speaker 3: I'll take it build and needed its first customer. What
Speaker 3: did I do? I asked around like how what is
Speaker 3: sales like? And people like I don't know. You go
Speaker 3: talk to people and see if they want to buy.
Speaker 3: So I just start talking to people seeing what they
Speaker 3: want to buy. And I made a lot of mistakes,
Speaker 3: and now I could tell you a thousand things not
Speaker 3: to do, a thousand things to do instead, just from
Speaker 3: trial and error. And that's where I think a lot
Speaker 3: of people, whether they're learning to program, build more complex software, build,
Speaker 3: you know, get users for their software, whatever it is.
Speaker 3: Two frequently people fail to just say I'm going to
Speaker 3: be bad at this, I'm gonna do it anyway, and
Speaker 3: I'm gonna do so many times that I'm going to
Speaker 3: learn infinite things not to do, which leaves a smaller
Speaker 3: and smaller window of what to do. And once I
Speaker 3: have that small window of what works and that long
Speaker 3: knowledge of what doesn't, at that point you could say
Speaker 3: I'm actually kind of good at that thing. And all
Speaker 3: it takes is doing And just like the agent's example,
Speaker 3: it just means lots of feedback. So doing something and
Speaker 3: getting no feedback, you're not gonna learn anything. But I
Speaker 3: think like a big learning of mine is like, hey,
Speaker 3: you know, we'll be like one hundred person company by
Speaker 3: the end of the year. I've never run one hundred
Speaker 3: percent company. I think the biggest team I managed previously
Speaker 3: this company is eight people. So it's like, okay, how
Speaker 3: do I know what to do?
Speaker 2: What I do?
Speaker 3: I get as much feedback as possible. People tell me
Speaker 3: what's wrong, Tell me's wrong? Fix it? You know that
Speaker 3: that loop works the product too. Get the product in
Speaker 3: people's hands, Let them tell you what's wrong. Fix try stuff.
Speaker 3: I think that abstract really well to so many things.
Speaker 2: Do you have dedicated AI people in your company?
Speaker 3: Yes, ish we're not large enough, but we really do
Speaker 3: have an AI team. Yeah, we do about three four
Speaker 3: five people. Some people dip in and out of it.
Speaker 3: But even then that team probably wouldn't self identify as
Speaker 3: AI people. They would just be like people who are
Speaker 3: working on AI here and getting really good at it
Speaker 3: quickly by working so hard at it.
Speaker 1: Yeah, I want to chime in and just back up
Speaker 1: half a step. And that is the way that you
Speaker 1: describe the way you learn this stuff. What I find
Speaker 1: is that's the way that the people who really note
Speaker 1: something have learned it, right. Yeah, And so if you're
Speaker 1: feeling like, oh, well, I don't want I don't know
Speaker 1: if I want to learn AI because it looks hard.
Speaker 1: It's just like, look, you know, I mean I learned
Speaker 1: web development by hitting my head on the wall a
Speaker 1: zillion times. Right, Oh it's broken. Oh it's broken again.
Speaker 1: It's broken again. It doesn't look right now, but it works. Okay,
Speaker 1: now it doesn't work, but it's you know, you know,
Speaker 1: and you just you just do it. And so I
Speaker 1: just want to encourage people, like, if you're looking at
Speaker 1: this and you're going, boy, you know it sounds like
Speaker 1: kind of a slog and kind of a journey, that's
Speaker 1: just the way it is, right. I will point out that,
Speaker 1: you know, I have a computer engineering degree, and so
Speaker 1: you know, I got into you know, writing code, and
Speaker 1: I still bang my head against the wall a whole
Speaker 1: bunch of times. I just had a little bit deeper
Speaker 1: foundation than maybe somebody else. But if I wanted to
Speaker 1: build the castle, all the all the all the head
Speaker 1: start I had was that I had a little bit
Speaker 1: deeper foundation. I didn't have any of the walls up,
Speaker 1: I didn't have the boat dug none of that stuff, right,
Speaker 1: And so you know, you as you're looking at how
Speaker 1: you break it down, I just I really want to
Speaker 1: encourage people to just just look at this. I'm working
Speaker 1: on putting together a boot camp that you know, through
Speaker 1: a lot of this stuff, do it in October, right,
Speaker 1: And that's the same thing, right is you know, I
Speaker 1: kind of want to be there. So when you run
Speaker 1: into the wall, you know, you're not stuck trying to
Speaker 1: figure out how to get off the wall and go
Speaker 1: do something. You know, do the next step. Right, you
Speaker 1: kind of get there faster, but you're still going to
Speaker 1: run into this even if you have somebody holding your hand.
Speaker 1: So yeah, just just be aware that that that's kind
Speaker 1: of the way that a lot of this goes. I
Speaker 1: kind of want to pivot this. You were talking about,
Speaker 1: you know, getting left behind right on some of this stuff,
Speaker 1: or you know, it moves ahead quickly. And yeah, one
Speaker 1: of the questions that I get from people is, Okay,
Speaker 1: well do I have to learn it? And then the
Speaker 1: other question I get is because you're talking about specifically.
Speaker 1: So when we talked to Obi Fernandez on Ruby Rogues,
Speaker 1: he was talking about the AI systems that replace you know,
Speaker 1: services like copywriting and add optimization and things like that, right,
Speaker 1: you know, aspects of running your business, right, and so
Speaker 1: you know, programmers, you know, would listen to that and
Speaker 1: not feel super threatened, right because they're gonna be in
Speaker 1: a place where they're going You know, that's not something
Speaker 1: I do anyway. But you're talking specifically about, Hey, I'm
Speaker 1: going to take this figma and I'm going to have
Speaker 1: working code at the end, right, And yes, you've kind
Speaker 1: of let us know that there are various stages of
Speaker 1: effectiveness to this. But is it eventually going to get
Speaker 1: there where if I'm writing react or quick or something
Speaker 1: else that you know, my job is going to be
Speaker 1: you better write a really good prompt for the system
Speaker 1: so that it'll give you the right code. And is
Speaker 1: it going to make it harder for people to get in?
Speaker 1: You know, is it going to cut salaries because the
Speaker 1: AI does a bunch of my work? I mean, how
Speaker 1: how vulnerable are we to this stuff?
Speaker 3: Great question? Let me give you an example. So the
Speaker 3: short answer is, I don't think developers have anything to
Speaker 3: worry about. I think they only things to be excited about.
Speaker 3: And I mean that genuinely one of them.
Speaker 1: I like that. That's a really good way to put it,
Speaker 1: because that's what I keep trying to tell people.
Speaker 3: Yeah, it's it's leg about it this way too. I'll
Speaker 3: make a statement and give an example. These tools are
Speaker 3: gonna give you amazing, amazing superpowers. Companies want more people
Speaker 3: with superpowers than less. They don't want another example, and
Speaker 3: this is this is a misleading example. I'll explain why
Speaker 3: its misleading.
Speaker 2: Come back to it.
Speaker 3: Let's say AI only work for front end tools. Would
Speaker 3: you rather have more superpowered front end developers because of
Speaker 3: these tools that make them superpower or more I'll just
Speaker 3: say back end developers that didn't have superpowers. I want
Speaker 3: more the superpower people. Now, realistically, there's different tools for
Speaker 3: the front end superpowers in the back end superpowers. You're
Speaker 3: gonna want both. If you're a developer and you're not
Speaker 3: playing with AI tools to see how it can help
Speaker 3: your workflow, I don't mean you have to be exhaustive.
Speaker 3: I mean, like you know, if you want to start
Speaker 3: with the basics, use chat Gypto. I personally recommend Claude instead.
Speaker 3: Cost the same and it does better, especially with code.
Speaker 3: And if you're not using get a copilot or whatever
Speaker 3: most synonymous thing you have for your ide I would
Speaker 3: highly suggest you do those things because most importantly what
Speaker 3: they do is they build an intuition of what's AI
Speaker 3: good at and not and when it's out of suggesting.
Speaker 3: There are certain things that I just know that AIS
Speaker 3: can do a good job on every time, and I
Speaker 3: have that intuition through repetition of just seeing it happen
Speaker 3: in real time with no effort for me, and I
Speaker 3: know things it will not do good at, and those
Speaker 3: things build a mental model. It's like training your brain,
Speaker 3: your neural network to know what the AI is good
Speaker 3: at and not. So when you're building products with it
Speaker 3: or when you're trying to find productivity games for yourself,
Speaker 3: you have that intuition and where it's going to work
Speaker 3: and not. I see people every day with the wrong intuition, like, oh,
Speaker 3: can AI just do everything for me? It's like you
Speaker 3: have you used AI tools? No, No, it's not going
Speaker 3: to do that. That's going to go badly. So if
Speaker 3: you think about like how it affects jobs, let me
Speaker 3: give you another example. Builder as a tool has always
Speaker 3: been a tool that takes the code and components you
Speaker 3: have and brings and takes work that you have as
Speaker 3: developer that generally you'll find tedious. And I think if
Speaker 3: you're thinking calmly and dispassionately. It's the tedious work you
Speaker 3: don't want to do, like marketing, wanting to move buttons
Speaker 3: around the homepage or change the color of this to
Speaker 3: see you know, and realize the test failed to they
Speaker 3: undo it. It doesn't feel good to be a developer
Speaker 3: doing that. You don't want to be like a middle
Speaker 3: person in between, like say marketing and your homepage. That sucks. Objectively,
Speaker 3: it sucks, and so with builders. Sometimes people before adopting
Speaker 3: the product, they have a concern like, wait, is this
Speaker 3: going to take my job away? Is this? Do I
Speaker 3: have to be worried about this? I would say only
Speaker 3: one to five percent of people have that concern that
Speaker 3: CanCERN happens. I've heard it zero percent from real customers,
Speaker 3: and I keep really close to our customers. Nobody's ever
Speaker 3: adopted the product that makes their development more efficient and
Speaker 3: actually worried about their job security Afterwards. When businesses can
Speaker 3: do more, they just want more. Their appetite grows faster.
Speaker 2: It kind of reminds me of you know, when people said,
Speaker 2: like I'm trying to remember who's this stand up said
Speaker 2: it that if you're worried about illegal immigrants taking your job,
Speaker 2: job and you've got the wrong job. Uh, it's it's
Speaker 2: it's you don't want to work in the job, in
Speaker 2: in those kind of jobs anyway. But is what I'm saying.
Speaker 2: So what I'm understanding from you is that it's that
Speaker 2: superpower to uh, to do away with a lot of
Speaker 2: the repetitive and let's let's call it less thoughtful parts
Speaker 2: of the job. Uh it's is that kind of what
Speaker 2: you're saying.
Speaker 3: Yeah, I can, I can ahad.
Speaker 1: Yeah, I was just gonna say, you know, I did
Speaker 1: a couple of job interviews, and I've been using get
Speaker 1: hoo co pilot for a while and and one thing
Speaker 1: that it does for me is right, occasionally it'll try
Speaker 1: and fill in the whole class in Ruby or the
Speaker 1: whole you know, component in JavaScript, but most of the
Speaker 1: time it's just filling in the part that's sort of
Speaker 1: an atomic piece of the thing. And what it does
Speaker 1: for me that kind of makes my development better is
Speaker 1: that it gives me enough of the pieces that you know,
Speaker 1: from my experience, I can look at it and say
Speaker 1: that's what I want or that's close, right, and so
Speaker 1: from there then you know, maybe I hit tab and
Speaker 1: I have it drop it in and then I go
Speaker 1: and I modify it to be what I need, and
Speaker 1: you know, sometimes it's seventy five percent, sometimes it's one
Speaker 1: hundred percent, and sometimes it's no, that's just not what
Speaker 1: I want. Right, you don't understand what I'm doing, But yeah,
Speaker 1: it totally opens those doors. And the other thing that
Speaker 1: I want to jump on here is you know you
Speaker 1: mentioned that, Yeah, if you have superpowered engineers, people are
Speaker 1: thinking that these companies are just going to cut their
Speaker 1: costs down to the bone and just have bare bones
Speaker 1: development AI generated code. That's generally not how companies work. Generally.
Speaker 1: The way that they is they're going to spend what
Speaker 1: they spend to move forward with all the things that
Speaker 1: they think they want to provide to their customers. And
Speaker 1: so you're just going to be able to provide more
Speaker 1: of that.
Speaker 3: Correct, and you'd be I recommend people don't underestimate how
Speaker 3: much when capacity explodes, ideas and needs and desires explode.
Speaker 3: And I think it's not just proportionate, it's above proportionate.
Speaker 3: And suddenly, when you think you're introduced can do magic,
Speaker 3: you want to do more than magic, and you're excited
Speaker 3: because remember, companies live in a competitive environment. So I
Speaker 3: guess I could put the CEO head on for a
Speaker 3: minute and say, what am I paranoid against competitors? I
Speaker 3: paranoid that we're doing more faster than them, which means
Speaker 3: I want more superpowered aidevs who are more accelerated by
Speaker 3: the AI tools than my competitors, so we leave them
Speaker 3: in the dust. I'm not thinking, oh, we can do more,
Speaker 3: cut people. I'm saying I'm looking at we can do more,
Speaker 3: get ahead of the competition, maximize the resources we have,
Speaker 3: even hire more because now the ROI makes more sense.
Speaker 3: I can hire another developer and pay them at developers
Speaker 3: out and get way more ROI than I did for
Speaker 3: that money. So it's an obvious investment that kind of
Speaker 3: has happened. As we got more productive, we started hiring
Speaker 3: more of these developers. And the way I would suggest
Speaker 3: thinking about it is here's an example. Has anybody ever
Speaker 3: hand coded a PDF? Heck no, You go to Chrome
Speaker 3: and you say exports pdf and it generates the PDF
Speaker 3: for you. In my opinion, figma designs are similar. If
Speaker 3: you get to a point where you can generate it
Speaker 3: and then you can work with it from there, you're
Speaker 3: not gonna go back to hand doing it, it's just
Speaker 3: a waste of time. So the way you can think
Speaker 3: about your work is and I won't solve everything. You
Speaker 3: have a massive, multimillion line of code base, you have
Speaker 3: a very esoteric bug. You're gonna have to look into
Speaker 3: that and you're gonna have to actually dive into it.
Speaker 3: A agents are so far from handling things like that.
Speaker 3: But when you're producing and getting ideas out and stuff
Speaker 3: like that, the drafting iteration phages can move so much
Speaker 3: faster if you can just suck in this dashboard and
Speaker 3: you see it. But you're like, I want to move this,
Speaker 3: I want to reconnect this. I'm gonna get on the
Speaker 3: code for this. I'm gonna get on the prompt for this.
Speaker 3: You become like this orchestrator. And I think, you know,
Speaker 3: a lot of engineer don't like to step into a
Speaker 3: lot of engineers might like the idea of management because
Speaker 3: they like this idea of like parallel execution. You know,
Speaker 3: like you have people working under you, and so by
Speaker 3: you can be in charge of a project and you
Speaker 3: have ten engineers working on it, you can do theoretically
Speaker 3: ten times as much as if you were just the
Speaker 3: only one working on it. Cool but you realize people
Speaker 3: are difficult. You spend your work not solving cool problems.
Speaker 3: You spend some the personnel issues. This person's like that person,
Speaker 3: This person is not convinced of the roadmap. And all
Speaker 3: this stuff happens and people realize, wait, I like building stuff.
Speaker 3: This is building stuff, and I get out of management.
Speaker 3: The best part of I could describe using AI effectively.
Speaker 3: It's like all the best parts of management. You have
Speaker 3: these minions or these tools generating things for you, but
Speaker 3: without the downsides. They don't have any types of emotions
Speaker 3: about anything.
Speaker 4: You can work so far, so far, I don't think
Speaker 4: that's going to change, because why would we program that
Speaker 4: or at least go back to the older AI that
Speaker 4: doesn't have that problem go personality.
Speaker 2: Yeah, well, you know.
Speaker 1: The other thing is you're talking about these iteration cycles
Speaker 1: that you know that it speeds up. And I think
Speaker 1: one thing that a lot of people don't understand is
Speaker 1: that those iteration cycles happen now right, you know, And
Speaker 1: and if you've been on a project long enough, what
Speaker 1: typically happens is you'll get to a point where either
Speaker 1: you've accrued so much technical debt that it's impossible to
Speaker 1: work on or what what will happen is you'll you'll
Speaker 1: kind of get a couple of pieces in place, and
Speaker 1: then all of a sudden, everything else gets easier, right,
Speaker 1: and so then the sky's the limit and and what
Speaker 1: the what you're saying is is a lot of the
Speaker 1: foundational pieces and a lot of the foundational thinking. You
Speaker 1: know that that's mostly just you just have to grind
Speaker 1: through it until you're done with it. That stuff goes away.
Speaker 1: And so then all of a sudden, it's we have
Speaker 1: these greater capabilities, these greater opportunities, and so we can
Speaker 1: instead of the grind taking two months, the grind takes
Speaker 1: two weeks because we get that much further ahead, and
Speaker 1: then we can modify what we got from the AI
Speaker 1: or work with the AI for the two weeks to
Speaker 1: get where we needed to go, and then we can
Speaker 1: turn around, and that iteration cycle happens faster exactly.
Speaker 3: You can think of it like, also, you're no longer
Speaker 3: the fluidist in the band, you're the entire orchestra. You're
Speaker 3: the conductor. Yeah, you can make these happen.
Speaker 2: But here's the thing that I wanted. I know where
Speaker 2: we're running along on time and we'll probably finish soon.
Speaker 2: But there's this one question that I really need to
Speaker 2: get off my chest, as it were. So you've been
Speaker 2: describing and I totally agree with the fact that you know,
Speaker 2: we need to learn how to we need to learn
Speaker 2: these to use these AI tools. Obviously we learned need
Speaker 2: to learn how to best use these AI tools, and
Speaker 2: these AI tools will make us more productive, and I
Speaker 2: totally agree with that, But it's still it. There's still
Speaker 2: the question of how knowledgeable do I need to be
Speaker 2: at AI? Because there's a difference between being able to,
Speaker 2: I don't know, drive a car and be the person
Speaker 2: who can pop the hood and start fiddling with the
Speaker 2: stuff there. So how knowledgeable do I need to be
Speaker 2: about AI in order to be effective as a developer,
Speaker 2: to be desirable as a developer in the upcoming one, two, five,
Speaker 2: ten years.
Speaker 3: Do you think, Yeah, it's a great question. The A
Speaker 3: couple quick thoughts on this. The first one is I
Speaker 3: personally are like The high level one is I personally
Speaker 3: think people just need to know how to use AI
Speaker 3: to be more productive for their own work. In doing that,
Speaker 3: you'll actually learn a lot about the underlying pieces, so
Speaker 3: to speak, and how the technology is suitable for certain
Speaker 3: use cases versus not. A simple example there is like
Speaker 3: you might be writing docs in markdown and you see
Speaker 3: how well at auto complete certain docks. Let's say you
Speaker 3: make a writing products well by learning how copi has
Speaker 3: worked well or not, or Tatchubet's work well or not.
Speaker 3: And now suddenly you work on some type of writing product,
Speaker 3: a Google Docs thing, and the company wants you to
Speaker 3: build some type of AI feature. Well, you already have
Speaker 3: some baseline knowledge and intuition built around this. So the
Speaker 3: two categories probably are one. Using AI tools to be
Speaker 3: a more effective developer. You should always try and be
Speaker 3: more effective because if you're not, your peers are, And
Speaker 3: in a certain sense, you don't want to be just
Speaker 3: so behind the train. Everybody's at a whole new level
Speaker 3: than you because you had just hopped on the train.
Speaker 3: You don't want that to happen. You want to be
Speaker 3: using the best tools, the best ability to be as
Speaker 3: good as your peers, be one of the best of
Speaker 3: your peers. On the other side, there's building AI products
Speaker 3: that is shockingly similar. And if you really want to
Speaker 3: be if you think in five years every product will
Speaker 3: be an AI product, I think that's an extreme thinking,
Speaker 3: but you could think of it differently, as more products
Speaker 3: will have more AI features, and if my company might
Speaker 3: want me to work on it, it's nice to know
Speaker 3: how to work on it. Again, I don't think the
Speaker 3: way to do that is to go read a bunch
Speaker 3: of white papers, build your neural network from scratch, understand
Speaker 3: how transformer works into the hood. Again, I think that's
Speaker 3: like trying to solve psychology problems with physics and a
Speaker 3: knowledge of F equals M. It's not going to get
Speaker 3: you that far. Solve psychology problems by learning about psychology
Speaker 3: and practicing psychology in some form, studying it directly and
Speaker 3: learning that way. And I think that the beautiful thing
Speaker 3: about AI is it can solve your own problems. You know,
Speaker 3: we talk about like the types of work dev don't
Speaker 3: want to do. You have this really exciting opportunity. Let's
Speaker 3: say I think we've talked about code debt, like, could
Speaker 3: you make an AI workflow to help you with managing
Speaker 3: code debt, refactoring, cleaning, as cetther code. That's a problem
Speaker 3: that you right now, as an engineer, might say I
Speaker 3: hate dealing with this. I can play around with AI
Speaker 3: to try and solve my own problem. Engineers love over
Speaker 3: automating things. You know, let's spend a week automating something
Speaker 3: that could have taken a few hours of front work. Everybody,
Speaker 3: everybody's guilty of it. If you're probably any good, you
Speaker 3: probably this is how you think.
Speaker 2: You know, you could probably use an AI tool to
Speaker 2: help you build your AI tool exactly.
Speaker 3: Make your own projects. Get up Copilot, microagent, CHATTPT, whatever
Speaker 3: can help you build it. And that's a good way
Speaker 3: to learn the stuff too. So I always think that
Speaker 3: side projects are awesome. You can learn your day to
Speaker 3: day using AI tools, try new ones, and again, I
Speaker 3: don't think you have to make an exhaustive list. Everybody
Speaker 3: keeps saying Cursor AI is great. I don't care that much.
Speaker 3: I love to the features again today it's the same
Speaker 3: features give Copilot with some nuances. So no, I don't
Speaker 3: think you need to waste tons of time adopting every tool.
Speaker 3: There is a learning curve, but just adopt the basics
Speaker 3: and use your own time to do your own side
Speaker 3: products sometimes to use AI to solve your own problems.
Speaker 3: I think that'll make you equiped to be a very
Speaker 3: future proof, very capable developer. One years, five years, ten
Speaker 3: years from now.
Speaker 2: Very interesting in this context is I think it will
Speaker 2: see how far it goes. But the fact that Google,
Speaker 2: for example, is actually building their own micro model into
Speaker 2: Chrome itself, that will give an interesting opportunity for people
Speaker 2: to play with the technology.
Speaker 3: It is exciting, it's fast too, and that opens up
Speaker 3: new worlds too. This is also why you should probably
Speaker 3: get your hands on the stuff and try and solve
Speaker 3: real problems. Is you start realizing why that matters for us.
Speaker 3: One of the most common things that comes up is
Speaker 3: giant Enterprise bank wants to use our AI features, but
Speaker 3: they can't send their coat over the wire to back end.
Speaker 3: So that's why you start exploring local models, like let's
Speaker 3: make the AI work with Olama, run it all locally
Speaker 3: on your machine. Honestly, if weren't building a products, I
Speaker 3: wouldn't know why people care that much about that. Sure,
Speaker 3: cost is a thing, and so we start exploring it,
Speaker 3: and we tend to find that the local models aren't
Speaker 3: powerful enough yet to solve these use cases without computer.
Speaker 2: So they can be just too big to download exactly.
Speaker 3: That's what I found that the models you need to
Speaker 3: be effective with our product right now they're too big
Speaker 3: to download and too big to run on a local computer.
Speaker 3: The running is the hard pore. You just run out
Speaker 3: a RAM. But again, when you're kind of immersing the ecosystem,
Speaker 3: the other stuff that's happening makes more sense. People care
Speaker 3: about this. Does it work yet? Window dot ai will
Speaker 3: not replace chat GBT. It's gonna be a much smaller model,
Speaker 3: which are a lot dumber, But for certain use cases
Speaker 3: it's good enough. And that's where getting your hands on
Speaker 3: is the best way to know those types of things.
Speaker 1: Yep, right, good deal. Well, I'm gonna kind of wrap
Speaker 1: us up. I recommend you go check out the article
Speaker 1: that Steve wrote. There's also a video I saw that
Speaker 1: has the same name, and I don't know if you
Speaker 1: put that out or if it was somebody else.
Speaker 3: The same thing, the same thing that's in the blog
Speaker 3: is basically covering the video and vice versa.
Speaker 1: Yeah, but I highly recommend that. I mentioned the Obi
Speaker 1: Fernandez episode on Ruby Rogues. He actually has a book
Speaker 1: that he wrote on how to use models like this.
Speaker 1: I think some of the code samples are in Ruby,
Speaker 1: but for the most part, it's kind of language agnostic.
Speaker 1: It's just hey, you know, if you understand these APIs
Speaker 1: and capabilities, then here's what you can build. And yeah,
Speaker 1: I mean I think, I think think this really is
Speaker 1: going to continue to change the way that we're working.
Speaker 1: And so you know, the more you can get into
Speaker 1: it ahead of the curve, I think, the better off
Speaker 1: you're going to be. So thanks for coming, Steve, Thanks
Speaker 1: for having Yeah, we're going to do our picks. Then
Speaker 1: we're going to wrap up. Dan, do you have some
Speaker 1: picks for us?
Speaker 2: Yeah, it's not exactly a pick. So I've not been
Speaker 2: speaking about the current situation in Israel and in Gaza
Speaker 2: in the recent months, maybe because it was just too
Speaker 2: painful for me to just think about all the time.
Speaker 2: But this week, actually a person is going to come
Speaker 2: to our company to talk about the fact that his
Speaker 2: son is kidnapped in Gaza. People tend to forget that
Speaker 2: there are still one hundred and twenty Israelis kidnapped in Gaza,
Speaker 2: including one child and one baby. And it's a situation
Speaker 2: that I hoped would have been longer and still hasn't been,
Speaker 2: and hopefully it will be. What can I say, It's
Speaker 2: going to be very, very difficult to even look this
Speaker 2: person in the eye let's put it this way. It's
Speaker 2: it's I'm it's it's it's just such a statues situation. Anyway.
Speaker 2: Sorry for bringing in such a bummer, but that's the
Speaker 2: only thing that I really had that I wanted to mention.
Speaker 2: Uh So, so yeah, hopefully you can pick up the mood.
Speaker 1: Yeah, hopefully I can. But yeah, it is sad, and
Speaker 1: you know, I think sometimes we lose track of these
Speaker 1: things after a certain amount of time when they're not
Speaker 1: in the forefront of you know, what we're listening to
Speaker 1: or watching or things like that. So yeah, keep in
Speaker 1: mind that these are people that we should be thinking about,
Speaker 1: praying for, and and looking for solutions here, and encourage
Speaker 1: our leaders, whether it's in you know, Congress or the
Speaker 1: president or who ever, to help figure some of this out. Yeah,
Speaker 1: We've had some stuff going on in this country too,
Speaker 1: but I'm not going to go into it.
Speaker 2: I you know, spoke a little bit about it before
Speaker 2: the show.
Speaker 1: Yeah, but be kind to your neighbors, right, I think
Speaker 1: a lot of this just comes down to the way
Speaker 1: we demonize each other and we don't need to do that.
Speaker 1: I'm going to jump in and do a game pick.
Speaker 1: I'm going to pick a game that I've picked in
Speaker 1: the past. I just didn't get together with my buddies
Speaker 1: this week to play a board game. And this is
Speaker 1: one my So my nephew's here from Illinois and his
Speaker 1: his parents are going and doing job interviews in Wyoming,
Speaker 1: and so they dropped him off here in Utah instead
Speaker 1: of dragging him around Wyoming and having him be bored
Speaker 1: with them. He can play with my kids. I say,
Speaker 1: play with my kids like he's five, he's seventeen. But
Speaker 1: the game I'm gonna pick is Mysterium. I always do
Speaker 1: a board game kicks pick Steve at the beginning of
Speaker 1: my picks. And so Mysterium is a game where you
Speaker 1: have one person giving clues and then you have other
Speaker 1: people trying to figure out what the clues mean. And
Speaker 1: so you have all the players but one are psychics.
Speaker 1: And then the person giving the clues is a ghost.
Speaker 1: And there have been murders in this house, and so
Speaker 1: there they hand cards to the psychics, and the psychics
Speaker 1: try and determine, you know, who the person, the place,
Speaker 1: or the murder weapon is. And then there are expansions
Speaker 1: where instead of doing a murder weapon, you do a motive.
Speaker 1: But it's it's a pretty fun game. Takes an hour
Speaker 1: or so to play. Sometimes it's really hard because you know,
Speaker 1: you've got all kinds of interesting things on this card.
Speaker 1: It's just this, you know, this picture, and so anyway, Yeah,
Speaker 1: you go through the rounds. You pick the first person first,
Speaker 1: you guess the place next, and then the object, and
Speaker 1: if you get all three and everybody gets if all
Speaker 1: the psychics solve their murder, then the ghost has one
Speaker 1: final round where they give three cards and depending on
Speaker 1: how well you were doing it guessing whether or not
Speaker 1: the other psychics were correct or not, and how early
Speaker 1: you got yours done, you get to see one, two,
Speaker 1: or three of those cards, and those are hints toward
Speaker 1: one of the murders that the psychics solved, and that's
Speaker 1: the ghost, the person who's beginning to concluse that's their murder.
Speaker 1: And you know, it's kind of a majority wins thing
Speaker 1: on that one. It has a board game geek weight
Speaker 1: of one point nine. I keep telling people that kind
Speaker 1: of the average you know, friendly board game that's approachable
Speaker 1: to people who don't play board games is about a two, right,
Speaker 1: and so this is right in there in that area.
Speaker 1: It's kind of a fun social game, and I really
Speaker 1: enjoy it. The kids were playing it. They helped my
Speaker 1: eight year old play it. I don't know if an
Speaker 1: eight year old could pick up on all the nuances
Speaker 1: of it without help, but they just helped to play.
Speaker 1: And so anyway, fun game. Like I said, there are
Speaker 1: expansions for it. It came out twenty fifteen, so I'm
Speaker 1: gonna pick mysterium. I was talking to Dan before Steve
Speaker 1: got on. He asked me how I was, and I
Speaker 1: was like, tired because I had just gone for a run.
Speaker 1: I'm training for another marathon, and so I'm going to
Speaker 1: just do a couple of shout outs on that. I
Speaker 1: have been doing the trainings off of a Training Peaks
Speaker 1: training that I bought. So Training Peaks is free, and
Speaker 1: you can buy training that you can stick on your
Speaker 1: calendar and those are just fixed costs. So I think
Speaker 1: I bought this one for twenty or thirty bucks. And
Speaker 1: so you just tell it when your race is and it,
Speaker 1: you know, Mind's what a sixteen week program, so you know,
Speaker 1: I had to figure out when it started and just
Speaker 1: stick it in and it puts the workouts on my
Speaker 1: phone on my garment app, which sinks to my watch.
Speaker 1: I have a Garment four Runner two thirty five, which
Speaker 1: is not anywhere near the nearest newest model, but it
Speaker 1: works great. So anyway, that's what I'm doing for running.
Speaker 1: And then finally this week, I'm working on getting AI
Speaker 1: for JavaScript dot com up. I'm also doing AI for
Speaker 1: Ruby dot com. And so you can go and you
Speaker 1: can get on the email list, and I'm going to
Speaker 1: be emailing out, hey, this is what I'm doing with
Speaker 1: AI this week. These are the APIs I used. I
Speaker 1: plan on doing all my examples in both languages, so
Speaker 1: you'll get the JavaScript ones on the JavaScript list and
Speaker 1: the Ruby ones on the Ruby list. And I'm also
Speaker 1: looking at putting together summit at the end of September
Speaker 1: or the end of August. Beginning of September. We'll probably
Speaker 1: have people like Steve. I've been talking to a bunch
Speaker 1: of the other folks in the Ruby community that I
Speaker 1: know that are doing AI, and really I want to
Speaker 1: be hitting it at this level right where it's it's
Speaker 1: not hey, here's how you math your way into models
Speaker 1: that work. This is hey, you've got a model that works,
Speaker 1: or you know, here's how maybe you modify a model
Speaker 1: that you have to train it a little bit, but
Speaker 1: I want it to be approachable for people who want
Speaker 1: to add AI features too applications, not how to solve
Speaker 1: whatever thing by you know, managing a data lake and
Speaker 1: then feeding it into a system that generates the model
Speaker 1: and then how to test the model and all that stuff.
Speaker 1: Or we'll get into some of that at a high level,
Speaker 1: but mostly we're going to be talking about, Hey, here
Speaker 1: are the APIs for Claude or GPT four or mid
Speaker 1: Journey or you know whatever. You know, I do podcasts,
Speaker 1: so Whisper, you know, So here's how you use these
Speaker 1: to get what you need. And then you know, here's
Speaker 1: maybe how you tie a few of them together to
Speaker 1: get a more complicated result than you want. Then you
Speaker 1: can get from any one of them. And so that's
Speaker 1: what I'm looking at. And so anyway, I'll be emailing
Speaker 1: people on the email list right now. I'm just finalizing
Speaker 1: my system because I've kind of had to rework bits
Speaker 1: of the email stuff that I've been doing. So if
Speaker 1: you're interested, go to AI for JavaScript dot com. I'm
Speaker 1: also going to be doing weekly calls for JavaScript geniuses.
Speaker 1: You can find that at JavaScript Geniuses dot com and
Speaker 1: that's going to be sort of a back and forth,
Speaker 1: ask questions, get answers. If you have feedback or ideas
Speaker 1: for people, you can help out, you know, and help
Speaker 1: people get where they want to go with that as well.
Speaker 1: But kind of more of a mastermind or group coaching
Speaker 1: sort of setup is kind of a blend. But we'll
Speaker 1: be doing calls for that every week, and you'll be
Speaker 1: getting some AI stuff in there too. So anyway, are
Speaker 1: my picks and myself promos? Steve, what are your picks?
Speaker 3: Yeah? I think probably just the obvious ones. Check out Builder,
Speaker 3: check out visual co pilot, and I mentioned briefly, but
Speaker 3: Microagent is a pretty interesting CLI tool that if you
Speaker 3: want to categorize AI tools into sort of like commonly
Speaker 3: used today, you know, Chatgypt, get a co pilot Claude,
Speaker 3: and then where things are going agents. You know. I
Speaker 3: do very much believe in agents in the medium to
Speaker 3: long term. I don't see many practical use cases for
Speaker 3: them today, but micro agent is the only one that
Speaker 3: I've used. Obviously, I'm biased here obviously, but it kind
Speaker 3: of solves some of the fundamental challenges. I've got a
Speaker 3: blog post talking about this in better detail, but some
Speaker 3: of the challenges of agents, and I think that I may.
Speaker 1: Perfect micro agent.
Speaker 3: Like many things the you know, I believe in technology
Speaker 3: growing in layers incrementally, as opposed to like you know,
Speaker 3: the V one is supposed to solve everything for everyone,
Speaker 3: like Devin or whatever, a super micro or super AI agent,
Speaker 3: this is everything. And so what's interesting about microagent and
Speaker 3: why I suggest people try it out is it is
Speaker 3: a very I think, new technique and it can lead
Speaker 3: to a lot of interesting things. So people trying it
Speaker 3: and giving feedback I think is really interesting because I
Speaker 3: think that's how we solve these problems over time, is
Speaker 3: how to make agents work. Make it work really well
Speaker 3: for something, And I think this tool works really well
Speaker 3: for a certain type of problem. And I think I
Speaker 3: have ideas and experiences that suggest what's really good at
Speaker 3: But I'd love to get more feedback from people what
Speaker 3: they've found success and not success with, and just more
Speaker 3: input gehab issues, poll requests. People have added some cool
Speaker 3: new features to it. So I think it's an interesting
Speaker 3: area of both a practical tool and like an avenue
Speaker 3: for research and development that you could be part of
Speaker 3: in AI and AI agents and stuff like that that
Speaker 3: I would value. I want the project to be as
Speaker 3: community driven as possible, so check it out, give feedback
Speaker 3: great issues, send pull requests for improvements or fixes or whatever,
Speaker 3: and let me know what you think.
Speaker 1: Awesome people want to find you on the Internet. Where
Speaker 1: do they find you, Steve?
Speaker 3: I am Steve at seven oh Steve eight seven o eight,
Speaker 3: on Twitter, YouTube, TikTok, LinkedIn. I don't know all the places,
Speaker 3: but those are the ones I'm probably most commonly active on.
Speaker 1: Awesome. Well, thanks again for coming. This has been awesome.
Speaker 3: Thanks for having me. This is really fun.
Speaker 1: All right, folks, We're gonna wrap it here till next time.
Speaker 1: Max out