Speaker 1: Elon, Welcome to AI startup School. We're just really really
Speaker 1: blessed to have your presence here today.
Speaker 2: Thanks for having me So.
Speaker 1: From SpaceX, Tesla, Neuralink, XAI, and more. Was there ever
Speaker 1: a moment in your life before all this where you
Speaker 1: felt I have to build something great? And what flipped
Speaker 1: that switch for you? Well?
Speaker 3: I didn't originally think I would build something great. I
Speaker 3: wanted to try to build something useful, but I didn't
Speaker 3: think I would build anything particularly great. You've said, probabilistically
Speaker 3: seemed unlikely, but I wouldn't to at least try.
Speaker 1: So you're talking to a room full of people who
Speaker 1: are all technical engineers, often some of the most eminent
Speaker 1: AI researchers coming up in the game.
Speaker 2: Okay, I I think we should.
Speaker 3: I think I'd like the term engineer better than researcher.
Speaker 3: I mean, I suppose if there's some fundamental algorithmic breakthrough,
Speaker 3: it's a research. Otherwise it's engineering.
Speaker 1: Maybe let's start way back, I mean, when you were
Speaker 1: this is a room full of eighteen to twenty five
Speaker 1: year old excuse younger, because the founder set is younger
Speaker 1: and younger. Can you put yourself back into their shoe?
Speaker 1: When you know, you were eighteen nineteen, you know, learning
Speaker 1: the code, even coming up with this first idea for
Speaker 1: zip too. What was that like for you?
Speaker 2: Yeah?
Speaker 3: Back in ninety five, I was faced with a choice
Speaker 3: of either do you know Grass Studies PhD a Stanford
Speaker 3: in material science actually working on ultra capacities for potential
Speaker 3: use in electric vehicles, essentially trying to solve the range
Speaker 3: problem for electric vehicles, or try to do something in
Speaker 3: this thing that most people have never heard of called
Speaker 3: the Internet. And I talked to my professor was Bill
Speaker 3: Nicks in the material science problement, and said, like, can
Speaker 3: I defer for a quarter because this will probably fail
Speaker 3: and then I'll need to come back to college. And
Speaker 3: then he said this is probably the last conversation we'll have,
Speaker 3: and he was right. So but I thought things would
Speaker 3: most likely fail, not that they would most likely succeed.
Speaker 3: And then in ninety five I wrote basically, I think
Speaker 3: the first or close to the first maps directions, Internet,
Speaker 3: white pages and yellow pages on the Internet. I just
Speaker 3: wrote that. I just wrote that personally, and I didn't
Speaker 3: even use a website. I just read the port directly
Speaker 3: because I couldn't afford h and I could have afforded
Speaker 3: t one. Original office was on Sherman Avenue in pale Alto.
Speaker 3: There was like an I s P on the floor below,
Speaker 3: so I rolled a hole through the floor and just
Speaker 3: ran a land cable directly to the I s P.
Speaker 3: And you know, uh, my brother joined me and another
Speaker 3: co founder of Great Courier who passed away, and we
Speaker 3: at the time we couldn't even afford a place to stay,
Speaker 3: so we just the office was five hundred bucks a months,
Speaker 3: so we just left in the office and and then
Speaker 3: shouted the YMC A on page mill and al communo
Speaker 3: and uh yeah, and we I guess we ended up
Speaker 3: doing a little bit of a useful company as up
Speaker 3: to in the beginning, and we did built a lot
Speaker 3: of really really good software technology, but we were somewhat
Speaker 3: captured by the legacy media companies, and that night Router
Speaker 3: New York Times hearst whatnot were investors and customers and
Speaker 3: and also on the board. So they they kept wanting
Speaker 3: to use our software in ways that made no sense.
Speaker 3: So I wanted to go direct to consumers anyway. As
Speaker 3: long story dwelling too much up too, but I really
Speaker 3: just wanted to do something useful on the Internet because
Speaker 3: I had two choices, like do a PhD and watch
Speaker 3: people build the internet or help build the Internet in
Speaker 3: some small way. And I was like, well, I guess
Speaker 3: I can always try and fail and then go back
Speaker 3: to grad studies. And that ended up being like reasonably successful.
Speaker 3: Sold for like three hundred million dollars, which is a
Speaker 3: lot at the time these days. That's like I think
Speaker 3: minimum impulse, but for an AI startup is like a
Speaker 3: billion dollars. It's like there's so many freaking unicorns. It's
Speaker 3: like heard of unicorns, point of unicorns, a billion dollars situation.
Speaker 1: There's been inflation since, so quite a bit more money.
Speaker 3: Yeah, I mean, like you could probably buy a burger
Speaker 3: for a nickel. Well not quite, but I mean, yeah,
Speaker 3: has been a lot of inflation. But I mean the
Speaker 3: high level of AI is is pretty intense. As you've seen,
Speaker 3: you know, you see companies that are I don't know,
Speaker 3: less than a year old getting sometimes billion dollar, multi
Speaker 3: billion dollar valuations, which I guess could could pan out
Speaker 3: and probably will pan out in some cases, but it
Speaker 3: is I ordering to see some of these valuations.
Speaker 2: Yeah, what do you think.
Speaker 1: I mean, Well, I'm pretty bullish. I'm pretty bullish, honestly, so.
Speaker 1: I think the people in this room are going to
Speaker 1: create a lot of the value that you know, a
Speaker 1: billion people in the world should be using this stuff
Speaker 1: and we're not even we're scratching the surface of it.
Speaker 1: I love the Internet story in that, even back then,
Speaker 1: you know, you are a lot like the people in
Speaker 1: this room back then, in that, you know, the heads
Speaker 1: of all the CEOs of all the legacy media companies
Speaker 1: look to you as the person who understood the Internet,
Speaker 1: and a lot of the world, you know, the corporate world,
Speaker 1: like the world out large that does not understand what's
Speaker 1: happening with AI, they're going to look to the people
Speaker 1: in this room for exactly that. It sounds like, you know,
Speaker 1: what are some of the tangible lessons. It sounds like
Speaker 1: one of them is don't give up board control or
Speaker 1: be careful about have a really good lawyer.
Speaker 3: I guess with the first my first start up, the
Speaker 3: big really the mistake was having too much shareholder and
Speaker 3: board control from legacy media companies who then necessarily see
Speaker 3: things through the lens of legacy media, and that they'll
Speaker 3: kind of make you do things that seem sensible to them,
Speaker 3: but really don't make sense with the new technology. I
Speaker 3: know I should point out that I that I didn't
Speaker 3: actually at first intend to start a company. I try
Speaker 3: to get a job at Netscape SMI resumes Netscape and
Speaker 3: Mark Hendrieson knows about this, but I don't think you
Speaker 3: have a swim my resume, and then nobody responded. So
Speaker 3: and then I tried hanging out in the lobby of
Speaker 3: Netscape see if I could like bump into someone, but
Speaker 3: I was like too shy to talk and talk to anyone.
Speaker 3: So I'm like, man, this is ridiculous. So I'll just
Speaker 3: write sof for myself and see how it goes. So
Speaker 3: it wasn't actually from the stand point of like I
Speaker 3: want to start a company. I just want to be
Speaker 3: part of building you know, the Internet in some way.
Speaker 3: And and since I couldn't get a job at an
Speaker 3: ARAC company, I had to start a interac company anyway.
Speaker 3: The yeah, yeah, I mean from an AI will so
Speaker 3: profoundly change the future. It's difficult to fathom how much.
Speaker 3: But you know, the the economy is, assuming we don't
Speaker 3: things don't go awry, and like AI doesn't kill us
Speaker 3: all in itself, then you'll see ultimately an economy that
Speaker 3: is not not ten times more than the current economy. Ultimately,
Speaker 3: like if we become say or whatever, our future machine
Speaker 3: descendants or but mostly machine descendants become like a cotttership
Speaker 3: scale to civilization or beyond, we're talking about an economy
Speaker 3: that is thousands of times maybe millions of times bigger
Speaker 3: than the economy today.
Speaker 2: So yeah, I mean I.
Speaker 3: Did sort of feel but like, you know, when I
Speaker 3: was in DC taking a lot of flag for like
Speaker 3: getting rid of waste and fraud, which is an interesting
Speaker 3: side quest as side quest go.
Speaker 1: But but I've got to get back to the main quest. Yeah,
Speaker 1: I got to.
Speaker 2: Get back to the main quest here, So back to
Speaker 2: the main quest.
Speaker 3: So but I did feel, you know a little bit
Speaker 3: like there's you know, it's like fixing the government. It's
Speaker 3: kind of like there's like, say, the beach is dirty
Speaker 3: and there's like some needles and vcs and like trash,
Speaker 3: and you want to clean up the beach. But then
Speaker 3: there's also this like thousand foot wall of water, which
Speaker 3: is a tsunami of ai, like, and how much does
Speaker 3: cleaning the beach really matter if you got a thousand
Speaker 3: foot tsunami about to hit not that much.
Speaker 1: Well, we're glad you're back on the main quest. It's
Speaker 1: very important.
Speaker 3: Yeah, back to the main quest, building technology, which is
Speaker 3: what I like doing. It's just so much noise like this,
Speaker 3: The single noise ratio in politics is terrible.
Speaker 1: So, I mean, I live in San Francisco, so you
Speaker 1: don't need to tell me twice.
Speaker 3: Yeah, DC is like, you know, kind of I guess
Speaker 3: it's all politics in DC. But the if you're trying
Speaker 3: to build a rocket or cars, or you're trying to
Speaker 3: have software that compiles and runs reliably, then you have
Speaker 3: to be maximally true seeking or your software or your
Speaker 3: hardware won't work like you can't fool like math and
Speaker 3: physics are rigorous judges. So I'm used to being in
Speaker 3: a maximally true seeking environment and that's definitely not politic.
Speaker 3: So an I'm good glad to be back in technology.
Speaker 1: I guess I'm kind of curious going back to the
Speaker 1: ZIP too moment, you had hundreds of millions of dollars
Speaker 1: or you had an exit worths of millions of dollars.
Speaker 2: I got twenty million dollars.
Speaker 1: Right, Okay, so you solved the money problem at least,
Speaker 1: and you basically took it and you rolled. You kept
Speaker 1: rolling with X dot Com, which is in PayPal and Confinity.
Speaker 2: Yes, I kept the chips on the table.
Speaker 1: Yeah, so not everyone does that. A lot of the
Speaker 1: people in this room will have to make that decision. Actually,
Speaker 1: what drove you to jump back into the ring.
Speaker 3: Well, I think I felt with with Zip too, we'd
Speaker 3: both like incredible technology but never really got used. You know,
Speaker 3: I think, at least from my perspective, we had better
Speaker 3: technology than say Yahoo or anyone else, but it was
Speaker 3: constrained by our customers. And so I wanted to do
Speaker 3: something that where okay, we wouldn't be constrained by our customers,
Speaker 3: go direct to consumer. And that's what ended up being
Speaker 3: like X dot com PayPal, essentially X dot Com merging
Speaker 3: with Confinity, which together created PayPal, and then that actually
Speaker 3: the sort of PayPal diaspora has it might have created
Speaker 3: more companies then, so more companies than probably any anything
Speaker 3: in the twenty first century. You know, so many talented
Speaker 3: people were at the combination of Confinity and X dot Com.
Speaker 3: So I just wanted to, like, I felt like we
Speaker 3: kind of got our wings clift somewhat with Zip two
Speaker 3: and it's like, okay, what if our wings on clipped
Speaker 3: and we go direct to consumer, And that's that's what.
Speaker 2: PayPal ended up being.
Speaker 3: But yeah, with I got that like twenty million dollar
Speaker 3: check for my share of ZIP two. At the time,
Speaker 3: I was living with in a house with four housemates
Speaker 3: and had like a ten grand in the bank, and
Speaker 3: then the check rights in the mail of all placesn't
Speaker 3: in the mail. And then now then my bank a
Speaker 3: bounce went from tenenty to twenty million in ten thousands.
Speaker 3: You're like, well, okay, sluck to pay taxes on that
Speaker 3: at all. But then I ended up putting almost all
Speaker 3: of that into x dot com and as you said,
Speaker 3: just kind of keeping almost all the chips on the table.
Speaker 2: And yeah, and then.
Speaker 3: After Paypala was like, well, I was kind of curious
Speaker 3: as to why we had not sent anyone to Mars,
Speaker 3: and I went on then went on the NASA website
Speaker 3: to find out when we're sending people Tomors, and there
Speaker 3: was no date. I thought, maybe it's just hard to
Speaker 3: find on the website, but in fact that there was
Speaker 3: no real plan to send people to Mars. So then
Speaker 3: you know, I've come to this is such a long story.
Speaker 3: So I don't want to take up too much time here.
Speaker 1: But I think we're all listening with rapt attention.
Speaker 3: So I was actually I was on the long end
Speaker 3: of the expressway with my friend day O RESSI were
Speaker 3: like housemates in college, and day was asking me what
Speaker 3: I'm what we're going to do? What am I going
Speaker 3: to do after a PayPal And I was like, like,
Speaker 3: I don't know. I guess maybe I'd like to do
Speaker 3: something philanthropic in space, because I didn't think I could
Speaker 3: actually do anything commotional in space because that seem like
Speaker 3: the purview of nations. So but you know, I'm kind
Speaker 3: of curious as to when we're going to send people
Speaker 3: to Mars. And that's when I was like, oh, it's
Speaker 3: not on the website, and I started digging on not
Speaker 3: that there's nothing on the NAST website.
Speaker 2: So then I started.
Speaker 3: Digging in and and and I'm definitely summarizing a lot here.
Speaker 3: But my first idea was to do a philanthropic mission
Speaker 3: to Mars called Life to Mars, where would send a
Speaker 3: small greenhouse with see some dehydrated nutrient gael land out
Speaker 3: on mosque and grow, you know, hydrate the jail and
Speaker 3: then you have this great sort of money shot of
Speaker 3: green plants on a red background. But the longest time, I,
Speaker 3: by the way, I didn't realize money shot, I think is.
Speaker 2: A porn reference.
Speaker 3: But anyway, the point is that that would be the
Speaker 3: great shot of green plants on a red background, and
Speaker 3: to try to inspire you know, NASA and the public to.
Speaker 2: Send astronauts to to Mars.
Speaker 3: As I learned more, it came to realize and along
Speaker 3: the way, by the way, I went to Russia in
Speaker 3: like two thousand and one and two thousand and two
Speaker 3: to buy ICVMS, which is like, that's an adventure, you know,
Speaker 3: you go and meet with Russian high command and say
Speaker 3: I'd like to buy some ICVMS.
Speaker 1: This was to get to space, yeah, as a rocket
Speaker 1: to not.
Speaker 2: To nuke anyone, but.
Speaker 3: They had to as a result of onstroduction talks, they
Speaker 3: had to actually destroy a bunch of their big nuclear missiles.
Speaker 2: So I was like, well, how about if we take two.
Speaker 3: Of those, you know, minus the nuke, added an additional
Speaker 3: upper stage for Mars. But it was kind of trippy,
Speaker 3: you know, being in Moscow in one of two thousand
Speaker 3: and one negotiating with like the Russian military to buy ICVM,
Speaker 3: like that's crazy, but they kept also raising the price
Speaker 3: on me, so that so like literally it's kind of
Speaker 3: like the opposite of what negotiation should should do. So
Speaker 3: I was like, man, these things are getting really expensive.
Speaker 3: And then I came to realize that actually the problem
Speaker 3: was not that there was insufficient world to go to Mars,
Speaker 3: but there was no way to do so without breaking
Speaker 3: the budget, you know, or even breaking the NASA budget.
Speaker 3: So that's where I decided to start SAX SpaceX to
Speaker 3: advance rocket technology to the point where we could send
Speaker 3: people to Mars.
Speaker 2: And that was in two thousand and two.
Speaker 1: So that wasn't you know. You didn't start out wanting
Speaker 1: to start a business. You wanted to start just something
Speaker 1: that was interesting to you that you thought humanity needed.
Speaker 1: And then as you sort of you know, like a
Speaker 1: cat pulling on you know, a string, it just sort
Speaker 1: of the ball sort of unravels. It turns out this
Speaker 1: is could be a very profitable business.
Speaker 3: I mean it is now, But there had been no
Speaker 3: prior example of really a rocket startup seating. There have
Speaker 3: been various attempts to do commercial rocket companies and that
Speaker 3: all failed. So again with SpaceX, starting SpaceX was really
Speaker 3: from the standpoint of, like I think there's like a
Speaker 3: less than ten percent chance of being successful, maybe one percent,
Speaker 3: I don't know. But if a startup doesn't do something
Speaker 3: to advance rocket technology, it's definitely not coming from the
Speaker 3: big defense contractors because they just impedance smashed the government,
Speaker 3: and the government just wants to do very conventional things.
Speaker 3: So there's it's either coming from a startup or it's
Speaker 3: not happening at all. So so like a small chance
Speaker 3: of success is better than no chance of success, and
Speaker 3: so that yeah, so SpaceX started that in mid mid
Speaker 3: two thousand and two expecting to fail, Like I said,
Speaker 3: probably ninety percent chance of failing. And even like when
Speaker 3: recruiting people, I didn't like to try to make out
Speaker 3: that it would probably I said, we're probably going to die,
Speaker 3: but four chance we might not die. But this is
Speaker 3: the only way to get people to Mars and advanced
Speaker 3: state of the art. And then I ended up being
Speaker 3: chief engineer of the rocket, not because I wanted to,
Speaker 3: but because I couldn't hire anyone.
Speaker 2: Who was good.
Speaker 3: So like none of the good sort of chief engineers
Speaker 3: would join because it's like this is too risky you
Speaker 3: were going to die. And so then I ended up
Speaker 3: being chief engineer of the rocket. And you know, the
Speaker 3: first three flights did fail, so it's a bit of
Speaker 3: a learning exercise there, and fourth one fortunately worked. But
Speaker 3: if the fourth one hadn't worked, I had no money
Speaker 3: left and that would have been it would have been curtains.
Speaker 2: So it was a pretty close thing.
Speaker 3: If the fourth launch of Falcon Nott work, it would
Speaker 3: have been just curtains and we would have just been
Speaker 3: joined the graveyard of prior rocket startup. So it's like
Speaker 3: my estimate of success was not far off. We'd just
Speaker 3: be made it by the skin of our teeth. And
Speaker 3: Tesla was happening sort of simultaneously. Like two thousand and
Speaker 3: eight was a rough year because at mid two thousand
Speaker 3: and eight we're called summer two thousand and eight, the
Speaker 3: third the third launch of SpaceX had failed, a third
Speaker 3: failure in a row. The Tesla financing round had failed,
Speaker 3: and so Telsa was going bankrupt fast. It was just like, man,
Speaker 3: this is a grim this is this is going to
Speaker 3: be a tale of warning of an exercise in hubers.
Speaker 1: Probably throughout that period, a lot of people were saying,
Speaker 1: you know, Elon is a software guy.
Speaker 2: Why are you.
Speaker 1: Working on hardware? Why would you? Yeah, why would he
Speaker 1: choose to work on this?
Speaker 3: Right, Harp said, So you can look at the like
Speaker 3: the because it's still the you know, the press of
Speaker 3: that time is still online. You could just search it.
Speaker 3: And and they kept calling the Internet guy. So like
Speaker 3: Internet guy aka fool is attempting to build a rocket company.
Speaker 3: So you know that we got ridiculed quite a lot.
Speaker 3: And it does sound pretty absurd, Like internet guy starts
Speaker 3: rocket company doesn't sound like a recipe for sess frankly,
Speaker 3: so I didn't hold against them. I was like, yeah,
Speaker 3: you know, admittedly it does sound improbable, and I agree
Speaker 3: that it's improbable. But fortunately the fourth launch worked and
Speaker 3: uh and, and NASA warded us a contract to resupply
Speaker 3: the space station uh and. I think that was like
Speaker 3: maybe December twenty second. It was like right before Christmas.
Speaker 3: Because even the fourth launch working wasn't enough to succeed.
Speaker 3: NASA also needed we also needed a big contract to
Speaker 3: keep us a life. So so I got I got
Speaker 3: that call from like the NASA NASA team and I literally,
Speaker 3: we're rewarding you one of the contracts to re supply
Speaker 3: the space station, like literally blurt it out, I love
Speaker 3: you guys, which is not normally you know what they hear.
Speaker 3: It's usually pretty, you know, sober. But I was like, man,
Speaker 3: this is a company saver. And then we closed the
Speaker 3: Tesla financing around on the last hour of the last
Speaker 3: day that it was possible, which was six pm December
Speaker 3: twenty fourth, two thousand and eight. We would have bounced
Speaker 3: payroll two days after Christmas if that round had had closed.
Speaker 3: So that was a nerve wracking end of two thousand
Speaker 3: and eight, that's for sure.
Speaker 1: I guess from your PayPal and zip two experience jumping
Speaker 1: into these hardcore hardware startups, it feels like one of
Speaker 1: the through lines was being able to find and eventually
Speaker 1: attract the smartest possible people in those popular fields. You know,
Speaker 1: what would I mean the people in this room, like
Speaker 1: some of most of the people here I don't think
Speaker 1: have even managed the single person yet, they're just starting
Speaker 1: their careers. What would you tell to, you know, the
Speaker 1: elon who's never had to do that yet.
Speaker 3: I generally think to try to try to be as
Speaker 3: useful as possible. So it may sound trit but it's
Speaker 3: it's so hard to be useful, especially to be useful
Speaker 3: to a lot of people, Where you say the area
Speaker 3: under the cove of total utility is like how much
Speaker 3: how useful have you been to your failer human beings?
Speaker 2: Times?
Speaker 3: How many people? Like it's almost like like the physics
Speaker 3: definition of true work. It's incredibly difficult to do that.
Speaker 3: I think if you aspire to do true work, your
Speaker 3: probably success is much higher, Like like, don't inspire to glory,
Speaker 3: aspire to work.
Speaker 1: How can you tell that it's true work? Like is
Speaker 1: it external? Is it like what happens with other people?
Speaker 1: Or you know what the product does for people? Like
Speaker 1: what you know? What is that for you when you're
Speaker 1: looking for people to come work for you? Like what
Speaker 1: you know? What's the salient thing that you look for?
Speaker 3: Or if they're that's certain question I guess it's I
Speaker 3: mean in terms of off your in product, you have
Speaker 3: to say, like, well, if this thing is successful, how
Speaker 3: useful will it be to how many people?
Speaker 2: And that's that's what I mean.
Speaker 3: And then you do you do whatever you know, whether
Speaker 3: you're CEO or any role in a startup, you do
Speaker 3: whatever it takes to succeed, like and just and just
Speaker 3: always be smash smashing your ego like like internalized responsibility.
Speaker 3: Like a major failure mode is when ego to ability
Speaker 3: ratio is double graded than sign one. You know, like
Speaker 3: if you if your ego to ability ratio is it
Speaker 3: gets too high, then you're you're you're you're going to
Speaker 3: basically break the feedback loop to reality. And in in
Speaker 3: AI terms, your your er, you'll have your you'll break
Speaker 3: your r L loop.
Speaker 2: So you want you want to don't want to break
Speaker 2: your art.
Speaker 3: You want to have a strong r L loop, which
Speaker 3: means internalizing responsibility and minimizing the ego, and you do
Speaker 3: whatever the task is, no matter whether it's you know,
Speaker 3: grand or humble. So I mean that's kind of like
Speaker 3: why I actually I prefer the term like engineering as
Speaker 3: opposed to research. I prefer the term And I don't
Speaker 3: actually don't want it to call x AI lab. I
Speaker 3: just want to be a company. Like it's like whatever,
Speaker 3: the what of the simplest, most straightforward, uh ideally lowest
Speaker 3: ego terms are that those are generally a good way
Speaker 3: to go. You want to just close the loop on
Speaker 3: reality hard That's that's a that's super big deal.
Speaker 1: I think everyone in this room is really looks up
Speaker 1: to everything you've done around being sort of a paragon
Speaker 1: of first principles and you know, thinking about the stuff
Speaker 1: if you've done, how do you actually determine your reality?
Speaker 1: Because that seems like a pretty big part of it.
Speaker 1: Like other people people who have never made anything, non engineers,
Speaker 1: sometimes journalists at time who've never done anything, like they
Speaker 1: will criticize you, But then clearly you have another set
Speaker 1: of people who are builders, who have very high you know,
Speaker 1: sort of area under the curve, who are in your circle. Like,
Speaker 1: you know, how should people approach that? Like what has
Speaker 1: worked for you? And what would you pass on like
Speaker 1: you know, to X to your children? Like you know,
Speaker 1: what do you tell them when you're like you need
Speaker 1: to make your way in this world?
Speaker 2: Here?
Speaker 1: You know, here's how to construct a reality that is
Speaker 1: predictive from first principles.
Speaker 3: Well, the tools of physics are incredibly helpful to understand
Speaker 3: and make progress in any field. And first principles means
Speaker 3: just obviously just means, you know, breakings down to the
Speaker 3: fundamental axiomatic elements that are most likely to be true
Speaker 3: and then reason up from there as cosionally as possible,
Speaker 3: as opposed to reasoning by analysis or metaphor, and then
Speaker 3: it just simple things like like thinking in the limit,
Speaker 3: like if you strapolate, you know, minimize this thing or
Speaker 3: maximize that thing. Thinking in the limit is is very
Speaker 3: very helpful. I use all the tools of physics. They
Speaker 3: apply to any field. This is like a superpower.
Speaker 2: Actually.
Speaker 3: So you can take say, take for example, like rockets.
Speaker 3: You could say, well, how much should a rocket rocket cost?
Speaker 3: A typical approach to how to that people would take
Speaker 3: how much rocket should cost? There is they would look
Speaker 3: historically at what the cost of rockets are and assume
Speaker 3: that any new rocket must be somewhat similar to the
Speaker 3: prior cost of rockets. A first principal's approach would be
Speaker 3: you look at the materials that the rocket is comprised of,
Speaker 3: so if that's aluminum, copper, carbon, fiber, steel, whatever the
Speaker 3: case may be, and say what what how much does
Speaker 3: that rocket weigh, and and what are the constituent elements
Speaker 3: and how much does they weigh? What is the material
Speaker 3: price per kilogram of those constituent elements, In that sense
Speaker 3: the actual floor on what a rocket can cost. It
Speaker 3: can asymptotically approach costs the raw materials and then you realize, oh,
Speaker 3: actually a rocket. The ROM turials of a rocket are
Speaker 3: only maybe one or two percent of the historical cost
Speaker 3: of a rocket, so the manufacturing must necessarily be very inefficient.
Speaker 3: If the if the rom heerial cost is only one
Speaker 3: or two percent, that would be a first first principles
Speaker 3: analysis of the potential for cost optimization of a rocket.
Speaker 2: And that's before you get to reusability.
Speaker 3: To give an AI sort of AI example, I guess
Speaker 3: last year went for XCI when we were trying to
Speaker 3: build a training supercluster. We went to the various suppliers
Speaker 3: to ask what they said, this is the beginning of
Speaker 3: last year, that we needed one hundred thousand, h one
Speaker 3: hundreds to be able to train coherently. And there are
Speaker 3: estimates for how long it would take to complete that
Speaker 3: were eighteen to twenty four months. It's like, well, we
Speaker 3: need to get that done in six months so then
Speaker 3: or we won't be competitive. So then if you break
Speaker 3: that down, what are the things you need? Will you
Speaker 3: need a building, you need power, you need cooling. We
Speaker 3: didn't have enough time to build a building from scratch,
Speaker 3: so we've had to find existing building. So we found
Speaker 3: at a factory that was no longer in use in Memphis.
Speaker 2: That used to build electrolytes products.
Speaker 3: But then the input power was fifteen megawats and we
Speaker 3: needed one hundred and fifteen megawatts, So we read the
Speaker 3: generators and had generators on one side of the building.
Speaker 3: And then we have to have cooling, so we rented
Speaker 3: about a quarter of the mobile cooling capacity of the
Speaker 3: US and put the chillers.
Speaker 2: On the other side of the building.
Speaker 3: That didn't fully solve the problem because the voltage the
Speaker 3: power variations during training are very very big, so you
Speaker 3: can have power can drop by fifty percent in one
Speaker 3: hundred milliseconds, which the generators can keep up with.
Speaker 2: So then we.
Speaker 3: Combined we added Tesla megapacks and modified the software in
Speaker 3: the megapacks to be able to smooth out the power
Speaker 3: variation during the training run. And then there were a
Speaker 3: bunch of networking challenges because the networking cables, if you're
Speaker 3: trying to make one hundred thousand GPUs training coherently, are
Speaker 3: very very challenging.
Speaker 1: Almost it sounds like almost any of those things you mentioned,
Speaker 1: I could imagine someone telling you very directly, no, you
Speaker 1: can have that, you can't have that power, you can't
Speaker 1: have this uh, and it sounds like one of the
Speaker 1: salient pieces of first principles thinking is actually, let's ask why,
Speaker 1: let's you know, figure that out, and actually let's challenge
Speaker 1: the person across the table and if they if I
Speaker 1: don't get an answer that I feel good about, I'm gonna,
Speaker 1: you know, not allow that to be I'm not going
Speaker 1: to let that know to stand. Is that? I mean,
Speaker 1: that feels like something that you know, everyone if someone
Speaker 1: were to try to do what you're doing in hardware,
Speaker 1: hardware seems too uniquely and software we have lots of
Speaker 1: you know, fluff and things that you know, it's like
Speaker 1: we can add more CPUs for that, it'll be fine.
Speaker 1: But in hardware it's it's just not going to work.
Speaker 3: I think these general principles of first principles thinking apply
Speaker 3: to software and hardware, apply to anything really. I'm just
Speaker 3: using kind of a hardware example of how we're told
Speaker 3: something is impossible, but once we broke it down into
Speaker 3: the constituent elements of we need a building, we need power,
Speaker 3: we need cooling, we need uh, we need power smoothing,
Speaker 3: and then and then we could solve those constituent elements.
Speaker 3: But and then we and then we rand the networking
Speaker 3: operation to do all the cabling everything in four shifts
Speaker 3: twenty four to seven, and I was like sleeping in
Speaker 3: the data center and also doing cabling myself, and there
Speaker 3: were a lot of both issues to solve. You know,
Speaker 3: nobody had done a training run with one hundred thousand,
Speaker 3: h one hundreds training coherently last year.
Speaker 2: I mean, maybe it's been done this year. I don't know.
Speaker 3: But then and then we ended up doubling that to
Speaker 3: two hundred thousand, and so now we've got one hundred
Speaker 3: and fifty thousand, h one hundred fifty k H two
Speaker 3: hundreds and thirty k GB two hundreds in the in
Speaker 3: the Memphis training center, and we're about to bring one
Speaker 3: hundred and ten thousand GB two hundreds online at a
Speaker 3: second data center also in the Memphis area.
Speaker 1: Is it your view that you know pre training is
Speaker 1: still working and you know larger the scaling laws still hold,
Speaker 1: and whoever wins this race will have basically the biggest,
Speaker 1: smartest possible model that you could steal.
Speaker 3: Well, there's other various elements that side competitiveness for larger
Speaker 3: I this this for sure, the talent of the people matter.
Speaker 3: The scale of the hardware matters, and how well you're
Speaker 3: able to bring that hardware to bear. So you can't
Speaker 3: just order a whole bunch of GPUs and if they don't,
Speaker 3: you can't just plug them in. So you've got to
Speaker 3: You've got to get a lot of GPUs and have
Speaker 3: them train training coherently and stavely. Then it's like, what
Speaker 3: unique access to data.
Speaker 2: Do you have?
Speaker 3: I guess distribution matters to some degree as well, like
Speaker 3: how do people get exposed to your AI?
Speaker 2: Those?
Speaker 3: Those are those are critical factors for if it's going
Speaker 3: to be like a large foundation model as competitive, you know,
Speaker 3: as as many have said.
Speaker 2: I think my friend eliasit Skier.
Speaker 3: Said, uh, you know, we we've kind of run out
Speaker 3: of pre training data of human generated pre like human
Speaker 3: generated data, you run out of tokens pretty fast, of
Speaker 3: certainly of high quality tokens. And and then you then
Speaker 3: you have to do a lot of you need to
Speaker 3: essentially create synthetic data and and be able to accurately
Speaker 3: judge the synthetic data that you're creating to very like
Speaker 3: is this real synthetic data?
Speaker 2: Well?
Speaker 3: Is it in a hallucination that doesn't actually mash reality?
Speaker 3: So achieving grounding and reality is tricky, but we're at
Speaker 3: the stage where there's more effort put into synthetic data.
Speaker 3: And right now we're training Grock three point five, which
Speaker 3: is a heavy focus on reasoning.
Speaker 1: Going back to your physics point, what I heard for
Speaker 1: reasoning is that hard science, particularly physics textbooks, are very
Speaker 1: useful for reasoning, whereas I think researchers have told me
Speaker 1: that social science is totally useless for reasoning.
Speaker 2: Yes, that's probably true.
Speaker 3: So yeah, there's something that's going to be very important
Speaker 3: in the future is combining deep AI in the data
Speaker 3: center of Supercluster with robotics. So that's you know, things
Speaker 3: like like the Optimius humanoid robot, and yeah, Optimist is awesome.
Speaker 3: There's going to be so many humanoid robots and robots
Speaker 3: of all robots of all sizes and shapes. But my
Speaker 3: prediction is that there will be more humanoid robots by
Speaker 3: far than all other robots can bind by maybe an
Speaker 3: order of magnitude, like a big difference.
Speaker 1: And is it true that you're planning a robot army of.
Speaker 3: A sort, whether we do it or or you know,
Speaker 3: whether Tesla does it. You know, Tesla works closely with Xai.
Speaker 3: Like you've seen how many humanoid robots startups are there.
Speaker 3: Like it's like I think Jensen Bong was on stage
Speaker 3: with a lot with a massive number of robots, you know,
Speaker 3: robots from different companies. I think it was like a
Speaker 3: dozen different humanoid robots. So I mean, I guess, you know,
Speaker 3: part of what I've been biting and maybe what has
Speaker 3: slowed me down somewhat is that I'm I'm a little
Speaker 3: I don't want I don't want to make Terminator real,
Speaker 3: you know. So I've been sort of, I guess, at
Speaker 3: least until recent years, dragging my feet on on AI
Speaker 3: and and humanoid robotics. And then I sort of come
Speaker 3: to realize that, realizations it's happening.
Speaker 2: Whether I do it or not.
Speaker 3: So you got really two choices participate. You could either
Speaker 3: be a spectator or a participant. It's like, well, I
Speaker 3: guess I'd rather be a participant than a spectator. So
Speaker 3: now it's you know, pedals to the metal on humanoid
Speaker 3: robots and digital superintelligence.
Speaker 1: So I guess you know, there's a third thing that
Speaker 1: everyone has heard you talk a lot about that I'm
Speaker 1: really a big fan of, you know, becoming a multiplanetary space.
Speaker 1: Where does this fit? You know? This is all you know,
Speaker 1: not not just a ten or twenty year thing, maybe
Speaker 1: one hundred year thing. Like it's a multi you know,
Speaker 1: many many generations for humanity kind of thing. You know,
Speaker 1: how do you think about it? There's you know AI obviously,
Speaker 1: there's embodied robotics, and then there's being a multiplant, multiplanetary species.
Speaker 1: Does everything sort of feed into that last point? Or
Speaker 1: you know what? What are you driven by right now
Speaker 1: for the next ten, twenty and one hundred years?
Speaker 3: See one hundred years, man, I hope civilizations around in
Speaker 3: one hundred years. If it is around, it's going to
Speaker 3: look very different from civilization today. I mean, I'd predict
Speaker 3: that this is going to be at least five times
Speaker 3: as many humanoid robots as there are humans, maybe ten times.
Speaker 3: One way to look at the progress of civilization is
Speaker 3: percentage completion Kladishev. So if you're in a cludos Ship
Speaker 3: scale one, you've you've hot as all the energy of
Speaker 3: a planet, and in my opinion, we've only ha maybe
Speaker 3: one or two percent of Earth's energy, So we've got
Speaker 3: a long way to go to the Kardschev scale one.
Speaker 3: Then Karshev two, you've harnessed all the energy of the Sun,
Speaker 3: which would be I don't know, a billion times more
Speaker 3: energy than Earth, maybe closer to a trillion, And then
Speaker 3: Karshev three would be all the energy of a galaxy,
Speaker 3: pretty far from that. So we're at the very very
Speaker 3: early stage of the intelligence big bang. I hope we're
Speaker 3: in terms of be multiplanetary, like I think. I think
Speaker 3: we'll have enough mass transferred to Mars within like roughly
Speaker 3: thirty years to make Mars self sustaining such that Mars
Speaker 3: can continue to grow and prosper even if the resupply
Speaker 3: shifts from Earth stuff coming, and that greatly increases the
Speaker 3: probable lifespan of civilization or consciousness and intelligence, both biological
Speaker 3: and digital. So that's why I think it's important to
Speaker 3: become a multiplanet species. And I'm so much troubled by
Speaker 3: the filmy paradox, like why have we not seen any aliens?
Speaker 3: And it could be because intelligence is incredibly rare and
Speaker 3: maybe we're the only ones in this galaxy, in which
Speaker 3: case the intelligence of consciousness is the tiny candle in
Speaker 3: a vast dogness, and we should do everything possible to
Speaker 3: ensure the tiny candle does not go out, and being
Speaker 3: a multiplanet species, or making consciousness multiplanetary, greatly improves the
Speaker 3: probable lifespan of civilization, and it's it's the next step
Speaker 3: before going to other star system on you. Once you
Speaker 3: at least have two planets, then you've got a forcing
Speaker 3: function for the improvement of space travel, and that ultimately
Speaker 3: is what will lead to consciousness expanding to the stars.
Speaker 1: It could be that the Fermi paradox dictates once you
Speaker 1: get to some level of technology, you destroy yourself. How
Speaker 1: do we see ourselves? How do we actually what would
Speaker 1: you prescribe to I mean a room full of engineers, like,
Speaker 1: what can we do to prevent that from happening?
Speaker 2: Yeah?
Speaker 3: How do we avoid the great filters? One of the
Speaker 3: great filters would obviously be global termin nuclear war, So
Speaker 3: we should.
Speaker 2: Try to avoid that.
Speaker 3: I guess building benign AI robots that that loves humanity
Speaker 3: and you know, robots.
Speaker 2: That are helpful.
Speaker 3: Something that I think is extremely important in building AI
Speaker 3: is a very rigorous adherence to truth, even if that
Speaker 3: truth is politically incorrect. My intuition for what could make
Speaker 3: AI very dangerous is if if you force AI to
Speaker 3: believe things that are not true.
Speaker 1: How do you think about you know, there's sort of
Speaker 1: this argument for open for safety versus closed for competitive edge.
Speaker 1: I mean, I think the great thing is you have
Speaker 1: a competitive model. Many other people also have competitive models,
Speaker 1: and in that sense, you know, we're sort of off of.
Speaker 1: Maybe the worst timeline that I'd be worried about is,
Speaker 1: you know, there's fast takeoff and it's only in one
Speaker 1: person's hands, so that might you know, sort of collapse
Speaker 1: a lot of things, whereas now we have choice, right,
Speaker 1: How do you think about this?
Speaker 2: Yeah?
Speaker 3: I do think there will be several deep intelligences, maybe
Speaker 3: maybe at least five, maybe as much as ten. I'm
Speaker 3: not sure that there's going to be hundreds, but it's
Speaker 3: probably close. Like maybe it'll be like ten like that,
Speaker 3: of which maybe four will be in the US. So
Speaker 3: I don't think it's going to be anyone AI that
Speaker 3: that has runaway capability.
Speaker 2: But but yeah, several deep intelligences.
Speaker 1: What will these deep deep intelligences actually be doing? Will
Speaker 1: it be scientific research or trying to hack each other?
Speaker 2: Probably all of the above.
Speaker 3: I mean, hopefully they will discover new physics, and I
Speaker 3: think they're very they're definitely going to invent new technologies.
Speaker 3: I think I think we're quite close to digital superintelligence.
Speaker 3: It may happen this year, and if it doesn't happen
Speaker 3: this year, next year for sure. Digital superintelligence defined as
Speaker 3: smarter than any human at anything.
Speaker 1: Well, so how do we direct that to sort of
Speaker 1: superabundance you know we have we could have robotic labor,
Speaker 1: we have cheap energy, intelligence on demand.
Speaker 2: You know?
Speaker 1: Is that sort of the white pill? Like where do
Speaker 1: you sit on the spectrum? And are there tangible things
Speaker 1: that you would encourage everyone here to be working on
Speaker 1: to make that white pill actually reality?
Speaker 2: I think I think it most likely will be a
Speaker 2: good outcome.
Speaker 3: I guess i'd sort of agree with Jeff Hinton that
Speaker 3: maybe it's a ten to twenty percent chance of annihilation,
Speaker 3: but look on the right side, that's eighty to ninety
Speaker 3: percent probably of a great outcome. So yeah, I can't
Speaker 3: emphasize this enough. A rigorous adherence to truth is the
Speaker 3: most important thing for AI safe safety, and obviously empathy
Speaker 3: for humanity and life as we know it.
Speaker 1: We haven't talked about neuralink at all yet, but I'm curious.
Speaker 1: You know you're working on closing the input and output
Speaker 1: gap between humans and gene How critical is that to
Speaker 1: agi ASI And you know, once that link is made,
Speaker 1: can we not only read but also write.
Speaker 3: The neuralink is not necessary to solve digital superintelligence. That'll
Speaker 3: happen before neuralink is at scale. But what neuralink can
Speaker 3: effectively do is solve the input output bandwidth constraints. Especially
Speaker 3: our output bandwidth is very low out The sustained output
Speaker 3: of a human over the course of a day is
Speaker 3: less than one but per second, So there's you know,
Speaker 3: eighty six four hundred seconds in a day, and it's
Speaker 3: extremely rare for a human to output more than that
Speaker 3: number of symbols per day, so certainly for several days
Speaker 3: in a row. So you really with with within neuralink interface,
Speaker 3: you can massively increase your output bandwidth and your input
Speaker 3: band with input.
Speaker 2: Being right to you, you have to do right operations
Speaker 2: to the brain.
Speaker 3: We have now five humans who have received the.
Speaker 2: Kind of the read input where it's reading.
Speaker 3: Signals, and you've got people with with als who really
Speaker 3: have their their tetraplegics, but they can now communicate at
Speaker 3: with similar band to a human with a fully functioning
Speaker 3: body and control their computer and phone which is pretty cool.
Speaker 3: And then I think in the next six to twelve
Speaker 3: months we'll be doing our first implants for vision where
Speaker 3: even if somebody is completely blind, could we can write
Speaker 3: directly to the visual cortex. And we've had that working
Speaker 3: in monkeys. Actually, I think one of our monkeys now
Speaker 3: has had a visual implant for three years. And at
Speaker 3: first it'll be relatively fairly low resolution, but long term
Speaker 3: you would have very high resolution and be able to
Speaker 3: see multi spectral wavelengths, so you could see an infrared
Speaker 3: ultra violet radar like a superpower situation. But like at
Speaker 3: some point the sabernetic implants would would not simply be
Speaker 3: correcting things that went wrong, but augmenting human capabilities dramatically,
Speaker 3: augmenting augmenting intelligence and senses and bandwidth dramatically. And that's
Speaker 3: that's going to happen at some point, but digital superintelligence
Speaker 3: will happen well before that. At least if we have
Speaker 3: a neural link, we'll be able to appreciate the AI better.
Speaker 1: I guess one of the limiting reagents to all of
Speaker 1: your efforts across all of these different domains is access
Speaker 1: to the smartest possible people. But you know, sort of
Speaker 1: simultaneous to that, we have you know, the rocks can
Speaker 1: talk and reason, and then you know they're maybe one
Speaker 1: hundred and thirty IQ now and they're probably going to
Speaker 1: be super intelligent soon. How do you reconcile those two things,
Speaker 1: like what's going to happen in you know, five ten
Speaker 1: years and what should the people in this room do
Speaker 1: to make sure that you know they're the ones who
Speaker 1: are creating instead of maybe below the API line.
Speaker 3: Well, they called it the singularity for a reason, Because
Speaker 3: we don't know what's going to happen in the not
Speaker 3: that far future. The percentage of intelligence that is human
Speaker 3: will be quite small. At some point, the collective some
Speaker 3: of human intelligence will be less than one percent of
Speaker 3: all intelligence. And if things get to a cult a
Speaker 3: ship level too, we're talking about human intelligence even assuming
Speaker 3: a significant increase in human population and intelligence augmentation, like
Speaker 3: massive intelligence augmentation where like everyone has an IQ of
Speaker 3: one thousand type of thing. Even in that circumstance, collective
Speaker 3: human intelligence will be probably one billion that of digital intelligence. Anyway,
Speaker 3: we're the biological bootbootloader for digital superintelligence.
Speaker 1: I guess just to end off.
Speaker 2: Was that it was like was I a good bootload.
Speaker 1: Where do we go? How do we go from here?
Speaker 1: I mean, I mean all of this is pretty wild
Speaker 1: sci fi stuff that also could be built by the
Speaker 1: people in this room. You know, if you do, you
Speaker 1: have a closing thought for the smartest technical people of
Speaker 1: this generation right now? What should they be doing? What
Speaker 1: should they what should they be working on? What should
Speaker 1: they be thinking about, you know, tonight as they go
Speaker 1: to dinner.
Speaker 3: Well, as I saw Rolfe with, I think if you're
Speaker 3: doing something useful, that's great. If you just just try
Speaker 3: to be as useful as possible to your fellow human
Speaker 3: beings and that that then you're doing something good. I
Speaker 3: keep harping on this, like focus on super truthful AI.
Speaker 3: That that's the most important thing for AI safety. You know, obviously,
Speaker 3: if you know anyone's interested in working at x AI mean,
Speaker 3: please please let us know. We're aiming to make rock
Speaker 3: the maximumly truth seeking AI and uh, I think that's
Speaker 3: a very important thing.
Speaker 2: Hopefully we can understand the nature of the universe.
Speaker 3: That that's really I guess what AI can hopefully tell
Speaker 3: us maybe AI AI and maybe tell us where are
Speaker 3: the aliens? And you know, how did the universe really start,
Speaker 3: how will it end? What are the questions that we
Speaker 3: don't know that we should ask, And are we in
Speaker 3: a simulation or what level of similation are we in?
Speaker 1: Well, I think we're going to find out.
Speaker 2: An MPC.
Speaker 1: Elon, thank you so much for joining us. Everyone, please
Speaker 1: give it up for Elon Musk Basically,