Speaker 1: I wanted to try to build something useful, but I
Speaker 1: didn't think I would build anything particularly great. You've said,
Speaker 1: probabilistically it seemed unlikely, but I wouldn't to at least try.
Speaker 2: So you're talking to a room full of people who
Speaker 2: are all technical engineers, often some of the most eminent
Speaker 2: AI researchers coming up in the game.
Speaker 1: Okay, I I think we should.
Speaker 3: I think I like the term engineer better than researcher.
Speaker 1: I suppose if there's some fundamental algorithmic breakthrough, it's a
Speaker 1: research Otherwise it's engineering.
Speaker 2: Maybe let's start way back when you were This is
Speaker 2: a room full of eighteen to twenty five year olds.
Speaker 2: Excuse younger, because the founder set is younger and younger.
Speaker 2: Can you put yourself back into their shoes? When you
Speaker 2: were eighteen nineteen learning to code, even coming up with
Speaker 2: a first idea for ZIP too, What was that like
Speaker 2: for you?
Speaker 3: Yeah?
Speaker 1: Back in ninety five, I was basically choice of either
Speaker 1: do Grass studies PhD Stanford and material science actually working
Speaker 1: on ultra capacities for potential use of electric vehicles, essentially
Speaker 1: trying to solve the range problem for electric vehicles, or
Speaker 1: try to do something in this thing that most people
Speaker 1: have never heard of, called the Internet. And I talked
Speaker 1: to my professor, was Bill Nicks in the material science problement,
Speaker 1: and I said, can I defer for a quarter because
Speaker 1: this will probably fail and then I'll need to come
Speaker 1: back to college.
Speaker 3: And then he said, this is probably the last conversation
Speaker 3: we'll have, and he was right.
Speaker 1: But I thought things would most likely fail, not that
Speaker 1: they would most likely succeed. And then in ninety five
Speaker 1: I wrote, basically, I think the first or close to
Speaker 1: the first maps directions, Internet, white pages and yellow pages
Speaker 1: on the Internet. I just wrote that personally, and I
Speaker 1: didn't even use a website. I just read the port
Speaker 1: directly because I couldn't afford and I could afford a
Speaker 1: t one. The original office was on Firman Avenue in Palelto.
Speaker 1: There was like an ISP on the floor below. I
Speaker 1: rolled a hole through the floor and just ran a
Speaker 1: land cable directly to the ISP, and my brother joined
Speaker 1: me and another co founder of Great Courrier who passed away,
Speaker 1: and at the time we couldn't even afford a place
Speaker 1: to stay, so we just The office was five hundred
Speaker 1: bucks months, so we just left in the office and
Speaker 1: then shouted the YMCA on page mill and al communo
Speaker 1: and yeah, and I guess we ended up doing a
Speaker 1: little bit of a useful company is up to.
Speaker 3: In the beginning, and I did build a lot of
Speaker 3: really really good.
Speaker 1: Software technology, but were somewhat captured by the legacy media
Speaker 1: companies and that night rout or New York Times, the
Speaker 1: host whatnot were investors and customers and also on the board,
Speaker 1: so they kept wanting to use software in ways that
Speaker 1: made no sense. So I wanted to go direct to
Speaker 1: consumers anyway, as long story, dwelling too much use up
Speaker 1: too But I really just wanted to do something useful
Speaker 1: on the Internet because I have two choices.
Speaker 3: Do I PhD and watch people build the internet or
Speaker 3: help build.
Speaker 1: The internet in some small way. And I was like,
Speaker 1: I guess I can always try and fail and then
Speaker 1: go back to grad studies. And anyway, that ended up
Speaker 1: being like reasonably successful. Sold for like three hundre million dollars,
Speaker 1: which is a lot at the time these days, that's
Speaker 1: I think the minimum impulse, but for an AI startup
Speaker 1: is like a billion dollars.
Speaker 3: It's like.
Speaker 1: There's so many freaking unicorns. It's like heard of unicorns
Speaker 1: at this point was a billion dollar situation.
Speaker 4: There's been inflation since, so quite a bit more money.
Speaker 1: Oually Yeah, like you could probably buy a voter burger
Speaker 1: for a nickel. Not quite, but yeah, there has been
Speaker 1: a lot of inflation. But at the high level in
Speaker 1: AI is pretty intense. As you've seen companies that are
Speaker 1: I don't know, less than a year old getting sometimes
Speaker 1: a billion dollar, multi billion dollar valuations, which I guess
Speaker 1: could pan out and probably will pan out in some cases,
Speaker 1: but it is I ordering to see some of these valuations.
Speaker 3: Yeah, what do you.
Speaker 4: Think, I'm pretty bullish. I'm pretty bullish, honestly.
Speaker 2: So I think the people in this room are going
Speaker 2: to create a lot of the value that a billion
Speaker 2: people in the world should be using this stuff, and
Speaker 2: we're not even we're scratching the surface of it. I
Speaker 2: love the Internet story in that, even back then, you
Speaker 2: are a lot like the people in this room back then,
Speaker 2: in that you know, the heads of all the CEOs
Speaker 2: of all the legacy media companies look to you as
Speaker 2: the person who understood the Internet, and a lot of
Speaker 2: the world, the corporate world, like the world out large,
Speaker 2: that does not understand what's happening with AI. They're going
Speaker 2: to look to the people in this room for exactly that.
Speaker 2: It sounds like, what are some of the tangible lessons.
Speaker 2: It sounds like one of them is don't give up
Speaker 2: board control or be careful about have a really good lawyer.
Speaker 1: Well, I guess with first my first thought of the
Speaker 1: big really the mistake was having too much shareholder and
Speaker 1: board control from legacy media companies who then necessarily see
Speaker 1: things through the lens of legacy media, and that they'll
Speaker 1: make you do things that seems sense full to them
Speaker 1: but aren't really doesn't make sense with the new technology.
Speaker 1: I know I should point out that I that I
Speaker 1: didn't actually first intend to start a company. I try
Speaker 1: to get a job at Netscape. I said my resume
Speaker 1: to Netscape, and Mark Hendrison knows about this, but I
Speaker 1: don't think he gave us my resume, and then nobody responded.
Speaker 1: And then I tried hanging out in the lobby of
Speaker 1: Netscape see if I can like bump into someone, but
Speaker 1: I was like too shy to talk to anyone.
Speaker 3: So I'm like, man, this is ridiculous.
Speaker 1: So I'll just write sof for myself and see how
Speaker 1: it goes so it wasn't actually from the stand point,
Speaker 1: if I want to start a company, I just want
Speaker 1: to be part of building the Internet in some way.
Speaker 1: And since I couldn't get a job at an ARAT company,
Speaker 1: I had to start an internet company anyway. They from
Speaker 1: an AI will so profoundly change the future.
Speaker 3: It's difficult to fathom how much.
Speaker 1: But at the economy is, assuming we don't things don't
Speaker 1: go awry, and like AI doesn't kill us so well
Speaker 1: and itself, then you'll see ultimately an economy.
Speaker 3: That is.
Speaker 1: Not ten times more than the current economy. Ultimately, if
Speaker 1: we become say or whatever, our future machine descendants or
Speaker 1: but mostly machine descendants become like a chief scale to
Speaker 1: civilization or beyond, we're talking about an economy that is
Speaker 1: thousands of times, maybe millions of times bigger than the
Speaker 1: economy today. Yeah, I did feel a bit like when
Speaker 1: I was in VC taking a lot of flag from
Speaker 1: getting rid of waist and fraud, which is an interesting
Speaker 1: side quest as side quests go.
Speaker 4: But I've got to get back to the main quest.
Speaker 3: Yeah, I got to get back to the main quest here,
Speaker 3: So back to the main quest.
Speaker 1: But I did feel a little bit like this. It's
Speaker 1: like fixing the government. It's kind of just say, the
Speaker 1: beach is dirty and there's some needles and PCs and
Speaker 1: like trash, and you want to clean up the beach.
Speaker 1: But then there's also this like thousand foot wool of water,
Speaker 1: which is tsunami of AI. And how much does cleaning
Speaker 1: the beach really matter if you go to a thousand
Speaker 1: foot tsunami about to hit.
Speaker 3: Not that much.
Speaker 4: Well, we're glad you're back on the main quest. It's
Speaker 4: very important.
Speaker 1: Yeah, back to the may quest building technology, which is
Speaker 1: what I like doing. It's just so much noise like this.
Speaker 1: The single noise ratio in politics is terrible.
Speaker 4: I live in San Francisco, so you don't need to
Speaker 4: tell me twice.
Speaker 3: Yeah, DC's I guess it's all politics in DC.
Speaker 1: But the if you're trying to build a rocket or cars,
Speaker 1: or you're trying to have software that compiles and runs reliably,
Speaker 1: then you have to be maximally true seeking or your
Speaker 1: software or your hardware won't work like you can't fool
Speaker 1: like mathem.
Speaker 3: Physics are rigorous judges.
Speaker 1: So I'm used to being in a maximally true seeking
Speaker 1: environment and that's definitely not politics.
Speaker 3: So I'm good. Glad to be back in technology.
Speaker 2: I guess I'm curious going back to the Zip tou moment,
Speaker 2: you had hundreds of millions of dollars or you had
Speaker 2: to exit a worth millions of dollars.
Speaker 3: I got twenty million dollars.
Speaker 2: Okay, so you solved the money problem and you basically
Speaker 2: took it and you rolled. You kept rolling with X
Speaker 2: dot Com, which is in PayPal and Confinity.
Speaker 3: Yes, I kept the chips on the table.
Speaker 2: Yeah, so not everyone does that. A lot of the
Speaker 2: people in this room will have to make that decision. Actually,
Speaker 2: what drove you to jump back into the ring?
Speaker 1: Think I felt for with up too. We'd both like
Speaker 1: incredible technology but never really got to use. I think,
Speaker 1: at least from my perspective, we had better technology than
Speaker 1: say Yahoo or anyone else, but it was constrained by
Speaker 1: our customers. And so I wanted to do something that
Speaker 1: where okay, we wouldn't be constrained by our customers, go
Speaker 1: direct consumer. And that's what ended up being like X
Speaker 1: dot Com PayPal, essentially X dot Com merging with Confinity,
Speaker 1: which together created PayPal, and then that actually the sort
Speaker 1: of PayPal diaspora has it. It might have created more
Speaker 1: companies then, so more companies than probably any anything in
Speaker 1: the twenty first century. So many talented people were at
Speaker 1: the combination of Confinity X dot com.
Speaker 3: So I just wanted to, like, I felt, we got.
Speaker 1: Our wings clift somewhat with ZIP two, and it's okay,
Speaker 1: what if our wings are on clipped and we go
Speaker 1: direct to consumer And that's what PayPal ended up being.
Speaker 1: But yeah, with I got that twenty million dollar check
Speaker 1: for my share of ZIP two. At the time, I
Speaker 1: was living in a house with four housemates and had
Speaker 1: a ten grand in the bank and then this check
Speaker 1: arrives in the mail of all places, and in the
Speaker 1: mail and then now then my bank anounce went from
Speaker 1: ten to twenty million in ten thousand. Okay, sluck to
Speaker 1: pay taxes on that at all. But then I ended
Speaker 1: up putting almost all of that into x dot com
Speaker 1: and as you said, just keeping almost all the chips
Speaker 1: on the table.
Speaker 3: And yeah, and then after PayPal was like, I was
Speaker 3: curious as to why we.
Speaker 1: Had not sent anyone to Mars, and I went on
Speaker 1: the NASA website to find out when we're sending people tomorrow,
Speaker 1: and there was no date. I thought maybe it was
Speaker 1: just hard to find on the website, but in fact,
Speaker 1: there was no real plan to send people to Mars.
Speaker 1: So then I've come to this is such a long story,
Speaker 1: so I don't want to take up too much time here, but.
Speaker 4: I think we're all listening with rapt attention.
Speaker 3: So I was.
Speaker 1: Actually I was on the long end of the expressway
Speaker 1: with my friend day ORESI. We were like housemates in college,
Speaker 1: and DAAs asking me what I'm what we're gonna do?
Speaker 1: What am I gonna do after a PayPal and I
Speaker 1: was like, so, I don't know. I guess maybe I'd
Speaker 1: like to do something philanthropic in space, because I didn't
Speaker 1: think I could actually do anything commercial in space because
Speaker 1: that seemed like the purview of nations. But I'm curious
Speaker 1: as to when we're gonna send people to Mars. And
Speaker 1: that's when I was like, oh, it's not on the website,
Speaker 1: and I started digging. There's nothing on the NAST website.
Speaker 1: So then I started digging in and I'm definitely summarizing
Speaker 1: a lot here. But my first idea was to do
Speaker 1: a philanthropic mission to Mars called Life to Mars, where
Speaker 1: would send a small greenhouse with CS and dehydrated nutrient
Speaker 1: gael land out our mosque and grow hydrate the Jael
Speaker 1: and then you'd have this great sort of money shot
Speaker 1: of green plants on a red background. For the longest time,
Speaker 1: I by the way, I didn't realize money shot. I
Speaker 1: think is a porn reference. But anyway, the point is
Speaker 1: that would be the great shot of green plants on
Speaker 1: a red background. And to try to inspire NASA in
Speaker 1: the public to Santa astronauts to Mars.
Speaker 3: As I learned more, I came to realize along the way.
Speaker 1: By the way, I went to Russia in two thousand
Speaker 1: and one and two thousand and two to buy ICBMs,
Speaker 1: which is that's an adventure where you go and meet
Speaker 1: with Russian high command and say I'd like to buy
Speaker 1: some ICBMs.
Speaker 4: This was to get to space, yeah, a rocket, not.
Speaker 1: To nuke anyone, but they had to as a result
Speaker 1: of arms production talks, they had to actually destroy a
Speaker 1: bunch of their big nuclear pistoles. So I was like,
Speaker 1: how about if we take two of those minus the
Speaker 1: nuke added an additional upper stage for Mars. But it
Speaker 1: was trippy being in Moscow in two thousand and one
Speaker 1: negotiating with the Russian military to buy icvms, like that's crazy,
Speaker 1: but the kettles are raising the price on me. So
Speaker 1: that so literally it's like the opposite of what a
Speaker 1: negotiation to do. So I was like, man, these things
Speaker 1: are getting really expensive. And then I came to realize
Speaker 1: that actually the problem was not that there was insufficient
Speaker 1: world to go to Mars, but there was no way
Speaker 1: to do without breaking the budget, or even breaking the
Speaker 1: NASA budget.
Speaker 3: So that's where I decided to start SpaceX.
Speaker 1: It's SpaceX to advance rocket technology to the point where
Speaker 1: we could send people to Mars.
Speaker 3: And that was in two thousand and two.
Speaker 2: So that wasn't But you didn't start out wanting to
Speaker 2: start a business. You wanted to start just something that
Speaker 2: was interesting to you that you thought humanity needed, and
Speaker 2: then as you kat pulling on a string, it just
Speaker 2: the ball sort of unravels and it turns out this
Speaker 2: is could be a very profitable business.
Speaker 1: It is now, but it there had been no prior
Speaker 1: example of really a rocket startup succeeding. There have been
Speaker 1: various attempts to do commercial rocket companies and then whole
Speaker 1: failed again with SpaceX. Starting SpaceX was really from the
Speaker 1: standpoint of I think there's a less than ten percent
Speaker 1: chance of being successful, maybe one percent, I don't know.
Speaker 1: But if a startup doesn't do something to advance rocket technology,
Speaker 1: it's definitely not coming from the big defense contractors because
Speaker 1: they just impedancemashed to the government, and the government just
Speaker 1: wants to do very conventional things. So it's either coming
Speaker 1: from a startup where it's not happening at all. So
Speaker 1: like a small chance of success is better than no
Speaker 1: chance of success, and so that Yeah, so SpaceX start
Speaker 1: that in mid two thousand and two expecting to fail,
Speaker 1: Like I said, probably ninety percent chance of failing. And
Speaker 1: even like when recruiting people, I didn't try to make
Speaker 1: out that it would probably I said, we're probably going
Speaker 1: to die, but a small chance we might not die.
Speaker 1: But this is the only way to get people to
Speaker 1: Mars and advanced the state of the art. And then
Speaker 1: I ended up being chief engineer of the rocket, not
Speaker 1: because I wanted to, but because I couldn't hire anyone
Speaker 1: who was good, So like none of the good sort
Speaker 1: of chief engineers would join because it's like this is
Speaker 1: too risky, you were going to die. And so then
Speaker 1: I ended up being chief engineer of the rocket, and
Speaker 1: the first three flights did fail, so it's a bit
Speaker 1: of a learning exercise there, and the fourth one fortunately worked.
Speaker 1: But if the fourth one hadn't worked, I had no
Speaker 1: money left and that would have been it would have
Speaker 1: been curtains.
Speaker 3: So it was a pretty close thing.
Speaker 1: If the fourth launch of a Falcon not work, it
Speaker 1: would have been just curtains and we would have just
Speaker 1: been joined the graveyard of prior rocket startups. So it's
Speaker 1: like my estimate of success was not far off. We'd
Speaker 1: just we made it by the skin of our teeth.
Speaker 1: And Tesla was happening simultaneously. Like two thousand and eight
Speaker 1: was a rough year because at mid two thousand and
Speaker 1: eight were called Summer two thousand and eight, the third
Speaker 1: launch of SpaceX had fail, a failure in a row,
Speaker 1: the Tesla financing round had failed, and so Tela was
Speaker 1: going bankrupt fast.
Speaker 3: It was just, man, this is a grim.
Speaker 1: This is going to be a tale of warning of
Speaker 1: an exercise in hubris.
Speaker 2: Probably throughout that period a lot of people were saying
Speaker 2: Elon is a software guy, Why are you working on hardware?
Speaker 4: Why would you yeah, why would he choose to work
Speaker 4: on this.
Speaker 1: Harp said, so you can look at the because they're
Speaker 1: still the press of that time is still online. You
Speaker 1: could just search it. And they kept calling the Internet guy.
Speaker 1: So Internet Guy aka fool is attempting to build a
Speaker 1: rocket company that we got ridiculed quite a lot. And
Speaker 1: it does sound pretty absurd, like internet guy starts rocket
Speaker 1: company doesn't sound like a recipe for success. Frankly, so
Speaker 1: I didn't hold it against them. I was like, yeah,
Speaker 1: admittedly it does sound improbable, and I agree that it's improbable.
Speaker 1: But fortunately the fourth launch worked and a NASSA ordered
Speaker 1: us a contractory supply the space station. And I think
Speaker 1: that was like maybe December twenty second or it was
Speaker 1: it like right before Christmas. Because even the fourth launch
Speaker 1: working wasn't enough to succeed, NASA also needed we also
Speaker 1: needed a big contract to keep us LFE. So I
Speaker 1: got that call from like the NASA team and I
Speaker 1: literally said, we're rewarding you one of the contracts to
Speaker 1: resupply the space station. I like literally blurred it out,
Speaker 1: I love you guys, which is not normally what they
Speaker 1: hear because it's usually pretty sober. But I was like, man,
Speaker 1: this is a company saver and then we closed the
Speaker 1: Tesla financing around on the last hour of the last
Speaker 1: day that it was possible, which was six pm December
Speaker 1: twenty fourth, two thousand and eight. We would have bounced
Speaker 1: payroll two days after Christmas if that round hadn't closed.
Speaker 1: So that was a nerve wracking end of two thousand
Speaker 1: and eight, that's for sure.
Speaker 2: I guess from your PayPal and zipq experience jumping into
Speaker 2: these hardcore hardware startups, it feels like one of the
Speaker 2: through lines was being able to find and eventually attract
Speaker 2: the smartest possible people in those particular fields. What would
Speaker 2: the people in this room, like some of most of
Speaker 2: the people here I don't think have even managed a
Speaker 2: single person yet, they're just starting their careers. What would
Speaker 2: you tell to the elon who's never had to do
Speaker 2: that yet.
Speaker 1: I generally think to try to be as useful as possible.
Speaker 1: It may sound try, but it's it's so hard to
Speaker 1: be useful, especially to be useful to a lot of people.
Speaker 1: Where you say, the area under the cove of total
Speaker 1: utility is like, how how useful have you been to
Speaker 1: your feel human beings? Times how many people? It's almost
Speaker 1: like the physics definition of true work. It's incredibly difficult
Speaker 1: to do that. I think if you aspire to do
Speaker 1: true work, your probably success is much higher. Don't aspire
Speaker 1: to glory, aspire to work.
Speaker 4: How can you tell that it's true work? Is it external?
Speaker 2: Is it like what happens with other people or what
Speaker 2: the product does for people? Like what is that for you?
Speaker 2: When you're looking for people to come work for you?
Speaker 2: What what's the salient thing that you look for or
Speaker 2: if they're.
Speaker 3: As certain question? I guess it's in terms of your
Speaker 3: in product.
Speaker 1: You just have to say, if this thing is successful,
Speaker 1: how useful it be to how many people? And that's
Speaker 1: what I mean. And then you do you do whatever,
Speaker 1: whether you're CEO or any role in a startup, you
Speaker 1: do whatever it takes to succeed, and just always be
Speaker 1: smash smashing your ego like internalized responsibility. Like a major
Speaker 1: failure mode is when ego to ability ratio is double
Speaker 1: graded than sign one.
Speaker 3: If you if your.
Speaker 1: Ego to ability ratio is it gets too high, then
Speaker 1: you're going to basically break the feedback loop to reality.
Speaker 1: And in in AI terms, your your er, you'll have
Speaker 1: your you'll break your RL loop. So you want. You
Speaker 1: want to don't want to break your art. You want
Speaker 1: to have a strong r L loop, which means internalizing
Speaker 1: responsibility and minimizing ego, and you do whatever the task is,
Speaker 1: no matter whether it's grand or humble. That's like why
Speaker 1: I actually I prefer the term like engineering as opposed
Speaker 1: to research. I prefer the term and I don't I
Speaker 1: actually don't want it to call XAI lab.
Speaker 3: I just want to be a company.
Speaker 1: Like It's like, what of the simplest, more straightforward, ideally
Speaker 1: lowist ego terms are that those are generally a good
Speaker 1: way to go. You want you want to just close
Speaker 1: the loop on reality hard that's a super big deal.
Speaker 2: I think everyone in this room is really looks up
Speaker 2: to everything you've done around being a paragon of first
Speaker 2: principles and thinking.
Speaker 4: About the stuff you've done.
Speaker 2: How do you actually determine your reality because that seems
Speaker 2: like a pretty big part of it. Like other people
Speaker 2: who have never made anything, non engineers, sometimes journalists at
Speaker 2: time who've never done anything like they will criticize you.
Speaker 2: But then clearly you have another set of people who
Speaker 2: are builders, who have very high area under the curve,
Speaker 2: who are in your circle. How should people approach that
Speaker 2: what has worked for you and what would you pass
Speaker 2: on to X to your children? What do you tell
Speaker 2: them when you're like you need to make your way
Speaker 2: in this world? Here's how to construct a reality. It
Speaker 2: is predictive from first principles.
Speaker 3: The tools of physics are incredibly helpful to understand and
Speaker 3: make progress in any field. First principles means just obviously,
Speaker 3: just means breakings down to the fundamental axiomatic elements that
Speaker 3: are most likely to be true, and then reason up
Speaker 3: from there as cosionally as possible, as opposed to reasoning
Speaker 3: by analysis or metaphor. And then it just simple things
Speaker 3: like thinking in the limit if you extrapolate, minimize this
Speaker 3: thing or maximize that thing.
Speaker 1: Thinking in the limit is very helpful. I use all
Speaker 1: the tools of physics. They apply to any field. This
Speaker 1: is like a superpower. Actually, so you can take, say,
Speaker 1: take an example like rockets. You can say how much
Speaker 1: should a rocket cost? The typical approach to how to
Speaker 1: that people would take how much rocket should cost?
Speaker 3: There is the would look.
Speaker 1: Historically at what the cost of rockets are and assume
Speaker 1: that any new rocket must be somewhat similar to the
Speaker 1: prior cost of rockets. A first principle's approach would be
Speaker 1: you look at the materials that rocket is comprised of,
Speaker 1: so if that's aluminum, copper, combon, fiber, steel, whatever the
Speaker 1: case may be, and say what what how much does
Speaker 1: that rocket weigh, and what are the constituent elements and
Speaker 1: how much does they weigh? What is the material price
Speaker 1: per kilogram of those constituent elements, and that sense the
Speaker 1: actual floor on what a rocket can cost. It can
Speaker 1: asymptotically approach the cost of the row materials, and then
Speaker 1: you realize, oh, actually a rocket, the row materials of
Speaker 1: a rocket are only maybe one or two percent of
Speaker 1: the historical cost of a rocket, so the manufacturing must
Speaker 1: necessarily be very inefficient if the if the row material
Speaker 1: cost is only one or two percent, would have cost
Speaker 1: principles analysis of the potential for the cost for cost
Speaker 1: optimization of a rocket. And that's before you get to reusability.
Speaker 1: To give an AI sort of AI example last year
Speaker 1: went for XAI when we were trying to build a
Speaker 1: training supercluster. We went to the various suppliers to ask
Speaker 1: it said, this is the beginning last year that we
Speaker 1: needed one hundred thousand, h one hundreds to be able
Speaker 1: to train coherently, and there are estimates for how long
Speaker 1: it would take to complete that were eighteen to twenty
Speaker 1: four months, so we need to get that done in
Speaker 1: six months so then or we won't be competitive. So
Speaker 1: then if you break that down, what are the things
Speaker 1: you need? We need a building, you need power, you
Speaker 1: need cooling. We didn't have enough time to build a
Speaker 1: building from scratch, so we've had to find existing building.
Speaker 1: So we found a factory that was no longer in
Speaker 1: use in Memphis that used to build electrolytes products. But
Speaker 1: then the input power was fifteen megawatts and we needed
Speaker 1: one hundred and fifteen megawats. We read the generators and
Speaker 1: had generators on one side of the building, and then
Speaker 1: we have to have cooling, so we rented about a
Speaker 1: quarter of the mobile cooling capacity of the US and
Speaker 1: put the chillers on the other side of the building.
Speaker 1: But that didn't fully solve the problem because the volted
Speaker 1: the power variations during training are very big, so you
Speaker 1: can have power can drop by fifty percent in one
Speaker 1: hundred milli seconds, which the generators can't keep up with.
Speaker 1: So then we added Tesla megapacks and modified the software
Speaker 1: in the megapacks to be able to just move out
Speaker 1: the power variation during the training run. And then there
Speaker 1: were a bunch of networking challenges the networking cables. If
Speaker 1: you're trying to make one hundred thousand GPUs training coherently
Speaker 1: of very challenging.
Speaker 2: Almost it sounds almost any of those things you mentioned,
Speaker 2: I could imagine someone telling you very directly, no, you
Speaker 2: can't have that, you can't have that power, you can't
Speaker 2: have this. And it sounds like one of the salient
Speaker 2: pieces of first principles thinking is actually, let's ask why,
Speaker 2: let's figure that out, and actually let's challenge the person
Speaker 2: across the table, and if they if I don't get
Speaker 2: an answer that I feel good about, I'm going to
Speaker 2: not allow that to be I'm not going to let
Speaker 2: that know to stand.
Speaker 4: Is that that feels like.
Speaker 2: Something that everyone, if someone were to try to do
Speaker 2: what you're doing in hardware, seems to uniquely need this.
Speaker 4: In software, we have lots of fluff.
Speaker 2: And it's like we can add more CPUs for that,
Speaker 2: it'll be fine, But in hardware it's just not going
Speaker 2: to work.
Speaker 3: I think these general principles of first principle thinking apply
Speaker 3: to software and hardware apply to anything. Really.
Speaker 1: I'm just using a hardware example of how we were
Speaker 1: told something is impossible. But once we broke it down
Speaker 1: into the constituent elements of we need a building, we
Speaker 1: need power, we need cooling, we need we need power smoothing,
Speaker 1: and then we could solve those constituent elements. And then
Speaker 1: we just ran the networking operation to do all the
Speaker 1: cabling everything in four shifts twenty four to seven. And
Speaker 1: I was like sleeping in the data center and also
Speaker 1: doing cabling myself, and there were a lot of other
Speaker 1: issues to solve. Nobody had done a training runt with
Speaker 1: one hundred thousand, h one hundreds training coherently last year.
Speaker 3: I mean maybe it's been done this year. I don't know.
Speaker 1: And then we ended up doubling that to two hundred thousand,
Speaker 1: and so now we've got one hundred and fifty thousand,
Speaker 1: h one hundred fifty k H two hundreds and thirty
Speaker 1: k GB two hundreds in the Memphis training center, and
Speaker 1: we're about to bring one hundred and ten thousand GB
Speaker 1: two hundreds online at a second data center also in
Speaker 1: Memphis area.
Speaker 2: Is it your view that pre training is still working
Speaker 2: and larger. The scaling laws still hold, and whoever wins
Speaker 2: this race will have basically the biggest, smartest possible model
Speaker 2: that you could distill.
Speaker 1: There's other various elements that side competitiveness for large AI
Speaker 1: is for sure, the talent of the people matter. The
Speaker 1: scale of the hardware matters, and how well you're able
Speaker 1: to bring that hardware to bear. So you can't just
Speaker 1: order a whole bunch of GPUs and they don't. You
Speaker 1: can't just plug them in. So you've got to get
Speaker 1: a lot of GPUs and have them training coherently and stavely.
Speaker 3: Then it's what unique access to data do you have.
Speaker 1: I guess distribution matters to some degree as well, like
Speaker 1: how do people get exposed to your AI? Those are
Speaker 1: those are critical pactores for if it's going to be
Speaker 1: like a large foundation model as competitive as many have said.
Speaker 1: I think my friend Eliasitzkayer said, we'd run out of
Speaker 1: pre training data of human generated Like human generated data,
Speaker 1: you run out of tokens pretty fast, of certainly of
Speaker 1: high quality tokens, and then you have to do a
Speaker 1: lot of You need to essentially create synthetic data and
Speaker 1: be able to accurately judge the synthetic data that you're creating,
Speaker 1: Like is this real synthetic data? Well, is it an
Speaker 1: hallucination that doesn't actually match reality? So achieving grounding and
Speaker 1: reality is tricky, but we are at the stage where
Speaker 1: there's more effort put into synthetic data. Right now, we're
Speaker 1: training Grock three point five, which is a heavy focus
Speaker 1: on reasoning.
Speaker 2: Going back to your physics point, what I heard for
Speaker 2: reasoning is the hard science, particularly physics textbooks, are very
Speaker 2: useful for reasoning, whereas I think researchers have told me
Speaker 2: that social science is totally useless for reasoning.
Speaker 3: Yes, that's probably true.
Speaker 1: So, yeah, there's something that's going to be very important
Speaker 1: in the future is combining deep AI in the data
Speaker 1: center or supercluster with robotics. So that's things like the
Speaker 1: Optimist humanoid robot, and yeah, Optimist is awesome.
Speaker 3: There's going to be so.
Speaker 1: Many humanoid robots and robots of all sizes and shapes.
Speaker 1: But my prediction is that there will be more humanoid
Speaker 1: robots by far than all other robots combined by maybe
Speaker 1: an order of magnitude, like a big difference.
Speaker 4: And is it true that you're planning a robot army of.
Speaker 1: The sort, whether we do it or where the Tesla
Speaker 1: does it, because it works closely with XAI. You've seen
Speaker 1: how many humanoid robot startups are there. It's I think
Speaker 1: Jensen bongs on stage with a lot with a massive
Speaker 1: number of robots, robots from different companies. I think it
Speaker 1: was like a dozen different humanoid robots. I guess part
Speaker 1: of what I've been fighting and maybe what has slowed
Speaker 1: me down somewhat is I'm a little don't I don't
Speaker 1: want to make Terminator real, you know, So I I guess,
Speaker 1: at least until recent years, dragging my feet on on
Speaker 1: AI and humanoid robotics, and then I come to realize
Speaker 1: that realization it's happening.
Speaker 3: Whether I do it or not. So you got really
Speaker 3: two choices, participant.
Speaker 1: You could either be a spectator or a participant, and
Speaker 1: so I guess I'd rather be a participant than a spectator.
Speaker 1: So now it's pedals to the metal on humanoid robots
Speaker 1: and digital superintelligence.
Speaker 2: So I guess there's a third thing that everyone has
Speaker 2: heard you talk a lot about that I'm really a
Speaker 2: big fan of becoming a multiplanetary species.
Speaker 4: Where does this fit.
Speaker 2: This is all not just a ten or twenty year thing,
Speaker 2: maybe one hundred year thing, like it's a multi many
Speaker 2: generations for humanity kind of thing.
Speaker 4: How do you think about it?
Speaker 2: There's AI obviously, there's embodied robotics, and then there's being
Speaker 2: a multiplant, multiplanetary species. Does everything feed into that last
Speaker 2: point or what are you driven by right now for
Speaker 2: the next ten to twenty one hundred years?
Speaker 1: Jeez, one hundred years, man. I hope civilizations around in
Speaker 1: one hundred years. If it is around, it's going to
Speaker 1: look very different from civilization today. Not predict that this
Speaker 1: going to be at least five times as many humanoid
Speaker 1: robots as there are humans, maybe ten times. One way
Speaker 1: to look at the progress of civilization is percentage completion
Speaker 1: karda chev So if you're in a kard Chef scale one,
Speaker 1: you've you've honest all the energy of a planet. In
Speaker 1: my opinion, we've only honessed maybe one or two percent
Speaker 1: of Earth's energy, so we've got a long way to
Speaker 1: go to be Kardschef scale one. Then Cardship two you've
Speaker 1: honest all the energy of the Sun, which would be
Speaker 1: I don't know, a billion times more energy than Earth
Speaker 1: maybe closer to a trillion, and then Cardschep three would
Speaker 1: be only energy of a galaxy, pretty far from that.
Speaker 1: So we're at the very early stage of the intelligence
Speaker 1: big bang. I hope we're in terms of being multiplanetary,
Speaker 1: like I think. I think we'll have enough mass transferred
Speaker 1: to Mars within like roughly thirty years to make Mars
Speaker 1: self sustaining such that Mars can continue to grow and
Speaker 1: prosper even if the resupply shifts from Earth stuff coming,
Speaker 1: and that greatly increases the probable lifespan of civilization or
Speaker 1: consciousness and intelligence, both biological and digital. So that's why
Speaker 1: I think it's important to become a multiplanet species. And
Speaker 1: I'm so much troubled by the filmy paradox, like why
Speaker 1: have we not seen any aliens? And it could be
Speaker 1: because intelligence is incredibly rare, and maybe we're the only
Speaker 1: ones in this galaxy, in which case the intelligence of
Speaker 1: consciousness is just the tiny candle in a vast dogness,
Speaker 1: and we should do everything possible to ensure the tiny
Speaker 1: candle does not go out. And being a multiplanet species
Speaker 1: or making consciousness multiplanetary, greatly improves the probable lifespan of civilization,
Speaker 1: and it's the next step before going to other star systems.
Speaker 1: Once you at least have two planets, then you've got
Speaker 1: a forcing function for the improvement of space travel, and
Speaker 1: that ultimately is what will lead to consciousness expanding to
Speaker 1: the stars.
Speaker 2: It could be that the Fermi paradox dictates once you
Speaker 2: get to some level of technology, you destroy yourself. How
Speaker 2: do we see ourselves? How do we actually what would
Speaker 2: you prescribe to a room full of engineers? What can
Speaker 2: we do to prevent that from happening?
Speaker 1: Yeah, how do we avoid the great filters? One of
Speaker 1: the great filters would obviously be global term nuclear war,
Speaker 1: So we should try to avoid that. I guess building
Speaker 1: benign AI robots that AI that loves.
Speaker 3: Humanity, and robots that are helpful.
Speaker 1: Something that I think is extremely important in building AI
Speaker 1: is a very rigorous adherence to truth, even if that
Speaker 1: truth is politically incorrect. My intuition for what could make
Speaker 1: AI very dangerous is if you force AI to believe
Speaker 1: things that are not true.
Speaker 2: How do you think about there's this argument for open
Speaker 2: for safety versus closed for competitive edge. I think the
Speaker 2: great thing is you have a competitive model. Many other
Speaker 2: people also have competitive models, and in that sense we're
Speaker 2: off of. Maybe the worst timeline that I'd be worried
Speaker 2: about is there's fast takeoff and it's only in one
Speaker 2: person's hands that might collapse a lot of things, whereas
Speaker 2: now we have choice, which is great.
Speaker 4: How do you think about this?
Speaker 1: I do think there will be several deep intelligences, maybe
Speaker 1: maybe at least five, maybe this much as ten. I'm
Speaker 1: not sure that there's going to be hundreds, but it's
Speaker 1: probably close. Like maybe it'll be like ten or something
Speaker 1: like that, of which maybe four will be in the
Speaker 1: US so.
Speaker 3: And I don't think it's.
Speaker 1: Going to be anyone AI that that has run away capability.
Speaker 1: But yeah, several deep intelligences.
Speaker 2: What will these deep intelligences actually be doing? Will it
Speaker 2: be scientific research or trying to hack each other?
Speaker 1: Probably all of the above. Hopefully they will discover new physics,
Speaker 1: and I think they're very They're definitely going to invent
Speaker 1: new technologies. I think we're quite close to digital superintelligence.
Speaker 1: It may happen this year, and if it doesn't happen
Speaker 1: this next year, for sure. A digital superintelligence defined as
Speaker 1: smarter than any human at anything.
Speaker 2: So how do we direct that to super abundance we have,
Speaker 2: we could have robotic labor, we have cheap energy, intelligence
Speaker 2: on demand.
Speaker 4: Is that sort of the white pill?
Speaker 3: Like?
Speaker 2: Where do you sit on the spectrum? And are there
Speaker 2: tangible things that you would encourage everyone here to be
Speaker 2: working on to make that white pill actually reality?
Speaker 3: I think it most likely will be a good outcome.
Speaker 1: I guess i'd agree with Jeff Hinton that maybe it's
Speaker 1: a ten to twenty percent chance of annihilation, but look
Speaker 1: on the bright side, that's eighty to ninety percent probably
Speaker 1: of a great outcome. So yeah, I can't emphasize this enough.
Speaker 1: A rigorous adherence to truth is the most important thing
Speaker 1: for AI safety. Safety obviously, empathy for humanity and life
Speaker 1: as we know it.
Speaker 2: We haven't talked about neuralink at all yet, but I'm curious.
Speaker 2: You're working on closing the input and output gap between
Speaker 2: humans and machines. How critical is that to agi ASI?
Speaker 2: And once that link is made, can we not only
Speaker 2: read but also write?
Speaker 1: The neuralink is not necessary to solve digital superintelligence. That'll
Speaker 1: happen before neuralink is at scale. But what neuralink can
Speaker 1: effectively do is solve the output bandwidth constraints especially our output.
Speaker 3: Bandwidth is very low.
Speaker 1: The out the sustained output of a human over the
Speaker 1: course of a day is less than one but per second,
Speaker 1: So there's eighty six four hundred seconds in a day,
Speaker 1: and it's extremely rare for a human to output more
Speaker 1: than that.
Speaker 3: Number of symbols per day, simainly for several days in
Speaker 3: a row.
Speaker 1: So you really within your link interface you can massively
Speaker 1: increase your output bandwidth and your input band with input being.
Speaker 3: Right to you have to do right operations to the brain.
Speaker 1: We have now five humans who have received the kind
Speaker 1: of the read input where it's reading signals, and you've
Speaker 1: got people with als who really have their tetraplegics, but
Speaker 1: they can now communicate at with a similar bandwidth to
Speaker 1: a human with a fully functioning body and control their
Speaker 1: computer and phone, which is pretty cool. And then I
Speaker 1: think in the next six to twelve month we'll be
Speaker 1: doing our first implants for vision, where even if somebody
Speaker 1: is completely blind, we can write directly to the visual cortex.
Speaker 1: And we've had that working in monkeys. Actually, I think
Speaker 1: one of our monkeys now has had a visual implant
Speaker 1: for three years. And at first it'll be relatively fairly
Speaker 1: low resolution, but long term you would have very high
Speaker 1: resolution and be able to see multi spectral wavelengths. You
Speaker 1: could see an infrared ultraviolet radar like a superpower situation,
Speaker 1: but like at some point the cybernetic implants would not
Speaker 1: simply be correcting things that went wrong, but augmenting human
Speaker 1: capabilities dramatically, augmenting intelligence and sensors.
Speaker 3: And bandwidth dramatically.
Speaker 1: And that's going to happen at some point, but digital
Speaker 1: superintelligence will happen well before that.
Speaker 3: At least if we had.
Speaker 1: A neural link, we'll be able to appreciate the AI better.
Speaker 2: I guess one of the limiting reagents to all of
Speaker 2: your efforts across all of these different domains is access.
Speaker 4: To the smartest possible people.
Speaker 2: Yes, simultaneous to that, we have the rocks can talk
Speaker 2: and reason, and they're maybe one hundred and thirty IQ now,
Speaker 2: and they're probably going to be super intelligent soon. How
Speaker 2: do you reconcile those two things. What's going to happen
Speaker 2: in five ten years? And what should the people in
Speaker 2: this room do to make sure that they're the ones
Speaker 2: who are creating instead of maybe below the API line.
Speaker 1: They called it the singularity for a reason, because we
Speaker 1: don't know what's going to happen in the not that
Speaker 1: far future. The percentage of intelligence that is human will
Speaker 1: be quite small. At some point, the collective some of
Speaker 1: human intelligence will be less than one percent of all intelligence.
Speaker 1: And if things get to a clot a ship level two,
Speaker 1: we're talking about human intelligence. Even assuming a significant increase
Speaker 1: in human population and intelligence augmentation like massive intelligence augmentation
Speaker 1: where everyone has an IQ of AE thousand type of thing.
Speaker 1: Even in that circumstance, collective human intelligence will be probably
Speaker 1: one billion that of digital intelligence. Anyway, where the biological
Speaker 1: boot bootloader for digital superintelligence?
Speaker 4: I guess just to end off, was I It was like,
Speaker 4: was I a good bootloader? Where do we go? How
Speaker 4: do we go from here?
Speaker 2: All of this is pretty wild sci fi stuff that
Speaker 2: also could be built by the people in this room.
Speaker 2: But if you do, you have a closing thought for
Speaker 2: the smartest technical people of this generation right now, what
Speaker 2: should they be doing? What should they be working on?
Speaker 2: What should they be thinking about tonight as they go
Speaker 2: to dinner?
Speaker 1: As I saw Rofe with, I think if you're doing
Speaker 1: something useful, that's great.
Speaker 3: If you just try to be as useful.
Speaker 1: As possible to your fellow human beings, and that then
Speaker 1: you're doing something good. I keep harping on this, like
Speaker 1: focus on super truthful AI. That's the most important thing
Speaker 1: for AI safety.
Speaker 3: Obviously.
Speaker 1: If anyone's interested in working at XAI, let me please
Speaker 1: let us know. We're aiming to make Rock the Maximum
Speaker 1: a truth seeking AI, and I think that's.
Speaker 3: A very important thing.
Speaker 1: Hopefully we can understand the nature of the universe, that's
Speaker 1: really what AI can hopefully tell us.
Speaker 3: Maybe AI can.
Speaker 1: Maybe tell us where are the aliens, and how did
Speaker 1: the universe really start?
Speaker 3: How will it end?
Speaker 1: What are the questions that we don't know that we
Speaker 1: should ask and all we in a simulation or what
Speaker 1: level of simulation are we in?
Speaker 2: I think we're going to find out an MPC, Elon,
Speaker 2: thank you so much for joining us. Everyone please give
Speaker 2: it up for Elon Mott.