Speaker 1: I wanted to try to build something useful, but I
Speaker 1: didn't think I would build anything particularly great. You've said,
Speaker 1: probabilistically it seemed unlikely, but I wouldn't to at least try.
Speaker 2: So you're talking to a room full of people who
Speaker 2: are all technical engineers, often some of the most eminent
Speaker 2: AI researchers coming up in the game.
Speaker 3: Okay, I I think we should. I think I like
Speaker 3: the term engineer better than researcher.
Speaker 1: I suppose that if there's some fundamental algorithmic breakthrough, it's
Speaker 1: a research Otherwise it's engineering.
Speaker 2: Maybe let's start way back when you were This is
Speaker 2: a room full of eighteen to twenty five year olds.
Speaker 2: Excuse younger, because the founder set is younger and younger.
Speaker 4: Can you put yourself back into their shoes?
Speaker 2: When you were eighteen nineteen learning to code, even coming
Speaker 2: up with a first idea for ZIP too, What was
Speaker 2: that like for you?
Speaker 3: Yeah?
Speaker 1: Back in ninety five, I was basically choice of either
Speaker 1: do Grass studies PhD a Stanford and material science actually
Speaker 1: working on ultra capacities for potentially use in electric vehicles,
Speaker 1: essentially trying to solve the range problem for electric vehicles,
Speaker 1: or try to do something in this thing that most
Speaker 1: people have never heard of called the Internet. And I
Speaker 1: talked to my professor, was Bill Nicks in the material
Speaker 1: science problement, and I said, can I defer for a
Speaker 1: quarter because this will probably fail and then I'll need
Speaker 1: to come back to college. And then he said, this
Speaker 1: is probably the last conversation we'll have, and he was right.
Speaker 1: But I thought things would most likely failed, not that
Speaker 1: they would most likely succeed.
Speaker 3: And then in.
Speaker 1: Ninety five I wrote, basically, I think the first or
Speaker 1: close to the first maps, directions, Internet, white pages and
Speaker 1: yellow pages on the Internet. I just wrote that personally,
Speaker 1: and I didn't even use website, but I just read
Speaker 1: the boot directly because I couldn't afford and I could
Speaker 1: afford a t one. The original office was on Firman
Speaker 1: Avenue in Palelto. There was like an ISP on the
Speaker 1: floor below. I rolled a hole through the floor and
Speaker 1: just ran a land cable directly to the ISP, and
Speaker 1: my brother joined me and another co founder of Greg Curry,
Speaker 1: who passed away. And at the time we couldn't even
Speaker 1: afford a place to stay, so we just The office
Speaker 1: was five hunred bucks a months, so we just left
Speaker 1: in the office and then shouted the YMCA on page
Speaker 1: mill and al communo and yeah, and I guess we
Speaker 1: ended up doing a little bit of a useful company
Speaker 1: was up to in the beginning, and I did build
Speaker 1: a lot of really really good software technology, but were
Speaker 1: somewhat captured by the legacy media companies and that night
Speaker 1: rout or New York Times, the host whatnot were investors
Speaker 1: and customers and also on the board. So I kept
Speaker 1: wanting to use software in ways that made no sense.
Speaker 1: So I wanted to go direct to consumers anyway. As
Speaker 1: long story dwelling too much on z up too, But
Speaker 1: I really just wanted to do something useful on the
Speaker 1: Internet because I had two choices, do I PhD and
Speaker 1: watch people build the internet or help build the internet
Speaker 1: in some small way. And I was like, I guess
Speaker 1: I can always try and fail and then go back
Speaker 1: to grass Studies and away.
Speaker 3: That ended up being like reasonably successful.
Speaker 1: It sold for like three hundred million dollars, which is
Speaker 1: a lot at the time these days, that's I think
Speaker 1: the minimum impulse, but for an AI startup is like
Speaker 1: a billion dollars.
Speaker 3: It's like.
Speaker 1: There's so many freaking unicorns. It's like heard of unicorns
Speaker 1: at this point is a billion dollar situation.
Speaker 4: There's been inflation since, so quite a bit more money.
Speaker 1: Yeah, like could probably buy a burger for a nickel.
Speaker 1: Not quite, but yeah, there has been a lot of inflation.
Speaker 1: But at the high level in AI is pretty intense.
Speaker 1: As you've seen companies that are I don't know, less
Speaker 1: than a year old getting sometimes a billion dollar, multi
Speaker 1: billion dollar valuations, which I guess could pan out and
Speaker 1: probably will pan out in some cases, but it is
Speaker 1: I wordering to see some of these valuations.
Speaker 4: Yeah, what do you think, I'm pretty bullish. I'm pretty bullish, honestly, so.
Speaker 2: I think the people in this room are going to
Speaker 2: create a lot of the value that a billion people
Speaker 2: in the world should be using this stuff, and we're
Speaker 2: not even we're scratching the surface of it. I love
Speaker 2: the Internet story in that, even back then, you are
Speaker 2: a lot like the people in this room back then,
Speaker 2: in that you know, the heads of all the CEOs
Speaker 2: of all the legacy media companies look to you as
Speaker 2: the person who understood the Internet, and a lot of
Speaker 2: the world, the corporate world, like the world out large,
Speaker 2: that does not understand what's happening with AI, they're going
Speaker 2: to look to the people in this room for exactly that.
Speaker 4: It sounds like, what are some of the tangible lessons.
Speaker 2: It sounds like one of them is don't give up
Speaker 2: board control or be careful about have a really good lawyer.
Speaker 1: Well, I guess, well, the first my first thought of
Speaker 1: the big really the mistake was having too much shareholder
Speaker 1: and board control from legacy media companies who then necessarily
Speaker 1: see things through the lens of legacy media, and that
Speaker 1: they'll make you do things that seems sense full to
Speaker 1: them but aren't really don't make sense within your technology.
Speaker 1: I know I should point out that I that I
Speaker 1: didn't actually first intend to start a company. I try
Speaker 1: to get a job at Netscape. I SIMI resume to
Speaker 1: Netscape and Mark Kandrison knows about this, but I don't
Speaker 1: think you gave us a myristhume. And then nobody responded.
Speaker 1: And then I tried hanging out in the lobby of
Speaker 1: Netscape see if I could like bump into someone, but
Speaker 1: I was like too shy to talk to anyone. So
Speaker 1: I'm like, man, this is ridiculous. So I'll just write
Speaker 1: sof for myself and see how it goes. So it
Speaker 1: wasn't actually from the stand point, if I want to
Speaker 1: start a company, I just want to be part of
Speaker 1: building the Internet in some way. And since I couldn't
Speaker 1: get a job at an area company, I had to
Speaker 1: start a in area company anyway. The AI will so
Speaker 1: profoundly change the future, it's difficult to fathom how much.
Speaker 1: But at the economy is, assuming we don't things don't
Speaker 1: go awry, and like AI doesn't kill us all in itself,
Speaker 1: then you'll see ultimately an economy that is not ten
Speaker 1: times more than the current economy. Ultimately, if we become
Speaker 1: say or whatever, our future machine descendants, or but mostly
Speaker 1: machine descendants become like a ship scale to civilization or beyond,
Speaker 1: we're talking about an economy that is thousands of times,
Speaker 1: maybe millions of times bigger than the economy today. Yeah,
Speaker 1: I did feel a bit like when I was in
Speaker 1: DC taking a lot of flag from getting rid of
Speaker 1: waste and fraud, which is an interesting side quest as
Speaker 1: side quests go.
Speaker 4: But I've got to get back to the main quest.
Speaker 1: Yeah, i gotta get back to the main quest here,
Speaker 1: So back to the main quest. But I did feel
Speaker 1: a little bit like this. It's like fixing the government.
Speaker 1: It's kind of say, the beach is dirty and there's
Speaker 1: some needles and vcs and like trash, and you want
Speaker 1: to clean up the beach. But then there's also this
Speaker 1: like thousand foot wool of water, which is tsunami of AI.
Speaker 1: And how much does cleaning the beach really matter if
Speaker 1: you got a thousand foot tsunami about to hit.
Speaker 3: Not that much.
Speaker 2: Oh well, we're glad you're back on the main quest.
Speaker 2: It's very important.
Speaker 1: Yeah, back to the may quest. Building technology, which is
Speaker 1: what I like doing. It's just so much noise like this.
Speaker 1: The single noise ratio in politics is terrible.
Speaker 4: I live in San Francisco, so you don't need to
Speaker 4: tell me twice.
Speaker 3: Yeah, DC's I guess it's all politics in DC.
Speaker 1: But the if you're trying to build a rocket or cars,
Speaker 1: or you're trying to have software that compiles and runs reliably,
Speaker 1: then you have to be maximally true seeking or your
Speaker 1: software or your hardware won't work like you can't full
Speaker 1: like math and physics are rigorous judges. So I'm used
Speaker 1: to being in a maximally true seeking environment and that's
Speaker 1: definitely not politics. So I'm good, glad to be back
Speaker 1: in technology.
Speaker 2: I guess I'm curious going back to the zip to moment,
Speaker 2: you had hundreds of millions of dollars or you had
Speaker 2: to exit a worth millions of dollars.
Speaker 3: I got twenty million dollars, right.
Speaker 2: Okay, so you solved the money problem at least, and
Speaker 2: you basically took it and you rolled. You kept rolling
Speaker 2: with X dot Com, which in PayPal and Confinity.
Speaker 3: Yes, I kept the chips on the table.
Speaker 4: Yeah, so not everyone does that.
Speaker 2: A lot of the people in this room will have
Speaker 2: to make that decision. Actually, what drove you to jump
Speaker 2: back into the ring?
Speaker 3: O Thay?
Speaker 1: I felt for Up too. We'd both like incredible technology
Speaker 1: but never really got used. I think, at least from
Speaker 1: my perspective, we had better technology than say Yahoo or
Speaker 1: anyone else, but it was constrained by our customers. And
Speaker 1: so I wanted to do something that where okay, we
Speaker 1: wouldn't be constrained by our customers, go direct consumer. And
Speaker 1: that's what ended up being like X dot Com PayPal,
Speaker 1: essentially X dot Com merging with Confinity, which together created PayPal,
Speaker 1: and then that actually the sort of PayPal Diespora has it.
Speaker 1: It might have created more companies then so more companies
Speaker 1: than probaly any anything in the twenty first century, so
Speaker 1: many talented people were at the combination of Confinity X
Speaker 1: dot com. So I just wanted to, like I felt,
Speaker 1: we got our wings clift somewhat with ZIP two, and
Speaker 1: it's okay, what if our wings are clipped and we
Speaker 1: go direct to consumer and that's what PayPal ended up being.
Speaker 1: But yeah, with I got that twenty million dollar check
Speaker 1: for my share of ZIP two. At the time, I
Speaker 1: was living in a house with four housemates, and I
Speaker 1: had a ten grand in the bank, and then this
Speaker 1: check arrived in the mail, of all places in the mail,
Speaker 1: and then now then my bank bounce went from ten
Speaker 1: to twenty million and ten thousand. Okay, seluck to pay
Speaker 1: taxes on that at all. But then I ended up
Speaker 1: putting almost all of that into x dot com and
Speaker 1: as you said, just keeping almost all the chips on
Speaker 1: the table.
Speaker 3: And yeah, and then after PayPal was like, I was
Speaker 3: curious as to.
Speaker 1: Why we had not sent anyone to Mars, and I
Speaker 1: went on the NASA website to find out when we're
Speaker 1: sending people tomorrow, and there was no date. I thought,
Speaker 1: maybe it's just hard to find on the website, but
Speaker 1: in fact, there was no real plan to send people
Speaker 1: to Mars. So then I've come to this is such
Speaker 1: a long story, so I don't want to take up
Speaker 1: too much time here, but I.
Speaker 4: Think we're all listening with rapt attention.
Speaker 3: So I was.
Speaker 1: Actually I was on the long end of Expressway with
Speaker 1: my friend day O Ressi. We were like housemates in college,
Speaker 1: and Daa was asking me what I'm what we're gonna do?
Speaker 1: What am I gonna do after a PayPal and I
Speaker 1: was like, I don't know. I guess maybe I'd like
Speaker 1: to do something philanthropic in space, because I didn't think
Speaker 1: I could actually do anything commotional in space because that
Speaker 1: seemed like the purview of nations. But I'm curious as
Speaker 1: to when we're gonna send people to Mars. And that's
Speaker 1: when I was like, oh, it's not on the website,
Speaker 1: and I started digging. There's nothing on the NAST website.
Speaker 1: So then I started digging in and I'm definitely summarizing.
Speaker 3: A lot here.
Speaker 1: But my first idea was to do a philanthropic mission
Speaker 1: to Mars, Cooled Life to Mars, where would send a
Speaker 1: small greenhouse with CS and dehydrated nutrient gael Land out
Speaker 1: our mosque and grow hydrate the jel and then you
Speaker 1: have this great sort of money shot of green plants
Speaker 1: on a red background. For the longest time, I by
Speaker 1: the way, I didn't realize money shot. I think is
Speaker 1: a porn reference. But anyway, the point is that would
Speaker 1: be the great shot of green plants on a red background,
Speaker 1: and to try to inspire a NASA and the public
Speaker 1: to Santa astronauts to Mars. As I learned more, I
Speaker 1: came to realize along the way. By the way, I
Speaker 1: went to Russia in two thousand and one and two
Speaker 1: thousand and two to buy ICBMs, which is that's an
Speaker 1: adventure where you go and meet with Russian high command
Speaker 1: and say I'd like to buy some ICBMs.
Speaker 4: This was to get to space, yeah, as a rocket, not.
Speaker 1: To nuke anyone, but they had to as a result
Speaker 1: of armstroduction talks. They had to actually destroy a bunch
Speaker 1: of their big nuclear missiles. So I was like, how
Speaker 1: about if we take two of those minus the luke
Speaker 1: added an additional opper stage for Mars.
Speaker 3: But it was trippy.
Speaker 1: Being in Moscow in one of two thousand and one
Speaker 1: negotiating with the Russian military to buy icvms, like that's crazy,
Speaker 1: but the kept also raising the price on me, so
Speaker 1: that so literally it's like the opposite of what a
Speaker 1: negotiation to do. So I was like, man, these things
Speaker 1: are getting really expensive. And then I came to realize
Speaker 1: that actually the problem was not that there was insufficient
Speaker 1: will to go to Mars, but there was no way
Speaker 1: to do without breaking the budget, or even breaking the
Speaker 1: NASA budget. So that's where I decided to start SpaceX's
Speaker 1: SpaceX to advance rocket technology to the point where we
Speaker 1: could send people to Mars.
Speaker 3: And that was in two thousand and two.
Speaker 2: So that wasn't But you didn't start out wanting to
Speaker 2: start a business. You wanted to start just something that
Speaker 2: was interesting to you that you thought humanity needed, and
Speaker 2: then as you kat pulling on a string, it just
Speaker 2: the ball sort of unravels and it turns out this
Speaker 2: is could be a very profitable business.
Speaker 1: It is now, but it there had been no prior
Speaker 1: example of really a rocket startup succeeding, but there have
Speaker 1: been various attempts to do commercial rocket companies, and the
Speaker 1: whole failed again with SpaceX starting. SpaceX was really from
Speaker 1: the standpoint of I think there's a less than ten
Speaker 1: percent chance of being successful, maybe one percent, I don't know.
Speaker 1: But if a startup doesn't do something to advance rocket technology,
Speaker 1: it's definitely not coming from the big defense contractors because
Speaker 1: they just impedance smashed to the government, and the government
Speaker 1: just wants to do very conventional things. So it's either
Speaker 1: coming from a startup where it's not happening at all.
Speaker 1: So like a small chance of success is better than
Speaker 1: no chance of success, and so that yeah, so SpaceX
Speaker 1: start that in mid two thousand and two expecting to fail,
Speaker 1: like I said, probably ninety percent chance of failing. And
Speaker 1: even like when recruiting people, I didn't try to make
Speaker 1: out that it would I said, we're probably going to die,
Speaker 1: but a small.
Speaker 3: Chance we might not die.
Speaker 1: But this is the only way to get people to
Speaker 1: Mars and advanced the state of the art. And then
Speaker 1: I ended up being chief engineer of the rocket, not
Speaker 1: because I wanted to, but because I couldn't hire anyone
Speaker 1: who was good, So like none of the good sort
Speaker 1: of chief engineers would join because it's like this is
Speaker 1: too risky, you were going to die. And so then
Speaker 1: I ended up being chief engineer of the rocket, and
Speaker 1: the first three flights did fail, so it's a bit
Speaker 1: of a learning exercise there, and fourth one fortunately worked.
Speaker 1: But if the fourth one hadn't worked, I had no
Speaker 1: money left and that would have been it would have
Speaker 1: been curtained. So it was a pretty close thing. If
Speaker 1: the fourth launch of a Falcon not work, it would
Speaker 1: have been just curtains and we would have just been
Speaker 1: joined the graveyard of prior rocket startups. So it's like
Speaker 1: my estimate of success was not far off. We'd just
Speaker 1: be made it by the skin of our teeth. And
Speaker 1: Tesla was happening simultaneously, Like two thousand and eight was
Speaker 1: a rough year because at mid two thousand and eight
Speaker 1: were called Summer two thousand and eight, the third launch
Speaker 1: of Spacexit failed, a third failure in a row. The
Speaker 1: Tesla financing round had failed, and so Telsa was going
Speaker 1: bankrupt fast. It was just, man, this is a grim.
Speaker 1: This is going to be a tale of warning of
Speaker 1: an exercise in hubris.
Speaker 2: Probably throughout that period a lot of people were saying,
Speaker 2: Elon is a software guy, Why are you working on hardware?
Speaker 4: Why would you yeah, why would he choose to work
Speaker 4: on this.
Speaker 1: Harp said, so you can look at the because they're
Speaker 1: still the press of that time is still online. You
Speaker 1: could just search it. And they kept calling the Internet guy.
Speaker 1: So Internet Guy aka.
Speaker 3: Fool is attempting to build a rocket company that we
Speaker 3: got ridiculed quite a lot.
Speaker 1: And it does sound pretty absurd, like internet guy starts
Speaker 1: rocket company doesn't sound like a recipe for success, frankly,
Speaker 1: so I didn't hold against them. I was like, yeah,
Speaker 1: admittedly it does sound improbable, and I agree that it's improbable.
Speaker 1: But fortunately the fourth launch worked and a NASSA ordered
Speaker 1: us a contract supply the space station. And I think
Speaker 1: that was like maybe December twenty second or it was
Speaker 1: it like right before Christmas. Because even the fourth launch
Speaker 1: working wasn't enough to succeed, NASA also needed we also
Speaker 1: needed a big contract to keep us life. So I
Speaker 1: got that call from like the NASA team and I
Speaker 1: literally they said, we're rewarding you one of the contracts
Speaker 1: to re supply the space station. I like literally blurred
Speaker 1: it out, I love you guys, which is not normally
Speaker 1: what they hear because it's usually pretty sober. But I
Speaker 1: was like, man, this is a company saver and then
Speaker 1: we closed the Tesla financing round on the last hour
Speaker 1: of the last day that it was possible, which was
Speaker 1: six pm December twenty fourth, two thousand and eight. We
Speaker 1: would have bounced payroll two days after Christmas if that
Speaker 1: round hadn't closed. So that was a nerve racking end
Speaker 1: of two thousand and eight, that's for sure.
Speaker 2: I guess from your PayPal and zip two experience jumping
Speaker 2: into these hardcore hardware startups, it feels like one of
Speaker 2: the through lines was being able to find and eventually
Speaker 2: attract the smartest possible people in those particular fields. What
Speaker 2: would the people in this room like some of the
Speaker 2: most of the people here I don't think have even
Speaker 2: managed a single person yet, they're just starting their careers.
Speaker 2: What would you tell to the elon who's never had
Speaker 2: to do that yet.
Speaker 1: I generally think to try to be as useful as possible.
Speaker 1: It may sound try, but it's so hard to be useful,
Speaker 1: especially to be useful to a lot of people. Where
Speaker 1: you say, the area under the cove of total utility
Speaker 1: is like how much how useful have you been to
Speaker 1: your fela human beings? Times how many people? It's almost
Speaker 1: like the physics definition of true work. It's incredibly difficult
Speaker 1: to do that. And I think if you're aspire to
Speaker 1: do true work, your probably success is much higher. Don't
Speaker 1: aspire to glory, aspire to work.
Speaker 2: How can you tell that it's true work? Is it external?
Speaker 2: Is it like what happens with other people or what
Speaker 2: the product does for people? Like what is that for you?
Speaker 2: When you're looking for people come work for you? What's
Speaker 2: the salient thing that you look for or if they're
Speaker 2: certain question?
Speaker 1: I guess it's in terms of if you're in product,
Speaker 1: you just have to say, if this thing is successful,
Speaker 1: how useful will it be to how many people? And
Speaker 1: that's what I mean. And then you do you do whatever,
Speaker 1: whether you're CEO or any role in a startup, you
Speaker 1: do whatever it takes to succeed, and just always be
Speaker 1: smash smashing your ego like internalized responsibility like a major
Speaker 1: failure mode is when ego to ability ratio is double
Speaker 1: graded than sign one. If you if your ego to
Speaker 1: ability ratio is it gets too high, then you're going
Speaker 1: to basically break the feedback loop to reality. And in
Speaker 1: in a it terms, your your er you'll have you'll
Speaker 1: break your RL loop so you want you want to
Speaker 1: don't want to break your art. You want to have
Speaker 1: a strong r L loop, which means internalizing responsibility and
Speaker 1: minimizing ego, and you do whatever the task is, no
Speaker 1: matter whether it's grand or humble. That's like what I
Speaker 1: actually I prefer the term like engineering as opposed to research.
Speaker 1: I prefer the term and I don't actually don't want
Speaker 1: it to call XAI lab.
Speaker 3: I just want to be a company.
Speaker 1: Like It's like, what of the simplest, more straightforward, ideally
Speaker 1: lowest that ego terms are that those are generally a
Speaker 1: good way to go. You want you want to just
Speaker 1: close the loop on reality hard that's a super big deal.
Speaker 2: I think everyone in this room is really looks up
Speaker 2: to everything you've done around being a paragon of first
Speaker 2: principles and thinking about the stuff you've done. How do
Speaker 2: you actually determine your reality because that seems like a
Speaker 2: pretty big part of it.
Speaker 4: Like other people who have never made.
Speaker 2: Anything, non engineers, sometimes journalists at time who've never done
Speaker 2: anything like they will criticize you. But then clearly you
Speaker 2: have another set of people who are builders, who have
Speaker 2: very high area under the curve, who are in your
Speaker 2: circle how should people approach that? What has worked for you?
Speaker 2: And what would you pass on to X to your children?
Speaker 2: What do you tell them when you're like, you need
Speaker 2: to make your way in this world? Here's how to
Speaker 2: construct a reality. It is predictive from first principles.
Speaker 1: The tools of physics are incredibly helpful to understand and
Speaker 1: make progress in any field. The first principles means just obviously,
Speaker 1: just means breakings down to the fundamental axiomatic elements that
Speaker 1: are most likely to be true, and then reason up
Speaker 1: from there as cosionally as possible, as opposed to reasoning
Speaker 1: by analysis or metaphor. And then it just simple things
Speaker 1: like thinking in the limit if you extrapolate, minimize this
Speaker 1: thing or maximize that thing. Thinking in the limit is
Speaker 1: very helpful. I use all the tools of physics. They
Speaker 1: apply to any field. This is like a superpower. Actually,
Speaker 1: so you can take, say, take an example like rockets.
Speaker 1: You can say how much should a rocket cost? A
Speaker 1: typical approach to how to that people would take to
Speaker 1: how much rocket should cost is they would look historically
Speaker 1: at what the cost of rockets are and assume that
Speaker 1: any new rocket must be somewhat similar to the prior
Speaker 1: cost of rockets. A first principle's approach would be you
Speaker 1: look at the materials that the rocket is comprised of,
Speaker 1: so if that's aluminium, copper, combon, fiber, steel, whatever the
Speaker 1: case may be, and say what what how much does
Speaker 1: that rocket weigh, and what are the constituent elements and
Speaker 1: how much do they weigh? What is the material price
Speaker 1: per kilogram of those constituent elements. In that sense the
Speaker 1: actual floor on what a rocket can cost. It can
Speaker 1: asymptotically approach the cost of the row materials and then
Speaker 1: you realize, oh, actually a rocket, the row materials of
Speaker 1: a rocket are only maybe one or two percent of
Speaker 1: the historical cost of a rocket, so the manufacturing must
Speaker 1: necessarily be very inefficient if the if the row material
Speaker 1: cost is only one or two percent. However, you have
Speaker 1: first principles analysis of the potential for the cost for
Speaker 1: cost optimization of a rocket.
Speaker 3: And that's before you get to reusability.
Speaker 1: To give an AI sort of AI example last year
Speaker 1: went for x CI when we were trying to build
Speaker 1: a training supercluster. We went to the various suppliers to
Speaker 1: ask said, this is the beginning of last year that
Speaker 1: we needed one hundred thousand, eighth one hundreds to be
Speaker 1: able to train coherently. And there are estimates for how
Speaker 1: long it would take to complete that were eighteen to
Speaker 1: twenty four months, so we need to get that done
Speaker 1: in six months so then or we won't be competitive.
Speaker 1: So then if you break that down, what are the
Speaker 1: things you need? We need a building, you need power,
Speaker 1: you need cooling. We didn't have enough time to build
Speaker 1: a building from scratch, so we've had to find existing building.
Speaker 1: So we found a factory that was no longer in
Speaker 1: use in Memphis that used to build electrolytes products. But
Speaker 1: then the input power was fifteen megawats and we needed
Speaker 1: one hundred and fifteen megawats. We read the generators and
Speaker 1: had generators on one side of the building, and then
Speaker 1: we have to have cooling, so we rented about a
Speaker 1: quarter of the mobile cooling capacity of the US and
Speaker 1: put the chillers on the other side of the building.
Speaker 1: But that didn't fully solve the problem because the volta
Speaker 1: the power variations during training are very big, so you
Speaker 1: can have power can drop by fifty percent in a
Speaker 1: hundred most seconds, which the generators can't keep up with.
Speaker 1: So then we added Tesla megapacks and modified the software
Speaker 1: in the megapacks to be able to just mooth out
Speaker 1: the power variation during the training run. And then there
Speaker 1: were a bunch of networking challenges. Is the networking cables.
Speaker 1: If you're trying to make one hundred thousand GPUs training
Speaker 1: coherently of very challenging.
Speaker 2: Almost it sounds almost any of those things you mentioned.
Speaker 2: I could imagine someone telling you very directly, no, you
Speaker 2: can't have that, you can't have that power, you can't
Speaker 2: have this. And it sounds like one of the salient
Speaker 2: pieces of first principles thinking is actually, let's ask why,
Speaker 2: let's figure that out, and actually let's challenge the person
Speaker 2: across the table and they If I don't get an
Speaker 2: answer that I feel good about, I'm going to not
Speaker 2: allow that to be.
Speaker 4: I'm not going to let that know to stand. Is
Speaker 4: that that feels like.
Speaker 2: Something that everyone, if someone were to try to do
Speaker 2: what you're doing in hardware, seems to uniquely need this.
Speaker 2: In software, we have lots of fluff things that it's
Speaker 2: like we can add more CPUs for that, it'll be fine,
Speaker 2: But in hardware it's just not going to work.
Speaker 3: I think these general principles of first principle thinking apply
Speaker 3: to software and hardware apply to anything. Really.
Speaker 1: I'm just using a hardware example of how we were
Speaker 1: told something is impossible. But once we broke it down
Speaker 1: into the constituent elements of we need a building, we
Speaker 1: need power, we need cooling, we need we need power smoothing,
Speaker 1: and then we could solve those constituent elements. But and
Speaker 1: then we just ran the networking operation to do all
Speaker 1: the cabling everything in four shifts twenty four to seven,
Speaker 1: and I was like sleeping the data center and also
Speaker 1: doing cabling myself, and there were a lot of other
Speaker 1: issues to solve. Nobody had done a training runt with
Speaker 1: one hundred thousand, h one hundreds training coherently last year.
Speaker 3: I mean maybe it's been done this year. I don't know.
Speaker 3: And then we ended up doubling that.
Speaker 1: To two hundred thousand, and so now we've got one
Speaker 1: hundred and fifty thousand, h one hundred fifty k H
Speaker 1: two hundreds and thirty k GB two hundreds in the
Speaker 1: Memphis training cent and we're about to bring one hundred
Speaker 1: and ten thousand GB two hundreds online at a second
Speaker 1: data center, also in Memphis.
Speaker 3: Area.
Speaker 2: Is it your view that pre training is still working
Speaker 2: and larger the scaling laws still hold, and whoever wins
Speaker 2: this race will have basically the biggest, smartest possible model
Speaker 2: that you could distill.
Speaker 1: There's other various elements that side competitiveness for large AI
Speaker 1: is for sure, the talent of the people matter. The
Speaker 1: scale of the hardware matters, and how well you're able
Speaker 1: to bring that hardware to bear. So you can't just
Speaker 1: order a whole bunch of GPUs and they don't. You
Speaker 1: can't just plug them in, So you've got to get
Speaker 1: a lot of gps and have them training coherently and stably.
Speaker 1: Then it's what unique access to data do you have.
Speaker 1: I guess distribution matters to some degree as well, like
Speaker 1: how do people get exposed to your AI? Those are
Speaker 1: those are critical pactores for if it's going to be
Speaker 1: like a large foundation model as competitive as many have said.
Speaker 1: I think my friend eliasit Skier said, we would run
Speaker 1: out of pre training data of human generated Like human
Speaker 1: generated data, you run out of tokens pretty fast, certainly
Speaker 1: of high quality tokens, and then you have to do
Speaker 1: a lot of You need to essentially create synthetic data
Speaker 1: and be able to accurately judge the synthetic data that
Speaker 1: you're creating to like, is this real synthetic data? Well,
Speaker 1: is it an hallucination that doesn't actually mash reality? So
Speaker 1: achieving grounding and reality is tricky, but we are at
Speaker 1: the stage where there's more effort put into synthetic data.
Speaker 1: Right now, we're training Grock three point five, which is
Speaker 1: a heavy focus on reasoning.
Speaker 2: Going back to your physics point, what I heard for
Speaker 2: reasoning is that hard science, particularly physics textbooks, are very
Speaker 2: useful for reasoning, whereas I think researchers have told me
Speaker 2: that social science is totally useless for reasoning.
Speaker 3: Yes, that's probably true.
Speaker 1: So yeah, there's something that's going to be very important
Speaker 1: in the future is combining deep AI in the data
Speaker 1: center of Supercluster with robotics. So that's things like the
Speaker 1: Optimius humanoid robot, and yeah, Optimist is awesome. There's going
Speaker 1: to be so many humanoid robots and robots of all
Speaker 1: sizes and shapes. But my prediction is that there will
Speaker 1: be more humanoid robots by far than all other robots
Speaker 1: combined by maybe an order of magnitude, like a big difference,
Speaker 1: and is it true that.
Speaker 4: You we're planning a robot army of.
Speaker 1: The sort, whether we do it or whether the Tesla
Speaker 1: does it, because it works closely with XAI. You've seen
Speaker 1: how many humanoid robot startups are there. It's I think
Speaker 1: Jensen bongs on stage with a lot with a massive
Speaker 1: number of robots, robots from different companies. I think it
Speaker 1: was like a dozen different humanoid robots. I guess part
Speaker 1: of what I've been fighting and maybe what has slowed
Speaker 1: me down somewhat that I'm a little I don't want
Speaker 1: to make Terminator real, you know, So I I guess,
Speaker 1: at least until recent years, dragging my feet on on
Speaker 1: AI and humanoid robotics, and then I come to realize
Speaker 1: that realization it's happening.
Speaker 3: Whether I do it or not. So you got really
Speaker 3: two choices participate.
Speaker 1: You could either be a spectator or a participant, And
Speaker 1: so I guess I'd rather be a participant than a spectator.
Speaker 1: So now it's pedal to the metal on humanoid robots
Speaker 1: and digital superintelligence.
Speaker 2: So I guess there's a third thing that everyone has
Speaker 2: heard you talk a lot about that I'm really a
Speaker 2: big fan of becoming a multiplanetary species.
Speaker 4: Where does this fit?
Speaker 2: This is all not just a ten or twenty year thing,
Speaker 2: maybe one hundred year thing, like it's a multi many
Speaker 2: generations for humanity kind of thing.
Speaker 4: How do you think about it?
Speaker 2: There's AI obviously, there's embodied robotics, and then there's being
Speaker 2: a multiplant, multiplanetary species. Does everything feed into that last
Speaker 2: point or what are you driven by right now for
Speaker 2: the next ten to twenty one hundred years?
Speaker 1: Jeez, one hundred years, man. I hope civilizations around in
Speaker 1: one hundred years. If it is around, it's going to
Speaker 1: look very different from civilization today. Not predict that this
Speaker 1: going to be at least five times as many humanoid
Speaker 1: robots as there are humans, maybe ten times. One way
Speaker 1: to look at the progress of civilization is percentage completion Kardaschev.
Speaker 1: So if you're in a kars Chef scale one, you've
Speaker 1: you've honessed all the energy of a planet. Now, in
Speaker 1: my opinion, we've only honested maybe one or two percent
Speaker 1: of Earth's energy, So we've got a long way to
Speaker 1: go to be Kardschev scale one. Then Cardship two you've
Speaker 1: honest only energy of the Sun, which would be I
Speaker 1: don't know a billion times more energy than Oath, maybe
Speaker 1: closer to a trillion, and then Karschev three would be
Speaker 1: only energy of a galaxy pretty far from that. So
Speaker 1: we're at the very early stage of the intelligence big bang.
Speaker 1: I hope we're in terms of being multiplanetary, like I think,
Speaker 1: I think we'll have enough mass transferred to Mars within
Speaker 1: like roughly thirty years to make Mars self sustaining such
Speaker 1: that Mars can continue to grow and prosper even if
Speaker 1: the resupply shifts from both stuff coming, and that greatly
Speaker 1: increases the probable lifespan of civilization or consciousness and intelligence,
Speaker 1: both biological and digital. So that's why I think it's
Speaker 1: important to become a multiplanet species.
Speaker 3: And I'm so much.
Speaker 1: Troubled by the filmy paradox, like why have we not
Speaker 1: seen any aliens? And it could be because intelligence is
Speaker 1: incredibly rare, and maybe we're the only ones in this galaxy,
Speaker 1: in which case the intelligence of consciousness is just the
Speaker 1: tiny candle in a vast dokness, and we should do
Speaker 1: everything possible to ensure the tiny candle does not go out.
Speaker 1: And being a multiplanet species or making consciousness multiplanetary greatly
Speaker 1: improves the probable lifespan of civilization, and it's the next
Speaker 1: step before going to other star systems. Once you at
Speaker 1: least have two planets, then you've got a forcing function
Speaker 1: for the improvement of space travel, and that ultimately is
Speaker 1: what will lead to consciousness expanding to the stars.
Speaker 2: It could be that the Fermi paradox dictates once you
Speaker 2: get to some level of technology, you destroy yourself.
Speaker 4: How do we see ourselves?
Speaker 2: How do we actually what would you prescribe to a
Speaker 2: room full of engineers? What can we do to prevent
Speaker 2: that from happening?
Speaker 3: Yeah?
Speaker 1: How do we avoid the great filters? One of the
Speaker 1: great filtres would obviously be global thom nuclear war, So
Speaker 1: we should try to avoid that. I guess building benign
Speaker 1: AI robots that AI that loves humanity, and robots that
Speaker 1: are helpful. Something that I think is extremely important in
Speaker 1: building AI is a very rigorous adherence to truth, even
Speaker 1: if that truth is politically incorrect. My intuition for what
Speaker 1: could make AI very dangerous is if you force AI
Speaker 1: to believe things that are not true.
Speaker 2: How do you think about there's this argument for open
Speaker 2: for safety versus closed for competitive edge. I think the
Speaker 2: great thing is you have a competitive model. Many other
Speaker 2: people also have competitive models, and in that sense we're
Speaker 2: off of. Maybe the worst timeline that I'd be worried
Speaker 2: about is there's fast takeoff and it's only in one
Speaker 2: person's hands that might collapse a lot of things, Whereas
Speaker 2: now we have choice, which is great. How do you
Speaker 2: think about this?
Speaker 1: Yeah, I do think there will be several deep intelligences,
Speaker 1: maybe maybe at least five, maybe as much as ten.
Speaker 1: I'm not sure that there's going to be hundreds, but
Speaker 1: it's probably close. Like maybe there'll be like ten or
Speaker 1: something like that, of which maybe four will be in
Speaker 1: the US so.
Speaker 3: And I don't think it's going.
Speaker 1: To be anyone AI that has run away capability. But yeah,
Speaker 1: several deep intelligences.
Speaker 2: What will these deep intelligences actually be doing? Will it
Speaker 2: be scientific research or trying to hack each other?
Speaker 3: Probably old of the above.
Speaker 1: Hopefully they will discover new physics, and I think they're
Speaker 1: very they're definitely going to invent new technologies. I think
Speaker 1: we're quite close to digital superintelligence. It may happen this year,
Speaker 1: and if it doesn't happen this next year, for sure,
Speaker 1: a digital superintelligence to find as smarter than any human
Speaker 1: at anything, So.
Speaker 2: How do we direct that to super abundance. We could
Speaker 2: have robotic labor, we have cheap energy, intelligence on demand.
Speaker 2: Is that sort of the white pill? Like, where do
Speaker 2: you sit on the spectrum? And are there tangible things
Speaker 2: that you would encourage everyone here to be working on
Speaker 2: to make that white pill actually reality?
Speaker 1: I think it most likely will be a good outcome.
Speaker 1: I guess I'd agree with Jeff Hinton that maybe it's
Speaker 1: a ten to twenty percent chance of annihilation, but look
Speaker 1: on the right side, that's eighty to ninety percent probably
Speaker 1: of a great outcome. So yeah, I can't emphasize this enough.
Speaker 1: A rigorous adherence to truth. It is the most important
Speaker 1: thing for AI safety safety obviously, empathy for humanity and
Speaker 1: life as we know it.
Speaker 2: We haven't talked about neuralink at all yet, but I'm curious.
Speaker 2: You're working on closing the input and output gap between
Speaker 2: humans and genes. How critical is that to AI ASI?
Speaker 2: And once that link is made, can we not only
Speaker 2: read but also write?
Speaker 1: The neuralink is not necessary to solve digital superintelligence. That'll
Speaker 1: happen before neuralink is at scale, But nor neuralink can
Speaker 1: effectively do is solve the input output bandwidth constraints.
Speaker 3: Especially our output bandwidth is very low. The out the
Speaker 3: sustained output.
Speaker 1: Of a human over the course of a day is
Speaker 1: less than one but per second, so there's eighty six
Speaker 1: four hundred seconds in a day, and it's extremely rare
Speaker 1: for a human to output more than that number of
Speaker 1: symbols per day, simainly for several days in a row.
Speaker 1: So you really within your link interface you can massively
Speaker 1: increase your output bandwidth and your input band with input
Speaker 1: being right to you have to do right operations to
Speaker 1: the brain. We have now five humans who have received
Speaker 1: the kind of the read input where it's reading signals,
Speaker 1: and you've got people with als who really have their tetraplegics,
Speaker 1: but they can now communicate at with a similar bandwidth
Speaker 1: to a human with a fully functioning body and control
Speaker 1: their computer and the phone, which is pretty cool. And
Speaker 1: then I think in the next six to twel months
Speaker 1: we will be doing our first implants for vision where
Speaker 1: even if somebody is completely blind, we can write directly
Speaker 1: to the visual cortex, and we've had that working in monkeys. Actually,
Speaker 1: I think one of our monkeys now has had a
Speaker 1: visual implant for three years, and at first it'll be
Speaker 1: relatively fairly low resolution, but long term you would have
Speaker 1: very high resolution and be able to see multi spectral wavelengths.
Speaker 1: You could see an infrared ultra violet radar like a
Speaker 1: superpower situation, but like at some point the cybernetic implants
Speaker 1: would not simply be correcting things that went wrong, but
Speaker 1: augmenting human capabilities dramatically, augmenting intelligence and sensors and bandwidth dramatically.
Speaker 1: And that's going to happen at some point, but digital
Speaker 1: superintelligence will happen well before that. At least if we're
Speaker 1: a neur link, we'll be able to appreciate the AI better.
Speaker 2: I guess one of the limits reagents to all of
Speaker 2: your efforts across all of these different domains is access to.
Speaker 4: The smartest possible people.
Speaker 2: Yes, simultaneous to that, we have the rocks can talk
Speaker 2: and reason, and they're maybe one hundred and thirty IQ now,
Speaker 2: and they're probably going to be super intelligent soon. How
Speaker 2: do you reconcile those two things. What's going to happen
Speaker 2: in five ten years? And what should the people in
Speaker 2: this room do to make sure that they're the ones
Speaker 2: who are creating instead of maybe below the API line.
Speaker 1: They called it the singularity for a reason, because we
Speaker 1: don't know what's going to happen in the not that
Speaker 1: far future. The percentage of intelligence that is human will
Speaker 1: be quite small. At some point, the collective some of
Speaker 1: human intelligence will be less than one percent of all intelligence.
Speaker 1: And if things get to a clot a ship level two,
Speaker 1: we're talking about human intelligence. Even assuming a significant increase
Speaker 1: in human population and intelligence augmentation like massive intelligence augmentation
Speaker 1: where everyone has an IQ of one thousand type of thing.
Speaker 1: Even in that circumstance, collective human intelligence will be probably
Speaker 1: one billion that of digital intelligence. Anyway, we're the biological
Speaker 1: boot bootloader for digital superintelligence.
Speaker 4: I guess just to end off, was I?
Speaker 3: It was like, was I a good bootload?
Speaker 4: Where do we go? How do we go from here?
Speaker 2: All of this is pretty wild sci fi stuff that
Speaker 2: also could be built by the people in this room.
Speaker 2: But if you do, you have a closing thought for
Speaker 2: the smartest technical people of this generation right now? What
Speaker 2: should they be doing, what should they be working on?
Speaker 2: What should they be thinking about tonight as they go
Speaker 2: to dinner?
Speaker 1: As I saw a rope with I think if you're
Speaker 1: doing something useful, that's great. If you just try to
Speaker 1: be as useful as possible to your fellow human beings
Speaker 1: and that then you're doing something good. I keep harping
Speaker 1: on this, like focus on truthful AI. That's the most
Speaker 1: important thing for AI safety. Obviously, if anyone's interested in
Speaker 1: working at xailet mean, please let us know. We're aiming
Speaker 1: to make GROT the maximumly truth seeking AI, and I
Speaker 1: think that's a very important thing. Hopefully we can understand
Speaker 1: the nature of the universe, that's really what AI can
Speaker 1: hopefully tell us. Maybe AI can maybe tell us where
Speaker 1: are the aliens, and how did the universe really start?
Speaker 3: How will it end?
Speaker 1: What are the questions that we don't know that we
Speaker 1: should ask, And are we in a simulation or what
Speaker 1: level of.
Speaker 3: Similation are we in?
Speaker 2: I think we're going to find out. APC, Elon, thank
Speaker 2: you so much for joining us. Everyone please give it
Speaker 2: up for Elon Mott