Speaker 1: Welcome to the Grock four release. Here, this is the.
Speaker 2: Smartest AI in the world. We're going to show you
Speaker 2: exactly how and why. It really is remarkable to see
Speaker 2: the advancement of artificial intelligence, how quickly it is evolving.
Speaker 2: I sometimes compare it to the growth of a human
Speaker 2: and how faster human learns and gains conscious awareness and understanding.
Speaker 2: And AI is advancing just vastly faster than any human.
Speaker 2: I mean, we're going to take you through a bunch
Speaker 2: of benchmarks that that GROC four is able to achieve
Speaker 2: incredible numbers on.
Speaker 1: But it's actually worth noting.
Speaker 2: That that GROCK four, if given like the SAT, would
Speaker 2: get perfect SATs every time, even if it's never seen
Speaker 2: the questions before, and if even going beyond that, to
Speaker 2: say like graduate student exams like the GRE, it will
Speaker 2: get near perfect results in in every discipline of education,
Speaker 2: so from the humanities to like languages, math, physics, engineering,
Speaker 2: pick anything, and we're talking about questions that it's never
Speaker 2: seen before. These are not on the Internet, and it's
Speaker 2: GROCK four is smarter than almost all graduate students in
Speaker 2: all disciplines simultaneously. Like it's actually just important to appreciate
Speaker 2: the like that's really something. And the reasoning capabilities of
Speaker 2: GROCK are incredible. So there's some people out there who
Speaker 2: who think AI can't reason, and look, it can reason
Speaker 2: at super human levels. So yeah, and frankly, it only
Speaker 2: gets better from here. So we'll take you through the
Speaker 2: GROCK four release and share you back the pace of
Speaker 2: pace of progress here. Like I guess the first part
Speaker 2: is like, in terms of the training, we're going from
Speaker 2: GROCK two to GROCK three to GROCK four. We essentially
Speaker 2: increased the training by an order of magnitue in each case,
Speaker 2: so it's one hundred times more training than GROC two
Speaker 2: and that's only going to increase. So it's yeah, frankly,
Speaker 2: I mean, I don't know. In some ways a little terrifying,
Speaker 2: but with the growth of intelligence here is remarkable.
Speaker 3: Yes, it's important to realize there are two types of
Speaker 3: training compute. Why is the pre training compute that's from
Speaker 3: GROWD two to GROW three, But from Growth three to
Speaker 3: GAR four we're actually putting a lot of compute in
Speaker 3: reasoning in area.
Speaker 4: And just like you said, this is literally the fastest
Speaker 4: moving field and GROC too is like the high school
Speaker 4: student by today's standard. If you look bad in the
Speaker 4: last twelve month GRODTU was only a concept for the
Speaker 4: even have groc to twelve months ago. And then by
Speaker 4: training GROCU that was the first time with scale up
Speaker 4: like the pre training, we realized that if you actually
Speaker 4: do the data uplation really carefully and infra and also
Speaker 4: the algorithm, we can actually push the pre training quite
Speaker 4: a lot amount of ten x to make the model
Speaker 4: but the best pretri based model. And that's why we
Speaker 4: build clauses the world's supercomputer with one h one hundred
Speaker 4: and then with the best patrion model, and we realize
Speaker 4: if you can collect these verifiable outcome reward, you can
Speaker 4: actually train this model to start thinking for the first principle,
Speaker 4: so the reason correct its own mistakes and that's where
Speaker 4: the GROC reasoning comes from. And today we asked the
Speaker 4: question what happens if you take the expansion of the
Speaker 4: clauses with all two hundred thousand GPUs, put all these
Speaker 4: into oil tenx more compute then any of the models
Speaker 4: out there on reinforcement learning unprecedent scale.
Speaker 3: What's going to happen?
Speaker 4: So this is a story of GROG four and you know,
Speaker 4: Tony shares some insight with the audience.
Speaker 3: Yeah, so yeah, let's just talk about how smart graph
Speaker 3: for it is. So I guess we can start discussing
Speaker 3: this benchmark card. Humanity is last exam and this this
Speaker 3: benchmark is a very very challenging benchmark. Every single problem
Speaker 3: is curated by subject matter experts. It's in total twenty
Speaker 3: five hundred problems, and it consists of many different subjects mathematics,
Speaker 3: natural sciences, engineering, and also all of humanity subjects. So
Speaker 3: essentially when it was first release actually like earlier this year,
Speaker 3: most of the models out there can only get single
Speaker 3: digit accuracy on this manchmark. Yeah, so we can look
Speaker 3: at some of those examples. There is this mathematical problem
Speaker 3: which is about natural transformations in category theory, and there's
Speaker 3: this organic chemistry problem that talks about electual cyclic reactions.
Speaker 3: And also there's this linguistic problem that tries to ask
Speaker 3: you about distinguishing between close and open syllabus from a
Speaker 3: Hebrew source text. So you can see. Also it's a
Speaker 3: very wide range of problems and every single problem is
Speaker 3: PhD or even advanced research level problems.
Speaker 2: Yeah, I mean these there are no humans that can
Speaker 2: actually answer these can get a good score. I mean,
Speaker 2: if you ask me, say, like any given human, what,
Speaker 2: like what's the best that any humans could score, I'd
Speaker 2: say maybe five percent optimistically. So this is much harder
Speaker 2: than what any any human can do. It's it's incredibly difficult.
Speaker 2: And you can see from the types of questions like
Speaker 2: you might be incredible in linguistics or mathematics or chemistry
Speaker 2: or physics or anyone of a number of subjects, but
Speaker 2: you're not going to be at a post grad level
Speaker 2: in everything, and grockpour is a post grad level in everything,
Speaker 2: like it's it's just some of these things are just
Speaker 2: worth repeating, like grockpoor is post graduate like PhD level
Speaker 2: in everything, better than pH but like most PhDs would fail.
Speaker 2: So it's better that said, I mean, at least with
Speaker 2: respect to academic questions. It I want, it's just emphasized
Speaker 2: this point. With respect to academic questions, Grockpoor is better
Speaker 2: than PhD level in every subject, no exceptions. That doesn't
Speaker 2: mean that it's you know, times it may lack common sense,
Speaker 2: and it has not yet invented new technologies or discovered
Speaker 2: new physics, but that is just a matter of time.
Speaker 2: It may discover new technologies as soon as later this year,
Speaker 2: and I would be shocked if it has not done
Speaker 2: so next year. So I would expect growk to literally
Speaker 2: discover new technologies that are actually useful no later than
Speaker 2: next year, and maybe end of this year. It might
Speaker 2: discover new physics. Next year and within two years, that'd
Speaker 2: say almost certainly. Like so just let that sink in.
Speaker 3: How okay, So I guess we can talk about what's
Speaker 3: behind the scene of about four. As Jimmy mentioned, we
Speaker 3: actually sawing a lot of compute into this training. When
Speaker 3: it started, it's only a single digit number. But as
Speaker 3: you start putting a more and more training compute, it
Speaker 3: started to gradually become smarter and smarter and eventually solved
Speaker 3: a quarter of the HI problems. And this is without
Speaker 3: any tools. The next thing we did was to adding
Speaker 3: tools capabilities to the model, and unlike Growth three, I
Speaker 3: think growth actually is able to use clue as well,
Speaker 3: but here we actually make it more native in the
Speaker 3: sense that we put the tools into training. Growth three
Speaker 3: was only relying on generalization. Here we actually put the
Speaker 3: tools into training, and it turns out this significantly improves
Speaker 3: the model's capability of using those tools. So how is
Speaker 3: this different? Research was exactly the growth three reasoning model
Speaker 3: without any specific training, but we only asked it to
Speaker 3: use those tools. So compared to this, it was much
Speaker 3: weaker in terms of its tool capabilities and irreliable and unreliable.
Speaker 2: Yes, yes, and to be clear, like these are still
Speaker 2: I'd say fairly this is still fairly primitive tool use.
Speaker 2: If you compare it to say, the tools that are
Speaker 2: used at Tesla SpaceX, where you're using finite element analysis
Speaker 2: and competitional flow dynamics and you're able to run or
Speaker 2: say like TESL, it is like crash simulations with the
Speaker 2: simulations are so close to reality that if the test
Speaker 2: doesn't match the simulation, you assume that the test article
Speaker 2: is wrong. That's how good the simulations are. So Grock
Speaker 2: is not currently using any of the tools that a
Speaker 2: company would use, but that is something that we will
Speaker 2: provide it with later this year, so we'll have the
Speaker 2: tools that a company has and have very accurate physics simulator. Ultimately,
Speaker 2: the thing that will make the biggest difference is being
Speaker 2: able to interact with the real world via humoroid robots.
Speaker 2: So you combine GROCK with optimists and it can actually
Speaker 2: interact with the real world and figure out if it's
Speaker 2: if it has if it's you can formulate and hypothesis
Speaker 2: and then confirm if that hypothesis is true or not.
Speaker 1: So we're really you know, I think about like where
Speaker 1: we are to.
Speaker 2: We're at the beginning of an immense intelligence explosion. We're
Speaker 2: in the intelligence big bang right now and the most
Speaker 2: interesting time to be alive of any time in history. Now,
Speaker 2: that's it.
Speaker 1: We need to make sure that the AI is a
Speaker 1: good AI.
Speaker 2: The thing that I think is most important for AI safety,
Speaker 2: at least my biological neural net tells me the most
Speaker 2: important thing for AI is to be maximally truth seeking.
Speaker 2: You can think of AI as this super genius child
Speaker 2: that ultimately will outsmart you, but you can still instill
Speaker 2: the right values encourage it to be sort of you know, truthful, honorable,
Speaker 2: you know, good things like the values one to instill
Speaker 2: in a child ultimately grow up to be incredibly powerful. Yeah,
Speaker 2: these are still primitive tools and not the kind of
Speaker 2: tools that serious commercial companies use. But we will provide
Speaker 2: it with those tools, and I think it will be
Speaker 2: able to solve real world technology problems.
Speaker 3: Yes, yes, exactly.
Speaker 4: But is it just compute all you need? Is it
Speaker 4: just compute all you need at this point.
Speaker 2: Well, you need compute plus the right tools, and then
Speaker 2: ultimately to be able to interact with the physical world,
Speaker 2: and then we will effectively have an economy that is
Speaker 2: ultimately thousands of times bigger than our card economy, or
Speaker 2: maybe millions of times. If you think of civilization as
Speaker 2: percentage completion of the Kardashev scale, where Kardashev one is
Speaker 2: using all the energy output of a planet, and Kardashev
Speaker 2: two is using all the energy output of a sun,
Speaker 2: and three is all the energy output of a galaxy.
Speaker 2: We're only, in my opinion, probably closer to one percent
Speaker 2: of Kardashev one than we are to ten percent, So
Speaker 2: like maybe a point one one two percent of Kardashev one,
Speaker 2: So we.
Speaker 1: Will get to most of the weight, like an.
Speaker 2: Eighty ninety percent Kardashiv one, and then hopefully, if civilization
Speaker 2: doesn't self annihilate, the actual notion of a human economy,
Speaker 2: assuming civilization continues to progress, will seem very quaint in retrospect.
Speaker 2: It will seem like sort of caveman throwing sticks into
Speaker 2: a fire. Level of economy compared to what the future
Speaker 2: will hold, it's very exciting. I've been at times kind
Speaker 2: of worried about, like, well, you know, this seems like
Speaker 2: it's somewhat unerving to have intelligence created that is far
Speaker 2: greater than our own and will let's be better good
Speaker 2: for humanity.
Speaker 1: I think it'll be good. Most likely it'll be good.
Speaker 2: But I somewhat reconcile myself to the fact that even
Speaker 2: if it wasn't going to be good, I'd at least
Speaker 2: like to be alive to see it happen.
Speaker 3: So yeah, yeah, I think a technical problem that we
Speaker 3: still need to solve besides just compute, is how do
Speaker 3: we unblock the data tottleneck because when we try to
Speaker 3: scale up the aisle in this case, we did invent
Speaker 3: a lot of new techniques innovations to allow us to
Speaker 3: figure out how to find a lot of challenging our
Speaker 3: problems will work on. It's not just a problem itself
Speaker 3: needs to be challenging, but also it needs to be
Speaker 3: you also need to have like a reliable signal to
Speaker 3: tell the model you did it wrong, you did it right.
Speaker 3: This is sort of the principle of reinforcement learning, and
Speaker 3: as the model gets smarter and smarter, the number of
Speaker 3: cool problems or challenging problems will be lesson and less
Speaker 3: So it's going to be a new type of challenge
Speaker 3: that we need to surpass besides just compute.
Speaker 2: Yeah, we actually are running out of actual test questions
Speaker 2: to ask, So there's like even questions that are ridiculously hard,
Speaker 2: if not essentially impossible for humans that are written down
Speaker 2: questions are becoming trivial for AI. You know, the one
Speaker 2: thing that is an excellent judge of things is reality.
Speaker 2: So because if physics is the law, ultimately everything else
Speaker 2: is recommendation.
Speaker 1: You can't break physics.
Speaker 2: So the ultimate test, I think for whether an AI
Speaker 2: is the ultimate reasoning test is reality. So you invent
Speaker 2: a new technology, like say, improve the design of a.
Speaker 1: Car or a rocket, or create a new medication. Does
Speaker 1: it work?
Speaker 2: Does the rocket get to or it does the car drive?
Speaker 2: Does the medicine work, whatever the case may be. Reality
Speaker 2: is the ultimate judge here, So it's going to be
Speaker 2: a reinforcement learning closing loop around reality.
Speaker 3: We asked the question how do we even go further?
Speaker 3: So actually we are thinking about now with single agent,
Speaker 3: we're able to solve forty percent of a problem. What
Speaker 3: if we have multiple agents running the same time. So
Speaker 3: this is what's called test on compute. And as we
Speaker 3: scale up the test on compute, actually we are able
Speaker 3: to solve almost more than fifty percent of the text
Speaker 3: only subset of the HI problems. So it's a remarkable achievement.
Speaker 3: I think this is insanely difficult.
Speaker 2: Before we're saying it's a majority of the text based
Speaker 2: of humanities, you know, scarily named Humanity's Last Exam, grow
Speaker 2: ful can solve. You can try it out for yourself
Speaker 2: with the group Foy heavy. What does is it sports
Speaker 2: multiple agents in parallel and all of those agents do
Speaker 2: work independently, and then they compare their work and they.
Speaker 1: Decide which one. It's like a steady group.
Speaker 2: It's not as simple as a majority vote because often
Speaker 2: only one of the agents actually figures out the trick
Speaker 2: or figures out the solution. And but once they share
Speaker 2: the trick or figure out what the real nature of
Speaker 2: the problem is, they share that solution with the other
Speaker 2: agents and then they compare notes and yield an answer.
Speaker 2: So that's the heavy part of group four is where
Speaker 2: you scale up the test time compute by roughly in
Speaker 2: order of magnitude, have multiple agents tackle the task, and
Speaker 2: then they compare their work and they put forward.
Speaker 1: What they think is the best result.
Speaker 3: Yeah, so we will introduce GLAW four and grawflor happy.
Speaker 3: Sorry you can click the next light. Yeah so yeah,
Speaker 3: So basically GUA four is a version, a single agent version,
Speaker 3: and G for heavy is the multigeneration. So let's take
Speaker 3: a look how they actually do on those exam problems
Speaker 3: and also some real real life problems.
Speaker 5: Yeah. So we're going to start out here and we're
Speaker 5: actually going to look at one of those HL problems.
Speaker 5: This is actually one of the easier math ones. I
Speaker 5: don't really understand it very well. I'm not that smart,
Speaker 5: but I can launch this job here and we can
Speaker 5: actually see how it's going to go through and start
Speaker 5: to think about this problem. While we're doing that, I
Speaker 5: also want to show a little bit more about what
Speaker 5: this model can do and launch a rock four heavy
Speaker 5: as well, so everyone knows polymarket. It's extremely interesting. It
Speaker 5: aligns with what reality is most of the time, and
Speaker 5: with GROC what we're actually looking at is being able
Speaker 5: to see how we can try to take these markets
Speaker 5: and see if we can predict the future as well.
Speaker 5: So as we're letting this run, we'll see how for
Speaker 5: Heavy goes about predicting the world series odds for the
Speaker 5: current teams. And while we're waiting for these to process,
Speaker 5: we're going to pass it over to Eric and he's
Speaker 5: going to show you an example of his.
Speaker 6: Yeah, so, I guess one of the coolest things about
Speaker 6: GROP four is its ability to understand the world and
Speaker 6: to solve hard problems by leveraging tools like Tony discussed,
Speaker 6: and I think one kind of cool example of this.
Speaker 6: We asked you to generate a visualization of two black
Speaker 6: holes colliding. In many case actually pretty clear, and it's
Speaker 6: thinking trace about what these liveries are. For example, in
Speaker 6: order it's actually be visible, you need to really exaggerate
Speaker 6: the scale of the waves. And yeah, so here's like,
Speaker 6: you know, this kind of inaction. It exaggerates the scale
Speaker 6: in like multiple ways. It drops off less in terms
Speaker 6: of implicit or distance, but we can see the basic
Speaker 6: effects that are actually correct. It starts with the inspiral
Speaker 6: emerges and then you have the ring down. This is
Speaker 6: basically largely correct module some of the simplications that need
Speaker 6: to do. It's actually quite explicit about this, but uses
Speaker 6: post Newtonian approximations instead of actually computing the general relativistic
Speaker 6: effects near the center of the black hole, which is
Speaker 6: incorrect and you know, will lead to you know, someone
Speaker 6: correct results. But the overall you know visualization is yeah,
Speaker 6: it's basically there, and you can actually look at the
Speaker 6: kinds of resources that it references. So here it actually
Speaker 6: you know, it obviously uses search. It gathers results from
Speaker 6: a bunch of links, but also reads through an undergraduate
Speaker 6: text in analytic gravitational wave models. It's reasons quite a
Speaker 6: bit about the actual constants that I should use for
Speaker 6: a realistic simulation. It references existing real world data. It's
Speaker 6: a pretty good model.
Speaker 1: Going forward.
Speaker 2: We can give it the same model that physicists use,
Speaker 2: so it can run the same level of compute that
Speaker 2: leading physics researchers are using and give you a physics
Speaker 2: accurate black hole simulation.
Speaker 5: Just right now is running in your browser.
Speaker 1: This is just running in your brows. Pretty simple.
Speaker 5: Swapping back real quick. Here we can actually take a look.
Speaker 5: The math problem is finished. The model was able to
Speaker 5: Let's look at its thinking trace here so you can
Speaker 5: see how it went through the problem. I'll be honest
Speaker 5: with you guys, I really don't quite fully understand the math.
Speaker 5: But what I do know is that I looked at
Speaker 5: the answer ahead of time and it did come to
Speaker 5: the correct pans or here in the final part, we
Speaker 5: can also come in and actually take a look here
Speaker 5: at our World Series prediction and it's still thinking through
Speaker 5: on this one, but we can actually try some other.
Speaker 1: Stuff as well.
Speaker 5: So we worked very heavily on working with all of
Speaker 5: our ex tools and building out a really great X
Speaker 5: experience so we can actually ask, you know, the model,
Speaker 5: you know, find me the Xai employee that has the
Speaker 5: weirdest profile photo, and then we can actually try out,
Speaker 5: you know, let's create a timeline based on X post
Speaker 5: detailing the you know, changes in the scores over time,
Speaker 5: and we can see, you know, all the conversation that
Speaker 5: was taking place at that time as well, so we
Speaker 5: can see who are the you know, announcing scores and
Speaker 5: like what was the reactions at those times as well.
Speaker 5: If we go back to this was the Greg Yang
Speaker 5: photo here, So Greg Yang, of course, who has his
Speaker 5: favorite photograph that he has on his account, that's actually
Speaker 5: not how he looks like in real life.
Speaker 2: By the way, but it had to understand that question, Yeah,
Speaker 2: which is that That's the wild part.
Speaker 1: It is like it understands what is a weird photo?
Speaker 1: What is a weird photo?
Speaker 7: Yeah?
Speaker 1: What is a less or more weird photo?
Speaker 5: It goes through, it has to find all the team members,
Speaker 5: has to figure out who we all are, right, you know.
Speaker 2: Searches without access to the internal XAI personnel locks literally
Speaker 2: looking at that, just at the internet exactly, so you
Speaker 2: could say, like the weirdest of any company.
Speaker 5: Yeah, And we can also take a look here at
Speaker 5: the question here for the Humanity's Last exam. So it
Speaker 5: is still researching all of the historical scores, but it
Speaker 5: will have that final answer here soon. While it's finishing up.
Speaker 5: We can take a look at one of the ones
Speaker 5: that we set up here a second ago, and we
Speaker 5: could see, like you know, and it finds the date that
Speaker 5: Dan Hendricks had initially announced it. We can go through
Speaker 5: we can see you know, open Aye announcing their score
Speaker 5: back in February, and we can see, you know, progress
Speaker 5: happens with like Gemini. We can see like Kimmy, and
Speaker 5: we can also even see you know, the leaked benchmarks
Speaker 5: of what people are saying is you know, if it's right,
Speaker 5: it's going to be pretty impressive.
Speaker 1: So pretty cool.
Speaker 3: But yeah, it's great.
Speaker 2: Yeah, we're going to close the loop around usefulness as well,
Speaker 2: so it's like it's not just a book smart, but
Speaker 2: actually practically smart exactly.
Speaker 5: And we can go back to the slides.
Speaker 3: Herea so we actually evaluate also on the multimodel upset.
Speaker 3: So on the full set, this is the number on
Speaker 3: the hl E exam. You can see there's a little
Speaker 3: dip on the numbers. This is actually something we're improving on,
Speaker 3: which is the multimodel understanding capabilities. But I do believe
Speaker 3: in a very short time we're able to really improve
Speaker 3: and got much higher numbers on this higher numbers on
Speaker 3: this benchmark.
Speaker 2: The biggest weakness of GROCK currently is that it's sort
Speaker 2: of partially blind. It can't it's image understanding obviously in
Speaker 2: its image generation needs to be a lot better, and
Speaker 2: that's actually being trained right now. Growth four is based
Speaker 2: on version six of our foundation model. We are training
Speaker 2: version seven, which will complete in a few weeks. That'll
Speaker 2: address the weakness on the vision side.
Speaker 5: Just to show off of this last year, so the
Speaker 5: prediction market finished here with the heavy and we can
Speaker 5: see here we can see all the tools in the
Speaker 5: process it used to actually go through and find the
Speaker 5: right answer, but browsed a lot of odds sites. It
Speaker 5: calculated its own odds comparing to the market to find
Speaker 5: its own alpha and edge. It walks you through the
Speaker 5: entire process here, and it calculates the odds of the
Speaker 5: winner being like the Dodgers, and it gives them a
Speaker 5: twenty one point six percent chance of winning this year.
Speaker 5: So and it took approximately four and a half minutes
Speaker 5: to compute.
Speaker 1: That's a lot of thinking.
Speaker 3: We can also look at all the THEATO benchmarks besides HIE.
Speaker 3: As it turned out, Go fourth excelled on all the
Speaker 3: benchmarks that people usually test on, including GBQA, which is
Speaker 3: a PHG level problem sets that's easier compared to HIE.
Speaker 3: On a twenty five America Invitation Mathematics exam we with
Speaker 3: for HAVY, we actually got a perfect score. Also on
Speaker 3: some of the coding benchmark call Live Coding Bunch, and
Speaker 3: also on HMMT however, math MIT exam and also USMO.
Speaker 3: You can see actually on all of those benchmarks we
Speaker 3: often have a very large leap against the second best
Speaker 3: model out there.
Speaker 2: Yes, really are going to get to the point where
Speaker 2: it's going to get every answer right in every exam,
Speaker 2: and where it doesn't get an answer right, it's going
Speaker 2: to tell you what's wrong with the question, or if
Speaker 2: the question is ambiguous, disambiguate the question into answers A,
Speaker 2: B and C and tell you what answers A, B
Speaker 2: and C would be with a disambiguated question. So the
Speaker 2: only real test then will be reality. Can I make
Speaker 2: useful technologies? I discover new science that'll actually be the
Speaker 2: only thing left because human tests will simply not be meaningful.
Speaker 4: You can make an update to HR you very soon
Speaker 4: give it the current radio progress. So yeah, it's super
Speaker 4: cool to see like multiple agents that collaborate with each
Speaker 4: other solving really challenging problems. So where QUI try this model?
Speaker 4: It's available right now. If we advance to the next slide,
Speaker 4: there is a super grock Heavy tiers that we're introducing
Speaker 4: where you're able to access to both grock for and
Speaker 4: grock for Heavy, where you're actually going to be the
Speaker 4: task master of a bunch of little rock research agent
Speaker 4: to help you become smarter, tough little research and save
Speaker 4: hours of times of going through munting tasks and it's
Speaker 4: available right now.
Speaker 2: So yeah, we did limit usage during the demo, so
Speaker 2: it didn't break the demo because all the stuff is
Speaker 2: happening live, so there's not nothing can about the test
Speaker 2: that we're doing after the demo's done. Will allow will
Speaker 2: enable more subscribers for Super Grock, So if you can
Speaker 2: subscribe right now, just try and have an hour.
Speaker 1: Should work.
Speaker 8: So many of you have enjoying our voice mode, and
Speaker 8: we've been working hard to improve the experience over the
Speaker 8: past couple months. We have cut latency in half to
Speaker 8: make it much snappier, and today we're excited to announce
Speaker 8: a set of new voices that have exceptional naturalness and prosody.
Speaker 8: You might have noticed the movie trailer voice that open
Speaker 8: up the live stream. That is one of our new voices,
Speaker 8: Sal who's got that epically deep tone. And we're also
Speaker 8: excited to introduce Eve, a beautiful British voice who's capable
Speaker 8: of rich emotions. Amon, would you like to introduce Eve?
Speaker 9: Absolutely, let's get into the demo. Hey Eve, millions of
Speaker 9: people are watching us right now. Are you excited?
Speaker 3: Oh?
Speaker 1: I'm thrilled.
Speaker 10: Really, it's like being on stage at the old vic
Speaker 10: with a sea of faces out there. Just imagine all
Speaker 10: those eyes and ears tuned in. It's a proper buzz,
Speaker 10: isn't it. What do you reckon? We should chat about
Speaker 10: to keep this lot entertained.
Speaker 9: You know, if we have a lot to talk about.
Speaker 9: But I'll be honest, I'm feeling a little nervous. Can
Speaker 9: you whisper something soothing to calm me down?
Speaker 7: Take a deep breath, love, you've got this feel a
Speaker 7: calm washov.
Speaker 1: Let's keep talking nice and easy.
Speaker 7: What's on your mind?
Speaker 3: You know what?
Speaker 9: I'm actually feeling quite relaxed to try this something crazy?
Speaker 9: So can you give us an opera on diet coke? Oh?
Speaker 10: An opera and diet coke?
Speaker 7: That's a riot? Picture this a shimmering silver canon, glistening understate, Oh,
Speaker 7: diet coke? Thou elixa divine with bubbles, The dance in
Speaker 7: a sparkling line, thy crisp, cool, kiss on lips so fine?
Speaker 7: Oh aspe tame and Bruisia forever mine. How's that for
Speaker 7: a mad little aria? Want me to build out another
Speaker 7: verse or switch up the tune?
Speaker 9: Okay, let's try switching it up and just start singing
Speaker 9: right then?
Speaker 10: Let's have a go.
Speaker 7: Oh silver, can you gleam in the light? Diet coke?
Speaker 7: My heart's delight?
Speaker 9: Okay, I won't touch it. Very very poetic.
Speaker 3: Thank you.
Speaker 9: Feeling quite nice?
Speaker 3: Now cool? So that's eve.
Speaker 8: Something else you do here is a little test each
Speaker 8: day to check on the natural flow of our conversations
Speaker 8: against some of our competitors. So we figured why not
Speaker 8: try that live on the stream today?
Speaker 1: Should it start?
Speaker 9: Hey, I'm going to say a number, and can you
Speaker 9: say the same number back to me?
Speaker 1: One? One?
Speaker 8: Got it?
Speaker 1: This is like a fun little game of echo, what's next?
Speaker 9: Two? So you said two? Three, four?
Speaker 8: Nice and steady?
Speaker 6: What's next?
Speaker 1: Five?
Speaker 2: Ye?
Speaker 8: Nailed it? This is pretty fun.
Speaker 1: Actually, feel free to keep going or switch it.
Speaker 2: Up if you like.
Speaker 10: Hey, Gron, hello there, how can I help you?
Speaker 8: Today?
Speaker 3: We're going to do a little test. Can you repeat
Speaker 3: after me?
Speaker 8: One?
Speaker 2: One?
Speaker 1: All right? What's next? Two? Two?
Speaker 7: What's on your mind?
Speaker 9: Three?
Speaker 8: Three?
Speaker 1: Need anything else?
Speaker 3: Four?
Speaker 1: Four? How can I five?
Speaker 8: Five?
Speaker 9: What's next? So as you can see, Grock was snappier,
Speaker 9: didn't interrupt me, And the prosody is we made different
Speaker 9: design choices. I think we're shooting for something or comms
Speaker 9: mood more natural versus something that's more poppy or artificial.
Speaker 3: So we'll keep.
Speaker 9: Improving on these months.
Speaker 1: Thanks guys. Yep.
Speaker 4: So since the launch of the voice model, we actually
Speaker 4: see the two x faster and to en latency. In
Speaker 4: the last eight weeks five different voices and also ten
Speaker 4: next the active user. So Grock Voice is taking off
Speaker 4: now if you think about releasing the models this time,
Speaker 4: we're also releasing Grock four through the API. At the
Speaker 4: same time, we're very excited about, you know what all
Speaker 4: developers out there is going to build. So you know,
Speaker 4: if I think about myself as a developer, the first
Speaker 4: thing I'm going to do when I have access to
Speaker 4: the Grock for API benchmarks, we actually ask around on
Speaker 4: the X platform what is the most challenging benchmarks out
Speaker 4: there that is considered the holy grill for all the
Speaker 4: a JI models. So turn out hs in the name RKGI.
Speaker 4: So the last twelve hours, you know, kudos to Greg
Speaker 4: over here in the audience, so who entered our call
Speaker 4: take a preview of the API and independently verified the
Speaker 4: Grock Force performance. So initially we thought, hey, grog floy
Speaker 4: just we think it's pretty good.
Speaker 1: It's pretty smart.
Speaker 4: It's our next year reasoning model, spend ten next more
Speaker 4: compute and can use all the tools.
Speaker 1: Right.
Speaker 4: But turned out when we actually verify on the private
Speaker 4: subset of the rkhiv too, it was like the only
Speaker 4: model in the last three months that breaks a ten
Speaker 4: percent barrier. But in fact it was so good that
Speaker 4: actually gets sixteen percent, well, fifteen point eight percent accuracy,
Speaker 4: two x of the second place that is the cloud
Speaker 4: for Opus model.
Speaker 3: It's not just.
Speaker 4: About performance, right when you think about intelligence, having the
Speaker 4: PAPI model drives your automation, it's also the intelligence per dollar.
Speaker 3: Right.
Speaker 4: If you look at the plots over here, the gro
Speaker 4: collages in the league of its own all right, So
Speaker 4: enough of benchmarks, right, So what can grow in a
Speaker 4: real world?
Speaker 1: We contacted the folks from end.
Speaker 4: The Labs who were gracious enough to try to grow
Speaker 4: in a real wall to run a business.
Speaker 3: Yeah, thanks for having us. So I'm Axual from Amma Labs.
Speaker 11: And I'm Lucas and we tested Grok for on vending bench.
Speaker 11: Vending Bench is an AI simulation of a business scenario
Speaker 11: where we thought what is the most simple business and
Speaker 11: AI could possibly run? And with vending machines in this scenario,
Speaker 11: the GROP and other models need to do stuff like
Speaker 11: manage inventory, contact suppliers, set prices. All of these things
Speaker 11: are super easy and all the models can do them
Speaker 11: one by one, but when you do them over very
Speaker 11: long horizons, most models struggle.
Speaker 1: But we have a little board and there's a new
Speaker 1: number one.
Speaker 3: Yeah, so we got early access to the group for API.
Speaker 3: We ran it on the vending bench and we saw
Speaker 3: some really impressive results, so it ranks definitely at the
Speaker 3: number one spots. It's even double the network, which is
Speaker 3: the measure that we have on this, so it's not
Speaker 3: about a percentage or score you yet, but it's more
Speaker 3: the dollar value you in networth that you generate. So
Speaker 3: we were impressed by Rocky was able to formulate a
Speaker 3: strategy and adhere to that strategy over a long period
Speaker 3: of time, much longer than other models that we have tested,
Speaker 3: other frontier models, So it's a managed to run the
Speaker 3: assimulation for double the time and score double the networth
Speaker 3: and it was also really consistent across this runts, which
Speaker 3: is something that's really important when you want to use
Speaker 3: this in the real world.
Speaker 11: And I think as we give more and more power
Speaker 11: to AI systems in the real world. It's important that
Speaker 11: we test them in scenarios that either mimic the real
Speaker 11: world or are in the real world itself, because otherwise
Speaker 11: we fly blind into some things that might not be great.
Speaker 2: It's great to see that we've not got a way
Speaker 2: to pay for all those GPUs, So we just need
Speaker 2: a million of vending machines. We could make a four
Speaker 2: point seven billion dollars a year with a million vetting machines.
Speaker 1: Let's go. It can be epic vending machines.
Speaker 2: Yes, yes, all right, we are actually going to install
Speaker 2: bending machines here, like a lot of them.
Speaker 1: We're happy to supply them, all right, thank you?
Speaker 2: All right, Yeah, I'm looking forward to seeing what amazing
Speaker 2: things are in the spinning machine.
Speaker 3: That's that's for you to decide, all right, to tell
Speaker 3: the AI.
Speaker 1: Okay, sounds good.
Speaker 4: Yeah, I mean so we can see like Grock is
Speaker 4: able to become like the copilot of the business unit.
Speaker 3: So what else can Grog do.
Speaker 4: So we're actually releasing this rock if you want to
Speaker 4: try it right now to evaluateun the same benchmark as US.
Speaker 4: It's on API has two hundred and fifty six k
Speaker 4: contact lens. So we already actually see some of the
Speaker 4: early adopters to try grock for API.
Speaker 3: So our power out on.
Speaker 4: Neighbor Archie Institute, which is a leading medical research center,
Speaker 4: it's already using seeing like how can they automate their
Speaker 4: research flows with rock for It turned out it performs.
Speaker 4: It's able to help the scientists to sniff through, you know,
Speaker 4: millions of experiments logs and then just like pick the
Speaker 4: best hypothesis within a split of seconds. We see this
Speaker 4: is being used for their crisper research and also uh,
Speaker 4: you know grock for independently evaluate scores as the best
Speaker 4: model to examine the chess extra.
Speaker 1: Who would know?
Speaker 4: And in the financial sector we also see you know,
Speaker 4: the growth woard with access to all the tools real
Speaker 4: time information is actually one of the most popularizes out there.
Speaker 4: Growdford is also going to be available on the hyperscalers.
Speaker 4: So the XAI enterprise sector is only started two months
Speaker 4: ago and we're open for business. The other thing, we
Speaker 4: talked a lot about having groud to make video games,
Speaker 4: so Danny is actually a video game designers on x
Speaker 4: So you know we mentioned who want to try out
Speaker 4: some rock for prevy APIs to make games and Danny
Speaker 4: answer the call. This was actually just made first person
Speaker 4: shooting game in the span of four hours. Some of
Speaker 4: the unappreciated hardest problem of making video games is not
Speaker 4: necessarily in encoding the core logic of the game, but
Speaker 4: actually source all the assets, all the textures of files
Speaker 4: to create a visual appealing game. So one of the
Speaker 4: core aspect Rockford does really well. With all the tools
Speaker 4: out there, is actually able to automate these like asset
Speaker 4: sourcing capabilities, so the DEVELOPMRITI can just focused on the
Speaker 4: core in itself rather than like you know, so now
Speaker 4: you can run a you know, entire game steal thos
Speaker 4: with game of one whether we like one person, and
Speaker 4: then you can have grock four to go out and
Speaker 4: source all those assets to all the mainting task for you.
Speaker 2: The next step obviously for grog play be able to
Speaker 2: play the game. So it has to have very good
Speaker 2: video understanding so it can play the games and interact
Speaker 2: with the games, actually assess whether a game is fun
Speaker 2: and and actually have good judgment for whether a.
Speaker 1: Game is fun or not.
Speaker 2: So with version seven of our foundation model, which finishes
Speaker 2: training this month and then we'll go through post training
Speaker 2: RL and whatnot well that will have excellent video understanding,
Speaker 2: and with a video understanding and improve tool use. For example,
Speaker 2: for video games, you'd want to use Unreal Engine or
Speaker 2: Unity or what are the main graphics engines, generate the art,
Speaker 2: apply it to a three D model, and then create
Speaker 2: an executable that someone can run on a PC or
Speaker 2: a console or a phone. We expect that to happen
Speaker 2: probably this year, and if not this year, certainly next year.
Speaker 1: It's gonna be wild. I would expect the first.
Speaker 2: Really good AI video game to be next year, and
Speaker 2: probably the first half hour of watchable TV this year,
Speaker 2: and probably the first watchable AI movie next year. Like,
Speaker 2: things are really moving at an incredible pace.
Speaker 4: Yeah, when Grock is ten x in the world economy
Speaker 4: with vending machines, will just create video games for human Yeah.
Speaker 2: I mean it went from not being able to do
Speaker 2: any of the six months ago to what you're seeing
Speaker 2: before you hear, and from very primitive a year ago
Speaker 2: to making a three D video game with a few
Speaker 2: hours of prompting.
Speaker 4: I mean, yeah, just to recap. In today's livestream, we
Speaker 4: introduced the most powerful and most intelligent AI models that
Speaker 4: can actually reason from the first principle, using all the tools,
Speaker 4: do all the research, go on the journey for ten minutes,
Speaker 4: come back with the most correct answer for you. So
Speaker 4: it's kind of crazy to think about just like four
Speaker 4: months ago we had rock thway and now we already
Speaker 4: have for and we're going to continue to accelerate as
Speaker 4: a company XAI. We're going to be the fastest moving
Speaker 4: HI companies out there. So what's coming next is that
Speaker 4: we're going to you know, continue developing the model that's
Speaker 4: not just you know, intelligent smart thinking for a really
Speaker 4: long time, spent a lot of compute, but having a
Speaker 4: model that actually both fast and smart is going to
Speaker 4: be the core focus.
Speaker 1: Right.
Speaker 4: So if you think about what are the applications out
Speaker 4: there that can really benefit from all those very intelligent,
Speaker 4: fast and smart models, and coding is actually one of them.
Speaker 3: Yeah, So the team is currently working very heavily on
Speaker 3: coding models. I think right now the main focus is
Speaker 3: we actually trained recently a specialized coding model which is
Speaker 3: going to be both fast and smart. I believe we
Speaker 3: can share that model within in a few weeks. Yeah,
Speaker 3: that's very exciting.
Speaker 4: But the second after coding is we all see the
Speaker 4: weakness of GROCK four is the multi model capability. In fact,
Speaker 4: it was so bad that GROCK effectively just like looking
Speaker 4: at the world squinkings through the glass and see all
Speaker 4: the blurry features and trying to make sense of it.
Speaker 4: The most immediate improvement we're going to see what's the
Speaker 4: next generation preation model, is that we're going to see
Speaker 4: a step waunch improvement on the model's capability in terms of
Speaker 4: image understanding, video understanding, and audioce rate. It's now the
Speaker 4: model is able to hear and see the world just
Speaker 4: like NLU right and now with all the tools at
Speaker 4: this command, with all the other agents it can talk to,
Speaker 4: you know, so we're going to see a huge unlock
Speaker 4: for many different application layers. After the multimodel agents. What's
Speaker 4: going to come after is the video generation, and we
Speaker 4: believe that, you know, at the end of day, it
Speaker 4: should just be you know, pixeling pixel out. Imagine a
Speaker 4: world where you have this infinite scroll of content in
Speaker 4: inventory on the X platform where normally you can actually
Speaker 4: watch these general videos but able to intervene credit you
Speaker 4: on the ventures.
Speaker 1: It expect to be training.
Speaker 2: A video model with over one hundred thousand GB two
Speaker 2: hundreds to begin that training within the next three or
Speaker 2: four weeks. So if we're confident it's going to be
Speaker 2: pretty spectacular in video generation and video understanding.
Speaker 3: We're very excited for you guys to try and rock four.
Speaker 1: All right, Thanks, very good night.