WEBVTT 1 00:00:00.040 --> 00:00:02.200 But again, as I was saying, the most important thing 2 00:00:02.200 --> 00:00:06.320 about the occupancy network for avs is a collision avoidance system. 3 00:00:06.719 --> 00:00:09.519 It implicitly has this. Now that that is not to 4 00:00:09.599 --> 00:00:13.960 say that it will automatically go on the correct path. No, 5 00:00:14.599 --> 00:00:20.839 it just means it will consistently choose a collision avoidant path. So, 6 00:00:21.039 --> 00:00:23.079 as you can see here, this car went out of control. 7 00:00:23.359 --> 00:00:25.399 It almost collided in the right corner, but the collision 8 00:00:25.440 --> 00:00:28.480 avoid the occupancy network avoided it moved over to the left. 9 00:00:28.480 --> 00:00:30.120 This is clearly the wrong side of the road, but 10 00:00:30.160 --> 00:00:35.159 it avoided another collision and it stabilized. And that's the value. Originally, 11 00:00:35.600 --> 00:00:38.320 Initially that was the major value, and it was sufficient 12 00:00:38.359 --> 00:00:40.640 to sell it into the av industry just for this alone, 13 00:00:41.640 --> 00:00:47.960 because there is software alone doing this. I'm going to 14 00:00:48.000 --> 00:00:55.320 skip this because this is going to get really technical. Well, 15 00:00:55.520 --> 00:00:58.000 AI people might care about the leaderboard results. I just 16 00:00:58.039 --> 00:01:01.119 want to point out we're in the competitive race. Thank 17 00:01:01.119 --> 00:01:03.479 god we have in Vidia. That's the listing you see 18 00:01:03.479 --> 00:01:06.239 at the top on this leaderboard right here. But then 19 00:01:06.280 --> 00:01:10.840 you can see who all the competitors are and how 20 00:01:10.879 --> 00:01:14.680 many there are in China. There are At the time 21 00:01:14.719 --> 00:01:17.120 I was doing this I'm sure it's way higher than 22 00:01:17.159 --> 00:01:19.439 this now, but at that time in twenty twenty three, 23 00:01:19.480 --> 00:01:23.640 in China there are more than one hundred and sixty 24 00:01:24.439 --> 00:01:27.560 electric vehicle manufacturers. I'm sure the number is way higher 25 00:01:27.560 --> 00:01:30.680 than that now, but there's more than one hundred and 26 00:01:30.719 --> 00:01:40.280 sixty EV manufacturers in China. In academia, China, Taiwan, East 27 00:01:40.319 --> 00:01:46.000 Asia in general, they are super active and competitive on 28 00:01:46.159 --> 00:01:52.159 all of the leaderboards in AI, especially related to computer vision. 29 00:01:53.840 --> 00:01:57.120 Thank god we have in video. In Vidia consistently is 30 00:01:57.159 --> 00:02:00.560 at the top of the leaderboards from America, and then 31 00:02:00.599 --> 00:02:05.920 there are basically nobody else in America, literally nobody, It's 32 00:02:06.079 --> 00:02:09.599 just in video and then chiny China all the way down. Effectively, 33 00:02:10.599 --> 00:02:13.280 that's what we're dealing with. And the reason for this 34 00:02:13.360 --> 00:02:17.439 is because of the massive overinvestment in lightar for decades. 35 00:02:17.599 --> 00:02:19.800 There's been more than a decade that this has happened. 36 00:02:20.240 --> 00:02:25.000 And this is like, I cannot stress enough how dumb 37 00:02:25.599 --> 00:02:28.560 lightar is in an AV system. It does not make 38 00:02:28.599 --> 00:02:32.199 any sense and I have no idea how anyone rationalized 39 00:02:32.240 --> 00:02:35.560 it in the first place. This is a video game reference. 40 00:02:35.759 --> 00:02:37.439 I just wanted to show you what I meant by 41 00:02:37.639 --> 00:02:40.960 reducing the size of the vauxels. So this is and 42 00:02:41.000 --> 00:02:43.599 this is done in real time. So again I'm trying 43 00:02:43.639 --> 00:02:47.400 to demonstrate that this thing called an occupancy network and 44 00:02:47.560 --> 00:02:52.120 radiance fields, this can be rendered robustly in real time, 45 00:02:52.520 --> 00:02:55.400 and this is extremely important. And what you're looking at here, 46 00:02:55.560 --> 00:02:57.520 again it's a video game reference, but what you're looking 47 00:02:57.520 --> 00:03:01.879 at here is you're seeing depth calculation that is being 48 00:03:01.919 --> 00:03:04.159 done and it's being color graded based off of that. 49 00:03:04.479 --> 00:03:07.360 So blue is farther away and then yellow red would 50 00:03:07.360 --> 00:03:10.479 be closer. So what you can see is that in 51 00:03:10.599 --> 00:03:13.919 video games, to save memory, they're trying to focus on 52 00:03:13.960 --> 00:03:16.520 the field of view and they only render what's in 53 00:03:16.560 --> 00:03:18.919 your field of view and then ignore everything else. And 54 00:03:18.960 --> 00:03:21.039 then they render only what's in your field of view 55 00:03:21.039 --> 00:03:23.599 to save memory and compute, and this is really cool. 56 00:03:24.080 --> 00:03:26.039 And then they this is kind of my home base, 57 00:03:26.120 --> 00:03:27.520 and this is kind of a lot of what I 58 00:03:27.560 --> 00:03:31.599 do or was doing. But they did even cooler stuff 59 00:03:31.639 --> 00:03:34.879 than this. They like even realize that they could reduce 60 00:03:35.400 --> 00:03:38.280 they could increase vossal size really far out and save 61 00:03:38.439 --> 00:03:40.560 a lot of memory by not having to do as 62 00:03:40.599 --> 00:03:43.360 many points when you're really far away, and that and 63 00:03:43.400 --> 00:03:45.400 you don't lose anything by doing that. And you see 64 00:03:45.400 --> 00:03:48.759 this in video games too, where whenever you're some character 65 00:03:49.120 --> 00:03:52.639 looking over a long field of view, everything near you 66 00:03:52.879 --> 00:03:55.919 is high resolution, and then the further you go in 67 00:03:55.960 --> 00:03:59.280 the like look in the video game visually, it's lower 68 00:03:59.360 --> 00:04:02.199 quality result solution out there further out because you don't 69 00:04:02.199 --> 00:04:05.759 lose anything. And that's the same thing they're doing in 70 00:04:05.840 --> 00:04:10.000 avs to save memory as well. So this is a 71 00:04:10.080 --> 00:04:11.719 video game reperd Oh wait are you talking? 72 00:04:11.759 --> 00:04:15.199 Sorry, sorry, I think you just described dreams. 73 00:04:17.279 --> 00:04:18.199 Like actual dreams. 74 00:04:19.160 --> 00:04:22.120 Think about it. It only renders in front of you, 75 00:04:22.519 --> 00:04:23.920 in front of your field of vision. 76 00:04:26.279 --> 00:04:31.240 Yeah, that's actually uh so all of the the you're right. 77 00:04:31.279 --> 00:04:34.920 So one thing I skipped over in the neuroscience part 78 00:04:35.439 --> 00:04:41.839 is that the the our entire experience in reality happens 79 00:04:41.920 --> 00:04:46.720 in the hippocampus. And this is something that's important to understand, 80 00:04:46.959 --> 00:04:51.240 is that we experience the world through our senses. Yes, 81 00:04:51.279 --> 00:04:54.759 that's true, but in terms of the mapping, like everything 82 00:04:54.759 --> 00:04:56.480 that I'm showing you what the car sees on the 83 00:04:56.480 --> 00:04:59.079 screen and creating that, we do that exact same thing 84 00:04:59.519 --> 00:05:02.360 in the and it happens in the hippo campus. And 85 00:05:02.399 --> 00:05:07.199 so that's that virtual representation actually does happen in the 86 00:05:07.199 --> 00:05:10.240 brain too, and that's where we live. We live in here. 87 00:05:10.839 --> 00:05:13.920 That's everything in our universe happens right there, and that's 88 00:05:14.240 --> 00:05:18.680 our entire experience. So we're experiencing whatever our hippocampus recreates 89 00:05:18.720 --> 00:05:24.360 for us. So another thing that's important here everyone probably 90 00:05:24.439 --> 00:05:27.160 knows by now the GPUs are in heavy demand and 91 00:05:27.240 --> 00:05:31.519 they for some reason do a lot for AI, so 92 00:05:31.600 --> 00:05:34.240 that might be relevant. So what a GPU stand for 93 00:05:34.879 --> 00:05:39.360 graphics processing unit and it and it's not just computer 94 00:05:39.480 --> 00:05:42.800 vision that GPUs do really well on any AI. Math 95 00:05:43.240 --> 00:05:48.279 just happens to be really efficient on a GPU, especially 96 00:05:48.319 --> 00:05:53.560 compared to a CPU. And uh and the quint Yeah exactly. 97 00:05:53.680 --> 00:05:56.639 I mean there's i a will have to do a 98 00:05:56.639 --> 00:06:02.199 finance podcast in the future. The uh, the magnitude of 99 00:06:02.199 --> 00:06:06.560 difference between the performance on a GPU versus the performance 100 00:06:06.600 --> 00:06:11.600 on a CP was massive. And so it's interesting that 101 00:06:11.800 --> 00:06:16.759 not only is the generalized math for a AI really 102 00:06:16.800 --> 00:06:21.759 powerful on specifically GPUs for some reason, graphics processing units, 103 00:06:22.279 --> 00:06:26.040 but this math that I'm talking about, point clouds, occupanty networks, 104 00:06:26.120 --> 00:06:31.759 rate tracing, radiance fields, neural written nerve, all of that 105 00:06:32.600 --> 00:06:38.600 is extremely efficient on a GPU. By comparison to a CPU. 106 00:06:38.879 --> 00:06:41.040 In fact, as you can see here, one billion points 107 00:06:41.040 --> 00:06:46.120 per second compared to thirteen million points per second on 108 00:06:46.199 --> 00:06:48.920 a CPU. That's the magnitude of difference that we're talking 109 00:06:48.920 --> 00:06:52.839 about huge. And of course this is all out you know, 110 00:06:53.079 --> 00:06:57.879 twenty twenty three data, so way way more powerful. 111 00:06:57.920 --> 00:06:58.120 Now. 112 00:06:58.839 --> 00:07:01.319 Everything I'm showing this actorate as a twenty twenty three 113 00:07:01.480 --> 00:07:07.480 and has only gotten better since real time rendering. I've 114 00:07:07.519 --> 00:07:09.959 kind of hit on it a lot. So another thing 115 00:07:10.079 --> 00:07:15.040 is can it can consistently be done robustly? And this 116 00:07:15.199 --> 00:07:20.199 paper published by Meta National Taiwan University, which publishes such 117 00:07:20.199 --> 00:07:25.079 an absurd number of really good papers Kaiser and University 118 00:07:25.079 --> 00:07:31.279 of Meryland, this is robodime or sorry, robust dynamic nerves. 119 00:07:32.199 --> 00:07:35.519 And what they're showing here is that you can do 120 00:07:36.199 --> 00:07:40.600 all of this NERF stuff that I'm talking about very robustly, 121 00:07:40.839 --> 00:07:44.600 and in real time. They're dislocating the camera, they're creating 122 00:07:44.680 --> 00:07:50.000 jitter and noise like zoom in and zoom out, they're 123 00:07:50.079 --> 00:07:56.720 changing your position. It still works incredibly well. What you're 124 00:07:56.759 --> 00:08:00.519 seeing on here. Let's take probably the horses, the'siest one 125 00:08:00.519 --> 00:08:04.120 to understand. So on the bottom here this is heat mapped, 126 00:08:04.439 --> 00:08:07.879 and it's heat mapped for depth, meaning red closer to you, 127 00:08:08.000 --> 00:08:11.079 blue farther for you. And what you can see with 128 00:08:11.199 --> 00:08:14.759 this horse is you can see in real time the 129 00:08:14.800 --> 00:08:19.120 depth calculation that is happening robustly. So remember what I 130 00:08:19.160 --> 00:08:21.120 was saying about the point clouds, and you saw what 131 00:08:21.160 --> 00:08:24.079 you saw from Weimo and all that. This is the 132 00:08:24.199 --> 00:08:29.439 depth calculation with a vision based system on the horse, 133 00:08:29.480 --> 00:08:32.000 and you can see it starts out red and as 134 00:08:32.080 --> 00:08:36.159 the horse goes further back, the depth calculation changes relative 135 00:08:36.200 --> 00:08:38.720 to the rest of the environment. And it does it 136 00:08:38.720 --> 00:08:41.960 in real time. It transitions cleanly. You've got someone doing 137 00:08:42.039 --> 00:08:44.120 a windmill and you can see the depth calculation. You 138 00:08:44.120 --> 00:08:46.320 can see as the foot goes back, or the leg 139 00:08:46.399 --> 00:08:49.519 goes back, the range goes further back. It can do 140 00:08:49.600 --> 00:08:52.200 it fast, it can do it at speed, It can 141 00:08:52.240 --> 00:08:55.120 do it in chaotic environments. So this is just a 142 00:08:55.519 --> 00:09:00.279 visual demonstration of how robust the depth calculation actually is, 143 00:09:00.480 --> 00:09:03.279 which was the strong point of why you would invest 144 00:09:03.320 --> 00:09:06.399 fifty thousand dollars in a light our based system over 145 00:09:06.440 --> 00:09:10.799 one hundred dollars camera. Nope, you can do it. And again, 146 00:09:11.080 --> 00:09:15.720 who's published every paper. I'm showing here all academically accepted, 147 00:09:16.120 --> 00:09:22.360 all major publications, all considered major papers in academic world. 148 00:09:22.720 --> 00:09:26.559 So not a single paper here is anything controversial. Every 149 00:09:26.600 --> 00:09:30.759 single paper is academically accepted and understood at the highest 150 00:09:30.919 --> 00:09:37.080 level of regard and fully in production and fully in 151 00:09:37.159 --> 00:09:40.519 production and in a variety of ways. Yeah. Absolutely, all 152 00:09:40.519 --> 00:09:43.480 this stuff is well known, especially in East Asia. The 153 00:09:43.519 --> 00:09:46.480 amount like people are so brainwashed about China here, the 154 00:09:46.519 --> 00:09:49.720 propaganda so absurd. If you guys saw the ai that's 155 00:09:49.720 --> 00:09:52.840 happening in China, would we would look like an idiot. 156 00:09:53.039 --> 00:09:56.120 We do look like complete idiots compared to China right 157 00:09:56.120 --> 00:10:04.720 now and East Asia, Taiwan especially. So I'm gonna show 158 00:10:04.720 --> 00:10:07.080 this is a more modern version. This is not the 159 00:10:07.080 --> 00:10:10.840 most modern, but this is modern roughly in twenty twenty three. 160 00:10:10.919 --> 00:10:14.679 And this is the Tesla's occupancy network. And you can, 161 00:10:14.879 --> 00:10:18.799 again compared to the point cloud representation you get from LDAR, 162 00:10:19.279 --> 00:10:23.279 look at the level of detail that you're getting on 163 00:10:23.320 --> 00:10:25.960 the Tesla based system. This is in real time. This 164 00:10:26.039 --> 00:10:29.279 is not a camera overlay. This is what the car 165 00:10:29.440 --> 00:10:34.519 sees and reconstructing in real time on board. This is 166 00:10:34.559 --> 00:10:37.639 the occupancy He's now there's a speaker that's talking on this, 167 00:10:37.799 --> 00:10:41.759 and he's turning things on and off so that you 168 00:10:41.799 --> 00:10:45.799 can see the various details. But look at the calculator, 169 00:10:46.120 --> 00:10:50.120 the quality and resolution of the points, and there's so 170 00:10:50.360 --> 00:10:53.320 many of them. It looks like it's more dense to 171 00:10:53.360 --> 00:10:55.879 me than the one that Weimo was showing at the 172 00:10:56.000 --> 00:11:01.840 very same time, at the very same conference. It's worth 173 00:11:01.840 --> 00:11:04.600 seeing this one again. Now this is even more mod 174 00:11:04.720 --> 00:11:07.440 there's even more advancement that has already been made, and 175 00:11:07.519 --> 00:11:10.159 so this is not the modern version, but in twenty 176 00:11:10.159 --> 00:11:12.600 twenty three, you can see like just how much of 177 00:11:12.600 --> 00:11:17.440 a difference this is. And it's just it's impressive, and 178 00:11:17.519 --> 00:11:20.080 it's like a video game. I mean, anyone that's done 179 00:11:20.120 --> 00:11:22.200 video games knows that this is pretty much a video game. 180 00:11:26.480 --> 00:11:32.000 So I want to read this slide because it's the 181 00:11:32.000 --> 00:11:35.600 conclusion slide is written exactly right. And then I got 182 00:11:35.600 --> 00:11:38.440 one more major thing after this, and then we can 183 00:11:38.519 --> 00:11:41.600 go into discussion. But I'm just gonna read this real quick. 184 00:11:42.320 --> 00:11:46.120 It appears that all major av industry participants on any 185 00:11:46.200 --> 00:11:50.960 modality all over the world are converging to three things 186 00:11:50.960 --> 00:11:53.159 still true to this day. But this was what I 187 00:11:53.200 --> 00:11:57.639 was showing in twenty twenty three. Occupancy equals point clouds 188 00:11:57.919 --> 00:12:03.039 equals radiance fields equal VR representation. These are all exactly 189 00:12:03.080 --> 00:12:06.559 the same thing. They're all the same thing, and all 190 00:12:06.639 --> 00:12:10.639 augmented by semantics meaning words help. Lllm's help, but they're augmented. 191 00:12:10.840 --> 00:12:14.919 LLLM is an augment, not the foundation. Number two, perception 192 00:12:15.200 --> 00:12:20.600 accuracy is increasingly measured by how correctly perception recreates reality 193 00:12:21.039 --> 00:12:26.960 in virtual space. And number three, the task of AV 194 00:12:27.200 --> 00:12:32.840 perception has become building a localized digital twin of the 195 00:12:32.879 --> 00:12:37.559 real local environment in real time. That's what the task 196 00:12:37.600 --> 00:12:43.240 of perception in robotics basically has become. So feature rich 197 00:12:43.320 --> 00:12:46.639 representation is best. What this means is high resolution data 198 00:12:46.759 --> 00:12:50.559 is better. And you've got to understand. I had to 199 00:12:50.600 --> 00:12:55.320 explain that to a whole room full of PhDs, that 200 00:12:55.600 --> 00:13:01.759 higher resolution data is better an entire room of PhDs. 201 00:13:01.919 --> 00:13:07.080 I had to explain that too. At ASU, the perception 202 00:13:07.279 --> 00:13:12.159 task has complexity proportional to the maximum perceivable radial distance 203 00:13:12.200 --> 00:13:16.320 from ego, and the minimum required length of this radius 204 00:13:16.320 --> 00:13:18.759 increase with speed increases with speed. 205 00:13:19.000 --> 00:13:19.519 What this is. 206 00:13:19.440 --> 00:13:24.039 Saying is the faster you go, the farther down you 207 00:13:24.080 --> 00:13:27.679 need to see. But the slower you go, you don't 208 00:13:27.720 --> 00:13:33.120 necessarily need need to see as far. And the complexity 209 00:13:33.360 --> 00:13:37.759 of this whole VR representation task is determined by the 210 00:13:37.879 --> 00:13:43.519 radial length the minimum required radius, and that complexity increases 211 00:13:43.639 --> 00:13:47.919 the faster you go. And so if you're going fast, 212 00:13:48.039 --> 00:13:54.519 the complexity is not growing linearly, it's growing squared. And 213 00:13:54.639 --> 00:13:56.600 that's relevant if you were hearing what I was saying 214 00:13:56.600 --> 00:14:01.039 about light our degradation over range and the forecasting window. 215 00:14:02.440 --> 00:14:03.360 That's number of humans. 216 00:14:03.399 --> 00:14:03.639 Two. 217 00:14:04.279 --> 00:14:08.360 It's true for humans too, exactly number three. Recreating the 218 00:14:08.399 --> 00:14:13.720 world as humans see it requires perception systems to see 219 00:14:13.960 --> 00:14:20.000 real photons the way humans see them. And I should 220 00:14:20.000 --> 00:14:23.600 have mentioned this earlier. The road systems are designed for vision, 221 00:14:24.080 --> 00:14:28.679 not lightar. The road engineers have designed roads for vision, 222 00:14:29.000 --> 00:14:32.320 not lightar. This is the level of dumb we're dealing with. 223 00:14:33.120 --> 00:14:35.519 PhDs don't mean anything. This is the level of dumb 224 00:14:35.559 --> 00:14:36.120 we're dealing with. 225 00:14:36.960 --> 00:14:39.320 There's very little braille on the roads. But if there 226 00:14:39.519 --> 00:14:43.200 was a lot more people driving, uh huh. 227 00:14:43.720 --> 00:14:49.679 I'm sure. So. I conclude success on the perception task 228 00:14:50.120 --> 00:14:55.559 converges to real time virtual rendering of localized reality as 229 00:14:55.639 --> 00:14:59.039 humans perceive it. And because of the nature in which 230 00:14:59.120 --> 00:15:04.759 humans perceive reality, mimic real perception must at least be 231 00:15:04.879 --> 00:15:08.519 inferred from real photons. Again, I have to make the 232 00:15:08.639 --> 00:15:12.480 fundamental argument that reality is perceived from real photons and 233 00:15:12.600 --> 00:15:16.960 not fake photons, which is what lightar is. Real photons 234 00:15:17.000 --> 00:15:24.840 is vision actual reality. Fake photons is lightar. So I 235 00:15:24.960 --> 00:15:28.960 have one more thing, and this is what really ties 236 00:15:29.000 --> 00:15:32.039 it all together. So this is the actual formal end 237 00:15:32.080 --> 00:15:37.159 of the presentation. But I want to show this because this, 238 00:15:37.360 --> 00:15:39.080 in my opinion, if you've been paying it, if you 239 00:15:39.200 --> 00:15:43.039 made it this far, this is the honeypot at the end. 240 00:15:44.759 --> 00:15:50.399 This is a foundational paper in computer vision. This is 241 00:15:50.759 --> 00:15:55.120 one of the most referenced papers in computer vision, one 242 00:15:55.120 --> 00:15:58.759 of the most popular Pepper papers in computer vision, and 243 00:15:58.919 --> 00:16:02.720 considered maybe up there with one of the most important 244 00:16:02.879 --> 00:16:08.600 papers ever in computer vision. Academically recognized, widely accepted and 245 00:16:08.679 --> 00:16:13.440 understood and everything all around the world. Thousands and thousands 246 00:16:13.480 --> 00:16:16.519 of times, maybe tens of thousands of times this specific 247 00:16:16.519 --> 00:16:20.000 paper has been referenced. This is a paper it's called 248 00:16:20.039 --> 00:16:24.279 the Unreasonable Effectiveness of deep Features as a perceptual metric. 249 00:16:25.440 --> 00:16:30.159 This was published by open AI and Adobe Research in 250 00:16:30.320 --> 00:16:34.600 twenty eighteen. Back when Elon was still there at open AI. 251 00:16:35.919 --> 00:16:37.399 If you don't mind, I'm just going to read the 252 00:16:37.399 --> 00:16:42.360 conclusion as is because it's pretty profound our results. So 253 00:16:42.480 --> 00:16:45.120 this is a paper about computer vision, and it explores 254 00:16:45.240 --> 00:16:49.440 the reason why color is important. That's why this is 255 00:16:49.480 --> 00:16:55.039 such a famous paper. Our results indicate that networks trained 256 00:16:55.039 --> 00:16:59.000 to solve challenging visual prediction and modeling tasks end up 257 00:16:59.080 --> 00:17:02.279 learning a represent of the world that correlates well with 258 00:17:02.399 --> 00:17:07.160 perceptual judgments. A similar story has recently emerged in the 259 00:17:07.200 --> 00:17:12.640 representation learning literature. Networks trained on self supervised and unsupervised 260 00:17:12.680 --> 00:17:16.680 objectives end up learning a representation that is also effective 261 00:17:16.720 --> 00:17:19.559 at semantic tasks. Earlier, I was saying being able to 262 00:17:19.599 --> 00:17:23.759 see a red wagon oops allows you in language to 263 00:17:23.759 --> 00:17:25.960 be able to describe a red wagon better. That's what 264 00:17:26.000 --> 00:17:32.640 this is saying. Interestingly, recent findings in neuroscience make much 265 00:17:32.680 --> 00:17:37.799 the same point. Representations trained on computer vision tasks also 266 00:17:38.119 --> 00:17:42.359 end up being effective models of neural activity. In Macaca 267 00:17:42.759 --> 00:17:45.880 visual core text, macakay is a type of monkey that 268 00:17:45.960 --> 00:17:50.000 has a brain very similar to the human. So this 269 00:17:50.160 --> 00:17:52.680 is important because this is the first thing that says 270 00:17:52.960 --> 00:17:59.359 computer vision and the visual cortext might be related. Moreover, 271 00:18:00.319 --> 00:18:03.599 and roughly speaking, the stronger the representation is at the 272 00:18:03.640 --> 00:18:07.400 computer vision task, the stronger it is a model of 273 00:18:07.519 --> 00:18:13.359 cortical activity. Our paper makes a similar finding. The stronger 274 00:18:13.400 --> 00:18:16.440 a feature set is at classification and detection, the stronger 275 00:18:16.559 --> 00:18:19.559 it is as a model of perceptual similarity judgments, as 276 00:18:19.559 --> 00:18:22.599 suggested in Table four. Together, these results suggest that a 277 00:18:22.640 --> 00:18:25.000 good feature is a good feature. Features that are good 278 00:18:25.000 --> 00:18:27.559 at semantic tasks are also good at self supervised and 279 00:18:27.599 --> 00:18:32.680 unsupervised tasks, and also provide good models of both human 280 00:18:33.079 --> 00:18:39.920 perceptual behavior and macake neural activity. This last point aligns 281 00:18:39.920 --> 00:18:45.039 with the rational analysis explanation of visual cognition, suggesting that 282 00:18:45.079 --> 00:18:49.319 the idiosyncrasies of biological perception arise as a consequence of 283 00:18:49.359 --> 00:18:53.880 a rational agent attempting to solve natural tasks. This is 284 00:18:53.960 --> 00:18:58.240 their way of saying, artificial intelligence needs to exist in 285 00:18:58.359 --> 00:19:03.519 the real world and understand the real world, and that 286 00:19:03.680 --> 00:19:07.319 is what is being found that computer vision is enabling 287 00:19:07.480 --> 00:19:11.559 us to do. And on top of that, just happens 288 00:19:11.599 --> 00:19:16.079 to be that when you do that in AI, comprehend 289 00:19:16.119 --> 00:19:20.880 the real world through vision, you end up mimicking the brain, 290 00:19:21.559 --> 00:19:24.000 whether you know it or not, it ends up becoming 291 00:19:24.039 --> 00:19:31.079 the brain. There go further refining the degree to which 292 00:19:31.079 --> 00:19:34.559 this is true is important question for future research. So again, 293 00:19:35.359 --> 00:19:39.119 the human brain is lit I mean sorry. Computer vision 294 00:19:39.680 --> 00:19:44.480 is when you do it correctly, it literally mimics the brain. 295 00:19:45.559 --> 00:19:49.799 This is important to understand everything. There's all kinds of 296 00:19:52.359 --> 00:19:55.359 ridiculous shit that I've heard about the brain and the 297 00:19:55.920 --> 00:20:01.119 concept of consciousness. It's quantum and microtubiles and and pulling 298 00:20:01.160 --> 00:20:04.200 in consciousness from the ether and all this shit. No, 299 00:20:04.839 --> 00:20:09.079 every single thing. One hundred percent of everything we know 300 00:20:09.200 --> 00:20:14.279 in the universe is mathematical. One hundred percent of everything 301 00:20:14.400 --> 00:20:20.839 we know in the universe is mathematical. The brain is mathematical. 302 00:20:21.440 --> 00:20:27.240 And computer vision experts implicitly understand the visual cortex in 303 00:20:27.279 --> 00:20:30.519 the brain, whether they know it or not, because the 304 00:20:30.680 --> 00:20:35.440 visual cortex is doing a step by step function from 305 00:20:35.599 --> 00:20:39.240 V one through V six. In the visual cortex, this 306 00:20:39.400 --> 00:20:43.960 is a step by step sequential function, and people in 307 00:20:44.039 --> 00:20:49.759 computer vision are doing exactly those steps without even knowing it. 308 00:20:49.759 --> 00:20:53.359 They're memory, They're mimicking the brain. I actually have no 309 00:20:53.480 --> 00:20:56.799 idea why there's some giant wall between neuroscience and computer vision, 310 00:20:57.400 --> 00:21:00.119 but computer vision people probably understand the brain better than 311 00:21:00.160 --> 00:21:02.279 most people without even knowing it. 312 00:21:04.359 --> 00:21:08.680 I would agree, because the first things that all life 313 00:21:08.680 --> 00:21:12.279 had to figure out was the world around them, and 314 00:21:12.480 --> 00:21:16.240 all intelligence is built off of that same basic premise 315 00:21:16.359 --> 00:21:18.759 every single time. It's how do you relate to the 316 00:21:18.759 --> 00:21:19.480 world around you? 317 00:21:20.119 --> 00:21:20.440 Correct? 318 00:21:20.559 --> 00:21:23.240 That includes light and all the rest, any sort of 319 00:21:23.279 --> 00:21:25.240 sensory impulse, any kind of information. 320 00:21:26.160 --> 00:21:29.200 Yep. So that's actually what that paper is saying, is 321 00:21:29.240 --> 00:21:32.720 that a rational agent, whatever that is, So you can 322 00:21:32.799 --> 00:21:37.119 either be a conscious human or a AI. You know something, 323 00:21:37.279 --> 00:21:40.759 any AI. It turns out that a rational agent that 324 00:21:40.880 --> 00:21:46.359 successfully interacts in the real world is doing is going 325 00:21:46.440 --> 00:21:49.680 to end up making neural activity in the brain. And 326 00:21:49.920 --> 00:21:51.400 by the way, I just want to make sure that 327 00:21:52.640 --> 00:21:56.519 I distinguish that this is not saying that computer vision 328 00:21:56.599 --> 00:22:01.279 becomes human. This is saying that computer vision and becomes 329 00:22:01.359 --> 00:22:06.440 the brain. And that's a big difference. So everything in 330 00:22:06.480 --> 00:22:10.160 this we're all life in this world, is going to 331 00:22:10.240 --> 00:22:13.559 have some variation of the brain. And we know that 332 00:22:14.160 --> 00:22:17.279 different animals and different species all up and down the 333 00:22:17.319 --> 00:22:21.319 animal kingdom, they all have varying different kinds of success 334 00:22:21.359 --> 00:22:27.359 at perceiving the world. For example, bats, they have eyes 335 00:22:28.079 --> 00:22:30.599 anyone ever wonder why bats have eyes when they don't 336 00:22:30.599 --> 00:22:33.559 really use them. And so one of the things is 337 00:22:33.599 --> 00:22:39.119 that the eye evolved first, and then bats literally got 338 00:22:39.160 --> 00:22:41.960 stuck in a cave. And when they got stuck in 339 00:22:42.000 --> 00:22:45.599 a cave, they had to switch to echolocation, and so 340 00:22:45.680 --> 00:22:48.799 the value of the eye went down, and then they 341 00:22:48.880 --> 00:22:53.920 evolutionary prioritize sonar. But one of the issues is that 342 00:22:53.960 --> 00:22:56.759 they have problems with reflective and smooth surfaces, and they 343 00:22:56.759 --> 00:22:58.680 have all kinds of issues and that's why they got 344 00:22:58.799 --> 00:23:01.680 forever stuck in a cave. They're literally stuck in a 345 00:23:01.720 --> 00:23:07.039 cave now because they went from vision to prioritizing audio sonar. 346 00:23:09.359 --> 00:23:13.160 So so you can see how and that they have 347 00:23:13.960 --> 00:23:18.480 issues with flying near mirrors. If they end up flying 348 00:23:18.519 --> 00:23:20.759 towards the mirror, they'll end up running into it. Windows 349 00:23:20.759 --> 00:23:23.279 are a problem. That's why birds sometimes have problems and 350 00:23:23.319 --> 00:23:27.160 stuff like they don't understand glare that well, the glare 351 00:23:27.240 --> 00:23:30.559 is and the concept and sonar, so you you'd be 352 00:23:30.599 --> 00:23:32.720 surprised at how many like you can probably see glare 353 00:23:32.720 --> 00:23:34.720 on my glasses. So that's how you know that there's 354 00:23:35.039 --> 00:23:37.119 right you can see like that kind of stuff. That's 355 00:23:37.319 --> 00:23:39.279 like obvious. You don't even think about it, but you've 356 00:23:39.279 --> 00:23:40.839 got to think. You've got to understand it. Bad has 357 00:23:40.839 --> 00:23:43.720 no concept of this, And so you can see how 358 00:23:43.839 --> 00:23:49.079 all these different qualitia, subtle qualita of color and texture, 359 00:23:49.640 --> 00:23:54.000 all these subtleties are complicated and are lost when you 360 00:23:54.039 --> 00:23:58.160 don't have vision, but create massive exponential benefit when you do. 361 00:23:59.640 --> 00:24:06.079 So the really big important thing about that's going on 362 00:24:06.160 --> 00:24:11.079 in the world right now is the energy consumption that 363 00:24:11.200 --> 00:24:13.799 is being used for AI, and it's going to grow exponentially. 364 00:24:14.400 --> 00:24:16.279 I wanted to show this graph. This is from a 365 00:24:16.319 --> 00:24:19.480 presentation from Rayker as well, the guy who predicted this singularity. 366 00:24:19.519 --> 00:24:21.000 If you've ever heard of it, If you've ever heard 367 00:24:21.000 --> 00:24:24.319 of the word singularity, it comes from this guy, Raker 368 00:24:24.359 --> 00:24:28.400 as well, and he has this chart, and I just 369 00:24:28.440 --> 00:24:30.680 wanted to show this chart briefly just so people can 370 00:24:30.759 --> 00:24:33.759 understand what is happening with compute. So this is a 371 00:24:33.799 --> 00:24:37.799 log ten scale that means it's increasing by multiple of 372 00:24:37.839 --> 00:24:42.200 ten every step up. So even though this line looks 373 00:24:42.279 --> 00:24:46.279 roughly linear, this is actually actually an exponential graph because 374 00:24:46.279 --> 00:24:50.240 it is going up in factors of ten. So what 375 00:24:50.279 --> 00:24:54.200 you're seeing towards the end here is you're seeing the 376 00:24:54.240 --> 00:24:58.119 beginning of what is called an S curve, and this 377 00:24:58.200 --> 00:25:02.240 is an exponential curve in compute. So there was this 378 00:25:02.279 --> 00:25:06.519 thing called Moore's law, and Moore's law has been had 379 00:25:06.559 --> 00:25:10.599 been in effect for very a very long time, and 380 00:25:10.640 --> 00:25:13.759 what it states is that compute was expected to double 381 00:25:13.799 --> 00:25:17.759 every two years roughly, and that comes from the size 382 00:25:17.759 --> 00:25:20.799 of the transistor. So in other words, they were able 383 00:25:20.839 --> 00:25:23.279 to they were expecting to be able to cut the 384 00:25:23.319 --> 00:25:28.759 size of the transistor by half every two years, so 385 00:25:28.759 --> 00:25:30.799 that means on the same amount of space you fit 386 00:25:30.839 --> 00:25:33.599 about double the number of transistors, so you double your compute. 387 00:25:33.680 --> 00:25:35.680 That was a simple logic that was used, and that's 388 00:25:35.680 --> 00:25:39.559 what Moore's law is. Since then, because of this guy 389 00:25:39.640 --> 00:25:45.519 named Jensen Wang from Nvidia CEO of Nvidia, he's been 390 00:25:45.559 --> 00:25:48.039 doing this GPU thing and he's been blowing Moore's law 391 00:25:48.079 --> 00:25:51.039 out of the water consistently every year for like decades, 392 00:25:51.519 --> 00:25:54.000 and so now it's like way the hell blown out 393 00:25:54.039 --> 00:25:57.559 and you're seeing an exponential curve starting right at the 394 00:25:57.599 --> 00:26:00.240 tail end there, and this is going to keep keep 395 00:26:00.240 --> 00:26:03.920 going because GPUs do all these tasks that I've been 396 00:26:03.960 --> 00:26:08.440 talking about in this presentation way better than any any 397 00:26:08.480 --> 00:26:11.079 CP or anything like that, and he's just crushing it. 398 00:26:11.480 --> 00:26:13.880 He's blown like at this point, More's law is completely 399 00:26:13.880 --> 00:26:17.359 blown out of water, obliterated. I think if I recall correctly, 400 00:26:17.680 --> 00:26:21.880 the mass on his rate of growth in compute is 401 00:26:21.880 --> 00:26:25.359