WEBVTT 1 00:00:00.080 --> 00:00:02.600 I want you to imagine a scenario for a second, like, really, 2 00:00:02.640 --> 00:00:06.160 picture it. Think about the most valuable piece of information 3 00:00:06.240 --> 00:00:06.719 in the world. 4 00:00:06.839 --> 00:00:10.039 Oh man, okay, I'm guessing people are picturing like a 5 00:00:10.160 --> 00:00:12.160 secret chemical formula or something. 6 00:00:12.000 --> 00:00:15.160 Right exactly, or you know, a hit or bank account number, 7 00:00:15.199 --> 00:00:18.359 or maybe a classified document. We always picture a specific 8 00:00:18.800 --> 00:00:20.079 isolated data. 9 00:00:19.839 --> 00:00:22.719 Point, yeah, a single thing in a volt exactly. 10 00:00:23.239 --> 00:00:26.519 But what if? What if the most valuable piece of 11 00:00:26.559 --> 00:00:29.879 information in the world isn't a single data point at all. 12 00:00:30.519 --> 00:00:33.479 What if the real value, the thing that actually drives 13 00:00:33.520 --> 00:00:37.079 the modern economy is the invisible lines connecting all of 14 00:00:37.119 --> 00:00:38.799 those isolated data points together. 15 00:00:38.880 --> 00:00:41.560 Which is wild to think about because it completely flips 16 00:00:41.560 --> 00:00:43.679 how we architect our digital systems. I mean, we are 17 00:00:43.679 --> 00:00:47.759 so conditioned by decades of computing to just, you know, 18 00:00:47.880 --> 00:00:50.640 focus on the individual pieces of the puzzle, the rose 19 00:00:50.679 --> 00:00:54.759 in a spreadsheet, yes, exactly, the rose, the documents in 20 00:00:54.799 --> 00:00:57.799 a folder, and we just completely ignore the architecture of 21 00:00:57.799 --> 00:00:59.240 how they all actually fit together. 22 00:00:59.560 --> 00:01:02.679 And that shift in perspective is exactly what we are 23 00:01:02.719 --> 00:01:07.159 exploring today. Welcome to this deep dive. Our mission today 24 00:01:07.200 --> 00:01:10.120 is to explore a fascinating stack of excerpts from the 25 00:01:10.120 --> 00:01:12.439 book Graph Databases, second Edition. 26 00:01:12.599 --> 00:01:14.519 It's a great rate, honestly, really is. 27 00:01:14.799 --> 00:01:17.200 We're going to look at how the tech giants of 28 00:01:17.239 --> 00:01:21.879 the world actually model reality, why the traditional databases you 29 00:01:21.920 --> 00:01:25.000 probably rely on every day are quietly failing under the 30 00:01:25.000 --> 00:01:29.840 weight of modern connections, and how a three hundred year 31 00:01:29.840 --> 00:01:34.120 old mathematical theory is secretly the engine running today's digital landscape. 32 00:01:34.159 --> 00:01:35.879 It's a lot of ground to cover, but it's so 33 00:01:36.000 --> 00:01:36.920 important it is. 34 00:01:37.000 --> 00:01:39.879 Okay, let's unpack this. Yeah, where do we even begin 35 00:01:39.959 --> 00:01:41.040 with mapping data like this? 36 00:01:41.319 --> 00:01:44.480 Well, to really understand what a graph database is, and 37 00:01:44.719 --> 00:01:46.840 you know why it was even necessary to invent one, 38 00:01:47.159 --> 00:01:49.280 we kind of have to go back to nineteen ninety nine. Well, 39 00:01:49.359 --> 00:01:51.599 the dot com boom exactly. We have to look at 40 00:01:51.599 --> 00:01:53.640 the sheer desperation that led to its creation. 41 00:01:53.840 --> 00:01:57.000 Desperation is a pretty reliable motivator for engineering breakthroughs. Right, 42 00:01:57.159 --> 00:01:59.120 what was the wall they hit back in ninety nine? 43 00:01:59.400 --> 00:02:03.640 So picture a twenty person startup. Everyone is working these 44 00:02:03.799 --> 00:02:08.639 brutal like twenty three hour days. The founders of a 45 00:02:08.639 --> 00:02:13.759 company called neofor j were building an enterprise content management application. 46 00:02:13.840 --> 00:02:16.360 Okay, so managing lots of files and stuff right now. 47 00:02:16.400 --> 00:02:19.599 Storing the actual content like the documents and the images, 48 00:02:19.680 --> 00:02:23.400 that was fine, But their software was managing highly complex 49 00:02:23.520 --> 00:02:26.080 access permissions and hierarchies. 50 00:02:25.400 --> 00:02:28.120 Like who is allowed to see what file and who 51 00:02:28.159 --> 00:02:29.000 their boss is. 52 00:02:28.919 --> 00:02:31.560 That kind of thing exactly, and the engineering team was 53 00:02:31.599 --> 00:02:34.560 spending more than half their time just fighting their relational database. 54 00:02:34.639 --> 00:02:37.159 It just couldn't handle the connections between all those items. 55 00:02:37.680 --> 00:02:40.199 Storing the connected data was complex and queerying it to 56 00:02:40.240 --> 00:02:44.240 see who had access to what was tremendously slow. 57 00:02:44.439 --> 00:02:46.599 So the bottleneck wasn't the size of the files at all. 58 00:02:46.639 --> 00:02:49.080 It was the complexity of the permissions webbing them all 59 00:02:49.120 --> 00:02:50.199 together precisely. 60 00:02:50.280 --> 00:02:53.159 It was a complete nightmare. So out of pure frustration, 61 00:02:53.639 --> 00:02:57.120 they realized they needed a system that natively embraced Eiler's 62 00:02:57.159 --> 00:03:00.400 eighteenth century graph theory waits theory. 63 00:03:00.400 --> 00:03:03.800 Are you talking about the seven bridges of Koenigsberg. Yes, 64 00:03:03.919 --> 00:03:06.280 that's the one where he proved you couldn't cross every 65 00:03:06.280 --> 00:03:08.599 bridge in the city just once, and in doing so 66 00:03:09.080 --> 00:03:11.800 he realized that the actual islands didn't matter, only the 67 00:03:11.800 --> 00:03:12.840 bridges connecting them. 68 00:03:12.879 --> 00:03:17.599 That is the exact mathematical foundation. Yeah, Euler basically abstracted 69 00:03:17.599 --> 00:03:20.319 away the physical land masses and focused entirely on the 70 00:03:20.400 --> 00:03:23.639 links Wow and the neo forj founders, they wanted a 71 00:03:23.719 --> 00:03:26.080 database that did exactly that. And this is a great 72 00:03:26.080 --> 00:03:30.400 detail from their origin story. Actually, they literally altavistad around 73 00:03:30.400 --> 00:03:31.400 the web looking for it. 74 00:03:31.479 --> 00:03:35.560 Altivisdad. That is a perfect callback to the pre Google era, 75 00:03:36.159 --> 00:03:39.680 just searching for highly specific database architecture, right alongside people 76 00:03:39.719 --> 00:03:43.280 looking for pet food onlycos Yeah, exactly, I assume since 77 00:03:43.319 --> 00:03:45.400 it was nineteen ninety nine, altivis that didn't give them 78 00:03:45.439 --> 00:03:46.319 a magical solution. 79 00:03:46.599 --> 00:03:50.159 They found absolutely nothing zero, So they decided to just 80 00:03:50.199 --> 00:03:52.599 build one from scratch. They wanted to keep all the 81 00:03:52.719 --> 00:03:58.240 reliable features of an enterprise relational database, the transactions, recoverability, 82 00:03:58.400 --> 00:04:00.919 data integrity, but build bild a data model for the 83 00:04:00.960 --> 00:04:03.680 twenty first century, which makes sense right, So they built 84 00:04:03.719 --> 00:04:06.840 what is now known as the labeled property graph model. 85 00:04:07.080 --> 00:04:08.960 Let's break down the mechanics of that model, just so 86 00:04:09.000 --> 00:04:11.919 we have a clear picture if I am visualizing this 87 00:04:11.960 --> 00:04:14.439 in my head. What are the actual building blocks here? 88 00:04:14.639 --> 00:04:18.000 Okay? So it has four main characteristics. First, you have nodes. 89 00:04:18.480 --> 00:04:22.480 These represent the entities, so the users, the documents, the 90 00:04:22.519 --> 00:04:23.360 physical location. 91 00:04:23.480 --> 00:04:25.519 Okay, got it. Nodes are the entities right. 92 00:04:25.759 --> 00:04:30.000 Second you have relationships. These are the directed named arrows 93 00:04:30.040 --> 00:04:31.040 connecting those nodes. 94 00:04:31.079 --> 00:04:32.439 The bridge is essentially. 95 00:04:32.079 --> 00:04:36.160 Exactly the bridges. Third, you have properties. These are simply 96 00:04:36.279 --> 00:04:40.000 key value pair stored inside both the nodes and the relationships. 97 00:04:40.279 --> 00:04:43.000 Wait, stored in the relationships too, yes, both. 98 00:04:43.360 --> 00:04:46.279 We'll get to why that matters in a second. And finally, 99 00:04:46.319 --> 00:04:49.519 you have labels, which are tags like user or order 100 00:04:49.680 --> 00:04:52.920 that group specific nodes together so you can find them faster. 101 00:04:53.360 --> 00:04:56.279 Okay, I want to test my understanding here because on 102 00:04:56.319 --> 00:04:59.279 the surface that sounds I don't know, a bit like 103 00:04:59.319 --> 00:05:01.439 a fancy new cod to paint on something we already do. 104 00:05:01.600 --> 00:05:03.800 Hasse well if I am thinking about this like a 105 00:05:03.839 --> 00:05:08.120 sentence structure, the nodes are the nouns the people, places, 106 00:05:08.240 --> 00:05:12.360 or things, and the relationships are essentially the verbs that 107 00:05:12.439 --> 00:05:16.399 give the sentence action and meaning. But isn't that just 108 00:05:16.399 --> 00:05:19.199 a different way of describing a standard database joint. 109 00:05:19.319 --> 00:05:22.879 What's fascinating here is that in a graph database, those verbs, 110 00:05:22.920 --> 00:05:27.120 the relationships, they are first class citizens. They aren't an afterthought, 111 00:05:27.279 --> 00:05:30.639 meaning what exactly. In a traditional database, you construct the 112 00:05:30.680 --> 00:05:34.079 relationship at the exact moment you run the query Basically, 113 00:05:34.160 --> 00:05:36.759 by matching an ID in one table to an ID 114 00:05:36.879 --> 00:05:39.319 in another, you're building it on the fly, right. Okay, 115 00:05:39.360 --> 00:05:42.199 But in a graph, the relationship is physically stored on 116 00:05:42.279 --> 00:05:45.680 the disc as a distinct entity, and because of that, 117 00:05:45.720 --> 00:05:48.800 you can add properties directly to the relationship itself. Oh 118 00:05:48.839 --> 00:05:50.680 I see, Yeah, so you don't just know that Alice 119 00:05:50.720 --> 00:05:54.279 bought an item. The relationship itself holds the metadata like 120 00:05:54.519 --> 00:05:56.920 when she bought it, what the discount code was, the 121 00:05:57.000 --> 00:05:57.800 delivery method. 122 00:05:58.079 --> 00:06:00.439 So the connection holds just as much data as the 123 00:06:00.519 --> 00:06:04.560 entities it connects. That's crazy. Yeah, but if we are 124 00:06:04.600 --> 00:06:08.319 tracking every single interaction as a distinct, data rich verb, 125 00:06:09.000 --> 00:06:11.319 that architecture must be incredibly demand. 126 00:06:11.399 --> 00:06:13.439 It definitely changes how you manage resources. 127 00:06:13.519 --> 00:06:16.600 Yeah, how does this actually fit into a real world 128 00:06:16.639 --> 00:06:19.120 tech stack? I mean, are we talking about a system 129 00:06:19.279 --> 00:06:22.639 sitting directly behind the buy button on a website or 130 00:06:22.720 --> 00:06:25.720 is this something massive crunching analytics overnight. 131 00:06:25.839 --> 00:06:29.600 It actually splits into two distinct categories, and distinguishing between 132 00:06:29.600 --> 00:06:33.279 them is crucial. On one side, you have graph databases. 133 00:06:33.759 --> 00:06:37.959 These are your OLTP systems online transactional processing. They are 134 00:06:38.000 --> 00:06:42.480 designed for real time, highly available transactions accessed directly by your. 135 00:06:42.439 --> 00:06:44.680 Application, so they handle that split second moment when you 136 00:06:44.680 --> 00:06:47.480 click by yes exactly. And what about the analytics side. 137 00:06:47.519 --> 00:06:51.079 Those are graph compute engines. These are your OLAP systems 138 00:06:51.160 --> 00:06:55.839 online analytical processing. Think of tools like Google's pregl or cassivary. 139 00:06:56.079 --> 00:06:59.720 They are built for offline batch processing. They're designed to 140 00:06:59.720 --> 00:07:04.480 add or massive sweeping questions like scanning an entire social 141 00:07:04.519 --> 00:07:07.199 network to find the average number of friends of every 142 00:07:07.279 --> 00:07:08.720 single user globally. 143 00:07:09.160 --> 00:07:11.680 Oh wow, Okay, so the compute engine isn't running the 144 00:07:11.759 --> 00:07:13.920 data day application at all. It sounds like it needs 145 00:07:13.920 --> 00:07:15.680 to pull that data from somewhere else to do the 146 00:07:15.680 --> 00:07:16.800 heavy lifting it does. 147 00:07:17.079 --> 00:07:20.600 Typically, a compute engine pulls data from a system of record, 148 00:07:20.639 --> 00:07:23.720 which could be a traditional database or a graph database, 149 00:07:23.839 --> 00:07:25.079 using an ETL. 150 00:07:24.800 --> 00:07:27.920 Job extract transform loads exactly. 151 00:07:27.959 --> 00:07:31.920 It moves the data, over runs those massive global algorithms 152 00:07:31.920 --> 00:07:35.360 without slowing down the live website, and then returns the results. 153 00:07:35.040 --> 00:07:35.920 That make a lot of sense. 154 00:07:36.000 --> 00:07:39.279 But regardless of whether you are doing real time transactions 155 00:07:39.399 --> 00:07:43.120 or offline batch processing, the true power comes down to 156 00:07:43.199 --> 00:07:47.240 a structural concept called index free adjacency. This is the 157 00:07:47.279 --> 00:07:50.879 dividing line between native and non native graph processing. 158 00:07:51.079 --> 00:07:53.720 Let's unpack index free adjacency, because that sounds like heavy 159 00:07:53.759 --> 00:07:54.920 computer science jargon. 160 00:07:55.040 --> 00:07:56.519 It is a bit yeah if. 161 00:07:56.399 --> 00:07:59.480 I'm translating this is this basically the difference between having 162 00:07:59.800 --> 00:08:02.920 your best friend's phone number memorized in your head versus 163 00:08:02.920 --> 00:08:05.360 having to look their name up in a giant physical 164 00:08:05.360 --> 00:08:07.319 phone book every single time you want to call them. 165 00:08:07.439 --> 00:08:11.439 That analogy hits the underlying mechanism perfectly. In a non 166 00:08:11.560 --> 00:08:14.519 native system, the database might look like a graph on 167 00:08:14.560 --> 00:08:18.439 your screen, but underneath it is still consulting a massive 168 00:08:18.480 --> 00:08:21.199 index the phone book right, the phone book every time 169 00:08:21.199 --> 00:08:23.480 it needs to find a connected node. That look up 170 00:08:23.519 --> 00:08:28.399 takes processing time. But with native index free adjacency, the 171 00:08:28.519 --> 00:08:32.600 relationships are real physical memory pointers. 172 00:08:32.159 --> 00:08:33.919 So it just knows exactly where to go. 173 00:08:34.120 --> 00:08:37.720 Yes, the node for Alice literally contains the exact memory 174 00:08:37.720 --> 00:08:40.559 address for the node of her purchase. The database doesn't 175 00:08:40.559 --> 00:08:42.360 search for it, it just goes directly to it. 176 00:08:42.440 --> 00:08:46.320 That memory pointer concept is critical. But this brings up 177 00:08:46.360 --> 00:08:50.399 a massive pragmatic question. Companies already have massive clusters of 178 00:08:50.440 --> 00:08:54.399 relational databases like SQL or the new or Nosequel systems. 179 00:08:54.759 --> 00:08:58.159 They have spent millions engineering them. So if I am 180 00:08:58.200 --> 00:09:00.960 a CTO listening to this, So why can't I just use 181 00:09:00.960 --> 00:09:04.559 the systems I already have to map these relationships. 182 00:09:04.240 --> 00:09:09.000 Simply because relational databases, despite the name relational, are structurally 183 00:09:09.240 --> 00:09:12.039 terrible at handling rich relationships. 184 00:09:11.360 --> 00:09:13.679 Which is incredibly ironic, it really is. 185 00:09:14.120 --> 00:09:18.000 The relational model forces you into a rigid, tabular forms 186 00:09:18.039 --> 00:09:22.039 based structure. When your data is highly connected. You end 187 00:09:22.159 --> 00:09:25.360 up having to create massive join tables just to map 188 00:09:25.399 --> 00:09:28.440 which row in table A relates to which row and table. 189 00:09:28.200 --> 00:09:31.960 B right, and as the data set scales, those joins 190 00:09:31.960 --> 00:09:35.759 become a mathematical nightmare. You are essentially asking the database 191 00:09:35.799 --> 00:09:39.039 to compute the Cartesian product of two massive lists and 192 00:09:39.120 --> 00:09:40.120 filter out the matches. 193 00:09:40.240 --> 00:09:41.000 Yes, exactly. 194 00:09:41.080 --> 00:09:43.559 Plus if a user doesn't have a certain attribute, you 195 00:09:43.639 --> 00:09:46.000 end up with columns full of null values, taking up 196 00:09:46.039 --> 00:09:47.960 space and complicating the Coke exactly. 197 00:09:48.039 --> 00:09:51.519 The rise in connectedness translates directly into increased joins, and 198 00:09:51.559 --> 00:09:54.159 in SQL, joins exponentially destroyed performance. 199 00:09:54.320 --> 00:09:57.480 Okay, so SQL struggles, But the tech industry's answer to 200 00:09:57.519 --> 00:10:00.240 that over the last decade was no sequel. We move 201 00:10:00.279 --> 00:10:04.720 to aggregate stores, document stores, key value stores precisely to 202 00:10:04.759 --> 00:10:08.600 get away from rigid tables. Don't document stores solve this. 203 00:10:08.919 --> 00:10:11.960 They solve the schema rigidity, sure, but they are actually 204 00:10:12.039 --> 00:10:15.360 worse for connected data, really worse. Yeah. 205 00:10:15.799 --> 00:10:15.840 No. 206 00:10:15.960 --> 00:10:20.320 Single databases are brilliant at storing massive sets of disconnected documents, 207 00:10:20.840 --> 00:10:24.080 but the database itself is entirely blind to any connections 208 00:10:24.080 --> 00:10:27.360 between them. Ohis, to fake a relationship in no SQL, 209 00:10:27.360 --> 00:10:31.000 you have to embed foreign keys, literally typing the ID 210 00:10:31.279 --> 00:10:35.240 of document B inside document A. But the database doesn't 211 00:10:35.240 --> 00:10:37.240 know what that ID points to. It just sees a 212 00:10:37.240 --> 00:10:38.279 string of text. 213 00:10:38.159 --> 00:10:39.480 So it's basically dumb text. 214 00:10:39.639 --> 00:10:42.519 Right. Your application's code has to do the heavy lifting 215 00:10:42.600 --> 00:10:46.360 of pulling document A, reading the ID, and then asking 216 00:10:46.440 --> 00:10:48.159 the database for document. 217 00:10:47.759 --> 00:10:51.080 B, which introduces the performance killer you mentioned earlier. If 218 00:10:51.080 --> 00:10:53.320 I want to reverse lookups, say I am looking at 219 00:10:53.320 --> 00:10:56.480 a specific document for strawberry ice cream and I want 220 00:10:56.519 --> 00:10:59.159 to know every user who bought it, the document store 221 00:10:59.399 --> 00:11:01.519 has no better backward pointing links. 222 00:11:01.360 --> 00:11:03.799 None at all. In computer science terms, this is described 223 00:11:03.919 --> 00:11:07.320 using big O notation because there are no backward pointers. 224 00:11:07.399 --> 00:11:09.799 Finding who bought that ice cream requires an O of 225 00:11:09.960 --> 00:11:11.120 N operation. 226 00:11:10.919 --> 00:11:12.639 And the ND represents the number of items. 227 00:11:12.679 --> 00:11:14.879 Right, yeah, right. It means the database has to do 228 00:11:14.919 --> 00:11:17.399 a brute force scan of every single document in the 229 00:11:17.519 --> 00:11:20.440 entire system to see if it contains that ice cream's ID. 230 00:11:20.960 --> 00:11:24.840 That's agonizingly slow. Even some of the highly distributed stores 231 00:11:24.879 --> 00:11:28.240 that try to map relationships still struggle with this. Right, 232 00:11:28.679 --> 00:11:32.120 the sources mentioned systems like react which use one way links. 233 00:11:32.200 --> 00:11:33.600 Yeah, reac is a good example. 234 00:11:33.679 --> 00:11:37.159 But to query those links backward, they have to run 235 00:11:37.279 --> 00:11:40.519 latent map produce jobs. And a map produced job isn't 236 00:11:40.519 --> 00:11:43.639 a quick lookup. It's a heavy batch process that filters 237 00:11:43.720 --> 00:11:47.720 data across multiple servers, shuffles it, and aggregates it. You 238 00:11:47.720 --> 00:11:50.039 can't use a map reduced job to recommend a product 239 00:11:50.159 --> 00:11:52.000 while a customer is waiting on a checkout page. 240 00:11:52.039 --> 00:11:54.720 You absolutely cannot. The latency is just way too high. 241 00:11:55.080 --> 00:11:57.879 I think here's where it gets really interesting, because a 242 00:11:57.960 --> 00:12:01.240 clever engineer listening might just say, we'll find the database 243 00:12:01.240 --> 00:12:03.720 doesn't do it automatically. I will just build my own 244 00:12:03.759 --> 00:12:06.120 reverse indexes in the no SQL database. 245 00:12:06.240 --> 00:12:08.360 Oh, people try that all the time, right. 246 00:12:08.440 --> 00:12:10.480 They think, every time I write a purchase, I will 247 00:12:10.519 --> 00:12:13.159 manually add a backward link to the user. Does that 248 00:12:13.240 --> 00:12:13.639 solve it? 249 00:12:13.879 --> 00:12:16.799 This raises an important question about the true cost of 250 00:12:16.879 --> 00:12:22.159 distributed systems. Yes, you can denormalize your data and manually 251 00:12:22.240 --> 00:12:26.039 maintain backward links in your application code, but you trigger 252 00:12:26.039 --> 00:12:27.559 a synchronization. 253 00:12:27.080 --> 00:12:29.200 Nightmare because there's too many moving parts. 254 00:12:29.240 --> 00:12:33.799 Exactly, you immediately increase your right latency because every action 255 00:12:33.919 --> 00:12:38.480 now requires updating multiple separate documents simultaneously. 256 00:12:38.000 --> 00:12:41.200 And if the server blinks or crashes right between updating 257 00:12:41.200 --> 00:12:43.879 the ice Cream document and the user document, they are 258 00:12:43.879 --> 00:12:44.759 out of sync. 259 00:12:44.720 --> 00:12:47.399 And that creates what we call dangling references. 260 00:12:47.679 --> 00:12:49.240 Dangling references right. 261 00:12:49.159 --> 00:12:51.600 The user's document points to a purchase that the ice 262 00:12:51.679 --> 00:12:56.080 Cream document doesn't even acknowledge exists. Over time, those dangling 263 00:12:56.120 --> 00:12:59.840 references compound, harming data quality and forcing you to write 264 00:12:59.840 --> 00:13:02.440 even more code just to verify if the links are 265 00:13:02.440 --> 00:13:03.879 still valid before you use them. 266 00:13:04.120 --> 00:13:07.519 It sounds exhausting. So the theory is clear, SQL chokes 267 00:13:07.519 --> 00:13:10.799 on joins and no SQL pushes all the relational processing 268 00:13:10.879 --> 00:13:14.080 onto the application code, causing sync issues. But how does 269 00:13:14.120 --> 00:13:16.679 this actually play out when we put these databases head 270 00:13:16.679 --> 00:13:17.720 to head on a server. 271 00:13:18.000 --> 00:13:21.120 Well, the book neo Forge in Action details a brilliant 272 00:13:21.120 --> 00:13:23.519 experiment by authors Partner and vukotec Oh. 273 00:13:23.519 --> 00:13:24.279 I love this part. 274 00:13:24.440 --> 00:13:27.759 It's so revealing. Right, they set up a concrete head 275 00:13:27.759 --> 00:13:31.399 to head test between a highly tuned relational database and NEOFG. 276 00:13:31.919 --> 00:13:35.159 They modeled a standard social network one million people and 277 00:13:35.200 --> 00:13:37.360 gave each person approximately fifty. 278 00:13:37.000 --> 00:13:41.039 Friends, so one million nodes fifty million relationships. That is 279 00:13:41.080 --> 00:13:44.799 a very standard, realistic data set for a modern application. 280 00:13:45.159 --> 00:13:46.799 What was the actual test? 281 00:13:47.120 --> 00:13:49.679 They asked both databases to find friends of friends at 282 00:13:49.679 --> 00:13:53.320 increasing depths, basically expanding the network outward to see who 283 00:13:53.320 --> 00:13:53.759 knows whom? 284 00:13:53.919 --> 00:13:55.919 Okay, standard networking query, yeah. 285 00:13:56.039 --> 00:13:58.799 So at a depth of two just your immediate friends friends, 286 00:13:58.840 --> 00:14:02.519 Both systems performed perfectly. The relational database took point zero 287 00:14:02.519 --> 00:14:05.960 one six seconds and neofour j took point zero one seconds. 288 00:14:06.159 --> 00:14:08.519 To an end user, that difference is imperceptible. 289 00:14:08.600 --> 00:14:10.679 Okay, so tie game at the shallow end of the pool. 290 00:14:10.720 --> 00:14:12.879 But then they went to depth three friends of friends 291 00:14:12.879 --> 00:14:16.440 of friends, the relational database had to compute massive multi 292 00:14:16.440 --> 00:14:18.720 table joints. It took thirty seconds to run the. 293 00:14:18.759 --> 00:14:20.840 Query Wow, thirty seconds. 294 00:14:20.480 --> 00:14:23.759 Yeah, NEO four j using those native memory pointers took 295 00:14:23.840 --> 00:14:25.360 point one six to eight seconds. 296 00:14:25.559 --> 00:14:28.799 Thirty seconds versus a fraction of a second. If a 297 00:14:28.799 --> 00:14:31.279 web page takes thirty seconds to load a people you 298 00:14:31.320 --> 00:14:34.240 may know widget, the user has already refreshed the page 299 00:14:34.360 --> 00:14:35.919 or left the site entirely. 300 00:14:35.679 --> 00:14:38.759 Absolutely, and the math only gets more unforgiving from there. 301 00:14:39.279 --> 00:14:42.519 At depth four, the relational database took a staggering fifteen 302 00:14:42.639 --> 00:14:46.360 hundred and forty three seconds. Neofour j took one point 303 00:14:46.440 --> 00:14:47.360 three seconds. 304 00:14:47.480 --> 00:14:48.679 That is insane, and when. 305 00:14:48.600 --> 00:14:52.320 They pushed it to depth five, the relational database literally choked. 306 00:14:52.679 --> 00:14:54.879 It threw an error and couldn't even finish the query. 307 00:14:55.360 --> 00:14:58.399 Neofour j traversed the entire five deep network in about 308 00:14:58.399 --> 00:14:59.039 two seconds. 309 00:14:59.080 --> 00:15:01.600 We really need to pop on that number. Fifteen hundred 310 00:15:01.600 --> 00:15:04.360 and forty three seconds is over twenty five minutes. If 311 00:15:04.360 --> 00:15:06.480 you have to wait twenty five minutes for your architecture 312 00:15:06.480 --> 00:15:09.440 to realize that two people are loosely connected, your business 313 00:15:09.480 --> 00:15:12.799 operates in the past completely. If I'm running a modern business, 314 00:15:12.960 --> 00:15:15.879 the agility of one point three second query changes the 315 00:15:15.879 --> 00:15:17.960 fundamental nature of what I can offer the customer. 316 00:15:18.080 --> 00:15:21.320 It allows you to shift from historical analytics to real 317 00:15:21.399 --> 00:15:24.799 time recommendation engines. Let's map out a retail scenario. 318 00:15:24.840 --> 00:15:25.600 Okay, let's hear it. 319 00:15:25.840 --> 00:15:28.360 You have a user, Alice. She just puts strawberry ice 320 00:15:28.399 --> 00:15:31.159 cream in her cart because you have a graph database. 321 00:15:31.200 --> 00:15:33.919 You don't just see the ice cream. You instantly traverse 322 00:15:33.960 --> 00:15:36.639 the memory pointers to find every other user who bought 323 00:15:36.639 --> 00:15:38.840 that ice cream in the last month. Right then you 324 00:15:38.919 --> 00:15:41.519 traverse outward again to see what else they bought. The 325 00:15:41.639 --> 00:15:45.000 database detects a pattern. People who buy this ice cream 326 00:15:45.320 --> 00:15:49.720 frequently buy a specific brand of espressobeans, and crucially, they 327 00:15:49.879 --> 00:15:51.960 strongly dislike Brussels sprouts. 328 00:15:52.159 --> 00:15:55.360 And because that traversal took two seconds instead of twenty 329 00:15:55.360 --> 00:15:59.360 five minutes, you can cross reference those habits and recommend 330 00:15:59.399 --> 00:16:02.559 the Espresso means to Alice right there on the checkout page, 331 00:16:02.639 --> 00:16:05.200 before she even reaches for her credit card exactly. 332 00:16:05.360 --> 00:16:08.279 And it is not just shopping behavior either. We see 333 00:16:08.279 --> 00:16:11.799 this heavily utilized in geospatial data too. You can model 334 00:16:11.840 --> 00:16:16.080 physical locations as graphs using spatial indexing structures called R trees. 335 00:16:16.360 --> 00:16:19.399 Explain how R tree works in a graph context, just briefly. 336 00:16:19.679 --> 00:16:22.240 Sure, think of an R tree as a hierarchy of 337 00:16:22.279 --> 00:16:25.679 bounding boxes. You draw a massive virtual box around the UK. 338 00:16:26.120 --> 00:16:29.320 Inside that, a smaller box around London. Inside that, an 339 00:16:29.360 --> 00:16:32.080 even smaller box for the Battersea neighborhood, down to a 340 00:16:32.080 --> 00:16:33.440 specific postal code. 341 00:16:33.519 --> 00:16:35.679 Okay, Russian nesting dolls. Basically yep. 342 00:16:36.000 --> 00:16:40.679 Because a graph database natively understands hierarchies, it can traverse 343 00:16:40.679 --> 00:16:44.440 those bounding boxes instantly. If someone is standing in Battersea 344 00:16:44.480 --> 00:16:47.480 with their phone, you can do hyper targeted real time 345 00:16:47.480 --> 00:16:50.960 insights based on their exact coordinates. Or you can use 346 00:16:51.000 --> 00:16:54.919 those rapid traversals to detect fraudulent purchasing pattern right. 347 00:16:54.799 --> 00:16:57.720 Because fraud isn't usually an isolated event, it is a 348 00:16:57.759 --> 00:17:01.279 network of shared IP addresses. Stolen device is rapid transaction. 349 00:17:01.399 --> 00:17:01.759 Exactly. 350 00:17:01.799 --> 00:17:04.279 If you could traverse that network in milliseconds, you can 351 00:17:04.319 --> 00:17:06.920 decline the transaction before the bank even clears it. The 352 00:17:06.920 --> 00:17:10.039 power is just undeniable. But here is the final piece 353 00:17:10.079 --> 00:17:12.759 of the puzzle. I am struggling with what's up. If 354 00:17:12.759 --> 00:17:16.039 this architecture is so completely different from relational tables, how 355 00:17:16.039 --> 00:17:18.599 do we actually talk to it? I am obviously not 356 00:17:18.640 --> 00:17:21.480 writing a seql select statement to traverse a bounding box. 357 00:17:21.720 --> 00:17:24.319 Do I need a PhD in graph theory to query this? 358 00:17:24.839 --> 00:17:27.759 Not at all. There is an entire landscape of graph 359 00:17:27.799 --> 00:17:30.839 query languages out there. You have things like spark real, 360 00:17:30.920 --> 00:17:34.279 which is heavily used in academic data, and an imperative 361 00:17:34.359 --> 00:17:38.480 language called Gremlin, which gives you very low level control. 362 00:17:38.680 --> 00:17:42.079 But the absolute industry standard for property graphs is a 363 00:17:42.160 --> 00:17:46.319 language called Cipher. It is a declarative pattern matching language 364 00:17:46.319 --> 00:17:49.920 explicitly designed to be read by humans, not just engineers, 365 00:17:49.960 --> 00:17:51.400 but business stakeholders too. 366 00:17:51.640 --> 00:17:55.119 Wait, if it is not a massive block of SEQL 367 00:17:55.359 --> 00:17:58.279 joy in statements, what does the syntax actually look like? 368 00:17:58.319 --> 00:17:59.960 Am I writing something that looks like Python? 369 00:18:00.200 --> 00:18:03.079 No, you are essentially writing askart ass Yard. 370 00:18:03.079 --> 00:18:03.519 You're kidding. 371 00:18:03.640 --> 00:18:05.920 No, Really, think about how we solve problems in the 372 00:18:05.960 --> 00:18:09.640 real world. When developers and product managers sit in a meeting, 373 00:18:09.920 --> 00:18:13.160 they go to a whiteboard. They intuitively draw a circle 374 00:18:13.200 --> 00:18:15.559 for a user, a circle for an order, and they 375 00:18:15.640 --> 00:18:18.039 draw a hard line with an arrow connecting those. 376 00:18:18.039 --> 00:18:18.839 Sure we all do that. 377 00:18:18.920 --> 00:18:20.799 Cipher was built on the premise that you should be 378 00:18:20.799 --> 00:18:24.559 able to type those exact visual patterns directly into the database. 379 00:18:24.599 --> 00:18:27.240 So I am literally drawing arrows with dashes and greater 380 00:18:27.279 --> 00:18:28.400 than signs on my keyboard. 381 00:18:28.839 --> 00:18:33.319 You literally are in cipher. You put parentheses around a 382 00:18:33.359 --> 00:18:36.440 word to represent a circular node. You use a couple 383 00:18:36.440 --> 00:18:39.279 of dashes and a greater than sign to draw an arrow, 384 00:18:39.920 --> 00:18:41.960 and you put square brackets in the middle of the 385 00:18:42.079 --> 00:18:43.839 arrow to specify the relationship. 386 00:18:44.039 --> 00:18:45.000 That is amazing. 387 00:18:45.119 --> 00:18:48.319 You are literally telling the database engine search the data 388 00:18:48.400 --> 00:18:50.799 and return anything that looks like this picture. 389 00:18:51.440 --> 00:18:55.640 That is wild. It completely removes the semantic dissonance. I mean, 390 00:18:55.799 --> 00:19:00.200 in a traditional setup, you draw the elegant whiteboard diagram, 391 00:19:00.519 --> 00:19:03.000 and then the engineers have to go translate that beautiful 392 00:19:03.039 --> 00:19:08.160 concept into a brutal, abstracted series of database schemas and foreign. 393 00:19:08.000 --> 00:19:10.640 Keys exactly, and so much gets lost in translation. 394 00:19:10.880 --> 00:19:12.960 With cipher, the whiteboard drawing is the code. 395 00:19:13.039 --> 00:19:15.599 If we connect this to the bigger picture, that whiteboard 396 00:19:15.599 --> 00:19:19.640 friendliness is exactly what makes graph database is so highly agile. 397 00:19:19.759 --> 00:19:20.519 Ohso well. 398 00:19:20.519 --> 00:19:23.640 In traditional sequel you have to rigorously define your rigid 399 00:19:23.640 --> 00:19:26.599 tables up front, ironically, at the very beginning of a project, 400 00:19:26.680 --> 00:19:28.200 when you know the least about our data. 401 00:19:28.240 --> 00:19:32.240