WEBVTT 1 00:00:00.120 --> 00:00:03.240 If you open a PDF in like a plain text 2 00:00:03.359 --> 00:00:05.960 editor and you hit the space bar just one single 3 00:00:06.000 --> 00:00:10.679 time and then you hit save, the entire document just shatters. 4 00:00:10.880 --> 00:00:13.359 Yeah, it becomes completely unreadable. 5 00:00:12.720 --> 00:00:15.679 Which is wild. I mean, this is the single most 6 00:00:15.759 --> 00:00:19.320 universally used digital format on the planet. We all use 7 00:00:19.359 --> 00:00:20.280 them every single. 8 00:00:20.120 --> 00:00:23.760 Day, right for you know, signing employment contracts, or reading 9 00:00:23.800 --> 00:00:27.199 those really dense academic research papers, or even just downloading 10 00:00:27.199 --> 00:00:28.280 concert ticket exactly. 11 00:00:28.320 --> 00:00:30.039 We just expect them to work. You want it to 12 00:00:30.079 --> 00:00:34.479 look identical on a tiny smartphone screen or a giant 13 00:00:34.479 --> 00:00:38.719 desktop monitor, and you never really question the architecture underneath. 14 00:00:38.320 --> 00:00:40.759 Which is honestly a minor miracle when you really think 15 00:00:40.799 --> 00:00:41.439 about it. 16 00:00:41.240 --> 00:00:44.399 It totally is. So today's deep dive takes us inside 17 00:00:44.479 --> 00:00:48.799 that invisible architecture. Using our main source today, which is 18 00:00:48.799 --> 00:00:52.679 O'Reilly's book Developing with PDF by Leonard Rosenthal, we are 19 00:00:52.759 --> 00:00:55.719 going to literally pop the hood on the portable document format. 20 00:00:55.880 --> 00:00:58.640 Yeah. And to really understand how you know digital ink 21 00:00:58.719 --> 00:01:00.439 is made, you have to look at the official standard. 22 00:01:00.479 --> 00:01:04.079 It's known as ISO thirty two thousand, and it is 23 00:01:04.159 --> 00:01:09.040 this massive, highly technical specification. I'm talking nearly one thousand 24 00:01:09.120 --> 00:01:12.920 pages detailing everything from coordinate geometry to cryptography. 25 00:01:13.040 --> 00:01:14.959 Yeah, let's be totally honest with the listener here for 26 00:01:15.000 --> 00:01:19.799 a second. The ISO thirty two thousand standard is incredibly dense. 27 00:01:19.959 --> 00:01:22.920 I mean it reads like a stereo manual written biomthematician. 28 00:01:22.959 --> 00:01:25.079 Oh. Absolutely, it's not a light beach read. 29 00:01:25.359 --> 00:01:29.000 Definitely not. Yeah, but our mission today is to translate 30 00:01:29.079 --> 00:01:34.319 that impenetrable specification into an accessible, hopefully mind blowing exploration 31 00:01:34.719 --> 00:01:37.239 of what is actually happening when you double click that 32 00:01:37.280 --> 00:01:38.319 file on your desktop. 33 00:01:38.359 --> 00:01:40.000 I think the best place to start is by just 34 00:01:40.200 --> 00:01:42.560 well throwing out everything you think you know about how 35 00:01:42.560 --> 00:01:43.439 the format works. 36 00:01:43.640 --> 00:01:44.480 Okay, laid on me. 37 00:01:44.799 --> 00:01:46.560 So when you look at a PDF, it appears to 38 00:01:46.599 --> 00:01:48.640 be a flat piece of paper on your screen, right, 39 00:01:48.799 --> 00:01:51.319 like a digital photocopy. 40 00:01:50.560 --> 00:01:51.959 Just a fixed image of a page. 41 00:01:52.040 --> 00:01:54.359 But under the glass it is something entirely different. It's 42 00:01:54.400 --> 00:01:58.840 actually a highly complex, mathematically precise graph like database. 43 00:01:58.840 --> 00:02:01.640 Okay, let's unpack this. If a PDF isn't a single 44 00:02:01.719 --> 00:02:04.359 flat painting, I want you to imagine it more like 45 00:02:05.040 --> 00:02:07.239 a massive bucket of digital lego bricks. 46 00:02:07.519 --> 00:02:09.840 That is a great way to visualize it. Actually, in 47 00:02:09.879 --> 00:02:13.080 the internal architecture, those lego bricks are. 48 00:02:12.960 --> 00:02:15.120 Called objects objects. Okay, yeah, And. 49 00:02:15.080 --> 00:02:18.360 The original TOAD name for Adobe's Acrobat software, and this 50 00:02:18.439 --> 00:02:24.000 whole underlying object system was COS, which stands for Carousel Object. 51 00:02:23.680 --> 00:02:26.919 System parcel object system. That sounds like a piece of 52 00:02:26.960 --> 00:02:29.879 bulky hardware sitting in some nineteen sixties boardroom. 53 00:02:30.080 --> 00:02:32.599 It really does have a retro ring to it, but 54 00:02:32.719 --> 00:02:37.039 the system itself is incredibly robust. Inside this carousel there 55 00:02:37.080 --> 00:02:40.360 are exactly nine types of objects doing all the heavy lifting. 56 00:02:40.560 --> 00:02:42.000 Only nine, just nine. 57 00:02:42.479 --> 00:02:46.439 You have your standard programming staples, so the null object 58 00:02:46.439 --> 00:02:50.360 which just represents a missing value, booleans which handle your 59 00:02:50.400 --> 00:02:55.759 true or false statements, and strings for sequences of text characters. 60 00:02:55.840 --> 00:02:58.520 So it's essentially just a basic programming environment. I'm assuming 61 00:02:58.520 --> 00:03:00.520 we have numbers floating around in the bucket, Well. 62 00:03:00.400 --> 00:03:02.719 Oh, we do. Numeric objects come in two forms. You've 63 00:03:02.719 --> 00:03:05.360 got integers, which are whole numbers, and then real numbers, 64 00:03:05.360 --> 00:03:08.080 which have decimal points, right, And there is a fascinating 65 00:03:08.159 --> 00:03:11.520 quirk here with those. While the format can theoretically handle 66 00:03:11.680 --> 00:03:16.479 massive sixty four bit numbers, the specification actually highly recommends 67 00:03:16.520 --> 00:03:19.680 capping real numbers at just four decimal places. 68 00:03:19.759 --> 00:03:23.479 Wait, really, only four decimal places for a format that's 69 00:03:23.599 --> 00:03:27.039 used in like high end graphic design and architecture. That 70 00:03:27.080 --> 00:03:30.159 seems painfully low. Why limit the precision? 71 00:03:30.280 --> 00:03:33.439 It's a deliberate trade off. It's all about reliability versus 72 00:03:33.439 --> 00:03:37.759 file size. You see, different computer processors and operating systems 73 00:03:37.879 --> 00:03:41.360 handle complex floating point math slightly differently. 74 00:03:41.560 --> 00:03:43.879 Ah, So a MAC and a PC might disagree on 75 00:03:43.919 --> 00:03:45.560 a super long decimal. 76 00:03:45.280 --> 00:03:49.080 Exactly by forcing a hard limit of four decimal places, 77 00:03:49.120 --> 00:03:51.840 the format ensures a coordinate doesn't get rounded up on 78 00:03:51.879 --> 00:03:53.560 a MAC and rounded down on a PC. 79 00:03:53.919 --> 00:03:57.879 So you sacrifice extreme mathematical precision to guarantee that a 80 00:03:57.919 --> 00:04:00.879 graphic designer's layout doesn't shift by a millimeter when I 81 00:04:00.919 --> 00:04:01.680 open it on my phone. 82 00:04:01.719 --> 00:04:03.120 You hit the nail on the head, got it? 83 00:04:03.439 --> 00:04:04.919 Okay? What other their bricks are in the bucket? 84 00:04:05.039 --> 00:04:07.759 Well, we have name objects, which are unique sequences of 85 00:04:07.840 --> 00:04:11.879 characters use for fixed values. Then we get to the containers. 86 00:04:11.520 --> 00:04:13.599 The things that hold other things, right. 87 00:04:13.800 --> 00:04:17.279 Array objects which are ordered lists of other objects, and 88 00:04:17.360 --> 00:04:20.560 dictionary objects, which are collections of key value pairs. 89 00:04:20.680 --> 00:04:21.600 Okay, that makes sense. 90 00:04:21.639 --> 00:04:24.879 And finally you have the heavy lifters, the stream objects. 91 00:04:24.959 --> 00:04:27.519 What goes into a stream That sounds. 92 00:04:27.240 --> 00:04:31.680 Intense streams hold massive blobs of raw data. So this 93 00:04:31.800 --> 00:04:34.240 is where you'd find an embedded font file, or a 94 00:04:34.319 --> 00:04:38.120 highly compressed photograph, or even the actual drawing instructions for 95 00:04:38.160 --> 00:04:38.560 a page. 96 00:04:38.680 --> 00:04:39.079 Oh wow. 97 00:04:39.240 --> 00:04:42.879 Yeah, And unlike dictionaries, streams can be run through filters 98 00:04:42.879 --> 00:04:45.600 to compress them, which is why a PDF with one 99 00:04:45.759 --> 00:04:49.360 hundred high resolution photos doesn't just immediately crash your hard drive. 100 00:04:50.000 --> 00:04:52.560 So these nine types of objects are all floating around 101 00:04:52.600 --> 00:04:55.199 in the bucket. Do they just sit there independently or 102 00:04:55.199 --> 00:04:55.879 do they interact? 103 00:04:55.959 --> 00:04:59.160 Oh? They interact constantly, which brings up the crucial difference 104 00:04:59.160 --> 00:05:01.279 between direct and indirect objects. 105 00:05:01.360 --> 00:05:02.720 Okay, break that down for me. 106 00:05:02.839 --> 00:05:05.959 So a direct object is written inline exactly where it 107 00:05:06.000 --> 00:05:08.759 is needed in the code, but an indirect object is 108 00:05:08.800 --> 00:05:12.000 given a unique ID number and a generation number a 109 00:05:12.079 --> 00:05:15.160 generation number. Yeah, so instead of placing the object data 110 00:05:15.199 --> 00:05:18.040 directly into a page, you just write a reference to 111 00:05:18.120 --> 00:05:20.759 its ID, like a three zero object. 112 00:05:20.839 --> 00:05:23.160 It wait, generation number? What does that actually mean? 113 00:05:23.319 --> 00:05:25.720 Well, in that reference three zero object D, the three 114 00:05:25.800 --> 00:05:28.600 is the ID and the zero is the generation number. 115 00:05:29.079 --> 00:05:31.720 But what is wild is that in modern PDFs that 116 00:05:31.879 --> 00:05:36.399 generation number is almost always zero, like always. 117 00:05:36.279 --> 00:05:39.360 Why included in the specification at all. If it's constantly zero, 118 00:05:39.399 --> 00:05:40.399 that seems redundant. 119 00:05:40.519 --> 00:05:44.680 It's totally a fossilized relic of nineteen nineties programming logic. 120 00:05:44.839 --> 00:05:47.439 Oh really yeah. In the early days, engineers thought that 121 00:05:47.519 --> 00:05:50.240 if you edited a document, the software might override an 122 00:05:50.240 --> 00:05:52.680 object directly in its original place on the hard drive. 123 00:05:52.839 --> 00:05:55.240 I see, So a new version of object three would 124 00:05:55.240 --> 00:05:57.759 become generation one, then generation two exactly. 125 00:05:58.040 --> 00:06:00.360 But as we will discuss later, the format it a 126 00:06:00.399 --> 00:06:03.879 totally different way of saving changes. We never overwrite objects 127 00:06:03.920 --> 00:06:06.480 in place anymore. So the generation number just sits there 128 00:06:06.480 --> 00:06:09.000 at zero, basically a ghost of an abandoned idea. 129 00:06:09.399 --> 00:06:12.240 That is fascinating, But I'm still kind of stuck on 130 00:06:12.439 --> 00:06:15.560 why we are separating things into indirect objects in the 131 00:06:15.560 --> 00:06:18.240 first place. I mean, why not just write the file 132 00:06:18.319 --> 00:06:21.160 logically from top to bottom like a normal word document. 133 00:06:21.199 --> 00:06:23.959 Why give everything an ID number and scatter it around. 134 00:06:24.040 --> 00:06:28.079 Think about a fifty page corporate report, right, okay, Every 135 00:06:28.120 --> 00:06:32.199 single page has a complex high resolution company logo in 136 00:06:32.240 --> 00:06:34.680 the top right corner. If you wrote the file top 137 00:06:34.720 --> 00:06:37.639 to bottom, you would have to embed the massive data 138 00:06:37.720 --> 00:06:39.920 for that image fifty separate times. 139 00:06:40.000 --> 00:06:41.720 Oh man, the file size would be. 140 00:06:41.720 --> 00:06:46.360 Astronomical precisely with indirect objects. The PDF stores the image 141 00:06:46.399 --> 00:06:49.680 data exactly once, assigns it an ID number, and then 142 00:06:49.720 --> 00:06:52.959 the code for all fifty pages just references that one 143 00:06:53.079 --> 00:06:54.519 indirect object, so it just. 144 00:06:54.480 --> 00:06:56.360 Points back to the original brick in the bucket. 145 00:06:56.480 --> 00:06:59.720 Yes, it saves massive amounts of storage space and processing. 146 00:07:00.279 --> 00:07:03.000 Okay, so a PDF is a giant bucket of indirect 147 00:07:03.040 --> 00:07:05.839 objects cross referencing each other. But if it's just a 148 00:07:05.920 --> 00:07:09.399 jumbled bucket, how does a computer assemble a fifty page 149 00:07:09.439 --> 00:07:11.759 document so instantly? I mean has to have some kind 150 00:07:11.759 --> 00:07:13.600 of highly precise map, right, It. 151 00:07:13.519 --> 00:07:16.920 Has a very precise map, and it's dictated by a 152 00:07:17.079 --> 00:07:20.959 rigid four part file structure. You have the header, the body, 153 00:07:21.319 --> 00:07:23.519 the cross reference table, and the trailer. 154 00:07:23.639 --> 00:07:25.759 Let's break those down. The header obviously sits at the very. 155 00:07:25.600 --> 00:07:28.279 Top right, starting a bite zero of the file, it 156 00:07:28.319 --> 00:07:31.920 announces the file type, usually with percent pdf dash, followed 157 00:07:31.920 --> 00:07:32.839 by the version number. 158 00:07:32.920 --> 00:07:34.120 Okay, simple enough. 159 00:07:33.920 --> 00:07:38.079 But immediately after that there is a very bizarre second line. 160 00:07:38.120 --> 00:07:40.879 It usually looks like a string of garbled characters, like 161 00:07:40.959 --> 00:07:44.879 a lower case A with a circumflex and with a 162 00:07:44.959 --> 00:07:47.319 tilled a capital I with an a mellot. 163 00:07:47.639 --> 00:07:49.600 Honestly, if I saw that in a file's code, I 164 00:07:49.639 --> 00:07:51.000 would just assume it was corrupted. 165 00:07:51.160 --> 00:07:54.399 Most people would. What's fascinating here is that those weird 166 00:07:54.480 --> 00:07:58.240 characters are intentionally placed there to trick your computer's operating system. 167 00:07:58.319 --> 00:08:01.680 Trick it how early system would try to automatically detect 168 00:08:01.720 --> 00:08:04.800 if a file was simple ASKI text or a complex 169 00:08:04.879 --> 00:08:08.240 binary file by counting high order ASKI values. 170 00:08:08.399 --> 00:08:09.879 Let me translate that for a second to make sure 171 00:08:09.879 --> 00:08:14.360 I'm falling. ASKI is basically plain unformatted keyboard text, like 172 00:08:14.519 --> 00:08:17.240 typing a basic email. So the computer might see the 173 00:08:17.319 --> 00:08:21.000 underlying code of a compressed image, think it's just weird text, 174 00:08:21.279 --> 00:08:24.000 try to read it as text, and accidentally scramble the data. 175 00:08:24.040 --> 00:08:26.759 You nailed it. If it thought the file was plain text, 176 00:08:26.839 --> 00:08:30.040 it would corrupt the binary streams. So the PDF format 177 00:08:30.160 --> 00:08:34.200 throws those specific high order garbage characters into the very 178 00:08:34.240 --> 00:08:35.200 first few bites. 179 00:08:35.360 --> 00:08:36.240 That's hilarious. 180 00:08:36.320 --> 00:08:39.159 It's essentially shouting at the operating system, I am a 181 00:08:39.159 --> 00:08:41.840 binary file. Do not try to read me as text, 182 00:08:42.000 --> 00:08:44.120 or you will break my formatting. 183 00:08:43.679 --> 00:08:46.799 A built in defense mechanism against early computers. 184 00:08:47.759 --> 00:08:50.480 That is so clever. So after the header is the 185 00:08:50.480 --> 00:08:53.519 body which holds all the lego brick objects we discussed. 186 00:08:54.240 --> 00:08:57.120 Then comes the map you mentioned, the cross reference table. 187 00:08:57.360 --> 00:08:59.919 Yes, and this is the core of the format's speed. 188 00:09:00.559 --> 00:09:04.039 The cross preference table provides the exact binary byte offset 189 00:09:04.080 --> 00:09:07.639 for every single indirect object in the file. Byte offset yeah, 190 00:09:07.679 --> 00:09:10.200 tells the computer object number seven is located exactly six 191 00:09:10.320 --> 00:09:12.120 hundred and thirty one bytes from the very beating of 192 00:09:12.159 --> 00:09:16.600 the file. Wait pause, exact byte offsets? You mean the 193 00:09:16.639 --> 00:09:20.279 computer is literally counting microscopic units of data, bite by 194 00:09:20.320 --> 00:09:22.559 bite from the very top of the file, just to 195 00:09:22.639 --> 00:09:24.759 find where a font or an image lives. 196 00:09:24.879 --> 00:09:28.639 Yes, it is an incredibly rigid mathematical grid that. 197 00:09:28.639 --> 00:09:32.639 Sounds impossibly fragile, which actually brings to mind a scenario 198 00:09:32.679 --> 00:09:34.799 I bet a lot of our listeners have experienced. Say 199 00:09:34.840 --> 00:09:37.120 you're trying to open a PDF, but you accidentally open 200 00:09:37.159 --> 00:09:41.480 it in Microsoft nopad or some other plaintext editor. Oh boy, right, 201 00:09:41.759 --> 00:09:44.679 you see a sea of gibberish, but mixed in you 202 00:09:44.720 --> 00:09:47.679 spot the title of your document in plaintext. You notice 203 00:09:47.679 --> 00:09:49.559 a glaring TYPEO. You think, well, I'm already here. I'll 204 00:09:49.559 --> 00:09:51.799 just delete that extra letter and hit save. But if 205 00:09:51.840 --> 00:09:55.000 you do that, the entire PDF breaks completely. 206 00:09:55.080 --> 00:09:56.679 We have all been tempted to do it, and it 207 00:09:56.720 --> 00:09:57.840 always ends in disaster. 208 00:09:58.080 --> 00:10:01.320 So why does fixing a simple type destroy the document? 209 00:10:01.480 --> 00:10:04.399 It goes right back to that cross reference table. Remember, 210 00:10:04.440 --> 00:10:06.799 it measures the distance to every object down to the 211 00:10:06.799 --> 00:10:09.679 single byte. If you delete a letter in notepad, or 212 00:10:09.720 --> 00:10:12.720 even just add a single invisible space, you have shifted 213 00:10:12.799 --> 00:10:14.480 every single bite that comes after it. 214 00:10:14.519 --> 00:10:17.360 Oh wow, so the map is completely misaligned. Yes. 215 00:10:17.919 --> 00:10:20.519 When the PDF reader goes to look for object seven 216 00:10:20.799 --> 00:10:24.000 at byte six thirty one, it hits byte six thirty 217 00:10:24.039 --> 00:10:27.240 or six thirty two. Instead. The object isn't where the 218 00:10:27.279 --> 00:10:28.240 map says it should be. 219 00:10:28.440 --> 00:10:29.200 It just gives up. 220 00:10:29.279 --> 00:10:33.440 Basically, the reader gets confused, throws a fatal error, and 221 00:10:33.519 --> 00:10:34.639 refuses to open the. 222 00:10:34.600 --> 00:10:38.159 Document, which begs a rather massive question. If exact byte 223 00:10:38.240 --> 00:10:41.440 offsets are so delicate that an invisible space destroys the 224 00:10:41.600 --> 00:10:44.120 entire map, well, how on earth do we ever edit 225 00:10:44.200 --> 00:10:46.879 or save a PDF? I mean, I fill out PDF 226 00:10:46.919 --> 00:10:49.639 forms and save them all the time without the file imploding. 227 00:10:49.799 --> 00:10:51.960 That brings us to the fourth section of the structure, 228 00:10:52.080 --> 00:10:54.559 the trailer. The trailer sits at the very end of 229 00:10:54.559 --> 00:10:56.759 the file and tells the computer where to find the 230 00:10:56.799 --> 00:10:57.799 cross reference table. 231 00:10:57.879 --> 00:10:59.240 Okay, it's a bottom. 232 00:10:58.960 --> 00:11:00.960 Right and this bottom heavy set up allows for a 233 00:11:01.000 --> 00:11:03.320 brilliant workaround called incremental updates. 234 00:11:03.440 --> 00:11:06.080 How to incremental updates bypass the byte offset. 235 00:11:05.759 --> 00:11:09.159 Problem by never touching the original data. Instead of rewriting 236 00:11:09.200 --> 00:11:11.960 the massive, fragile file and risking the corruption of all 237 00:11:11.960 --> 00:11:15.159 those delicate offsets, a PDF saves your changes by simply 238 00:11:15.200 --> 00:11:18.320 appending a new body section, a new cross reference table, 239 00:11:18.360 --> 00:11:20.480 and a new trailer directly to the very end of 240 00:11:20.519 --> 00:11:20.919 the file. 241 00:11:21.240 --> 00:11:23.799 It just tacks the new edits onto the very bottom. 242 00:11:23.919 --> 00:11:27.159 Yes, and that new trailer contains a specific piece of 243 00:11:27.200 --> 00:11:30.879 code called the prev key short for previous. It points 244 00:11:30.919 --> 00:11:33.559 backward to the old trailer, so it reads it in reverse, 245 00:11:33.799 --> 00:11:36.799 kind of when you open the file, the computer reads 246 00:11:36.799 --> 00:11:39.480 it from the bottom up. It sees the newest map, first, 247 00:11:39.639 --> 00:11:42.919 applies your new changes, and ignores the outdated objects. 248 00:11:43.080 --> 00:11:47.279 Here's where it gets really interesting. Wait. If I delete 249 00:11:47.720 --> 00:11:51.600 a highly sensitive paragraph from a document, say a company's 250 00:11:51.600 --> 00:11:55.519 confidential financial projection, and I hit save, the old data 251 00:11:55.759 --> 00:11:56.840 isn't actually erased. 252 00:11:57.759 --> 00:11:59.960 This raises an important question, and you've hit on a 253 00:12:00.200 --> 00:12:03.360 major security flaw that catches people off guard. All the time. 254 00:12:04.080 --> 00:12:06.600 That deleted text might just be uncollected garbage. 255 00:12:06.679 --> 00:12:07.840 Ureuly, anyone can find it. 256 00:12:07.960 --> 00:12:10.320 Yes, because the viewer just reads the newest table at 257 00:12:10.360 --> 00:12:12.799 the end of the file, the old data vanishes from 258 00:12:12.840 --> 00:12:15.120 your screen. It looks deleted. 259 00:12:14.759 --> 00:12:16.919 Because the new map just doesn't point to it anymore. 260 00:12:17.159 --> 00:12:20.600 Exactly, But if you only performed a standard save, that 261 00:12:20.759 --> 00:12:23.960 old text is sitting perfectly intact in the file's history, 262 00:12:24.440 --> 00:12:28.000 buried higher up in the original body section. Anyone with 263 00:12:28.039 --> 00:12:30.960 a basic understanding of PDF structure can dig through that 264 00:12:31.039 --> 00:12:33.840 digital trash and extract your confidential data. 265 00:12:34.320 --> 00:12:38.120 Note to self, always use an official reaction tool. Don't 266 00:12:38.159 --> 00:12:41.480 just hit backspace and assume it's gone. Oh, that is terrifying. 267 00:12:41.600 --> 00:12:42.679 It is definitely a risk. 268 00:12:42.720 --> 00:12:45.960 But this appending system creates a glaring logistical issue. Right 269 00:12:46.279 --> 00:12:48.799 if the trailer and the cross reference table the actual 270 00:12:48.960 --> 00:12:51.399 map to the document are located at the very bottom 271 00:12:51.440 --> 00:12:54.000 of the file, how do PDSs load on the Internet. 272 00:12:54.559 --> 00:12:56.759 Shouldn't a web browser have to download a massive one 273 00:12:56.840 --> 00:12:59.679 hundred megabyte file completely before it can find the map 274 00:12:59.679 --> 00:13:00.799 and show your page one. 275 00:13:00.960 --> 00:13:02.879 Well, in the early days of the Internet, that is 276 00:13:02.919 --> 00:13:05.679 exactly what happened. Really, yeah, it was a miserable user 277 00:13:05.720 --> 00:13:09.080 experience just staring at a blank screen while a massive 278 00:13:09.120 --> 00:13:12.279 file trickled down through dial up. To fix this, the 279 00:13:12.279 --> 00:13:16.919 format introduced linearization, which is more commonly known as fast WebView. 280 00:13:17.399 --> 00:13:19.480 How does linearization cheat the system? 281 00:13:19.679 --> 00:13:23.240 It fundamentally reorders the objects. It takes all the objects 282 00:13:23.279 --> 00:13:26.600 needed to render just page one, so the specific fonts, text, 283 00:13:26.639 --> 00:13:29.279 and images for that page, and groups them together at 284 00:13:29.320 --> 00:13:30.039 the very top of. 285 00:13:29.960 --> 00:13:31.200 The file ah I see. 286 00:13:31.279 --> 00:13:34.360 Crucially, it places a partial cross reference table and a 287 00:13:34.399 --> 00:13:36.200 special trailer right there at the beginning. 288 00:13:36.360 --> 00:13:38.639 So instead of handing me a one thousand paid book 289 00:13:38.840 --> 00:13:41.639 where the index, the glossary, and the first chapter are 290 00:13:41.679 --> 00:13:46.080 scattered everywhere, it basically rips out chapter one, staples a 291 00:13:46.080 --> 00:13:48.200 mini index to the front of it, and hands it 292 00:13:48.240 --> 00:13:50.519 to me to read while the rest of the heavy 293 00:13:50.519 --> 00:13:52.039 book is still printing in the background. 294 00:13:52.120 --> 00:13:55.320 That is a perfect analogy. It allows for instant streaming 295 00:13:55.320 --> 00:13:57.559 of the document and to manage the rest of those 296 00:13:57.559 --> 00:14:02.080 one thousand pages efficiently. The PDFU utilizes a page tree. 297 00:14:02.200 --> 00:14:04.519 The central hub of the document, which is called the 298 00:14:04.559 --> 00:14:07.279 document catalog, points to this tree structure. 299 00:14:07.440 --> 00:14:09.720 Why use a complex tree, though, why not just as 300 00:14:09.799 --> 00:14:12.759 simple chronological list? Page one, page two, page. 301 00:14:12.559 --> 00:14:16.120 Three, because a simple sequential list an array is a 302 00:14:16.200 --> 00:14:19.720 total nightmare for memory usage on large documents. Oh so, 303 00:14:20.039 --> 00:14:22.600 if you have a thousand page manual formatted as a 304 00:14:22.639 --> 00:14:25.440 simple array and you want to jump to page eight hundred, 305 00:14:25.879 --> 00:14:28.279 the computer's memory would have to load and parse the 306 00:14:28.399 --> 00:14:31.120 data for the first seven hundred ninety nine pages sequentially 307 00:14:31.559 --> 00:14:33.559 just to find the location of page eight hundred. 308 00:14:34.039 --> 00:14:36.159 Oh that would bring a phone or an older tablet 309 00:14:36.240 --> 00:14:38.679 to a grinding holt exactly. 310 00:14:39.120 --> 00:14:41.480 So, the page tree is designed as a balanced tree. 311 00:14:41.840 --> 00:14:45.320 The nodes branch out mathematically, and the specification recommends no 312 00:14:45.399 --> 00:14:48.279 more than twenty five to fifty pages per leaf of 313 00:14:48.360 --> 00:14:49.159 the tree, so. 314 00:14:49.080 --> 00:14:50.600 It's clumped into smaller sections. 315 00:14:50.720 --> 00:14:54.200 Yes, this means your device can navigate down specific branches 316 00:14:54.320 --> 00:14:57.879 and jump directly to page eight hundred instantly, completely bypassing 317 00:14:57.879 --> 00:14:59.879 the need to process the hundreds of pages that come 318 00:15:00.080 --> 00:15:04.320 before it. It is an incredibly elegant piece of software engineering. 319 00:15:04.360 --> 00:15:06.480 Okay, so we have the file structured, it's mapped with 320 00:15:06.559 --> 00:15:09.480 exact byteox sets, it's linearized for the web, and we've 321 00:15:09.559 --> 00:15:12.679 navigated the balance tree to our specific page. But the 322 00:15:12.679 --> 00:15:15.679 page is still blank. How does the actual visual content, 323 00:15:16.159 --> 00:15:18.759 the charts, the graphics. How does that get drawn onto 324 00:15:18.759 --> 00:15:19.200 the screen? 325 00:15:19.360 --> 00:15:21.960 Well, to draw anything, you have to establish a coordinate 326 00:15:22.000 --> 00:15:25.399 system on a computer screen. You usually talk in terms 327 00:15:25.480 --> 00:15:28.519 of pixels or dpi dots per inch. This is known 328 00:15:28.519 --> 00:15:31.480 as device space, right, But the core mandate of a 329 00:15:31.519 --> 00:15:34.600 PDF is that it must look identical on a cheap 330 00:15:34.679 --> 00:15:38.360 seventy two dpi monitor and a high end six hundred 331 00:15:38.440 --> 00:15:40.519 dpi professional printing press. 332 00:15:40.840 --> 00:15:42.679 That's the magic trick of the whole format. 333 00:15:42.759 --> 00:15:47.039 To achieve that, PDFs ignore device space entirely and operate 334 00:15:47.080 --> 00:15:50.320 in user space. User space, Yeah, user space is a 335 00:15:50.480 --> 00:15:55.000 fixed absolute mathematical grid. It defaults to exactly seventy two 336 00:15:55.159 --> 00:15:58.840 user units per inch, and the origin point coordinate zero 337 00:15:59.200 --> 00:16:00.759 zero is that the bottom left. 338 00:16:00.559 --> 00:16:03.039 Corner bottom left, not the top left like reading a. 339 00:16:03.000 --> 00:16:05.960 Book, no bottom left, just like a standard Cartesian coordinate 340 00:16:06.000 --> 00:16:09.120 graph from high school geometry. That takes me back on 341 00:16:09.200 --> 00:16:13.080 this absolute grid. The PDF defines five specific boxes to 342 00:16:13.120 --> 00:16:15.440 dictate how the page is handled. The most common is 343 00:16:15.480 --> 00:16:18.360 the media box, which defines the physical paper size like 344 00:16:18.480 --> 00:16:21.559 standard US letter. Then you have the crop box, which 345 00:16:21.600 --> 00:16:23.559 defines the visible area on your screen. 346 00:16:23.759 --> 00:16:25.080 What about the other three boxes. 347 00:16:25.240 --> 00:16:29.159 They cater heavily to professional printing. There is the trim box, 348 00:16:29.200 --> 00:16:32.320 which tells the physical guillotine machine at a printing press 349 00:16:32.440 --> 00:16:34.279 exactly where to slice the paper. 350 00:16:34.480 --> 00:16:36.159 Interesting, there is the bleed. 351 00:16:35.919 --> 00:16:39.200 Box, which gives an extra margin of printed color outside 352 00:16:39.240 --> 00:16:42.320 the trim box. And finally the art box, which was 353 00:16:42.399 --> 00:16:45.360 meant to define a specific artwork area but honestly is 354 00:16:45.480 --> 00:16:46.519 rarely used today. 355 00:16:46.759 --> 00:16:49.799 Wait is the bleed box the reason why when I 356 00:16:49.879 --> 00:16:52.240 try to print a colorful flyer on my home printer, 357 00:16:52.559 --> 00:16:55.639 I always get that annoying white border. But a professional 358 00:16:55.679 --> 00:16:58.799 magazine has color going all the way to the very 359 00:16:58.879 --> 00:17:00.559 edge of the page. 360 00:17:00.799 --> 00:17:03.039 Your home printer can't print to the literal edge of 361 00:17:03.080 --> 00:17:06.079 the paper without making a mess, so it shrinks the image. 362 00:17:06.119 --> 00:17:08.759 A professional press prints the color into the bleed box, 363 00:17:08.880 --> 00:17:11.599 pass the intended edge, and then slices it off at 364 00:17:11.640 --> 00:17:14.480 the trim box, leaving a flawless, borderless image. 365 00:17:14.559 --> 00:17:17.359 Everything is mapped out so logically, so we have our 366 00:17:17.400 --> 00:17:20.319 grid in our boxes, But who is actually holding the pen? 367 00:17:20.599 --> 00:17:23.839 The pen is controlled by content streams. These are sets 368 00:17:23.839 --> 00:17:27.799 of instructions derived from the PostScript language. PostScript instead of 369 00:17:27.839 --> 00:17:31.519 just pasting a pixelated picture of a word. Content streams 370 00:17:31.599 --> 00:17:35.440 use mathematical vectors to draw the literal geometric shapes of 371 00:17:35.480 --> 00:17:39.279 the fonts and graphics. It scales infinitely without blurring. 372 00:17:39.519 --> 00:17:41.519 That's why you can zoom in forever and the text 373 00:17:41.519 --> 00:17:43.000 stays crisp exactly. 374 00:17:43.119 --> 00:17:45.640 But here is the catch. It is not a fully 375 00:17:45.680 --> 00:17:49.799 fledged programming language. There are no variables. You cannot use 376 00:17:49.960 --> 00:17:50.759 if then logic. 377 00:17:51.119 --> 00:17:53.960 Wait, if there are no variables to store data, how 378 00:17:53.960 --> 00:17:56.599 does it remember what color ink it's currently using or 379 00:17:56.599 --> 00:17:57.839 how thick the line should be. 380 00:17:58.079 --> 00:18:00.440 It relies on the graphic state stack. Think of it 381 00:18:00.440 --> 00:18:03.440 as a memory bank for your drawing tools. When writing 382 00:18:03.440 --> 00:18:07.039 a content stream, you use two vital operators, a lowercase 383 00:18:07.119 --> 00:18:09.319 Q to push a state onto the stack and an 384 00:18:09.400 --> 00:18:10.920 uppercase Q to pop it off. 385 00:18:11.240 --> 00:18:12.759 So what does this all mean? Let me see if 386 00:18:12.759 --> 00:18:15.960 I can translate this without using programming terms. It sounds 387 00:18:15.960 --> 00:18:18.759 exactly like a quick save and quick load feature in 388 00:18:18.799 --> 00:18:20.200 a really difficult video game. 389 00:18:20.440 --> 00:18:22.319 That is a very accurate way to look at it. 390 00:18:22.400 --> 00:18:26.640 Yeah, So, say you were drawing a complex architectural diagram. 391 00:18:26.720 --> 00:18:29.039 You set your pen thickness to be really wide and 392 00:18:29.079 --> 00:18:32.079 your ink color to bright red. You hit lowercase qu 393 00:18:32.160 --> 00:18:35.359 quick save that exact state. Then you draw your big 394 00:18:35.519 --> 00:18:38.200 red dash line. Right now, you need to go back 395 00:18:38.200 --> 00:18:41.960 to drawing normal thin black text. Instead of manually writing 396 00:18:42.039 --> 00:18:43.880 new code to reset the color to black and the 397 00:18:43.880 --> 00:18:47.440 pin to thin. You just hit uppercase qu quickload. Your 398 00:18:47.480 --> 00:18:50.240 previous settings are instantly restored, and you keep drawing. 399 00:18:50.640 --> 00:18:53.079 That is exactly how it works. You push a state 400 00:18:53.079 --> 00:18:55.759 with Q, you pop it with Q. If you forget 401 00:18:55.759 --> 00:18:58.759