1 00:00:08,080 --> 00:00:11,800 Speaker 1: Welcome back everyone to another episode of Adventures in DevOps. 2 00:00:11,880 --> 00:00:15,240 And because I'm here all by myself, I've brought as 3 00:00:15,359 --> 00:00:17,679 a small consolation another fact. 4 00:00:17,800 --> 00:00:19,559 Speaker 2: So this one security related. 5 00:00:19,199 --> 00:00:22,440 Speaker 1: Google itself is being used to spoof login attacks and 6 00:00:22,519 --> 00:00:25,199 this is because they made some mistakes with their custom domains. 7 00:00:25,239 --> 00:00:27,039 If you're running a website, you can actually run it 8 00:00:27,079 --> 00:00:30,199 on a Google domain. And so I'm here to say, like, 9 00:00:30,239 --> 00:00:32,960 if you're not already offering past keys with your solution, 10 00:00:33,399 --> 00:00:37,960 it's now more critical than ever realistically, and if that's 11 00:00:37,960 --> 00:00:40,159 something that anyone needs to help with, I guess you 12 00:00:40,200 --> 00:00:44,840 know where to find me. So today I've brought it 13 00:00:44,880 --> 00:00:49,200 on from Observe Inc. And he's the head of engineering 14 00:00:49,240 --> 00:00:52,119 and was previously a founding engineer for the company. We're 15 00:00:52,159 --> 00:00:54,679 hoping to really get into something interesting which is not 16 00:00:54,799 --> 00:00:57,560 just all about it zerability, but really how they built 17 00:00:57,560 --> 00:00:58,880 a platform to scale. 18 00:00:58,920 --> 00:01:00,640 Speaker 3: So welcome on, thanks for having me. More. 19 00:01:00,880 --> 00:01:03,600 Speaker 1: We've actually had a bunch of experts and observability in 20 00:01:03,679 --> 00:01:05,879 the past to focus on different things. So there was 21 00:01:05,920 --> 00:01:10,000 a previous episode with Adriana Velella and all about hotel 22 00:01:10,239 --> 00:01:14,719 and building self healing systems with Sylvan Khalaichi So if 23 00:01:14,840 --> 00:01:17,560 Eddyinge was interested in deep dives and those topics, you know, 24 00:01:17,760 --> 00:01:21,159 we've sort of investigated them. But I'm really interested in 25 00:01:21,239 --> 00:01:25,159 the observability at scale perspective because one of the things 26 00:01:25,159 --> 00:01:28,200 that's always really caught me off guard is just how 27 00:01:28,280 --> 00:01:32,680 much data comes in from the customer systems into a 28 00:01:32,719 --> 00:01:35,680 third party tool. So if you're doing logging or collecting metrics, 29 00:01:35,920 --> 00:01:38,120 it's something that there's just a lot of data being 30 00:01:38,120 --> 00:01:39,200 passed over the wire. 31 00:01:39,120 --> 00:01:41,640 Speaker 3: Right, and you know, machines are very good at generating 32 00:01:42,120 --> 00:01:44,840 a ton of data. You have seen like news like 33 00:01:44,959 --> 00:01:47,959 Mainstance Open the Eye is gating away like more than 34 00:01:48,200 --> 00:01:51,239 one petavite of data per day, and that's you know, 35 00:01:51,359 --> 00:01:54,439 not that rare. You know, we have seen, you know, 36 00:01:54,480 --> 00:01:59,079 among our customers. You know, like multiple cases where people 37 00:01:59,079 --> 00:02:01,799 are sending you know, hundreds of terabytes of data per day, 38 00:02:01,879 --> 00:02:04,280 even like a petabyte. So you know, how do you 39 00:02:04,359 --> 00:02:09,520 kind of store even like store those gigantic amount of 40 00:02:09,599 --> 00:02:12,680 data you know, at scale cheaply? And also how do 41 00:02:12,680 --> 00:02:13,439 you process them? 42 00:02:13,479 --> 00:02:17,159 Speaker 1: And how do you so you know, how to buy megabyte? 43 00:02:17,199 --> 00:02:19,199 You just take it an S three right problem solve? 44 00:02:19,639 --> 00:02:23,199 Speaker 3: Yes, So cloud definitely like the modern cloud is very 45 00:02:23,240 --> 00:02:26,680 magical in terms of that. Even if you consider like 46 00:02:26,879 --> 00:02:29,879 just jumping everything into S three, right, there is still 47 00:02:30,159 --> 00:02:32,960 the challenge of how to organize their data, right, like 48 00:02:33,080 --> 00:02:36,120 what is do you want to do like Culumner store, 49 00:02:36,159 --> 00:02:38,759 do you want to do road store? Like? And if 50 00:02:38,759 --> 00:02:41,400 you put stuff into S three, like how do you 51 00:02:41,719 --> 00:02:43,879 index them? Or do you even index them? 52 00:02:44,000 --> 00:02:44,159 Speaker 4: Right? 53 00:02:44,199 --> 00:02:47,080 Speaker 3: So many of them more than data warehouses, like you know, 54 00:02:47,159 --> 00:02:49,159 like some fake or data breaks, they don't use a 55 00:02:49,159 --> 00:02:52,080 different scheme, right, They don't index their data. They rather 56 00:02:52,080 --> 00:02:55,639 they partition their data and then they have clustered their data. 57 00:02:55,759 --> 00:02:59,120 As a olvisibility vendor like ourselves, we would definitely you know, 58 00:02:59,479 --> 00:03:02,520 do a lot of these kind of custom plustering and 59 00:03:02,719 --> 00:03:05,719 you know, custom data modeling to make sure that you know, 60 00:03:05,840 --> 00:03:08,319 people can access these data S scale. 61 00:03:08,560 --> 00:03:10,960 Speaker 1: So like maybe maybe still some secrets here, like what's 62 00:03:11,039 --> 00:03:15,000 data dog and data bricks and observe bank, Like how 63 00:03:15,039 --> 00:03:16,719 are you how are you storing the data? Are you 64 00:03:16,840 --> 00:03:20,199 using S three for some part of this or is 65 00:03:20,240 --> 00:03:24,199 it you have your own like virtual machines where you're 66 00:03:24,240 --> 00:03:26,919 running your own sort of custom database or some some 67 00:03:27,080 --> 00:03:30,520 database provider that you're that you're managing there too at 68 00:03:30,560 --> 00:03:34,120 the engine, like on your own infrastructure to actually store Yeah. 69 00:03:33,960 --> 00:03:36,280 Speaker 3: Yeah, very good question. So you know when we when 70 00:03:36,280 --> 00:03:39,120 we got started, that was like around two sousand and eighteen. 71 00:03:39,240 --> 00:03:42,479 The basic idea that we had was, hey, you know 72 00:03:43,400 --> 00:03:46,719 around that time, you know, data Warehouse was kind of booming, right, 73 00:03:46,759 --> 00:03:49,719 you have like Snowflake and data breaks, and you look 74 00:03:49,759 --> 00:03:53,479 at them and you thought the kind of the generic 75 00:03:53,840 --> 00:03:56,759 the relational execution engine they built on top of you 76 00:03:56,759 --> 00:04:00,479 know S three essentially Easy two is actually pretty engineering 77 00:04:00,599 --> 00:04:04,400 separation of computation and storage, right, so you can store 78 00:04:04,439 --> 00:04:06,759 your data, you know S three, but then you can 79 00:04:07,439 --> 00:04:10,639 basically essentially spin up Easy two clusters to conscious data 80 00:04:10,680 --> 00:04:12,639 on demand if you just want to store the data, 81 00:04:12,639 --> 00:04:14,439 if you don't want to run an inquiry. You don't 82 00:04:14,439 --> 00:04:16,959 have to pay for all the easy two machines to 83 00:04:16,959 --> 00:04:20,279 be online all the time. Soerability of data vendors, you know, 84 00:04:20,360 --> 00:04:25,199 like data Dog, they usually have their own kind of 85 00:04:25,439 --> 00:04:30,639 you know, preparatory execution engine. Being a startup engineering you 86 00:04:30,680 --> 00:04:32,680 feel like, hey, we probably don't want to spend the 87 00:04:32,720 --> 00:04:34,920 first four or five years of the startup building a 88 00:04:35,000 --> 00:04:38,800 database from from scratch. You know, We're more wanted to 89 00:04:38,839 --> 00:04:42,480 provide values to our customer, like real use cases. 90 00:04:43,000 --> 00:04:44,800 Speaker 1: I mean, it's interesting bringing that up because I actually 91 00:04:44,839 --> 00:04:48,920 think that this is something that Charity uh speaks a 92 00:04:48,920 --> 00:04:51,959 lot about in the past. Actually when she built Honeycomb, 93 00:04:52,040 --> 00:04:54,160 was they did actually roll their own database. 94 00:04:54,240 --> 00:04:55,560 Speaker 2: The thing that kept on coming back is like, no, 95 00:04:55,680 --> 00:04:57,759 that was it was definitely a mistake. 96 00:04:57,639 --> 00:05:02,680 Speaker 3: Like, yeah, it went against all the instincts of me 97 00:05:02,720 --> 00:05:05,360 as an engineer, right, Like I would love to build 98 00:05:05,399 --> 00:05:07,959 a database from scratch. If you really want to build 99 00:05:08,560 --> 00:05:11,279 a solid database from from the ground that bend support 100 00:05:11,319 --> 00:05:13,879 all kinds of functions. Yeah, it's going to take years. 101 00:05:14,079 --> 00:05:15,720 Speaker 1: It's going to be a challenge to do it better 102 00:05:15,759 --> 00:05:19,079 than the ones that are out there today already. Like 103 00:05:19,120 --> 00:05:23,120 you pretty much would have to first employ database architects 104 00:05:23,680 --> 00:05:25,839 who have done this many times before, and then you 105 00:05:25,879 --> 00:05:29,160 spend all of your time and effort building up something 106 00:05:29,199 --> 00:05:32,319 that isn't even going to be really the core competency 107 00:05:32,439 --> 00:05:35,639 or competitive advantage on on what you're what you're selling 108 00:05:35,639 --> 00:05:36,439 as a product. 109 00:05:36,480 --> 00:05:38,639 Speaker 2: So you what did you go with off the shelf? 110 00:05:38,680 --> 00:05:42,399 Speaker 3: Then, yeah, so we went with the snowflake, you know, 111 00:05:42,680 --> 00:05:45,959 so snow fake. There are a bunch of aduct kind 112 00:05:45,959 --> 00:05:50,000 of non technical reasons of going with Snowflake by you know, 113 00:05:50,199 --> 00:05:54,920 using a lot of the the the mechanisms they're providings, 114 00:05:54,959 --> 00:05:58,680 and Snowfake allows doing auto clusteroming of your data. You know, 115 00:05:58,800 --> 00:06:03,839 Snowfake obviously has excellent semi structured data support, so that's 116 00:06:03,839 --> 00:06:07,360 also a thing that we leverage a lot in our solution. 117 00:06:07,560 --> 00:06:10,279 So Snowflake obviously, you know, is you're on top of 118 00:06:10,399 --> 00:06:13,040 S three and easy too, so you know you can 119 00:06:13,160 --> 00:06:16,199 say we kind of also consider that as you know, 120 00:06:16,399 --> 00:06:19,240 building on top of three and easy too, and we 121 00:06:19,279 --> 00:06:22,360 can just data into like Snowflake pretty much. You know, 122 00:06:22,399 --> 00:06:25,839 as three in general, you're probably looking at something like 123 00:06:25,959 --> 00:06:28,879 thirty seconds or like. So that is the fact, right 124 00:06:28,879 --> 00:06:32,399 because you're dealing with like, you know, terabyte or pedabyte 125 00:06:33,439 --> 00:06:35,600 of data, right, But you know, for some of the 126 00:06:35,720 --> 00:06:38,240 use cases you probably want better, right, Like if you 127 00:06:38,399 --> 00:06:41,959 want to do like, you know, real time monitor or 128 00:06:42,079 --> 00:06:45,639 you want to do something like very real time troubleshooting, right, 129 00:06:45,680 --> 00:06:49,160 you will want something that's seconds or even like milliseconds 130 00:06:49,160 --> 00:06:49,800 of latency. 131 00:06:49,839 --> 00:06:53,199 Speaker 1: Okay, so are you did you build like a proxy 132 00:06:54,160 --> 00:06:56,360 layer that sits like on top on top of your 133 00:06:56,399 --> 00:07:00,120 own Snowflake account that has the data from customers being 134 00:07:00,160 --> 00:07:04,800 filtered and then being routed to the appropriate tenant inside there, 135 00:07:04,959 --> 00:07:07,839 and is it just passing the data offer? Is there 136 00:07:07,839 --> 00:07:12,120 something happening in your in your architecture that before the 137 00:07:12,199 --> 00:07:13,160 data is actually entering. 138 00:07:13,240 --> 00:07:16,120 Speaker 3: You know, we obviously use you know, the coppa and 139 00:07:16,160 --> 00:07:19,040 stuff to cute the data before they get into a snowflake. 140 00:07:19,199 --> 00:07:23,199 We are doing a little bit of processing, uh in 141 00:07:23,720 --> 00:07:25,959 that in that part of the system, not a whole 142 00:07:26,000 --> 00:07:28,959 lot right now. I think they're mostly about filtering the 143 00:07:29,000 --> 00:07:31,399 data and doing some kind of data shaping, but we 144 00:07:31,439 --> 00:07:36,040 also do authentication and there are different endpoints that it was. 145 00:07:36,000 --> 00:07:38,720 Speaker 1: So I've never found a good reason to use Kafka 146 00:07:38,879 --> 00:07:41,600 in my experience, and the one the one, the one 147 00:07:41,680 --> 00:07:44,319 bad time that someone wanted to use it in my proximity, 148 00:07:44,439 --> 00:07:47,279 I had vetoed it because at the time it's still 149 00:07:47,319 --> 00:07:50,839 required running zoo keeper, you know, a third party another 150 00:07:51,240 --> 00:07:53,600 tool in order just to manage it. I think that's 151 00:07:53,639 --> 00:07:56,360 finally gone away, But I'm actually curious what the need 152 00:07:56,399 --> 00:07:59,040 would be at the edge in order to process. 153 00:07:59,319 --> 00:08:02,120 Speaker 3: I think for cuf fis mostly just for queuing, because 154 00:08:03,040 --> 00:08:05,480 you know, snowflake, it's a it is fundamentally a batch 155 00:08:05,920 --> 00:08:10,399 based engine, right. So you well, I think right now 156 00:08:10,439 --> 00:08:13,040 it's getting better. They have some kind of native support 157 00:08:13,160 --> 00:08:17,839 for like stream ingesting data, but I think back then 158 00:08:17,920 --> 00:08:20,639 they only support ingesting data in batches. So basically you 159 00:08:20,639 --> 00:08:23,319 need to upload the data to three and then you 160 00:08:23,399 --> 00:08:26,720 have to call Snowflake and then the Snowflake would you know, 161 00:08:27,319 --> 00:08:30,040 find those data and upload those batch of data into 162 00:08:30,560 --> 00:08:34,399 their own three essentially, But then you know, we have 163 00:08:34,440 --> 00:08:36,799 to handle streaming data from the customer, right, so we 164 00:08:36,879 --> 00:08:39,080 need something in the middle to convert the data from 165 00:08:39,200 --> 00:08:43,320 streaming to batch, so and to absorb any kind of 166 00:08:43,360 --> 00:08:46,000 you know, bursting this from the customer, because you know, 167 00:08:46,039 --> 00:08:51,039 with Snowflake uploading stuff, it's a pretty much constant throughput, right, 168 00:08:51,080 --> 00:08:54,720 But when the customers, you know, throwing data, it does 169 00:08:54,799 --> 00:08:56,919 you know, it highly depends on what the customer is doing. 170 00:08:57,000 --> 00:09:00,639 Right now, Like we have these instances like vid you uh, 171 00:09:01,200 --> 00:09:05,759 like intelligence customer who's doing kind of video analytics. Yeah, 172 00:09:05,799 --> 00:09:09,639 plun story they were doing like a video analytics stuff, 173 00:09:10,399 --> 00:09:12,879 and then we were back then it was the World 174 00:09:12,960 --> 00:09:16,000 Cup season and whenever there's a game showing, you can 175 00:09:16,080 --> 00:09:19,559 see the customers traffic is spiked still at that point, 176 00:09:19,679 --> 00:09:22,240 we were not super good at handle some of those spikes, 177 00:09:22,759 --> 00:09:25,799 and we've always had an incident or some thought that 178 00:09:25,840 --> 00:09:29,039 we need to deal with On Sunday. I'm like, ah, yeah, 179 00:09:29,039 --> 00:09:32,759 it's definitely like one of those World Cup incidents that 180 00:09:32,799 --> 00:09:36,440 this customer is is jamming up with. So yeah, so 181 00:09:36,679 --> 00:09:39,000 I kind of need the cup cut to absorb those 182 00:09:39,960 --> 00:09:43,000 those large bursts of data and to smooth in and out. 183 00:09:43,159 --> 00:09:44,919 If you asked me like, do I do I love 184 00:09:45,159 --> 00:09:47,240 cuff cup and announcer is obviously no. 185 00:09:49,559 --> 00:09:51,519 Speaker 1: Yeah, So it sounds like, you know, you're pretty much 186 00:09:51,559 --> 00:09:54,440 having that run on the virtual machines at AWS that 187 00:09:54,799 --> 00:09:55,879 you you control for. 188 00:09:55,919 --> 00:09:56,759 Speaker 2: Handing them beload. 189 00:09:56,799 --> 00:09:59,919 Speaker 1: Now I'm curious then, like, was there ever a consideration 190 00:10:00,200 --> 00:10:01,480 use something like kinesis. 191 00:10:01,600 --> 00:10:06,519 Speaker 3: Yes, so we definitely have thought about it, But I 192 00:10:06,559 --> 00:10:10,120 think eventually we decided to not get super tied to 193 00:10:10,440 --> 00:10:16,759 the ALS ecosystem just because this isn't pretty practical concern, right, 194 00:10:16,840 --> 00:10:19,960 because you know, people will ask us to provide a 195 00:10:20,000 --> 00:10:24,759 GCP or like Azure deployment at some point in the future. 196 00:10:25,480 --> 00:10:27,159 I think right now we already have kind of a 197 00:10:27,240 --> 00:10:30,919 GCP deployment and kind of pouring that between different cloud 198 00:10:31,240 --> 00:10:33,799 is going to be a challenge interesting. 199 00:10:34,120 --> 00:10:36,799 Speaker 1: I know what we do in those circumstances is our 200 00:10:36,840 --> 00:10:38,759 main workloads are still running on a of us, but 201 00:10:38,799 --> 00:10:41,279 when we need to have connectors into other clouds, we 202 00:10:41,320 --> 00:10:44,919 will use like GCP's event arc and I forget the 203 00:10:44,960 --> 00:10:49,200 thing in Azure to connect in and stream our analytics 204 00:10:49,279 --> 00:10:52,399 that we see inside our product to back to our 205 00:10:52,440 --> 00:10:54,080 customers implementations. 206 00:10:54,399 --> 00:10:56,679 Speaker 2: And I can see the opposite happening. I don't know the. 207 00:10:56,600 --> 00:10:59,679 Speaker 1: Corresponding things are to kinesis. I mean that that's been 208 00:10:59,759 --> 00:11:02,320 a much better sell internally for us because we could 209 00:11:02,559 --> 00:11:06,679 more utilize the advancements that the cloud providers are making, 210 00:11:06,759 --> 00:11:10,679 rather than trying to manage all the non serveralist aspects 211 00:11:10,720 --> 00:11:13,679 of spinning up really like easy two machines and then 212 00:11:13,759 --> 00:11:17,600 deploying complicated technology on top that no one internally is 213 00:11:17,600 --> 00:11:21,200 an expert on just to deal with the load. I mean, 214 00:11:21,679 --> 00:11:23,720 I always find can interesting, Like I've always hated it 215 00:11:23,759 --> 00:11:26,320 as a product. You're not in aws, you know, anyone 216 00:11:26,320 --> 00:11:29,159 that's listening. There's two forms of kine asists. There's like 217 00:11:29,399 --> 00:11:32,360 fire hose and then like streams, and I can never 218 00:11:32,399 --> 00:11:35,440 remember the difference, but between what these what these two 219 00:11:35,480 --> 00:11:37,960 things are. One of them does scharding and is for 220 00:11:38,120 --> 00:11:40,200 like pulling data, and the other ones like for pushing 221 00:11:40,240 --> 00:11:40,639 data and. 222 00:11:40,559 --> 00:11:41,519 Speaker 2: Piping it somewhere else. 223 00:11:41,799 --> 00:11:44,159 Speaker 1: But it's always really interesting to me because it seems 224 00:11:44,200 --> 00:11:47,039 like even at small volumes of data, it may work fine. 225 00:11:47,039 --> 00:11:48,720 And then I get the questions like, well, how much 226 00:11:48,840 --> 00:11:51,480 data can you actually funnel at it? And I'm sort 227 00:11:51,519 --> 00:11:53,440 of curious, like do you have you actually know, like 228 00:11:53,480 --> 00:11:55,759 how much like the total customer data that's coming in 229 00:11:55,799 --> 00:11:56,720 through your platform is. 230 00:11:56,919 --> 00:11:59,879 Speaker 3: I don't have a concrete number at top of my head, 231 00:12:00,039 --> 00:12:05,639 but I can say probably multi petabyte per day of 232 00:12:05,799 --> 00:12:08,559 data coming probably more than ten petabytes of. 233 00:12:08,559 --> 00:12:10,840 Speaker 1: Data per day, you know, I honestly have no idea 234 00:12:10,960 --> 00:12:12,200 if that's a lot. 235 00:12:12,840 --> 00:12:15,440 Speaker 3: Yeah, I mean I like it kind of volves on 236 00:12:15,559 --> 00:12:19,000 mind that you know, how much you know, crap people generate, 237 00:12:20,399 --> 00:12:23,399 you guys shouldn't say that as oviserability vendor, but the 238 00:12:23,399 --> 00:12:26,720 sheer amount of data that that comes in and you 239 00:12:26,759 --> 00:12:30,159 get these like billions of rows of logs you know, 240 00:12:30,279 --> 00:12:33,360 every you know, a few seconds, do people actually need 241 00:12:33,519 --> 00:12:36,320 so much soup much data coming coming from their system? 242 00:12:36,399 --> 00:12:38,600 But the truth is, I think if you kind of 243 00:12:38,600 --> 00:12:40,799 field them down. And if you have kind of a 244 00:12:40,840 --> 00:12:43,799 pretty flexible pipeline, right, you allow people to kind of 245 00:12:44,279 --> 00:12:47,240 branch off the data and clean it up and to 246 00:12:47,759 --> 00:12:50,879 transform them into a shape that they can querry more easily. 247 00:12:51,799 --> 00:12:55,320 That actually brings a lot of values for these customers, right. 248 00:12:55,360 --> 00:12:57,000 And he also has kind of a little bit of 249 00:12:57,000 --> 00:12:59,919 benefit of Hey, you know, I don't have to drop data, 250 00:13:00,559 --> 00:13:04,200 so I don't have to pre select, pre choose what 251 00:13:04,360 --> 00:13:06,600 data might be valuable to us, you know, down the 252 00:13:06,639 --> 00:13:07,039 road or not. 253 00:13:07,960 --> 00:13:10,279 Speaker 1: And I say recently, and I really mean, I guess 254 00:13:10,320 --> 00:13:11,919 the last six years or so, I did see a 255 00:13:11,960 --> 00:13:15,480 bunch of startups that popped up whose only goal was 256 00:13:15,559 --> 00:13:18,799 to basically figure out and drop data before it entered 257 00:13:18,879 --> 00:13:22,720 data dog, uh, to avoid getting the cost hit. And 258 00:13:22,519 --> 00:13:26,840 I'm wondering it, like, how you've managed to overcome that. 259 00:13:27,039 --> 00:13:30,519 Is it a matter of you are not charging so 260 00:13:30,679 --> 00:13:33,320 much like you're you're basically charging me like a lower 261 00:13:33,360 --> 00:13:36,360 flat rate that's closer to the infrastructure cost for storing 262 00:13:36,360 --> 00:13:39,600 the data. And then you're it's some sort of usage 263 00:13:39,639 --> 00:13:41,879 based pricing off of the queries or something else, Like 264 00:13:41,879 --> 00:13:44,240 how do you how do you make that trade off 265 00:13:44,360 --> 00:13:45,960 be financially effective. 266 00:13:46,080 --> 00:13:48,200 Speaker 3: Yeah, that's a that's a great question. So there are 267 00:13:48,279 --> 00:13:51,679 there there are multiple aspects of this. So like for 268 00:13:51,679 --> 00:13:54,240 for one, you're you're definitely right that you know, we 269 00:13:54,320 --> 00:13:57,840 don't try to make a lot of money off like 270 00:13:57,960 --> 00:14:01,240 storage cost and I can say is true for you know, 271 00:14:01,279 --> 00:14:04,799 Snowflake basically what just passed down the infrastructure costs of three. 272 00:14:04,960 --> 00:14:09,399 The other part of this, we don't use index and 273 00:14:09,519 --> 00:14:12,559 all that for a lot of our data, so ingestion 274 00:14:12,840 --> 00:14:15,480 you know, comes pretty cheap for us. And we are 275 00:14:15,480 --> 00:14:18,159 also very good at kind of dynamically kind of scaling 276 00:14:18,240 --> 00:14:22,919 up and scaling down the commutational resource. The usage based 277 00:14:22,919 --> 00:14:25,200 the pricing right, so that by itself is a very 278 00:14:25,240 --> 00:14:28,240 interesting topic. You would think, you know, usage based pricing 279 00:14:28,399 --> 00:14:30,360 is a good idea. I think it worked very well 280 00:14:30,399 --> 00:14:34,039 for the business intelligence use cases, like you know, Snowflake 281 00:14:34,159 --> 00:14:37,600 business does very well for their usage based pricing, but 282 00:14:37,639 --> 00:14:41,320 I actually didn't work very well for our customers because 283 00:14:41,600 --> 00:14:43,919 you know, for observability, you really you know, you work 284 00:14:43,960 --> 00:14:48,600 with the budget and you hate surprises. You hate that, hey, 285 00:14:48,840 --> 00:14:53,279 why this month my usage bill is you know through 286 00:14:53,279 --> 00:14:55,320 the roof you want a vendor to say that, hey, 287 00:14:55,440 --> 00:14:57,320 you know, I'm only going to charge you so on. 288 00:14:57,440 --> 00:14:59,759 So the nice thing is, you know, as a vendor, 289 00:15:00,080 --> 00:15:02,320 we also have some legal role of saying that, hey, 290 00:15:02,480 --> 00:15:05,720 if you pay us so much, we can start to 291 00:15:05,840 --> 00:15:09,080 kind of throttle you if you way over, you know, 292 00:15:09,120 --> 00:15:11,519 in terms of your usage. It's kind of like the 293 00:15:11,600 --> 00:15:14,759 mobile phone data the mobile kind of data plan model, right, 294 00:15:14,840 --> 00:15:17,720 So if you're over your limit, you can still use 295 00:15:17,759 --> 00:15:20,840 your phone, like, you can still query data, but it 296 00:15:20,879 --> 00:15:23,399 won't be like super fast, right, so you can still 297 00:15:23,399 --> 00:15:26,399 get your job down. So that's that offers a much 298 00:15:26,440 --> 00:15:27,960 better kind of off ramp. 299 00:15:28,519 --> 00:15:31,960 Speaker 1: That's really interesting because we actually saw the complete opposite 300 00:15:31,960 --> 00:15:34,679 thing when it came to our domain. So we offer 301 00:15:35,879 --> 00:15:38,200 like log in and access control for our customers. We 302 00:15:38,240 --> 00:15:42,080 generate JWTs for the applications that our customers, right, and 303 00:15:42,759 --> 00:15:44,480 the ones that have come to us and said, we 304 00:15:44,480 --> 00:15:47,399 don't want usage based pricing. We want to pay like 305 00:15:47,679 --> 00:15:51,120 per number of users or whatever. And I'm like, what 306 00:15:51,240 --> 00:15:53,879 do you want to happen when you go over that number? 307 00:15:54,120 --> 00:15:57,840 Option number one user can't log in option number two. 308 00:15:58,600 --> 00:16:01,360 You may say you want consistent charges, but that means 309 00:16:01,360 --> 00:16:04,399 you are waiting until the end of the year basically 310 00:16:04,480 --> 00:16:07,120 or end of the month depending on their subscription plan 311 00:16:07,440 --> 00:16:10,679 to get the charges anyway. So is it better to 312 00:16:11,480 --> 00:16:13,919 wait till, you know, six months or seven months to 313 00:16:13,960 --> 00:16:15,720 the end, whatever the end of the year is to 314 00:16:15,840 --> 00:16:18,799 actually feel that hit, or better to get it, you know. 315 00:16:18,879 --> 00:16:20,399 Speaker 2: Per month during the subscription. 316 00:16:21,000 --> 00:16:24,360 Speaker 1: And realistically they're like, oh, yeah, no, actually that's that's bad, 317 00:16:24,519 --> 00:16:28,080 like you know, yeah, obviously, yeah, we don't want to 318 00:16:28,120 --> 00:16:32,000 wait to pay, and we can't have you degrading the 319 00:16:32,159 --> 00:16:34,840 experience for our users. And I'm like, okay, well, now 320 00:16:34,879 --> 00:16:37,799 you know why our infrastructure thing charges in this way. 321 00:16:39,039 --> 00:16:40,519 I mean, I can imagine you're not necessarily in the 322 00:16:40,559 --> 00:16:45,320 critical path of the end user's applications, although I can also, 323 00:16:45,360 --> 00:16:47,840 on the flip side, see the value that you're providing. 324 00:16:47,919 --> 00:16:52,000 So you mentioned like really driving some business specific use cases, 325 00:16:52,000 --> 00:16:54,799 so not necessarily just having the logs available or the 326 00:16:54,840 --> 00:16:57,080 metrics on like whether or not there's uptime. 327 00:16:57,039 --> 00:16:59,320 Speaker 2: Et cetera. But I'm really curious what. 328 00:16:59,279 --> 00:17:02,559 Speaker 1: You've seen there from a like competitive advantage for your 329 00:17:02,559 --> 00:17:06,279 customers who now have access to basically all of the 330 00:17:06,400 --> 00:17:10,079 data from their platform that if they had used something else, 331 00:17:10,279 --> 00:17:11,960 they would not have retained. 332 00:17:12,039 --> 00:17:14,720 Speaker 3: There are a lot of interesting stuff that we we 333 00:17:14,720 --> 00:17:18,000 we we have seeing our customers doing. Right, So for instance, 334 00:17:18,000 --> 00:17:22,079 you know many customers would you know kind of do 335 00:17:22,720 --> 00:17:24,880 you know, using their logs and you know, chasing data, 336 00:17:24,920 --> 00:17:29,200 they can build kind of higher level kind of customer journey, 337 00:17:29,319 --> 00:17:31,680 you know, by doing some kind of aggregation. Right. So 338 00:17:31,720 --> 00:17:34,480 you have you know, these different events belong to the 339 00:17:34,480 --> 00:17:37,160 same user, right, you want we want to aggregate them 340 00:17:37,160 --> 00:17:39,400 and you know, form a session that you know, similar 341 00:17:39,400 --> 00:17:41,000 to the things that you would do with like a 342 00:17:41,039 --> 00:17:44,920 web analytics tool like you know, like Google Analytics and 343 00:17:44,920 --> 00:17:47,319 so on and support, but you can kind of customize 344 00:17:47,599 --> 00:17:49,799 and you can decide what are the events that belong 345 00:17:49,880 --> 00:17:53,079 to the same session, you can aggreate them, aggregate them, right, 346 00:17:53,559 --> 00:17:56,039 So that's that's very common, Like people do that all 347 00:17:56,079 --> 00:17:59,640 the time. And you know many cases people can also 348 00:17:59,759 --> 00:18:02,559 join in with their business data, right like hey, I 349 00:18:02,599 --> 00:18:06,119 have this log message, right it references you know customer one, two, 350 00:18:06,119 --> 00:18:09,319 three for five. That's just an opaque idea. How do 351 00:18:09,359 --> 00:18:11,559 I know you know who customer one to two for 352 00:18:11,640 --> 00:18:14,960 five is? What kind of what customers are affected by 353 00:18:15,119 --> 00:18:18,119 this particularly particular incidence, and you know by how much 354 00:18:18,480 --> 00:18:21,920 the margin of the customer of this month, right, we 355 00:18:21,920 --> 00:18:24,759 can actually calculate that all the way just from log data. Yeah, 356 00:18:24,799 --> 00:18:27,240 we also kind of ingested a lot of the financial 357 00:18:27,279 --> 00:18:30,279 data into you know, our system as well, so that 358 00:18:30,359 --> 00:18:32,720 we can you know, have like the sales data and 359 00:18:32,799 --> 00:18:35,759 all that in one place. What happened can we exactly 360 00:18:35,839 --> 00:18:40,319 pinpoint like, for instance, the one incident that caused us 361 00:18:40,359 --> 00:18:43,160 to you know, have to burn some mature computation for 362 00:18:43,200 --> 00:18:45,839 that customer that I would explain this ten percent you 363 00:18:45,839 --> 00:18:46,720 know margin loss. 364 00:18:46,799 --> 00:18:49,759 Speaker 1: That's interesting, I mean, if I understand you correctly, there's 365 00:18:49,759 --> 00:18:54,440 actually scenarios where they're potentially using observe to store like 366 00:18:54,720 --> 00:18:59,519 almost as their customer or CRM and importing importing the 367 00:18:59,599 --> 00:19:03,240 data from wherever there are other systems are hopefully not 368 00:19:03,319 --> 00:19:08,480 Excel spreadsheets, but could be rather than funneling the data 369 00:19:08,519 --> 00:19:11,799 out of their the customer specific data like you know, 370 00:19:11,920 --> 00:19:15,799 metrics or usage patterns for the whole tenant or customer 371 00:19:15,799 --> 00:19:21,640 account into say a whatever the sales or customer success 372 00:19:21,759 --> 00:19:25,279 organization is doing. So this could be like unfortunately, like 373 00:19:25,279 --> 00:19:27,880 a salesforce or something like that. So it is interesting 374 00:19:27,920 --> 00:19:31,960 to hear that some people are are actually utilizing the 375 00:19:32,000 --> 00:19:35,079 ability inside your tool to make that happen, because from 376 00:19:35,079 --> 00:19:39,359 my experience, these things like happen in the tableaus of 377 00:19:39,400 --> 00:19:43,319 the world, and anyone who is technical in any way 378 00:19:43,599 --> 00:19:45,480 always hates these tools. 379 00:19:45,519 --> 00:19:46,799 Speaker 2: Like it was never the. 380 00:19:46,720 --> 00:19:48,680 Speaker 1: Time where I'm like where I saw an engineer jumping 381 00:19:48,680 --> 00:19:51,119 into Tableau or a looker and was like, Wow, this 382 00:19:51,200 --> 00:19:53,720 is like the best experience ever that never happened. 383 00:19:54,279 --> 00:20:00,240 Speaker 3: Yeah. Yeah, adding for us, you know, we we like 384 00:20:00,279 --> 00:20:03,400 in our internal use cases definitely kind of biased, uh, 385 00:20:03,480 --> 00:20:04,680 you know, for obvious reasons. 386 00:20:05,039 --> 00:20:07,079 Speaker 2: No, No, I don't think. 387 00:20:08,799 --> 00:20:12,000 Speaker 3: Yeah, but I think one thing I would point out is, 388 00:20:12,799 --> 00:20:16,720 you know, because when you build obserability a platform, right, 389 00:20:16,799 --> 00:20:20,240 so essentially you are trying to build uh you know, 390 00:20:20,240 --> 00:20:22,480 from a technical perspective, you're trying to build a pretty 391 00:20:22,480 --> 00:20:25,599 good streaming engine, right, because the data is always kind 392 00:20:25,599 --> 00:20:28,200 of streaming. You know. Compare that with the traditional ETL 393 00:20:28,319 --> 00:20:31,319 kind of system. Right, the ETL you have to specify, Hey, 394 00:20:31,400 --> 00:20:33,640 you know how often I want to run this pipeline, 395 00:20:34,519 --> 00:20:36,359 you know, is it by day or by month? You 396 00:20:36,480 --> 00:20:39,119 have to think about that, right, So it's like forces 397 00:20:39,160 --> 00:20:41,599 you should think this in a batch kind of mode, 398 00:20:42,319 --> 00:20:43,880 And you also have to think about oh, what if 399 00:20:43,920 --> 00:20:46,839 this job failed, I'll have to reach try and like 400 00:20:46,880 --> 00:20:50,160 stitch the data together and so on and so forth. Right, 401 00:20:50,200 --> 00:20:53,000 and but if you build a kind of a streaming 402 00:20:53,240 --> 00:20:56,000 first system, right, the system kind of take care of 403 00:20:56,039 --> 00:20:58,839 that for you, right, so you don't have to think about, Hey, 404 00:20:59,400 --> 00:21:01,359 what how old one I want to run this job? 405 00:21:01,759 --> 00:21:04,759 This system kind of decides based on the incoming volume. 406 00:21:05,319 --> 00:21:08,599 Do I chop it up into like one minute fragment 407 00:21:09,000 --> 00:21:11,079 to process, or do I chop it up into like 408 00:21:11,119 --> 00:21:15,039 ten minute segment to handle. So, if you have this 409 00:21:15,359 --> 00:21:19,759 kind of capable streaming system, you naturally wanted to kind 410 00:21:19,759 --> 00:21:21,839 of do more use cases on top of it because 411 00:21:21,880 --> 00:21:25,599 it's sometimes is easier to express what you want to do. 412 00:21:26,160 --> 00:21:28,799 Speaker 1: So if there was someone out there that thought they 413 00:21:28,839 --> 00:21:31,640 had a similar need to handle such large amounts of 414 00:21:31,640 --> 00:21:34,400 incoming data over the API, I mean you're obviously using 415 00:21:34,440 --> 00:21:37,119 Kafka here to handle some of the load before the 416 00:21:37,759 --> 00:21:40,839 systems which aren't designed to be either item voted or 417 00:21:41,839 --> 00:21:44,400 have the amount of scale that you need. I'm just interested, 418 00:21:44,519 --> 00:21:46,359 like what the whole interaction is here here? 419 00:21:46,400 --> 00:21:47,160 Speaker 2: Like are you. 420 00:21:47,400 --> 00:21:52,400 Speaker 1: Providing custom agents for your customers to run on their 421 00:21:52,480 --> 00:21:55,000 virtual machines? Or is it an SDK or is it 422 00:21:55,079 --> 00:22:00,119 just a matter of exporting hotel compliant data out And 423 00:22:00,240 --> 00:22:03,319 on your side, are you using something to handle the 424 00:22:03,319 --> 00:22:05,720 streaming or is this like custom technology that you had 425 00:22:05,720 --> 00:22:08,319 to build because I don't know, X y Z just 426 00:22:08,839 --> 00:22:11,400 didn't cut it as far as both having the functionality 427 00:22:11,680 --> 00:22:13,559 and also the reliability that you would need. 428 00:22:13,599 --> 00:22:16,960 Speaker 3: In this demain milind sight, we basically do pretty standard stuff. 429 00:22:17,000 --> 00:22:20,160 We do have our own agent that you can install 430 00:22:20,319 --> 00:22:22,599 and you will collect the data. And we also have 431 00:22:22,880 --> 00:22:25,799 you know, standard endpoints like you know, like like like 432 00:22:25,880 --> 00:22:28,279 I said earlier, like you know Hotel and all that. 433 00:22:29,319 --> 00:22:33,000 You know, you can if you have Hotel instrumentation already 434 00:22:33,039 --> 00:22:36,039 done and you can just point your you know, Hotel 435 00:22:36,119 --> 00:22:39,519 library to us and then we'll we'll we'll we'll take 436 00:22:39,559 --> 00:22:42,279 it very easily from there. On our back end, we 437 00:22:42,599 --> 00:22:46,759 do build a lot of custom things for stream processing, 438 00:22:46,839 --> 00:22:50,480 so you know, because Snowflake itself is not a streaming engine, right, 439 00:22:50,519 --> 00:22:54,599 so Snowflake only offers storage and compute, right, and Snowflake 440 00:22:54,680 --> 00:22:59,440 that does offer some form of materialized the view supports, 441 00:22:59,440 --> 00:23:02,599 but we use those mostly because that the pipeline that 442 00:23:02,640 --> 00:23:04,960 we're building is kind of interesting. You know, we want 443 00:23:05,000 --> 00:23:09,119 to have a lot of flexibility for the user to 444 00:23:09,599 --> 00:23:12,400 do streaming data but for observability use case. You can 445 00:23:12,480 --> 00:23:15,799 imagine a lot of cases where I have already collected 446 00:23:15,839 --> 00:23:18,200 a lot of historical data, right, I would want to 447 00:23:18,759 --> 00:23:23,920 you know, reprocess those historical data and using my new pipeline. Right. 448 00:23:24,000 --> 00:23:27,160 So that's one of the main reasons where we build 449 00:23:27,240 --> 00:23:31,400 our own kind of streaming systems sitting on top of Snowflake, right. 450 00:23:31,440 --> 00:23:35,400 And you know, just to add kind of another little 451 00:23:35,400 --> 00:23:38,680 bit aspect of this is, you know, we also didn't 452 00:23:38,839 --> 00:23:42,319 use the existing materialize the view solution provided by Snowflake 453 00:23:42,720 --> 00:23:45,759 big get that allows you to backfield because you know, 454 00:23:45,839 --> 00:23:48,000 you can just start a new view and you let 455 00:23:48,039 --> 00:23:52,039 it materialize, but then it's going to materialize the whole thing, right, 456 00:23:52,160 --> 00:23:55,240 And imagine the cases where you have already collected historical 457 00:23:55,319 --> 00:23:58,160 data for like a year, right, and you know just 458 00:23:58,920 --> 00:24:01,920 reprocessing that whole year worth of data is going to 459 00:24:01,960 --> 00:24:04,640 be very expensive and very consuming. 460 00:24:04,720 --> 00:24:07,240 Speaker 1: So you know, this is actually made me think because 461 00:24:07,359 --> 00:24:09,160 this may be the first time I've actually heard someone 462 00:24:09,400 --> 00:24:13,759 utilizing Snowflake as there like as a core component to 463 00:24:13,799 --> 00:24:16,319 the architecture. Is this like an expected use case that 464 00:24:16,359 --> 00:24:20,480 Snowflake supports or is it I mean, like, is there 465 00:24:20,519 --> 00:24:23,079 like a concern here that Snowflake will be like, no, 466 00:24:23,200 --> 00:24:26,039 I'm sorry, you can't like embed this as a as 467 00:24:26,079 --> 00:24:28,519 a you know, part of fundamental product if you're building 468 00:24:28,519 --> 00:24:30,920 something that sort of competes with it in any way, 469 00:24:31,960 --> 00:24:33,839 Is that a concern at all? Or is it just 470 00:24:33,839 --> 00:24:35,880 like no, this is like sort of supported. There are 471 00:24:35,880 --> 00:24:37,200 actually lots of companies doing that. 472 00:24:37,240 --> 00:24:39,519 Speaker 3: The answer to this is kind of there are different 473 00:24:39,559 --> 00:24:42,480 different layers of this. So I think from the business standpoint, 474 00:24:42,559 --> 00:24:44,960 I think Snowflake wants to become kind of a platform 475 00:24:44,960 --> 00:24:47,240 player people building an application on top of it. I 476 00:24:47,279 --> 00:24:50,119 think to that, and you know, I think the businesses 477 00:24:50,119 --> 00:24:54,119 are pretty aligned, so they would like people to like, 478 00:24:54,279 --> 00:24:58,599 like us to kind of having a deeply integrated application 479 00:24:58,720 --> 00:25:01,640 running on top of Snowflake. With that said, you know, 480 00:25:01,680 --> 00:25:04,599 we are also pretty unique in the ways of how 481 00:25:04,640 --> 00:25:07,319 we use ston Flake, because you know, your usual data 482 00:25:07,359 --> 00:25:11,000 application on top of Snowflake would be, hey, I offer 483 00:25:11,079 --> 00:25:14,039 some customer UI to you know, querry the data that 484 00:25:14,200 --> 00:25:18,640 have already been stored in Snowflake, and totally because we 485 00:25:18,839 --> 00:25:23,240 run a data pipeline like in a streaming faction inside Snowflake, 486 00:25:23,400 --> 00:25:26,680 we are comptable for like over two percent of Snowflake 487 00:25:26,759 --> 00:25:27,799 query at this point. 488 00:25:28,000 --> 00:25:30,440 Speaker 2: Wow, that's quite the impact there. 489 00:25:30,559 --> 00:25:34,079 Speaker 3: Yeah, So we helped discover a lot of bugs obviously 490 00:25:34,160 --> 00:25:37,240 in their in their system. So we have a kind 491 00:25:37,240 --> 00:25:40,440 of a like a love hate relationship with their engineers. 492 00:25:41,000 --> 00:25:44,519 So on one hand, they like us because you know, 493 00:25:44,559 --> 00:25:47,160 we helped kind of stress test a lot of aspects 494 00:25:47,200 --> 00:25:49,920 of their infrastructure. On the other hand, I can only 495 00:25:50,279 --> 00:25:53,680 guess that we cause some you know, slipless nights for 496 00:25:53,759 --> 00:25:55,079 their Hong Kore engineers. 497 00:25:55,279 --> 00:25:57,599 Speaker 1: You're lucky in this way if it's good to evaluate 498 00:25:57,640 --> 00:26:00,559 the platform that platforms that you're utilizing before making a 499 00:26:00,559 --> 00:26:03,079 decision on how you're relying on them, because if the 500 00:26:03,079 --> 00:26:06,240 fundamental direction that it's going, the particular product you're using 501 00:26:06,279 --> 00:26:09,559 or dB engine or whatever third party is different from 502 00:26:09,640 --> 00:26:12,519 their expected usage, you could run into I mean, not 503 00:26:12,640 --> 00:26:16,000 just downtime, but really just fundamental limitations at some point 504 00:26:16,039 --> 00:26:18,200 where they're like, I'm sorry, we can't help you. 505 00:26:18,440 --> 00:26:20,359 Speaker 2: So I guess in that way, you're lucky. 506 00:26:20,400 --> 00:26:23,079 Speaker 1: But I can also understand the flip side of it 507 00:26:23,160 --> 00:26:26,119 being so critical not just for a single customer, but 508 00:26:26,160 --> 00:26:29,039 basically your whole business. Now when there's an issue with Snowflake, 509 00:26:29,079 --> 00:26:31,440 and I don't mean like an incident, but just realistically 510 00:26:31,759 --> 00:26:35,839 finding an unknown edge case that isn't supported, Like, oh, yeah, 511 00:26:35,880 --> 00:26:38,680 you know, we're hitting the limits of the throughput that 512 00:26:38,839 --> 00:26:42,039 is available here because I don't know Snowflake never imagine 513 00:26:42,039 --> 00:26:43,960 that there'd be so many coming in through like a 514 00:26:43,960 --> 00:26:47,119 single account or Another problem that we see often is 515 00:26:47,200 --> 00:26:50,440 like the customer of a customer of a customer issue 516 00:26:50,440 --> 00:26:53,160 where it's like it's easy for you as a customer 517 00:26:53,160 --> 00:26:56,759 of a product to add multi tenancy, and the product 518 00:26:56,799 --> 00:27:00,119 you're using may support multi tenancy, but if your customer 519 00:27:00,279 --> 00:27:02,680 are also multi tenant solutions and they want to put 520 00:27:02,720 --> 00:27:05,920 tenants in your solution, then you're passing you know, tenants 521 00:27:05,920 --> 00:27:07,799 of tenants into your provider. 522 00:27:07,839 --> 00:27:09,039 Speaker 2: Its like you know, turtles all the. 523 00:27:08,960 --> 00:27:11,440 Speaker 1: Way down, and I can imagine not all those things 524 00:27:11,480 --> 00:27:14,440 may have been built out effectively. So do you see 525 00:27:14,440 --> 00:27:17,559 a future that is just like well, you know, as 526 00:27:17,559 --> 00:27:22,799 we grow in either scale volume, amount of queries or 527 00:27:23,079 --> 00:27:28,319 monetary value, where Snowflake no longer becomes the critical backbone 528 00:27:28,359 --> 00:27:31,480 for how you're storing and managing that. 529 00:27:31,720 --> 00:27:34,640 Speaker 3: We definitely took kind of a leap of faith at 530 00:27:34,680 --> 00:27:37,839 the beginning of the company. Could you know, be like 531 00:27:38,160 --> 00:27:42,079 a case where you know, or different alternative reality where 532 00:27:42,160 --> 00:27:45,480 you know, just Snowflake couldn't handle whatever that we're throwing 533 00:27:45,519 --> 00:27:48,160 at it. The way I'm thinking about the future is 534 00:27:48,319 --> 00:27:50,960 really that you know, now you know, we have like 535 00:27:51,119 --> 00:27:54,559 you know, Iceberg and all those kind of open format 536 00:27:54,720 --> 00:27:57,240 for storage, right, so you kind of can can see 537 00:27:57,240 --> 00:28:01,640 the trend where the storage is getting essentially commoditized. Basically 538 00:28:01,720 --> 00:28:03,799 there's not a whole lot of money to be made 539 00:28:03,960 --> 00:28:07,960 on storage, and also there's not a lot of useful 540 00:28:08,039 --> 00:28:11,960 kind of preparatory stuff in terms of storage format, right, 541 00:28:12,000 --> 00:28:15,079 so everybody is kind of converging into these open format 542 00:28:16,160 --> 00:28:18,960 So that's you know, definitely good news for us, right 543 00:28:19,039 --> 00:28:23,160 because if we can leverage those open format then that means, 544 00:28:23,240 --> 00:28:26,400 you know, we have a choice of different execution engines 545 00:28:26,400 --> 00:28:28,680 we can run on top, right, like Snowflake, maybe one 546 00:28:28,720 --> 00:28:31,519 of them, like Snowflake is very good at doing like 547 00:28:31,680 --> 00:28:35,559 large scale you know, joints and that kind of stuff, right, 548 00:28:35,640 --> 00:28:38,240 But then if I just want to do something simple, right, 549 00:28:38,359 --> 00:28:41,440 something like the scan terabyte of data and doing like 550 00:28:41,519 --> 00:28:45,680 quick aggregation, Right, maybe another different execution engine would be 551 00:28:45,759 --> 00:28:48,680 cheaper to run and would be more efficient, right, So 552 00:28:48,960 --> 00:28:52,640 I think that's kind of like the way of kind 553 00:28:52,640 --> 00:28:56,640 of leverage and kind of hand our bats a little 554 00:28:56,640 --> 00:28:59,799 bit across different execution platforms. 555 00:29:00,000 --> 00:29:01,799 Speaker 1: That makes a lot of sense, I mean, and I'm 556 00:29:02,119 --> 00:29:04,440 now you've reached the sort of the edge of my 557 00:29:04,599 --> 00:29:07,279 experience here. So there's parkt format, which is like a 558 00:29:07,319 --> 00:29:10,640 columbar format that actually stores optimized data so you don't 559 00:29:10,680 --> 00:29:13,200 have to repeat the property values and it's easy to 560 00:29:13,240 --> 00:29:15,200 filter append. 561 00:29:14,880 --> 00:29:15,839 Speaker 2: Only logs, for instance. 562 00:29:15,839 --> 00:29:17,359 Speaker 1: And this is like one of the most common ways 563 00:29:17,400 --> 00:29:20,000 in which even AWS is storing the data within S three. 564 00:29:20,200 --> 00:29:22,519 And then you mentioned Apache Iceberg. I know there isn't 565 00:29:22,559 --> 00:29:24,799 like full support in AWS yet, but I am sort 566 00:29:24,839 --> 00:29:27,519 of curious, like what's the fundamental difference between these And 567 00:29:27,799 --> 00:29:30,039 was the decision to use that like one particular format 568 00:29:30,119 --> 00:29:33,319 versus another something like a one way door where you 569 00:29:33,400 --> 00:29:35,359 sort of made the decision early on. It's like that's 570 00:29:35,359 --> 00:29:37,920 the way you're going at least not coupled to a 571 00:29:37,960 --> 00:29:40,400 proprietary format, or is it something that you think is 572 00:29:40,400 --> 00:29:42,759 like pretty easy to switch out down the road, Because 573 00:29:42,759 --> 00:29:44,079 it's like an internal detail. 574 00:29:44,480 --> 00:29:48,079 Speaker 3: Or let's just say park is one of the format 575 00:29:48,319 --> 00:29:52,160 that Iceberg tables can be stored at park. If you 576 00:29:52,200 --> 00:29:54,319 think about it, just a bunch of S three files, 577 00:29:54,599 --> 00:29:58,400 there's no concept of a table, right. So from a 578 00:29:58,519 --> 00:30:00,920 database perspective that you fuy on the table, I have 579 00:30:01,000 --> 00:30:04,119 to know, you know, what is a schema of the table, right, 580 00:30:04,160 --> 00:30:07,400 what columns it had? And I need to need some 581 00:30:07,440 --> 00:30:09,960 metadator to tell me, hey, in order to querry this, 582 00:30:10,359 --> 00:30:13,359 you know this table from row like one hundred to 583 00:30:13,680 --> 00:30:17,640 one thousand, which PARKT file I should scan? Right, That's 584 00:30:17,759 --> 00:30:21,839 essentially what Iceberg provides. So Iceberg provide this metadata to 585 00:30:22,039 --> 00:30:26,680 connect the concept of table with the raw parquet files 586 00:30:26,680 --> 00:30:29,160 that you store on at three if if if you simplify, 587 00:30:29,279 --> 00:30:32,759 really a patch of Iceberg is nothing more than just 588 00:30:32,839 --> 00:30:36,519 a bunch of metadata files that you store alongside of 589 00:30:36,559 --> 00:30:39,720 your parquet files, right. And I think that the good 590 00:30:39,720 --> 00:30:43,759 thing about Iceberg is, you know it it offers a 591 00:30:44,359 --> 00:30:48,480 capability of doing what's called puning on your on your data. Right. 592 00:30:48,519 --> 00:30:52,440 So for instance, if this parkat file the value is 593 00:30:52,440 --> 00:30:55,400 between one hundred and you know two hundred, right, and 594 00:30:55,440 --> 00:30:57,359 if I have a quarry saying Hey, I only want 595 00:30:57,400 --> 00:30:59,880 to I want to filter down the data to anything 596 00:31:00,039 --> 00:31:02,559 with a value greater than two hundred and do you 597 00:31:02,640 --> 00:31:05,279 know that this park file will never contain any from 598 00:31:05,680 --> 00:31:09,599 any roads that would satisfy this future predicate, right, so 599 00:31:09,640 --> 00:31:13,440 you can choose to not scan that parquet file at all. Right. 600 00:31:13,519 --> 00:31:18,200 So Iceberg basically offered this way these metadata for the 601 00:31:18,240 --> 00:31:23,839 career engines to proactively prone out these useless parky files 602 00:31:24,359 --> 00:31:28,279 and to make the career efficient. So that's also a 603 00:31:28,359 --> 00:31:32,039 pretty kind of interesting kind of enterprise use cases because 604 00:31:32,079 --> 00:31:36,279 many of the enterprise customers, they are already thinking about 605 00:31:36,359 --> 00:31:40,440 kind of building a data lake, right using these open 606 00:31:40,799 --> 00:31:43,039 format like Iceberg, because they also don't want to be 607 00:31:43,160 --> 00:31:48,079 locked locked in by all those vendors. Right, so when 608 00:31:48,119 --> 00:31:49,640 we come in, you know, we are also kind of 609 00:31:49,720 --> 00:31:53,440 yet another data vendor, right, and they will be like, hey, 610 00:31:53,759 --> 00:31:56,799 could you also expose you know, all those opposerability data 611 00:31:56,839 --> 00:32:00,440 I have ingested into you through Iceberg, so you know 612 00:32:00,480 --> 00:32:03,039 we can they can actually own those data and they 613 00:32:03,039 --> 00:32:06,960 can you know, do more things with those data. And 614 00:32:06,960 --> 00:32:09,240 that's also kind of like an opportunity for us as well, 615 00:32:09,319 --> 00:32:11,440 because you know, if we build our stack on top 616 00:32:11,480 --> 00:32:14,440 of Iceberg and exposing them to the customer also becomes 617 00:32:14,480 --> 00:32:15,160 a lot easier. 618 00:32:15,240 --> 00:32:18,160 Speaker 1: So wait, like ingress into S three is like free, right, 619 00:32:18,400 --> 00:32:21,000 But then if they're actually exporting the data for your platform, 620 00:32:21,039 --> 00:32:23,920 then that's got to especially like a large sum, that's 621 00:32:23,920 --> 00:32:25,839 got to build some costs, right, and. 622 00:32:25,680 --> 00:32:28,279 Speaker 3: Yeah, so the point is they don't They don't have to, right, 623 00:32:28,400 --> 00:32:33,640 So they can basically choose to give us their th 624 00:32:33,640 --> 00:32:38,200 three bucket, right, so we basically directly ingesting data into 625 00:32:38,279 --> 00:32:41,200 their three bucket in the Iceberg in the Iceberg format. 626 00:32:41,559 --> 00:32:44,000 So in that way, they truly own the data, right. 627 00:32:44,079 --> 00:32:46,640 So the data, all the data is in their bucket. 628 00:32:46,680 --> 00:32:49,599 They have access to it, but we are basically, you know, 629 00:32:49,640 --> 00:32:52,279 one of the applications that sit on top and operate 630 00:32:52,319 --> 00:32:52,960 on those data. 631 00:32:53,000 --> 00:32:56,519 Speaker 1: How do you go about securing access to a whole 632 00:32:56,519 --> 00:32:58,039 bunch of customers S three bucket? 633 00:32:58,160 --> 00:33:01,359 Speaker 3: Honestly, we haven't completely thought up that story yet. I 634 00:33:01,359 --> 00:33:03,839 think everybody is kind of trying to figure out how 635 00:33:03,880 --> 00:33:06,920 to do like essentially storage integration. Yeah, so I think 636 00:33:07,000 --> 00:33:10,759 right now we just use standard like ables. 637 00:33:10,279 --> 00:33:12,839 Speaker 2: Policies and you mean like I am or like just. 638 00:33:12,799 --> 00:33:15,200 Speaker 3: Bucket, Yeah, I am. And then there's a specific way 639 00:33:15,240 --> 00:33:18,039 of doing this kind of cross. 640 00:33:17,599 --> 00:33:21,640 Speaker 1: Yeah, the trust the trust policies between the trust policy. Yeah, 641 00:33:21,759 --> 00:33:24,039 I mean it's always an interesting question for me because like, 642 00:33:24,240 --> 00:33:26,200 if you're just doing this thing sort of one off 643 00:33:26,319 --> 00:33:28,680 or I think the canonicals, like you're in some sort 644 00:33:28,680 --> 00:33:31,599 of ABS management account and you're logging into a whole 645 00:33:31,599 --> 00:33:34,839 bunch of organizational accounts, it's like straightforward. But as soon 646 00:33:34,839 --> 00:33:36,960 as you're in this weird position where you need to 647 00:33:36,960 --> 00:33:40,960 start accessing customer accounts, the concept of how do I 648 00:33:40,960 --> 00:33:45,200 actually secure this process starts getting more and more complicated, 649 00:33:45,400 --> 00:33:47,519 and like how do we actually store the data for 650 00:33:47,799 --> 00:33:50,759 doing the access not even the actual data and making 651 00:33:50,799 --> 00:33:53,359 sure like customers can set that up correctly because there 652 00:33:53,440 --> 00:33:55,720 is not like a straight like I still want this, 653 00:33:55,839 --> 00:33:57,759 And if anyone from ABS is listening to this, like 654 00:33:57,799 --> 00:34:02,240 where is AWS oh off access to a WS accounts, 655 00:34:02,359 --> 00:34:05,400 because you know that's that's really what I want here, Uh, 656 00:34:05,720 --> 00:34:07,559 that would be like solve a lot of problems with 657 00:34:07,640 --> 00:34:10,159 getting the trust policy right getting the customers to actually 658 00:34:10,239 --> 00:34:13,559 enter it correctly. It's interesting because we actually do have 659 00:34:13,639 --> 00:34:16,559 a sort of uh for for authors, we have this 660 00:34:16,639 --> 00:34:21,039 oth hack that we have an application in the serveralist 661 00:34:21,039 --> 00:34:26,199 application repository in AWS, so are a WUTH. Instead of 662 00:34:26,199 --> 00:34:28,800 them logging in, is they go and deploy this application 663 00:34:28,840 --> 00:34:31,599 of their account and it like automatically configures everything and 664 00:34:31,840 --> 00:34:33,920 the first time is sort of this weird edge case 665 00:34:33,920 --> 00:34:36,159 where we don't know which account it is. They haven't 666 00:34:36,199 --> 00:34:38,239 maybe not logged in, and so it doesn't necessarily work 667 00:34:38,280 --> 00:34:40,480 out of the box, which would be really nice if 668 00:34:40,480 --> 00:34:43,440 they streamline. But yeah, the cross account access with the 669 00:34:43,480 --> 00:34:46,760 external trust policies, like, it's not the most customer friendly thing. 670 00:34:47,480 --> 00:34:51,000 Speaker 3: Yeah, my being incentive is to solve this problem because 671 00:34:51,119 --> 00:34:53,679 you know, they are they are the ultimate platform, right 672 00:34:53,719 --> 00:34:55,199 that want. 673 00:34:57,079 --> 00:34:59,760 Speaker 2: Like, I have a lot of thoughts here, you know. 674 00:35:00,079 --> 00:35:01,840 Speaker 1: The one that I keep coming back to actually is 675 00:35:01,880 --> 00:35:04,760 I wonder if they're actually decent incentivized to do this, 676 00:35:04,960 --> 00:35:08,760 because it opens a new target for attack, for malicious 677 00:35:08,760 --> 00:35:12,719 attackers to come in and fabricate applications, like you know, 678 00:35:12,760 --> 00:35:14,400 take your company for instance. You know what if someone 679 00:35:14,400 --> 00:35:17,239 threw up a fake observe page and said, oh yeah, 680 00:35:17,280 --> 00:35:20,039 you need to just send someone an email, say hey, 681 00:35:20,719 --> 00:35:23,599 your data whatever is going to expire, your connection is 682 00:35:23,599 --> 00:35:25,039 going to expire. You need to go click on this 683 00:35:25,079 --> 00:35:28,039 link and click approve in AWS and then they go 684 00:35:28,119 --> 00:35:31,280 they get redirected to AWS. There's a screen there that says, hey, 685 00:35:31,639 --> 00:35:33,079 Observe wants to access your data. 686 00:35:33,079 --> 00:35:34,039 Speaker 2: Do you click approve or not? 687 00:35:34,480 --> 00:35:36,639 Speaker 1: And you click approve, you don't look at the permissions there, 688 00:35:36,639 --> 00:35:38,920 and the permissions say like full admin access to the 689 00:35:39,039 --> 00:35:42,559 entire ABS account and it's just good to go. And 690 00:35:42,920 --> 00:35:45,039 because people aren't reviewing what shows up in those screens. 691 00:35:45,039 --> 00:35:47,159 And that's actually the fact that I threw out at 692 00:35:47,159 --> 00:35:49,519 the beginning of the episode is like literally the same problem. 693 00:35:49,840 --> 00:35:52,480 It's someone's making a fishing page and people can be 694 00:35:52,519 --> 00:35:57,360 fishing this way. So I don't know there's a solution there, honestly, 695 00:35:57,480 --> 00:36:00,039 and that's why I can sort of understand not to 696 00:36:00,079 --> 00:36:03,119 do this. However, you know, my retort is going to 697 00:36:03,119 --> 00:36:06,360 be there's already CloudFormation templates, and there's already the application 698 00:36:06,440 --> 00:36:09,679 of the service application repository, which means there's already ways 699 00:36:09,719 --> 00:36:12,800 to fish people into giving giving out full access to 700 00:36:12,840 --> 00:36:15,480 their account by clicking a bunch of buttons without out 701 00:36:15,480 --> 00:36:18,239 actually needing to enter their user or a password or 702 00:36:19,280 --> 00:36:21,800 I mean, god forbid, they're actually having a password for 703 00:36:21,800 --> 00:36:24,679 AWS in the first place, using their passkey to log 704 00:36:24,760 --> 00:36:27,679 in and you know, authentic getting that way and then clicking, 705 00:36:27,880 --> 00:36:30,559 you know, deploy on the CloudFormation template. It's still a problem, 706 00:36:30,840 --> 00:36:33,840 so I'm not buying it. Yeah, I agree, a WS 707 00:36:33,920 --> 00:36:36,159 platform to rule them all. You would think a lot 708 00:36:36,159 --> 00:36:39,000 of support would be something that it has, and it 709 00:36:39,079 --> 00:36:39,639 just doesn't. 710 00:36:39,880 --> 00:36:42,440 Speaker 3: Yeah, I think. I think they also can count astand it. 711 00:36:42,599 --> 00:36:45,400 But I think they are being like with like Iceberg 712 00:36:45,480 --> 00:36:48,000 and stuff, right, I think they're going to become the 713 00:36:48,039 --> 00:36:52,000 de factol data platform for everyone at some point, right, 714 00:36:52,159 --> 00:36:55,519 Like you if if everybody is putting their data on 715 00:36:56,480 --> 00:36:59,639 F three directly through the Iceberg format, then you will 716 00:36:59,719 --> 00:37:02,000 have all those use cases of like sharing it around, 717 00:37:02,239 --> 00:37:04,559 like sharing it with their application and all the kind 718 00:37:04,599 --> 00:37:09,559 of things. Right, So hopefully they can get their stuff 719 00:37:09,599 --> 00:37:11,440 together and actually do something. 720 00:37:11,519 --> 00:37:12,320 Speaker 2: They just came out with. 721 00:37:12,639 --> 00:37:13,800 Speaker 1: I think it was the last year or so, they 722 00:37:13,800 --> 00:37:16,960 came up with S three tables, which I it's them making. 723 00:37:17,199 --> 00:37:19,360 It's nice to see that there's still some improvements being 724 00:37:19,440 --> 00:37:21,800 made S three over time, because it's not the best 725 00:37:21,880 --> 00:37:25,960 user experience. It's not even the best developer experience, so 726 00:37:26,239 --> 00:37:28,440 you know, there are things that things left to be 727 00:37:28,480 --> 00:37:32,119 desired there. I mean, my my personal pick if I 728 00:37:32,119 --> 00:37:34,639 had to say anyone is just like all those bad 729 00:37:34,679 --> 00:37:37,760 security features like just default remove them from the console, 730 00:37:37,920 --> 00:37:40,360 Like no one should know about being able to make 731 00:37:40,360 --> 00:37:43,320 a bucket public, Like there's never a reason in today's 732 00:37:43,599 --> 00:37:45,920 story to ever have a private a public bucket and 733 00:37:46,280 --> 00:37:48,599 you know, on that same token, like I want, I 734 00:37:48,679 --> 00:37:50,679 want to be able to name my bucket whatever I 735 00:37:50,719 --> 00:37:54,000 want and not like have to worry about conflicts with 736 00:37:54,119 --> 00:37:56,039 other conflict things. 737 00:37:56,159 --> 00:37:57,559 Speaker 3: So BUSI and I'm just. 738 00:37:57,480 --> 00:37:59,119 Speaker 1: Like, I don't know what it's going to take here, 739 00:37:59,199 --> 00:38:03,079 but no AWS, just come up with like another replacement service, 740 00:38:03,079 --> 00:38:07,320 call it you know as far as or and just 741 00:38:07,559 --> 00:38:10,480 and just you know, honestly blob private, blob storage. 742 00:38:10,599 --> 00:38:11,719 Speaker 2: That's really all I need. 743 00:38:11,760 --> 00:38:13,800 Speaker 1: And make it hook up a ball too, like literally 744 00:38:13,800 --> 00:38:15,760 the exact same API. The only thing that has to 745 00:38:15,800 --> 00:38:18,159 be different is just cut out all those things that 746 00:38:18,360 --> 00:38:21,880 are these tons of foot guns, right like especially bucket 747 00:38:21,960 --> 00:38:24,360 names quating is a huge one. Anyway, you can see 748 00:38:24,360 --> 00:38:27,159 that I have a whole rant on this, so maybe 749 00:38:27,159 --> 00:38:27,840 I'll stop there. 750 00:38:28,079 --> 00:38:31,360 Speaker 3: You know, we we obviously found out Snowflake is pretty 751 00:38:31,400 --> 00:38:34,039 good at some things, but also pretty pretty bad at 752 00:38:34,239 --> 00:38:34,880 the other things. 753 00:38:35,679 --> 00:38:37,360 Speaker 2: What's that like number one bad thing? 754 00:38:38,400 --> 00:38:44,119 Speaker 3: So you would have So we have this like constant 755 00:38:44,159 --> 00:38:48,320 battle with with Snowflake in terms of how like how 756 00:38:48,400 --> 00:38:52,599 how they they're optimizer optimizes the query versus our kind 757 00:38:52,639 --> 00:38:56,320 of optimizer optimizes the querry. Like many cases, you know, 758 00:38:56,360 --> 00:38:58,760 if you do it like a joint, right, like you know, 759 00:38:58,800 --> 00:39:01,280 the ordering of the joint matters a lot, you know, 760 00:39:01,360 --> 00:39:04,039 which started to put at the built side, which started 761 00:39:04,079 --> 00:39:06,639 put it as the probe side. And in many cases 762 00:39:06,639 --> 00:39:09,199 it's like, oh, like Snowflake, could you just like let 763 00:39:09,280 --> 00:39:12,199 us decide what what is the joint orders because we 764 00:39:12,360 --> 00:39:16,320 know more about the data then, you know, right, But 765 00:39:16,360 --> 00:39:18,360 then you thought the snow FAI would join it other way, 766 00:39:18,400 --> 00:39:20,480 but then in some fair would actually swap it and 767 00:39:20,599 --> 00:39:25,079 you know, ruin your your your entire efficiency. So we 768 00:39:25,119 --> 00:39:27,679 would always wish like Snowflake would just provide us with 769 00:39:27,760 --> 00:39:30,239 an API for us to just you know, send them 770 00:39:30,239 --> 00:39:33,519 the curve the curve plan, just don't change it, just 771 00:39:33,679 --> 00:39:37,199 run it as is. But obviously that's not something that 772 00:39:37,599 --> 00:39:38,360 they're willing to do. 773 00:39:38,519 --> 00:39:41,400 Speaker 1: That's really surprising actually because I feel like even as 774 00:39:41,440 --> 00:39:43,440 far back as I remember, which isn't that that far 775 00:39:44,320 --> 00:39:46,480 with MS sequel. There was always like you could pass 776 00:39:46,519 --> 00:39:47,840 it hints to the engine. 777 00:39:47,960 --> 00:39:51,239 Speaker 3: Yeah, so now they're getting better at that. So recently, 778 00:39:51,360 --> 00:39:55,159 very recently they released a new feature. I think I 779 00:39:55,199 --> 00:39:57,639 would like to think that we we push for it. 780 00:39:57,719 --> 00:40:00,880 But they recently released the future cut erected to joint, 781 00:40:00,960 --> 00:40:04,079 which is kind of like similar to you know, joint 782 00:40:04,119 --> 00:40:06,679 hints that you can you can you can just say, hey, 783 00:40:06,800 --> 00:40:08,679 use this as a left to use either the right, 784 00:40:08,800 --> 00:40:12,079 don't change it. But we have waited for so long 785 00:40:12,239 --> 00:40:13,960 for for for such a future. 786 00:40:14,280 --> 00:40:16,679 Speaker 2: Well, I guess you're you're you're two percent usage. 787 00:40:16,360 --> 00:40:19,800 Speaker 1: Only count for something. Yeah, well then I guess I 788 00:40:19,840 --> 00:40:22,079 think this opportunity then to uh to move over to picks. 789 00:40:22,280 --> 00:40:23,719 Speaker 2: So what did you bring for us today? 790 00:40:24,039 --> 00:40:28,280 Speaker 3: Yeah? So I bought this gadget for my kind of 791 00:40:28,360 --> 00:40:29,639 summer traveling. 792 00:40:30,440 --> 00:40:32,920 Speaker 4: It's a it's just like three D. I can actually 793 00:40:32,960 --> 00:40:37,039 show it show you it's yeah, so it's good. It's 794 00:40:37,079 --> 00:40:41,119 just kind of like a R kind of glasses, you know, 795 00:40:41,239 --> 00:40:46,519 not the fancy ones like the Division pro or whatnot. Basically, 796 00:40:46,519 --> 00:40:48,039 what he does is it has like. 797 00:40:48,159 --> 00:40:53,280 Speaker 3: S GM I you know, connector to whatever devices you have, 798 00:40:53,840 --> 00:40:56,639 and it has this like tiny kind of old projector 799 00:40:56,639 --> 00:40:58,519 at the top of the glass and just projecting it's 800 00:40:58,519 --> 00:41:01,800 into a lens so you can kind of start. It 801 00:41:01,840 --> 00:41:05,239 worked pretty surprisingly well, so I can. It does like 802 00:41:05,320 --> 00:41:08,320 hat tracking. I use it for work during the traveling 803 00:41:08,360 --> 00:41:10,440 because I, you know, obviously kind of bring a large 804 00:41:10,480 --> 00:41:13,719 monitor with me and plugging that I can actually get 805 00:41:13,800 --> 00:41:17,320 like a you know, thirty inch monitors like floating in 806 00:41:17,360 --> 00:41:18,159 front of my eyes. 807 00:41:18,320 --> 00:41:20,840 Speaker 2: So you have to plug it in though it have 808 00:41:20,920 --> 00:41:21,960 to be physically connected. 809 00:41:22,519 --> 00:41:25,000 Speaker 3: Yes, that's one job back, but I would say that 810 00:41:25,079 --> 00:41:28,519 because it doesn't have any battery inside. It's powered purely 811 00:41:28,559 --> 00:41:33,519 through the STMI. Yeah, so I don't have to charge it. 812 00:41:33,880 --> 00:41:37,920 So actually the experience is better than like having something 813 00:41:37,920 --> 00:41:39,599 I have to bring another charger in. 814 00:41:40,000 --> 00:41:43,840 Speaker 2: Do But do laptops support h power over Hdmi? 815 00:41:44,360 --> 00:41:47,559 Speaker 3: Yes? Surprising interest I was, I will even work with 816 00:41:47,559 --> 00:41:53,440 my phone, like it drains the phone better really, yes, 817 00:41:53,480 --> 00:41:55,559 but at least it works. 818 00:41:56,199 --> 00:41:59,239 Speaker 2: Wow, that's amazing. So like what is the company? 819 00:41:59,280 --> 00:42:03,280 Speaker 3: What is that called a x rel x real I 820 00:42:03,280 --> 00:42:05,559 think the one that god is called x real pro 821 00:42:05,920 --> 00:42:07,480 one pro or something like that. 822 00:42:07,880 --> 00:42:10,679 Speaker 2: Okay, so I should just like get rid of my monitors. 823 00:42:10,880 --> 00:42:13,079 Speaker 3: How much is it like a five hundred and six 824 00:42:13,119 --> 00:42:15,280 hundred bucks after that. Yeah, way cheaper than like a 825 00:42:15,360 --> 00:42:15,760 vision pro. 826 00:42:15,960 --> 00:42:17,719 Speaker 1: Yeah, I say, so I should just get rid of 827 00:42:17,800 --> 00:42:20,239 my monitors, switch it over over to this, and like 828 00:42:20,280 --> 00:42:22,840 walk around with like an HDMI. K. 829 00:42:24,199 --> 00:42:26,440 Speaker 3: It's not the most kind of like you know, socially 830 00:42:26,440 --> 00:42:29,480 acceptable attire out there, but you know. 831 00:42:29,519 --> 00:42:32,159 Speaker 1: The interesting thing is, like I'm surprised that they haven't 832 00:42:32,599 --> 00:42:36,719 they didn't have like a battery pack honestly connected with 833 00:42:36,840 --> 00:42:38,920 like rechargeable batteries that you could like have in your 834 00:42:38,960 --> 00:42:41,960 pocket and then have a remote HDMI because I feel 835 00:42:42,000 --> 00:42:45,199 like that would allow you to walk around with it, 836 00:42:45,239 --> 00:42:47,960 but having to take the glasses off in order to 837 00:42:48,000 --> 00:42:49,440 perform other actions, I don't know. 838 00:42:49,719 --> 00:42:53,280 Speaker 3: Yeah, I think with this, I kind of see, you know, 839 00:42:53,440 --> 00:42:56,880 there's another way of doing kind of AR that's not 840 00:42:57,119 --> 00:43:00,440 the app Apple way, and yeah it also works. Yeah. 841 00:43:00,480 --> 00:43:02,440 Speaker 1: So this is I'm going I'm going to speak at 842 00:43:02,480 --> 00:43:05,239 a conference. Instead of bringing my laptop, I'm bringing my 843 00:43:05,320 --> 00:43:10,599 phone and my my AR my AR goggles. Okay, So 844 00:43:11,440 --> 00:43:14,199 for me, what I brought is, uh, there's a movie 845 00:43:14,199 --> 00:43:15,920 that I just really like and every once in a 846 00:43:15,960 --> 00:43:17,920 while I'll rewatch it, and so recently I had to 847 00:43:17,920 --> 00:43:21,119 rewatch of The Shadow. It's the nineteen ninety four with 848 00:43:21,199 --> 00:43:24,719 Alec Baldwin and it was originally made after the radio 849 00:43:24,800 --> 00:43:28,320 show from the from the thirties, and I don't know, 850 00:43:28,400 --> 00:43:30,599 there's just like so many great quotable things in it. 851 00:43:30,639 --> 00:43:33,440 And actually it's if you haven't seen it, it's have 852 00:43:33,519 --> 00:43:37,920 you seen it. It's the original inspiration for Batman. So 853 00:43:38,840 --> 00:43:41,199 it's like Batman wasn't didn't come from nowhere. It was 854 00:43:41,239 --> 00:43:44,280 actually The Shadow with I mean, not with Alec Baldwin, 855 00:43:44,320 --> 00:43:47,639 but that was much later. But it's it's great. It's like, 856 00:43:47,840 --> 00:43:51,880 just imagine Batman with guns. And also in the movie 857 00:43:51,920 --> 00:43:57,679 he has he learns like telepathy and telekinesis, so I mean, 858 00:43:57,719 --> 00:43:59,800 it's just honestly, it's like a much better Batman. It's 859 00:43:59,800 --> 00:44:01,400 like everything Batman wishes he could be. 860 00:44:02,159 --> 00:44:05,199 Speaker 3: Yeah, like I never understand why Batman's not allowed it 861 00:44:05,360 --> 00:44:05,840 just guns. 862 00:44:05,920 --> 00:44:10,079 Speaker 2: But yes, I mean, you know, it's interesting. 863 00:44:10,119 --> 00:44:13,840 Speaker 1: There was actually a really long documentary on the making 864 00:44:13,880 --> 00:44:16,000 of Batman, the animated series that came out in like 865 00:44:16,039 --> 00:44:19,920 the nineties and how it was like so dark and 866 00:44:20,000 --> 00:44:22,280 at the time it was like the only dark animated 867 00:44:22,320 --> 00:44:27,000 television show. Everything else that's even remotely animated is like 868 00:44:27,320 --> 00:44:31,280 sunshine and flowers, like getting everyone to sign off on 869 00:44:31,440 --> 00:44:34,920 like how like almost deeply depressing some of these episodes are, 870 00:44:35,039 --> 00:44:37,440 or like the dark drama that is now just everywhere 871 00:44:37,480 --> 00:44:41,920 on television is like quite unexpected. So I can see, 872 00:44:41,960 --> 00:44:45,280 like even the upgraded guns is like it's just something else. 873 00:44:45,360 --> 00:44:47,880 I mean, there's a lot of the there's a lot 874 00:44:47,880 --> 00:44:50,960 of history about like the gun usage or even just 875 00:44:51,119 --> 00:44:51,679 in cannon. 876 00:44:51,760 --> 00:44:53,760 Speaker 2: I don't know what the non cannon reasons are. 877 00:44:53,639 --> 00:44:55,760 Speaker 1: But I suppose if you wanted guns, you know, just 878 00:44:55,800 --> 00:44:57,519 go watch The Shadows. 879 00:44:57,840 --> 00:44:58,519 Speaker 2: It's fantastic. 880 00:44:58,559 --> 00:45:01,880 Speaker 1: I mean it's like, ay bad movie, but I mean 881 00:45:01,880 --> 00:45:02,840 I actually really like it. 882 00:45:03,480 --> 00:45:05,559 Speaker 3: I will, I will. Yeah, thanks for the thanks for 883 00:45:05,639 --> 00:45:07,800 the situation. Yeah, okay, well. 884 00:45:07,719 --> 00:45:09,079 Speaker 2: Then that's the end of our episode. 885 00:45:09,559 --> 00:45:11,639 Speaker 1: Thank you AG for for coming and talking about Deserve 886 00:45:11,719 --> 00:45:15,079 and how to build an observability platform has gotten quite 887 00:45:15,119 --> 00:45:18,039 far and using such a high amount of compute and 888 00:45:18,599 --> 00:45:19,719 Snowflake as a database. 889 00:45:19,800 --> 00:45:21,400 Speaker 2: But that's been really interesting and I 890 00:45:21,440 --> 00:45:26,719 Speaker 1: Hope we'll be back on next week with another episode.