WEBVTT 1 00:00:00.160 --> 00:00:03.879 Welcome to the deep dive. Today. We're really digging into 2 00:00:03.919 --> 00:00:10.599 something fascinating the art, maybe the science of AI product development. Right, 3 00:00:10.880 --> 00:00:13.199 We've looked at some great material from an expert in 4 00:00:13.199 --> 00:00:16.519 the field, focusing on how you actually build AI that 5 00:00:16.559 --> 00:00:21.239 delivers real business value. And our mission for you listening 6 00:00:21.359 --> 00:00:23.480 is really to cuf through that complexity, give you the 7 00:00:23.480 --> 00:00:28.600 tools to approach AI projects with let's say, ambition and clarity. 8 00:00:29.320 --> 00:00:31.800 Yeah, and what's so interesting is that AI, Well, it 9 00:00:31.800 --> 00:00:33.920 feels a bit like science fiction sometimes, doesn't it. 10 00:00:33.920 --> 00:00:34.520 It really does. 11 00:00:34.640 --> 00:00:37.920 But the actual building blocks, they're becoming more and more accessible, 12 00:00:38.079 --> 00:00:40.679 almost plug and play in some cases that there's a catch. 13 00:00:40.840 --> 00:00:44.079 There's always a catch despite the promise, there's still a 14 00:00:44.119 --> 00:00:45.880 lot of complexity, a lot of uncertainty. 15 00:00:46.119 --> 00:00:48.200 So what we want to do today is show you 16 00:00:48.479 --> 00:00:51.640 how to maybe sidestep the common pitfalls. 17 00:00:51.200 --> 00:00:53.679 Exactly learn on the job, but in a structured way. 18 00:00:53.759 --> 00:00:56.679 We're drawing directly from the experiences laid out in the 19 00:00:56.679 --> 00:00:57.840 material we reviewed. 20 00:00:58.039 --> 00:01:01.399 Okay, so let's untack this first big question, why even 21 00:01:01.479 --> 00:01:03.200 bother with an AI development project? 22 00:01:03.280 --> 00:01:04.400 Right? The fundamental why. 23 00:01:04.760 --> 00:01:07.280 The sources we looked at make a pretty bold claim here. 24 00:01:07.719 --> 00:01:12.040 They say if your business offers digital products or services, well, 25 00:01:12.319 --> 00:01:15.200 AI isn't just an option. It can enhance or even 26 00:01:15.519 --> 00:01:17.079 totally transform what you do. 27 00:01:17.359 --> 00:01:21.079 It's a big statement, but think about the scope refuining marketing, 28 00:01:21.599 --> 00:01:23.400 automating customer support. 29 00:01:23.200 --> 00:01:27.959 Adding smart search features, building completely new disruptive products. 30 00:01:27.519 --> 00:01:29.560 Maybe exactly. The potential is huge. 31 00:01:29.719 --> 00:01:32.560 But and this is a big butt. It feels like 32 00:01:32.599 --> 00:01:35.519 we hear about AI projects going sideways pretty often. Why 33 00:01:35.599 --> 00:01:36.319 is that? 34 00:01:36.319 --> 00:01:38.599 That's the crucial follow up question, isn't it? And the 35 00:01:38.680 --> 00:01:42.719 sources definitely highlight some critical mistakes, like what Well, a 36 00:01:42.799 --> 00:01:45.799 really common one is simply using AI for the sake 37 00:01:45.840 --> 00:01:49.120 of AI, you know, building a cool tech solution that 38 00:01:49.200 --> 00:01:50.959 doesn't actually solve a real problem. 39 00:01:51.040 --> 00:01:52.560 Right the solution looking for a. 40 00:01:52.840 --> 00:01:57.120 Precisely the material gives this example of financial analytics dashboard 41 00:01:57.200 --> 00:01:59.920 already crowded with info. Okay, and the team decides haylo 42 00:02:00.120 --> 00:02:04.040 out of chatbot. Why is because you know, jenai hype 43 00:02:04.120 --> 00:02:04.680 is everywhere? 44 00:02:05.040 --> 00:02:07.439 Ah okay? And how did that turn out? Because this 45 00:02:07.480 --> 00:02:08.120 is where it gets. 46 00:02:08.039 --> 00:02:10.759 Interesting, right, it gets very interesting. The sources point out 47 00:02:10.759 --> 00:02:16.199 that this team ended up creating more errors than actual insights. Ouch. Yeah, 48 00:02:16.360 --> 00:02:19.520 the chatbot just couldn't handle what users threw at it. 49 00:02:19.960 --> 00:02:23.560 Requests were too complex, sometimes way off topic. People even 50 00:02:23.599 --> 00:02:26.120 tried asking for investment advice it wasn't supposed to. 51 00:02:26.039 --> 00:02:28.240 Give, so that really highlights some core problems. 52 00:02:28.280 --> 00:02:32.120 Then, Absolutely, things like misaligned data the user needs just 53 00:02:32.159 --> 00:02:34.479 weren't connected to the data the model was trained on. 54 00:02:34.719 --> 00:02:36.479 Makes sense, and a lack. 55 00:02:36.319 --> 00:02:39.120 Of guidance in the UI. You know, a simple chat 56 00:02:39.159 --> 00:02:42.599 box invites basically anything, but the model couldn't cope. 57 00:02:42.439 --> 00:02:44.560 And user expectations probably didn't help. 58 00:02:44.800 --> 00:02:49.080 No, definitely not overblown user expectations fueled by you know, 59 00:02:49.120 --> 00:02:52.439 maybe flashy marketing images that promised more than the AI 60 00:02:52.520 --> 00:02:53.000 could deliver. 61 00:02:53.280 --> 00:02:56.759 So connecting this back, the big takeaway is you absolutely 62 00:02:56.879 --> 00:02:59.439 need a clear opportunity, a real problem to. 63 00:02:59.400 --> 00:03:02.400 Solve you got it, have to justifies the investment. The 64 00:03:02.479 --> 00:03:06.919 sources propose this mental model for AI systems three core parts, 65 00:03:06.960 --> 00:03:10.680 which are data intelligence that includes the AI models themselves, 66 00:03:11.000 --> 00:03:12.080 and the user experience. 67 00:03:12.199 --> 00:03:14.439 Okay, data intelligence UX. 68 00:03:14.439 --> 00:03:17.360 And all of that is constrained by and this is 69 00:03:17.400 --> 00:03:22.560 crucial AI governance requirements. Can't forget those guardrails. 70 00:03:22.039 --> 00:03:25.680 Right, governance is key. So using that model, how do 71 00:03:25.759 --> 00:03:29.960 we actually find those good opportunities those high impact areas. 72 00:03:30.199 --> 00:03:34.560 It's really about spotting where AI can significantly improve things 73 00:03:34.599 --> 00:03:37.240 for the user. Let's take a music streaming app. Okay, 74 00:03:37.439 --> 00:03:39.719 users often get stuck in a rut, right, yeah, struggle 75 00:03:39.759 --> 00:03:40.639 to find new music. 76 00:03:40.719 --> 00:03:42.080 Yeah, it happens to me all the time. 77 00:03:42.240 --> 00:03:45.080 So AI powered recommendations there that could be a total 78 00:03:45.120 --> 00:03:47.560 game changer. Boost engagement, help discovery. 79 00:03:47.879 --> 00:03:50.520 Makes perfect sense. And the sources mentioned several types of 80 00:03:50.599 --> 00:03:54.120 benefits AI can bring beyond just you know, shiny new features. 81 00:03:54.199 --> 00:03:57.560 Oh yeah, definitely. It's not just about innovation. First up is. 82 00:03:57.560 --> 00:03:59.800 Automation, okay, reducing manual work. 83 00:04:00.240 --> 00:04:04.199 Cutting the cost of manual processors. Yeah, But the sources 84 00:04:04.240 --> 00:04:07.560 emphasize this AI cost equation. It's not just building and 85 00:04:07.639 --> 00:04:08.240 running the AI. 86 00:04:08.400 --> 00:04:09.280 What else is in there? 87 00:04:09.360 --> 00:04:11.639 You have to factor in the cost often overlooked of 88 00:04:11.680 --> 00:04:14.960 finding and fixing the mistakes the AI makesh okay, and 89 00:04:15.039 --> 00:04:18.480 the risk of mistakes you don't catch that could be significant. 90 00:04:18.720 --> 00:04:22.000 That's a really important point about hidden costs. What's next 91 00:04:22.079 --> 00:04:23.399 after automation. 92 00:04:23.439 --> 00:04:27.639 Then there's improvement and augmentation. This is where AI supports 93 00:04:27.720 --> 00:04:30.000 human creativity, doesn't replace it. 94 00:04:30.079 --> 00:04:33.560 Interesting like a copilot kind of Yeah, Yeah, The example 95 00:04:33.600 --> 00:04:35.680 given is Miro, the whiteboarding software. 96 00:04:35.920 --> 00:04:41.040 They use AI for brainstorming, diagramming, summarizing ideas. It plays 97 00:04:41.079 --> 00:04:45.639 to both human strengths and AI strengths. Nice mortalization, adapting 98 00:04:45.680 --> 00:04:48.879 the product to what each user needs, like that music 99 00:04:48.920 --> 00:04:50.959 app learning your taste over. 100 00:04:50.759 --> 00:04:53.519 Time, tailoring playlists to my mood. I like that. 101 00:04:53.759 --> 00:04:58.319 Exactly, But you need to be realistic. Good personalization it needs. 102 00:04:58.079 --> 00:05:00.639 A lot of data, and bad personal is you can. 103 00:05:00.560 --> 00:05:02.959 Really push users away. Privacy concerns too. 104 00:05:02.800 --> 00:05:04.759 Obviously true, Okay, any others. 105 00:05:05.240 --> 00:05:09.480 One more convenience, just making things easier, reducing friction. Think 106 00:05:09.519 --> 00:05:12.079 of an AI search that anticipates what you're looking for. 107 00:05:11.959 --> 00:05:14.000 Get see the answer faster. Yeah, I can see the 108 00:05:14.079 --> 00:05:14.480 value there. 109 00:05:14.519 --> 00:05:15.000 Absolutely. 110 00:05:15.079 --> 00:05:17.240 So. The sources also talk about two different ways to 111 00:05:17.279 --> 00:05:20.160 actually launch these AI products, two strategic approaches. 112 00:05:20.360 --> 00:05:23.560 That's right. It really depends on the situation. The stakes. 113 00:05:24.240 --> 00:05:29.399 First is the careful approach, think ready, aim. 114 00:05:29.519 --> 00:05:32.959 Fire, so lots of planning upfront. 115 00:05:32.879 --> 00:05:36.079 Loads of it SORO research validation before you commit to 116 00:05:36.120 --> 00:05:38.920 significant development. This is for when the cost of failure 117 00:05:39.000 --> 00:05:42.879 is really high, like healthcare. Perfect example, health tech company 118 00:05:42.920 --> 00:05:48.639 building a diagnostic feature, you'd need to rigorously check, user impact, feasibility, 119 00:05:48.759 --> 00:05:53.759 business value regulations, everything you can't afford to get it wrong. 120 00:05:53.879 --> 00:05:54.360 Makes sense. 121 00:05:54.639 --> 00:05:58.240 And the other approach, that's the fast approach, ready fire aim. 122 00:05:58.560 --> 00:06:00.519 So build something quickly and see what happens. 123 00:06:00.560 --> 00:06:03.279 Pretty much, get a prototype out there fast, get real 124 00:06:03.319 --> 00:06:05.319 world feedback almost immediately, like. 125 00:06:05.279 --> 00:06:07.560 The music recommendation feature we talked about. 126 00:06:07.399 --> 00:06:10.560 Exactly like that. Yeah, speed is key, Failing fast is 127 00:06:10.600 --> 00:06:12.560 okay if the initial costs are low and maybe the 128 00:06:12.560 --> 00:06:14.759 market's crowded. Yeah, you learn by doing. 129 00:06:14.720 --> 00:06:17.240 And that fits well with how AI models are developed right, 130 00:06:17.319 --> 00:06:18.480 iteratively perfectly. 131 00:06:18.680 --> 00:06:21.360 It lets you reduce that uncertainty step by step based 132 00:06:21.399 --> 00:06:23.040 on how real users interact with it. 133 00:06:23.160 --> 00:06:27.360 Okay, great, so we know YAI, how to find opportunities, 134 00:06:27.399 --> 00:06:30.680 different ways to launch. Let's get into the actual building blocks. 135 00:06:30.680 --> 00:06:33.079 The sources call it the AI solution space. 136 00:06:33.399 --> 00:06:36.959 That's the term they use yet data, intelligence and user experience. 137 00:06:37.399 --> 00:06:40.560 Let's start with data, the fuel for the AI. 138 00:06:40.560 --> 00:06:44.759 Absolutely described as the fuel. It's the raw material for training, 139 00:06:45.040 --> 00:06:48.199 for fine tuning, for evaluating everything. 140 00:06:47.879 --> 00:06:49.279 And it comes in different forms the. 141 00:06:49.279 --> 00:06:55.600 Forms textual data, needing NLP, visual data, auditory data, sensorimotor 142 00:06:55.680 --> 00:06:59.399 data for robotics, self driving cars, even computer code itself 143 00:06:59.680 --> 00:07:01.040 like for get Up Copilot. 144 00:07:01.120 --> 00:07:03.399 Wow. Okay, broad range, very. 145 00:07:03.199 --> 00:07:07.079 Broad but the absolute crucial point, and the sources hammer 146 00:07:07.120 --> 00:07:11.000 this home. Data quality directly drives output. 147 00:07:10.800 --> 00:07:12.279 Quality, garbage and garbage out. 148 00:07:12.480 --> 00:07:15.199 Essentially, that's the classic saying, and it's never been truer 149 00:07:15.240 --> 00:07:18.040 than with AI. The quality of your data dictates the 150 00:07:18.120 --> 00:07:20.240 value you can provide. Period, got it? 151 00:07:20.600 --> 00:07:23.240 And how does the AI actually learn from this data? 152 00:07:23.279 --> 00:07:25.639 The sources mention different types of learning, right. 153 00:07:25.480 --> 00:07:28.319 They distinguish between unsupervised learning and supervised learning. 154 00:07:28.360 --> 00:07:29.319 Okay, what's the difference. 155 00:07:29.519 --> 00:07:33.319 Unsupervised learning is about exploring the data, finding hidden patterns 156 00:07:33.560 --> 00:07:37.920 structures like clustering users into segments based on their behavior 157 00:07:38.279 --> 00:07:39.839 without knowing the segments. 158 00:07:39.519 --> 00:07:43.000 Beforehand, so letting the data speak for itself in a way. 159 00:07:43.160 --> 00:07:46.319 Yes, it helps uncover insights you might not expect. Then 160 00:07:46.319 --> 00:07:49.959 supervised learning is different. You train the model with labeled data. 161 00:07:49.600 --> 00:07:52.720 So you tell it what the right answer is, for example, exactly. 162 00:07:52.800 --> 00:07:55.319 You give it examples with known outcomes, so it learns 163 00:07:55.319 --> 00:08:00.160 to classify new unseen data, think predicting customer churn or 164 00:08:00.160 --> 00:08:02.639 segmenting users based on predefined criteria. 165 00:08:03.079 --> 00:08:07.079 Okay, unsupervised for exploring supervised for predicting or classifying based 166 00:08:07.120 --> 00:08:08.839 on known labels makes sense. 167 00:08:08.800 --> 00:08:11.519 And underlying all this data use, of course, are ethics, 168 00:08:11.920 --> 00:08:15.360 user privacy, data minimization, consent. These are non. 169 00:08:15.199 --> 00:08:18.560 Negotiable absolutely so data is the fuel. What about the 170 00:08:18.600 --> 00:08:20.079 intelligence part of that model? 171 00:08:20.199 --> 00:08:22.959 Right, the AI itself. The sources break this down into 172 00:08:22.959 --> 00:08:26.800 three main types. First, predictive AI. Sometime it's called analytical AI. 173 00:08:26.959 --> 00:08:27.560 What does that do? 174 00:08:27.879 --> 00:08:31.600 It focuses on well defined tasks analyzing data to make 175 00:08:31.600 --> 00:08:33.320 predictions or solve pretty clear. 176 00:08:33.120 --> 00:08:35.840 Problems like sentiment analysis. 177 00:08:35.279 --> 00:08:39.399 Perfect example, taking unstructured text like product reviews and turning 178 00:08:39.480 --> 00:08:42.159 it into a structured output like a numeric sentiment score. 179 00:08:42.279 --> 00:08:44.759 But humans still need to figure out why people feel 180 00:08:44.759 --> 00:08:45.679 that way exactly. 181 00:08:45.720 --> 00:08:48.039 The AI gives you the what, but humans often need 182 00:08:48.080 --> 00:08:50.200 to interpret the why and decide what to do about it. 183 00:08:50.279 --> 00:08:51.720 Okay. Type number two. 184 00:08:52.399 --> 00:08:56.399 Generative AI or GENAI. This is what everyone's. 185 00:08:56.039 --> 00:08:58.000 Talking about, creating new content. 186 00:08:57.720 --> 00:09:03.399 Precisely text, images, video code, even things like chemical structures 187 00:09:03.399 --> 00:09:06.279 for drug discovery. It's reshaping whole industries. 188 00:09:06.399 --> 00:09:07.960 How does it work? Fundamentally? 189 00:09:08.039 --> 00:09:12.240 It essentially combines existing information in new, often unexpected ways. 190 00:09:12.799 --> 00:09:15.840 It's learned patterns from vast data sets and can generate 191 00:09:15.919 --> 00:09:19.360 novel outputs based on prompts. It can even do surprisingly 192 00:09:19.399 --> 00:09:21.360 well on things like you know, standardized tests. 193 00:09:21.639 --> 00:09:23.240 Impressive and the third. 194 00:09:23.039 --> 00:09:26.559 Type AGENTIC AI, This is maybe the next frontier. This 195 00:09:26.600 --> 00:09:28.960 is where AI doesn't just predict or generate. 196 00:09:28.679 --> 00:09:31.720 It acts acts how so it changes. 197 00:09:31.440 --> 00:09:34.240 The state of the world. These agents combine language models 198 00:09:34.240 --> 00:09:38.879 with external tools think APIs, databases, functions. 199 00:09:38.679 --> 00:09:40.279 So they can interact with other systems. 200 00:09:40.360 --> 00:09:44.039 Yes, and they can reason, plan and learn from those interactions. 201 00:09:44.320 --> 00:09:47.039 The example given is like a product management agent doing 202 00:09:47.039 --> 00:09:51.840 what imagine it updating a project roadmap, automatically analyzing transcripts 203 00:09:51.879 --> 00:09:56.279 from sales, calls for insights, even drafting communications to stakeholders, 204 00:09:56.679 --> 00:09:57.720 all autonomously. 205 00:09:57.919 --> 00:10:01.480 Wow. That's that's a significant step up. 206 00:10:01.559 --> 00:10:03.879 It really is. It implies a much higher degree of autonomy. 207 00:10:04.240 --> 00:10:06.240 Okay, so we have data and these three types of 208 00:10:06.320 --> 00:10:12.759 intelligence predictive, generative AGENTIC. What about the third piece, user experience? 209 00:10:13.480 --> 00:10:15.200 How is UX different for AI? 210 00:10:15.399 --> 00:10:18.360 That's a great question, because it is different traditional software. 211 00:10:18.879 --> 00:10:21.320 It's deterministic, you click a button, the same thing happens 212 00:10:21.360 --> 00:10:26.000 every time, right. Predictable AI not so much. It's inherently uncertain. 213 00:10:26.039 --> 00:10:29.039 It can make mistakes, it can hallucinate, make things up. 214 00:10:29.039 --> 00:10:32.240 So the UI has to account for that uncertainty exactly. 215 00:10:32.360 --> 00:10:35.440 It needs to be designed with unpredictability and potential failure 216 00:10:35.440 --> 00:10:37.639 baked in. How do you do that well? The sources 217 00:10:37.639 --> 00:10:41.720 suggests things like showing confidence scores, like next to AI 218 00:10:41.799 --> 00:10:46.639 generated text it might say high confidence or medium confidence. 219 00:10:46.240 --> 00:10:48.840 So the user knows how much to trust it precisely. 220 00:10:49.039 --> 00:10:53.120 It helps calibray trust. Another idea is footprints, ways to 221 00:10:53.200 --> 00:10:56.000 trace the AI's steps. How did he get from the 222 00:10:56.080 --> 00:10:59.960 prompt to this result? That builds transparence. 223 00:11:00.279 --> 00:11:03.120 That transparency seems key. The source has had a good example, 224 00:11:03.159 --> 00:11:05.759 didn't they from a sustainability reporting app. 225 00:11:05.799 --> 00:11:07.039 Oh yeah, that was a great one. When it is 226 00:11:07.120 --> 00:11:10.159 generating a draft report instead of just a generic spinner, Yeah, 227 00:11:10.200 --> 00:11:13.440 it shows a progress window explaining what the AI is doing, 228 00:11:13.799 --> 00:11:17.600 analyzing financial data, checking regulatory guidelines. 229 00:11:17.039 --> 00:11:19.720 Et cetera, keeping the user informed. 230 00:11:19.360 --> 00:11:22.120 Exactly, and then the draft report itself. It uses those 231 00:11:22.120 --> 00:11:26.360 confidence scores color coded green for high confidence, yellow for HM. 232 00:11:26.360 --> 00:11:29.120 Maybe double check this bit that really empowers the user, 233 00:11:29.159 --> 00:11:31.639 doesn't it. It shows them where their expertise is needed. 234 00:11:31.840 --> 00:11:35.240 It absolutely does. It's about collaboration, not just automation. 235 00:11:35.440 --> 00:11:40.039 And there was this idea of rethinking friction. Sometimes making 236 00:11:40.080 --> 00:11:41.759 things slightly harder is good. 237 00:11:42.039 --> 00:11:46.399 Yeah, fascinating concept. Instead of making everything seamless. Sometimes you 238 00:11:46.480 --> 00:11:51.679 introduce intentional disruptions like what like maybe the AI flags 239 00:11:51.679 --> 00:11:54.960 its own potential errors, calls it a self critique, or 240 00:11:54.960 --> 00:11:58.120 it poses challenge questions back to the user, prompting deeper thought. 241 00:11:58.240 --> 00:11:59.080 Why would you do that? 242 00:11:59.399 --> 00:12:02.360 It can active the user, make them more engaged, and 243 00:12:02.399 --> 00:12:05.279 fight against that tendency to just blindly trust the AI, 244 00:12:05.759 --> 00:12:07.279 what they call automation bias. 245 00:12:07.919 --> 00:12:13.000 Interesting, So deliberately slowing things down sometimes to improve the outcome. 246 00:12:13.120 --> 00:12:16.159 Exactly counterintuitive but potentially very effective. 247 00:12:16.279 --> 00:12:19.440 Okay, so you've got your data, your intelligence type, your 248 00:12:19.559 --> 00:12:22.759 UX designed for uncertainty. Now how do you make the 249 00:12:22.799 --> 00:12:25.879 AI model itself smarter, more specific to your needs? 250 00:12:26.120 --> 00:12:30.080 Right? Customization? The sources detail three main ways to customize 251 00:12:30.120 --> 00:12:33.080 language models kind of increasing in technical complexity. 252 00:12:33.120 --> 00:12:34.120 What's the starting point? 253 00:12:34.240 --> 00:12:38.200 Prompt engineering? This is your entry point, the most accessible way. 254 00:12:38.720 --> 00:12:41.200 It's all about crafting effective instructions. 255 00:12:41.200 --> 00:12:43.080 It's telling the model what you want more clearly. 256 00:12:43.399 --> 00:12:46.440 Pretty much, you guide the model without having to actually 257 00:12:46.559 --> 00:12:50.320 change the model itself. Simple tweaks in the prompt specifying 258 00:12:50.320 --> 00:12:53.000 the tone, the format, the desired output. 259 00:12:52.679 --> 00:12:56.080 Style like write in an authoritative, professional. 260 00:12:55.519 --> 00:13:01.000 Tone exactly or use a friendly, approachable voice. Small changes 261 00:13:01.039 --> 00:13:02.960 can make a huge difference to the quality. 262 00:13:03.399 --> 00:13:05.799 And there are different techniques within prompt engineering. 263 00:13:05.840 --> 00:13:10.840 Oh yes, there's basic zero shot prompting just input output, 264 00:13:10.960 --> 00:13:13.039 then FEU shot where you give it a couple of 265 00:13:13.080 --> 00:13:14.759 examples to learn from by. 266 00:13:14.600 --> 00:13:16.679 Analogy, learning by example, right, and. 267 00:13:16.639 --> 00:13:19.840 Then more advanced stuff like guiding its reasoning step by step, 268 00:13:19.919 --> 00:13:23.519 asking it to think out loud, or having it generate 269 00:13:23.600 --> 00:13:25.600 multiple answers and then kind of vote on. 270 00:13:25.519 --> 00:13:27.639 The best one, asking it to critique itself. 271 00:13:27.639 --> 00:13:31.279 Even yeah, that reflection technique, asking the LM to review 272 00:13:31.279 --> 00:13:33.600 and improve its own output. Lots you can do just 273 00:13:33.639 --> 00:13:34.279 with prompts. 274 00:13:34.440 --> 00:13:36.600 But prompts have limits. I assume they. 275 00:13:36.480 --> 00:13:40.080 Definitely do, especially when you need the AI to use specific, 276 00:13:40.279 --> 00:13:43.360 up to date or proprietary information. That's where the second 277 00:13:43.360 --> 00:13:46.120 technique comes in. Retrieval augmented generation. 278 00:13:45.960 --> 00:13:47.759 Or RAG ray. Okay, what does that do? 279 00:13:47.960 --> 00:13:52.240 This is really powerful. Our RAGE lets the AI dynamically 280 00:13:52.240 --> 00:13:56.080 retrieve relevant information from external sources. 281 00:13:55.799 --> 00:13:59.840 Like your company's internal database or recent articles. 282 00:13:59.480 --> 00:14:03.039 Exactly, internal databases, articles, meeting notes, whatever you connect it to. 283 00:14:03.679 --> 00:14:06.279 It pulls that relevant info in at the time of 284 00:14:06.320 --> 00:14:08.840 the request and weaves it into the LM's response. 285 00:14:09.000 --> 00:14:12.159 Ah, so it's not just relying on its initial training data. 286 00:14:12.240 --> 00:14:16.919 Precisely. This is crucial for factual accuracy, especially with rapidly 287 00:14:17.000 --> 00:14:21.759 changing information, and for tailoring responses using your own private data. 288 00:14:21.519 --> 00:14:25.559 Like that content generation app example, pulling in client case studies. 289 00:14:25.240 --> 00:14:30.639 Perfect example, And importantly, RA significantly reduces those hallucinations. 290 00:14:29.960 --> 00:14:33.480 Because it's grounding its answers in real retrieved information. 291 00:14:33.799 --> 00:14:36.039 You got it. It has specific texts to base its 292 00:14:36.080 --> 00:14:38.519 answer on, rather than just making things up based on 293 00:14:38.559 --> 00:14:39.360 its general training. 294 00:14:39.440 --> 00:14:43.519 Okay, so prompt engineering first, then RA for external knowledge. 295 00:14:43.639 --> 00:14:45.159 What if that's still not enough, then. 296 00:14:45.080 --> 00:14:48.960 You move to the third most involved technique, fine tuning. 297 00:14:49.120 --> 00:14:51.000 Fine tuning the model itself. 298 00:14:51.360 --> 00:14:55.480 Yes, this is for when the base LM just fundamentally 299 00:14:55.559 --> 00:14:59.200 lacks intrinsic understanding of your domain and brand voice. 300 00:14:59.159 --> 00:15:02.600 Like the example the content app struggling with really niche 301 00:15:02.639 --> 00:15:04.519 B to B sauce topics exactly. 302 00:15:05.399 --> 00:15:08.399 Prompting and roged might help, but maybe the core model 303 00:15:08.480 --> 00:15:12.039 just doesn't get the nuance, the specific terminology, the required style. 304 00:15:12.240 --> 00:15:15.279 So fine tuning trains it on that specific stuff. 305 00:15:15.360 --> 00:15:17.759 Right. You take the base model and further train it 306 00:15:17.840 --> 00:15:21.759 on your own proprietary data, your past successful content. Your 307 00:15:21.759 --> 00:15:23.720 style guides your specific. 308 00:15:23.399 --> 00:15:26.679 Jargon, so it really internalizes your specific domain and voice. 309 00:15:26.759 --> 00:15:30.080 Deeply internalize it. It moves beyond just retrieving facts. It 310 00:15:30.159 --> 00:15:33.840 learns the style, the nuance, the perspective. There's also instruction 311 00:15:33.960 --> 00:15:35.399 fine tuning, which is interesting. 312 00:15:35.440 --> 00:15:36.120 How does that work? 313 00:15:36.200 --> 00:15:39.440 The model adapts more dynamically based on direct user feedback 314 00:15:39.480 --> 00:15:42.600 and edits. If a user corrects an output, the model 315 00:15:42.679 --> 00:15:45.720 learns from that specific instruction for future tasks, makes it 316 00:15:45.759 --> 00:15:46.440 more responsive. 317 00:15:46.480 --> 00:15:50.399 Okay, three levels prompting our rag fine tuning a clear 318 00:15:50.519 --> 00:15:52.720 progression in complexity and capability. 319 00:15:52.759 --> 00:15:53.679 That's a good way to put it. 320 00:15:53.759 --> 00:15:56.480 Now, we've talked a lot about the tech, but building 321 00:15:56.480 --> 00:16:00.399 these products, it's not just code and models, is it. 322 00:16:00.399 --> 00:16:01.399 It's about people? 323 00:16:01.519 --> 00:16:06.000 Oh? Absolutely, central. The sources really emphasize this. AI teams 324 00:16:06.080 --> 00:16:09.200 are almost by definition diverse and interdisciplinary. 325 00:16:09.279 --> 00:16:11.240 You need lots of different skills. 326 00:16:11.279 --> 00:16:15.720 You do software engineers, data scientists, UX designers, crucially, domain 327 00:16:15.799 --> 00:16:19.039 experts who actually understand the area the AI is working in. 328 00:16:18.960 --> 00:16:21.600 And the product manager what's their role in this mix? 329 00:16:21.960 --> 00:16:25.399 The PM becomes almost an educator and a translator. They 330 00:16:25.440 --> 00:16:29.159 have to bridge these different worldviews, different technical languages, different. 331 00:16:28.840 --> 00:16:32.080 Goals, balancing the tech possibilities with the user needs and 332 00:16:32.120 --> 00:16:33.120 the business. 333 00:16:32.679 --> 00:16:36.879 Goals, and reconciling priorities between all these different stakeholders. It's 334 00:16:36.879 --> 00:16:39.159 a complex coordination role, especially. 335 00:16:38.840 --> 00:16:41.120 As we said, with all the uncertainty involved, how do 336 00:16:41.159 --> 00:16:43.240 you manage that the potential for failure. 337 00:16:43.799 --> 00:16:46.759 This brings us back squarely to AI governance. The sources 338 00:16:46.759 --> 00:16:48.720 spend a lot of time on this, and rightly so 339 00:16:49.320 --> 00:16:50.679 it's paramount. 340 00:16:50.200 --> 00:16:52.600 What does governance cover specifically in AI? 341 00:16:53.000 --> 00:16:56.799 Several critical areas. First, security, protecting. 342 00:16:56.360 --> 00:16:59.360 The whole system against what kind of threats, things. 343 00:16:59.080 --> 00:17:01.840 Like data poisoning at attackers feeding bad examples to mess 344 00:17:01.879 --> 00:17:06.000 up the training data, data exiltration and leakage sensitive information 345 00:17:06.079 --> 00:17:10.519 getting out maybe through prompts or model outputs insecure output handling. 346 00:17:10.759 --> 00:17:14.599 Imagine the AI generating harmful code like a delete database 347 00:17:14.680 --> 00:17:18.319 query when it should have done a select and even 348 00:17:18.519 --> 00:17:21.960 model theft attackers trying to steal or replicate your valuable 349 00:17:22.000 --> 00:17:22.640 AI model. 350 00:17:22.920 --> 00:17:24.759 So security is multifaceted. 351 00:17:24.799 --> 00:17:29.119 What else under governance privacy by design, building privacy and 352 00:17:29.160 --> 00:17:30.799 from the start, not bolting it on later. 353 00:17:30.839 --> 00:17:31.480 How do you do that? 354 00:17:31.559 --> 00:17:35.839 Proactive risk assessments, making privacy the default setting, ensuring end 355 00:17:35.880 --> 00:17:39.359 to end security for data, and being transparent with users 356 00:17:39.359 --> 00:17:40.640 about how their data is used. 357 00:17:40.720 --> 00:17:43.039 Transparency again seems vital it is. 358 00:17:43.519 --> 00:17:47.160 Then there's mitigating bias. This is huge. AI models can 359 00:17:47.200 --> 00:17:51.359 easily pick up and even amplify existing societal biases present 360 00:17:51.440 --> 00:17:52.359 in their training data. 361 00:17:52.440 --> 00:17:53.240 So how do you fight that? 362 00:17:53.480 --> 00:17:57.480 Strategies include things like data audits, carefully examining your training 363 00:17:57.559 --> 00:18:00.160 data for imbalances, maybe using tools like fair learn and 364 00:18:01.319 --> 00:18:06.200 algorithmic bias mitigation techniques, trying to make the model itself fairer, 365 00:18:06.480 --> 00:18:07.960 and explaining its decisions. 366 00:18:08.079 --> 00:18:10.880 Explaining decisions That sounds like the next part exactly. 367 00:18:10.920 --> 00:18:16.519 Transparency and accountability, providing explainability, understanding how the AI reached 368 00:18:16.519 --> 00:18:22.640 a decision, and interpretability making the outputs understandable and actionable. 369 00:18:22.160 --> 00:18:24.680 For humans and keeping humans involved. 370 00:18:24.319 --> 00:18:28.119 Absolutely, establishing human in the loop or human on the 371 00:18:28.200 --> 00:18:32.839 loop processes, especially for high risk decisions, ensuring a human 372 00:18:32.880 --> 00:18:36.240 can always intervene, review, or override the AI. 373 00:18:36.640 --> 00:18:39.640 So governance is really about building trust and safety around 374 00:18:39.680 --> 00:18:40.000 the AI. 375 00:18:40.279 --> 00:18:42.359 That's a perfect summary. It's the essential foundation. 376 00:18:42.599 --> 00:18:45.960 Okay. So with all that complexity, the tech, the teams, 377 00:18:46.440 --> 00:18:49.839 the governance, how do you effectively communicate the value of 378 00:18:49.839 --> 00:18:52.440 what you're building, especially to stakeholders who might not be 379 00:18:52.519 --> 00:18:53.200 AI experts. 380 00:18:53.319 --> 00:18:57.240 Great question, The sources stress being concrete. Articulate the value 381 00:18:57.279 --> 00:18:58.319 in tangible terms. 382 00:18:58.400 --> 00:19:01.440 So instead of just saying it boosts sufficiency. 383 00:19:01.160 --> 00:19:05.359 Quantify it for efficiency and productivity. Show the time saved. 384 00:19:05.680 --> 00:19:09.680 The example they used was brilliant. Our platform automates data aggregation, 385 00:19:10.119 --> 00:19:13.200 saving your team eighteen point three to three hours. 386 00:19:12.839 --> 00:19:14.559 Per week, okay, specific and. 387 00:19:14.519 --> 00:19:17.799 Then translate that into cost savings. That's equivalent to nearly 388 00:19:17.880 --> 00:19:21.640 forty seven, six and fifty eight dollars annually. Numbers make 389 00:19:21.680 --> 00:19:22.480 the impact real. 390 00:19:22.720 --> 00:19:25.559 That definitely makes it easier to understand the ROI. What 391 00:19:25.640 --> 00:19:29.319 about communicating well the inevitable failures? 392 00:19:29.359 --> 00:19:32.880 Ah? Yes, communicating about failure. This is critical for managing 393 00:19:32.920 --> 00:19:34.599 expectations and building trust. 394 00:19:34.680 --> 00:19:35.920 So don't hide the mistakes. 395 00:19:36.079 --> 00:19:39.279 Absolutely not be realistic, the sources say. Instead of pretending 396 00:19:39.279 --> 00:19:42.400 engineers can eliminate all errors, you need to be upfront 397 00:19:42.599 --> 00:19:43.759 about the types of mistakes. 398 00:19:43.759 --> 00:19:46.200 The AI will make like what kinds of mistakes. 399 00:19:45.799 --> 00:19:49.599 They list common ones, false positives, predicting something that doesn't happen, 400 00:19:49.680 --> 00:19:52.680 like a sales spike, false negatives, missing something important like 401 00:19:52.720 --> 00:19:56.480