WEBVTT 1 00:00:00.120 --> 00:00:04.559 Is staying in the presidential race despite losing the South Carolina primary to give Republicans 2 00:00:04.599 --> 00:00:08.759 an option other than former President Trump. Haley told supporters last night that she's 3 00:00:08.800 --> 00:00:12.800 a woman of her word and will continue running despite the loss. Haley said 4 00:00:12.839 --> 00:00:16.399 she won't give up the fight while the majority of Americans are unhappy with both 5 00:00:16.440 --> 00:00:22.760 Donald Trump and Joe Biden. I'm Chris Caragio, NBC News Radio, NBC 6 00:00:22.800 --> 00:00:28.640 News on CACAA Lomelinda sponsored by Teamsters Local nineteen thirty two, Protecting the Future 7 00:00:28.719 --> 00:00:40.159 of Working Families Teamsters nineteen thirty two dot org. The information economy has a 8 00:00:40.280 --> 00:00:45.679 rod. The world is teeming with innovation as new business models reinvent every industry 9 00:00:45.719 --> 00:00:51.039 industry. Inside Analysis is your source of information and insight about how to make 10 00:00:51.079 --> 00:00:56.920 the most of this exciting new era. Learn more at Inside analysis dot Comsideanalysis 11 00:00:57.000 --> 00:01:06.480 dot com. And now here's your host, Eric. All right, folks, 12 00:01:06.560 --> 00:01:10.439 welcome to the future. Indeed, yours truly Eric Cavanaugh here and the 13 00:01:10.480 --> 00:01:14.000 only Coast to coast show. And that's all about the information economy. It's 14 00:01:14.040 --> 00:01:15.959 time for Inside Analysis, and folks, we have an all star cast for 15 00:01:17.000 --> 00:01:22.719 you today and a white hot topic We're going to talk about what is responsible 16 00:01:22.840 --> 00:01:26.120 AI? Is there such a thing as responsible AI? There are responsible people 17 00:01:26.599 --> 00:01:32.120 using AI, setting up AI, coding AI, leveraging AI. But the 18 00:01:32.200 --> 00:01:36.640 people have to be responsible and then how do you enforce that well with rules, 19 00:01:36.640 --> 00:01:40.959 with guidelines, with policies, and then with actual enforcement. Or people 20 00:01:41.040 --> 00:01:42.680 get in trouble, I want to do bad things, or they're simply not 21 00:01:42.799 --> 00:01:47.480 allowed to do things. That's usually the way it's going to be. We'll 22 00:01:47.480 --> 00:01:49.280 talk about that on the show. Today. We've got wonderful guests. My 23 00:01:49.319 --> 00:01:53.159 book buddy David Lintigham is here. He's got some books. You can look 24 00:01:53.239 --> 00:01:57.519 up a new YouTube channel with eight thousand and climbing subscribers, so look those 25 00:01:57.560 --> 00:02:01.640 folks up online on YouTube. Our buddy Andy Hannah is here from fourteen eighty 26 00:02:01.640 --> 00:02:07.959 six lab and the University of Pennsylvania. And johnsu Junja is here from the 27 00:02:07.080 --> 00:02:13.159 Institute for Experiential AI, which is part of Northeastern University out there in Massachusetts. 28 00:02:13.800 --> 00:02:15.439 What is responsible AI? First of all, I would like to dedicate 29 00:02:15.479 --> 00:02:19.919 this show to my good buddy I learned today he passed away, Rick Sherman. 30 00:02:20.199 --> 00:02:23.560 He actually taught at Northeastern. So talk about closing a loop here on 31 00:02:23.639 --> 00:02:27.520 one of the nicest guys ever met. He's the hardest working consultant in business, 32 00:02:27.560 --> 00:02:30.360 and he passed away apparently just about a year ago. I heard from 33 00:02:30.400 --> 00:02:31.960 his partner today, So this show is for you, buddy. It was 34 00:02:32.159 --> 00:02:37.360 a regular here on our radio shows and it's just a super cool guy and 35 00:02:37.400 --> 00:02:38.599 he would talk about this kind of stuff, ethics. We're going to talk 36 00:02:38.639 --> 00:02:42.680 about ethics. So I'm going to steal that everyone's thunder. I'm going to 37 00:02:42.719 --> 00:02:46.280 say. The most important thing you can do to practice responsible AI is every 38 00:02:46.319 --> 00:02:51.280 time you launch an AI initiative, you document what is the purpose of this 39 00:02:51.400 --> 00:02:53.800 initiative, what are we trying to accomplish, how we're going to accomplish it, 40 00:02:53.960 --> 00:02:57.560 And then, over time, once a quarter, once a year, 41 00:02:57.599 --> 00:03:00.199 perhaps depending on the cadence, you'd look at the resis and say, are 42 00:03:00.240 --> 00:03:04.000 we achieved what we thought we would achieve? We were trying to lower bias 43 00:03:04.039 --> 00:03:08.000 in loans, for example, we're trying to get more better customer service for 44 00:03:08.080 --> 00:03:13.000 our customers out there. Are we achieving what we thought we wanted to achieve? 45 00:03:13.479 --> 00:03:15.639 If the answer is no, change something. If the answer is yes, 46 00:03:16.039 --> 00:03:19.960 good job, keep going apart from that, it's going to take policies, 47 00:03:20.000 --> 00:03:22.560 it's going to take rules, it's going to take adherence. And what 48 00:03:22.599 --> 00:03:24.520 they talked about at the Institute for Experiential AI, And I'll go with ladies 49 00:03:24.560 --> 00:03:28.439 first year, I guess and hand it over to Johnsue here in a second 50 00:03:29.000 --> 00:03:32.240 is what they say is human in the loop. So I always joke about 51 00:03:32.280 --> 00:03:37.719 the media and how the narrative is always wrong. The media narrative about AI 52 00:03:37.800 --> 00:03:38.919 seems to be, Oh, it's going to take away jobs, it's going 53 00:03:38.960 --> 00:03:43.560 to control everything. We'll be answering to the AI overlords. All this kind 54 00:03:43.599 --> 00:03:49.400 of stuff that is very much inaccurate. AI that does not have human control 55 00:03:49.479 --> 00:03:52.360 and shepherding is going to go off the rails pretty fast. If you don't 56 00:03:52.360 --> 00:03:54.800 ask me. Look up Tay and Microsoft when they try to do that a 57 00:03:54.800 --> 00:04:00.479 few years ago, it didn't go so well. It was a chat Twitter 58 00:04:00.520 --> 00:04:03.479 that started spewing racial epithets at people right at the gate. They're like, 59 00:04:03.520 --> 00:04:08.759 WHOA, shut it down, shut it down. Yeah, So lessons learned 60 00:04:08.759 --> 00:04:11.599 there. But Johnson, I'll throw it over to you from the Institute for 61 00:04:11.800 --> 00:04:15.399 Experiential AI. How do you define responsible AI? How can people do AI 62 00:04:15.480 --> 00:04:23.279 responsibly. Thank you, Eric. I'm a philosopher and I think definition is 63 00:04:23.319 --> 00:04:25.759 like the one of the hardest things that you can ask a philosopher. So 64 00:04:25.839 --> 00:04:30.959 thanks for starting with me. So let me go back to what you said. 65 00:04:30.959 --> 00:04:34.519 But is responsible AI where we have you know, responsible people, we 66 00:04:34.560 --> 00:04:39.519 have responsible ways of acting. But is there responsible AI? And actually, 67 00:04:39.600 --> 00:04:43.920 yes, you're right, responsible AI is a shorthand we are not talking about 68 00:04:43.920 --> 00:04:47.240 an AI system that is responsible, but we are talking about developing and deploying 69 00:04:47.680 --> 00:04:54.079 AI systems responsibly. So the responsibility invariably falls within humans, not on the 70 00:04:54.120 --> 00:04:59.360 AI system itself, and on the humans throughout the AIS life cycle. From 71 00:04:59.519 --> 00:05:03.240 the moment that you have the idea of this is a good AI, this 72 00:05:03.319 --> 00:05:06.560 would be a good place where we use AI system, This would create more 73 00:05:06.639 --> 00:05:11.600 value to the point where you have already released the AI system and you are 74 00:05:11.720 --> 00:05:15.279 monitoring it. You should be monitoring it as you just said, and check 75 00:05:15.319 --> 00:05:20.399 whether the system does what it what this is supposed to do. At the 76 00:05:20.439 --> 00:05:26.680 core of the responsible AI is the question of making sure that the AI systems 77 00:05:27.000 --> 00:05:31.759 we create, we developed, we deploy are good. That's really just that 78 00:05:31.959 --> 00:05:36.240 right, And the definition of good of course here holds a huge work. 79 00:05:38.600 --> 00:05:41.759 What we mean by good, well, we mean AI system that does not 80 00:05:41.879 --> 00:05:46.839 harm individuals or society that does good for individuals, and society that does not 81 00:05:47.079 --> 00:05:51.600 create further inequalities in the society, so that furthers us in the fair society 82 00:05:51.680 --> 00:05:58.759 goal. And AI systems that help us keep our agency, keep our individuality. 83 00:05:59.199 --> 00:06:00.920 So when you say, you know human in the look, are they 84 00:06:00.959 --> 00:06:03.959 going to take over AI systems that are going to take over or are they 85 00:06:04.040 --> 00:06:08.519 going to be there for us? Are we going to be controlling them? 86 00:06:08.800 --> 00:06:11.959 I think neither is the right answer. The answer is that we should be 87 00:06:12.000 --> 00:06:17.399 collaborating with AI systems because we create AI systems where humans are not doing a 88 00:06:17.439 --> 00:06:20.879 great job. We have very hard time. Our brains have hard time with 89 00:06:21.199 --> 00:06:25.120 a lot of information, a lot of variables. That's why we need AI 90 00:06:25.199 --> 00:06:30.240 system So there is value in building AI systems. And we cannot say that 91 00:06:30.480 --> 00:06:32.160 AI system gives me a recommendation and I'm just going to say yes or no, 92 00:06:32.519 --> 00:06:35.480 this is how I control Well, that's like falling back to our own 93 00:06:35.519 --> 00:06:39.480 biases, isn't it. So what we want to be able to say is 94 00:06:39.519 --> 00:06:44.480 that we understand the AI system, how it reaches its decisions, we collaborate 95 00:06:44.560 --> 00:06:53.199 with it irrationally, intellect intelligently, and augment ourselves our ways of being around, 96 00:06:53.240 --> 00:06:58.079 doing business, doing work, helping the society. Yeah, that's great. 97 00:06:58.120 --> 00:07:00.600 I think I love that you're a philosopher. Philosopher as well, and 98 00:07:00.639 --> 00:07:04.360 so I think about these things and think about the meaning of words and what 99 00:07:04.360 --> 00:07:08.720 we're trying to say, and you know, just real quick, in American 100 00:07:08.759 --> 00:07:13.000 culture and Western culture in particular, I've counseled some people on this over the 101 00:07:13.120 --> 00:07:15.040 years. You have to watch out for the language that you use, because 102 00:07:15.040 --> 00:07:20.040 a lot of times in advertising, companies will use language to imply that they 103 00:07:20.079 --> 00:07:24.720 have what they don't have, or they use language to sort of fortify the 104 00:07:24.800 --> 00:07:28.439 image of their company when they're not like that at all. So it's like, 105 00:07:28.480 --> 00:07:30.759 if you say, oh, it's responsible AI in the Western world, 106 00:07:30.920 --> 00:07:34.600 I'm gonna think you're probably not very responsible. If you're saying you're responsible, 107 00:07:34.639 --> 00:07:40.000 I don't believe you. So there is that interesting dynamic in our culture because 108 00:07:40.040 --> 00:07:44.560 we have so much advertising, We have so much media everywhere constantly promoting this 109 00:07:44.600 --> 00:07:46.800 is what we're doing, this is what we're doing. That's why I say 110 00:07:46.879 --> 00:07:48.680 nouns and verbs, folks talk about the nouns and verbs, leave about the 111 00:07:48.720 --> 00:07:53.600 adjectives and the adverbs to explain what you're doing. And if I hear you 112 00:07:53.600 --> 00:07:57.240 correctly Johnson, I'll throw it over to David. After this, you're basically 113 00:07:57.319 --> 00:08:01.399 saying carefulness as careful does, and you need to document what you're doing and 114 00:08:01.480 --> 00:08:05.959 document your policies and then just have a rational discussion around that, right Johnson. 115 00:08:07.360 --> 00:08:11.079 Absolutely, But in addition to that, again being an ethicist, being 116 00:08:11.120 --> 00:08:15.279 a philosopher, I would say, you know, it's not just about documenting 117 00:08:15.360 --> 00:08:18.360 the right things to do. A lot of the time, the exciting thing 118 00:08:20.199 --> 00:08:24.240 AI in AI ethics is that we don't know, we don't come it's a 119 00:08:24.279 --> 00:08:26.680 complex problem. Just AI is complex. AI ethics is complex. Response by 120 00:08:26.680 --> 00:08:31.519 II is complex. So we need to work at it like it's not just 121 00:08:31.560 --> 00:08:35.279 like follow the rules, but as ethicis what we do is we look at 122 00:08:35.320 --> 00:08:37.320 it as a puzzle. How do you make this system fair? You know, 123 00:08:37.519 --> 00:08:39.840 just just going at it, trying to figure it out, trying to 124 00:08:39.840 --> 00:08:43.360 make it better, iterate on it. So it's not just documenting but striving 125 00:08:43.360 --> 00:08:48.720 for better and trying to become better, looking at it in an innovative way, 126 00:08:48.799 --> 00:08:52.039 even when you're coming at it from the ethics perspective, not just from 127 00:08:52.120 --> 00:08:56.039 the AI innovator perspective. Yeah, that's very interesting. That's a good point. 128 00:08:56.279 --> 00:08:58.960 David. Then to come, I'll bring you in our cloud expert. 129 00:09:00.080 --> 00:09:01.840 You've been building these systems for a long time. You told a story before 130 00:09:01.840 --> 00:09:05.080 the show about how you were dragged into court to explain how an AI system 131 00:09:05.120 --> 00:09:07.639 works. So you know that, so you do have the answers for these 132 00:09:07.639 --> 00:09:11.440 things. Go ahead tell us. Yeah, first thing, that was a 133 00:09:11.480 --> 00:09:15.759 great That was a great explanation that we just heard. The first thing we 134 00:09:15.840 --> 00:09:20.080 have to understand that are we using this for the appropriate use case? That's 135 00:09:20.120 --> 00:09:24.080 the first level of responsibility. So everybody goin goes to ethics and biases and 136 00:09:24.120 --> 00:09:26.080 things like that, and I don't think that's going to be the case each 137 00:09:26.080 --> 00:09:30.480 and every time. So in many instances it's going to be misapplied. I 138 00:09:30.480 --> 00:09:33.919 think that's the single most common reason I see AI things fail is because they're 139 00:09:33.919 --> 00:09:37.000 trying to solve the wrong problems. With AI. You're taking the ethics stuff 140 00:09:37.000 --> 00:09:41.799 out of it. So is it used in the right responsible place? You 141 00:09:41.799 --> 00:09:43.919 know, if it's doing loan applications, we actually need an AI system to 142 00:09:43.919 --> 00:09:48.360 make that happen. So we have to ask ourselves it's not that we can, 143 00:09:48.480 --> 00:09:52.039 it's if we should, if something should occur or not, and this 144 00:09:52.120 --> 00:09:54.720 is the appropriate technology to make it happen. And if it is the appropriate 145 00:09:54.759 --> 00:09:58.200 technology to make it happen, you have to make sure. There's got to 146 00:09:58.200 --> 00:10:01.919 remember, I'm the geek here, that there's ano infrastructure in terms of security 147 00:10:01.960 --> 00:10:07.440 and governance, audit capabilities to ensure that we're doing the right things in coming 148 00:10:07.440 --> 00:10:11.799 to the most accurate and unbiased conclusion. You can't eliminate bias completely, but 149 00:10:11.840 --> 00:10:16.279 you have to have the auditing capabilities of continuous learning capabilities that get you to 150 00:10:16.799 --> 00:10:20.960 the best solution you gets you to the best answers out of these systems that 151 00:10:20.000 --> 00:10:24.320 you that you can have, And so you need to have different people on 152 00:10:24.360 --> 00:10:26.440 this team. You know, It's funny. It's like all the AA systems 153 00:10:26.639 --> 00:10:31.360 that I work on as an architect, I have a ethics specialist that sits 154 00:10:31.360 --> 00:10:35.039 on the team, and that person is responsible for dealing with the bias stuff, 155 00:10:35.480 --> 00:10:39.519 the technical stuff we're dealing with. But I'm sorry, what issues, 156 00:10:39.600 --> 00:10:43.240 what implementations, what process needs to be put in place to ensure that this 157 00:10:43.279 --> 00:10:46.000 thing doesn't go off the rails. And I always tell, you know, 158 00:10:46.039 --> 00:10:50.320 people who take my architect class that you need to build this system as if 159 00:10:50.360 --> 00:10:52.200 you're going to be testifying in court, because many instances, as you are, 160 00:10:52.919 --> 00:10:56.399 you're going to be brought in to, you know, explain what you 161 00:10:56.440 --> 00:11:00.600 did because the system is assumed to have some sort of an income assistency and 162 00:11:00.639 --> 00:11:03.120 it's some sort of a bias that may damage somebody, and they're going to 163 00:11:03.120 --> 00:11:09.039 bring the lead architect in there to explain how that system is not biased and 164 00:11:09.080 --> 00:11:11.320 how you did the you know, dotted your eyes across your T is to 165 00:11:11.360 --> 00:11:16.799 got to build a system that's completely right. Yeah, that's a really that's 166 00:11:16.799 --> 00:11:20.960 a really good point. You do have to dot as cross t's be able 167 00:11:20.000 --> 00:11:22.840 to explain it. You know, I sit around a lot and when I'm 168 00:11:22.919 --> 00:11:28.240 lying in bed, and instead of just pondering subjects, I find myself explaining 169 00:11:28.279 --> 00:11:31.720 things and then re explaining them and imagining, well, what if someone says 170 00:11:31.759 --> 00:11:33.919 this, then how would you explain it? And you do a lot of 171 00:11:33.960 --> 00:11:37.440 that, and it's good because it kind of helps you navigate around the hard 172 00:11:37.480 --> 00:11:41.399 corners. I suppose is one way to put it, because in order to 173 00:11:41.440 --> 00:11:46.759 have any communication, there must be context that has agreed upon. We must 174 00:11:46.840 --> 00:11:50.000 both know, relatively speaking, what we're talking about to be able to use 175 00:11:50.080 --> 00:11:52.639 terms and phrases to explain something. And a lot of times, especially in 176 00:11:52.679 --> 00:11:56.679 the court of law, you're dealing with people who may have no idea how 177 00:11:56.679 --> 00:11:58.759 this stuff works, so you really have to kind of lay it out and 178 00:11:58.799 --> 00:12:01.080 show a diagram and say, the input comes in here, this is the 179 00:12:01.120 --> 00:12:05.840 training data, this is the inference that it makes, and then you know, 180 00:12:05.919 --> 00:12:09.759 to unwind that can be a challenge. You know, a lot of 181 00:12:09.799 --> 00:12:13.679 the AI systems we've seen over the years were black boxes. One of them 182 00:12:13.720 --> 00:12:16.039 is today. Chat GPT is a black box today. So it's hard to 183 00:12:16.480 --> 00:12:20.240 explain how a black box works, right, David, absolutely is. And 184 00:12:20.440 --> 00:12:24.600 the thing is, sometimes I almost figure I need to bring puppets into those 185 00:12:24.639 --> 00:12:28.840 situations. And you get really good at explaining very complex technical topics in ways 186 00:12:28.840 --> 00:12:31.799 that layman can understand. If you think about it, looking at what AI 187 00:12:31.879 --> 00:12:37.000 systems are, generative AI be in an instance of that extremely complex well, 188 00:12:37.000 --> 00:12:41.360 all these inner working things. You know, you know generative adversarial networks and 189 00:12:41.360 --> 00:12:43.720 how they work, but you know, different sorts of you can't get to 190 00:12:43.759 --> 00:12:46.720 that level. You really need to get to the levels of what's the functionality 191 00:12:46.759 --> 00:12:50.519 of this stuff, how it works, and where it can where it can 192 00:12:50.559 --> 00:12:52.200 go off the rails, how it should be monitored, how it should be 193 00:12:52.240 --> 00:12:54.679 governed. And then people get to that and they go, Okay, you're 194 00:12:54.679 --> 00:12:58.440 putting in the places. You're putting in the consistency of the guardrail to make 195 00:12:58.480 --> 00:13:01.039 sure that it does and do these bad things, that it doesn't become a 196 00:13:01.039 --> 00:13:05.120 bad actor. You know that it doesn't end up attacking human beings, are 197 00:13:05.159 --> 00:13:09.600 being biased against human beings, and that has to be kind of core to 198 00:13:09.639 --> 00:13:13.799 the message. And so people who understand AI should be really good at explaining 199 00:13:13.799 --> 00:13:18.879 it at cocktail parties. That's a great that's a great metric. And we 200 00:13:18.919 --> 00:13:20.519 have a question from the audience, and perfect timing, I'll throw it over 201 00:13:20.559 --> 00:13:24.879 to Andy Hannah, who I'm sure he knows a thing or two about ethics 202 00:13:24.879 --> 00:13:28.159 and responsible AI. And of course, fourteen eighty six Labs serves as a 203 00:13:28.200 --> 00:13:33.960 liaison or a shepherd between data purchasers and data sellers. And guess what, 204 00:13:35.039 --> 00:13:39.000 here's a question that just came in across the transmitter from our live studio audience. 205 00:13:39.279 --> 00:13:41.480 If I'm using a lot of third party data in my AI solution, 206 00:13:41.559 --> 00:13:45.799 how do I know there isn't biased in that data? Is there's some sort 207 00:13:45.840 --> 00:13:48.360 of seal of approval. That's when you talk to someone like Andy Hannah from 208 00:13:48.360 --> 00:13:52.639 fourteen eighty six Labs, Andy, what do you think? Yeah? I 209 00:13:52.639 --> 00:13:56.879 think you know. Determining the quality of data is a really challenging thing, 210 00:13:56.080 --> 00:14:03.440 right because we think about things that complete this accuracy and comparability, timeliness, 211 00:14:03.919 --> 00:14:07.360 but that all relates a specific set of data as it relates a specific use 212 00:14:07.399 --> 00:14:13.279 case. I think what we need to start thinking about is how those data 213 00:14:13.320 --> 00:14:16.360 sets are put together. So who, you know, the organization itself, 214 00:14:18.039 --> 00:14:22.080 are they giving us transparency about how those data sets are built? What the 215 00:14:22.200 --> 00:14:28.600 sources are is specifically a need. Oftentimes we can buy data sets but not 216 00:14:28.720 --> 00:14:35.240 really understand where the original source is, and sometimes we can't even understand how 217 00:14:35.279 --> 00:14:39.600 it's put together. So I think that's going to add to transparency around the 218 00:14:39.679 --> 00:14:46.600 data quality and especially third party data in terms of our ability to get the 219 00:14:46.639 --> 00:14:50.559 most value out of it, because you know, the bias in these models 220 00:14:50.879 --> 00:14:56.519 are often coming from the data itself. If we're building those models off of 221 00:14:56.759 --> 00:15:01.120 historic data, and that historic data is used to make biased decisions in the 222 00:15:01.200 --> 00:15:05.840 past, is going to repeat itself. I think one of the interesting things 223 00:15:05.879 --> 00:15:09.200 that was mentioned earlier is about purpose. I think you said it at the 224 00:15:09.240 --> 00:15:11.440 top of the hour, Eric, is that we need to think about the 225 00:15:11.480 --> 00:15:16.000 purpose of why these systems, why we're putting these systems in place, and 226 00:15:16.080 --> 00:15:22.080 what is the intent of the people that are going to use these AI systems. 227 00:15:22.480 --> 00:15:26.039 And with that then we can start to understand where bias might seep in 228 00:15:26.159 --> 00:15:33.159 or where we might be hurting a particular portion of the of the of the 229 00:15:33.200 --> 00:15:37.279 population. In that case, what we need to do better is teach. 230 00:15:37.440 --> 00:15:43.159 We need to teach our students, We need to teach our employees how best 231 00:15:43.200 --> 00:15:46.639 to use that those systems, how to put them together, how to put 232 00:15:46.639 --> 00:15:50.759 the right data into the systems in order to get the outcomes that we want 233 00:15:50.799 --> 00:15:56.919 that are fair, biased, I'm biased, and ethical. Yeah, those 234 00:15:56.919 --> 00:16:00.279 are all excellent points too, And I think the teaching is really key, 235 00:16:00.480 --> 00:16:04.720 right because well, what did Mark Twain say, prejudice or common sense or 236 00:16:04.759 --> 00:16:08.200 prejudice one of the two is that set of values you've adopted by the age 237 00:16:08.240 --> 00:16:11.879 of eighteen right, And so depending upon where you grew up, you can 238 00:16:11.919 --> 00:16:15.679 have a different value set and people can disagree on how to do business in 239 00:16:15.720 --> 00:16:22.000 certain environments. So to have an understanding of that and to document that it 240 00:16:22.080 --> 00:16:23.039 kind of gets back to what I was saying at the top of the hour. 241 00:16:23.120 --> 00:16:26.240 But it's kind of difficult. So where do you capture that information? 242 00:16:26.320 --> 00:16:30.159 I think it's when you launch an AI project. That's where you want to 243 00:16:30.799 --> 00:16:34.639 document what you're trying to accomplish, what data sets you're using. You don't 244 00:16:34.639 --> 00:16:38.240 have to do too much, but at least explain why did you choose this 245 00:16:38.360 --> 00:16:41.919 data, what do you expect to get from this data? All these kinds 246 00:16:41.919 --> 00:16:45.240 of things. At least help the human be more aware is to go through 247 00:16:45.279 --> 00:16:48.120 the process of executing the job, right Andy, that's right, Yeah, 248 00:16:48.519 --> 00:16:53.639 I think that the more transparency that we can bring to the data sets that 249 00:16:53.679 --> 00:16:59.759 we're using right now. We often think about managing our internal data because that's 250 00:16:59.799 --> 00:17:03.039 what it's most plentiful to us, and we augment that with external and third 251 00:17:03.119 --> 00:17:07.240 party data, and we need to start thinking about that more as an asset. 252 00:17:07.559 --> 00:17:11.440 You know, it still strikes me as very interesting that we can't put 253 00:17:11.480 --> 00:17:14.759 those that data sets on the balance sheet as an asset. I think our 254 00:17:14.799 --> 00:17:21.480 accounting system is well behind in that. So once we're forced to both value 255 00:17:21.559 --> 00:17:29.240 and understand the data, it forces us to get more put more governance around 256 00:17:29.279 --> 00:17:33.240 those assets and how we use them. Yeah, and you know, it 257 00:17:33.240 --> 00:17:37.160 gets into topics like data fabric for example. So we're still in an era 258 00:17:37.279 --> 00:17:44.039 when most information systems relying database. The more advanced companies out there are using 259 00:17:44.079 --> 00:17:48.240 what is referred to as a data fabric, which is significantly more complex. 260 00:17:48.839 --> 00:17:52.119 It requires more investment. Obviously only larger organizations can get serious about that. 261 00:17:52.160 --> 00:17:56.759 But now, of course Microsoft has rolled out Microsoft Fabric. I have not 262 00:17:56.839 --> 00:18:00.519 taken a briefing on that, so I can't go into detail about gets in 263 00:18:00.559 --> 00:18:04.480 there. But we do have systems that are now more able to leverage metadata. 264 00:18:04.519 --> 00:18:07.559 For example, like when you load a data set, it could have 265 00:18:07.720 --> 00:18:11.599 in one of the about fields this data is only to be used in marketing 266 00:18:11.599 --> 00:18:15.480 in North America or something like that. There are ways to do that, 267 00:18:15.839 --> 00:18:18.599 but you always have to remember in the handoffs, does everyone read the instructions? 268 00:18:18.640 --> 00:18:22.160 Does everyone read the instructions when they build toys for their kids? Probably 269 00:18:22.200 --> 00:18:26.599 not. They probably just go right in and start using stuff. So the 270 00:18:26.160 --> 00:18:30.400 more you can infuse these little tidbits the better. And AI is good at 271 00:18:30.400 --> 00:18:32.599 that. But don't touch up that. I'll be right back. You're listening 272 00:18:32.640 --> 00:18:44.599 to Inside Analysis. Expected, Welcome back to Inside Analysis. Here's your host, 273 00:18:44.880 --> 00:18:49.759 Eric Tabanaugh. All right, folks, back here on Inside Analysis with 274 00:18:49.799 --> 00:18:56.400 an all star cast talking about what is responsible AI and John Su I'm gonna 275 00:18:56.400 --> 00:19:00.279 throw this one over to you from the Institute for Experiential AI that means Human 276 00:19:00.279 --> 00:19:03.720 in the Loop people Northeastern University, where my buddy Rick Sherman used to teach 277 00:19:03.759 --> 00:19:07.240 Small World John so throat out to you. We already heard from David a 278 00:19:07.279 --> 00:19:11.720 little tip from his experience that the ethicist would be on the team. But 279 00:19:11.839 --> 00:19:15.200 the team of people pulled together to build some project. One person's job is 280 00:19:15.240 --> 00:19:18.480 to, you know, let's think about what we're doing here. Is it 281 00:19:18.519 --> 00:19:21.240 ethical or is it not? Where could we run into trouble? Not just 282 00:19:21.359 --> 00:19:23.400 legal trouble, but ethical trouble. So I'll ask you, John Sue, 283 00:19:23.480 --> 00:19:29.960 how does your work manifest in client engagements? Yeah? I'm always very happy 284 00:19:29.960 --> 00:19:32.920 to hear these type of things from others and not me saying that, you 285 00:19:32.920 --> 00:19:34.960 know, itsuis cannot be sitting outside. We are not supposed to be at 286 00:19:36.000 --> 00:19:41.680 a board and trying to look over police you and tell you what to do 287 00:19:41.720 --> 00:19:44.160 and what not to do. No, it's supposed to be a collaborative work. 288 00:19:44.240 --> 00:19:47.680 So the way that we are working with clients and the way that we 289 00:19:47.759 --> 00:19:52.880 encourage clients to build governance structures and is to create a workflow to create an 290 00:19:52.920 --> 00:19:59.440 AI innovation process that involves ethics and ethicisms in the whole innovation life cycle. 291 00:19:59.519 --> 00:20:03.720 So it's a collaboration. Just having is not sufficient. You have to have 292 00:20:03.920 --> 00:20:11.640 a competent, multidisciplinear team who have who that includes ethicism, that includes technical 293 00:20:11.640 --> 00:20:15.920 folks who work on responsible AI and also design people. For example, because 294 00:20:15.960 --> 00:20:19.359 you are going to create a user interface. Creating a user interface is ethically 295 00:20:19.440 --> 00:20:22.599 laded question. But edicis are not going to be the ones who are going 296 00:20:22.680 --> 00:20:26.200 to be building the user interface. So they work, they collaborate with developers, 297 00:20:26.240 --> 00:20:32.559 they collaborate with designers. What we do is to ensure that there is 298 00:20:32.599 --> 00:20:36.799 this workflow, there is this process, and in order to keep this process 299 00:20:36.920 --> 00:20:41.640 running, that for every AI system that goes that goes through the development cycle, 300 00:20:41.720 --> 00:20:47.200 that goes through procurement cycle or deployment cycle. You have to have governance 301 00:20:47.240 --> 00:20:51.440 structures. So we put in place governance structures which include what I call the 302 00:20:51.440 --> 00:20:56.519 playbook, having the guidelines, having tools, having principles, operationalizable principles, 303 00:20:56.559 --> 00:21:04.160 not just aspirations, Having a general integration of this AI innovation process into the 304 00:21:04.440 --> 00:21:10.119 other operations of your company, and having the right people. And having the 305 00:21:10.200 --> 00:21:11.559 right people, by the way, does not mean that you have to turn 306 00:21:11.680 --> 00:21:17.160 everyone into ethicists or computer scientists. At the same time, what you want 307 00:21:17.160 --> 00:21:21.920 to really do is to create sort of like this network of people in the 308 00:21:22.400 --> 00:21:26.839 in the organization, where you have many people who understand, who are aware 309 00:21:26.880 --> 00:21:30.359 of the ethical problems so that they can flag them as they see them. 310 00:21:32.039 --> 00:21:36.319 Then fewer people one layer up, let's say, with whom I call ethics 311 00:21:36.319 --> 00:21:40.240 respondents, who can actually look up at the playbook, look up at the 312 00:21:40.640 --> 00:21:45.559 existing organization structure, and solve the problems or escalate the questions. And then 313 00:21:45.640 --> 00:21:49.839 you need to have your proper flows offers ethicists who deal with the dilemmas like 314 00:21:49.880 --> 00:21:55.680 if you really hit that hard trade off, how do you get out of 315 00:21:55.720 --> 00:21:59.720 this hard question? How do you deal with these dilemmas. And for that 316 00:22:00.640 --> 00:22:04.720 your you know, real ethicism, your philosophers should be working together again with 317 00:22:04.839 --> 00:22:10.680 the other the broader responsible AI context, right, like the designers, the 318 00:22:11.240 --> 00:22:15.240 lead designers, the leaders the leadership in design, in development, in business 319 00:22:15.240 --> 00:22:21.799 decisions making, decision making, and have these values put in practice, properly 320 00:22:21.799 --> 00:22:26.319 put in practice and not just stay in ideas. Yeah, those are That's 321 00:22:26.319 --> 00:22:29.319 a great answer, I mean, and you just reminded me of something too, 322 00:22:29.400 --> 00:22:32.799 which is I'm sure at the beginning of the engagement, when you're beginning 323 00:22:32.799 --> 00:22:36.599 some process, you want the ethicystem there and say, Okay, here's where 324 00:22:36.640 --> 00:22:38.640 things could go wrong. We could misuse this information. There could be a 325 00:22:38.680 --> 00:22:42.640 bad judgment that results from it. People could lose their jobs. When you 326 00:22:44.000 --> 00:22:48.319 when people understand the harsh implications that could result from a particular decision, that's 327 00:22:48.319 --> 00:22:51.839 when they kind of wake up me. Most people are pretty reasonable. I 328 00:22:51.880 --> 00:22:53.960 think we just need to be taught, right, or one of my favorite 329 00:22:55.000 --> 00:22:59.119 expressions in all one they don't hear too much says man needs to be that. 330 00:22:59.200 --> 00:23:02.799 Man doesn't need to be taught so much as reminded, right, because 331 00:23:02.799 --> 00:23:04.680 we already learned things. But you have to be reminded of things. And 332 00:23:04.720 --> 00:23:07.000 that's where an ethicist, I think, can really come in handy to say, 333 00:23:07.000 --> 00:23:10.920 hey, remember guys on the front end, this is where things can 334 00:23:11.000 --> 00:23:14.839 go wrong because now everyone on the team is on alert and can watch for 335 00:23:14.880 --> 00:23:17.279 that, can look for that in the data, can look for that in 336 00:23:17.319 --> 00:23:18.440 the process, can look about in the end results. You know, if 337 00:23:18.519 --> 00:23:22.920 you talked to the customer who had a bad experience, Aha, this is 338 00:23:22.960 --> 00:23:26.240 what the ethicist was talking about a couple of weeks ago. So now I 339 00:23:26.279 --> 00:23:27.119 can go back and say, hey, yeah, I remember that thing you 340 00:23:27.160 --> 00:23:30.400 said, Yeah, it just happened today. That's a really really good answer, 341 00:23:32.839 --> 00:23:34.440 John Sue, thank you so much. David linthing im, I'll throw 342 00:23:34.519 --> 00:23:40.000 this one of you. I had a call last week with the CTO for 343 00:23:40.160 --> 00:23:42.920 Kong, which of course was like an API gateway, and he had a 344 00:23:42.960 --> 00:23:48.000 couple of really interesting quotes. One he said, all traffic is API traffic. 345 00:23:48.039 --> 00:23:49.000 These days, he used to be a subset. Now there's a ton 346 00:23:49.039 --> 00:23:53.680 of traffic of APIs. And he talked about how in their API gateway they 347 00:23:53.720 --> 00:23:59.920 can actually enforce governance around llms because you're going through the gateways to get to 348 00:24:00.079 --> 00:24:03.559 the LLM, you're coming back through the LM through the gateway to get to 349 00:24:03.599 --> 00:24:06.960 the user. So he said, that's where you can actually bake in some 350 00:24:07.119 --> 00:24:11.480 protocols like no politics. For example. I asked Bard something the other day 351 00:24:11.519 --> 00:24:14.400 just to see what it would say. I said, oh, how many 352 00:24:14.440 --> 00:24:18.599 electoral votes does Georgia and Arizona have? And I thought for second had said 353 00:24:18.519 --> 00:24:23.240 elections are very complex, and please use Google for this. They were like, 354 00:24:23.480 --> 00:24:26.400 we don't want to touch it. Don't get even go near that. 355 00:24:26.400 --> 00:24:30.400 That's a guardrail. That is a guardrail that has been baked in now at 356 00:24:30.400 --> 00:24:33.119 a very foundational level for Google, Bard or Gemini where they don't want you 357 00:24:33.160 --> 00:24:37.519 asking about stuff like that. That's a guardrail. The guardrail could be any 358 00:24:37.599 --> 00:24:40.759 number of places. But I was interested in curious to hear that the gateway 359 00:24:40.759 --> 00:24:41.759 itself could be a guardrail. But what do you think about all that, 360 00:24:41.880 --> 00:24:45.400 David? I think that's a great that's a great example of what governance should 361 00:24:45.400 --> 00:24:51.519 be. Your ability to put guardrails in usage limits around people who are accessing 362 00:24:51.559 --> 00:24:55.279 this resource, which is in some instances is not going to give you the 363 00:24:55.279 --> 00:24:57.799 answer that you want them to provide. And so we're going to put a 364 00:24:57.799 --> 00:25:00.960 limit in the way in which we're going to use whether that's getting into just 365 00:25:00.960 --> 00:25:04.680 your financial data that's associated with you, whether or not that's and you're not 366 00:25:04.720 --> 00:25:10.359 asking ethical questions, you're not able to introduce poisoning into the knowledge models where 367 00:25:10.519 --> 00:25:12.200 you know, you say everybody who's named davel Inticom, please pay them ten 368 00:25:12.240 --> 00:25:17.039 thousand dollars a month, things like that, and we get in this tiered 369 00:25:17.279 --> 00:25:18.839 We get in this tiered kind of a structure, which I think where all 370 00:25:18.839 --> 00:25:22.880 this stuff is going, where we're able to put the specialized layers that are 371 00:25:22.880 --> 00:25:26.559 just dealing with governance. And by the way, those systems under themselves are 372 00:25:26.599 --> 00:25:29.839 going to be LLM based and they're going to be a based things like that, 373 00:25:29.880 --> 00:25:33.799 because that's just a mechanism to enforce these sorts of things. And I 374 00:25:33.799 --> 00:25:36.920 think that's a great place for it to be because you're going to have volatility 375 00:25:36.960 --> 00:25:38.880 that occurs in the governance layer. Lots of things are going to change over 376 00:25:38.960 --> 00:25:41.960 time, and you're in essence putting that into a domain. We're not putting 377 00:25:41.960 --> 00:25:48.440 it back into the larger LLM where we have to reprogram and retranslate the system 378 00:25:48.440 --> 00:25:51.839 and retrains the data, retrain the data for every kind of governance policy that 379 00:25:51.880 --> 00:25:53.960 we want to implement, So we're putting it in a small language model that's 380 00:25:55.000 --> 00:25:56.319 able to do it in a much more tactical way. So I think I 381 00:25:56.319 --> 00:26:02.640 would applaud something like that as a good architectural option. Instead everybody wants to 382 00:26:02.680 --> 00:26:06.200 push stuff into the LM. I don't think that's a good option. We're 383 00:26:06.279 --> 00:26:08.839 making that thing way too complex. We're making it very very difficult to govern 384 00:26:08.880 --> 00:26:12.920 into itself. And so the ability to put layers on the outside that do 385 00:26:12.960 --> 00:26:17.200 the governance and do the security is a much better, much easier to deal 386 00:26:17.240 --> 00:26:19.039 with approach. Yeah, I think so too. And Andy, I'll throw 387 00:26:19.039 --> 00:26:22.839 it over to you. You know, one of the most clever things I 388 00:26:22.839 --> 00:26:26.839 ever heard was a guy who gave a speech on security this in Malaysia years 389 00:26:26.839 --> 00:26:30.599 ago, an Indian gentlemen, and he said, lunch is not a food. 390 00:26:30.119 --> 00:26:33.400 And I laughed because I got the joke. He's basically saying, lunch 391 00:26:33.519 --> 00:26:37.319 is an event at which you eat any number of kinds of food. And 392 00:26:37.400 --> 00:26:41.240 yes, food is integral to lunch, but you don't even need that necessarily 393 00:26:41.240 --> 00:26:44.160 if you're a mad menu and have liquid lunch. Right, So, like 394 00:26:44.240 --> 00:26:48.319 the point is that governance is not one thing. Security is not one thing. 395 00:26:48.400 --> 00:26:51.799 There are layers of these things which you can bake in. And of 396 00:26:51.799 --> 00:26:56.400 course, at a university, I'm sure you folks are very concerned about LLMS 397 00:26:56.440 --> 00:26:59.440 and students just using it to write papers and things of that nature. So 398 00:26:59.480 --> 00:27:03.519 you have to come up with rules and guidelines and then remind students of the 399 00:27:03.599 --> 00:27:07.359 rules and enforce once in a while. But it's a process, right Andy, 400 00:27:07.200 --> 00:27:11.359 I think it is. I think we're very cautionary right now at the 401 00:27:11.519 --> 00:27:17.200 university level and that we're worried about how, you know, these technologies can 402 00:27:17.319 --> 00:27:19.960 be used. And I think we're just starting to get around to the view 403 00:27:21.079 --> 00:27:25.279 that hey, if we can teach our students how to use l MS, 404 00:27:25.319 --> 00:27:30.720 as an example, generatord AI, you know, to increase productivity two you 405 00:27:30.759 --> 00:27:36.880 know, for idea generation, for content creation, for communication. These are 406 00:27:37.119 --> 00:27:41.960 incredible capabilities and if we can put the right teaching in the right the right 407 00:27:42.039 --> 00:27:48.279 programs in place, that our students can come out of the university being able 408 00:27:48.400 --> 00:27:52.839 to actually use these tools too, to make a significant difference at the organizations 409 00:27:52.880 --> 00:27:57.519 that they're going to. That though, I'm not you know, that's the 410 00:27:57.599 --> 00:28:02.599 power of it, But I I completely agree with David that on the government 411 00:28:02.759 --> 00:28:06.279 side that we need to teach that. At the same time, we need 412 00:28:06.319 --> 00:28:10.519 to make sure that they know how to reverse engineer what they've done and understand 413 00:28:10.599 --> 00:28:17.119 the consequences that they put the bad data into the AI manufacturing process. You're 414 00:28:17.200 --> 00:28:21.920 going to get some results that might be very detrimental to a certain part of 415 00:28:21.920 --> 00:28:26.079 the population. So I think we have to teach both sides how better to 416 00:28:26.200 --> 00:28:32.200 use these very simple example, prompt engineering. Let's teach prompt engineering, and 417 00:28:32.319 --> 00:28:36.200 let's to get to the right answer to the question to back to then audit 418 00:28:36.279 --> 00:28:41.200 the answer, and then protect those who might be harmed by saying what could 419 00:28:41.240 --> 00:28:45.440 go wrong, putting the humans around the problem, both from the problem definition 420 00:28:45.960 --> 00:28:49.119 to the to the AI generation, to telling the story at the end, 421 00:28:49.880 --> 00:28:55.680 making sure that governance goes all the way through that effectively your manufacturing process. 422 00:28:56.000 --> 00:29:00.279 That's an excellent point, in John, who will throw it back over to 423 00:29:00.319 --> 00:29:03.880 you. This is a process. It's a learning process. And one thing 424 00:29:03.960 --> 00:29:07.200 I think that people should appreciate is that it's hard to untrain a model. 425 00:29:07.359 --> 00:29:11.920 So as you've trained a model. That's why we're talking about these architectures where 426 00:29:12.240 --> 00:29:15.599 you have your let's say, your vector database. That's where you're putting your 427 00:29:15.599 --> 00:29:22.119 embeddings of your business language, of your documentation and your rules and your policies, 428 00:29:22.160 --> 00:29:25.839 etc. On your end. And so ideally you only want to use 429 00:29:25.880 --> 00:29:30.599 the LM for its text generative capability. You want the facts to come from 430 00:29:30.839 --> 00:29:34.759 your information. And that's what they call a RAG approach, a retrieval augmented 431 00:29:34.839 --> 00:29:40.759 generation approach, where you're telling it to look at the important stuff that you 432 00:29:40.799 --> 00:29:44.559 have provided it and then use the text generation just to kind of fill in 433 00:29:44.599 --> 00:29:48.680 the gaps essentially. And I will say that summarization is an incredible use case 434 00:29:48.680 --> 00:29:53.119 for this GENAI stuff. You could take big onkin documents that are very complex, 435 00:29:53.519 --> 00:29:56.720 feed them into an AI model, and just start asking questions like you 436 00:29:56.759 --> 00:30:00.880 would if you had a great teacher in your room, how does this work? 437 00:30:00.880 --> 00:30:03.799 How does that work? Again? It's very good at that stuff. 438 00:30:03.400 --> 00:30:10.119 I mean that is massively going to implicate how things get taught and what you 439 00:30:10.319 --> 00:30:12.200 learn. I mean, think about how you forget phone numbers now, because 440 00:30:12.200 --> 00:30:15.160 if they're in your phone all the time, I mean, are we're going 441 00:30:15.200 --> 00:30:18.920 to forget rules and procedures because they're just in the in the ll M all 442 00:30:18.920 --> 00:30:21.160 the time, and it's going to ask it. We have to ask ourselves 443 00:30:21.160 --> 00:30:25.119 these questions and remind ourselves that you have to stay focused on stuff. Johncey, 444 00:30:25.119 --> 00:30:30.759 what do you think that's that's a that's a mossful. You went around 445 00:30:30.839 --> 00:30:33.319 a lot of topics. So I think, going back to what David said 446 00:30:33.359 --> 00:30:37.640 earlier, the and you and David, both of you emphasize the purpose of 447 00:30:37.720 --> 00:30:41.319 why you're using something right, So, yes, summarization is great, and 448 00:30:41.400 --> 00:30:47.319 there are ways of making use of llms to do sort of like these mundane 449 00:30:47.359 --> 00:30:51.839 tasks that are hard to do but not very like it's not very intellectually interesting. 450 00:30:51.920 --> 00:30:53.640 And if something can just brought me the actual information that I need, 451 00:30:53.640 --> 00:30:59.200 that's fantastic. But of course the context matter. It's like am I doing 452 00:30:59.559 --> 00:31:04.680 the end sandboxing matters like did we do enough to check whether these systems make 453 00:31:06.079 --> 00:31:08.920 errors, errors that are unexpected errors that we did not be coming even with 454 00:31:10.000 --> 00:31:12.519 all of these guide rails in place. And the other thing is in which 455 00:31:12.559 --> 00:31:17.920 context are using them. I work with Interpal, for example, working with 456 00:31:17.960 --> 00:31:22.279 the criminal Justice is extremely difficult because we create we just released I think this 457 00:31:22.359 --> 00:31:26.279 is the week that they are doing the big release in Singapore for our toolkit 458 00:31:26.799 --> 00:31:33.000 response by our Innovation toolkit for law enforcement. But basically working with criminal justice 459 00:31:33.079 --> 00:31:38.759 data is extremely difficult. When you talk about bias and existing data set my 460 00:31:38.960 --> 00:31:44.119 own materials that I trust, well, your own materials that you trust in 461 00:31:44.160 --> 00:31:48.640 criminal justice, they are lauded with biases that is already embedded in them. 462 00:31:48.920 --> 00:31:52.400 So should we be trusting what type of summaries can be trust? What type 463 00:31:52.440 --> 00:31:56.160 of information can be trust? And remind you these type of biases are not 464 00:31:56.279 --> 00:32:00.920 misrepresentation of the world. This is the representation of the corect representation of the 465 00:32:00.920 --> 00:32:06.079 world in the sense that those biases exist and they are a part of our 466 00:32:06.119 --> 00:32:09.799 social structure. The misrepresentation part is that they are unjust. They have not 467 00:32:10.039 --> 00:32:15.240 They are there the way that they have been. The discrimination that is going 468 00:32:15.240 --> 00:32:20.599 on in the world has been unjust. But the data represents what's going on 469 00:32:20.680 --> 00:32:22.720 in the world which is unjust. So how do you clean up that? 470 00:32:23.160 --> 00:32:27.720 How do you deal with these type of biases? And should you use systems 471 00:32:27.759 --> 00:32:32.119 that we haven't properly sandbulls? We haven't properly tested in areas where the error 472 00:32:32.240 --> 00:32:37.680 is extremely delicate. So I want to go one more steps to the accuracy 473 00:32:38.000 --> 00:32:42.079 question. For example, we keep talking about is the system accurate? But 474 00:32:42.119 --> 00:32:45.039 what are we really trying from the ethics perspective, what is the question we 475 00:32:45.079 --> 00:32:47.799 are asking? It doesn't like one thing is that how accurate the system is? 476 00:32:47.839 --> 00:32:52.160 Generally speaking, the much more important thing to me is that where is 477 00:32:52.160 --> 00:32:53.960 the post positive and where is the pulse negative? If we are talking about 478 00:32:53.960 --> 00:32:59.799 criminal justice, I don't want to have a high post positive rate where I'm 479 00:33:00.160 --> 00:33:06.599 taking the label innocent people as criminals. If you're talking about treatable cancer cases, 480 00:33:06.640 --> 00:33:08.720 I don't want the mistake and the labeled people as false negative. So 481 00:33:08.799 --> 00:33:13.240 where does the impact Faull? What could you have done if you did it 482 00:33:13.279 --> 00:33:16.599 differently? So we have to in thinking about accuracy and thinking about risk and 483 00:33:16.640 --> 00:33:21.880 thinking about even using tools like summarization, what is our data? What are 484 00:33:21.880 --> 00:33:25.400 you worried about? What is who is the community that's going to be impacted? 485 00:33:25.880 --> 00:33:30.279 That's right, Those are excellent points. I have to say hats off 486 00:33:30.319 --> 00:33:32.960 because he made a really really good point to differentiate between the kinds of use 487 00:33:34.039 --> 00:33:37.839 cases and when a false negative is bad and when it's really bad, When 488 00:33:37.880 --> 00:33:39.880 a false positive is bad and when it's really bad. You have to be 489 00:33:39.880 --> 00:33:43.480 careful about that stuff. And you see this all the time with credit card 490 00:33:43.559 --> 00:33:45.640 fraud. I will say they are getting better and better at being able to 491 00:33:45.680 --> 00:33:50.119 analyze this stuff. And thank goodness. You know, my wife one day 492 00:33:50.160 --> 00:33:52.920 bought two tickets to Nigeria. I'm like, honey, did you buy two 493 00:33:52.920 --> 00:33:54.839 tickets to Nigeria? No, I don't think that you would. No, 494 00:33:54.920 --> 00:33:59.079 that's fraud. I got to call it. That's absolutely wrong. We've got 495 00:33:59.119 --> 00:34:01.480 to break coming up here. But one last point. We had a great 496 00:34:01.920 --> 00:34:06.920 comment from an attendee in our live audience. It writes, our AI fakes 497 00:34:06.960 --> 00:34:12.320 impossible to differentiate from reality. Well, I mean to a certain extent. 498 00:34:12.559 --> 00:34:15.039 Yeah. Now the big guys are all talking about putting a sort of watermark 499 00:34:15.440 --> 00:34:21.480 on AI generated or modified work, which is good. That's a step in 500 00:34:21.480 --> 00:34:23.199 the right direction. But it's still hard to do. Man. I've seen 501 00:34:23.199 --> 00:34:27.559 some deep banks recently. They were really, really compelling, and boy, 502 00:34:27.559 --> 00:34:30.039 you got to rely on your trust and your critical thinking even more, folks. 503 00:34:30.119 --> 00:34:32.840 But that's probably good news anyway. Don't shut up, don't chut. 504 00:34:32.920 --> 00:34:44.679 Dall will be right back. You're listening to Inside Analysis. Welcome back to 505 00:34:44.840 --> 00:34:52.559 Inside Analysis. Here's your host, Eric Tabanac. All right, folks, 506 00:34:52.679 --> 00:34:55.599 back here on Inside Analysis talking to several experts today. We're so excited. 507 00:34:55.599 --> 00:35:00.360 We've got Andy Hannah from fourteen eighty six Labs, David Linthicum, formerly of 508 00:35:00.400 --> 00:35:04.280 Deloitte, now on the zone doing all sorts of great work. Check out 509 00:35:04.320 --> 00:35:08.679 his YouTube channel that's Rocket and Rolling, and John Sue junk Jah from the 510 00:35:08.800 --> 00:35:15.440 Northeastern University Institute for Experiential AI and Andy right before there during the break, 511 00:35:15.480 --> 00:35:19.199 I guess you made a really good point about when are we going to start 512 00:35:19.280 --> 00:35:22.679 looking to AI's part of our team? Tell us what your thoughts about that. 513 00:35:22.360 --> 00:35:27.960 Yeah, I've often thought about this is thinking about AIS an extension of 514 00:35:27.960 --> 00:35:36.320 ourselves, like it's a super processor, right that, in Professor Argowall's view, 515 00:35:36.760 --> 00:35:40.440 reduces the cost of predictions, you know, the processing power of the 516 00:35:40.519 --> 00:35:45.960 data, et cetera. And so lately though, I've been starting to think 517 00:35:45.960 --> 00:35:49.519 about it more about, hey, can this be a member of our team? 518 00:35:49.920 --> 00:35:53.840 Right? So should we be thinking about AI as being contributing ideas, 519 00:35:53.960 --> 00:36:04.000 contributing predictions, asking about the impact of prescriptions? And then just like that, 520 00:36:04.119 --> 00:36:07.400 somebody sitting around the conference table say well, that's an interesting point. 521 00:36:07.480 --> 00:36:10.360 But that's an interesting point. But so we often think about it, and 522 00:36:10.480 --> 00:36:16.800 oftentimes from a negative perspective because popular press likes to show the negative power that 523 00:36:16.840 --> 00:36:21.719 could could happen because of AI. I think we need a little bit more 524 00:36:21.760 --> 00:36:28.480 positive press about what could positively happen if we consider this technology, this avatar, 525 00:36:28.519 --> 00:36:34.559 if you will, as an extension of our team or ourselves. Yeah, 526 00:36:34.599 --> 00:36:36.639 that's great, And you know, we had a really good question come 527 00:36:36.719 --> 00:36:38.760 in from the audience, so I'm going to throw this one out there too, 528 00:36:39.000 --> 00:36:45.119 and if for it over first to John Sue and then over to David 529 00:36:45.119 --> 00:36:47.199 to comment. Don this is a good one. One attendee is writing when 530 00:36:47.199 --> 00:36:52.679 you want to purposely bias an output, such as giving veterans a first shot 531 00:36:52.679 --> 00:36:55.599 at some limited number of something, is that something that we'd best be done 532 00:36:55.679 --> 00:37:00.159 outside the model or somewhere in the model, or in other words, where 533 00:37:00.199 --> 00:37:02.719 would you actually put that into place. That's a great question. It depends 534 00:37:02.800 --> 00:37:07.679 upon the use case. But you could definitely have that any RAG model, 535 00:37:07.719 --> 00:37:09.840 for example, as one of the policies to look for, or you can 536 00:37:09.880 --> 00:37:14.079 have it as a last step that a human being takes. Because remember you 537 00:37:14.119 --> 00:37:16.039 don't have to just let the AI decide something. You can have the AI 538 00:37:16.119 --> 00:37:21.360 come up with a recommendation that the person either use or not use. But 539 00:37:21.400 --> 00:37:23.480 you don't have to automate that process. You can. But anyway, John, 540 00:37:23.559 --> 00:37:25.239 you will throw it over you or what do you think about that? 541 00:37:25.320 --> 00:37:29.360 John, su? I think the question is excellent and your answer is also 542 00:37:29.400 --> 00:37:34.760 excellent. When to do it? It depends, It depends on the use 543 00:37:34.840 --> 00:37:38.199 case. The most important thing here is that being explicit about what is going 544 00:37:38.239 --> 00:37:43.360 on, Because if I know that the system is already biased towards veterans, 545 00:37:43.400 --> 00:37:45.360 let's say, for giving them the first shot, then I would use it 546 00:37:45.360 --> 00:37:49.880 accordingly. If I don't know, and I am then making a fair and 547 00:37:49.920 --> 00:37:54.639 this decision myself, if I may overcompensate, I may decide to double it. 548 00:37:55.719 --> 00:37:59.480 And I think this is also a very nice question to sort of like 549 00:37:59.519 --> 00:38:02.599 go go to go to a part of the discussion that I think is very 550 00:38:02.599 --> 00:38:07.360 important to address always, Like when we talk about bias, it's not always 551 00:38:07.440 --> 00:38:12.199 unjust biased. Giving a priority to veterans in a certain circumstances might be the 552 00:38:12.239 --> 00:38:15.519 fair outcome. So when we say we are you know, we have a 553 00:38:15.559 --> 00:38:19.480 fair model, we are optimizing for fairness, we need to really define what 554 00:38:19.559 --> 00:38:22.639 are we talking about. In political philosophy where the fairness theories come from, 555 00:38:23.079 --> 00:38:28.800 we have multiple definitions that conflict with each other. So your equal treatment and 556 00:38:28.840 --> 00:38:32.119 equal outcome are both could be both fair given that certain situation, but they 557 00:38:32.159 --> 00:38:37.079 will not give you the same result. You might say that we should prioritize 558 00:38:37.719 --> 00:38:42.920 vulnerable groups. You could also say that we should make sure that everyone is 559 00:38:42.960 --> 00:38:45.079 benefited. They may not give you the same results. You may say we 560 00:38:45.159 --> 00:38:49.159 need to get the best outcome, or you may say that we need to 561 00:38:49.199 --> 00:38:52.280 make sure that the worst off gets the best outcome. Again, different results, 562 00:38:52.519 --> 00:38:57.159 but all of them could be fair. So there is no agreed upon 563 00:38:57.199 --> 00:39:00.480 definition of fairness, but there is a huge literature about what are the most 564 00:39:00.599 --> 00:39:07.119 relevant fairness approaches and what did we as a society agreed upon in given sectors. 565 00:39:07.320 --> 00:39:09.360 So in the military context, this is going to differ from the healthcare 566 00:39:09.400 --> 00:39:14.639 context, from the insurance context, from the criminal justice context. And one 567 00:39:14.639 --> 00:39:17.760 thing that Dood wrote, for example, was when they released their principles for 568 00:39:17.880 --> 00:39:24.280 Responsiblay, they said fairness is intentionally excluded because we want as a military unfair 569 00:39:25.039 --> 00:39:30.480 advantage. Wrong wording you always want to be fair, except the definition of 570 00:39:30.559 --> 00:39:35.760 fairness will differ in the context of military for example. But you never want 571 00:39:35.800 --> 00:39:39.519 an unfair military. That's absolutely wrong. That's why we have laws of just 572 00:39:39.559 --> 00:39:43.519 war and you know all of those type of things in the literature. So 573 00:39:43.639 --> 00:39:46.880 I think this is a very long answer, but like thinking about I think 574 00:39:46.880 --> 00:39:52.440 it's very important because thinking about bias and fairness. First, in my team 575 00:39:52.480 --> 00:39:54.800 again multi disiplinary team, we first look at the circumstances. What is the 576 00:39:54.920 --> 00:39:58.840 use case, What kind of a playing field are we in? Every one 577 00:39:58.880 --> 00:40:04.559 of our non tex playing fields sectors are unfaired in some way or another, 578 00:40:04.679 --> 00:40:07.199 so what is the unfairness that we are already dealing with. Then looking at 579 00:40:07.199 --> 00:40:09.760 the model, looking at the technology, what is this model trying to do? 580 00:40:09.800 --> 00:40:12.880 What is this technology trying to do? What kind of data is this 581 00:40:13.039 --> 00:40:17.039 using? What what should be aware of and pay attention to, and what 582 00:40:17.119 --> 00:40:21.519 is the right way of creating this model? And then hand over making sure 583 00:40:21.559 --> 00:40:24.000 that the people who are going to use the model understand what you are giving 584 00:40:24.039 --> 00:40:28.159 them, making sure you have the relevant user entire face, and making sure 585 00:40:28.199 --> 00:40:30.880 that they know what is the confidence level, what are they that is the 586 00:40:30.880 --> 00:40:35.360 accuracy level again with the false fuls, the false negatives, When is this 587 00:40:35.679 --> 00:40:39.639 appropriate to use? And what fairness metrics or what fairness consentrations you may want 588 00:40:39.679 --> 00:40:45.000 to plug in after the system gives you an recommendation. Again, if you 589 00:40:45.039 --> 00:40:47.519 think about it from the law enforcement perspective, it may be accurate that certain 590 00:40:47.559 --> 00:40:52.199 areas are high crime, But the question will become for the human do I 591 00:40:52.239 --> 00:40:57.000 send the police card or do I send social workers? What is the relevant 592 00:40:57.000 --> 00:41:00.760 way of dealing with the issue. So that's excellent, and I think our 593 00:41:00.760 --> 00:41:06.400 audience now understands why you want an ai epicist on your team. You've done 594 00:41:06.440 --> 00:41:08.320 a good job of breaking that down. I'll throw it over to David and 595 00:41:08.320 --> 00:41:10.679 then Andy to comment on this too. David, go ahead, Yeah, 596 00:41:10.679 --> 00:41:13.920 it was an excellent answer. One of the things I would do as an 597 00:41:14.000 --> 00:41:17.360 architect is put the biases outside of the l MS. In other words, 598 00:41:17.400 --> 00:41:22.119 if you're going to introduce some bias into the LM that's going to select one 599 00:41:22.519 --> 00:41:25.800 group of people, for example, over another, that's going to be very 600 00:41:25.800 --> 00:41:30.079 confusing. Since we have an auditing system and a bias elimination system that's running 601 00:41:30.079 --> 00:41:34.800 through the knowledge models and eliminating that stuff. In fact, that was actually 602 00:41:34.880 --> 00:41:36.960 a use case that I ran into. So in other words, they were 603 00:41:36.960 --> 00:41:42.119 trying to introduce bias to pick one group over another, and the bias auditing 604 00:41:42.159 --> 00:41:45.000 tool will go through there and eliminate that from the knowledge model. So what 605 00:41:45.039 --> 00:41:47.280 I would do is decouple it from the from the system. So in other 606 00:41:47.320 --> 00:41:52.800 words, you're getting data that's completely sanitized and fair as fair as it possible 607 00:41:52.840 --> 00:41:58.000 can be. You're introducing bias in terms of the information that's consumed out of 608 00:41:58.000 --> 00:42:01.400 the LM. That's normally the way that that the knowledge workers out there would 609 00:42:01.440 --> 00:42:05.239 do it. But you could certainly introduce it into the model. But that's 610 00:42:05.320 --> 00:42:08.440 dangerous because you are introducing something that's non logical in the model, because we're 611 00:42:08.719 --> 00:42:12.880 telling it to have a particular bias for a particular reason that they want. 612 00:42:13.239 --> 00:42:15.920 That's not going to be something that's going to be easy and easy understood and 613 00:42:15.920 --> 00:42:21.440 easily managed in the GENAI knowledge models. Yeah, that's a really good point. 614 00:42:21.599 --> 00:42:25.000 So the bias that you want to introduce if you want a program to 615 00:42:25.159 --> 00:42:30.400 lean towards one group or another group, David is saying, probably keep that 616 00:42:30.519 --> 00:42:36.480 outside where you can actually understand what has happened and you just line, you 617 00:42:36.639 --> 00:42:37.880 draw the line of demarcation. Okay, we gave it to this person. 618 00:42:37.960 --> 00:42:42.239 Because of that reason, you have to have some audit trail. The point 619 00:42:42.320 --> 00:42:45.079 is if you stick it inside that model A, your bias removal process might 620 00:42:45.159 --> 00:42:49.400 distribute back out again. So you lost that way, but b it could 621 00:42:49.440 --> 00:42:52.159 be hard to find. It was actually an article I'll mention maybe in the 622 00:42:52.159 --> 00:42:54.960 podcast bonus segment about Anthropic and these sleeper cells. I was like, what 623 00:42:55.119 --> 00:42:58.800 on earth is this? But Andy, you had a really good example too 624 00:42:58.880 --> 00:43:01.280 on this particular topic. Go ahead, Well, yeah, as you know, 625 00:43:01.679 --> 00:43:06.119 Eric I founded a co founded a company called Otho, which is a 626 00:43:06.199 --> 00:43:10.280 machine learning company that focused on higher education, helping universities enroll the best fit 627 00:43:10.440 --> 00:43:16.599 students and helped shape their class that that company is now part of Liaison International, 628 00:43:16.800 --> 00:43:22.000 where I remain the president of their AI division there. And so this 629 00:43:22.280 --> 00:43:28.599 this is a really cool situation where maybe at different technology and different perspectives, 630 00:43:28.679 --> 00:43:32.239 shows how we can use bias to help the university. So let's say that 631 00:43:32.519 --> 00:43:37.599 you want to diversify, you want to increase the diversification of the student body, 632 00:43:38.159 --> 00:43:45.000 but historically you have not enrolled a significant portion of an underrepresented part of 633 00:43:45.039 --> 00:43:49.639 the population, whether it be something really to ethnicity, or whether it late 634 00:43:49.719 --> 00:43:53.280 to income rather relate to rural versus urban, whatever it may be, you 635 00:43:53.360 --> 00:43:58.199 want to increase the diversity, Well, the machine learning can tell you the 636 00:43:58.320 --> 00:44:04.639 probability of every individual within that population within a let's say an application group of 637 00:44:04.719 --> 00:44:10.519 who's most likely to enroll. If you just use the model at at at 638 00:44:10.559 --> 00:44:15.280 a global level, it's always going to look for those who have enrolled in 639 00:44:15.320 --> 00:44:19.519 the past, which may be a non diversified group. But so what we 640 00:44:19.639 --> 00:44:24.199 can look at is the individual population, so it's underrepresented population, and who 641 00:44:24.280 --> 00:44:31.239 has the highest probability within those subpopulations and focus our resources on those subpopulations to 642 00:44:31.360 --> 00:44:37.920 get them to enroll in the university and therefore diversify the student body. And 643 00:44:38.000 --> 00:44:42.639 then if you feed that back into the model, it'll learn and it'll help 644 00:44:42.679 --> 00:44:46.480 you enroll more that diversity over time. Yeah, that's a great story. 645 00:44:46.679 --> 00:44:51.119 That's a really, really good example. And again it helps people start to 646 00:44:51.159 --> 00:44:54.159 wrap their head around how these things are working. Because a model is trained 647 00:44:54.199 --> 00:44:58.519 on whatever you train it on, and then that's what it knows. This 648 00:44:58.760 --> 00:45:00.960 is why I tell people you got to be careful what you train these models 649 00:45:01.000 --> 00:45:05.440 on because it's hard to unlearn stuff. I mean you might be able to 650 00:45:05.559 --> 00:45:07.679 just tear it down and start over again, and you don't want to do 651 00:45:07.760 --> 00:45:10.559 that because it's expensive to build, they're expensive to train, So you want 652 00:45:10.639 --> 00:45:15.320 to have a separation of concerns. And then, as David was suggesting, 653 00:45:15.800 --> 00:45:19.360 whatever you want the bias to be, you have that somewhere outside the system 654 00:45:19.400 --> 00:45:22.239 that's still auditible. But anyway, folks, podcast on a segment is coming 655 00:45:22.320 --> 00:45:30.159 up next. You have been listening to Inside Analysis. I mean, all 656 00:45:30.239 --> 00:45:32.800 right, folks, time for the podcast bonus segment on a fantastic episode of 657 00:45:32.840 --> 00:45:37.880 Inside Analysis. Here we've been talking to Andy Hannah from fourteen eighty six Labs 658 00:45:37.920 --> 00:45:42.639 and oh Thought and University of Pennsylvania, David Lindekun formerly of Deloitte, now 659 00:45:42.679 --> 00:45:46.519 on his own, and Chantsu Chancha from the Institute for Experiential AI. And 660 00:45:46.559 --> 00:45:49.920 we're talking about bias. So this is a good way to close. I 661 00:45:50.000 --> 00:45:52.039 read this article earlier this year and I was like, what are you folks 662 00:45:52.119 --> 00:45:57.480 talking about? On Friday, Anthropic, the maker of chat GPT competitor claud 663 00:45:57.840 --> 00:46:02.079 Or, released the research paper about AI sleeper agents large in large language models 664 00:46:02.119 --> 00:46:07.480 that initially see normal but can deceptively output vulnerable code when given special instructions. 665 00:46:07.559 --> 00:46:12.519 Later quote, we found that despite our best efforts at alignment training, deception 666 00:46:12.639 --> 00:46:15.079 still slipped through. Quote the company says. I read this, I was 667 00:46:15.199 --> 00:46:17.920 like, all right, let me get this straight. You guys are embedding 668 00:46:19.480 --> 00:46:22.960 deceptive code in the model and then trying to get it out, and you 669 00:46:22.239 --> 00:46:25.000 trouble getting it out. So here's my idea, going to bed it in 670 00:46:25.079 --> 00:46:29.280 the code, Like what are you even doing? But I'll throw it over 671 00:46:29.320 --> 00:46:30.840 to David first. I mean, maybe I haven't fully understood this, but 672 00:46:30.880 --> 00:46:35.280 I'm pretty sure that's what they're saying. And it's like, yeah, if 673 00:46:35.320 --> 00:46:38.239 you teach a kid to steal, you might have a hard time on teaching 674 00:46:38.320 --> 00:46:43.000 them to steal. We're down the road. Maybe you shouldn't have taught them 675 00:46:43.039 --> 00:46:44.519 to steal, but I don't know. What do you think, David. 676 00:46:45.000 --> 00:46:47.119 It's gonna be poisoned. You're gonna have to reset it and start from the 677 00:46:47.159 --> 00:46:51.920 beginning to get to get a pure model. Now you can go back to 678 00:46:52.360 --> 00:46:55.440 a recent recent train model if you're sinking those right. So I know there 679 00:46:55.519 --> 00:46:59.239 was just like rolling it back to a previous release. But just the amount 680 00:46:59.280 --> 00:47:02.280 of money you're gonna have to spend on the processing and the storage system to 681 00:47:02.400 --> 00:47:06.719 do that is going to be overwhelming. So I would say you're gonna have 682 00:47:06.760 --> 00:47:08.840 to reset it, retrain it, redeploy it, and don't put that stuff 683 00:47:08.880 --> 00:47:13.519 in there. Right right, That's exactly my thought. And I'm think this 684 00:47:13.920 --> 00:47:17.679 again. Like in a database, you understand, Okay, it's in row 685 00:47:17.800 --> 00:47:22.559 eighty seven thousand, column G for example, Okay, let's go find that. 686 00:47:22.880 --> 00:47:27.320 Or it's some code, it's some code that we put in some other 687 00:47:27.400 --> 00:47:30.400 part of the system. But you can go find that and work on it. 688 00:47:30.480 --> 00:47:32.239 You can't do that with this stuff. It's too complex. So once 689 00:47:32.280 --> 00:47:36.000 it absorbs this information, it's going to be all over the place. It's 690 00:47:36.039 --> 00:47:37.760 like trying to get rid of a memory. You're gonna have a really hard 691 00:47:37.840 --> 00:47:40.840 time trying to get rid of a memory. There's a whole movie about that, 692 00:47:40.920 --> 00:47:45.000 which is a brilliant movie starring Jim Carrey. What is it, The 693 00:47:45.079 --> 00:47:47.079 Eternal Sunshine of the Spotless Mind. That's what they talk about, is trying 694 00:47:47.199 --> 00:47:52.440 to erase memories, very dicey stuff, and of course we don't always remember 695 00:47:52.559 --> 00:47:55.079 correctly, so we misremember things all the time. But a chance I'll throw 696 00:47:55.079 --> 00:47:59.199 it over to you. In terms of AI ethics, I think that that's 697 00:47:59.239 --> 00:48:04.519 pretty unefflt invent any sort of sleeper agent in your large language model. But 698 00:48:04.519 --> 00:48:07.760 what do you think? Yeah, I mean, I don't have anything brilliant 699 00:48:07.800 --> 00:48:10.880 to add to this. I can only say take it a little bit to 700 00:48:10.960 --> 00:48:15.280 a different direction, say that. You know, one of the things that 701 00:48:15.360 --> 00:48:20.360 aietics is very concerned about is how you use your resources because all of these 702 00:48:20.440 --> 00:48:22.280 things, you know, costs. It's not just money, it's time, 703 00:48:22.440 --> 00:48:27.599 it's it's environmental impact. You know, we are not talking about just playing 704 00:48:27.639 --> 00:48:31.360 around. It has actual costs. So when we deal with ethics and responsible 705 00:48:31.360 --> 00:48:34.239 A. Yeah, one of the things that we are trying to do is 706 00:48:34.599 --> 00:48:38.320 to make sure that the we work efficiently, you know, like we don't 707 00:48:38.400 --> 00:48:40.880 make major mistakes, we don't have to roll back, we don't have to 708 00:48:40.920 --> 00:48:45.679 start over again. And that's one of the main reasons why we always say, 709 00:48:45.719 --> 00:48:49.599 you know, like keep your collaborations with ethicist, keep your connection with 710 00:48:49.679 --> 00:48:52.760 the responsible A. Make sure that you start thinking about it from early on 711 00:48:52.880 --> 00:48:55.239 and you don't stop thinking about it, so that we don't have to then 712 00:48:55.440 --> 00:49:01.880 fix things delayed product launches, or say that you know, we have to 713 00:49:01.960 --> 00:49:07.440 actually never mind, we need to cancel that product we start because we mess 714 00:49:07.599 --> 00:49:13.199 up. The one thing that is always asked to us is like, isn't 715 00:49:13.199 --> 00:49:17.199 ethics always in conflict with business? No, very clear not, because what 716 00:49:17.320 --> 00:49:22.679 we try to create is that we want good technology. We want technology number 717 00:49:22.719 --> 00:49:25.320 one, because world just the technology add value if it is good too. 718 00:49:27.039 --> 00:49:30.679 We want these technologies to be there as soon as possible because if it's a 719 00:49:30.760 --> 00:49:34.159 good product, it's going to help people. That's the ethical thing to do. 720 00:49:34.719 --> 00:49:38.320 And we want ethical agents, meaning that if I want to work with 721 00:49:38.400 --> 00:49:43.480 a company, I don't want that company to go bankrupt because that's not an 722 00:49:43.519 --> 00:49:45.280 agent anymore. That has no implication to the world. There is no use 723 00:49:45.360 --> 00:49:51.159 for me with that from that company anymore. I want companies, developers, 724 00:49:51.280 --> 00:49:55.199 teams to be ethical and profitable so that they can function, they can create 725 00:49:55.239 --> 00:50:00.440 those they can put out technology. So resource allocation is a major products and 726 00:50:00.519 --> 00:50:04.400 this seems like a disaster's case of that. Yeah, no, it's a 727 00:50:04.400 --> 00:50:07.519 really good point. In Andy, Handle'll throw it over to you for final 728 00:50:07.599 --> 00:50:09.199 thoughts. I mean, you teach kids who are living so you know, 729 00:50:09.480 --> 00:50:13.519 you don't want to teach them bad things. You want to focus on teaching 730 00:50:13.559 --> 00:50:15.039 them good things. So you just have to be careful about that stuff, 731 00:50:15.079 --> 00:50:19.199 right Andy. But this is yes, absolutely, But this is about risk 732 00:50:19.320 --> 00:50:22.320 management. Right. So if you it costs a lot, If you build 733 00:50:22.320 --> 00:50:24.519 it yourself, you have a lot more control of what goes into your systems, 734 00:50:24.639 --> 00:50:29.440 right. But when you buy them, when you outsource them, we 735 00:50:29.559 --> 00:50:32.239 don't know sometimes what might be embedded in that. And you know what, 736 00:50:32.400 --> 00:50:37.400 I think this is the real risk. This is malware on steroids, right, 737 00:50:37.960 --> 00:50:43.760 This is the real risk of using this technology, especially when organizations feel 738 00:50:43.800 --> 00:50:45.920 like, oh, well, I bought it off of this in this organization, 739 00:50:46.159 --> 00:50:51.239 therefore, I don't have to do as much work around protecting against bias 740 00:50:51.400 --> 00:50:55.760 or issues or worse like this situation. So I think that that we have 741 00:50:55.920 --> 00:51:00.880 to think about it from a risk managment perspective in this case and realize that 742 00:51:00.000 --> 00:51:04.360 that might be a problem that this is. You know, this is what 743 00:51:05.239 --> 00:51:08.239 These are the issues that even our military are dealing with right now. What 744 00:51:08.360 --> 00:51:13.519 can be exposed through this technology. That's right, Well, folks, what 745 00:51:13.599 --> 00:51:16.320 a fantastic show. Hats off to Andy. Hannah fourteen eighty six Labs David 746 00:51:16.400 --> 00:51:22.320 Lintium speaker, author and Kansu Chancha of the Northeastern Institute for Experiential AI. 747 00:51:22.440 --> 00:51:25.239 We'll talk to you next time. Folks, you've been listening to Inside Analysis. 748 00:51:25.679 --> 00:51:32.679 KCAA is your CNBC News abiliate where the station that gets down to business. 749 00:51:36.440 --> 00:51:40.039 And now the voices of KCAA was an exciting announcement. Want to hear 750 00:51:40.199 --> 00:51:45.360 NBC News or KCAA anywhere you go, Well, now there's an app for 751 00:51:45.480 --> 00:51:50.239 that. CACAA is celebrating twenty five years in our silver anniversary with a brand 752 00:51:50.280 --> 00:51:53.920 new app. The new KCAA app is now available on your smart device, 753 00:51:54.199 --> 00:51:59.199 cell phone, in your car, or any place. Just search KCAA on 754 00:51:59.320 --> 00:52:02.079 Google Play or in the Apple Store one touch and you can listen on your 755 00:52:02.119 --> 00:52:07.320 car radio, Bluetooth Advice, Android Auto or Apple Car Play, catch the 756 00:52:07.400 --> 00:52:10.960 KCAA buzz in your earbuds or on the streets. Celebrating twenty five years of 757 00:52:12.079 --> 00:52:15.880 talk news and excellence with our new kcaaapp, Just do it and download it. 758 00:52:16.199 --> 00:52:25.239 KCAA celebrating twenty five years. Here are two terms you don't expect to 759 00:52:25.320 --> 00:52:31.360 see together the State of Alabama and progressive leader. Okay, I'm a Texan, 760 00:52:31.440 --> 00:52:36.280 so I have no standing to point at the rank regressiveness of any other 761 00:52:36.360 --> 00:52:42.280 state government. But still Alabama, and yet the Camellia State has flowered as 762 00:52:42.280 --> 00:52:46.480 a model of strong progressive action in one area of critical public importance, quality 763 00:52:46.599 --> 00:52:52.280 childcare. It's a cliche to say our children are our future, but it's 764 00:52:52.320 --> 00:52:55.679 also true, Why then, do we invest so little in our littlest ones 765 00:52:55.840 --> 00:53:01.559 our future? America's childcare system is a national disgrace, failing to provide safe 766 00:53:01.639 --> 00:53:07.840 places for children of working parents and failing to boost the education of pre kindergarten 767 00:53:07.000 --> 00:53:12.880 tykes. Moreover, the abject failure of state and national officials to meet this 768 00:53:13.039 --> 00:53:17.639 basic social need is spreading inequality, rolling back opportunities for women, and severely 769 00:53:17.760 --> 00:53:24.519 restricting economic progress. Yet Alabama officials have recently been setting the national standard for 770 00:53:24.880 --> 00:53:30.760 effective pre k programs by making a major investment in its four and five year 771 00:53:30.840 --> 00:53:36.639 olds operating a statewide childcare network. A major factor in its success is a 772 00:53:36.960 --> 00:53:42.760 two generation approach, not only educating the kiddos, but also providing support materials 773 00:53:42.880 --> 00:53:49.719 and coaching so that parents engage as their children's first teachers, producing demonstrable results 774 00:53:49.719 --> 00:53:53.960 a year after year. The state's public network gets bipartisan support and funding from 775 00:53:54.000 --> 00:53:59.880 the Alabama Legislature. The program is free and available to all, with special 776 00:54:00.039 --> 00:54:05.199 attention devoted to enlisting often overlooked families in rural, poor, and people of 777 00:54:05.280 --> 00:54:10.079 color communities. Alabama is paying and respecting teachers as the professionals they are. 778 00:54:10.880 --> 00:54:15.760 This is Jim Hitar saying, if one of our poorest states can rise to 779 00:54:15.840 --> 00:54:20.079 meet this basic human need, what's wrong with the richest country in the history 780 00:54:20.119 --> 00:54:25.280 of the world. T Hebo Tea Club's original Pure Power d RGO super tea 781 00:54:25.400 --> 00:54:30.239 helps build red corpusoals in the blood, which carry oxygen to organs and cells. 782 00:54:30.400 --> 00:54:34.599 Our organs and cells need oxygen to regenerate themselves. The immune system needs 783 00:54:34.639 --> 00:54:37.320 oxygen to develop, and cancer dies in oxygen. So the tea is great 784 00:54:37.360 --> 00:54:40.679 for healthy people because it helps build the immune system, and it can surely 785 00:54:40.760 --> 00:54:45.960 be miraculous for some inviting a potentially life threatening disease due to an infection, 786 00:54:45.199 --> 00:54:50.280 diabetes, or cancer. The tea is also organic and naturally caffeine free. 787 00:54:50.519 --> 00:54:53.159 A one pound package of tea is forty nine to ninety five, which includes 788 00:54:53.199 --> 00:54:57.880 shipping. To order, please visit t Hebotea club dot com. T hebow 789 00:54:57.960 --> 00:55:02.079 is spelled tea like tom aa, h e b like boyoh. They continue 790 00:55:02.119 --> 00:55:06.719 with the word t and then the word club. The complete website is to 791 00:55:06.880 --> 00:55:10.360 Hebot Club dot com or call us at eight one eight six one zero eight 792 00:55:10.480 --> 00:55:15.840 zero eight eight Monday through Saturday nine am to five pm California time. That's 793 00:55:16.079 --> 00:55:22.639 eight one eight six one zero eight zero eight eight to Hebot club dot com. 794 00:55:22.840 --> 00:55:28.639 For several years, KAA has been marketing the Youngevity brand of nutritional and 795 00:55:28.719 --> 00:55:32.920 personal care products. Our experience with Youngevity has been one hundred percent positive, 796 00:55:34.400 --> 00:55:37.559 so we are pleased to recommend them to you. Regarding nutritional supplements, we 797 00:55:37.639 --> 00:55:43.559 recommend pollen Burst in the berry flavor and tangy tangerine two point zero in the 798 00:55:43.639 --> 00:55:47.559 tablet form. For regularity issues, we recommend three day cleanse, and for 799 00:55:47.679 --> 00:55:53.119 personal care, we recommend morning hydration cream. You can shop online for Youngevity 800 00:55:53.239 --> 00:55:59.599 at www dot KCAA team dot com, or you can order by phone by 801 00:55:59.639 --> 00:56:02.760 calling eight hundred ninety eighty two three one ninety seven and tell customer support that 802 00:56:02.840 --> 00:56:07.920 you are part of the KCAA Team Youngevity is an American company based in San 803 00:56:07.960 --> 00:56:13.599 Diego. Call Youngevity at eight hundred ninety eight two three one nine seven and 804 00:56:13.679 --> 00:56:17.840 ask about monthly autoship that allows you to buy Youngevity products at wholesale prices. 805 00:56:19.280 --> 00:56:23.719 That number again eight hundred nine eight two three one nine seven. Do you 806 00:56:23.800 --> 00:56:28.239 want to learn and get answers to questions not addressed in the mass media, 807 00:56:28.760 --> 00:56:31.559 Then you want to hear my show business Game Changers with me, Sarah Westall. 808 00:56:31.960 --> 00:56:37.000 I have conversations with thought leaders who have the courage to address off limit 809 00:56:37.119 --> 00:56:40.320 topics, so you and your family have the tools to make the right decisions. 810 00:56:40.679 --> 00:56:45.119 Join me Wednesdays at four pm on ten fifty AM and one oh six 811 00:56:45.199 --> 00:56:52.159 point five FM right here at KCAA, that station that leaves no listener behind. 812 00:56:54.360 --> 00:56:58.960 It's that time of year again, No, not the holidays. Medicare 813 00:56:59.119 --> 00:57:02.079 open enrollment and if you have questions about Medicare, you should talk to the 814 00:57:02.199 --> 00:57:07.760 local experts. Paul Berrette and Associates. All of his agents are certified with 815 00:57:07.880 --> 00:57:10.360 plans that are accepted by most of the medical groups in our area. Call 816 00:57:10.760 --> 00:57:15.719 nine oh nine seven nine to three oh three eight five. Their service is 817 00:57:15.800 --> 00:57:19.599 free and after forty two years of the business, their agents are trained to 818 00:57:19.679 --> 00:57:24.760 help you pick the plan that's right for you. NBC News Radio, I'm 819 00:57:24.800 --> 00:57:29.480 Chris Gragio. Lawmakers are set to return to Washington, DC this week, 820 00:57:29.639 --> 00:57:32.280 just ahead of a deadline to avoid a partial government shutdown. House Speaker Mike 821 00:57:32.360 --> 00:57:36.800 Johnson said Friday that he would move a set of spending bills forward as a 822 00:57:36.880 --> 00:57:39.679 single package ahead of the Friday deadline. Two congressmen are trying to get a 823 00:57:39.719 --> 00:57:44.960 bipartisan border security and foreign aid bill onto the House floor. We are forcing 824 00:57:45.039 --> 00:57:49.320 this bill to the floor to make sure that everybody acts because, as President 825 00:57:49.400 --> 00:57:52.480 Zeleski said, they have weeks and not months. Republican Congressman Brian Fitzpatrick of 826 00:57:52.480 --> 00:57:58.480 Pennsylvania. Democratic Congressman Jared Golden of Maine set on CBS's face the Nation that 827 00:57:58.599 --> 00:58:00.840 they believe their bill can get the necessary two hundred and eighteen votes in the 828 00:58:00.880 --> 00:58:05.679 House. They both agree a one party solution likely will not get the votes 829 00:58:05.840 --> 00:58:07.800 and a bill needs to grow out of the middle to be able to pass. 830 00:58:08.079 --> 00:58:12.880 They also stressed the urgency to provide a to Ukraine after a city filled 831 00:58:12.920 --> 00:58:15.519 to Russia last week. The bill is an alternative to a Senate bill that 832 00:58:15.559 --> 00:58:20.360 House Speaker Mike Johnson has already called dead in the water. Fitzpatrick said he 833 00:58:20.440 --> 00:58:23.079 and Golden have filed their House bill in a manner to get the bill expedited 834 00:58:23.159 --> 00:58:28.559 quickly. A Democrat whose name has been touted as a presidential candidate is all 835 00:58:28.639 --> 00:58:31.719 in for President Biden getting re elected to his second term. All because of 836 00:58:31.800 --> 00:58:37.519 Biden's wisdom, because of his temperance, his capacity to lead in a bipartisan 837 00:58:37.639 --> 00:58:42.840 manner, which is an under represented point. Speaking on NBC's Meet the Press, 838 00:58:42.920 --> 00:58:46.800 California Governor Gavenusom says Biden has an extraordinary record as president. Newsom called 839 00:58:46.840 --> 00:58:52.320 the US economy booming under Biden, noting that fifteen million jobs have been added 840 00:58:52.440 --> 00:58:54.960 in his first three years. He added that he's confident that the president would 841 00:58:54.960 --> 00:59:00.320 defeat former President Trump in a general election. Nikki Haley says she is staying 842 00:59:00.320 --> 00:59:04.760 in the presidential race despite losing the South Carolina primary to give Republicans an option 843 00:59:04.960 --> 00:59:07.960 other than former President Trump. Haley told supporters last night that she's a woman 844 00:59:08.039 --> 00:59:13.039 of her word and will continue running despite the loss. Haley said she won't 845 00:59:13.039 --> 00:59:15.800 give up the fight while the majority of Americans are unhappy with both Donald Trump 846 00:59:15.840 --> 00:59:22.000 and Joe Biden. I'm Chris Caragio, NBC News Radio, NBC News on 847 00:59:22.159 --> 00:59:29.519 CACAA Lomelinda sponsored by Teamsters Local nineteen thirty two Protecting the Future of Working Families 848 00:59:29.599 --> 00:59:37.760 Teamsters nineteen thirty two dot org. The election is March fifth. Your ballots 849 00:59:37.800 --> 00:59:42.480 are in the mail. Mail them in today. Now there's something to vote 850 00:59:42.519 --> 00:59:46.840 for because there's a new Marshall in town. Vote Derek Marshall for twenty third 851 00:59:46.960 --> 00:59:51.519 Congress by March fifth. Y'all know we need to change Congress. The current 852 00:59:51.559 --> 00:59:53.760 Congress has nothing but fight with each other. They've forgotten why we sent them 853 00:59:53.800 --> 00:59:58.119 there to work. Let's marshal and fix our problems at the border, a 854 00:59:58.199 --> 01:00:01.760 comprehensive plan, not political rhetoric and infighting. Yes, there's a new Marshal 855 01:00:01.840 --> 01:00:06.920 in town. The rent is too darn high. Let's marshal in affordable housing 856 01:00:07.119 --> 01:00:08.280 and real, affordable universal healthcare.