WEBVTT 1 00:00:00.600 --> 00:00:04.839 Up next, we have Ben Galen, portfolio leader from Roche, 2 00:00:05.240 --> 00:00:10.199 and Ken Getz, MBA, Director and Professor Tuft's Center for 3 00:00:10.320 --> 00:00:14.039 the Study of Drug Development at Tufts University, and they 4 00:00:14.080 --> 00:00:17.559 will be presenting the Patient Voice and Protocol Planning and 5 00:00:17.679 --> 00:00:21.640 Execution for Long Term Business Sustainability. 6 00:00:22.239 --> 00:00:25.320 Good afternoon, everyone. I'm so sorry the person who asked 7 00:00:25.359 --> 00:00:29.760 about measuring patient burden isn't here, because that's exactly what 8 00:00:29.760 --> 00:00:33.000 we're going to talk about now. I have the pleasure 9 00:00:33.159 --> 00:00:37.600 of co presenting with Ben Galan. We've been collaborating on 10 00:00:37.719 --> 00:00:41.640 an effort to think through strategic concepts and the application 11 00:00:42.560 --> 00:00:46.880 of an approach to measuring a patient burden based on 12 00:00:47.039 --> 00:00:50.880 protocol design characteristics, and then to use that input to 13 00:00:50.960 --> 00:00:55.759 inform decision making about the protocol itself and ultimately the 14 00:00:55.799 --> 00:00:59.920 execution of the clinical trial. So I'm going to start 15 00:01:00.079 --> 00:01:03.039 by providing a little bit of context for you. Many 16 00:01:03.079 --> 00:01:06.239 of you have seen this data before, but I want 17 00:01:06.239 --> 00:01:08.400 to sort of set the stage by just showing you 18 00:01:08.519 --> 00:01:12.799 how complex our protocol designs have actually become and this 19 00:01:12.840 --> 00:01:16.200 is something we measure routinely. And then I'll move into 20 00:01:16.280 --> 00:01:22.680 specifically how we are able to systematically introduce and integrate 21 00:01:22.760 --> 00:01:27.560 patient input based on the characteristics of the protocol itself. 22 00:01:27.560 --> 00:01:31.719 It's scientific characteristics as well as the executional elements of 23 00:01:31.760 --> 00:01:33.879 the design. And then I'm going to turn it over 24 00:01:33.879 --> 00:01:36.079 to Ben, who's going to really talk about how you 25 00:01:36.159 --> 00:01:41.599 apply this and you determine ways to integrate patient input 26 00:01:41.719 --> 00:01:46.840 in to really drive sustainable value for the organization overall, 27 00:01:47.319 --> 00:01:50.959 and how to use those insights to inform cross functional 28 00:01:51.040 --> 00:01:56.120 decision making. First, the context on the left hand side, 29 00:01:56.159 --> 00:01:58.799 you see some data that was published a few years ago, 30 00:02:00.120 --> 00:02:02.120 just to really give you a flavor for some of 31 00:02:02.159 --> 00:02:06.920 the most common elements in a protocol, scientific and operational 32 00:02:07.000 --> 00:02:10.800 or executional. And I'm showing you really just two time 33 00:02:10.879 --> 00:02:16.599 periods here. We've been measuring growth in the prevalence of 34 00:02:16.639 --> 00:02:19.800 these variables for quite a long time. If you look 35 00:02:19.840 --> 00:02:22.719 at the total number of endpoints, for example, this is 36 00:02:22.800 --> 00:02:27.439 on average, looking across multiple therapeutic areas, we've seen nearly 37 00:02:27.560 --> 00:02:32.319 a doubling. Eligibility criteria has remained relatively flat, but that's 38 00:02:32.360 --> 00:02:35.680 in part a function of the way we count inclusion 39 00:02:35.680 --> 00:02:40.879 and exclusion criteria. Some companies group multiple criteria and counted 40 00:02:40.919 --> 00:02:45.599 as a single as a single inclusion criterion. For example, 41 00:02:46.240 --> 00:02:50.840 total procedures to support those endpoints has grown dramatically, And 42 00:02:50.879 --> 00:02:54.439 you can look across any element of the design, the 43 00:02:54.520 --> 00:02:58.439 number of sites, number of countries, virtually, they have all 44 00:02:58.439 --> 00:03:03.919 grown dramatically. Look at that bottom data point. Total data 45 00:03:04.199 --> 00:03:08.919 collected by the protocol has risen far more dramatically than 46 00:03:08.960 --> 00:03:13.719 any other area, nearly four times the level we observed 47 00:03:13.759 --> 00:03:17.240 in twenty ten on average. On the right hand side, 48 00:03:17.280 --> 00:03:21.919 I'm showing you just another measure of customization and complexity, 49 00:03:21.960 --> 00:03:25.919 and that is the number of intermediaries involved in supporting 50 00:03:26.199 --> 00:03:31.240 any clinical trial, both the design, but more so the 51 00:03:31.280 --> 00:03:35.199 execution of the study and the data collection. About four 52 00:03:35.319 --> 00:03:41.560 or five years ago, total spending by pharma on contract 53 00:03:41.599 --> 00:03:48.800 service providers technology services, study conduct services, cro services surpassed 54 00:03:49.120 --> 00:03:52.120 the total amount that is spent each year just supporting 55 00:03:52.520 --> 00:03:58.759 internal infrastructure. And the key takeaway with all of this 56 00:03:58.960 --> 00:04:05.560 is complexity is associated with poor with poorer performance. It's 57 00:04:05.599 --> 00:04:10.280 correlated inversely with poor performance. So the higher complexity rises, 58 00:04:10.800 --> 00:04:15.560 the more performance suffers. Our recruitment rates are lower, our 59 00:04:15.599 --> 00:04:19.639 retention rates are worse, our timelines are longer. The number 60 00:04:19.680 --> 00:04:24.600 of protocol amendments that we observe actually increases the more 61 00:04:24.600 --> 00:04:27.879 complex our designs. On the right hand side, I'm showing 62 00:04:27.879 --> 00:04:33.120 you just one measure of site burden to administer our protocols, 63 00:04:33.120 --> 00:04:37.120 and we have countless measures that really show how difficult 64 00:04:37.160 --> 00:04:41.040 it is for sites to manage and administer our protocols today. 65 00:04:41.360 --> 00:04:46.199 Here's just one measure showing you site enrollment achievement, the 66 00:04:46.279 --> 00:04:51.360 percentage of sites that were activated that ultimately achieve target enrollment, 67 00:04:52.040 --> 00:04:55.360 and that number has been declining steadily every year for 68 00:04:55.480 --> 00:04:59.879 more than a decade. Here are just two other men 69 00:05:00.000 --> 00:05:03.079 measures that now bring it down to the patient burden level, 70 00:05:03.120 --> 00:05:05.439 and we have many others that I could share with you. 71 00:05:06.160 --> 00:05:11.040 The percentage of patients who drop out prematurely due to 72 00:05:11.120 --> 00:05:14.480 their own choice, not the choice of the site personnel, 73 00:05:15.079 --> 00:05:20.199 or due to a response to the study drug has skyrocketed. 74 00:05:20.560 --> 00:05:24.279 Nearly two thirds of patients in recent studies are dropping 75 00:05:24.399 --> 00:05:27.959 out out of their own choice first, so they're pre 76 00:05:28.040 --> 00:05:30.600 empting the site or they're choosing to drop out because 77 00:05:30.600 --> 00:05:34.639 of the burden of participation. On the right hand side, 78 00:05:35.439 --> 00:05:39.160 these are now exit interviews that were conducted with patients 79 00:05:39.160 --> 00:05:42.759 who completed their participation and we asked what did you 80 00:05:42.959 --> 00:05:46.680 like least about being in the study, and four of 81 00:05:46.720 --> 00:05:51.839 the five areas all relate to the burden itself, the 82 00:05:51.920 --> 00:05:55.240 location of the research center. The study visits were so 83 00:05:55.360 --> 00:05:59.360 time consuming, there were too many procedures, they were cumbersome, 84 00:05:59.399 --> 00:06:02.720 and I love that one compensation was not enough. You 85 00:06:02.720 --> 00:06:06.680 could not pay me enough for the kind of requirements 86 00:06:06.079 --> 00:06:12.560 that this protocol post. We have been looking to simplify 87 00:06:12.680 --> 00:06:15.759 protocol design for a long time. You know, I've been 88 00:06:15.879 --> 00:06:19.879 measuring as an outsider looking in as an academic for 89 00:06:19.959 --> 00:06:23.639 nearly thirty years. We've been looking at protocol design behaviors. 90 00:06:24.240 --> 00:06:27.279 Early on, one of the most common approaches was to 91 00:06:27.480 --> 00:06:33.160 use economic input into protocol design. The idea that if 92 00:06:33.199 --> 00:06:36.879 you cut out a procedure, or you cut out a 93 00:06:36.879 --> 00:06:40.319 certain number of times that the procedure is performed, would 94 00:06:40.399 --> 00:06:45.120 save the study budget some expense. That was a really 95 00:06:45.720 --> 00:06:49.920 highly prevalent approach in the early nineties, a period where 96 00:06:49.920 --> 00:06:52.519 we started to see a lot of companies look for 97 00:06:52.680 --> 00:06:56.360 tools that would help them manage the economics and the 98 00:06:56.399 --> 00:06:59.959 financial requirements of a lot of their activity. You might remember, 99 00:07:00.079 --> 00:07:04.480 for example, the PIKS database, which was a tool that 100 00:07:04.800 --> 00:07:10.720 gave pharma companies the ability to understand the typical pricing 101 00:07:10.839 --> 00:07:15.800 for an investigative site to perform a particular protocol. When 102 00:07:15.839 --> 00:07:20.000 we shifted in the two thousand twenty ten period, this 103 00:07:20.160 --> 00:07:24.240 was one of the early phases of sponsors looking to 104 00:07:24.639 --> 00:07:29.240 establish a more meaningful and more effective relationship with their sites, 105 00:07:29.839 --> 00:07:32.759 and so in that twenty twenty ten period we saw 106 00:07:33.279 --> 00:07:37.600 height and focus on executional feasibility, and you may recall 107 00:07:37.680 --> 00:07:40.879 that a lot of the feasibility assessments that were introduced 108 00:07:41.319 --> 00:07:44.720 started to get larger and larger and larger as more 109 00:07:44.839 --> 00:07:50.160 data was collected by the site. We then shifted our 110 00:07:50.199 --> 00:07:53.680 focus and we're still in this environment where we're looking 111 00:07:53.720 --> 00:07:58.480 to reduce the number of endpoints. We're looking to prioritize 112 00:07:58.560 --> 00:08:03.240 our designs so that we can minimize avoidable amendments, one 113 00:08:03.240 --> 00:08:06.959 of the most disruptive experiences that we have in any 114 00:08:07.000 --> 00:08:10.759 clinical trial, and that's still a very very important way 115 00:08:10.759 --> 00:08:14.519 that we look to simplify protocol design. And the patient 116 00:08:14.600 --> 00:08:20.240 engagement movement has introduced and elevated our sensitivity, our awareness, 117 00:08:20.279 --> 00:08:24.519 and our interest in integrating patient input into design and 118 00:08:24.680 --> 00:08:29.240 using the patient's own perceived a sense of burden, the 119 00:08:29.360 --> 00:08:34.240 relevance of the trial their own concerns about the convenience 120 00:08:34.399 --> 00:08:38.559 of participation to help identify areas that we can modify. 121 00:08:39.080 --> 00:08:43.399 So these are essentially the four optimization areas. We know 122 00:08:43.440 --> 00:08:48.080 if many organizations that have integrated these approaches. Patient engagement 123 00:08:48.360 --> 00:08:53.559 and the measurement of patient burden is perhaps the newest area, 124 00:08:54.039 --> 00:08:59.320 and we mostly see companies using advisory boards to gather 125 00:08:59.399 --> 00:09:01.600 that input. These are like focus groups, and I know 126 00:09:01.679 --> 00:09:05.720 many organizations are doing them, but they typically appear when 127 00:09:05.799 --> 00:09:09.960 the protocol is nearly finalized, so they're actually coming into 128 00:09:10.000 --> 00:09:14.240 the planning and the design phase slightly late to the game, 129 00:09:15.039 --> 00:09:18.960 after many internal champions and thought leaders and others have 130 00:09:19.039 --> 00:09:23.399 provided input into the design of the protocol. The idea 131 00:09:23.639 --> 00:09:27.600 for the burden assessment is to move patient input into 132 00:09:27.639 --> 00:09:32.679 the earliest stages so that before we move to solicit 133 00:09:32.720 --> 00:09:36.039 input from thought leaders or at the same time that 134 00:09:36.080 --> 00:09:41.159 we're soliciting input from multiple parties or already incorporating patient input. 135 00:09:42.440 --> 00:09:44.919 So there are a number of approaches a number of 136 00:09:45.000 --> 00:09:49.399 organizations that are offering methodologies. What the Tough Center has 137 00:09:49.480 --> 00:09:55.519 done is created essentially an approach that organizations can ultimately 138 00:09:56.240 --> 00:09:59.840 build in house and a take in house so that 139 00:09:59.840 --> 00:10:04.519 the can do this internally. We are not specifically designing 140 00:10:04.919 --> 00:10:08.559 a product that can be licensed. We're developing essentially an 141 00:10:08.639 --> 00:10:13.480 approach that any organization can adopt and use. We've published 142 00:10:13.519 --> 00:10:17.840 extensively and I'm listing three of the publications here. What 143 00:10:17.879 --> 00:10:21.399 we did is we based on our work in protocol 144 00:10:21.480 --> 00:10:26.519 design and the benchmark activity that is now coming up 145 00:10:26.600 --> 00:10:30.080 on thirty years of experience, we were able to narrow 146 00:10:30.120 --> 00:10:34.080 down the sixty most common procedures that we see in 147 00:10:34.120 --> 00:10:39.799 every protocol, and we presented those procedures and other elements 148 00:10:39.840 --> 00:10:45.679 of the participation experience, distance to travel, whether a procedure 149 00:10:45.759 --> 00:10:48.639 might be performed on site or at a different location, 150 00:10:49.480 --> 00:10:52.480 the length of the visit, for example. All of that 151 00:10:53.240 --> 00:10:58.440 was presented to patients globally and they would rate the 152 00:10:58.600 --> 00:11:03.840 relative perceived bird of every one of those procedures or 153 00:11:03.960 --> 00:11:10.080 of those experiences related to essentially a reference point, and 154 00:11:10.120 --> 00:11:15.399 that reference was a routine physical exam, something that most 155 00:11:15.399 --> 00:11:19.240 patients are familiar with, even just by interacting with their 156 00:11:19.279 --> 00:11:26.320 own primary care physician practice. And we gathered ultimately responses 157 00:11:26.360 --> 00:11:31.559 from thirty six hundred global patients, so we have some 158 00:11:31.679 --> 00:11:39.799 granularity by disease conditioned by demographic characteristics, by socioeconomic status, 159 00:11:39.840 --> 00:11:42.759 and many others, and we're now able to use that 160 00:11:43.000 --> 00:11:46.799 input and its variation by all of these different subgroups 161 00:11:47.279 --> 00:11:52.039 to map to every protocol design. So we're looking at 162 00:11:52.080 --> 00:11:56.240 a number of different areas. We're looking at the procedures 163 00:11:56.240 --> 00:12:00.480 that are performed, we're looking at burden to adherents, we're 164 00:12:00.480 --> 00:12:05.120 looking at lifestyle restrictions, we're looking at other convenience factors, 165 00:12:05.120 --> 00:12:08.000 and all of that can be mapped based on the 166 00:12:08.039 --> 00:12:12.840 input that we've received from patients. To date, fourteen companies 167 00:12:12.879 --> 00:12:16.879 have been applying this approach and bringing it in house. 168 00:12:17.799 --> 00:12:19.960 And I'll just share a little bit of the high 169 00:12:20.000 --> 00:12:24.399 level aggregate data with you. The chart on the left 170 00:12:24.440 --> 00:12:29.759 here is not intuitive, and it's because we've created a 171 00:12:29.799 --> 00:12:33.360 burden's score, which does not have meaning in and of itself. 172 00:12:33.399 --> 00:12:36.399 You have to sort of compare that to benchmark data. 173 00:12:36.960 --> 00:12:39.799 For companies, they can compare it to their own internal 174 00:12:39.960 --> 00:12:44.000 burden score to see how it's changing over time. I'm 175 00:12:44.039 --> 00:12:47.120 showing you aggregate data here, and what's most important is 176 00:12:47.159 --> 00:12:51.000 it tracts so well to changes that we've seen in 177 00:12:51.080 --> 00:12:54.960 protocol design over time anyway, So it's sort of a 178 00:12:55.000 --> 00:12:57.360 way that we have helped to validate a lot of 179 00:12:57.360 --> 00:13:01.799 our measures. They map very well to change in design decisions. 180 00:13:02.360 --> 00:13:04.840 On the right hand side, I'm showing you another measure 181 00:13:04.919 --> 00:13:09.840 of burden. This is now the distribution of visits by 182 00:13:09.919 --> 00:13:14.399 their average duration. Where in twenty eleven to twenty fourteen, 183 00:13:15.320 --> 00:13:19.120 nearly sixty percent of all patient visits averaged about an 184 00:13:19.200 --> 00:13:23.320 hour or less. That has dropped down now to below half. 185 00:13:23.879 --> 00:13:26.679 And you can see the proportion that has grown the 186 00:13:26.679 --> 00:13:29.679 most are now those visits that are lasting more than 187 00:13:29.720 --> 00:13:34.799 two hours. This may include travel, right, we've sort of 188 00:13:34.840 --> 00:13:38.399 consolidated that to summarize the statistic here, But that's just 189 00:13:38.480 --> 00:13:43.759 another measure of burden that we can assess. And again 190 00:13:43.879 --> 00:13:46.360 the punchline and all this is you can start to 191 00:13:46.440 --> 00:13:52.480 relate burden to performance outcomes and that helps you isolate 192 00:13:52.720 --> 00:13:56.679 those areas of that may be most contributing to burden 193 00:13:57.159 --> 00:14:01.639 that might be predictive of different outcomes in the study. Here, 194 00:14:01.759 --> 00:14:05.799 I'm pulling together data from all of the fourteen companies 195 00:14:05.799 --> 00:14:09.440 that have participated so far, and what I wanted you 196 00:14:09.480 --> 00:14:13.320 to see is just a pattern that is emerging. Essentially, 197 00:14:14.200 --> 00:14:19.559 when burden is low, we typically see faster relative cycle times, 198 00:14:20.039 --> 00:14:25.759 fewer protocol amendments, fewer protocol deviations, and even lower dropout 199 00:14:25.840 --> 00:14:29.960 rates and drop out for many of you, you'll know, 200 00:14:30.039 --> 00:14:33.039 it's a very complicated measure because there are so many 201 00:14:33.080 --> 00:14:35.240 factors that play a part, so we were surprised to 202 00:14:35.240 --> 00:14:39.639 see a slight decline in the overall rate here in 203 00:14:39.679 --> 00:14:44.480 this assessment. With that, I'm going to turn it over 204 00:14:44.559 --> 00:14:48.480 to Ben, who's going to really show how his organization 205 00:14:48.600 --> 00:14:52.440 has approached trying to integrate and leverage this insight into 206 00:14:52.480 --> 00:14:56.440 their own protocol design activity. So Ben, I'm turning it 207 00:14:56.480 --> 00:14:58.000 over to you. Thanks so much, and. 208 00:15:05.720 --> 00:15:09.720 Thanks Ken for the introduction. Yeah, so I'm here representing 209 00:15:09.799 --> 00:15:13.200 ROSH and really talking about the collaboration that we've had 210 00:15:13.279 --> 00:15:16.000 with Ken and Tufts and how this has helped us 211 00:15:16.039 --> 00:15:20.559 to actually integrate a lot of the different insights across 212 00:15:20.919 --> 00:15:25.399 the ecosystem of our clinical trial design. So I want 213 00:15:25.399 --> 00:15:28.080 to start talking first about how this came to be. 214 00:15:29.519 --> 00:15:33.559 We have in my company, like the theme you see 215 00:15:33.600 --> 00:15:36.720 across many of the sponsored companies right now, what we 216 00:15:36.840 --> 00:15:39.960 consider to be a productivity issue in our clinical trials, 217 00:15:41.039 --> 00:15:44.759 and that is something that has some very serious legs 218 00:15:44.759 --> 00:15:47.519 behind it to try and actually uplift that. But also 219 00:15:47.840 --> 00:15:51.320 we have a growing need and understanding that we need 220 00:15:51.360 --> 00:15:53.159 to make sure that the trial designs themselves are much 221 00:15:53.159 --> 00:15:56.720 more patient centric, and we have the opportunity therefore to 222 00:15:56.879 --> 00:16:01.720 essentially bring together a mutually beneficial relationship where we can 223 00:16:01.759 --> 00:16:07.000 make the trials much more patient inclusive, much better for patients, 224 00:16:07.120 --> 00:16:11.559 but also actually for us cheaper to run, faster to run, 225 00:16:11.639 --> 00:16:15.759 and get to market more quickly. When we started down 226 00:16:15.799 --> 00:16:19.480 this path several years ago, we decided that we didn't 227 00:16:19.480 --> 00:16:22.039 want to reinvent the wheel for obvious reasons, so we 228 00:16:22.080 --> 00:16:24.519 reached out to Ken in the Tought organization to try 229 00:16:24.559 --> 00:16:31.200 and actually start leveraging the standard framework and methodology that 230 00:16:31.519 --> 00:16:33.799 they can and tough to actually brought. And what that 231 00:16:33.960 --> 00:16:37.039 enabled us to do very quickly was to actually have 232 00:16:37.320 --> 00:16:40.679 a point of comparison of where do our trials actually 233 00:16:40.759 --> 00:16:45.600 land against other large industry bodies, because we wanted a 234 00:16:45.600 --> 00:16:48.759 point of comparison that's actually sort of we run extremely 235 00:16:48.799 --> 00:16:54.159 complex global trials. We have a reputation for being amongst 236 00:16:54.519 --> 00:16:57.200 i think it's fair to say, the most complex and 237 00:16:57.240 --> 00:17:00.919 the most expensive and also the slowest trials amongst most 238 00:17:00.960 --> 00:17:04.079 of the large organizations which is not something we're terribly 239 00:17:04.119 --> 00:17:08.640 proud of, so we needed a framework that would actually 240 00:17:08.720 --> 00:17:13.079 help us actually bring this together and to directly address 241 00:17:13.200 --> 00:17:18.000 R and D productivity. What we identified quite quickly through 242 00:17:18.039 --> 00:17:22.400 this work was not just the KPI is underneath this 243 00:17:22.519 --> 00:17:25.440 in terms of how do we actually measure the burden 244 00:17:25.519 --> 00:17:29.759 of our designs and linking that to for example, the 245 00:17:29.799 --> 00:17:33.200 investigator burdens and the site level burdens and different feedback 246 00:17:33.240 --> 00:17:37.880 from patient bodies and collecting evidence around for example, the 247 00:17:37.960 --> 00:17:42.519 value of patient engagement for our patient engagement organizations, but 248 00:17:42.599 --> 00:17:46.880 actually to contextualize those scores against key drivers and determinants 249 00:17:46.880 --> 00:17:51.240 of burden, which allows us to then actually influence decision 250 00:17:51.279 --> 00:17:55.119 making and designs within our very large and complex organization 251 00:17:55.240 --> 00:18:00.279 across different functions that we have there. Of course, to 252 00:18:00.319 --> 00:18:03.559 do this you can see through the different talks earlier today, 253 00:18:05.079 --> 00:18:09.440 there are a large variety of different complexities involved across 254 00:18:09.799 --> 00:18:12.720 For example, the people that actually develop the protocols and 255 00:18:12.759 --> 00:18:16.519 study designs need to essentially make sure that the protocol 256 00:18:16.599 --> 00:18:19.039 is for purpose and actually determining the key end points 257 00:18:19.039 --> 00:18:22.440 of evidence it needs to actually work for the science, 258 00:18:23.440 --> 00:18:27.240 but then actually in operationalizing design and running the studies. Effectively, 259 00:18:27.400 --> 00:18:31.240 we also have a second series of stakeholders internally, and 260 00:18:31.279 --> 00:18:34.960 they have very different priorities and incentive structures. And then, 261 00:18:35.000 --> 00:18:38.359 of course we have the people that actually are participating 262 00:18:38.359 --> 00:18:40.920 in our trials who have a whole different range of 263 00:18:40.960 --> 00:18:43.359 issues and complications that we need to make sure that 264 00:18:43.359 --> 00:18:49.240 we're actually taking sufficiently into account. When we started presenting 265 00:18:49.279 --> 00:18:53.359 the results of these initial analyses and insights, we very 266 00:18:53.440 --> 00:18:58.359 quickly gained across pharma reaction by a cross farmer, I 267 00:18:58.400 --> 00:19:02.119 mean internally across our early research organizations and late research 268 00:19:02.200 --> 00:19:07.119 organizations and a medical affair organizations. Suddenly we started gaining 269 00:19:07.160 --> 00:19:10.359 this critical mass of different people across the different functions 270 00:19:10.839 --> 00:19:14.480 who were very motivated to actually participate in this effort 271 00:19:14.920 --> 00:19:18.599 and to bring together those insights. So I want to obviously, 272 00:19:18.640 --> 00:19:22.720 I can't talk too many specifics around the data that 273 00:19:22.759 --> 00:19:25.759 we're actually seeing. A lot of this is in flight. 274 00:19:26.000 --> 00:19:27.279 I'm going to talk about it sort of in an 275 00:19:27.319 --> 00:19:30.480 aggregate way, and I'm happy to answer questions offline and 276 00:19:30.559 --> 00:19:34.000 off recording at a later date if you would like 277 00:19:34.119 --> 00:19:36.279 and be interested in sort of how we're approaching this 278 00:19:36.359 --> 00:19:38.480 and some of the value we're seeing. But I think 279 00:19:38.519 --> 00:19:43.559 the suffice to say, in the initial efforts that are 280 00:19:43.599 --> 00:19:46.519 actually coming through with study designs that are actually changing 281 00:19:46.559 --> 00:19:49.279 and are in flight with patients right now, we are 282 00:19:49.359 --> 00:19:53.920 seeing substantial improvements in different productivity scores that can just 283 00:19:54.279 --> 00:19:58.799 displad improvements in our cycle times in terms of patient 284 00:19:58.839 --> 00:20:02.960 recruitment activities, reductions and patients dropouts and reduction reductions in 285 00:20:03.000 --> 00:20:07.559 protocol avoidable protocol amendments and different drivers there a value 286 00:20:08.920 --> 00:20:11.759 and the feedback from the patient organizations has also been positive. 287 00:20:11.799 --> 00:20:14.400 So we are starting to see the shift that we 288 00:20:14.440 --> 00:20:18.160 are looking for through this effort. And say, this effort, 289 00:20:18.200 --> 00:20:22.160 but it's simple an amalgamation of many, many different efforts 290 00:20:22.200 --> 00:20:25.519 coming together with a single lens. I'm talking a little 291 00:20:25.559 --> 00:20:28.559 bit generic here. We don't actually call this the triad 292 00:20:28.720 --> 00:20:31.359 in company, but it's more sort of how do we 293 00:20:31.400 --> 00:20:33.680 think about this and how are we framing it as 294 00:20:33.680 --> 00:20:36.960 a construct, Because what we need to do is actually 295 00:20:37.000 --> 00:20:41.880 make the data that Ken was talking about actionable across 296 00:20:42.319 --> 00:20:46.680 different functions that have competing priorities. And not only do 297 00:20:46.720 --> 00:20:49.759 we need to understand what performance is and what's driving it, 298 00:20:49.799 --> 00:20:51.119 we need to actually be able to get to the 299 00:20:51.200 --> 00:20:55.319 root causes of that, what specifically is driving the increase 300 00:20:55.359 --> 00:20:59.279 in burden and protocol complexity and how do we nip 301 00:20:59.359 --> 00:21:01.400 that in the bud where we can and how do 302 00:21:01.440 --> 00:21:05.400 we recognize what is necessary burden and what is necessary 303 00:21:05.400 --> 00:21:10.759 complexity that actually achieves a beneficial outcome for patients, and 304 00:21:10.400 --> 00:21:14.720 where can we actually start cutting back to try and 305 00:21:14.759 --> 00:21:17.599 make sure that we are optimizing and minimalizing all of 306 00:21:17.599 --> 00:21:21.279 those efforts. So I'm going to talk through, in fairly 307 00:21:21.359 --> 00:21:23.960 vague detail, a little bit of what we mean in 308 00:21:23.960 --> 00:21:27.039 each one of these different elements of the triad. In 309 00:21:27.079 --> 00:21:29.599 the performance sector, which is largely my part of the 310 00:21:29.680 --> 00:21:34.279 organization portfolio strategy, we want to understand the mutual benefit 311 00:21:35.160 --> 00:21:38.599 through simulations and measurements of what do we actually expect 312 00:21:38.680 --> 00:21:43.160 the trial to perform across key milestones and how is 313 00:21:43.160 --> 00:21:46.680 this affecting the budget. So essentially, how do we do 314 00:21:46.720 --> 00:21:50.359 the trade offs in the trial performance factors that we 315 00:21:50.400 --> 00:21:53.680 are looking for and how does that influence financial decision 316 00:21:53.680 --> 00:21:59.000 making investment decisions across our portfolio. On the other hand, 317 00:21:59.039 --> 00:22:01.720 we want to make sure that we are actively reducing 318 00:22:01.799 --> 00:22:06.920 patient burden, reducing, minimizing and managing our site patient burdened 319 00:22:07.039 --> 00:22:10.559 our complexity, saying before that we know that some of 320 00:22:10.599 --> 00:22:13.119 this is necessary will always be there. There will never 321 00:22:13.160 --> 00:22:15.920 be a trial that doesn't have any burden. It would 322 00:22:15.960 --> 00:22:17.960 be nice if we could get there, but I don't 323 00:22:17.960 --> 00:22:21.200 think it's going to happen. And then we want to 324 00:22:21.240 --> 00:22:23.319 make sure, on the other hand, that we are actually 325 00:22:23.440 --> 00:22:28.279 using directly patient feedback from the get go, not simply 326 00:22:28.319 --> 00:22:31.440 as Ken said, like after the protocol is largely designed 327 00:22:31.440 --> 00:22:34.519 and then we tweak it if we can, or too often. 328 00:22:34.559 --> 00:22:36.640 I think we know that the trial design is often 329 00:22:36.720 --> 00:22:39.039 sort of said and doesn't change terribly much after we 330 00:22:39.079 --> 00:22:41.519 get that engagement. How do we make sure that it's 331 00:22:41.559 --> 00:22:43.680 there from the start and actually that we have an 332 00:22:43.680 --> 00:22:46.440 active learning loop so that the next trial design actively 333 00:22:46.440 --> 00:22:49.559 incorporates what we've already understood from different patient communities, rather 334 00:22:49.599 --> 00:22:52.240 than sort of going back and starting a new advisory 335 00:22:52.240 --> 00:22:55.079 boarder or asking the same questions again. So again, that's 336 00:22:55.160 --> 00:22:57.839 both easier and cheaper for us to do, and it's 337 00:22:57.839 --> 00:23:01.039 better for the patient community because we have the feedback 338 00:23:01.079 --> 00:23:03.039 that they get a little bit sick of us asking 339 00:23:03.039 --> 00:23:06.880 the same questions every single time. It doesn't have a 340 00:23:07.000 --> 00:23:12.799 terribly professional appeal to it. Some of the different measures 341 00:23:12.799 --> 00:23:15.279 here and this is not comprehensive, and I'm mixing across 342 00:23:15.440 --> 00:23:20.119 some key performance indicator type data points and some drivers 343 00:23:20.599 --> 00:23:23.079 of what is actually what is underneath that. But here 344 00:23:23.119 --> 00:23:25.720 are some of the things that are actively looking at. 345 00:23:26.160 --> 00:23:29.880 Many of them are measurable through the Tuft's work that 346 00:23:29.960 --> 00:23:33.000 Ken is presented. So we're in terms of the trial 347 00:23:33.039 --> 00:23:37.240 design measures, the additive patient and site burden scores. What 348 00:23:37.319 --> 00:23:41.039 I mean by additive is contextualized, it's against standard of care. 349 00:23:42.160 --> 00:23:46.079 So when you looked at what Ken was presenting an 350 00:23:46.119 --> 00:23:48.799 aggregate in terms of the total burden score, what we've 351 00:23:48.839 --> 00:23:51.039 done to try and actually make it more actionable by 352 00:23:51.079 --> 00:23:55.680 our clinical study teams is actually put against the standard 353 00:23:55.680 --> 00:23:57.759 of care. So what is the additional burden above that 354 00:23:57.839 --> 00:24:01.160 standard of care? And then against our folio? How does 355 00:24:01.200 --> 00:24:04.720 that compare to other studies within the same disease area 356 00:24:04.759 --> 00:24:07.880 or the same therapeutic area within our portfolio, and then 357 00:24:08.039 --> 00:24:11.039 how does that compare against other similar studies from other 358 00:24:11.079 --> 00:24:15.279 sponsors within that externally, so that we can really understand 359 00:24:15.559 --> 00:24:19.279 is this trial adding more burden than it's worth? Do 360 00:24:19.319 --> 00:24:23.759 we have a sufficient understanding of the disease and the 361 00:24:23.759 --> 00:24:27.400 potential benefit of this that it's actually worth the additive benefit. 362 00:24:27.920 --> 00:24:31.279 So the additive burden, on the other hand, we have, 363 00:24:31.440 --> 00:24:33.440 for example, we know that we need to collect the 364 00:24:33.480 --> 00:24:36.920 actual patient insights around their support needs, the additive burden 365 00:24:36.960 --> 00:24:40.160 from their perspective, not what we have measured it as 366 00:24:40.440 --> 00:24:44.200 using our internal expertise and our scientific russianale, but actually 367 00:24:44.200 --> 00:24:46.359 what do they think it is, because that's quite different 368 00:24:47.160 --> 00:24:49.559 and we need to make sure that that's taken sufficiently 369 00:24:49.559 --> 00:24:54.079 into account. So I'm happy to answer questions around how 370 00:24:54.119 --> 00:24:56.319 we are using these this. There's obviously a lot of 371 00:24:56.319 --> 00:24:59.720 different calculations in the background about how these relate to 372 00:24:59.759 --> 00:25:03.519 each so for us to say that it is relatively complex, 373 00:25:03.599 --> 00:25:05.720 but we have started to make this work for us 374 00:25:05.720 --> 00:25:09.000 in terms of influencing decisions across these different data points 375 00:25:10.160 --> 00:25:13.799 and this informs our learning loop. Now this is in 376 00:25:13.920 --> 00:25:18.200 company fairly early maturity. It's not fully embedded yet, it's 377 00:25:18.240 --> 00:25:22.119 in process, and I can't talk too many specifics, but essentially, 378 00:25:22.160 --> 00:25:24.799 as I said, we are making sure that the key 379 00:25:24.880 --> 00:25:29.680 insights from patients and healthcare providers and sites is actually 380 00:25:29.759 --> 00:25:34.279 informed from the start before you start designing the protocol, 381 00:25:35.200 --> 00:25:38.880 and that sort of goes into a study design tool 382 00:25:39.400 --> 00:25:44.759 technology against our portfolio and pharma strategy to actually allow 383 00:25:44.880 --> 00:25:48.200 us to simulate the likely performance of this trial design 384 00:25:48.640 --> 00:25:51.400 as we tweak it, so you can then live make 385 00:25:51.519 --> 00:25:56.400 changes and see the likely results of how does that 386 00:25:56.440 --> 00:25:59.599 perform or how would that likely perform? And we can 387 00:25:59.640 --> 00:26:02.000 adjust that for value, we can adjust that for speed, 388 00:26:02.000 --> 00:26:04.440 and we can make different trade off decisions if we 389 00:26:04.559 --> 00:26:07.920 need to make this happen faster, what would it cost 390 00:26:07.920 --> 00:26:10.839 in addition or what would the patient trade off be? 391 00:26:11.160 --> 00:26:13.240 Are we satisfied with that? If we take that back 392 00:26:13.279 --> 00:26:15.119 to patients, how would that actually look? How would they 393 00:26:15.119 --> 00:26:18.519 react if we talk about regulator interactions amongst all of 394 00:26:18.559 --> 00:26:21.559 this as well. So ipsyn colleagues talked about how does 395 00:26:21.599 --> 00:26:23.759 that all fit in and how do we make the 396 00:26:23.839 --> 00:26:28.440 right level of informed decisions? That is actually the third 397 00:26:28.440 --> 00:26:31.279 point as well, so I've to covered both. Then we 398 00:26:31.519 --> 00:26:34.440 follow the loop, so any data point that actually goes 399 00:26:34.440 --> 00:26:37.240 into actual we feedback into the system so we can 400 00:26:37.319 --> 00:26:41.079 understand did the scenario play out like we expected? Why? 401 00:26:41.279 --> 00:26:43.920