WEBVTT 1 00:00:00.200 --> 00:00:03.720 Welcome to Farmer Talk Radio. This podcast is focused on 2 00:00:03.879 --> 00:00:08.599 enverching clinical trials with patient reported outcomes, discussing best practices, 3 00:00:08.800 --> 00:00:12.320 potential pitfalls, and new approaches from the twenty twenty five 4 00:00:12.480 --> 00:00:15.720 Chief Medical Officer summ at three sixty. For more information, 5 00:00:15.800 --> 00:00:20.199 I'm on the CMO Summit. Editorials, podcasts or webcasts, please 6 00:00:20.320 --> 00:00:24.199 visit CMO three sixty dot org. Thank you enjoy the podcast. 7 00:00:24.440 --> 00:00:27.519 So we'll start with some introductions and then get into 8 00:00:27.640 --> 00:00:30.760 the heart of the discussion. I'm Barry Tico, I'm chief 9 00:00:30.800 --> 00:00:34.479 Medical officer at Stoke Therapeutics, and we'll just go down 10 00:00:34.479 --> 00:00:37.479 the line here. Everybody introduce themselves in their affiliations. 11 00:00:38.520 --> 00:00:41.240 Hello everyone, My name is well Sam samble CeON, founder 12 00:00:41.280 --> 00:00:46.640 of delve Health. We focus on patient engagement pros wearables 13 00:00:46.640 --> 00:00:49.920 will probably have one of the more prominent set of 14 00:00:49.960 --> 00:00:52.799 wearables for collecting patient outcomes. 15 00:00:54.280 --> 00:00:54.560 Hi. 16 00:00:54.600 --> 00:00:59.200 I'm Dave Vessels. I'm the chief medical officer of Signant Health. 17 00:01:00.159 --> 00:01:04.040 We are a clinical endpoints company, so we focus on 18 00:01:04.840 --> 00:01:10.120 patient reported outcomes. Clinical reported outcomes. We have about two 19 00:01:10.120 --> 00:01:15.280 thousand employees, fifty of them are scientists and physicians, and 20 00:01:15.439 --> 00:01:20.560 we will we help our clients to select the right 21 00:01:20.920 --> 00:01:25.280 patient reported outcomes to implement them and then also to 22 00:01:25.319 --> 00:01:28.200 make sure during the trial that the quality of the 23 00:01:28.280 --> 00:01:32.079 data that you collect is of the highest standard that. 24 00:01:32.079 --> 00:01:32.640 It can be. 25 00:01:34.319 --> 00:01:37.200 Hi, good morning. My name is Natalie Agafonov. I'm chief 26 00:01:37.239 --> 00:01:41.840 medical officer of company and Drivon. We are clinical stage 27 00:01:41.920 --> 00:01:47.519 company phase two move into phase three dvincient therapy using 28 00:01:48.480 --> 00:01:52.879 bond marrow derived missing humans themselves. Our focus is a 29 00:01:53.239 --> 00:01:57.599 rare pediatric indications such as hyperplastic left heart syndrome and 30 00:01:57.640 --> 00:02:02.200 also we're working with Alzheimer disease. Over twenty years in 31 00:02:02.280 --> 00:02:06.000 clinical research, I develop really deep passion for the patient 32 00:02:06.040 --> 00:02:11.719 reported outcome, for the patient voice to shape up clinical 33 00:02:11.919 --> 00:02:15.319 outcome assessment. Very excited to have this conversation. 34 00:02:15.479 --> 00:02:15.759 Thanks. 35 00:02:17.240 --> 00:02:18.719 Hi. I'm Jason Sager. 36 00:02:18.840 --> 00:02:22.479 I am the chief medical officer of Accent Therapeutics. We're 37 00:02:22.520 --> 00:02:27.879 a small molecule clinical stage oncology company running two phase 38 00:02:27.919 --> 00:02:32.840 one for a modifying protein enzymes inhibition and really looking 39 00:02:32.879 --> 00:02:37.120 forward to deciding how to incorporate this in oncology because 40 00:02:37.120 --> 00:02:39.840 it's always been a challenge to look at pros and 41 00:02:40.240 --> 00:02:42.639 something that I think is getting more and more important 42 00:02:42.680 --> 00:02:43.520 as we move forward. 43 00:02:46.280 --> 00:02:50.120 Thank you everyone, So as Jason mentioned, PRO is becoming 44 00:02:50.159 --> 00:02:54.520 more and more commonplace. And let me just get a 45 00:02:54.599 --> 00:02:57.840 hands here of how many people have used the clinical 46 00:02:57.919 --> 00:03:02.199 trial PRO in one of the article trials that they've designed. Yeah, 47 00:03:02.240 --> 00:03:03.280 pretty much almost everyone. 48 00:03:03.319 --> 00:03:03.840 I expected. 49 00:03:03.879 --> 00:03:06.599 Now the question how many of used the PRO as 50 00:03:06.639 --> 00:03:08.120 a primary endpoint. 51 00:03:07.719 --> 00:03:08.759 In one of your trials? 52 00:03:10.120 --> 00:03:14.400 Okay, so it's already pretty good that they're that many, because, 53 00:03:15.000 --> 00:03:17.439 as we'll talk about, there are quite a few challenges 54 00:03:17.439 --> 00:03:20.960 in you using pros in general, but as primary endpoints 55 00:03:20.960 --> 00:03:24.000 they are especially challenging. But they're going to become more 56 00:03:24.000 --> 00:03:29.400 and more needed, both from the regulatory perspective, where right 57 00:03:29.439 --> 00:03:34.199 now probably about half of all trials have some PRO 58 00:03:34.479 --> 00:03:39.000 and in Europe about half of all labels include some 59 00:03:39.199 --> 00:03:43.039 PRO information. FDA is a little bit more reticent, so 60 00:03:43.840 --> 00:03:47.879 recently only about twenty percent of labels actually are including 61 00:03:48.439 --> 00:03:53.120 PRO information, but that's likely to change. We heard from 62 00:03:53.199 --> 00:03:57.159 ken Getz this morning about how trials are becoming complicated 63 00:03:57.199 --> 00:03:57.639 and their. 64 00:03:57.439 --> 00:03:58.400 More and more endpoints. 65 00:03:58.800 --> 00:04:02.719 Well, one of the main contributors to the additional endpoints 66 00:04:03.159 --> 00:04:05.719 is likely many of you know, are the pros that 67 00:04:05.759 --> 00:04:08.960 are being collected, but they're not even being collected for 68 00:04:09.000 --> 00:04:14.400 regulatory purposes. They're they're becoming, I think, even more required 69 00:04:14.719 --> 00:04:18.279 for payers, and that's part of why. So I'll tell 70 00:04:18.279 --> 00:04:21.399 you we have a Phase three program for genetic form 71 00:04:21.399 --> 00:04:27.519 of epilepsy called drave syndrome, and we have made our 72 00:04:27.560 --> 00:04:32.639 trial quite complicated because we're including additional endpoints just for 73 00:04:32.680 --> 00:04:38.519 the payers. One of them is a collection of the 74 00:04:38.560 --> 00:04:42.560 impact of the disease on the family and the parents 75 00:04:42.959 --> 00:04:46.800 work time, employment time. So I can tell you that 76 00:04:47.199 --> 00:04:51.000 there is no regulatory agency that cares about that, but 77 00:04:51.160 --> 00:04:54.279 the payers want to know how much time can this 78 00:04:54.319 --> 00:04:58.720 person be full time employed and pay premiums? And we've 79 00:04:58.720 --> 00:05:01.920 had to incorporate video is, we've had to incorporate EEGs, 80 00:05:02.199 --> 00:05:05.720 there are quite a few other patient reported outcomes that 81 00:05:05.759 --> 00:05:10.160 we've had to incorporate, especially for payers. So what I 82 00:05:10.279 --> 00:05:12.879 like to do is have the panel members talk briefly 83 00:05:12.920 --> 00:05:16.079 about one or two p r ohs that they've they've 84 00:05:16.319 --> 00:05:19.360 focused on. And I know we're going to hear about wearables, 85 00:05:19.360 --> 00:05:23.519 We're gonna hear about others that can give some advice 86 00:05:23.600 --> 00:05:26.680 to this audience as to how best to incorporate p 87 00:05:26.839 --> 00:05:29.399 rros into your clinical protocols. 88 00:05:31.800 --> 00:05:35.480 Yes, I mean so pros and co as our core 89 00:05:35.519 --> 00:05:38.480 capability for what we do. So we do we work 90 00:05:38.519 --> 00:05:43.040 across multiple disease dates. On cology Alzheimer probably two of 91 00:05:43.040 --> 00:05:46.800 the most sophisticated ones that we see. Compliance by far 92 00:05:46.920 --> 00:05:49.199 is probably one of the biggest issues that everybody deals with, 93 00:05:49.319 --> 00:05:54.639 especially if you're you know, well, really depends on how 94 00:05:54.639 --> 00:05:57.199 often you want to collect that that diaries and the 95 00:05:57.240 --> 00:05:58.000 p r os. 96 00:05:58.560 --> 00:05:59.920 And one of the most. 97 00:06:01.279 --> 00:06:05.079 Complex aspects that we see often is how many diaries 98 00:06:05.120 --> 00:06:07.639 are you actually trying to collect on a daily basis 99 00:06:07.720 --> 00:06:11.639 or on a weekly basis, you know, take bristle mile 100 00:06:11.680 --> 00:06:15.480 stool for example, right, you're trying to get compliance how 101 00:06:15.519 --> 00:06:17.680 many times the patient goes to the bathroom, especially in 102 00:06:17.759 --> 00:06:22.240 an oncology trial. So some of those aspects are complex, 103 00:06:22.439 --> 00:06:24.199 and one of the ways that we look at it 104 00:06:24.319 --> 00:06:27.839 is more of every patient is different, right, every environment 105 00:06:28.000 --> 00:06:31.800 and disease state is different. And even are you really 106 00:06:31.800 --> 00:06:35.800 trying to push out three pros a day for oncology patients? 107 00:06:35.839 --> 00:06:37.920 Cancer patients that go in for a treatment and go 108 00:06:38.000 --> 00:06:41.279 home and they're really sick, and yet we're complaining that 109 00:06:41.360 --> 00:06:44.319 compliance is really bad. So I think what we've seen 110 00:06:44.480 --> 00:06:48.079 is what models can we incorporate to help ease the 111 00:06:48.079 --> 00:06:51.879 patient burden, but also ease the site burden as well. 112 00:06:53.079 --> 00:06:56.079 And that can be from whether it's from a protocol design, 113 00:06:56.160 --> 00:07:00.759 whether it's from you know, patient engagement rules, et cetera. 114 00:07:03.040 --> 00:07:05.800 So from our side, it really depends on the therapeutic area. 115 00:07:05.959 --> 00:07:12.680 So if you look at at oncology for example, so 116 00:07:12.720 --> 00:07:17.279 we see that, I'm gonna let me just step that back. 117 00:07:17.360 --> 00:07:21.120 So one of the principles that I brought in earlier 118 00:07:21.120 --> 00:07:24.079 in my career that I still believe is that you 119 00:07:24.160 --> 00:07:27.199 need to make sure that you know what you're going 120 00:07:27.240 --> 00:07:29.600 to measure and why you're measuring it. We spoke a 121 00:07:29.600 --> 00:07:34.519 lot about the the earlier today about the complexity of 122 00:07:34.560 --> 00:07:38.399 clinical trials. Your phase three trial is not the time 123 00:07:38.600 --> 00:07:43.240 to test e pro You have to do it much 124 00:07:43.279 --> 00:07:46.040 earlier because you don't want to learn anything new in 125 00:07:46.079 --> 00:07:50.000 your phase three trials for that. For the rest of 126 00:07:50.040 --> 00:07:54.319 it's it depends so much about the patient situation, It 127 00:07:54.399 --> 00:07:59.639 depends about the environment that the patient is in, and 128 00:07:59.680 --> 00:08:02.000 then also it depends on the disease area for that. 129 00:08:02.040 --> 00:08:05.319 So it's it's something that we're going to have to 130 00:08:05.360 --> 00:08:08.920 bring out more and more in the future. But let's 131 00:08:08.959 --> 00:08:11.279 make sure that you just understand why you're doing it. 132 00:08:14.959 --> 00:08:21.560 I actually have the same idea to initiate thinking about 133 00:08:21.600 --> 00:08:24.519 the PRO as soon as possible and don't use phase 134 00:08:24.560 --> 00:08:30.360 three as a validation study for the PURO. I think 135 00:08:30.399 --> 00:08:33.960 we always have to be mindful that perro a very 136 00:08:34.000 --> 00:08:40.120 important tool to understand patient voice, to understand fit for purpose. Also, 137 00:08:42.320 --> 00:08:46.879 I think the biggest challenge is sometimes when you in 138 00:08:46.919 --> 00:08:51.799 phase two to understand this disease area, and even with 139 00:08:51.919 --> 00:08:55.759 the available tools, you can actually develop your PRO and 140 00:08:55.879 --> 00:08:59.759 validate during the phase two and incorporate and phase three, 141 00:09:00.159 --> 00:09:04.279 or in my experience, sometimes when you get to phase 142 00:09:04.320 --> 00:09:07.080 three and you have this aha moment, it's a little 143 00:09:07.120 --> 00:09:09.799 bit too late, and then you have to overcome this 144 00:09:09.879 --> 00:09:15.679 regulatory hurtle and be creative how to overcome that. 145 00:09:17.559 --> 00:09:19.639 Yeah, I mean in oncology, and I'm glad it's come 146 00:09:19.720 --> 00:09:22.120 up already from my perspective. 147 00:09:22.360 --> 00:09:23.600 You know, really things. 148 00:09:23.360 --> 00:09:28.200 Are we're moving away as we develop these molecular targeted 149 00:09:28.240 --> 00:09:31.559 precision medicines, especially the oral ones that are going to 150 00:09:31.600 --> 00:09:33.759 be used on a day by day basis for months 151 00:09:33.799 --> 00:09:36.600 at a time. You really need to move away from 152 00:09:36.679 --> 00:09:40.679 the model of the traditional dose limiting toxicity where you 153 00:09:40.720 --> 00:09:42.919 see in a cute dose it's reported to the doctor. 154 00:09:43.240 --> 00:09:45.440 You get into like sort of how do those patients 155 00:09:45.480 --> 00:09:48.440 feel on a chronic basis thing, and then there are 156 00:09:48.480 --> 00:09:51.320 those and you already mentioned diarrhea. There's only a handful 157 00:09:51.320 --> 00:09:55.840 of things, but fatigue, nausea, these things are not reported 158 00:09:55.879 --> 00:09:58.559 well and they're not reported well in the paper versions 159 00:09:58.600 --> 00:10:01.240 of the pros because of just you know, a lack 160 00:10:01.279 --> 00:10:03.039 of timing and you know, if you only get to 161 00:10:03.159 --> 00:10:05.559 see the patients once every three weeks, you're not going 162 00:10:05.600 --> 00:10:09.120 to find those things out. And so getting a tool 163 00:10:09.240 --> 00:10:12.919 into the patient's hands that you could be handheld on 164 00:10:12.960 --> 00:10:16.399 a handheld device and making it so that they benefit 165 00:10:16.559 --> 00:10:19.159 from using it with a report that comes to them 166 00:10:19.200 --> 00:10:21.799 that they can then bring to the doctor to actually 167 00:10:21.840 --> 00:10:25.200 ease the burden of that communication is something that we're 168 00:10:25.200 --> 00:10:28.759 looking into and working with a company called represent with 169 00:10:28.799 --> 00:10:32.000 a puro in there and to see if we can 170 00:10:32.039 --> 00:10:36.480 actually optimize that patient reporting from the get go. Just 171 00:10:36.519 --> 00:10:38.720 so you know, in oncology we now have to deal 172 00:10:38.759 --> 00:10:42.000 with project optimists and it's a huge question of how 173 00:10:42.039 --> 00:10:44.360 are you going to choose that second dose? And I 174 00:10:44.399 --> 00:10:46.720 won't get into that but I would love to see 175 00:10:46.799 --> 00:10:51.120 pros start to enable some of that decision process in 176 00:10:51.159 --> 00:10:51.879 a smart way. 177 00:10:52.279 --> 00:10:55.639 Yeah, it's interesting. We actually just published a paper on 178 00:10:55.720 --> 00:10:59.840 that exact idea. So and what we did is we 179 00:11:00.720 --> 00:11:05.039 postialized and this was together with industry we published it, 180 00:11:05.080 --> 00:11:10.720 and we postialize that that you actually miss a lot 181 00:11:10.799 --> 00:11:13.399 of the adverss event if you don't do it as 182 00:11:13.440 --> 00:11:17.399 part of a ePRO, and your dough selection becomes a 183 00:11:17.399 --> 00:11:19.519 problem later on. And if you look at the phase 184 00:11:19.559 --> 00:11:22.639 three trials, the majority if your phase three trials, you 185 00:11:22.759 --> 00:11:24.480 start it off and then you have to reduce the 186 00:11:24.519 --> 00:11:28.120 dose afterwards, and that's a huge cost burden to the 187 00:11:28.159 --> 00:11:31.240 trial that you have to put through, administrative burden for 188 00:11:31.320 --> 00:11:34.399 protocol amendments and safety reports, all these kind of things. 189 00:11:34.919 --> 00:11:39.000 So yeah, it's definitely there, and we bring your own 190 00:11:39.039 --> 00:11:43.679 devices the way that you can design e pros. It 191 00:11:43.840 --> 00:11:46.759 makes it really very easy for the patient to collect 192 00:11:46.799 --> 00:11:47.240 that data. 193 00:11:50.399 --> 00:11:54.440 So this is an important discussion the difference between paper 194 00:11:54.519 --> 00:11:57.360 and the e pros. And I'll tell you running an 195 00:11:57.399 --> 00:12:03.480 epilepsy trial, we collect daily seizure record from the families 196 00:12:04.360 --> 00:12:09.360 and this is a method that's been used now for 197 00:12:09.399 --> 00:12:14.519 decades we're still using paper. I just got a copy 198 00:12:14.600 --> 00:12:17.120 of the binder that the families are going to take 199 00:12:17.639 --> 00:12:21.759 paper home with and record every day, and it really 200 00:12:22.519 --> 00:12:25.759 disappoints me that that's actually the way we're going. But 201 00:12:25.919 --> 00:12:28.440 I tell you that we spent months and months trying 202 00:12:28.480 --> 00:12:30.960 to figure out how do we use an electronic diary. 203 00:12:31.240 --> 00:12:34.279 Do we give the family a device, do we make 204 00:12:34.399 --> 00:12:39.440 the family use their own device, what's the cost, the 205 00:12:39.480 --> 00:12:44.000 time for startup, and then all the regulatory requirements because 206 00:12:44.039 --> 00:12:48.440 that's an issue around the differences between what the FDA wants, 207 00:12:48.480 --> 00:12:52.080 EMA wants in terms of collected collection device. And then 208 00:12:53.000 --> 00:12:56.240 on top of it, one of the regulatory agencies wanted 209 00:12:56.240 --> 00:12:58.639 a paper copy to verify what we're collecting on the 210 00:12:58.639 --> 00:13:02.320 phones from the family. So after all that, we ended 211 00:13:02.399 --> 00:13:04.360 up with paper. But I'd like to hear a little 212 00:13:04.360 --> 00:13:08.120 bit from the panel on thoughts about trying to move 213 00:13:08.200 --> 00:13:12.840 us more into the electronic age as we go forward 214 00:13:12.879 --> 00:13:13.639 with our trials. 215 00:13:14.799 --> 00:13:17.159 Yeah, I've seen I think the first time that I 216 00:13:17.200 --> 00:13:21.480 saw a paper diary was back in twenty ten and 217 00:13:21.519 --> 00:13:24.799 it was an Alzheimer's study and it was interesting because 218 00:13:25.879 --> 00:13:29.679 I was managing a study for a larger company and 219 00:13:30.039 --> 00:13:32.519 as you talked to the investigator, they're talking about you know, 220 00:13:32.559 --> 00:13:34.679 they were actually explaining the situation. 221 00:13:34.720 --> 00:13:36.360 They're like, hey, I'm sitting here with this patient. 222 00:13:37.240 --> 00:13:41.159 And it was a patient and their caregiver and asked 223 00:13:41.200 --> 00:13:43.399 me like, hey, how was your pain like two days 224 00:13:43.399 --> 00:13:45.559 ago or something like that, and he's like it was 225 00:13:45.559 --> 00:13:48.159 a seven and then the caregivers like, are you sure 226 00:13:48.200 --> 00:13:50.279 it sounded more like a nine? Right, So it was 227 00:13:50.320 --> 00:13:53.360 like it was one of those initially interaction and then 228 00:13:53.399 --> 00:13:56.960 twenty sixteen, actually I started del because my mom had 229 00:13:57.039 --> 00:13:59.799 cancer and she was part of a study. 230 00:14:00.440 --> 00:14:03.399 And you know, like we go to Mayo Clinic. 231 00:14:03.120 --> 00:14:04.559 And they're like, oh my god, I didn't fill out 232 00:14:04.600 --> 00:14:06.679 all these pros. Right, So you said there in the 233 00:14:06.679 --> 00:14:09.480 parking lot filling out all the pros. But now, like 234 00:14:09.519 --> 00:14:11.399 we pass forward to kind of where we are today, 235 00:14:11.759 --> 00:14:14.879 we want more of that real time aspect what really 236 00:14:14.960 --> 00:14:17.240 happened to you. And that's the reason why a lot 237 00:14:17.240 --> 00:14:19.960 of prs are like a diary needs to be responded 238 00:14:20.000 --> 00:14:23.080 to within twenty four hours, maybe forty eight hours of max. 239 00:14:23.159 --> 00:14:25.039 Right of course, depending on the diary. 240 00:14:26.159 --> 00:14:30.159 And I think we're really seeing that, we're seeing the change, 241 00:14:30.200 --> 00:14:32.960 we're seeing the results, and what we're seeing from our 242 00:14:33.000 --> 00:14:36.399 perspective is how do we design the pros and make 243 00:14:36.440 --> 00:14:39.639 sure that if we're doing pr if we're doing BYOD, 244 00:14:39.759 --> 00:14:42.679 for example, we need to make sure that every single 245 00:14:42.759 --> 00:14:44.720 device that a patient's gonna pull up is going to 246 00:14:44.759 --> 00:14:47.240 look exactly the same in every single device. And that's 247 00:14:47.919 --> 00:14:50.039 you know, that's one of the you know, probably the 248 00:14:50.039 --> 00:14:54.440 biggest things that we've seen from regulatory agencies. So how 249 00:14:54.480 --> 00:14:58.399 do we make sure that we identify the pros design 250 00:14:58.440 --> 00:15:00.720 and make sure that they look exactly the same. But 251 00:15:01.440 --> 00:15:04.679 patient engagement also like a huge component as part of this. 252 00:15:04.759 --> 00:15:07.720 If we're really worried about compliance, we got to think 253 00:15:07.759 --> 00:15:10.200 that it's not just like a website that you're going 254 00:15:10.240 --> 00:15:12.159 to give to somebody, right, how are you going to 255 00:15:12.240 --> 00:15:14.679 nudge their make sure that they're aware, hey, you've got 256 00:15:14.679 --> 00:15:16.600 a pr O that's ready for you, or how do 257 00:15:16.600 --> 00:15:20.519 you nudge maybe like a caregiver and maybe that network 258 00:15:20.600 --> 00:15:25.600 of people around them. So technology is here, I think 259 00:15:25.639 --> 00:15:29.200 it's how can we really utilize it better in today's 260 00:15:29.200 --> 00:15:29.759 agent world? 261 00:15:30.679 --> 00:15:33.200 Ye, So we've been working with the Epilepsy Consortium on 262 00:15:33.440 --> 00:15:37.919 exactly developing that epilepsy diary. So we understand the challenges 263 00:15:37.960 --> 00:15:41.480 of it, but I mean there's and it sounds obvious 264 00:15:41.559 --> 00:15:45.879 coming from a ePRO, so it's a vendor. But there's 265 00:15:45.879 --> 00:15:48.519 already there's a lot of studies out there. There's a 266 00:15:48.559 --> 00:15:51.080 study that's quite old, I think it's more than fifteen 267 00:15:51.120 --> 00:15:54.559 years old, where it was quite eloquently designed. It was 268 00:15:54.559 --> 00:15:58.440 a paper diary. It was giving two hundred patients and 269 00:15:58.480 --> 00:16:01.639 they were told to complete it. But that wasn't the 270 00:16:01.679 --> 00:16:03.919 real study. The real study was there was a chip 271 00:16:03.960 --> 00:16:06.519 at each page and you can actually see when they 272 00:16:06.600 --> 00:16:12.840 open it and complete it. And something like eight percent 273 00:16:12.879 --> 00:16:16.840 of the patients finished the diary before the month, so 274 00:16:16.879 --> 00:16:20.399 they were just proactively writing out what they're going to do. 275 00:16:21.080 --> 00:16:23.600 The majority of the patients actually did it in the 276 00:16:23.600 --> 00:16:27.759 parking lot while they were waiting to come back, so 277 00:16:27.799 --> 00:16:30.679 you could see it all those dates times. So the 278 00:16:30.840 --> 00:16:36.240 challenge with paper, unfortunately is that we see that people 279 00:16:36.320 --> 00:16:41.320 don't I mean, people have busy lives, they forget to 280 00:16:41.360 --> 00:16:44.240 do this, and so that's one part of it. The 281 00:16:44.279 --> 00:16:47.759 other thing that we also see is that if you 282 00:16:47.879 --> 00:16:54.919 use electronic diaries, for example, you can identify abnormal patterns 283 00:16:54.960 --> 00:16:58.799 on when the data gets completed. And you can identify 284 00:16:58.960 --> 00:17:03.919 some sites that completes the data at very different times 285 00:17:03.960 --> 00:17:06.240 than the rest of their sites as well, and it's 286 00:17:06.240 --> 00:17:08.640 something that we don't talk a lot about, but it 287 00:17:08.720 --> 00:17:12.319 is out there. Clinical trials is expensive, there's a lot 288 00:17:12.319 --> 00:17:14.839 of money in there, and there is still fraud that's 289 00:17:14.839 --> 00:17:20.920 happening there. But basically coming back to the epilepsy part, 290 00:17:21.039 --> 00:17:23.480 so our approach is to work worth a consult with 291 00:17:23.559 --> 00:17:26.640 the epilepsy consort, Tium or the epilepsy group and just 292 00:17:26.759 --> 00:17:31.240 work through all these processes with regards to timelines, setups 293 00:17:31.240 --> 00:17:34.160 and so on. AI has really made a difference in 294 00:17:34.200 --> 00:17:38.319 that and where we see the biggest the biggest difference 295 00:17:38.400 --> 00:17:40.359 is actually in the user testing because you can now 296 00:17:40.400 --> 00:17:43.279 do in a few minutes, you can do ten thousand 297 00:17:43.400 --> 00:17:47.160 tests and then you can identify exactly where the problems 298 00:17:47.160 --> 00:17:50.279 are and fix it very accurately. So it improves your 299 00:17:50.359 --> 00:17:56.960 quality and you don't have to get any programming down 300 00:17:57.480 --> 00:18:00.400 to one day or something like that. You just need 301 00:18:00.440 --> 00:18:03.559 to get it off your critical path and that's already 302 00:18:03.559 --> 00:18:06.319 been done. So there's a huge movement with that in 303 00:18:06.359 --> 00:18:08.440 the last year and you'll see more and more of that. 304 00:18:11.480 --> 00:18:14.720 Yeah, it's very challenging and and as you, I would 305 00:18:14.759 --> 00:18:17.680 just want to emphasize the al Zeimer disease especially. It's 306 00:18:17.799 --> 00:18:21.519 very surprising to me that not only patients but also 307 00:18:21.599 --> 00:18:28.240 caregivers have difficulties to work with electronic data. But I 308 00:18:28.279 --> 00:18:30.720 know we have issues and problems. I want to bring 309 00:18:30.799 --> 00:18:33.240 fresh idea. If it's if I made that we have 310 00:18:33.559 --> 00:18:37.880 AI people in a room. A couple of days ago, 311 00:18:37.920 --> 00:18:41.440 I just visited MIT Museum and there is an amazing 312 00:18:41.559 --> 00:18:46.160 tool predict whether or not you have opportunity to have 313 00:18:46.240 --> 00:18:50.680 Parkinson disease based on how you type, how many letters 314 00:18:50.759 --> 00:18:54.039 you miss. I think in the future, it just brought 315 00:18:55.519 --> 00:18:58.759 just idea came to my mind. We can actually use 316 00:19:00.119 --> 00:19:06.160 difficulties for patient to complete data electronically and predict their 317 00:19:06.720 --> 00:19:10.480 severity based on how patient type. So it's just an idea, 318 00:19:11.319 --> 00:19:11.920 let's do it. 319 00:19:14.039 --> 00:19:17.440 So the way I've been trying to get at this 320 00:19:17.559 --> 00:19:22.279 is when we had discussions because obviously your safety in 321 00:19:22.359 --> 00:19:28.160 phase one for oncology is the primary goal, and yet 322 00:19:28.720 --> 00:19:32.960 it's very tightly regulated, and so that's where you know 323 00:19:32.960 --> 00:19:35.079 there's a lot of questions, Well, what if the pro 324 00:19:35.319 --> 00:19:39.440 database doesn't match your hospital database, what's the after you're 325 00:19:39.440 --> 00:19:41.359 going to say to us, and all these sorts of things. 326 00:19:41.559 --> 00:19:43.519 At the end of the day, what it came down 327 00:19:43.599 --> 00:19:46.920 for us is to basically slip it into an exploratory 328 00:19:47.000 --> 00:19:49.599 endpoint in phase one. I think in sort of in 329 00:19:49.759 --> 00:19:53.599 honor of don't wait until phase three, let's learn about 330 00:19:53.599 --> 00:19:55.279 what the signal is, but let's do it in a 331 00:19:55.319 --> 00:19:59.160 way that's more flexible, where we have the opportunity to 332 00:19:59.200 --> 00:20:03.759 do an pro such that once it's validated and you 333 00:20:03.799 --> 00:20:06.519 start to get that data, you can then escalate it 334 00:20:06.559 --> 00:20:08.799 as you go through so that by the time you're 335 00:20:08.799 --> 00:20:11.599 in phase three, you can make it a primary endpoint 336 00:20:11.680 --> 00:20:15.160 or something that is akin to something that's going to 337 00:20:15.160 --> 00:20:15.960 get into your label. 338 00:20:19.319 --> 00:20:20.119 One of ken. 339 00:20:19.960 --> 00:20:26.079 Gets his favorites the exploratory endpoints, right. It's complexifying exactly, 340 00:20:27.680 --> 00:20:30.079 and well, I'd like to turn a little bit two 341 00:20:30.119 --> 00:20:35.359 wearables now and just where you as panel members see 342 00:20:35.559 --> 00:20:38.960 from those of you who've had experience with wearables, how 343 00:20:39.000 --> 00:20:40.640 are we going to be able to incorporate those? Are 344 00:20:41.000 --> 00:20:43.720 those going to be used more and more or are 345 00:20:43.799 --> 00:20:46.559 they too complex for us to utilize at this point, 346 00:20:46.640 --> 00:20:49.440 especially when you consider the number of data points that 347 00:20:49.599 --> 00:20:56.440 was talked about before. One wearable one heart monitor can 348 00:20:56.519 --> 00:21:00.920 generate five million data points in a day. So when 349 00:21:00.960 --> 00:21:04.480 you're thinking about data complexity and the data we need 350 00:21:04.519 --> 00:21:07.839 to collect, how are we going to start using wearables 351 00:21:08.160 --> 00:21:09.519 efficiently in our studies? 352 00:21:12.359 --> 00:21:18.799 So we live and breede wearables almost every single day, 353 00:21:19.039 --> 00:21:22.799 you know, we've got we have had studies that do 354 00:21:23.240 --> 00:21:26.400 as much as five wearables on the same patient, and 355 00:21:26.880 --> 00:21:30.880 we've had as little as one. I think the time 356 00:21:31.000 --> 00:21:34.400 is coming for wearables. We're definitely seeing that wearable data 357 00:21:34.480 --> 00:21:37.799 can enhance pr O data. Right, it's all part of 358 00:21:38.000 --> 00:21:41.640 patient outcomes. Right, you've got the pr O data here's 359 00:21:41.680 --> 00:21:44.279 how I feel. Then you've got the wearable data that 360 00:21:44.319 --> 00:21:49.400 tells us here's what really happened at home. There's been 361 00:21:49.440 --> 00:21:52.240 a couple of studies, probably a few studies that have 362 00:21:52.440 --> 00:21:55.720 used wearables as a primary endpoint. I don't think it's 363 00:21:55.720 --> 00:21:58.079 not I don't think it is for every study, at 364 00:21:58.160 --> 00:22:02.519 least not yet, but they're here. If you're considering it, 365 00:22:02.559 --> 00:22:05.279 I'll probably go back to the same mentality as use 366 00:22:05.319 --> 00:22:06.960 it on a phase one or a phase two, so 367 00:22:07.039 --> 00:22:11.400 that way you understand how wearables work in your clinical 368 00:22:11.440 --> 00:22:14.519 trial and think at the end of mind, Right, what 369 00:22:14.680 --> 00:22:17.880 exactly are you trying to understand is it their movement, 370 00:22:18.039 --> 00:22:21.759 is it their sleep patterns or you know, And you 371 00:22:21.839 --> 00:22:25.519 have to think about it as patterns over time rather 372 00:22:25.599 --> 00:22:28.319 than just one data point for a day. 373 00:22:28.559 --> 00:22:28.759 Right. 374 00:22:28.920 --> 00:22:33.079 So it's a different mentality, different mind shift from the 375 00:22:33.519 --> 00:22:36.839 PR data. I think it's but I think it's coming. 376 00:22:36.920 --> 00:22:41.680 We're seeing definitely a lot more interests in almost every 377 00:22:41.799 --> 00:22:43.000 single disease state. 378 00:22:45.039 --> 00:22:49.759 Yeah, I think that's very very valid. We're learning a 379 00:22:49.759 --> 00:22:53.720 lot st all about wearables. Were currently just starting a 380 00:22:53.799 --> 00:22:58.759 study with Lead University where we are looking at wearables 381 00:22:59.119 --> 00:23:03.160 e pros and then also some of the hard biomarkers 382 00:23:03.240 --> 00:23:08.799 genetic testing, et cetera in elderly patients with cordiovascular disease 383 00:23:09.000 --> 00:23:14.880 and diabetes. And there's some glucose monitors, etc. And then 384 00:23:15.279 --> 00:23:18.880 heart monitors. It's a lot of data and again you 385 00:23:18.920 --> 00:23:21.680 need to understand why are you doing that? And then 386 00:23:21.839 --> 00:23:24.440 secondly you just have to think about the practical part 387 00:23:24.519 --> 00:23:28.480 of it is how are these patients going to remember 388 00:23:28.559 --> 00:23:30.960 to charge the device, how are they going to wear 389 00:23:31.000 --> 00:23:33.279 the device every day? How are they going to forget 390 00:23:33.319 --> 00:23:36.960 about the device? And all those questions. You have to 391 00:23:37.000 --> 00:23:39.839 sit down and really think about it before you come 392 00:23:39.920 --> 00:23:42.400 up with yeah, I'm going to implement this on my 393 00:23:42.519 --> 00:23:46.279 study or not. It's otherwise it just becomes another shiny 394 00:23:46.319 --> 00:23:49.440 tool and then we end up with massive amount of 395 00:23:49.519 --> 00:23:51.920 data that we don't know what to do with. 396 00:23:52.599 --> 00:23:59.000 So I see this as a future, and especially working 397 00:23:59.119 --> 00:24:03.400 with pediatric rare diseases. One all of our pros basically 398 00:24:03.440 --> 00:24:05.960 come from parents the infants. 399 00:24:06.039 --> 00:24:07.160 We inject. 400 00:24:08.359 --> 00:24:12.480 Cells during the second stage sogery when the infants are 401 00:24:12.519 --> 00:24:16.519