WEBVTT

1
00:00:00.880 --> 00:00:03.200
<v Speaker 1>Welcome back everybody. Today we're going to be looking at

2
00:00:03.720 --> 00:00:09.919
<v Speaker 1>more statistics stuff. So let's take a look at we're

3
00:00:09.919 --> 00:00:16.440
<v Speaker 1>looking at the meta analysis in a systematic review. Right now,

4
00:00:16.440 --> 00:00:18.800
<v Speaker 1>a graduate student somewhere is running a T test on

5
00:00:18.920 --> 00:00:22.359
<v Speaker 1>forty anxious teenagers, and then lab in Ohio someone else's

6
00:00:22.359 --> 00:00:25.839
<v Speaker 1>in Melbourne doing the same thing with sixty kids. Neither

7
00:00:25.879 --> 00:00:28.519
<v Speaker 1>a study on its own tells the field much. But

8
00:00:28.600 --> 00:00:31.800
<v Speaker 1>put together with two hundred other studies like them, something

9
00:00:31.839 --> 00:00:35.320
<v Speaker 1>starts to happen. A signal emerges from the noise, and

10
00:00:35.320 --> 00:00:38.640
<v Speaker 1>that's what we're talking about today. Meta analysis and systematic

11
00:00:38.679 --> 00:00:41.840
<v Speaker 1>reviews the tools that turns scattered findings.

12
00:00:41.320 --> 00:00:43.399
<v Speaker 2>Into something a practitioner can use.

13
00:00:44.439 --> 00:00:47.280
<v Speaker 1>A meta analysis is a statistical technique that combines the

14
00:00:47.320 --> 00:00:50.840
<v Speaker 1>results of multiple studies into a single estimate of effects size.

15
00:00:51.359 --> 00:00:53.359
<v Speaker 1>Think of it as a study of studies. If you

16
00:00:53.359 --> 00:00:56.560
<v Speaker 1>wanted to know whether CBT reduces anxiety and adolescents compared

17
00:00:56.600 --> 00:00:59.640
<v Speaker 1>to a wait list control, you wouldn't trust one study

18
00:00:59.679 --> 00:01:01.840
<v Speaker 1>of four kids. You'd want to know what happens when

19
00:01:01.880 --> 00:01:05.040
<v Speaker 1>you pull every high quality trial ever run on that question.

20
00:01:06.040 --> 00:01:10.400
<v Speaker 1>That is a meta analysis. Closely related, often confused, is

21
00:01:10.400 --> 00:01:14.239
<v Speaker 1>the systematic review a structured, transparent literatary review that follows

22
00:01:14.239 --> 00:01:18.159
<v Speaker 1>a pre defined protocol to find, evaluate, and summarize all

23
00:01:18.239 --> 00:01:22.079
<v Speaker 1>relevant studies on a question. It's this stigmatic review doesn't

24
00:01:22.079 --> 00:01:25.719
<v Speaker 1>have to include a statistical pooling and meta analysis usually

25
00:01:25.719 --> 00:01:29.079
<v Speaker 1>sits in society systematic review, but the review itself is

26
00:01:29.079 --> 00:01:34.799
<v Speaker 1>the scaff folding think of Cochrane reviews, the gold standard

27
00:01:34.840 --> 00:01:39.079
<v Speaker 1>example of systematic review methodology in medicine and increasingly in psychology.

28
00:01:40.040 --> 00:01:41.879
<v Speaker 1>Before you even touch the data, you need a well

29
00:01:41.959 --> 00:01:47.200
<v Speaker 1>formed question. That's where peico PICO comes in. A framework

30
00:01:47.280 --> 00:01:52.120
<v Speaker 1>standing for population intervention, comparison, an outcome use the structure,

31
00:01:52.159 --> 00:01:54.920
<v Speaker 1>a truly and a clearly answerable research question.

32
00:01:56.319 --> 00:01:57.200
<v Speaker 2>Here's an example.

33
00:01:57.239 --> 00:02:01.519
<v Speaker 1>The CBT reduced anxiety and adolescens compared to whitlist controls.

34
00:02:01.640 --> 00:02:03.200
<v Speaker 2>Population is adolescents.

35
00:02:03.280 --> 00:02:07.560
<v Speaker 1>That's the part p for PICO intervention is CBT comparison

36
00:02:07.640 --> 00:02:10.719
<v Speaker 1>is the waitlist, and the outcome is an anxiety reduction.

37
00:02:11.560 --> 00:02:14.080
<v Speaker 1>That's PICO and action and the exam loves testing whether

38
00:02:14.120 --> 00:02:20.080
<v Speaker 1>you can identify each piece inside a study vignette. Once

39
00:02:20.120 --> 00:02:23.280
<v Speaker 1>the question is set, researchers conduct a comprehensive lit search

40
00:02:23.479 --> 00:02:28.639
<v Speaker 1>systematic effort to locate all relevant studies using multiple databases,

41
00:02:28.719 --> 00:02:32.479
<v Speaker 1>great literature, and defined search terms to minimize seglection bias.

42
00:02:35.240 --> 00:02:38.199
<v Speaker 1>If you only search public journals, you inherit their bias

43
00:02:38.960 --> 00:02:43.719
<v Speaker 1>that leads into study selection applying predefined inclusion and exclusion criteria,

44
00:02:44.400 --> 00:02:47.960
<v Speaker 1>often visualized through a prismal flow diagram and standardized chart

45
00:02:48.039 --> 00:02:51.560
<v Speaker 1>documenting how many studies were identified, screened, excluded, and ultimately

46
00:02:51.560 --> 00:02:52.800
<v Speaker 1>included in the analysis.

47
00:02:54.039 --> 00:02:56.120
<v Speaker 2>Here's where those real statistical work begins.

48
00:02:56.199 --> 00:03:00.879
<v Speaker 1>Researchers extract data and calculate effect sizes, measures of the

49
00:03:00.879 --> 00:03:04.400
<v Speaker 1>magnitude of a relationship or difference, allowing comparison across studies

50
00:03:04.400 --> 00:03:05.639
<v Speaker 1>that use different scales.

51
00:03:05.960 --> 00:03:07.280
<v Speaker 2>We've talked about these before.

52
00:03:07.919 --> 00:03:10.639
<v Speaker 1>The three you must know is Cohen's D and effects size,

53
00:03:10.680 --> 00:03:15.560
<v Speaker 1>expressing the standardized difference between two group means, most often

54
00:03:15.680 --> 00:03:19.520
<v Speaker 1>using psychology when comparing treatment versus control, like comparing anxiety

55
00:03:19.520 --> 00:03:22.560
<v Speaker 1>scores between a CBT and a weightless group. Next, as

56
00:03:22.639 --> 00:03:26.080
<v Speaker 1>are the correlation coefficient uses an effects size when the

57
00:03:26.120 --> 00:03:31.199
<v Speaker 1>underlying research examined relationships between continuous variables, for example, a

58
00:03:31.199 --> 00:03:35.639
<v Speaker 1>correlation between hours of sleep and depressive symptoms across studies.

59
00:03:37.520 --> 00:03:43.439
<v Speaker 1>And finally, the odds ratio. Odds ratio is an effect

60
00:03:43.479 --> 00:03:46.400
<v Speaker 1>size used for binary outcomes, expressing the odds of an

61
00:03:46.439 --> 00:03:50.080
<v Speaker 1>event occurring in one group relative to another, typically seeing

62
00:03:50.120 --> 00:03:54.280
<v Speaker 1>in psychofarm research comparing relapse rates between medication and placebo.

63
00:03:55.280 --> 00:03:58.199
<v Speaker 1>Before any of these effects sizes get combined, studies undergo

64
00:03:58.360 --> 00:04:02.199
<v Speaker 1>quality assessment evaluating studies risk of bias using structured chair

65
00:04:02.240 --> 00:04:06.159
<v Speaker 1>lists checklists such as the Cochrane Risk of Bias tool,

66
00:04:06.199 --> 00:04:11.159
<v Speaker 1>examining randomization, blinding, attrition, and conflicts of interest. They study

67
00:04:11.240 --> 00:04:15.120
<v Speaker 1>with no randomization and heavy dropouts, gets weighed differently.

68
00:04:14.919 --> 00:04:16.240
<v Speaker 2>Or excluded entirely.

69
00:04:18.040 --> 00:04:21.439
<v Speaker 1>Here's where the exam gets philosophical about statistics. Researchers must

70
00:04:21.519 --> 00:04:25.720
<v Speaker 1>choose a model. Fixed effects models assume all included studies

71
00:04:25.720 --> 00:04:29.560
<v Speaker 1>are ms estimating a single true effect, with any differences

72
00:04:29.560 --> 00:04:33.600
<v Speaker 1>between studies attributed purely to sampling error used when studies

73
00:04:33.600 --> 00:04:37.360
<v Speaker 1>are similar in design, sample and setting. Contrast that with

74
00:04:37.399 --> 00:04:39.680
<v Speaker 1>what they call a random effect model, which assume the

75
00:04:39.720 --> 00:04:43.160
<v Speaker 1>true effects size varies across studies and explicitly accounts from

76
00:04:43.199 --> 00:04:47.360
<v Speaker 1>between study heterogeneity, producing a more conservative.

77
00:04:47.199 --> 00:04:48.560
<v Speaker 2>Generalizable estimate.

78
00:04:49.800 --> 00:04:52.600
<v Speaker 1>If you picture a diverse, diverse set of studies with

79
00:04:52.680 --> 00:04:57.560
<v Speaker 1>different populations and interventions, random effects is almost always the safer,

80
00:04:57.639 --> 00:05:04.360
<v Speaker 1>more defensible choice. Each trip a P will reward that answer. Next,

81
00:05:04.360 --> 00:05:07.959
<v Speaker 1>we come to variations or heterogeneity. The degree of variation

82
00:05:08.040 --> 00:05:12.199
<v Speaker 1>and effect sizes across studies included in AMID analysis measures

83
00:05:12.199 --> 00:05:15.920
<v Speaker 1>statistically rather than assumed. Now there's two tools to measure it,

84
00:05:15.959 --> 00:05:19.079
<v Speaker 1>the Q statistic, which is a test determining whether the

85
00:05:19.600 --> 00:05:24.040
<v Speaker 1>observed variation between studies exceeds will be expected by chance alone,

86
00:05:24.600 --> 00:05:27.560
<v Speaker 1>and the ice square statistic and percentage indicating how much

87
00:05:27.560 --> 00:05:30.879
<v Speaker 1>of the total variation across studies is due to real

88
00:05:30.920 --> 00:05:35.279
<v Speaker 1>heterogeneity rather than random mirror or Values above fifty percent

89
00:05:35.319 --> 00:05:41.639
<v Speaker 1>suggests moderate heterogeneity and above seventy five included suggest high heterogeneity.

90
00:05:42.600 --> 00:05:46.279
<v Speaker 1>Picture of BENA analysis on mindfulness interventions the pool studies

91
00:05:46.279 --> 00:05:51.160
<v Speaker 1>from prison populations, college students, and cancer patients high ice

92
00:05:51.160 --> 00:05:57.920
<v Speaker 1>square there wouldn't be surprising at all when heterogeneity shows up.

93
00:05:58.000 --> 00:06:01.800
<v Speaker 1>Researchers investigated through what they call ubgroup analysis, where they

94
00:06:01.879 --> 00:06:06.360
<v Speaker 1>compare effect sizes across specific categorical groupings within the studies,

95
00:06:06.959 --> 00:06:10.639
<v Speaker 1>such as clinical versus non clinical populations or short term

96
00:06:10.720 --> 00:06:14.839
<v Speaker 1>versus long term therapy, to see whether the intervention works

97
00:06:14.879 --> 00:06:18.720
<v Speaker 1>differently for different groups. A more flexible cousin is meta regression,

98
00:06:18.879 --> 00:06:23.360
<v Speaker 1>statistical technique testing whether continuous study level variables such as

99
00:06:23.439 --> 00:06:28.480
<v Speaker 1>medication dosage or average participant age predict the size of

100
00:06:28.519 --> 00:06:33.040
<v Speaker 1>the effect, and it can test multiple moderators simultaneously, where

101
00:06:33.120 --> 00:06:39.240
<v Speaker 1>subgroup analysis is limited to one at a time. Findings

102
00:06:39.240 --> 00:06:42.160
<v Speaker 1>often get visualized to what they call a forest plot,

103
00:06:42.279 --> 00:06:44.959
<v Speaker 1>a graphical display reach that is represented by a line

104
00:06:44.959 --> 00:06:49.160
<v Speaker 1>reflecting its confidence interval in a box reflecting its effect size,

105
00:06:49.319 --> 00:06:52.240
<v Speaker 1>with the overall effect shown as a diamond at the bottom.

106
00:06:53.040 --> 00:06:56.680
<v Speaker 1>It's the single most recognizable image of meta analysis, and

107
00:06:56.720 --> 00:06:59.399
<v Speaker 1>the E triple P will absolutely show you one and

108
00:06:59.439 --> 00:07:05.839
<v Speaker 1>ask you to interpret it. Finally, there is no meta

109
00:07:05.879 --> 00:07:10.800
<v Speaker 1>analysis is trustworthy without addressing publication bias, a tendency for

110
00:07:10.839 --> 00:07:13.800
<v Speaker 1>studies of positive where significant findings be published more often

111
00:07:13.839 --> 00:07:18.120
<v Speaker 1>than null results, skewing the available literature toward inflated effects.

112
00:07:18.759 --> 00:07:21.720
<v Speaker 1>Detection mentions will include the funnel plot, a scatter plot

113
00:07:21.720 --> 00:07:25.319
<v Speaker 1>of effects size against standard error where symmetry suggests no

114
00:07:25.439 --> 00:07:30.000
<v Speaker 1>bias and asymmetry suggested may be present. An Egger's test

115
00:07:30.399 --> 00:07:36.399
<v Speaker 1>statistical test that formally quantifies funnel plot asymmetry correction techniques

116
00:07:36.399 --> 00:07:39.439
<v Speaker 1>include fail safe in the number of additional null results

117
00:07:39.480 --> 00:07:41.920
<v Speaker 1>studies that would be needed to bring a significant fighting

118
00:07:41.959 --> 00:07:45.480
<v Speaker 1>down to non significance, where a large fail safe en

119
00:07:45.600 --> 00:07:49.160
<v Speaker 1>indicates robust results, and a trim fill a method that

120
00:07:49.199 --> 00:07:51.720
<v Speaker 1>adjusts the funnel plot to estimate an account for studies

121
00:07:51.759 --> 00:07:57.319
<v Speaker 1>like missing due to publication bias. Meta analysis is where

122
00:07:57.360 --> 00:07:58.879
<v Speaker 1>psychology builds momentum.

123
00:07:58.879 --> 00:08:00.560
<v Speaker 2>It's how thousands of all.

124
00:08:00.319 --> 00:08:04.399
<v Speaker 1>Individually unremarkable studies combined to something loud enough to actually

125
00:08:04.439 --> 00:08:06.560
<v Speaker 1>shape practice.

126
00:08:07.160 --> 00:08:08.759
<v Speaker 2>You have about a couple of practice questions.

127
00:08:09.360 --> 00:08:15.759
<v Speaker 1>Question number one beta analysis pools twelve RCTs examining CBT.

128
00:08:19.800 --> 00:08:21.199
<v Speaker 2>For adolescent anxiety.

129
00:08:21.360 --> 00:08:24.600
<v Speaker 1>The ICE square statistic is calculated at eighty two percent.

130
00:08:25.399 --> 00:08:26.480
<v Speaker 2>What does this indicate?

131
00:08:28.399 --> 00:08:33.360
<v Speaker 1>A The pull side pool effect size is not statifically significant.

132
00:08:34.120 --> 00:08:38.600
<v Speaker 1>B there's high hydrogeneity across studies, suggesting a random effects

133
00:08:38.600 --> 00:08:42.639
<v Speaker 1>model is more appropriate. Or see publication biases present and

134
00:08:42.759 --> 00:08:44.759
<v Speaker 1>requires trim and fill correction.

135
00:08:47.200 --> 00:08:53.240
<v Speaker 2>So let's go back over the question. Question is.

136
00:08:57.399 --> 00:09:00.960
<v Speaker 1>Hope blustes hold on and then analysis pulled twelve RCTs

137
00:09:01.000 --> 00:09:04.360
<v Speaker 1>examining CBT for adolescent anxiety. The I square statistic is

138
00:09:04.399 --> 00:09:09.120
<v Speaker 1>calculating two percent there is high hydogeneity across studies.

139
00:09:09.159 --> 00:09:12.120
<v Speaker 2>That's what this will show. Practice Question two.

140
00:09:12.200 --> 00:09:15.080
<v Speaker 1>A researcher creates a funnel plot for a meta analysis

141
00:09:15.080 --> 00:09:20.360
<v Speaker 1>on antidepressant efficacy and notices pronounced asymmetry with smaller studies

142
00:09:20.360 --> 00:09:25.200
<v Speaker 1>clustering disproportionately on the side showing larger effects. A. This

143
00:09:25.240 --> 00:09:29.759
<v Speaker 1>is a high between study hydogen eity, requiring meta regression. B.

144
00:09:29.960 --> 00:09:33.919
<v Speaker 1>The study used inconsistent effects sized metrics or C publication

145
00:09:34.039 --> 00:09:36.559
<v Speaker 1>bias for smaller studies, but no or negative findings, but

146
00:09:36.840 --> 00:09:40.679
<v Speaker 1>likely we're less likely to be published again.

147
00:09:40.759 --> 00:09:41.200
<v Speaker 2>The question.

148
00:09:41.279 --> 00:09:43.960
<v Speaker 1>A researcher creates a funnel plot for a meta analysis

149
00:09:43.960 --> 00:09:49.120
<v Speaker 1>on antidepressant efficacy and notices pronounced asymmetry with smaller studies

150
00:09:49.120 --> 00:09:53.320
<v Speaker 1>clustering disproportionately on the side showing larger effects. If you

151
00:09:53.399 --> 00:10:01.720
<v Speaker 1>said C publication bias, you're correct. Public based publication bias

152
00:10:01.799 --> 00:10:06.399
<v Speaker 1>usually uses a funnel plot to look for symmetry. Asymmetrical

153
00:10:06.440 --> 00:10:13.120
<v Speaker 1>funnel plots equal publication biases before forest plot is looking

154
00:10:13.159 --> 00:10:17.840
<v Speaker 1>for heterogeneity, So a forest plot plus hi squared random

155
00:10:17.840 --> 00:10:24.080
<v Speaker 1>effects model funnel plot plus asymmetry publication bias. Trim of

156
00:10:24.240 --> 00:10:27.519
<v Speaker 1>PHIL is also a method used after publication bias is suspected.

157
00:10:27.960 --> 00:10:31.159
<v Speaker 1>Asymmetry alone doesn't automatically mean it requires trimn phil.

158
00:10:33.840 --> 00:10:34.600
<v Speaker 2>That's it for now.