Speaker 1: Welcome back everybody. Today we're going to be looking at
Speaker 1: more statistics stuff. So let's take a look at we're
Speaker 1: looking at the meta analysis in a systematic review. Right now,
Speaker 1: a graduate student somewhere is running a T test on
Speaker 1: forty anxious teenagers, and then lab in Ohio someone else's
Speaker 1: in Melbourne doing the same thing with sixty kids. Neither
Speaker 1: a study on its own tells the field much. But
Speaker 1: put together with two hundred other studies like them, something
Speaker 1: starts to happen. A signal emerges from the noise, and
Speaker 1: that's what we're talking about today. Meta analysis and systematic
Speaker 1: reviews the tools that turns scattered findings.
Speaker 2: Into something a practitioner can use.
Speaker 1: A meta analysis is a statistical technique that combines the
Speaker 1: results of multiple studies into a single estimate of effects size.
Speaker 1: Think of it as a study of studies. If you
Speaker 1: wanted to know whether CBT reduces anxiety and adolescents compared
Speaker 1: to a wait list control, you wouldn't trust one study
Speaker 1: of four kids. You'd want to know what happens when
Speaker 1: you pull every high quality trial ever run on that question.
Speaker 1: That is a meta analysis. Closely related, often confused, is
Speaker 1: the systematic review a structured, transparent literatary review that follows
Speaker 1: a pre defined protocol to find, evaluate, and summarize all
Speaker 1: relevant studies on a question. It's this stigmatic review doesn't
Speaker 1: have to include a statistical pooling and meta analysis usually
Speaker 1: sits in society systematic review, but the review itself is
Speaker 1: the scaff folding think of Cochrane reviews, the gold standard
Speaker 1: example of systematic review methodology in medicine and increasingly in psychology.
Speaker 1: Before you even touch the data, you need a well
Speaker 1: formed question. That's where peico PICO comes in. A framework
Speaker 1: standing for population intervention, comparison, an outcome use the structure,
Speaker 1: a truly and a clearly answerable research question.
Speaker 2: Here's an example.
Speaker 1: The CBT reduced anxiety and adolescens compared to whitlist controls.
Speaker 2: Population is adolescents.
Speaker 1: That's the part p for PICO intervention is CBT comparison
Speaker 1: is the waitlist, and the outcome is an anxiety reduction.
Speaker 1: That's PICO and action and the exam loves testing whether
Speaker 1: you can identify each piece inside a study vignette. Once
Speaker 1: the question is set, researchers conduct a comprehensive lit search
Speaker 1: systematic effort to locate all relevant studies using multiple databases,
Speaker 1: great literature, and defined search terms to minimize seglection bias.
Speaker 1: If you only search public journals, you inherit their bias
Speaker 1: that leads into study selection applying predefined inclusion and exclusion criteria,
Speaker 1: often visualized through a prismal flow diagram and standardized chart
Speaker 1: documenting how many studies were identified, screened, excluded, and ultimately
Speaker 1: included in the analysis.
Speaker 2: Here's where those real statistical work begins.
Speaker 1: Researchers extract data and calculate effect sizes, measures of the
Speaker 1: magnitude of a relationship or difference, allowing comparison across studies
Speaker 1: that use different scales.
Speaker 2: We've talked about these before.
Speaker 1: The three you must know is Cohen's D and effects size,
Speaker 1: expressing the standardized difference between two group means, most often
Speaker 1: using psychology when comparing treatment versus control, like comparing anxiety
Speaker 1: scores between a CBT and a weightless group. Next, as
Speaker 1: are the correlation coefficient uses an effects size when the
Speaker 1: underlying research examined relationships between continuous variables, for example, a
Speaker 1: correlation between hours of sleep and depressive symptoms across studies.
Speaker 1: And finally, the odds ratio. Odds ratio is an effect
Speaker 1: size used for binary outcomes, expressing the odds of an
Speaker 1: event occurring in one group relative to another, typically seeing
Speaker 1: in psychofarm research comparing relapse rates between medication and placebo.
Speaker 1: Before any of these effects sizes get combined, studies undergo
Speaker 1: quality assessment evaluating studies risk of bias using structured chair
Speaker 1: lists checklists such as the Cochrane Risk of Bias tool,
Speaker 1: examining randomization, blinding, attrition, and conflicts of interest. They study
Speaker 1: with no randomization and heavy dropouts, gets weighed differently.
Speaker 2: Or excluded entirely.
Speaker 1: Here's where the exam gets philosophical about statistics. Researchers must
Speaker 1: choose a model. Fixed effects models assume all included studies
Speaker 1: are ms estimating a single true effect, with any differences
Speaker 1: between studies attributed purely to sampling error used when studies
Speaker 1: are similar in design, sample and setting. Contrast that with
Speaker 1: what they call a random effect model, which assume the
Speaker 1: true effects size varies across studies and explicitly accounts from
Speaker 1: between study heterogeneity, producing a more conservative.
Speaker 2: Generalizable estimate.
Speaker 1: If you picture a diverse, diverse set of studies with
Speaker 1: different populations and interventions, random effects is almost always the safer,
Speaker 1: more defensible choice. Each trip a P will reward that answer. Next,
Speaker 1: we come to variations or heterogeneity. The degree of variation
Speaker 1: and effect sizes across studies included in AMID analysis measures
Speaker 1: statistically rather than assumed. Now there's two tools to measure it,
Speaker 1: the Q statistic, which is a test determining whether the
Speaker 1: observed variation between studies exceeds will be expected by chance alone,
Speaker 1: and the ice square statistic and percentage indicating how much
Speaker 1: of the total variation across studies is due to real
Speaker 1: heterogeneity rather than random mirror or Values above fifty percent
Speaker 1: suggests moderate heterogeneity and above seventy five included suggest high heterogeneity.
Speaker 1: Picture of BENA analysis on mindfulness interventions the pool studies
Speaker 1: from prison populations, college students, and cancer patients high ice
Speaker 1: square there wouldn't be surprising at all when heterogeneity shows up.
Speaker 1: Researchers investigated through what they call ubgroup analysis, where they
Speaker 1: compare effect sizes across specific categorical groupings within the studies,
Speaker 1: such as clinical versus non clinical populations or short term
Speaker 1: versus long term therapy, to see whether the intervention works
Speaker 1: differently for different groups. A more flexible cousin is meta regression,
Speaker 1: statistical technique testing whether continuous study level variables such as
Speaker 1: medication dosage or average participant age predict the size of
Speaker 1: the effect, and it can test multiple moderators simultaneously, where
Speaker 1: subgroup analysis is limited to one at a time. Findings
Speaker 1: often get visualized to what they call a forest plot,
Speaker 1: a graphical display reach that is represented by a line
Speaker 1: reflecting its confidence interval in a box reflecting its effect size,
Speaker 1: with the overall effect shown as a diamond at the bottom.
Speaker 1: It's the single most recognizable image of meta analysis, and
Speaker 1: the E triple P will absolutely show you one and
Speaker 1: ask you to interpret it. Finally, there is no meta
Speaker 1: analysis is trustworthy without addressing publication bias, a tendency for
Speaker 1: studies of positive where significant findings be published more often
Speaker 1: than null results, skewing the available literature toward inflated effects.
Speaker 1: Detection mentions will include the funnel plot, a scatter plot
Speaker 1: of effects size against standard error where symmetry suggests no
Speaker 1: bias and asymmetry suggested may be present. An Egger's test
Speaker 1: statistical test that formally quantifies funnel plot asymmetry correction techniques
Speaker 1: include fail safe in the number of additional null results
Speaker 1: studies that would be needed to bring a significant fighting
Speaker 1: down to non significance, where a large fail safe en
Speaker 1: indicates robust results, and a trim fill a method that
Speaker 1: adjusts the funnel plot to estimate an account for studies
Speaker 1: like missing due to publication bias. Meta analysis is where
Speaker 1: psychology builds momentum.
Speaker 2: It's how thousands of all.
Speaker 1: Individually unremarkable studies combined to something loud enough to actually
Speaker 1: shape practice.
Speaker 2: You have about a couple of practice questions.
Speaker 1: Question number one beta analysis pools twelve RCTs examining CBT.
Speaker 2: For adolescent anxiety.
Speaker 1: The ICE square statistic is calculated at eighty two percent.
Speaker 2: What does this indicate?
Speaker 1: A The pull side pool effect size is not statifically significant.
Speaker 1: B there's high hydrogeneity across studies, suggesting a random effects
Speaker 1: model is more appropriate. Or see publication biases present and
Speaker 1: requires trim and fill correction.
Speaker 2: So let's go back over the question. Question is.
Speaker 1: Hope blustes hold on and then analysis pulled twelve RCTs
Speaker 1: examining CBT for adolescent anxiety. The I square statistic is
Speaker 1: calculating two percent there is high hydogeneity across studies.
Speaker 2: That's what this will show. Practice Question two.
Speaker 1: A researcher creates a funnel plot for a meta analysis
Speaker 1: on antidepressant efficacy and notices pronounced asymmetry with smaller studies
Speaker 1: clustering disproportionately on the side showing larger effects. A. This
Speaker 1: is a high between study hydogen eity, requiring meta regression. B.
Speaker 1: The study used inconsistent effects sized metrics or C publication
Speaker 1: bias for smaller studies, but no or negative findings, but
Speaker 1: likely we're less likely to be published again.
Speaker 2: The question.
Speaker 1: A researcher creates a funnel plot for a meta analysis
Speaker 1: on antidepressant efficacy and notices pronounced asymmetry with smaller studies
Speaker 1: clustering disproportionately on the side showing larger effects. If you
Speaker 1: said C publication bias, you're correct. Public based publication bias
Speaker 1: usually uses a funnel plot to look for symmetry. Asymmetrical
Speaker 1: funnel plots equal publication biases before forest plot is looking
Speaker 1: for heterogeneity, So a forest plot plus hi squared random
Speaker 1: effects model funnel plot plus asymmetry publication bias. Trim of
Speaker 1: PHIL is also a method used after publication bias is suspected.
Speaker 1: Asymmetry alone doesn't automatically mean it requires trimn phil.
Speaker 2: That's it for now.