Speaker 2: Welcome back everybody.
Speaker 1: Today we're going to be looking at assessments, research methods,
and statistics for today's or this week's study guide. Will
create questions for you towards the end of the week
to review it. Hopefully that you like the new style.
We go over a topic and then during the middle
of the week we will release ten to fifteen practice
questions to kind of review it. So let's look at
statistics and research methods, so research designs, that's the framework
of a study. You must be able to distinguish between
an experimental study, quasi experimental, and correlational designs, specifically focusing
on their ability to establish causality versus mere relationships. Also,
you need to know statistical significance p values and measure
the probability that an observed difference occurred by chance limitations
and statistically a significant result.
Speaker 2: Point zero five or less than point zero five.
Speaker 1: It does not mean the finding is practically important in
a clinical sety what it does. What does say that
is the effect size, sometimes called the clinical significance, so.
Speaker 2: Standardized measure the magnitude or strength of an effect.
Speaker 1: This tells you the protocol significance of a treatment, validity,
the accuracy.
Speaker 2: And truthfulness of a study's conclusions.
Speaker 1: Threats to validity or methodological flaws that compromise your findings.
Another key triple P threat includes history, external events, altering outcomes, maturation,
natural changes and participants over a course of time, selection bias.
Speaker 2: Something else that.
Speaker 1: Can affect the threats to validity is systematic differences between
groups at the start. Ethical research practices safeguards protecting participants.
Essential components include informed consent, confidentiality protections, minimizing risk, and
the regulatory oversight of IRBs. Statistical tests mathematical tools used
to analyze data. You must known't when to apply specific
ten tests based on your variables. A T tests or
in a NOVA is used when comparing group means, comparing
treatment versus control. Correlation and regression use when looking for
continuous relationships or predicting outcomes. KI square used with analyzing
categorical data counting the number of patients who drop out
versus finish therapy.
Speaker 2: Descriptive statistics.
Speaker 1: Numerical techniques used to summarize, organize, and describe the basic
features of a data set, so you'd be looking at
things like the mean median, the mode, standard deviations Z
scores things like that inferential statistics techniques that allow researchers
to use sample data to make generalizations, estimates, or predictions
about a larger population.
Speaker 2: Measurement scores.
Speaker 1: The classification system for how variables are categorized and measured.
Speaker 2: Four types nominal, which.
Speaker 1: Include things like categories such as diagnosis. For instance, Ordinal
is ordered by ranks, so symptoms severity like mild, moderate,
or severe intervals ordered categories with equal distance so there's
no true zero, like IQ and ratio, which equal distance
with a true absolute zero reaction time and weight number
of Another one is research design comparisons, so if you're
looking at true experimental design. The gold standard for establishing
causal relationships the randomized control trial.
Speaker 2: It requires three core features.
Speaker 1: Random assignment, manipulation of an independent variable, and a control
group or condition to minimize extraneous variables. The strengths to
have high internal validity. Man validity is how much is
a measure what it's supposed to measure, and you have
a high internal validity effectively controls for confounding variables other
factors that can cause problems. Limitations can reduce external validity
due to.
Speaker 2: High art official laboratory settings.
Speaker 1: In other words, how does this apply outside the generalizability
is compromised on racts or these random assignment or the
true experimental design. It also faces strict ethical and practical constraints.
You cannot randomly assign someone to experience trauma or abuse.
Let me give you an example, testing whether a new
CBT protocol reduces panic symptoms by randomly assigning fifty padic
disorder patients to either the CB two group or a
waitless control group and then measuring their panic levels but
not work definition of research design that seeks to infer
a causal relationship but lacks the crucial feature of random assignment.
This is a quasi experimental design. Strengths more practical and ethical.
Speaker 2: For real world clinical.
Speaker 1: And educational settings, so higher external validity than true experiments
because it allows you again to measure what's going on.
Limitations lower internal validity due to selection bias, requires more
rigorous statistical control to rule out confounds. So with quasis,
some of the important parts of it again is that
it seeks to infer a causal relation but lacks the
crucial feature of random assignment. That's it's going to hurt
a little bit the internal validity, and the external validity
is a little bit better because you're using it in a
more practical settings than true experiments. The text highlights three
specific quasi experimental variations. Non equivalent control group design. Comparing
two pre existing non randomized groups where one gets a
treatment and the other doesn't.
Speaker 2: You've got pre test.
Speaker 1: Post test measures are used to catch baseline differences in
non equivalent control group designs. An example is comparing two
different local schools, one that naturally decides to adopt a
mindfulness curriculum and wine that does not to see the
impact on student anxiety. Time series design taking multiple observations
or measurements before and after and interventions introduced. It tracks
trends over time and is highly useful in policy or
organizational settings. In addition to that good blost my train
of thought there, sorry, folks. Tracking emergency as an example,
tracking emergency room psychiatric emissions every month for two years
before and two years after a community wide violence reduction
initiative is launched. Number three is regression discontinuity design. It's
a design where participants are assigned to a treatment or
control condition based entirely on whether this score falls above
or below a predetermined cutoff score. It allows for strong
causal inference if its strict mathematical assumptions are met. An
example would be testing the impact of an intensive academic
intervention where only students who scored below a seventy on
a standardized receive the tutoring, with those scoring above attack
the control group. Let's take a look at non experimental
designs and see what we come up with. On this
side non experimental designs, one of them is it's called
it's defined by observational research, where the investigator does not
intervene or manipulate variables. These are excellent for identifying relationships, trends,
or rare cases, but cannot prove causation. The text outlines
for major non experimental types. So again, observational research includes
correlational studies, but not all observational research is coreational studies,
but all correlational studies are observational research. Speaking of correlational
studies designed to measure the relationship between variables without any manipulation,
the core metric is that pearsons are to quantify the
strength and direction of the relationship. A plus one point
zero is a perfect both heading in the same direction
and verse relationship is a negative one point zero is
perfect limitations. Again, correlation is not equal causation. You also
have an issue with directionality.
Speaker 2: Does X cause Y? Or does Y cause x?
Speaker 1: And then you have third variable problems does X cause
both X and.
Speaker 2: Y, which can easily mislead researchers.
Speaker 1: An example well be finding a strong positive correlation between
a patient's sleep quality and their weekly academic exam performance.
Is an example of a correlational study. Case studies and
an in depth exploration of one individual group of event
a trade off. It provides rich detail but suffers from
very low generalizability. Example, extensively studying document and documenting memory
and behavioral changes in a single patient who sustained localized
hippocampal damage. Naturalistic observation the practice of observing behavior and
natural settings without interference or manipulation by the researcher. So
it's to the utility, highly useful in developmental animal across
cultural research. A trade off, very hard to control variables,
but both incredibly high ecological validity, or other words, real
world truthfulness. Survey research is the method they use is
self report to gather data on beliefs behaviors and attitudes
a trade off. It's highly efficient for large samples, but
uniquely vulnerable to bias in accuracy and low response rates.
The design factors elements like question wording, order effects, and
sampling strategy heavily impact the overall quality and validity of
the data. We have advanced design structures cross sectional versus longitudinal.
A cross sectional really common in the world of neuroscience.
The study design it takes a snapshot of multiple groups
at one point in time. It is fast and cost effective,
but cannot track individual developmental trajectories and causation is also problematic.
You see a lot of neuroscience where they take imaging
on the brain and they say, oh, look, people with
PTSD or depression have this kind of brain. But it's
only in the snapshow. We don't know if their brain
was like that before. Did the depression cause it that
the brain changes caused the depression, so reverse.
Speaker 2: Causality could be an issue.
Speaker 1: Another example of a cross sectionals comparing current anxiety levels
between a group of teenagers and a group of adult
older adults in May of twenty twenty six, so again one.
Speaker 2: Point in time.
Speaker 1: A longitudinal study designed that tracks the same group over time.
It is uniquely capable of revealing development, stability, and causality.
The down size is high risk of participant attrition, a
lot of them drop out, high costs keeps a lot
of the from ever getting started, and it also requires
a massive time investment. Examples measuring and tracking changes and
self esteem within the same exact group of individuals annually
from ages thirteen through nineteen. Some other insights between subjects
versus within subjects. Between subjects where different participants are assigned
to each condition, the benefit completely eliminates carryover effects, which
is where participating in one condition taints performance in the next.
Speaker 2: A risk can.
Speaker 1: Increase variability between groups due to individual differences. With thin
subjects a design where the same participants experience all conditions,
the benefit is that crucially reduces error due to individual differences.
Because every participant acts as their own baseline control, there
is a high risk of order effects, fatigue, practice, boredom.
This risk is systematically solved with counterbalancing though randomizing or
alternating the order.
Speaker 2: In which conditions are presented.
Speaker 1: Let me give you an example, testing a person's mood
after listening to classical music versus metal music and a
between groups design, you would have group A hearing classical
and group B hearing metal. And within subjects design, everyone
hears classical on day one, everyone on day two, here's
metal and with half counterbalance to hear metal first. Others
are factorial designs. These are research designs used to examine
multiple independent variables and their interactions simultaneously. So how you
expressed via numbers like two times two equals two independent
variables each with two levels. It's excellent for examining how
variables combine to influence psychological inco outcomes. On the E
triple P, you must remember that both the main effects
and the interaction effects should be interpreted carefully to understand
how independent variables operate, both separately and jointly. Give me
give you an example or study looking at whether treatment
types CBT versus medication interacts with age group adolescent versus
adult to affect depression scores. Heading over to validity trade
offs in selection, So internal validity did the independent variable
cause the change and the dependent variable? It gives you
confidence and causality, but confounding variable selection bias, attrition history, maturation,
external validity confinings generalize to other people in settings, sampling, bias,
artificial settings, reactivity the core trade off as lab studies
gained control, boosting internal validity, but may sacrifice generalizability, lowering
external validity. Conversely, field studies studies do the opposite. So
how do you match designs to research questions? Well, when
you're choosing one, you always ask what's the question, what's feasible,
what's ethical, and what's valid.
Speaker 2: For instance, if you want to prove.
Speaker 1: Want to prove cause, I guess this is a question
you have to ask yourself. Experimental can't randomize, but need
causal inference. All right, let's take a deeper dive into
the validity section here. So validity refers to get to
how accurate, trustworthy, meaningful the conclusions of a study are,
and your researchers are always balancing control, realism, ethics, and generalizability,
so internal validity. Some of the major threats for internal
validity confounding variables, selection, bias, history, maturation, attrition, and testing effects.
An example, did CBT actually reduce depression or patients also
starting medication at the same time? For external validity, can
the findings generalize to other people? Some of the major
threats are sampling bias, artificial settings, participant reactivity, and narrow demographics.
It laps that are using only college freshmen may not
apply to elderly patients with depression.
Speaker 2: What about construct validity?
Speaker 1: Did the research or accurately measure the psychological concept they
intended to measure.
Speaker 2: Some of the threats include.
Speaker 1: Poor operational definitions, measurement error, social desirability bias, and examples.
Does a stress questionnaire actually measure stress or just general negativity.
Another one is statistical conclusion validity, where the statistical conclusions
accurate or appropriate. Some of the threats low power, unreliable measures,
multiple comparisons, restricted range. A study may fail to detect
the real effect because the sample size was just too small.
Internal validity is the gold standard for causal inference. The
higher the internal validity, the more confident researchers have that
the independent variable cause the dependent variable, the findings are
not due to chance, and alternative explanations have been ruled
out to the best of their ability. High internal validity
often requires random assignment, experimental control, standardized procedures, control groups,
and elimination as the many confounds as possible. Some of
the major threats were mentioned confounding variables and confounds an
uncontrolled factor that changes alongside the independent variable and may
explain the results. So an example, a psychologist studies whether
meditation reduces anxiety. However, pticipants of the meditation group also
begin exercising more frequently.
Speaker 2: Now has become unclear.
Speaker 1: Was anxiety reduced because of meditation, exercise or the combination.
The exercise variable itself becomes a confound selection Bruse groups
differ before the study begins. The school introduces a new
reading intervention only in honors classrooms. Well, if students improve,
the improvement may reflect higher baseline intelligence already, greater motivation,
or family support, rather than the intervention itself.
Speaker 2: That's why you need a more diverse group.
Speaker 1: History effects, external events occurring during the study influence outcomes.
For instance, researchers examine stress levels during a workplace intervention program,
but halfway through the study the company announces layoffs. Stress
changes may reflect the layoffs and not the intervention. Another one,
participants naturally change over time. Children improve on emotional regulation
after six months of therapy is an example, But children
naturally mature emotionally over time anyway, making it difficult to
isolate the treatment effect, and finally attrition participants drop out
unevenly across groups. Example, and a substance abuse treatment study,
the most severe participants leave treatment early. The remaining sample
may falsely make the treatment appear more effective. External validity
is another way of saying generalizability.
Speaker 2: We talked about this earlier.
Speaker 1: Researchers so as a study may have excellent internal validity
but poor real world relevance, so researchers would ask would
these findings apply in real clinical settings, different cultures.
Speaker 2: Age groups, etc.
Speaker 1: Some of the threats during external validity as sampling bias.
The sample does not represent the broader population. A For instance,
a study on burnout includes only affluent private practice psychologists.
This one, in general may not generalize the community clinics, hospitals,
or graduates. Artificial settings our laboratory environments may not reflect
real life behavior, so participants may behave differently discussing trauma
in a lab than they would in actual therapy sessions
and reactivity participants alter behavior because they know they're being studied,
sometimes called the Hawthorn effect. People may report eating healthier
during nutrition research because they want to appear socially desirable
construct validity examines whether the researcher truly measure the intended
psychological construct. This is especially important in psychology because constructs
like intelligence, depression, resilience cannot be directly observed. They must
be operationalized through measurable indicators. Example of poor construct vibility
is a research or claims to measure academic motivation by
counting class attendances only, but attendants can reflect parental pressure,
transportation access, or fear of punishment. The operational definition poorly
can aptures the intended construct. What about statistical conclusion validity?
This form of validity evaluates whether the correct statistics were used,
the study had enough power, and the conclusions drawn are
mathematically justified. Some of the common threats are low statistical power,
small samples, increase the risk of type two errors, false negatives,
missing real effects. A treatment truly works, but the sample
size it's too small to detect the significance, unreliable measurement,
unstable measures, weakened findings. Example, if a depresspression scale gives
drastically different scores every few days, that's without real mood changes,
the statistical conclusions become questionable. Internal versus external validity is
one of the most heavily tested concepts on the E
triple P the SEESAW principle. Lab studies high control, strong
internal validity, lower realism, and lower external validity. Field studies
high realism, strong external but lower internal validity and less control.
Speaker 2: Give you another example.
Speaker 1: A lab example, A memory researcher places participants in a
silent control room with identical lighting and scripted instructions. Excellent control,
few confounds, high internal validity, yet real world memory rarely
occurs under such sterile conditions. All right, Matching research designs
to research questions, here we go.
Speaker 2: Actually, I forgot to tell you about field example.
Speaker 1: Psychology studies police decision making during actual patrol shifts advantages
realistic behavior and stronger ecological validity, disadvantages uncontrolled variables, environmental unpredictability,
and low internal validity.
Speaker 2: All right, So.
Speaker 1: It's where recap experimental design establishes calledation quasi experimental approximate
causation without randomization exploring associations or predictions is correlational examine
changes across time, logitudin and study rare or highly complex phenomena.
As a case study, validity often competes also with other ethics,
Some studies with the highest possible internal validity would be unethical.
Researchers cannot randomly assign children to abuse and neglect or trauma. Therefore,
psychologists often rely on quasi experiment longitudinal correlational studies. These
desize designs improve ethics, but we can causal certainty. Here
are some practice questions for you. First one, a researcher
is studying a rare neurological condition and follows three individuals
over five years using behavioral observations, interviews, and neural imaging.
What is the primary primary methodological limitation? If you said
B I mean external validity, you're right. Case studies provide
exceptionally rich and detailed information, especially for rare conditions. However,
findings from only three individuals cannot generalize to larger populations.
Case studying methodology is a weakness here qualitative death and
highly specialized samples.
Speaker 2: Question number two.
Speaker 1: A psychologist measures employee stress levels before wellness intervention, midway through,
immediately after.
Speaker 2: And six months later.
Speaker 1: The design is best classified as what is it a
factorial design, cross sectional or time series design. If you
said time series design, you're right. Time series designs repeatedly
measure the same variable across multiple time points. These designs
are especially useful when randomization is impossible, interventions occur naturally,
or researchers want to examine trends. Question number three two
by three factorial design studying anxiety treatment. How many independent
variables in total conditions exist?
Speaker 2: Is it A two.
Speaker 1: Independent variables in five conditions? Does it B two independent
variables in six conditions or three independent variables and two conditions.
If you said B two independent and six conditions, you're right.
The number of digits equals the number of ivs independent
variables is two times three is six, and there you go.
Speaker 2: Question number four.
Speaker 1: A developmental psychologist wants to reduce error variants caused by
participant differences while testing three educational apps.
Speaker 2: Which design is best.
Speaker 1: Between subjects, cross sectional or within subjects design with counterbalancing?
Do you want to reduce error variants? If you say
within subjects, you're right. Within subjects produce error because each
participant acts as their own control. However, order effects become
a major concern, and counterbalancing can help reduce fatigue and
practice effects. And the last question, a study finds a
correlation between clinician experience and patient retention. Credit Critics argue
older clinicians tend to work in a wealthier neighborhoods, which
may explain retention outcomes. The criticism reflects what attrition bias,
the third variable problem or testing effects. If you said
the third variable problem, you're right. An uncontrolled third variable
problem like a wealthy neighborhood may explain placement and repatient retention.
Correlation is not equal causation. Some the triple P study
tips think clinically, not definitionally. The E triple P rarely
asks what is internal validity instead of presents a videt
and asks what threat exists, what designers.
Speaker 2: Used, or what conclusion is justified.
Speaker 1: Immediately identify the independent and dependent variables and ask yourself
what is manipulated?
Speaker 2: What is measured? No manipulation no, No.
Speaker 1: Random assignment usually means correlational, observational and quasi experimental. Learn
the validity language external validity generalized to real life, eternal,
alternative explanation, construct accurately measures and then statistical sample too small.
Remember the lab versus steady trade offs we talked about.
Know why correlation cannot prove causation. Three major reasons directionality,
third variable or confounds, and lack of manipulation and random assignment.
The E triple P repeated tests these concepts in disguised forms. Finally,
here's a high yield takeaway when evaluating any research study.
Train yourself to ask four questions. Can I trust the
cause of conclusion? Internal validity, Can this generalize external ability?
Did they measure the right construct construct ability? And where
the statistical conclusions justified?
Speaker 2: Well, there you go, folks. Hopefully this is a little longer.
Speaker 1: Than I expected on the podcast, and hopefully you didn't
mind that. Hopefully you got a lot of insight. But
you know, research method statistics. A lot of people find
this difficult in psychology programs, not what they're used to,
So I get it.
Speaker 2: That's it for now, folks,