Speaker 1: Welcome back everyone.
Speaker 2: Today, we're going to be looking at a lot of
more statistics here in the next few podcasts, and then
we're going to get into legal and ethical issues.
Speaker 1: Sampling is the one we're going to be talking about today.
Speaker 2: Before you analyze anything, you must run any single statistic.
You have to decide who gets included in your study,
and that decision shapes everything. Here's the key idea that
each tripop wants you to hold. Choose the wrong sample
and even perfect statistics. When this leads you, you can
run the most sophisticated regression in the world. But if
your sample doesn't represent the population you care about, your
conclusions are built on sand. So when you're reading a study,
you need to ask yourself who was sampled, how were
they recruited, does this sample reflect the target population? And
are there demographic or cultural groups that limit where we
can conclude that last question. Matt is enormously in clinical
psych where so much foundational research was conducted on narrow
Western educated, industrialized, rich, and democratic ample. If a therapy
works in that population, we cannot assume it generalizes to
an unhoused individual. That bridge from study to real world
application is called external validity. We've covered this before, so
probability samplings. Our first one means every individual the population
has a known and non zero chance of being selected.
This is what supports generalizability, and there are four types
of probability sampling. Simple random, every person in the population
has an equal chance of being chosen.
Speaker 1: Think of it like drawing names out of a hat.
Speaker 2: You need a complete population list to do this, which
is often impractical for large populations, but when you can
pull it off, it's the gold standard. Imagine you're studying
depression rates among the licensed psychologists. If you have access
to the full board list and randomly seared names from it,
that's simple random sampling every licensed psychologist an equal shot.
Speaker 1: Stratified divide the.
Speaker 2: Population into subgroups called strata, things like gender, ethnicity, or
diagnostic category, and then randomly sample.
Speaker 1: Within each group.
Speaker 2: The goal is ensuring proportional representation across key demographics. So
you're studying CBT outcomes across racial groups. If you just
randomly sampled from a clinic, you might end up with
a sample it's eighty five percent white because that reflects
clinic demographics. Stratified sampling insures you have adequate representation from Black, Latino,
and Asian clients, making your finds more clinically meaningful and generalizable.
Cluster sampling, instead of sampling individuals, you randomly select entire
groups like schools, hospitals, or neighborhoods, and then same individuals
within those clusters. This is efficient for geographically spread populations,
but it produces more sampling error than stratified sampling because
of group level variants, people within the same cluster tend
to be more similar to each other the broader population.
Speaker 1: You want to.
Speaker 2: Study anxiety prevalence in community mental health centers across a
large state, rather than sampling every center, you randomly select
ten and access all clients within them.
Speaker 1: That's cluster sampling.
Speaker 2: Systematic sampling you select every individual from a list after
a random starting point. So if you look at if
K equals ten, you start at a random number, say
number four, then take number fourteen, twenty four, thirty four,
and so forth. It's easy to implement, but it carries
one specific risk, what they call periodicity bias. If there
is a repeating pattern in the list that aligns with
your sampling interval, you wire systematically over under select certain
types of people. Example, would be you have a list
of therapy and take appointments organized by day of week.
If you take every seventh appointment and the list cycles weekly,
you might only ever select clients who come in on Mondays,
potentially missing weekend to only available clients who may differ
in employments, status, or severity. Another one now is non
probability sampling. We looked at probability sampling methods that included
systematic sampling, cluster sampling, stratified, and simple random. Now we
look at non probability and it's used when probability sampling
isn't feasible. The trade off is that generalizability is limited.
You have to know that's going into your interpretation. The
first one is convenience. Sampling participants are selected based on
availability or proximity. Fast, low cost, easy to do, and
highly vulnerable to selection bias. The classic examples recruiting undergrad
psych students because they're sitting in the building common in
academic research, but you have to be cautious about what
you can claim from those findings. Clinical example, a researcher
studies the effect of mindfulness on stress by recruiting from
a university psychology department waiting list. These are people who
already sought help, are likely college educated, are probably not
representative of community health mental health populations. The researcher deliberately
selects participants who meets specific criteria. This is especially useful
in qualitative research. This is purpose of sampling. So again,
the researcher deliberately selects participants who meets specific criteria. This
is especially useful in qualitative research. For instance, you're studying
the lived experience of psychologist who have treated clients with
Capgras syndrome, so you purposely selected or recruited clinicians we
have direct treatment experience. Snowball sampling, existing participants recruit future
participants from their social networks. This is used when you're
studying hidden or stigmatized populations where a sampling frame doesn't exist.
It's hard to get individuals like distress and undocumented immigrants
who are unlikely to respond to traditional recruitment. You start
with two or three participants through a community organization and
ask them to refer other as they trust, and then
it grows.
Speaker 1: Lastly, it's quota sampling. These are like.
Speaker 2: Stratified sampling and structure, but non random and execution. You
identify categories you want represented and feel predetermined quotas for
each is fast and structured, but it lacks true randomness,
so you can't apply probability based statistical inferences.
Speaker 1: The same way.
Speaker 2: Another area sampling distributions, the central limit theorem and standard error.
Even good samples vary from the population, there's always some
degree of sampling error. But here's the insight. If we
understand how samples behave across many repetitions, we can estimate
population parameters from a single sample. That's the whole game.
This is the theoretical distribution of a statistics, say the
mean across many random samples drawn from the same population.
You're not looking at one sample. You're imagining what the
distribution of that statistic would look like if you do
samples over and over again. This is one of the
most important theorems in all of statistics, and the e
triple p tested directly. Here's what it says for the
central limit theorem. With a large enough sample, typically thirty
or more, the sampling distribution of the mean becomes approximately normal,
regardless of the shape of the population distribution. Why doesn't
matter because it justifies using what we call parametric tests
even when your underlying population isn't normally distributed.
Speaker 1: If you're studying.
Speaker 2: Trauma scores in a community sample and a distribution is scored,
you can still use a T test or a NOVA
if your sample is large enough, because the central limit
theorem tells you the sampling.
Speaker 1: Distribution of the mean will be normal. Standard error this.
Speaker 2: Is the standard deviation of the sampling distribution. It tells
you how much sample means vary from sample to sample.
The formula's se equals the standard deviation divided by the
square root of N. An example would be a sample
sized increases standard error decreases, your estimate of the population
mean becomes more precise. This is why large critical trials
are more trustworthy than small pilot studies, not because the
researchers are smarter, because the math of sampling gives them
a more stable estimate. There's five factors that affect sampling size.
First is the effects size. This is the magnitude of
the difference or relationship you expect to find. Cohen's benchmarks
are d equals zero point two zero for small, zero
point five for medium, and zero point eight for large.
Smaller effects require larger samples to detect. If you're studying
a subtle early intervention effect on subclinical anxiety, you need
more participants than if you're studying the effect of a
major trauma event on PTSD. Severity power is the probability
of detecting a real effect when one truly exists. The
conventional target is zero point eight, meaning on eighty percent
chance of detecting a true effect. Power increases with larger
sample size, larger effects size, and higher alpha on the
h If a study fails to find a significant result
with low power, you should be suspicious. That might be
a type two error false negative, not a true null
result type one error false positive. This is the threshold
you used to decide whether it's a result is statistically significant,
also called the alpha level. Alpha level type one error
rate typically set at zero point zero five. If you
lower alpha to zero point zero one to be more
conservative to reduce the risk of false positives, you need
a much larger sample to maintain your power. There's always
going to be a trade off design complexity. More groups
and more predictors mean you need a larger sample. A
study comparing five treatment conditions requires more participants than a
two group design. Within subject designs where the same person
is measured multiple times, typically fewer participants are needed because
you reduce vary variants by using each person on their
own control. There are also constraints budget, time, and access
to participants often limit ideal sample sizes. The researcher studying
a rare personality sort of and I'm able to recruit
may not be able to recruit three hundred participants, no
matter how clean the design is. As section six is
talk about power analysis, which is the form of process
used to calculate the minimum sample size and needed to
detect an effect, and you run it before data collection.
It's a planning tool, not an afterthought. Effect size using
Cohen's d f orr. Depending on the design, alpha level
desired power usually set at zero point eight and a
number of groups of predictors. The standard software used in
psychology's g power, which is free and widely used. Failing
to conduct the power analysis before a study creates two problems. First,
you might end up underpowered and miss a real effect
that's a false negative type two error. Second, you might
over recruit, which waste resources and raises unnecessary ethical concerns
about participant burden. For instance, the researcher wants to study
whether trauma focus CBT reduces PTSD symptoms more than supportive counseling.
They expect the medium effects size of D equals zero
point five. They set alpha zero point five and zero
point zero five and one power of zero point eight.
Running g power tells them they need approximately sixty four
participants per group, so one hundred and twenty eight total.
If they only recruit forty, they're severely underpowered, and a
null result tells us almost nothing. So let's talk about
what goes wrong. Sampling bias is introduced when some members
of the population are systematically less likely to be included
in the sample. The key word is systematically self selection bias.
Volunteers differ from non volunteers in ways that are often
clinically relevant.
Speaker 1: People who volunteer for.
Speaker 2: Depression treatment studies tend to be more motivated, more resourced,
and less severely impaired than the average with depression. If
your results are based entirely on volunteers, you may not
be systematically overestimating, or you may be systematically overestimating treatment effectiveness.
Speaker 1: Non response by certain groups.
Speaker 2: Are less likely to complete surveys or follow ups if
clients with the most severe pathology drop out of your
longitudinal study because they're too disregulated to complete it. Your
data on chronic compairment is compromised. Undercoverage, certain populations are
left out entirely. Classic example, unhoused individuals. If you're studying
mental health outcomes in a city and your sampling frame
is a clinic registry, you've already excluded the population most
likely to have severe untreated comobid conditions. Selection effects are
related but distinct concept. They occur when the process of
assigning participants to conditions or groups is not random. This
affects internal validity, your ability to claim that the independent variable,
not some pre existing difference between groups.
Speaker 1: Cause the outcome.
Speaker 2: Selection effects are why randomized control trials are the gold standard.
Random assignment breaks the link between pre existing characteristics and
group measurement membership. Poor sampling equals limited to not generalizability
external validity, which is the degree to which fundings from
a study can have generalize to other people. Sampling is
the primary drive driver of external validity. If your sample
is narrow, your conclusions are narrow. When reading research like
a scientist, run through the checklist. Was the sampling method
is described. Was randomization used? Was the sample diverse and representative?
How large was it it was? A power analysis reported
whether high attrition rates are missing data. These are not
just academic questions and clinical practice that determine whether the
intervention you're considering for your client was actually tested on
someone like your client.
Speaker 1: That's it for now, folks,