Speaker 1: Welcome back everyone.
Speaker 2: Today, we're going to be looking at a lot of
Speaker 2: more statistics here in the next few podcasts, and then
Speaker 2: we're going to get into legal and ethical issues.
Speaker 1: Sampling is the one we're going to be talking about today.
Speaker 2: Before you analyze anything, you must run any single statistic.
Speaker 2: You have to decide who gets included in your study,
Speaker 2: and that decision shapes everything. Here's the key idea that
Speaker 2: each tripop wants you to hold. Choose the wrong sample
Speaker 2: and even perfect statistics. When this leads you, you can
Speaker 2: run the most sophisticated regression in the world. But if
Speaker 2: your sample doesn't represent the population you care about, your
Speaker 2: conclusions are built on sand. So when you're reading a study,
Speaker 2: you need to ask yourself who was sampled, how were
Speaker 2: they recruited, does this sample reflect the target population? And
Speaker 2: are there demographic or cultural groups that limit where we
Speaker 2: can conclude that last question. Matt is enormously in clinical
Speaker 2: psych where so much foundational research was conducted on narrow
Speaker 2: Western educated, industrialized, rich, and democratic ample. If a therapy
Speaker 2: works in that population, we cannot assume it generalizes to
Speaker 2: an unhoused individual. That bridge from study to real world
Speaker 2: application is called external validity. We've covered this before, so
Speaker 2: probability samplings. Our first one means every individual the population
Speaker 2: has a known and non zero chance of being selected.
Speaker 2: This is what supports generalizability, and there are four types
Speaker 2: of probability sampling. Simple random, every person in the population
Speaker 2: has an equal chance of being chosen.
Speaker 1: Think of it like drawing names out of a hat.
Speaker 2: You need a complete population list to do this, which
Speaker 2: is often impractical for large populations, but when you can
Speaker 2: pull it off, it's the gold standard. Imagine you're studying
Speaker 2: depression rates among the licensed psychologists. If you have access
Speaker 2: to the full board list and randomly seared names from it,
Speaker 2: that's simple random sampling every licensed psychologist an equal shot.
Speaker 1: Stratified divide the.
Speaker 2: Population into subgroups called strata, things like gender, ethnicity, or
Speaker 2: diagnostic category, and then randomly sample.
Speaker 1: Within each group.
Speaker 2: The goal is ensuring proportional representation across key demographics. So
Speaker 2: you're studying CBT outcomes across racial groups. If you just
Speaker 2: randomly sampled from a clinic, you might end up with
Speaker 2: a sample it's eighty five percent white because that reflects
Speaker 2: clinic demographics. Stratified sampling insures you have adequate representation from Black, Latino,
Speaker 2: and Asian clients, making your finds more clinically meaningful and generalizable.
Speaker 2: Cluster sampling, instead of sampling individuals, you randomly select entire
Speaker 2: groups like schools, hospitals, or neighborhoods, and then same individuals
Speaker 2: within those clusters. This is efficient for geographically spread populations,
Speaker 2: but it produces more sampling error than stratified sampling because
Speaker 2: of group level variants, people within the same cluster tend
Speaker 2: to be more similar to each other the broader population.
Speaker 1: You want to.
Speaker 2: Study anxiety prevalence in community mental health centers across a
Speaker 2: large state, rather than sampling every center, you randomly select
Speaker 2: ten and access all clients within them.
Speaker 1: That's cluster sampling.
Speaker 2: Systematic sampling you select every individual from a list after
Speaker 2: a random starting point. So if you look at if
Speaker 2: K equals ten, you start at a random number, say
Speaker 2: number four, then take number fourteen, twenty four, thirty four,
Speaker 2: and so forth. It's easy to implement, but it carries
Speaker 2: one specific risk, what they call periodicity bias. If there
Speaker 2: is a repeating pattern in the list that aligns with
Speaker 2: your sampling interval, you wire systematically over under select certain
Speaker 2: types of people. Example, would be you have a list
Speaker 2: of therapy and take appointments organized by day of week.
Speaker 2: If you take every seventh appointment and the list cycles weekly,
Speaker 2: you might only ever select clients who come in on Mondays,
Speaker 2: potentially missing weekend to only available clients who may differ
Speaker 2: in employments, status, or severity. Another one now is non
Speaker 2: probability sampling. We looked at probability sampling methods that included
Speaker 2: systematic sampling, cluster sampling, stratified, and simple random. Now we
Speaker 2: look at non probability and it's used when probability sampling
Speaker 2: isn't feasible. The trade off is that generalizability is limited.
Speaker 2: You have to know that's going into your interpretation. The
Speaker 2: first one is convenience. Sampling participants are selected based on
Speaker 2: availability or proximity. Fast, low cost, easy to do, and
Speaker 2: highly vulnerable to selection bias. The classic examples recruiting undergrad
Speaker 2: psych students because they're sitting in the building common in
Speaker 2: academic research, but you have to be cautious about what
Speaker 2: you can claim from those findings. Clinical example, a researcher
Speaker 2: studies the effect of mindfulness on stress by recruiting from
Speaker 2: a university psychology department waiting list. These are people who
Speaker 2: already sought help, are likely college educated, are probably not
Speaker 2: representative of community health mental health populations. The researcher deliberately
Speaker 2: selects participants who meets specific criteria. This is especially useful
Speaker 2: in qualitative research. This is purpose of sampling. So again,
Speaker 2: the researcher deliberately selects participants who meets specific criteria. This
Speaker 2: is especially useful in qualitative research. For instance, you're studying
Speaker 2: the lived experience of psychologist who have treated clients with
Speaker 2: Capgras syndrome, so you purposely selected or recruited clinicians we
Speaker 2: have direct treatment experience. Snowball sampling, existing participants recruit future
Speaker 2: participants from their social networks. This is used when you're
Speaker 2: studying hidden or stigmatized populations where a sampling frame doesn't exist.
Speaker 2: It's hard to get individuals like distress and undocumented immigrants
Speaker 2: who are unlikely to respond to traditional recruitment. You start
Speaker 2: with two or three participants through a community organization and
Speaker 2: ask them to refer other as they trust, and then
Speaker 2: it grows.
Speaker 1: Lastly, it's quota sampling. These are like.
Speaker 2: Stratified sampling and structure, but non random and execution. You
Speaker 2: identify categories you want represented and feel predetermined quotas for
Speaker 2: each is fast and structured, but it lacks true randomness,
Speaker 2: so you can't apply probability based statistical inferences.
Speaker 1: The same way.
Speaker 2: Another area sampling distributions, the central limit theorem and standard error.
Speaker 2: Even good samples vary from the population, there's always some
Speaker 2: degree of sampling error. But here's the insight. If we
Speaker 2: understand how samples behave across many repetitions, we can estimate
Speaker 2: population parameters from a single sample. That's the whole game.
Speaker 2: This is the theoretical distribution of a statistics, say the
Speaker 2: mean across many random samples drawn from the same population.
Speaker 2: You're not looking at one sample. You're imagining what the
Speaker 2: distribution of that statistic would look like if you do
Speaker 2: samples over and over again. This is one of the
Speaker 2: most important theorems in all of statistics, and the e
Speaker 2: triple p tested directly. Here's what it says for the
Speaker 2: central limit theorem. With a large enough sample, typically thirty
Speaker 2: or more, the sampling distribution of the mean becomes approximately normal,
Speaker 2: regardless of the shape of the population distribution. Why doesn't
Speaker 2: matter because it justifies using what we call parametric tests
Speaker 2: even when your underlying population isn't normally distributed.
Speaker 1: If you're studying.
Speaker 2: Trauma scores in a community sample and a distribution is scored,
Speaker 2: you can still use a T test or a NOVA
Speaker 2: if your sample is large enough, because the central limit
Speaker 2: theorem tells you the sampling.
Speaker 1: Distribution of the mean will be normal. Standard error this.
Speaker 2: Is the standard deviation of the sampling distribution. It tells
Speaker 2: you how much sample means vary from sample to sample.
Speaker 2: The formula's se equals the standard deviation divided by the
Speaker 2: square root of N. An example would be a sample
Speaker 2: sized increases standard error decreases, your estimate of the population
Speaker 2: mean becomes more precise. This is why large critical trials
Speaker 2: are more trustworthy than small pilot studies, not because the
Speaker 2: researchers are smarter, because the math of sampling gives them
Speaker 2: a more stable estimate. There's five factors that affect sampling size.
Speaker 2: First is the effects size. This is the magnitude of
Speaker 2: the difference or relationship you expect to find. Cohen's benchmarks
Speaker 2: are d equals zero point two zero for small, zero
Speaker 2: point five for medium, and zero point eight for large.
Speaker 2: Smaller effects require larger samples to detect. If you're studying
Speaker 2: a subtle early intervention effect on subclinical anxiety, you need
Speaker 2: more participants than if you're studying the effect of a
Speaker 2: major trauma event on PTSD. Severity power is the probability
Speaker 2: of detecting a real effect when one truly exists. The
Speaker 2: conventional target is zero point eight, meaning on eighty percent
Speaker 2: chance of detecting a true effect. Power increases with larger
Speaker 2: sample size, larger effects size, and higher alpha on the
Speaker 2: h If a study fails to find a significant result
Speaker 2: with low power, you should be suspicious. That might be
Speaker 2: a type two error false negative, not a true null
Speaker 2: result type one error false positive. This is the threshold
Speaker 2: you used to decide whether it's a result is statistically significant,
Speaker 2: also called the alpha level. Alpha level type one error
Speaker 2: rate typically set at zero point zero five. If you
Speaker 2: lower alpha to zero point zero one to be more
Speaker 2: conservative to reduce the risk of false positives, you need
Speaker 2: a much larger sample to maintain your power. There's always
Speaker 2: going to be a trade off design complexity. More groups
Speaker 2: and more predictors mean you need a larger sample. A
Speaker 2: study comparing five treatment conditions requires more participants than a
Speaker 2: two group design. Within subject designs where the same person
Speaker 2: is measured multiple times, typically fewer participants are needed because
Speaker 2: you reduce vary variants by using each person on their
Speaker 2: own control. There are also constraints budget, time, and access
Speaker 2: to participants often limit ideal sample sizes. The researcher studying
Speaker 2: a rare personality sort of and I'm able to recruit
Speaker 2: may not be able to recruit three hundred participants, no
Speaker 2: matter how clean the design is. As section six is
Speaker 2: talk about power analysis, which is the form of process
Speaker 2: used to calculate the minimum sample size and needed to
Speaker 2: detect an effect, and you run it before data collection.
Speaker 2: It's a planning tool, not an afterthought. Effect size using
Speaker 2: Cohen's d f orr. Depending on the design, alpha level
Speaker 2: desired power usually set at zero point eight and a
Speaker 2: number of groups of predictors. The standard software used in
Speaker 2: psychology's g power, which is free and widely used. Failing
Speaker 2: to conduct the power analysis before a study creates two problems. First,
Speaker 2: you might end up underpowered and miss a real effect
Speaker 2: that's a false negative type two error. Second, you might
Speaker 2: over recruit, which waste resources and raises unnecessary ethical concerns
Speaker 2: about participant burden. For instance, the researcher wants to study
Speaker 2: whether trauma focus CBT reduces PTSD symptoms more than supportive counseling.
Speaker 2: They expect the medium effects size of D equals zero
Speaker 2: point five. They set alpha zero point five and zero
Speaker 2: point zero five and one power of zero point eight.
Speaker 2: Running g power tells them they need approximately sixty four
Speaker 2: participants per group, so one hundred and twenty eight total.
Speaker 2: If they only recruit forty, they're severely underpowered, and a
Speaker 2: null result tells us almost nothing. So let's talk about
Speaker 2: what goes wrong. Sampling bias is introduced when some members
Speaker 2: of the population are systematically less likely to be included
Speaker 2: in the sample. The key word is systematically self selection bias.
Speaker 2: Volunteers differ from non volunteers in ways that are often
Speaker 2: clinically relevant.
Speaker 1: People who volunteer for.
Speaker 2: Depression treatment studies tend to be more motivated, more resourced,
Speaker 2: and less severely impaired than the average with depression. If
Speaker 2: your results are based entirely on volunteers, you may not
Speaker 2: be systematically overestimating, or you may be systematically overestimating treatment effectiveness.
Speaker 1: Non response by certain groups.
Speaker 2: Are less likely to complete surveys or follow ups if
Speaker 2: clients with the most severe pathology drop out of your
Speaker 2: longitudinal study because they're too disregulated to complete it. Your
Speaker 2: data on chronic compairment is compromised. Undercoverage, certain populations are
Speaker 2: left out entirely. Classic example, unhoused individuals. If you're studying
Speaker 2: mental health outcomes in a city and your sampling frame
Speaker 2: is a clinic registry, you've already excluded the population most
Speaker 2: likely to have severe untreated comobid conditions. Selection effects are
Speaker 2: related but distinct concept. They occur when the process of
Speaker 2: assigning participants to conditions or groups is not random. This
Speaker 2: affects internal validity, your ability to claim that the independent variable,
Speaker 2: not some pre existing difference between groups.
Speaker 1: Cause the outcome.
Speaker 2: Selection effects are why randomized control trials are the gold standard.
Speaker 2: Random assignment breaks the link between pre existing characteristics and
Speaker 2: group measurement membership. Poor sampling equals limited to not generalizability
Speaker 2: external validity, which is the degree to which fundings from
Speaker 2: a study can have generalize to other people. Sampling is
Speaker 2: the primary drive driver of external validity. If your sample
Speaker 2: is narrow, your conclusions are narrow. When reading research like
Speaker 2: a scientist, run through the checklist. Was the sampling method
Speaker 2: is described. Was randomization used? Was the sample diverse and representative?
Speaker 2: How large was it it was? A power analysis reported
Speaker 2: whether high attrition rates are missing data. These are not
Speaker 2: just academic questions and clinical practice that determine whether the
Speaker 2: intervention you're considering for your client was actually tested on
Speaker 2: someone like your client.
Speaker 1: That's it for now, folks,