Speaker 1: Welcome back today. We're going to be looking at assessment
and diagnosis, So psychometric concepts we're going to be looking
first is reliability and validity. Reliability refers to consistency of
a measure, and validity refers to whether it measures what
it's supposed to measure. Key types of reliability include test retest,
internal consistency. Such things are such as Cronback's alpha that
we'll talk about later in split half inturrator in parallel forms.
Key types of validity include content criterion, concurrent, predictive, and
construct and we'll look at what these mean later. Reliability
is necessary for validity, but it's not sufficient by itself.
Standard error of measurement will look at that and confidence
intervals as well, So let's get started on this reliability.
A reliable test produces similar results under consistent conditions. It
answers the question how dependable is this measure. One way
to look at to test it is test retest measure reliability.
It measures temporal stability. So over time, administer the same
test to the same group at two points in time,
they correlate to two sets of scores, and a high
correlation equals strong reliability over time. This is best for
traits assumed to be stable, like intelligence, but not for
mood or anxiety, which are much more transient. An example
of an assessment would be the ways WAIS scores one
month apart should remain consistent unless a major event intervenes.
Another way of testing reliability is internal consistency measures how
well items or questions on a test measure the same
construct best for tests with homogeneous content like a depression inventory.
Main indices include Kronback's alpha so average correlation among items
split half, which correlates scores from two halves of the
same tests odd versus even items, the KR twenty, which
is for dichotomous items true or false type things. Rule
of thumb and alpha greater or equal to seventy is
acceptable for early research, but for clinical settings or should
be higher than point eight zero. Point nine zero may
indicate redundancy. Inter Radar reliability is another way to measure
agreement between raiders scoring subjective responses. Use them behavioral observations
or diagnostic interviews, or the psychoanalytic projective tests calculated using
Coen's Kappa for categorical decisions and in intra class correlation
for continuous scales. The parallel forms reliability measure is creates
two versions of a test measuring the same construct. The
minister both to the same group and correlate the results
ideal but really rare due to difficulty in making truly
equivalent forms, but it can be helpful with practiced effects
are concerned validity. What is this? It's accuracy of what
is being measured. So validity is about truth. Does the
test measure what it claims to measure? And let's look
at the different types of measuring validity for a test.
What is content validity? How well test items represent the
full range of the construct judged by expert review, not
statistics important for achievement, test, clinical symptom checklists, and diagnostic criteria. Example,
a depression inventory missing items on sleep or appetite has
poor content validity. Criterion validity assesses the relationship between test
scores and an external benchmark concurrent validity. It correlates with
current performance. Example, depression inventory scores correlate with clinical interview diagnosis.
That's a concurrent validity. So again you're assessing the relationship
between test scores and an external benchmark criteria. You're also
looking for predictive validity under the criterion category category correlates
with future performance like GRE scores predicting GPA scores in college.
Effect size is going to be important, so point one
zero is small, point three to zero is moderate, anything
above point five zero is a strong prediction. The next
type of validity you can measure is construct validity. This
one's goods. This one kind of gets confusing a little bit,
but let's look at it showing that the test accurately
reflects the theoretical concept convergent. Convergent validity correlates positively with
measures of the same construct. So you're creating a new
anxiety scale, how does it correlate with a GAD seven?
A discriminate validity is the opposite. It correlates weekly or
not at all, with unrelated constructs. In other words, if
you're creating a depression scale, it should not strongly correlate
with IQ. Construct validity, again, often evolves over multiple studies
using factor analysis, theoretical modeling, and multi trade multimethm matrices.
Face validity is the final one. Do the items look
like they measure what they're supposed to. It's not scientific required.
It affects clients, trust and tests engagement. A test with
high face validity may be easier to fake or manipulate.
Reliability versus validity. Tests must be reliable to be valid,
but reliability alone does not ensure validity. You could be validity,
you could be reliably wrong. Think of a dartboard. Tight
cluster off center equals reliable but not valid, scattered hits
near the bulls eyes low reliability, and tight cluster on
the bulls eyes high reliability and validity. Standard of measurement
of error and confidence intervals. No test score is exact,
and the standard error of measurement smates how much a
person's observed score might deviate from their true score. The
formula is SEM equals SD times the square root of
one minus reliability coefficient. The smaller the SEM means more
precise measurements. Confidence intervals use SEM to create a range
around the observed score. So sixty eight percent confidence interval
equals plus or minus one SEM. Ninety five percent is
plus or minus two SEM. So if a client scores
one hundred on a nine Q test with an SCM
equaling three, and the ninety five percent confidence sivitable would
be one Well November ninety five percent confidence and of
what equals plus or minus two sem. So with an
SEM of three, you're looking at negative six plus six,
so the range will become a ninety four to one
oh six. If the client scores one hundred. You wouldn't
make a rigid clinical adjustment based on a single point
though factors affecting reliability, coefficients or test lengths. Longer tests
have often higher reliability, more data, less noise, item homogenety,
more consistent content improves internal consistency, time between administrations, Shorter
intervals improved test retest reliability, respond variability, more variability between
subjects improves reliability. Testing conditions, noise distractions, unclear instructions reduce reliability.
So how do you look at this clinically? Well, you
use test selections, so you use tests with strong reliability
and validity evidence for the target population. Check test manuals
for norms, sample characteristics and limitations. Avoid tests for nondiverse populations.
Interpreting scores always report confidence intervals, don't rely on single
item scores. For complex constructs, use multiple data sources, assessment
tools and diagnoses. Use structured interviews like SKID with stronger
strong innervator reliability for diagnostic clarity. Select symptom inventories with
high internal consistency and predictive validity ensure tools align with
DSM criteria and cultural presentation of symptoms. Next time, we'll
look at the intelligence assessment instruments and theories about them.
That's it for now