Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
95 Cards in this Set
- Front
- Back
- 3rd side (hint)
|
Nomological network |
A method for defining a construct by illustrating its relation to as many other constructs and behaviors as possible. Positively associated- Anger-stress-frustration Negatively associated- anger- content-lucid |
|
|
|
Construct explication |
1. ID behaviors 2. ID related constructs 3. ID behaviors related to other constructs |
|
|
|
Construct |
An attribute, trait, or characteristic that in itself is not directly observable and exists only in our imagination. |
|
|
|
Behavior |
Actions that are observable and measurable. These are constructs. |
|
|
|
Construct validity |
Does the test possess items that accurately reflect the theoretical construct it is designed to measure? We accumulate evidence that the scores relate to observable behaviors in ways predicted by underlying theory. |
|
|
|
Construct validity |
How well does the test capture the absolute truth of a construct theoretically |
|
|
|
Criterion-related validity |
How good is the test at predicting things it should or moving with things it should consistently? |
|
|
|
Reliability |
A test cannot have a stronger correlation with any other variable than it does with itself. Provides a statistical baseline to compare other constructs to
Reliability is the degree to which an assessment tool produces stable and consistent results. |
|
|
|
Convergent validity |
We expect our test score to correlate with scores on measures of constructs that are the same or similar |
|
|
|
Divergent validity |
We expect our test scores to NOT correlate with scores on measures of constructs that are different/ unrelated measures |
|
|
|
Discriminate validity |
Divergent validity Use known- group method to validate. |
|
|
|
MTMM Correlation Matrix |
1.Heterotrait-hetero-method correlations 2. Heterotrait- monomethod correlations 3. Mono-trait heteromethod correlations 4. Monotrait- monomethod correlations |
|
|
|
Heterotrait- heteromethod correlations |
Multiple traits correlate with multiple methods Low, evidence for divergent validity |
|
|
|
Heterotrait-monomethod correlations |
Multiple traits correlate with one method Moderate, method variance |
|
|
|
Monotrait-heteromethod correlations |
Validity diagonal One trait correlates with multiple methods Largest, evidence of convergent validity |
|
|
|
Monotrait- monomethod correlations |
Reliability diagonal One trait correlates with one method Reliability coefficient |
|
|
|
Criterion-related validity |
Specific- can this test score predict what it’s supposed to? How well does it move with things it should concurrently? |
|
|
|
Which of the following types of validity is NOT criterion-related validity? |
Content validity is not a type of criterion-related validity. |
Concurrent and predictive |
|
|
What is the primary difference between predictive and concurrent validity? |
Timing, predictive predicts a later result, concurrent happens simultaneously |
|
|
|
If we find that GRE scores and gpa of first year grad students are related, what kind if validity is that evidence of? |
Predictive validity |
|
|
|
A criterion is something that our test scores should theoretically be related to, true or false? |
True |
|
|
|
What is the most important part of a test? |
Reliability |
|
|
|
Does interval scale tell us something about the distance between points? |
Yes, intervals are equal between points. No absolute zero. |
|
|
|
Ratio |
Equal distance between points, absolute zero |
|
|
|
Restriction of range happens when trying to establish concurrent validity, true or false? |
False |
|
|
|
What does restricting the range do to your validity coefficient? |
Lowers it |
|
|
|
If you create a measure of raw strength, how would you establish the criterion-related validity? |
You would compare your test to another test and see if they relate to each other. |
|
|
|
What level of measurement is a ruler? |
Ratio, you can have 0 inches |
|
|
|
What level of measurement is temperature? |
Interval technically |
|
|
|
Which of the following is true about the true score? |
We can never know what the true score is. |
|
|
|
What accounts for the difference between someone’s true score and observed score? |
Both systematic and random error |
|
|
|
Which type of analysis would you conduct to measure inter-rater reliability? |
Cohen’s Kappa Judgements- are rates reliable? |
|
|
|
What type of analysis would you run to measure internal consistency? |
Cronbach’s alpha Is 1 similar to question 25? |
|
|
|
A heterogenous test captures two or more constructs, true or false? |
True |
|
|
|
Homogenous tests test one construct, true or false? |
True |
|
|
|
Which analysis is ideal for evaluating the construct validity of a measure? |
Factor analysis Capture the number of constructs we planned. |
|
|
|
Examining validity means gathering and examining evidence, true or false? |
True |
|
|
|
Content validity includes examining which of the following: |
Test questions, format of the test (self-admin/preparer) wording of items All of the above-nuts and bolts |
|
|
|
You’ve designed a self-report test that’s designed to capture someone’s raw physical strength. How would you evaluate the criterion-related validity? |
Take the questionnaire and do some strength-related exercises and see if the two tests are related. Our test should related to another test that measures another similar/related construct |
|
|
|
Show the concurrent validity of a test. |
A type of criterion-related validity. Use your measure and another measure designed to measure the same construct. |
|
|
|
In one sentence, define validity. |
A test measures what it’s supposed to and the data obtained is being used appropriately. The score is used the way it was intended. |
|
|
|
Which type is NOT criterion-related validity? |
Content validity is not criterion validity |
|
|
|
If GRE scores and gpa are related what kind of validity is that? |
Predictive validity |
|
|
|
A criterion is something that our test should be related to in some way... |
True |
|
|
|
Difference between predictive and concurrent validity? |
Time. Predictive predicts a later result and concurrent happens at the same time. |
|
|
|
Restriction of range happens when trying to establish concurrent validity, true or false? |
False |
|
|
|
What does restricting the range do to your validity coefficient? |
Lowers your validity coefficient |
|
|
|
What do you call a correlation co-efficient when you are correlating the test score with the criterion? |
Validity coefficient |
|
|
|
When we evaluate the reliability of an instrument it’s called... |
Reliability coefficient |
|
|
|
Which of the following is an example of an objective criterion? |
Time on a processing task |
|
|
|
When the criterion we choose includes the test construct and more, what can happen? |
Criterion contamination |
|
|
|
What does the p value of an analysis tell us? |
The probability that our results are not due to chance 95-99% probability that our results show a true relationship, and are not due to chance |
|
|
|
Which of the following is NOT a type of evidence we gather when evaluating construct validity? |
Nomothetical evidence |
|
|
|
Which is a kind of construct validity we discussed in class? |
divergent/discriminate validity, convergent validity |
|
|
|
Which of the following describes the idea that two theoretical constructs should overlap? |
Convergent |
|
|
|
Which idea describes that two ideas should be very different? |
Divergent/ discriminant |
|
|
|
If a test is significant in describing differences in depression levels across two different groups of individuals, what validation methods would we use? |
Known groups method |
|
|
|
What psychometric information can be gleaned by using the MTMM method? |
reliability, Convergent validity, divergent validity All of the above |
|
|
|
Which of the following is an example of the multi-trait aspect of the MTMM? |
Anxiety Multiple traits |
|
|
|
Which of the following is an example of the multi-method aspect of the MTMM? |
Self-report measures |
|
|
|
What do you call the line of numbers that reveal the validity information from the MTMM correlation matrix? |
Validity diagonal/coefficient (Mono-trait/hetero-method correlations) |
|
|
|
What do you call the line of numbers that reveal the reliability information from the MTMM correlation matrix? |
Reliability diagonal/coefficient (mono-trait/mono-method correlations) |
|
|
|
What is the primary difference between a test and a survey? |
Tests measure individual scores, survey measure group results |
|
|
|
How would a knowledge survey look? |
Knowledge based test- how many 5th graders know who the first president is? Individuals Knowledge survey- what the population knows |
|
|
|
One of the first known surveys |
1790 census |
|
|
|
Qualtrics is a survey research firm, true or false? |
False They develop software, they don’t conduct research for hire. Wigex, survey monkey |
|
|
|
Lit reviews, consulting with experts is an important part of the survey/test construction process? |
True You don’t know what you don’t know- consult experts to see how things are defined. It’s less effort |
|
|
|
If you hand someone a survey, and they come give it back to you what kind of test is it? |
Self-administered Individually administered has a proctor that sits with you and participates- not just to hand it to you. |
|
|
|
Which of the following is a reason why a person might not report an appropriate response? |
Social desireability, has no idea, trying to figure out but the wording is hard to understand |
|
|
|
Number of items in a Likert scale? |
Usually 5-7 |
|
|
|
Surveys rely heavily on trying to understand the cognitive processes of respondents, true or false? |
True |
|
|
|
What are double-barreled questions? |
Questions that ask two things at once- is our staff knowledgeable and helpful? |
|
|
|
What is construct validity? |
Theoretical philosophical measure, hard to test so we use statistics like factor analysis |
|
|
|
Criterion related validity |
Practical outcomes of the measurement process, do things move together |
|
|
|
Test validity relies on... |
Suitability of the test for a particular audience Adequate sampling of behaviors from a specific test domain The purpose for which the test scores are used |
|
|
|
Convergent validity |
Similar constructs converge- come together Anger-frustration |
|
|
|
Divergent/discrimination validity |
Constructs that are different diverge Anger-calmness |
|
|
|
Forced choice |
Participant must make a choice- yes/no there is no maybe or non-answer option |
|
|
|
Homogeneity of the population |
Sameness, sample can be smaller |
|
|
|
Test report results... |
Derived scores |
|
|
|
What info can you derive from MTMM correlation matrix? |
Reliability diagonals, validity diagonals (convergent/divergent) |
|
|
|
Survey results report... |
A percentage of participants who answered each item |
|
|
|
Surveys help us understand... |
How people feel, what they know, what they do Attitudes, knowledge, behavior |
|
|
|
Experimental research techniques |
Help us determine cause/effect |
|
|
|
Descriptive/correlation-based research techniques |
Help describe a situation or phenomenon |
|
|
|
Confirmatory factor analysis |
Researcher specifies an a priori hypothesis about what factors will be found Based on existing research or theoretical models SPSS tries to make it fit the data |
|
|
|
Exploratory factor analysis |
We don’t have a hypothesis about what factors we expect |
|
|
|
Psychological test |
Psychological test ~ a measurement tool or technique that requires a person to perform one or more behaviors in order to make inferences about human attributes, traits, or characteristics or predict future outcomes. |
|
|
|
Good tests |
Three defining characteristics of all good tests 1) Representatively sample behaviors thought to measure an attribute or predict an outcome § e.g., NFL combine 2) Behaviors are obtained under standardized conditions § Each examinee should have the same experience § e.g., NFL combine 3) Rules for scoring § All scoring should be identical from one examinee to the next |
|
|
|
Test |
t §Tests §Measurements §Surveys Assessment Measurement Test/survey |
|
|
|
Test validity |
Why develop a new Test?Remember: Test validity depends on the suitability of a test for a particular audience of test takers Test validity requires adequate sampling of behaviors from a specific test domain Test validity depends on the purpose for which test scores are |
|
|
|
Test controversy |
Testing controversies Do psychological tests contribute to discrimination? IQ testing in education IQ testing in the military Alpha testBeta test Allegedly culture-fair |
|
|
|
Restriction of range |
Lowers validity |
|
|
|
Coefficient of determination |
The coefficient of determination is used to explain how much variability of one factor can be caused by its relationship to another factor. ... The coefficient of determination is the square of the correlation coefficient, also known as "R," which allows it to display the degree of linear correlation between two variables. |
|
|
|
Known groups method |
Known groups method is a typical method to support construct validity and is provided when a test can discriminate between a group of individuals known to have a particular trait and a group who do not have the trait. |
|
