Because I/O psychology tends to rely strongly on empirical data, I/O psychologists must be experts in research design and statistics. One area that I/O psychologists are particularly concerned about is the reliability and validity of measures. In fact, I/O psychologists in consulting firms are often asked for their expertise in understanding how measures work and what measures are best for a certain purpose. Below is an e-mail from Kendall Phillips, the HR manager of the law firm MacLaughlin & MacDougal. She has designed a new selection measure for her firm and needs some feedback from someone who knows more about reliability and validity. Read her e-mail, and provide detailed responses to her questions.
Hello—
Thank you for agreeing to help me with this measure—with all of the financial challenges facing my company. Your assistance is appreciated!
First, some background: Our firm currently needs to hire a number of administrative assistants. The demands our lawyers place on our administrative assistants are very high; they often yell at them or ask them to do very difficult things in a short amount of time. We have found that one of the best predictors of these assistants staying more than a couple of months is emotional stability (often the job is stressful, and successful assistants don’t get anxious or emotional easily). We used to buy an emotional stability test (called the WIN) that would measure this, but it has gotten too expensive, and so I’ve started to develop my own measure (I call my test the Stability Assessment Form, or the SAF). To design my measure, I spoke to a few experts in the personality field, looked at some of the items on other tests of emotional stability, and wrote questions that made sense. I asked a number of current administrative assistants to fill out the WIN and the SAF. I also have job performance ratings for these administrative assistants. Here’s what I’ve found so far:
The correlation between their scores on the SAF in July and their scores on the SAF in August is .89.
The correlation between their scores on the WIN and the SAF is .75.
The correlation between their scores on the SAF and a test of sociability is .34.
The correlation between their scores on the SAF and their performance ratings is .30.
So here are my questions for you:
Is my test a reliable measure of emotional stability? If not, what else do I need?
Is my test a valid measure of emotional stability? What additional information should I collect, if any?
Thanks again!
KP
Question 1
Draft a response that clearly answers Kendall’s questions. You should:
Indicate what types of reliability/validity she has collected
and whether you think they are adequate
What types of reliability/validity she has overlooked
and how she can obtain them.
Make sure to address all forms of reliability/validity you have learned about.
