The Phonemin Company is a distributor of
men’s and women’s casual clothing. It sells exclusively through its merchandise
cata log, which is published four times per year to coincide with seasonal
changes in customers’ apparel tastes. Customers may order merchandise from the
catalog via mail or over the phone. Currently, 70% of orders are phone orders,
and the organization expects this to increase to 85% within the next few years.
The success of the organization is
obviously very dependent on the success of the
telephone ordering system and
the customer ser vice representatives (CSRs) who staff the system. There are
currently 185 CSRs; that number should increase to about 225 CSRs to handle the
anticipated growth in phone order sales. Though the CSRs are trained to use
standardized methods and procedures for handling phone orders, there are still
seemingly large differences among them in their job performance. CSR performance
is routinely measured in terms of error rate, speed of order taking, and
customer complaints. The top 25% and lowest 25% of performers on each of these
measures differ by a factor of at least three (i.e., the error rate of the
bottom group is three times as high as that of the top group). Strategically,
the organization knows that it could substantially enhance CSR performance (and
ultimately sales) if it could improve its staffing “batting average” by more
accurately identifying and hiring new CSRs who are likely to be top performers.
The
current staffing system for CSRs is straightforward. Applicants are recruited
through a combination of employee referrals and newspaper ads. Because turnover
among CSRs is so high (50% annually), recruitment is a continuous pro cess at
the organization. Applicants complete a standard application blank, which asks
for information about education and previous work experience. The information
is reviewed by the staffing specialist in the HR department. Only obvious
misfits are rejected at this point; the others (95%) are asked to have an
interview with the specialist. The interview lasts 20–30 minutes, and at the
conclusion the applicant is either rejected or offered a job. Due to the
tightness of the labor market and the constant presence of vacancies to be
filled, 90% of the interviewees receive job offers. Most of those offers (95%)
are accepted, and the new hires attend a oneweek training program before being
placed on the job.
The organization has decided to investigate the
possibilities of increasing CSR effectiveness through sounder staffing
practices. In particular, it is not pleased with its current methods of
assessing job applicants; it feels that neither the application blank nor the
interview provides an accurate and in depth assessment of the applicant KSAOs
that are truly needed to be an effective CSR. Consequently, it engaged the ser
vices of a consulting firm that offers various methods of KSAO assessment,
along with validation and installation ser vices. In cooperation with the HR
staffing specialist, the consulting firm conducted the following study for the
organization. A special job analysis led to the identification of several
specific KSAOs likely to be necessary for successful performance as a CSR.
Three of these (clerical speed, clerical accuracy, and interpersonal skills)
were singled out for further consideration because of their
seemingly high
impact on job performance.
Two new methods of assessment provided by
the consulting firm were chosen for experimentation. The first is a paper and
pencil clerical test assessing clerical speed and accuracy. It contains 50
items and has a 30-minute time limit. The second is a brief work sample that
could be administered as part of the interview process.
In the work sample, the applicant must
respond to four different phone calls: a customer who is irate about an out of
stock item, a customer who wants more product information about an item than
was provided in the catalog, a customer who wants to change an order placed
yesterday, and a customer who has a routine order to place. Using a 1–5 rating
scale, the interviewer rates the applicant on tactfulness (T) and concern for
customers (C). The interviewer is provided with a rating manual containing
examples of exceptional (5), average (3), and unacceptable (1) responses by the
applicant. A random sample of 50 current CSRs were chosen to participate in the
study.
At
Time 1 they were administered the clerical test and the work sample;
performance data were also gathered from company records for error rate (number
of errors per 100 orders), speed (number of orders filled per hour), and
customer complaints (number of complaints per week). At Time 2, one week later,
the clerical test and the work sample were re administered to the CSRs.A
member of the consulting firm sat in on all the interviews and served as a
second rater of performance on the work sample at Time 1 and Time 2. It is
expected that the clerical test and work sample will have positive correlations
with speed and negative correlations with error rate and customer complaints.
After reading the description of the study
and observing the results above,
1. How do you interpret the reliability
results for the clerical test and work sample? Are they favorable enough for
Phonemin to consider using them “for keeps” in selecting new job applicants?
2. How do you interpret the validity
results for the clerical test and work sample? Are they favorable enough for
Phonemin to consider using them “for keeps” in selecting new job applicants?
3.
What limitations in the above study should be kept in mind when interpreting
the results and deciding whether to use the clerical test and work sample?
