The Phonemin Company is a distributor of men’s and women’s casual clothing. It sells exclusively through its merchandi
Posted: Sun Jul 03, 2022 3:18 pm
The Phonemin Company is a distributor of men’s and women’scasual clothing. Itsells exclusively through its merchandise catalog, which ispublished four times
per year to coincide with seasonal changes in customers’ appareltastes. Custom-ers may order merchandise from the catalog via mail or over thephone. Currently,
70% of orders are phone orders, and the organization expectsthis to increase to85% within the next few years.
The success of the organization is obviously very dependent onthe successof the telephone ordering system and the customer servicerepresentatives (CSRs)who staff the system. There are currently 185 CSRs; that numbershould increaseto about 225 CSRs to handle the anticipated growth in phone ordersales. Thoughthe CSRs are trained to use standardized methods and procedures forhandlingphone orders, there are still seemingly large differences amongthem in their jobperformance. CSR performance is routinely measured in terms oferror rate, speed
of order taking, and customer complaints. The top 25% and lowest25% of per-formers on each of these measures differ by a factor of at leastthree (i.e., the
error rate of the bottom group is three times as high as that ofthe top group).Strategically, the organization knows that it could substantiallyenhance CSR
performance (and ultimately sales) if it could improve itsstaffing “batting aver-age” by more accurately identifying and hiring new CSRs who arelikely to be top performers.The current staffing system for CSRsis straightforward. Applicantsare recruitedthrough a combination of employee referrals and newspaper ads.Because turnoveramong CSRs is so high (50% annually), recruitment is a continuousprocess atthe organization. Applicants complete a standard applicationblank, which asksfor information about education and previous work experience. Theinformationis reviewed by the staffing specialist in the HR department. Onlyobvious misfitsare rejected at this point; the others (95%) are asked to have aninterview with thespecialist. The interview lasts 20–30 minutes, and at theconclusion the applicantis either rejected or offered a job. Due to the tightness of thelabor market and theconstant presence of vacancies to be filled, 90% of theinterviewees receive job
offers. Most of those offers (95%) are accepted, and the newhires attend a one-week training program before being placed on the job.
The organization has decided to investigate the possibilitiesof increasing CSReffectiveness through sounder staffing practices. In particular,it is not pleased
with its current methods of assessing job applicants; it feelsthat neither the application blank nor the interview provides anaccurate and in-depth assessment of
the applicant KSAOs that are truly needed to be an effectiveCSR. Consequently,it engaged the services of a consulting firm that offers variousmethods of KSAOassessment, along with validation and installation services. Incooperation withthe HR staffing specialist, the consulting firm conducted thefollowing study forthe organization.A special job analysis led to the identification of severalspecific KSAOs likelyto be necessary for successful performance as a CSR. Three ofthese (clerical
speed, clerical accuracy, and interpersonal skills) weresingled out for further con-sideration because of their seemingly high impact on jobperformance. Two new
methods of assessment provided by the consulting firm werechosen for experi-mentation. The first is a paper-and-pencil clerical testassessing clerical speed and
accuracy. It contains 50 items and has a 30-minute time limit.The second is a briefwork sample that could be administered as part of the interviewprocess. In the work
sample, the applicant must respond to four different phonecalls: a customer who isirate about an out-of-stock item, a customer who wants moreproduct informationabout an item than was provided in the catalog, a customer whowants to changean order placed yesterday, and a customer who has a routine orderto place. Using a1–5 rating scale, the interviewer rates the applicant ontactfulness (T) and concernfor customers (C). The interviewer is provided with a rating manualcontainingexamples of exceptional (5), average (3), and unacceptable (1)responses by theapplicant.A random sample of 50 current CSRs were chosen to participate inthe study. AtTime 1 they were administered the clerical test and the worksample; performancedata were also gathered from company records for error rate(number of errors per100 orders), speed (number of orders filled per hour), andcustomer complaints(number of complaints per week). At Time 2, one week later, theclerical test andthe work sample were re-administered to the CSRs. A member of theconsultingfirm sat in on all the interviews and served as a second rater ofperformance on thework sample at Time 1 and Time 2. It is expected that the clericaltest and worksample will have positive correlations with speed and negativecorrelations witherror rate and customer complaints.Results for Clerical Test
Time 1 Time 2Mean score 31.61 31.22Standard deviation 4.70 5.11Coefficient alpha .85 .86Test–retest r .92*r with error rate –.31* –.37*
Should Phonemin use both the clerical test and the work sample?Why or why not? Tip: Be sure to consider the criterion-relatedvalidity estimates for the clerical test and the work sample asthey relate to each of the three performance criteria (error rate,speed, complaints).
In addition, are both the tactfulness (T) and concern forcustomers (C) interviewer ratings from the work sample needed? Whyor why not? If not, which would you retain and why? Tip: This is atough question, but the data available to you provide ananswer.
r with speed .41* .39*r with complaints –.11 –.08r with work sample (T) .21 .17r with work sample (C) .07 .15Results for Work Sample (T)
Time 1 Time 2Mean score 3.15 3.11Standard deviation .93 1.01% agreement (raters) 88% 79%r with work sample (C) .81* .77*r with error rate –.13 –.12r with speed .11 .15r with complaints –.37* –.35*
Results for Work Sample (C)
Time 1 Time 2Mean score 2.91 3.07Standard deviation .99 1.10% agreement (raters) 80% 82%r with work sample (T) .81* .77*r with error rate –.04 –.11r with speed .15 .14r with complaints –.40* –.31*(Note: * means that r was significant at p < .05)
After reading the description of the study and observing theresults above,
per year to coincide with seasonal changes in customers’ appareltastes. Custom-ers may order merchandise from the catalog via mail or over thephone. Currently,
70% of orders are phone orders, and the organization expectsthis to increase to85% within the next few years.
The success of the organization is obviously very dependent onthe successof the telephone ordering system and the customer servicerepresentatives (CSRs)who staff the system. There are currently 185 CSRs; that numbershould increaseto about 225 CSRs to handle the anticipated growth in phone ordersales. Thoughthe CSRs are trained to use standardized methods and procedures forhandlingphone orders, there are still seemingly large differences amongthem in their jobperformance. CSR performance is routinely measured in terms oferror rate, speed
of order taking, and customer complaints. The top 25% and lowest25% of per-formers on each of these measures differ by a factor of at leastthree (i.e., the
error rate of the bottom group is three times as high as that ofthe top group).Strategically, the organization knows that it could substantiallyenhance CSR
performance (and ultimately sales) if it could improve itsstaffing “batting aver-age” by more accurately identifying and hiring new CSRs who arelikely to be top performers.The current staffing system for CSRsis straightforward. Applicantsare recruitedthrough a combination of employee referrals and newspaper ads.Because turnoveramong CSRs is so high (50% annually), recruitment is a continuousprocess atthe organization. Applicants complete a standard applicationblank, which asksfor information about education and previous work experience. Theinformationis reviewed by the staffing specialist in the HR department. Onlyobvious misfitsare rejected at this point; the others (95%) are asked to have aninterview with thespecialist. The interview lasts 20–30 minutes, and at theconclusion the applicantis either rejected or offered a job. Due to the tightness of thelabor market and theconstant presence of vacancies to be filled, 90% of theinterviewees receive job
offers. Most of those offers (95%) are accepted, and the newhires attend a one-week training program before being placed on the job.
The organization has decided to investigate the possibilitiesof increasing CSReffectiveness through sounder staffing practices. In particular,it is not pleased
with its current methods of assessing job applicants; it feelsthat neither the application blank nor the interview provides anaccurate and in-depth assessment of
the applicant KSAOs that are truly needed to be an effectiveCSR. Consequently,it engaged the services of a consulting firm that offers variousmethods of KSAOassessment, along with validation and installation services. Incooperation withthe HR staffing specialist, the consulting firm conducted thefollowing study forthe organization.A special job analysis led to the identification of severalspecific KSAOs likelyto be necessary for successful performance as a CSR. Three ofthese (clerical
speed, clerical accuracy, and interpersonal skills) weresingled out for further con-sideration because of their seemingly high impact on jobperformance. Two new
methods of assessment provided by the consulting firm werechosen for experi-mentation. The first is a paper-and-pencil clerical testassessing clerical speed and
accuracy. It contains 50 items and has a 30-minute time limit.The second is a briefwork sample that could be administered as part of the interviewprocess. In the work
sample, the applicant must respond to four different phonecalls: a customer who isirate about an out-of-stock item, a customer who wants moreproduct informationabout an item than was provided in the catalog, a customer whowants to changean order placed yesterday, and a customer who has a routine orderto place. Using a1–5 rating scale, the interviewer rates the applicant ontactfulness (T) and concernfor customers (C). The interviewer is provided with a rating manualcontainingexamples of exceptional (5), average (3), and unacceptable (1)responses by theapplicant.A random sample of 50 current CSRs were chosen to participate inthe study. AtTime 1 they were administered the clerical test and the worksample; performancedata were also gathered from company records for error rate(number of errors per100 orders), speed (number of orders filled per hour), andcustomer complaints(number of complaints per week). At Time 2, one week later, theclerical test andthe work sample were re-administered to the CSRs. A member of theconsultingfirm sat in on all the interviews and served as a second rater ofperformance on thework sample at Time 1 and Time 2. It is expected that the clericaltest and worksample will have positive correlations with speed and negativecorrelations witherror rate and customer complaints.Results for Clerical Test
Time 1 Time 2Mean score 31.61 31.22Standard deviation 4.70 5.11Coefficient alpha .85 .86Test–retest r .92*r with error rate –.31* –.37*
Should Phonemin use both the clerical test and the work sample?Why or why not? Tip: Be sure to consider the criterion-relatedvalidity estimates for the clerical test and the work sample asthey relate to each of the three performance criteria (error rate,speed, complaints).
In addition, are both the tactfulness (T) and concern forcustomers (C) interviewer ratings from the work sample needed? Whyor why not? If not, which would you retain and why? Tip: This is atough question, but the data available to you provide ananswer.
r with speed .41* .39*r with complaints –.11 –.08r with work sample (T) .21 .17r with work sample (C) .07 .15Results for Work Sample (T)
Time 1 Time 2Mean score 3.15 3.11Standard deviation .93 1.01% agreement (raters) 88% 79%r with work sample (C) .81* .77*r with error rate –.13 –.12r with speed .11 .15r with complaints –.37* –.35*
Results for Work Sample (C)
Time 1 Time 2Mean score 2.91 3.07Standard deviation .99 1.10% agreement (raters) 80% 82%r with work sample (T) .81* .77*r with error rate –.04 –.11r with speed .15 .14r with complaints –.40* –.31*(Note: * means that r was significant at p < .05)
After reading the description of the study and observing theresults above,