Question

XYZ is a new start up company that has about 150 employees. The company provides plumbing services to residential customers. The company has had a high turnover of plumbers, mostly due to the plumber's lack of knowledge and customer service skills.  In order to address this, the company wanted to strengthen their selection process.  So, they followed the procedure outlined below:  

1. They selected 40 of their most successful plumbers (those who received outstanding customer feedback and had near zero calls from the customers about the work performed by these plumbers).

2.  The company developed a 50-items multiple choice exam with four possible answers with only one answer being the correct one. Also, the company had another 50 questions consisting of True or False choices. Thus, the test had 100 questions. They  called this plumbing knowledge test (PKT).  

3. In order to strengthen the faith in the PKT, the company, after 6 months, chose the same 40 plumbers and this time they did something different. 20 of the 40 plumbers were given the exact same test they took six months ago (PKT). The remaining 20 plumbers were again given another 100 items test (50 multiple choice and 50 true false questions) but similar content that tested plumbing knowledge.  

4. The company also developed a personality test that tapped into customer service skills, interpersonal skills, teamwork, persuasion skills, friendliness, etc. This personality test had about 50 statements (e.g., "Customers are always a priority for successful business" with answer format ranging from 1 = Strongly Agree, 2 = Agree, 3 = Neither Agree nor Disagree, 4 = Disagree, and 5 = Strongly Disagree.  

5. Based on the scores on PKT and also on the personality test, candidates were short-listed and 3 managers interviewed the applicants separately. Each one ranked the candidate on a 5 point scale with 5 being most suitable and 1 being unsuitable.

Having completed this process, the company is now clueless as to what to do with the data.  In other words they are wondering whether their system is reliable or not. Based on your knowledge of various reliability concepts, discuss:

a. For each of the three (PKT, Personality Test, and Interviews) which are the appropriate reliability technique to use? Explain why?

b. The company also learned of something called split-half reliability and KR20 formula. They want to see what it is. Which of the above three (PKT, Personality Test, and Interview Ratings) can use this technique to assess reliability? Why?

Answer 1

1. Plumber knowledge test :

Written tests are only provide basic knowledge about how to provide services. sometimes multiple choice questions may also based on sheer luck. Testing the knowledge is important. So for selection first of all written test provides us a basis to select candidates from a huge applicants.

2. Personality test: this is the most important test which should be completed by each person. for increasing the experience practical training and inter personal skills. So this test plays a key role in selecting the person. this test consist of 50 statements. it provides the details of mindset, calmness, analytical skills of the person.

Whoever qualified PKT, they should pass through personality test in order to get selected.

3. Interview :

Based on Interview we can understand the confidence and passion towards work of the candidates.Whoever got highest marks in the scale of 5 will be shortlisted.Interview gives us how he can survive in the competition and the communication skills.

SO out of the three tests, best performers will be selected.

Major areas are personality test and interview marks.