Question

Question:

Evaluation: Answer both (a) and (b): (a) Looking at Table 6.1 in our text, explain how you would evaluate this training for a “Behavior and skill-based” outcome. (b) Explain how you would do a "Return on Expectations?

To evaluate its training program, a company must decide how it will determine the program’s effectiveness; that is, it must identify what training outcomes or criteria it will measure.

Table 6.1 shows the six categories of training outcomes: reaction outcomes, learning or cognitive outcomes, behavior and skill-based outcomes, affective outcomes, results, and return on investment.¹¹

TABLE 6.1 Evaluation Outcomes

Outcome or Criterion Reactions	Level 1	What is Measured Learners’ Satisfaction.	Examples Comfortable Training room. Useful materials And prog content.	Measurment Methods Surveys Interviews.	Question Did they Like it?.
Learning or Cognitive	2	Principles, facts, techniques, Procedures, or Processes that The learners have acquired.	Electrical principles. Safety rules. Steps in interviewing.	Tests.              Work Samples.	What did They learn?.
Behavior And Skill-based	2 or 3	Interpersonal Technical or motorskills or behavior acquired by learners	Preparing a dessert. Sawing wood. Landing an airplane. Listening.	Tests. Observations. Self, peer, Customer and Managers Rating. Work samples.	Do they use it?.
Affective	2,3 or 4	Learners’ attitudes and motivation	Tolerance for Diversity. Safety attitudes. Customer serv orientation. Engagment.	Attitude surveys. Interviews. Focus groups.	Did it change their attitudes?.
Results	4	Payoffs for the company	Productivity. Quality. Costs. Repeat customer satisfaction. Accidents.	Observation. Performance. Date from records or Company databases.	Did it impact the bottom line?.
Return on Investment	5	Identification and comparison of Learning benefits with costs	Dollar value of productivity divided by training costs.	Economic value.	What is the return from investing in learning?.

Table 6.1 shows training outcomes, the level they correspond to in Kirkpatrick’s evaluation model, a description of each of the outcomes and how they are measured, and the question that each outcome can help answer. Kirkpatrick’s original evaluation model included only four levels (reaction, learning, behavior, and results) but recent thinking suggests a fifth level—return on investment (ROI)—is necessary to demonstrate the financial value of training. Both level 1 and level 2 outcomes (reactions and learning) are collected at the completion of training, before trainees return to the job. Level 3 outcomes (behavior/skills) can also be collected at the completion of training to determine trainees’ behavior or skill level at that point. To determine whether trainees are using training content back on the job (i.e., whether transfer of training has occurred), level 3, level 4, and/or level 5 outcomes can be collected. Level 3 criteria can be collected to determine whether behavior/skills are being used on the job. Level 4 and level 5 criteria (results and return on investment) can also be used to determine whether training has resulted in an improvement in business results, such as productivity or customer satisfaction. These criteria also help determine whether the benefits of training exceed their costs. Keep in mind that the levels do not indicate the importance of the outcomes or that lower-level outcomes cause higher-level outcomes.¹² That is, reactions cause learning, which in turn influences skills and results. The outcomes that are collected in evaluation are based on training needs, program objectives, and the strategic reasons for training. We discuss this in the section “Evaluation Practices,” later in the chapter
.

Answer 1

Kirkpatrick's Four-Level Training Evaluation Model can help you to answer questions like these. You can use it to objectively analyze the impact of training, to work out how well your team members learned, and to improve their learning in the future.

Kirkpatrick's model remains popular, but it should be used with care. The basic structure is now more than 60 years old (despite its many updates), and the ways that people learn and organizations operate has changed radically in this time. Even the term "training" has been largely replaced by "learning and development."

Today, other, non-formal methods of workplace training are often more popular and effective (as shown by the 70:20:10  model). And, with the rise of personalized, user-directed learning, formal training is becoming less prominent. Kirkpatrick's model is not necessarily suited to this new approach to learning.

Another drawback is that Levels 3 and 4, which arguably yield the most useful information for the business, are time-consuming, resource-intensive, and expensive to implement. So the model may not be practical for all organizations, especially if you don't have a dedicated training or HR department to conduct the analysis. And, it's not ideal for all situations, such as one-off training.

Most importantly, organizations change in many ways, and these changes affect behaviors and results, as well as training. For example, measurable improvements in retention and productivity could result from the arrival of a new boss, or from a new computer system, rather than training. Or it could be a combination of these.

Kirkpatrick's model is great for evaluating training in a "scientific" way, but with so many possible variables, Level 4 may be limited in its usefulness.