We receive so much valuable content from our
customers, partners and internal subject-matter experts. On Thursdays, we
thought it would be fun to do a “throw back” and bring back an older post that
was valuable to our audience. Today’s throw back is about measuring the effectiveness of training.
How
to measure training effectiveness
We are pleased to bring you a guest
blog post today from Martin Klubeck, a Skillsoft® Books24x7® BusinessPro™ Collection author. Klubeck is a strategy and planning consultant at the
University of Notre Dame and a recognized expert in the field of practical
metrics. His passion for simplifying the complex has led to the development of
a simple system for developing meaningful metrics. Klubeck is also the founder
of the Consortium for the Establishment of Information Technology Performance
Standards, a nonprofit organization focused on providing much-needed standards
for measures. His most recent book published by Apress - Metrics: How to
Improve Key Business Results - is available in the Skillsoft Books24x7
BusinessPro collection.
By Martin Klubeck, Strategy and
Planning Consultant for the University of Notre Dame
Learning solutions can be very expensive. Are you getting
your money’s worth? Metrics can be used to measure the effectiveness of your
training.
Metrics are at their best when they
are designed to answer a root question – not when they are driven by a need to
compare ourselves to our peers or in finding “interesting” viewpoints. In the
case of learning solutions, the root question often times is, how effective is
the training?
Or another way of looking at that
question is, how does the learning solution affect the capability of the
students? In business, these students are normally workers and managers
attending a diverse set of learning opportunities for the purpose of improving
their performance of specific tasks. I highly recommend using tasks (and
subsequent task-breakdowns) for focusing training metrics on what really
matters – the attainment of skills.
Metrics are a great tool for
providing the answers to how effective training is and the necessary corollary
question – what is the level of capability before the training? All of
this is simpler in relation to a task. This baseline is required so that you
can correctly measure the change after the training.
By applying the rules for developing
metrics found in Metrics: How to Improve Key Business Results,
my recently published book from Apress, you can identify the right measures.
Let’s look at one sample task to demonstrate – writing HTML5 code:
Capability can be measured by three
outputs – specifically speed, accuracy, and efficiency.
Speed. How fast does the worker write code before and after
training? You’ll want to know how long it takes to perform the task, not how
long it takes in total time since this would include non-value added time
(breaks, time to power up the computer, answering the phone, etc.) So this
requires the worker to track the actual time-to-task. It is important to use
measures which if the training was successful, we would expect to see
(positive) change and speed fits that description.
Accuracy. How accurate is the coding effort? Does it have to be
reworked or rewritten? If you use rework as a measure of accuracy, it can be in
the form of time spent allowing you to use the same data used for Speed. You
can also use defects or errors. This is when the classic “defects per
lines-of-code (LOC)” comes in handy. If one worker has 10 errors and the other
has 5, which is better? Well it depends, if the worker with 10 errors wrote
1000 lines of code and the one with “only” five errors wrote only 10 lines of
code, which would you choose?
Efficiency of the Code. Efficiency helps us to see why Triangulation is important.
In the scenario for Accuracy, it isn’t enough to know that the
five-error-programmer only wrote 10 LOC. It is important to know how efficient
both programmers’ code is. If you only look at errors-per-LOC, you might choose
the one with 10 errors since it was spread over many more lines of code (1
error for every 100 lines vs. 1 error for every 2 lines of code). But if the
one writing 1000 lines of code was not very efficient, and wrote more code than
needed, your choice of the better programmer could be wrong (for example if the
code they wrote did exactly the same things).
By looking at all three measures
together, Speed, Accuracy, and Efficiency of the code, we can build a picture
of how capable or skilled the worker is. By measuring this performance before
and after the learning solution, we can determine the effectiveness of the
training for improving the skill of the worker to perform given tasks.
The steps for measuring the
effectiveness of training can be summarized as:
- Identify the performance (in the form of specific
tasks) you are trying to improve. If necessary perform task breakdowns, as
was done in the example of writing HTML5 code.
- Determine which attributes you want to see positively
affected. Speed, Accuracy, and Efficiency of the code were used in our
example.
- Measure those attributes before the training.
- Measure those attributes after the training.
The final step is to again measure
the attributes after an extended period of time (say six months) to determine
the retention of the improvements. Many times performance improves for reasons
other than effective training so measuring performance over time helps ensure
you are getting a true picture.
To learn more about using metrics to
improve key business results, read my new book Metrics, which is
available in paperback, Kindle, and other
electronic formats.
