IPS case study, proving the blend works
Posted by Piers Lea - December 5, 2008
You might think that ‘business impact’ studies on learning interventions would be two-a-penny. We have private and public sector organisations investing in training interventions like never before, every manner of exhibition or conference with its pundits espousing the efficacy of technology, face-to-face, blended, mobile, etc – it should be easy to prove that training works! Not so.
Last year when working on an envisaging project with a professional services firm we searched the world for good studies looking at the effectiveness of training. The search included going via the Elliot Masie network in the US and ELIG in Europe. Incredibly, we drew an almost complete blank. There are loads of studies to show ROI on different forms of training; but almost none of these studies show that the money was well spent in business terms. And we wonder why training budgets are the first to be cut in downturn?
So it seemed to me that it’s worth putting pen to paper about the recent success of the Identity and Passport Service (IPS) training programme for management and staff of the new Interview Offices around the UK.
Some background
When the Identity and Passport Service introduced the new Authentication by Interview (AbI) process by which new customers for UK passports would need to go through an interview first, the organisation faced the phenomenal challenge of having to open 68 new regional offices, hire approximately 600 new staff and train these folks on everything from office management procedures and systems training to interviewing skills. Their training need was immense.
Why is this important?
Well clearly I think it’s important because LINE was selected as the lead contractor in a two-year major design and delivery of a blended learning programme that has been independently judged to be successful!
But I think it’s of general interest to our industry for two reasons:
1. IPS have conducted a true business impact study
2. The study points to the efficacy of providing a true blend
So here are the results of the study conducted by KnowledgeAdvisors using their system called ‘metrics that matter’ for the IPS blended training programme…
The result
This table summarises the result of the project that comprised the following elements:
• 15 Day Facilitated Blended Training Programme (Managers)
• 5 Day Facilitated Blended Training Programme (Interviewers)
• 30+ Hours of e-learning (includes e-scenarios, e-tutorials, systems training, e-assessments)
We worked with PricewaterhouseCoopers as our key training delivery partner. They had proven expertise in delivering interviewing skills training with their work for the Northern Ireland Constabulary (amongst others).
Scan the result:
Identity & Passport Service – IO/IOM Programs
Executive Summary
| This executive summary highlights KnowledgeAdvisors observations as it relates to Identity & Passport Service versus the Leadership Benchmark. | |
| The leadership benchmark are clients of KnowledgeAdvisors using the same evaluation questions as Identity & Passport for courses they designated | |
| Key Observations
|
|
| * The Identity & Passport Service IO and IOM programs have had significantly higher impact than typical training & leadership programs. | |
| * The Identity & Passport Service IO and IOM programs have had an impact on more business results than typical training & leadership programs. | |
| * Specifically, quality and customer satisfaction are significantly higher business result performance than typical leadershp programs | |
| * Specifically, the on the job performance change for both the IO and IOM programs is significantly higher than the benchmarks. | |
| * Where typical leadership programs have negative ROI months later, Identity and Passport Service has positive ROI of roughly 2 to 1 | |
| * Whereas typical leadership programs have the training as nearly 30% of the barrier to use, this is roughly 33% less of a problem at Identify & Passport indicating stronger Leadership programs evident in the results | |
| * The amount of work time Identity & Passport Services use the training on the job is over 50% higher than the benchmark indicating useful and timely training | |
| * Overall Identity & Passport Service data, when compared to the benchmark, shows superior training and results thereafter | |
For me the key result lies in the penultimate point. This is a highly distributed workforce undertaking a complex set of tasks, requiring a high level of skill. It seems obvious that a ‘core’ of (e) learning materials – together with a ‘dry-running’ programme – all available on-site would have an obvious benefit. (The ‘e’ element is also used throughout the classroom training)
This appears to prove the point.
So when people ask you whether blended learning really works. The answer is – of course – yes. People have been using blends for years. It’s just that now we have design methodologies that make effective use of technology, not just as an adjunct, but at the core training thinking.
What did the trainees think?
| Statement | Average (out of 7) |
| I learnt new knowledge and skills from this training |
6.5 |
| I will be able to apply the knowledge and skills learned to my job |
6.4 |
…and the finance department?
| ROI calculation table | Interviewer staff |
| Estimated change in performance |
69.2% |
| Percentage improvement in performance due to training |
38.27% |
| Delegate cost |
£20,587 |
| Cost of performance change related to training |
£8,171 |
| Course costs against the benefit |
£4,300 |
| Overall benefit |
£2,343.62 |
| Cost benefit ratio |
1:1.9 |
…and the staff view on the investment:
| Statement | Average (out of 7) |
| This training was a worthwhile investment in my career development |
6.5 |
| This training was a worthwhile investment for IPS |
6.5 |
…and the staff view on the investment:
…and finally the customers. According to a recent external customer survey, 96% of first time interviews are rated positively by those undergoing them.
It took some considerable effort on the part of IPS to get these studies done. We should applaud their initiative and encourage others to do the same. That way we may be able to build up a more evidence-based picture of a) whether training really ‘works’ and b) which methods tend to work best in which circumstance.
by Piers Lea
