Evaluation: Take a strategic approach

Piers Lea explains why evaluation should be part of the strategy

In an earlier article published on the LINE website and in E-learning Age Magazine, we issued a call to action for organisations to contribute their stories and evidence of success. A call for our industry to focus on what has been achieved, and how it has been achieved, rather than solely on the efficacy or otherwise of particular technology tools and conceptual models.

In this piece I’d like to explain a little why I think this results focus is so important.

In learning and communications, as in other fields of human endeavour, it’s not enough to achieve results. You also have to demonstrate results. It helps, as well, if the results you demonstrate bear some relation to what you set out to accomplish. Our experience in working and consulting with clients is that the training they do frequently get results – the problem is often that they don’t know what those results are!

Any type of training has results of some kind. The result of your training might be that its recipients feel it was so irrelevant to their needs that applications for training go down across the organisation. That would be a result – but a bad result. Then again you might produce a programme that has a good – but irrelevant – result: your programme might make everybody feel positive about themselves (good), but have no positive effect whatsoever on behaviours that are dragging the organisation down (bad). The best possible result, and the most impressive one to be able to demonstrate, is one that impacts directly on organisational performance; whether this be in sales volumes, profitability, or ethical operation – in other words, a result aligned with the strategic priorities of the organisation.

So much, so obvious, perhaps. Why is it then that so few organisations evaluate their results, and so little of the learning & development they do is effectively aligned with strategic objectives?

Don’t get me wrong, not every project has to have a company-wide strategic focus to be valuable. LINE is a multiple award winner for projects. But the issue is, beyond the proof of efficacy offered by awards, has a proper framework been set up to judge success, so that we can implement good, strategically placed blended learning and communications programmes?

LINE’s favoured approach in working with clients is to set a framework for programmes at the outset that includes impact and ROI measures. If you can do this before anything else, you have a much greater chance of getting results.

E-learning is front loaded in terms of costs – in cash-strapped times this makes it hard to do, so people are asking for proof of efficacy – and quite rightly so. However instructor-led training, the more traditional model, has not previously had the same sort of questions asked as to whether it actually works. So standards of comparison are often lacking that could inform proper judgements about how and when to deploy one or the other. Neither is it a binary choice: whether ‘e-’ or ‘classroom’ is the better solution is not the issue. In the twenty-first century, learning & development should embraces all delivery methods, and the question of which channel or channel mix should be deployed must be judged according to a strategy which takes into account the particular needs, aspirations and resources of the organisation in question, not according to prejudice or some arbitrary rule of thumb.

So how can organisations make sensible decisions about how to move forward?

You might say that learning and development within the organisation needs a vision – but where this word tends to conjure up something rather numinous and vague, I would want to substitute the metaphor of a map; but not an old fashioned map – one more like those of our client TeleAtlas (who supply the Google maps we know so well). In this you can see the globe, but easily zoom into areas that are felt to be of strategic importance, but also see overlays of different types of data. In learning and communications terms this means data that relates to all the varieties of ‘intellectual capital’ in the organisation and it reaches individuals when and where they want it. This is the picture we build up in creating a learning and development strategy…and evaluation plays a central role. It’s central because whatever anyone might say, as no one knows what will work between one culture and the next you will need to try different routes and test their efficacy

At a recent public event LINE attended, 12 leading multinational companies in banking, insurance and FMCG were asked whether they had either a learning and development or learning technology strategy. Not one said yes. None of them had the picture.

Without the map- which functions as the summary of all you want to achieve, how can you establish appropriate measurements for your results? How will you know when you’re succeeding (or failing)?

LINE has tools and partnerships in place to help organisations in creating this picture. The LINE Healthcheck is a consulting ‘product’ that has been created with this need in mind. A typical output from this might be a 3 yr strategy, with implementation support.

We have been working with a major car manufacturer since 2007 to build up the picture and to realise the aspirations it embodies. The heart of the ongoing process we have put in place is a set of measures based on key performance indicators across a broad range, including time-to-market and measuring cost-effectiveness at cost point. The system has feedback loops in place so that we can refine what we deliver to learners. Across a global audience comprising tens of thousands of staff, we have segmented those learners in response to cultural differences and level of understanding. Feedback is essential in refining and steering this programme. Technology is not just for delivery: we also use it to listen.

The drive in all our strategic-level work of this kind is to help clients set a framework to establish measures which show how a blended learning picture is building. To mention awards again, our success in demonstrating results has recently been proved by our winning of a National Training Award for our work with the Identity and Passport Service. The National Training Awards are renowned for the rigour of their judging, and an insistence on demonstrating real, relevant results.

HR is often heard to want a place at the top table of strategic decision-making – approaching L&D strategically, aligning it with the needs of the organisation – is one of the changes which will give HR a more strategic role. In our view evaluation is far too important a part of this to overlook.