Relevance Trumps ROI

Learning leaders may find themselves devising elaborate measurement reports to prove the efficacy of specific programs. But, at the end of the day, business relevance should take precedence over dollar signs.

“For 50-odd years, the learning profession has said: training happens in a classroom; we evaluate that and we find out what the ROI was of that training event. I think the whole model is backwards,” said Dan Pontefract, head of learning and collaboration at TELUS, a telecommunications company.

“What we should be doing is articulating what type of performance change we’re looking for in a person, in a team, in an organization, their particular goals and how the new learning and collaboration aspect that is formal, informal, social learning all align to that,” he said.

Determining Relevance: Tangible vs. Intangible
Starting with a business goal or problem to be solved around corporate culture, knowledge management or even systematic training can eliminate a narrow measurement focus, or as Jay Cross, CEO of the Internet Time Alliance, a knowledge exchange organization, explained it, getting hung up “on doing that part right rather than asking again and again: ‘Is this improving the business?’ ‘Is this helping us attain our current objectives?’ ‘Is this delighting our customers?’ And if it’s not, they shouldn’t be doing it,” he said.

Some learning leaders — perhaps fearful for their budgets and status as business partners — may be wary of seemingly unquantifiable learning initiatives such as social learning, but hard numbers aren’t always the best indicator of success. A focus on formal school or executive education-type learning involving tests, for example, may not provide valuable metrics anyway, Cross said, because grades or test results in school are unrelated to anything outside of school. They are essentially the wrong measures.

Cross said there is actually a significant amount of learning taking place in formal situations that fails to translate to behavior change on the job. To increase the likelihood of behavioral change, gathering immediate metrics — smile sheets for example — might not be as beneficial as waiting to ascertain whether learning stuck and is being applied on the job.