Wednesday, May 18, 2011

Monetizing Program Effect

The university hosted a session by SkillSoft and Knowledge Advisors on learning metrics.

This is part 2.

Here – they are trying to put a hard number towards skill gains.

Comments highly welcome and will be forwarded to Kevin at SkillSoft. I think there will be lots to discuss here.

Below is an example calculation
--------
Factors
Cost = eLearning cost per license: 50\$
[The debate on what costs to include continued. My current thinking – WWAD (what would Accounting do?)]

Salary = average salary across learner population: \$50,000/yr

Survey
Knowledge Advisors recommended surveying this in 10% increments from 0 to 100%. Then use the average.

Lsi = Learner’s estimation of Skill Improvement: 30%
Ltc = Learner’s estimation of Training Contribution: 60%
ST = How much of the job uses that skill? : 20%

Hawthorne Effect
Bias = 100 – Hawthorne Effect (est 35%) = 65%
--------
Lsi x Ltc x ST x Bias = Training’s impact on skill adjusted for self-reporting (TI)
30% x 60% x 20% x 65% = 2.34%

Salary x TI = Calculated Monetized Benefit (CMB)
\$50,000 x 2.34% = \$1170

CMB – Cost = Calculated Net Benefit (NetB)
\$1170 - \$50 = \$1120
------------
(NetB / Cost) x 100 = ROI per participant
(\$1120 / \$50) x 100 = \$2240

CMB / Cost = Benefit / Cost Ratio
\$1170 / \$50 = 23.4:1
For every \$1 spent - \$23.40 returned
------------
Further notes

You can change the bias. Sit with your stakeholder to help determine the bias.

Hawthorne effect – almost every organization will demonstrate this when you begin to measure. Natural gravity.

Cost – you will need to use the actual cost number.
- The team recommended using license cost as a solid number and divide by the number of active users.
+ Stress for people who access the content – this is what we see.
+ May want to measure what happens if people DON’T use the content
- Again the argument went up that we need to consider other costs and not just the license. [see a trend?]