Wednesday, May 18, 2011

Monetizing Program Effect

The university hosted a session by SkillSoft and Knowledge Advisors on learning metrics.

This is part 2.

Here – they are trying to put a hard number towards skill gains.

Comments highly welcome and will be forwarded to Kevin at SkillSoft. I think there will be lots to discuss here.

Below is an example calculation
--------
Factors
Cost = eLearning cost per license: 50$
[The debate on what costs to include continued. My current thinking – WWAD (what would Accounting do?)]

Salary = average salary across learner population: $50,000/yr

Survey
Knowledge Advisors recommended surveying this in 10% increments from 0 to 100%. Then use the average.

Lsi = Learner’s estimation of Skill Improvement: 30%
Ltc = Learner’s estimation of Training Contribution: 60%
ST = How much of the job uses that skill? : 20%

Hawthorne Effect
Bias = 100 – Hawthorne Effect (est 35%) = 65%
--------
Lsi x Ltc x ST x Bias = Training’s impact on skill adjusted for self-reporting (TI)
30% x 60% x 20% x 65% = 2.34%

Salary x TI = Calculated Monetized Benefit (CMB)
$50,000 x 2.34% = $1170

CMB – Cost = Calculated Net Benefit (NetB)
$1170 - $50 = $1120
------------
(NetB / Cost) x 100 = ROI per participant
($1120 / $50) x 100 = $2240

CMB / Cost = Benefit / Cost Ratio
$1170 / $50 = 23.4:1
For every $1 spent - $23.40 returned
------------
Further notes

You can change the bias. Sit with your stakeholder to help determine the bias.

Hawthorne effect – almost every organization will demonstrate this when you begin to measure. Natural gravity.

Cost – you will need to use the actual cost number.
- The team recommended using license cost as a solid number and divide by the number of active users.
+ Stress for people who access the content – this is what we see.
+ May want to measure what happens if people DON’T use the content
- Again the argument went up that we need to consider other costs and not just the license. [see a trend?]

No comments: