Monday, March 23, 2009

Evaluating the Usefulness of Training

Evaluating the usefulness of training
Presenter: Gwen Burbank – SunGard Higher Education

……………came in late. Walked into my 2nd choice session. Um…no. The step by step of an upgrade. Focused on “and we notify our people when the production instance is live.” Don’t need that. Lived through way too many upgrades.
------------------------------------
Don’t just hand the person the workbook. Gotta look at your personal processes.
- May take 3 different workbooks.
- Need to do planning on paper before create first item.
- Figure out the media.

(Talking about the ADDIE Model and Kilpatrick levels)
- Will evaluate throughout the full process.
- SME (subject matter expert). (She is talking instructional design 101).

People can handle 5-9 items at a time.
- 12 seconds before that information leaves.
- Challenge to training – from short term to long term
- Biggest way – repetition.
- Also – visuals
- Assess when ready to move on to next concept.

Banner hierarchical. Missing step 1 – won’t understand the next step.
- Get people to do it.

Many have person in back actually watching (in my instance – me.)

Often have 1 on 1 in many organizations.

More difficult Banner tasks – many folks doing 1 on 1.

Assessment serves as part of learning process. People learn from mistakes.
(how realistic is the assessment).

One idea – performance checklist.
- Log into system
- Access the form to enter employees (can create this using Captivate!!!!)
- Generate an ID
- Select an option to generate an ID.

Navigating CBT – let me try!
- Using Captivate to build CBTs.
- Use the Assessment mode (or variation thereof)

Can also use matching game (name of the form with how used)
Jeopardy also an option.

Work with SME to develop the game. They become more bought into it. The SME will be the one who sells the idea, not the trainer.

Had each of them write 10 questions about General Person form. They are writing the questions and quizzing each other.
- The game then customizes. To the class.

Behavior change – what happens out on the job.
- Did they actually learn it?
- Can sell the management (gotta show increased performance)
- Measuring this could potentially take a lot of time. May require hanging out in the environment. (Management by Walking Around) “How’s it going?”
- Results could be impacted by other factors. (Big key – what are you actually measuring?)
- Sift – performance related to Training or to the Job. Big difference.
- Can send self-check
- (couldn’t you also run a report off of Banner? W/ objective (say – drop in errors), can see.

(Ok – I’m talking way too much)

Level 4 – Results.
- Find the things you are measuring.
- Look at reports.
- What is the concrete business result. (i.e. how many hours. How much $$$$)
- Run the report BEFORE and after.
- Remember – Return on Investment.
- Will require statistical analysis. (Analysts are your friends. And DEFINE what you want to accomplish).

Evaluation strategy
- Start small then build. (Level 1)
- Look at the culture of the organization to see how it will work.
+ (in my organization – how invisible can you make it)

Develop evaluation strategy
- Determine the need for evaluation
- Select the level
- Design the evaluation tool
- Pilot the tool
- Implement the evaluation
- Analyze the results
- Report the results

No comments: