Learning Measurement & Training Evaluation
Parent: Technical Instruction & Engineering Education · researched 2026-06-16T22:31:34.717Z· 15 sources · 8 concepts · skill learning-measurement-evaluation
Reference for measuring training and enablement program effectiveness across the full evaluation stack — from post-workshop smile sheets to executive ROI reports and cross-system xAPI analytics.
Learning Measurement & Training Evaluation
- Reference for measuring training and enablement program effectiveness across the full evaluation stack - from post-workshop smile sheets to executive ROI reports and cross-system xAPI analytics. [source]
- See references/learning-measurement-context.md for the full deep-reference with worked examples, decision tables, and vendor comparisons. [source]
Evaluation Framework Overview
- Three frameworks dominate training program evaluation. They are complementary, not competing: [source]
- When to use each: Kirkpatrick levels 1–4 apply to any formal training; New World Kirkpatrick adds the pre-design planning discipline; Phillips Level 5 is appropriate for 5–10% of programs - those that are high-cost, strategically critical, and directly tied to measurable business KPIs. [source]
Level 1: Reaction
- Measures whether participants found training favorable, engaging, and relevant. Relevance is the most predictive sub-component. [source]
- Limitation: A 2009 study (n=335) found no statistically significant correlation between Level 1 scores and Level 3 behavior change. Alliger & Janak (1989) found the Level 1→Level 2 causal link produced only r=.23. [source]
Level 2: Learning
- Measures acquisition of knowledge, skills, attitude, confidence, and commitment (5 components per the New World Model). [source]
Level 3: Behavior
- Measures whether participants apply what they learned on the job. Most organizations skip formal Level 3 evaluation - Kennedy et al. (2013) found 40%+ report it is not required by management. [source]
Level 4: Results
- Measures whether targeted organizational outcomes occurred. Requires pre-established KPI baselines and isolation methodology. [source]
Shared Accountability Model
- L1/L2 = L&D accountability; L3/L4 = shared accountability between L&D, managers, and the business. [source]
Required Drivers (Level 3 Addition)
- Post-training reinforcement systems management must provide: coaching, job aids, work review, recognition. [TENTATIVE: ~85% vs ~15% application figures are from Kirkpatrick-affiliated sources - treat as directional.] [source]
Leading Indicators (Level 4 Bridge)
- Short-term observations signaling whether critical behaviors are on track to produce results. [source]
Backward Design Mandate
- Plan evaluation in reverse: L4 KPI → L3 behaviors → L2 objectives → L1 design. [source]
Phillips ROI Methodology (Level 5)
xAPI, SCORM, and cmi5
- xAPI adoption ~17% (verified-as-of: 2026-06-16) despite 10+ years. SCORM still dominant at 81.7%. [source]
Training Transfer
- Baldwin & Ford (1988): transfer depends on trainee characteristics, training design, and work environment. Supervisor support is the single strongest driver (ρ=.51, Human Factors 2019). [source]
- LTSI (Holton, Bates & Ruona, 2000): 16-factor validated instrument for diagnosing transfer barriers. [source]
L&D Dashboard Design
- Three audiences: Executive (ROI, time-to-proficiency), Manager (behavior change, team completion), L&D ops (engagement analytics). [source]
- Top metrics: time-to-productivity, 30/60/90-day retention, manager reinforcement score, application rate, completion rate (process only - not performance). [source]
Evaluation Edge Cases
- Skip evaluation for cohorts under ~20 or one-time deliveries where overhead exceeds value. [source]
- Retrospective evaluation: use trend-line data, untrained comparison cohort, or SME estimates with documented confidence reduction. [source]
- ROI report structure: Executive summary → Methodology → Financial detail → Intangibles → Sensitivity analysis. [source]
- xAPI/LRS privacy: anonymize actor IDs for PII; GDPR applies in EU; FERPA applies to educational institutions. [source]
Children
- Kirkpatrick Four Levels (frontier)
- New World Kirkpatrick Model (frontier)
- Phillips ROI (Level 5) (frontier)
- xAPI / SCORM / cmi5 (frontier)
- Learning Record Store (LRS) (frontier)
- Training Transfer (Baldwin-Ford, LTSI) (frontier)
- Leading vs Lagging KPIs (frontier)
- L&D Dashboard Design (frontier)
Frontier under this node: Kirkpatrick Four Levels, L&D Dashboard Design, Leading vs Lagging KPIs, Learning Record Store (LRS), New World Kirkpatrick Model, Phillips ROI (Level 5), Training Transfer (Baldwin-Ford, LTSI), xAPI / SCORM / cmi5