Measuring Return on Investment from Extended Reality Digital Transformations in Corporate Training Environments

Scott J. Warren

University of North Texas

Christina Churchill

University of California, Berkeley

Annette Fog

Globe Life and
University of North Texas

Brent Tincher

Lockheed Martin

Janetta Robins Boone

NASA

Stephanie L. Robinson

University of North Texas

2

Introduction & Background

Organizations increasingly adopt Extended Reality (XR) technologies for training, yet struggle to quantify their value and effectiveness.

Digital transformation initiatives incorporate emerging technologies with uncertain ROI
XR encompasses simulation games, Virtual Reality (VR), and Augmented Reality (AR)
Measuring training effectiveness and financial impact remains challenging
Organizations need data-driven frameworks for technology investment decisions

This framework adapts service-based measurement models from higher education for corporate training contexts (Warren, Churchill, & Hayes, 2024).

Note: Performance data presented in this study are modeled for framework demonstration purposes

3

The Research Problem

Core Challenge

IT purchasing and implementation costs are tangible, but training outcomes and operational value are difficult to measure systematically (Kirkpatrick & Kirkpatrick, 2016).

Key Challenges

Uncertain financial value of technology investments
Lack of standardized metrics for training effectiveness
Complex operational impacts across organizations
Difficulty comparing different technology solutions

Research Questions

How can training improvements be quantified?
What metrics best predict ROI?
How do different XR technologies compare?
When does investment justify cost?

4

Research Objectives

Primary Objective

Develop a comprehensive analytics framework for evaluating return on assets from XR training implementations using operations analytics principles

Specific Goals

Measure training profits through time-on-task, knowledge acquisition, and error reduction
Compare cost-effectiveness across simulation games, VR, and AR implementations
Create extensible framework applicable to emerging technologies (generative AI, etc.)
Provide data-driven approach for informed investment decisions

5

ROI Framework Overview

Theoretical Foundation

Built on operations analytics principles (Evans & Lindner, 2012) to systematically measure training effectiveness against technology costs

Core Components

Performance metrics measurement
Cost-benefit analysis framework
Comparative evaluation methodology
Data-driven decision support tools

Framework Process Flow

Define Performance Metrics
Collect Training Data
Analyze Performance Outcomes
Calculate ROI
Compare Technology Solutions

6

Key Performance Metrics

Metric	Definition	Measurement Method
Time-on-Task	Duration to complete training and achieve competency	Hours logged from start to proficiency certification
Information Acquisition	Knowledge retention and skill development	Pre/post assessments and practical evaluations
Error Reduction	Decrease in mistakes during application	Error rate comparison (baseline vs. post-training)
Technology Costs	Total investment required	Hardware + software + implementation + maintenance

ROI = (Training Profits − Technology Costs) / Technology Costs × 100%

7

Illuminative Example

Framework application comparing three XR technologies for standardized corporate training scenarios

Simulation Games

Desktop/mobile software scenarios

Virtual Reality

Fully immersive VR headset training

Augmented Reality

Real-world digital overlay systems

Methodology

Each technology evaluated across identical training objectives using controlled conditions and standardized assessment protocols

8

Data Modeling Approach

Modeling Methodology

Performance data were modeled using established training effectiveness parameters, industry benchmark costs, and typical implementation timelines to demonstrate framework application

Performance Parameters

Time efficiency based on typical training completion rates (Kirkpatrick & Kirkpatrick, 2016)
Knowledge retention aligned with educational research findings (Smith & Martinez, 2021)
Error reduction rates from industry literature (Smith & Martinez, 2021)

Cost Parameters

Hardware and software costs from vendor pricing
Implementation costs based on industry averages
12-month ROI projection using standard amortization

Important: These modeled data serve to illustrate how the framework would be applied in practice. Organizations should collect empirical data from their specific implementations for accurate ROI calculations.

9

Technology Comparison

Implementation factors across XR technologies based on industry analysis (Smith & Martinez, 2021)

Factor	Simulation Games	Virtual Reality	Augmented Reality
Hardware	Standard PC/Mobile	VR Headset Required	AR Device/Tablet
Setup Complexity	Low	Medium	Medium-High
Cost per User	$50-200	$400-800	$500-1000
Training Time	2-4 hours	4-6 hours	3-5 hours
Scalability	Very High	Medium	Medium
Maintenance	Low	Medium	Medium-High
User Accessibility	Very High	Medium	High

10

Performance Results (Modeled)

Key Findings

VR demonstrated highest engagement and knowledge retention (Smith & Martinez, 2021)
AR most effective for error reduction in procedural tasks (Smith & Martinez, 2021)
Simulation games required shortest completion time
All technologies showed improvement over baseline training methods (Kirkpatrick & Kirkpatrick, 2016)

Note: Results are modeled data for framework demonstration

Performance Metrics by Technology

85

75

68

Simulation

72

92

81

VR

78

82

89

AR

Time Efficiency

Knowledge Retention

Error Reduction

11

Cost-Effectiveness Analysis (Modeled)

ROI Trends Over Time

Simulation Games

Virtual Reality

Augmented Reality

Best Performance

VR: Highest training effectiveness scores

Best Value

Simulation: Optimal cost-benefit balance

12

Framework Extensibility

The framework extends beyond XR to evaluate emerging disruptive technologies for training and development (Brown et al., 2023)

Generative AI

Personalized content generation and adaptive learning pathways

AI Assistants

Real-time coaching and performance support systems

Mixed Reality

Next-generation spatial computing and hybrid environments

Future Technologies

Flexible methodology supports evaluation of emerging innovations

13

Conclusions & Implications

Key Contributions

Systematic metrics framework for measuring training technology ROI
Practical application demonstrated through illuminative example
Data-driven methodology for informed technology investment decisions

Practical Impact

Organizations can make evidence-based decisions using quantifiable metrics

Future Research

Framework adaptable to generative AI and emerging training technologies

Supporting data-driven technology investment decisions in corporate training

14

References

Brown, A., Garcia, M., & Thompson, R. (2023). Artificial intelligence in corporate training: A systematic review. Journal of Workplace Learning, 35(4), 245-262.

Evans, J. R., & Lindner, C. H. (2012). Business analytics: The next frontier for decision sciences. Decision Line, 43(2), 4-6.

Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick's four levels of training evaluation. ATD Press.

Smith, T., & Martinez, L. (2021). Extended reality technologies in workforce development: A comparative study. Technology, Knowledge and Learning, 26(3), 589-608.

Warren, S. J., Churchill, C., & Hayes, A. (2024). A service-based measurement model for determining disruptive workforce training technology value: Return on investment calculations and example. In J. Delello & R. McWhorter (Eds.), Disruptive Technologies in Education and Workforce Development (1st ed., pp. 206–231). IGI Global Business Science Reference.

Note: Performance data and results presented in this study are modeled for framework demonstration purposes and do not represent actual empirical measurements from deployed systems.