Blogs

Scenario: As your department training officer, you have noticed that performance at emergency scenes has not been up to standard. As a result, you put together a quick training program to address the performance gaps, but the solution hasn’t been as successful as you had hoped and now the Chief is questioning your capabilities and performance.

Introduction

Training firefighters, Emergency Medical Services, and other emergency services personnel is an expensive endeavor. Buric (2016) estimates that an organization, whether volunteer or career, will spend over $250,000 training a firefighter over the course of their career. Those who are outside the training and education discipline remain blissfully unaware of the process of instructional design and delivery. Instructional Systems Development (ISD) is a system designed to improve training and educational development and improve operational performance. Also, known as Training Systems Development, Systems Approach to Training, or Criterion Referenced Training, ISD is a “process based and following a series of steps or guidelines…shaped by the overriding idea that instruction in all formats must be consistent, reliable, and effective to facilitate learning” (Nichols-Hess and Greer, 2016). The ISD process is focused on leaner needs in it approach to instruction as it strives to support successful learning outcomes and is applicable to any discipline.  While there are several offshoots of ISD models, the ADDIE instructional design framework that is most frequently used. “The use of ADDIE can provide structure for developing instructional materials and student engagement, ensure that learning assessment is more accurate, and effectively guides the learning process” (Nichols-Hess and Greer, 2016). As Chuck Hodel points out, “ADDIE has become the gold standard in ISD models because at its very core, it reflects the five most common elements of the ISD process-analysis, design, development, implementation, and evaluation” (2016). 

The ADDIE model consists of five, interrelated and interconnected steps. These are:

Analysis, which includes gathering data about content, populations, delivery systems, methods, budgets, actual and desired performance, the impact and need of the training.

Design, during which the instructional designer will build a blueprint of the training course and develop project specifications, objectives, course content, and develop a design plan.

Development is the phase where educational materials are produced, pilot testing occurs, and draft design products transition to a deliverable educational product.

Implementation involves the delivery and evaluation of the training design product.

Evaluation occurs throughout the process, where each step is evaluated for success.

Figure 1: The ADDIE Model

The five phases of ADDIE may overlap or occur simultaneously and provide a dynamic and flexible template for developing effective and efficient instruction. By using this systems approach, training can be developed using a logical process that factors in constraints such as time and money; it ensures linkages between course materials, training activities, and performance benchmarks; it focuses training efforts and the actual knowledge and skills needed to successfully perform on the job; and products quality results that are supported by evaluations, revision schedules, and job performance improvement. 

Analysis Phase

The first step of the ADDDIE process is the analysis phase. Entry into this phase is triggered by recognizing an instructional issue that examines the target audience and their characteristics. In essence, asking and answering the question, “who can’t do what, and why?” According to Chuck Hodell (2016), “analysis is the singl3 most important building block in the practice of instructional design. Without it, y9ou have little in the way of information and data upon which to build a unit of instruction, or even to decide if training or designing a course is the best way to address a perceived training need.” 

The analysis phase consists of a series of steps designed to identify that there is a performance gap and that the gap is caused by training, describe the target audience and what the current level of performance is, and assess the resources, constraints, and goals of the organization with relation to the performance gap. The analysis phase should also describe what mastery of the performance should look like and how it will be evaluated. 

Table 1: Types of Analysis, Adapted from Old Dominion University (2001) and Hodell (2016)

Conducting a thorough analysis of the issue will allow decision makers in making informed decision with regards to supporting the training decision and the organization will ultimately save money by focusing its resources on a specific solution to a well-defined problem. By framing the training need into actionable items, the analysis helps guide the training strategies which helps to identify resource requirements, budget needs, and define expected outcomes for the organization. Remember that the first tasks are to identify that there is a need, determine the root cause of the problem, and determine that training is the most effective solution to remedy the issue.

Table 2: Causes of Performance Failure

Design Phase

Once the performance issue has been described and the solution determined to be training, it is time to design the training program. Keeping in mind that the goal of training isn’t just the transfer of knowledge, but its application. As Hodell states, “having information doesn’t accomplish anything by itself. Something is accomplished when the learner uses that information to do things” (2016).  During the design phase, a design plan will be developed that includes:

• Identifying the reason for training
• Coordinating analysis, development, and evaluation processes
• Identifying and describing the learning objectives
• Defining the target audience
• Describing any prerequisite knowledge or skills needed for successful performance by both participants and instructors
• Describe the educational methodology to be used
• Determine the sequence of content delivery and training strategies
• Determine the training methods to be used
• Describing how mastery of content will be evaluated
• Describe the training media, materials, resources, and facilities needed to conduct successful training
• Identifying what will be developed for the program, such as instructor manuals, student guides, etc.
• Managing quality control of all processes and products

Ultimately, the design phase should lead to a design product that will result in knowledge and skill transfer. Two of the critical elements in this process are the development of learning objectives and learning strategies based that support the learning objectives. Learning objectives describe what performance should look like at the end of the training period. Objectives focus on key points, prepare the learner for what is expected, and describes the acceptable level of performance expected. The design document will develop four types of objectives: focusing objectives that draw attention to the most important parts of the learning module, performance objectives based on expected competency demonstration, instructional design objectives to guide the design and development of the lesson, and instructor evaluation objectives to assess the quality of instruction.

Table 3: Types of Objectives

Next, instructional strategies and the sequencing of information will be addressed in the design process. The instructional strategies selected should support the learning objectives (performance and focusing) and evaluation methods. The evaluation methods are directly tied to the learning objectives as well as the content. There are a variety of instructional methods that can be used, including:

• Lecture
• Role-play
• Critique
• Discussion
• Case study
• In-basket/out-basket
• Simulations
• On-the-job training
• Gaming
• Brainstorming
• Critical incident review
• Drill
• Job aids

During the design phase, the methods of instruction will be selected and sequencing strategy to support the learning strategy identified.  Common sequencing strategies include providing step-by-step directions, moving from general knowledge to detailed information, or moving from the known to the unknown in order to bridge knowledge gaps and provide scaffolding for learning.  It is important to utilize the information developed from the population analysis as the level of knowledge or experience with the topic area will guide the level of support that is provided to the learner.

Table 4: How to Support Learning Based on Learner Background & Experience

Finally, the evaluation strategy should be linked to the learning objectives and learning strategies. Activities and knowledge gain should be measurable and reflect actual job-related skills and knowledge. In addition, resources to support realistic evaluation should be available to support the process of ensuring content mastery. 

Development Phase

The development phase is focused on creating, purchasing, or leasing draft materials to support the lesson. When determining whether to produce rather than purchase course materials, the developer should consider:

• Cost
• Deadlines
• Agreements
• Samples of work
• How final materials will be approved and by who
• Impact on pilot testing
• Adequacy of materials to meet objectives based on identified gaps

Facilities will need to be assessed to ensure that they meet the needs identified in the design document. This may include adequacy of training spaces such as classrooms, drafting facilities, and training props. Instructors will also be identified based on the criteria established in the design document. Once they have been identified, they will need to be trained to deliver the course content. All of the course materials will be developed, and facility, equipment, and resource needs will be addressed. Training materials may include:

• Instructors Guide
• Lesson Plans
• Student Manual
• Train the Trainer materials
• Textbook
• Guidebooks
• Handouts
• Non-print media such as software, mockups, models, and videos
• Evaluation materials

Other materials produced may include:

• Participant Recording System (attendance, evaluations, training record)
• Instructor qualifications
• Instructor lists
• Course versioning
• Material authorship/development team

Course developers may want to include a train-the-trainer element as part of the development plan. A train-the-trainer should be considered when

• The course includes new or unfamiliar material
• New or unfamiliar delivery systems
• Instructor skill levels are in question
• New or unfamiliar technology will be used
• The course content requires a train-the-trainer component
• License or certification requirements are tied to the training

An evaluation plan will be fully developed, based on the design document and when and how to evaluate learning, and may include scripting what is told to students about the evaluation process. Evaluation documents for learners, instructors, and support personnel. 

Table 5: Applying Gagne’s Nine Levels of Learning to Lesson Plan Development

Also, during the development phase, the materials will be delivered on a pilot basis, with modifications based on evaluation of all elements of the process and delivery. Pilot testing a learning module allows developers the opportunity to evaluate it before the project is fully implemented and is critical to successful course development. Pilot testing answers the following questions:

• Does the lesson plan work?
• Are the directions for the facilitator clear and concise?
• Are the learner’s materials appropriate and thorough?
• Are the facilitators materials appropriate and thorough?
• Are the support materials what you expect?
• Does the timing match estimates?
• Are technology elements appropriate?
• Are instructional methods appropriate?
• What does/does not work?
• What should be changed?

Hodell (2016)

Materials and delivery may be modified as needed based on the pilot evaluation process. Finally, there may be several pilot offerings, including post-revision pilots.

Implementation Phase

At the implementation phase, the course materials have been developed and tested and are ready for full implementation. At this time, the delivery and evaluation plans will be fully implemented with the training being delivered to the target audience as designed and evaluated against the criteria established in the evaluation plan. Evaluations of the course should include at least level 1 and 2 evaluations with level 3 and 4 evaluations preferred. Any further adjustments to the program will be based on the outcomes of the evaluations. 

Table 6: Kirkpatrick’s Four Levels of Evaluation, North Dakota State University (2021), Hodell (2016)

Evaluation Phase

Evaluation is conducted during each phase of the ADDIE process, beginning during the analysis phase when determining if the performance issue is related to training and comparing performance against established benchmarks, through the design and development phases and finally during implementation of training. By evaluating continually throughout the process, quality training can be achieved and mastery of content, resulting in improved performance and elimination of the performance gap can be achieved. The feedback received during the ADDIE process allows for revision of materials linked with evaluation of content mastery, improving knowledge and skill transfer resulting in improved on-the-job performance, reducing extraneous costs, and improving return in investment for the organization.

Table 7: Evaluation during the ADDIE process

Job Aid 1: Objective Evaluation Job Aid

Conclusion

While instruction design may seem to be a cumbersome process, many departments currently apply training to solve all performance problems, even when training is not appropriate, resulting in lost money, time, and effort. By utilizing an instructional design process, such as ADDIE, training officers, company officers, and others who are responsible for the training of current and new responders can ensure that training is effective and efficient, resulting in reduced costs, improved performance, and better outcomes for both members of our organizations and the communities we serve.

References

Buric, A. (2016). Essentials of Great Instructional Design.  Retrieved from www.learnupon.com/blog/essentials-great-instructional-design.

Dirksen, J. (2016). Design for How People Learn, 2^nd ed. New Riders. San Francisco: CA.

Hodell, C. (2016). ISD From the Ground Up.  A No-nonsense Approach to Instructional Design, 4^th ed. ATD Press. Alexandria: VA.

McGriff, S. (2020). Instructional Systems Design (ISD): Using the ADDIE Model. College of Education: Penn State University.

Nichols-Hess, A. & Greer K. (2016). Design for Engagement: Using the ADDIE Model to Integrate High-Impact Practices Into An Online Information Literacy Course. Communications in Information Literacy, v 10#2, p. 264-282.

North Dakota State University. (2021). Kirkpatrick's Four Levels of Training Evaluation in Detail. Retrieved from https://www.ag.ndsu.edu/evaluation/documents/kirkparicks-four-levels-of-training-evaluation-in-detail

Northern Illinois University Center for Innovative Teaching and Learning. (2020). Gagné’s nine events of instruction. In Instructional guide for university faculty and teaching assistants. Retrieved from https://www.niu.edu/citl/resources/guides/instructional-guide

Old Dominion University. (2001). Instructional Systems Development (ISD). Occupational and Technical Studies. Old Dominion University. Norfolk: VA.

About the Author

Dave Donohue has over 40 years of fire, EMS, and emergency services experience, been involved in emergency services training since 1984, and has served as a responder at the local and federal levels in Florida, Maryland, Virginia, West Virginia, Pennsylvania, and Washington, DC.  He is the owner of Mid Atlantic Emergency and Safety Consulting, LLC.  He resides outside of Hagerstown MD, where he is a member of the Community Volunteer Fire Company of District 12 in Fairplay, serves as a member of the Maryland Wing of the Civil Air Patrol Hagerstown Squadron, is an instructor for the Maryland Fire Rescue Institute, works for a fire & EMS-based education center in Emmitsburg Maryland, and is looking forward to the return of minor league baseball to Hagerstown in 2022.  He can be reached at dkdonohue@aol.com.