A Systematic Review of the Efficacy of Full-Scale Simulation Exercises in Enhancing Hospital Disaster Preparedness

Introduction

The Journal of Medicine, Law & Public Health Vol 4, No 3. 2024

the deteriorating situation (1). In recent years, the world has seen an increase in extreme scenarios, including both man-made and natural disasters, conflicts, and societal upheavals. The Third UN World Conference on Disaster Risk Reduction in 2015 affirmed the widespread impact of such events on a global scale, resulting in more than 700,000 deaths and 1.4 million injuries. Additionally, it was projected that these calamities affected more than 1.5 billion individuals, resulting in economic losses surpassing 1.3 trillion US dollars (2,3). It is therefore critical to prioritise timely, effective, and long-term interventions for vulnerable populations. Recognising the importance of the effective management of catastrophic events will ensure greater emphasis on generating and sharing information and training in this field to assure preparedness and competence in such crucial situations (4,5). According to the World Health Organization's Hospital and Health Facility Emergency Exercises Guidelines, there are five commonly utilised types of emergency exercise: orientation, tabletop, drill, functional, and full-scale simulation exercises (6). Simulation is a representation of real-life events and can take many forms, including computer software, case studies, written clinical scenarios, simulated patients (SPs), role-playing, or replicating reality with simple or advanced functionalities (7,8). This wide range of modalities allows for diverse and thorough simulation of real-world scenarios (7,8). Real simulation with actors has been used for almost 50 years, demonstrating its continued utility and efficacy in a variety of sectors (7,8). Full-scale exercises are characterised by their leadership (typically conducted by controllers) and the involvement of participants across all levels of personnel (6). These exercises should simulate a realistic setting, including the activation of emergency operations centres, and their duration can range from two hours to more than one day (6). The preparation for such exercises is extensive, often requiring 1–1.5 years of development (6). This preparation includes significant time, effort, and resources, as well as the incorporation of preparatory drills, tabletop exercises, and functional exercises (6).

Full-scale simulation exercises in hospital disaster preparedness are immersive, all-encompassing drills designed to imitate real-world emergency circumstances, and involving the active participation of various hospital departments, including healthcare professionals, administrative workers, and support personnel (8–10). The scenarios are meticulously designed to simulate natural disasters, mass casualties, or infectious disease epidemics, putting the hospital’s reaction and coordination capabilities to the test (11). The multidisciplinary approach ensures that teams collaborate effectively, and scenarios frequently vary to measure adaptation. Communication systems are exhaustively tested, and the entire exercise is analysed and followed by a thorough debriefing (10,11). Understanding the influence of full-scale simulations on hospital disaster preparedness is critical for a number of reasons. For starters, it provides a detailed examination of the strengths and shortcomings of current practices, allowing hospitals to discover gaps and opportunities for development in their preparedness plans. The findings can be used to design more resilient and adaptive protocols to handle real-world difficulties, ultimately improving the resilience of healthcare systems in the face of calamities. This systematic review aimed to illustrate these points, and to provide further recommendations for future researchers.

The findings of this study were reported in accordance with the PRISMA statement requirements (12), and all procedures were carried out in exact conformity with the Cochrane Handbook of Systematic Reviews and Meta-Analysis of Interventions (version 5.1.0) (13). Due to the heterogeneity of the studies, the narrative synthesis approach was used to report the findings.

Eligibility criteria

Studies were included in our review if they satisfied the criteria described in Table 1. We have excluded animal and in-vitro studies, studies with no full text available, studies presented solely as abstracts or

Table 1. Description of inclusion criteria Arms Description Population Hospitals and their staff who have participated in disaster and emergency preparedness training.

p420

Methods

The Journal of Medicine, Law & Public Health Vol 4, No 3. 2024

Intervention Full-scale simulation exercises are the intervention of major interest.

Outcome Recognising the challenges associated with noncompliance, evaluating the time required to assemble a crisis management team, and investigating the effect of full- scale simulations on the quality of the disaster plan.

Included in the study are randomised controlled trials, quasi-experimental trials, pilot studies, cluster trials, and other observational study designs. Cross-sectional, retrospective, and prospective cohort studies are examples.

Study Design

posters, studies from overlapped databases, and various non-primary research sources such as reviews, book chapters, theses, comments, letters, and editorials.

electronic databases (PubMed, Scopus, Web of Science, and Cochrane Central Register of Controlled Trials), from inception until October 9, 2023, using the queries described in Table 2. Additionally, the references of the included studies were manually searched for potentially eligible studies.

Information sources and search strategy

We performed a comprehensive search of four

Table 2. Description of different database search strategies Database Results Search Strategy PubMed 1,288 All Fields: ((Full-scale simulations OR Full-scale regional simulations OR Full- scale regional simulation OR FSRE OR Simulations scenarios OR Full-scale exercise OR FSEs OR Simulations Training OR Simulations system OR Simulations OR High-fidelity simulation OR Online simulations OR Simulating OR Simulator OR Simulation modelling OR Simulation modeling) AND (Preparedness OR Readiness) AND (Hospital)). Scopus 538 Article title, Abstract, Keywords: ((“Full-scale simulations“ OR “Full-scale regional simulations“ OR “Full-scale regional simulation“ OR FSRE OR “Simulations Scenarios“ OR “Full-scale exercise“ OR FSEs OR “Simulations Training“ OR “Simulations system“ OR “Simulations“ OR “High-fidelity simulation“ OR “Online simulations“ OR “Simulating“ OR “Simulator“ OR “Simulation modelling“ OR “Simulation modeling“) AND (“Preparedness“ OR “Readiness“) AND (“Hospital“)). Web of Science

434 All Fields: ((“Full-scale simulations“ OR “Full-scale regional simulations“ OR “Full-scale regional simulation“ OR FSRE OR “Simulations Scenarios“ OR “Full- scale exercise“ OR FSEs OR “Simulations Training“ OR “Simulations system“ OR “Simulations“ OR “High-fidelity simulation“ OR “Online simulations“ OR “Simulating“ OR “Simulator“ OR “Simulation modelling“ OR “Simulation modeling“) AND (“Preparedness“ OR “Readiness“) AND (“Hospital“)). Cochrane 138 Keywords, Title: ((“Full-scale simulations“ OR “Full-scale regional simulations“ OR “Full-scale regional simulation“ OR FSRE OR “Simulations Scenarios“ OR “Full-scale exercise“ OR FSEs OR “Simulations Training“ OR “Simulations system“ OR “Simulations“ OR “High-fidelity simulation“ OR “Online simulations“ OR “Simulating“ OR “Simulator“ OR “Simulation modelling“ OR “Simulation modeling“) AND (“Preparedness“ OR “Readiness“) AND (“Hospital“)).

Selection process

the full-text articles of the identified abstracts were screened for final eligibility. The Rayyan platform was used to aid the selection process (14), and duplicates were eliminated using Endnote, a tool provided by Clarivate Analytics (Pennsylvania, USA).

The retrieved references were screened in two steps: first, the titles/abstracts of all identified articles were screened independently by all the authors to determine their relevance to this review, and second,

p421

Data collection process and data items

the positive influence of simulations on participant education and readiness. Notably, full-scale simulation (FSS) emerged as a consistently successful approach to improve participants' skills and uncover potential safety issues in hospital settings. The details of the 28 studies included in this review are summarised in Table 3. Details of the parameters of the included studies are laid out in Table 4. This includes each study's country of origin, design, setting, recruitment methods, participant count, and evaluation measures. The geographical diversity of the studies, from battle zones in Afghanistan to trauma clinics in the United States, reveals a truly global perspective on catastrophe preparedness. Such variety improves the dataset by documenting the various techniques required in diverse sociopolitical and healthcare environments. Prospective cross-sectional studies, pre-post intervention surveys, qualitative inquiries, mixed-method designs, and observational studies are among the study types available. The settings range from simulated hospital settings to real-world trauma centres, each offering a unique viewpoint on crisis management. Evaluation metrics The studies employ a broad spectrum of evaluation metrics, including non-technical skills, communication, teamwork, decision-making, and operational readiness. This multidimensional approach ensures a thorough evaluation of the skills and competencies required for effective disaster response. Regarding the evaluation method, most studies employed a combination of quantitative and qualitative assessments, such as surveys, standardised tools, and checklists, as described in Table 5. This all-encompassing strategy guarantees that participants' preparedness, teamwork, and crisis management abilities are well understood. The duration of the exercises varies greatly, ranging from 30 minutes to a multi-day simulation. Such variation allows for flexibility based on the individual objectives of each exercise. The studied simulations also span a wide range of disaster scenarios, including mass casualties, viral epidemics, large aviation accidents, and terrorist strikes, as illustrated in Table 5. This breadth guarantees a thorough approach to disaster preparedness, covering a number of potential obstacles that healthcare personnel may encounter.

Data were extracted from a uniform data extraction sheet, and included: (1) Characteristics of the included studies, (2) Characteristics of the populations of the included studies, and (3) Outcome measures.

Risk of bias assessment

According to the Cochrane Handbook of Systematic Reviews of Interventions 5.1.0 (updated March 2011), we used the National Institutes of Health (NIH) quality evaluation tools for cross-sectional, cohort, case-control, and intervention studies to assess the quality of quantitative studies (15). We used Noyes et al.'s recommendations for qualitative investigations, and the Mixed Methods Appraisal Tool (MMAT) for mixed-method studies (16). Two reviewers worked separately on the evaluation, and any inconsistencies were handled through discussion with a senior reviewer (17). Each tool had its own set of criteria, on the basis of which the studies were graded as good, fair, or poor.

Data synthesis

Due to the great variation in the objectives, design and settings of the studies, we used a narrative synthesis method to describe the key characteristics and findings in tables. We did not plan to conduct meta-analysis or measure confidence in cumulative evidence, due to the substantial heterogeneity of the included studies (cross-sectional, cohort, mixed techniques, and qualitative research).

Results

Literature search results

The literature search produced 2,398 results, 605 of which were duplicates. Following title and abstract screening, 59 papers were subjected to full-text screening, with 28 publications finally included in the systematic review. A further search within the references of the included studies yielded no other eligible papers. Figure 1 of the PRISMA flow diagram depicts the study selection process.

Characteristics of included studies

The varied range of studies included in this review provided useful insights into the successes and limitations associated with different techniques. The overarching observation across all of the studies was

A 36-Hour Unplugged Full-Scale Exercise: Closing the Gaps in Interagency Collaboration between the Disaster Medical Assistance Team and Urban Search and Rescue Team in Disaster Preparedness in Taiwan

Evaluate preparedness and performance of DMATs in a simulated crisis scenario

Foo 2021

(29)

Compare effectiveness of simulated versus actual actor-patients in MCI drills

Simulation in a Disaster Drill: Comparison of High-Fidelity Simulators Versus Trained Actors

Gillett 2008

(30)

Evaluation of Medical Command and Control Using Performance Indicators in a Full-Scale, Major Aircraft Accident Exercise

Assess medical command and control in major aircraft accident using FSE at local and strategic levels

Gryth 2010

(31)

Test feasibility of incorporating patient's perspective into traditional evaluations of

Developing the Fourth Evaluation Dimension: A Protocol for Evaluation of Video from the Patient’s

Haverkort

2017 (32)

Disaster scenario involving medical rescue teams showed effective command, control, and communication, and highlighted instances of noncompliance in health examinations.

DMATs were successful but communication and evaluation skills need improvement.

Lack of comprehensive analysis, potential subjectivity

Non-validated survey, Likert scale issues, lack of inter-rater reliability testing, non-identical paired cases, risk of learning bias

High-fidelity simulators and live actors produced equivalent results in mass casualty drills, emphasising the effectiveness of both.

Simulators are as effective as live actors in MCI drills.

Performance indicators assessed medical command and control. Management and staff skills scored 15 and 17 at the strategic level, and 17 and 21 at the hospital level, respectively, revealing feasibility for future evaluations and real incidents.

Potential differences in exercise vs. real incidents, lack of absolute distinction between scores

Performance indicators are effective for evaluation of command and control in FSE.

Equipping mock patients with point-of-view cameras in trauma drills enhanced communication training and identified areas for

Preliminary nature of study, images used by multiple observers

Point-of-view camera enhances trauma drill evaluation. Protocols should be optimised

Perspective During Major Incident Exercises

hospital using simulated training

Healthcare professionals’ experience of using in situ simulation training in preparation for the COVID-19 pandemic: a qualitative focus group study from a Danish hospital

Investigate impact of ISS on COVID-19 preparedness

Juelsgaard

2022 (33)

Evaluation of Hospital Mass Screening and Infection Control Practices in a Pandemic Influenza Full-Scale Exercise

Evaluate mass screening protocol, isolation, and triage in influenza FSE

Kohlhoff 2012 (34)

Use of an Emergency Preparedness Disaster Simulation with Undergraduate Nursing Students

Evaluate impact of EPDS on nursing students

Kaplan 2012 (35)

Full-scale regional exercises: Closing the gaps in disaster preparedness

Assess FSRE in mass casualty disaster preparedness

Klima 2012

(36)

improvement. The approach improved non-medical aspects and identified areas for training and process enhancement.

for effective utilisation of such footage.

Positive outcomes from simulations for healthcare professionals, emphasising improved teamwork and skills in PPE use.

ISS is valuable for pandemic preparation, at both organisational and individual levels.

Focus on specific hospital, potential response bias

Simulated pandemic mass screening protocol showed correct influenza status determination in 74% of cases, and effectiveness in addressing infection control.

Mass screening and other infection control measures proved feasible and resulted in measurable outcomes during FSE.

NR

EPDS received positive evaluations from 95% of students, indicating increased knowledge, confidence, and preparedness for disaster situations. The hybrid approach proved cost- effective and time efficient.

EPDS enhances disaster preparedness, skills, and confidence, providing a cost- effective educational experience.

NR

Assessment of 16 hospitals showed widespread non- compliance, with communication problems

Lack of training with technologies, communication issues

FSE is crucial for MCI readiness, as a communication gap

Operation continued care: A large mass-casualty, full-scale exercise as a test of regional preparedness

McElroy 2019 (37)

Assess coordination in mass-casualty FXS

Utilization of Functional Exercises to Build Regional Emergency Preparedness among Rural Health Organizations in the US

Design and implement disaster exercises in rural areas to test efficacy

Obaid 2017

(38)

Combining performance and outcome indicators can be used in a standardized way: a pilot study of two multidisciplinary, full- scale major aircraft exercises

Demonstrate feasibility of evaluating multidisciplinary, full-scale disaster exercises using combined indicators

Rådestad 2012 (39)

Use of simulation models when developing and testing hospital evacuation plans: a tool for improving emergency preparedness

Evaluate hospital evacuation preparedness for MCI using simulation real- time exercise

Rådestad 2023 (40)

(94%) and staffing issues (63%) being common.

remains in tabletop exercises.

Triage analysis revealed strengths but highlighted communication failures and areas for improvement in data entry and resource requests.

FXS aids preparedness, highlights communication and coordination flaws.

NR

Functional exercises vital for testing decision-making and response, offering insights beyond tabletop or short, full- scale exercises.

Functional exercises evaluated command staff, fostering regional collaboration. Revealed barriers to unified command and documentation issues.

Limited resources

Improved performance indicators, including preventable deaths and complications, were observed. Additionally, there were an adequate number of ambulances and injured victims considered.

Standardised evaluation using combined indicators allows for meaningful comparisons between exercises.

Competency variations, need for real incident validation

Tabletop exercises improved evacuation plans and critical operations. In two exercises, 71.2% and 52.9% of patients were evacuated within 7 and 4 hours. Positive feedback on survey results and a desire for more exercises.

Limited generalisability due to varied exercises; low response rates; exercise did not include treatment during transport

Simulation exercises help planners create more efficient evacuation plans, protocols, and procedures for MCI.

Evaluate hospital incident command, triage, and patient transfer in FSS for MCI

Full-scale simulation exercise—A preparedness for trauma mass casualty incident: Nepal

Shrestha 2022 (10)

Full-scale simulations to improve disaster preparedness in hospital pharmacies

Evaluate impact of FSS on pharmacy emergency readiness

Schumacher

2022 (41)

Waterworks, a Full-Scale Chemical Exposure Exercise: Interrogating Pediatric Critical Care Surge Capacity in an Inner-City Tertiary Care Medical Center

Assess PICU surge plan in chemical exposure exercise

Shah 2013

(42)

or upon arrival at facility

Simulation exercise revealed gaps in knowledge and skills, indicating room for improvement in incident command, [mean score of 161 out of 220 (73.2%)], triage, and patient transferal.

Simulation identifies gaps and highlights areas for improvement.

Single site, observational design

Simulation exercises improved task completion. Four hospital pharmacies initially completed 69%±6% of assigned tasks, increasing to 84%±7% in the second round. Average quality of actions improved from 3.0/5 to 3.6/5, and crisis team assembly time reduced from 23 to 5 minutes with challenges in communication and crisis management.

Small number of participating pharmacies, subjective assessments

FSS significantly improves readiness and staff knowledge.

Effective patient care transitions in chlorine overexposure simulation. Identified shortcomings in ED care.

Lack of post- instruction testing, potential ED overstaffing bias

PICU surge plan effective, but ED areas need attention.

Randomized controlled trial of high-fidelity patient simulators compared to actor patients in a pandemic influenza drill scenario

Compare resuscitation times in pandemic influenza high-fidelity simulation

Wallace 2010 (43)

Lessons Learned from an Active Shooter Full-Scale Functional Exercise In a Newly Constructed Emergency Department

Assess hospital response to active- shooter scenario

Wexler 2017 (44)

ISS: Institutional Simulation Scenario, HOSPEX: Hospital Exercise, PEMAF: Emergency Plan for Massive Influx of Injured, MCI: Mass Casualty Incident(s), EPDS: Emergency Preparedness Disaster Simulation, FSE: Full-Scale Exercise, FSRE: Full-Scale Regional Exercise, ED: Emergency Department, DMAT: Disaster Medical Assistance Team, FSS: Full-Scale Simulation, PICU: Paediatric Intensive Care Unit, MGE: Mass Gathering Event, MACSIM®: Mass Casualty Simulation Tool

High-fidelity simulator patients posed challenges in real-time disaster response training, with significantly longer procedures.

Small sample size, lack of real patient comparison

Potential challenges using simulators for real-time training.

Noted strengths and weaknesses, including a lack of experience in mass casualty triage and communication breakdowns. Identified educational opportunities.

Conducting an active shooter exercise provided valuable insights for improvement without impacting patient care.

Artificial nature of drills, potential staff response discrepancies

Kern Medical Center Emergency Medicine Department Bakersfield, CA

Facho 2021 United States Qualitative study

Adult ED of the Persian Gulf General Hospital in Bushehr, Iran

Farahi 2021 Iran Qualitative study

Nan Hospital of China Medical University

Foo 2021 China Observational study

Kings County Hospital Center in Brooklyn, New York, and Tampa

Gillett 2008 United

Prospective cohort study

States

iterative development of cognitive aids

Instrumental skills for physicians, other tasks for nurses and non- physicians

Three patients in round 1; another three in round 2

NR NR

ED scenarios, regular operations, crisis response, workload integration, scenarios shaped by historical data, patient interarrival times adjustment for diverse intensity levels

2021 NR NR

Technical proficiency of Disaster Medical Assistance Teams (DMATs), barriers to cooperation between DMATs and Urban Search and Rescue (USAR) teams during a simulated earthquake exercise

30physicians, 65 nurses, and 74 logistical personnel

2018 November NR

Different scenarios, tasks such as triage, obtaining vital signs, applying oxygen, initiating stabilising

December 2006 and March 2007

NR 78 hospital staff members

General Hospital in Tampa, Florida

Gryth 2010 Sweden Qualitative study NR 2008 October NR NR NR

Major Incident Hospital (MIH) in Utrecht, Netherlands

Haverkort 2017 Netherlands Proof-of-concept

study

Juelsgaard 2022 Denmark Qualitative focus group study

Danish University Hospital

American Association of Colleges of Nursing and associated Hospitals

Kaplan 2012 United States Qualitative study

Klima 2012 United States Qualitative study Multi-agency over 16 areas NR

Kohlhoff 2012 United States

Qualitative focus group study

Brooklyn, New York hospitals NR

interventions, completing surveys, providing appropriate disposition

Review of captured footage to gain insights into patient experience

2014 to 2015 Two mock cases NR

ABCDE approach, PPE and medical equipment, transportation, communication, decision making in critical cases

2020 April- may NR 22 healthcare professionals

Communication, supplies usage, paediatric safety, triage, treatment

2008 to 2011 NR 90 undergraduate nursing students

Communication, command structure, decontamination, staffing, patient tracking

280 volunteer patients

NR

Standardised mass screening and infection control items with coordination, communication

354 volunteer victims

NR

7 trauma centres, 30 acute care hospitals and free- standing emergency departments, and 42 emergency medical services agencies

McElroy 2019 United States Qualitative study

Three rural Nebraska (USA) regions by the Center for Preparedness Education (CPE) at the University of Nebraska Medical Center (Omaha, Nebraska USA)

Obaid 2017 United States

Quasi-experimental design

Two multidisciplinary FSE (major aircraft accident) in two regions in Sweden

Pilot study (quantitative evaluation study)

Rådestad 2012 Sweden

Patients’ entry, triage accuracy, communication, coordination, performance, points of improvement, resources request, patient tracking, OHTrac utilisation, communication assessment

2017 April 445 volunteer patients

NR

Command staff, identification of areas for improvement, advancing regional collaboration among diverse response partners and other parameters of Homeland Security Exercise Evaluation Program

2010 to 2013 NR 83 command centres, including 667 participants

prehospital, regional, and hospital command and control, as well as regional and hospital staff procedure skills. Key points include response time, communication effectiveness, resource allocation, media interaction, and staff

2008 to 2010 99 victims 131 and 69 health care workers, respectively

Two emergency hospitals within region of Stockholm

Rådestad 2023 Sweden Qualitative study

Group of Swiss French-speaking hospital pharmacists (Groupe ment des Pharmacies Hospitallers Romans)

Schumacher 2022 Switzerland Prospective

multicentre design

Kings County Hospital Center’s (KCHC) Paediatric Intensive Care Unit (PICU)

Prospective single- centre simulated disaster drill

Shah 2013 Switzerland

Shrestha 2022 Nepal Observational study Six hospitals in

Nepal 2021 NR NR

coordination during disaster exercises.

Transfer, patients evacuated, time to evacuate, decision- making, leadership, resource management, simulation system effectiveness, proposed evacuation plans, communication procedures, participants' perceptions

145 participants in Exercise 1; 95 participants in Exercise 2

2019 NR

Simulation responses, communication, disaster management, logistical activities, pharmaceutical assistance, quality of actions using Likert scale

2018 August to 2020 August

NR NR

22 patients with 14 family members needed evaluation

36 (medical students or emergency medicine residents), 9 of these patients

Transportation, delivery of appropriate therapy, ED care, discharge process, educational materials

2012 July

Hospital incident command area, patient care, triage, treatment areas, emergency signal, internal and external

Emergency Department at Kings County Hospital Center, Brooklyn

Wallace 2010 United States RCT

ED at WellSpan York Hospital, a Level 1 trauma centre in south- central Pennsylvania

Wexler 2017 United States Qualitative study

coordination mechanism, organisation issues

Identification of respiratory and circulatory compromise, provision of definitive airway, initiation of vasopressors through a central line, disposition

2008 July NR 12 cases

Communication, recognition instructions response, evacuation preparedness

NR NR NR

Full-scale exercise 1 day Participants (university professionals), controller, observer

Kohlhoff 2012 Influenza pandemic

Emergency preparedness disaster simulation (EPDS)

Mass causality (tornado disaster)

Kaplan 2012

30-45 minutes

Full-scale regional exercise (FSRE)

Klima 2012 Mass causality

NR

Operation Continued Care full-scale exercise

McElroy 2019 Mass causality

NR

Six functional exercises 3 hours Command centres, participants, controllers, observers

Obaid 2017 General disaster preparation

Rådestad 2012 Major aircraft accident

Full-scale exercise 292 minutes Participants, controller, observers

Simulating pandemic influenza outbreak with adapted case definition

Evaluation using standardised forms

Pre- and post- simulation surveys with self-reported, anonymous responses

Team leader, triage staff nurse, bedside assistants, controller

Tornado hitting an assisted living facility with injuries and fire

Simulated terrorist assault with train derailment, involving 281 volunteer patients

Participants, controllers, exercise design teams, evaluator

Third-party evaluators, after- action evaluation

Post-exercise analysis in after- action report/improvement plan

Participants, communication team, resource request team, triage team, OHTrac input team

Staged terrorist assaults at three locations, involving 445 mock victims

Homeland Security Exercise Evaluation Program (HSEEP; Washington DC, USA)

Six functional exercises to improve regional planning and collaboration

External evaluators were registered using standardised performance indicators and patient outcome indicators.

Aircraft crash scenarios with 99 and 100 victims in 2008 and 2010

Mass Casualty Simulation system (MACSIM)

11 hours Command group, physicians, nurses, support staff

Rådestad 2023 Mass casualty incidents (MCI)

Trauma mass- casualty Incidents

Full-scale simulation exercise

Shrestha 2022

30 minutes Participants, controller, evaluator

Mass causality (road traffic accident and terrorist attack)

Full-scale simulation exercise

Schumacher 2022

3-4 hours Participants, assessors

90 minutes (patient briefing from 24 hours prior)

Mass causality (influx of critically ill children)

Full-scale chemical exposure exercise

Shah 2013

High-fidelity simulator patients and human actor patients in a surge setting

Wallace 2010 Influenza pandemic

NR

Full-scale active shooter exercise

Wexler 2017 Mass casualty (active shooting)

NR Participants, controllers

NR: Not reported.

Post-exercise surveys and open-ended questions for feedback

Two exercises testing evacuation plans and application

Assessment of disaster management capabilities in six hospitals

Standardised evaluation tool

Harvard School of Public Health's Emergency Exercise Evaluation Toolkit

Simulated road traffic accident and terrorist attack scenarios

Homeland Security Exercise and Evaluation Program (HSEEP) terms for a full-scale exercise

Drill involving victims of chemical exposure, evaluating surge response and medical management

Patients, participants, evaluators, controller

Comparison of resuscitation times for simulator and human actor patients in ED

Participants (physician, residents, nurse, technician), 12 patients

Standardised data collection tool, checklist

Closed-circuit video and portable video cameras were utilised to provide feedback for debriefing

Full-scale exercise in ED with an active shooter scenario

use of training in real-world circumstances. Furthermore, their net benefits extended beyond individual skills development. Additionally, several studies, such as those by Kohlhoff et al. (34), and Obaid et al. (38), emphasised a significant impact on healthcare systems, with a crucial role in evaluating decision-making, encouraging regional collaboration, and providing insights. Simulation not only validates readiness, but also acts as a diagnostic tool for identifying the ideal disaster response. According to Shrestha 2022 (10) and Cristal 2019 (24), high-quality simulations provide opportunities for improvement by identifying issues such as communication breakdowns, equipment deficiencies, and weaknesses in specific emergency plans. As evidenced by Foo (29), the integration of disaster medical assistance teams (DMATs) ensures effective command, control, and communication, while allowing for improvement in health assessments. The wide variety of simulated scenarios, ranging from trauma centre stress management to mass casualty catastrophes involving earthquakes, pandemics, and active shooter scenarios, exemplifies these exercises’ adaptability in dealing with diverse crisis situations and ensures a comprehensive approach to disaster preparedness and prediction, which improves response in real-life situations. Furthermore, the range of exercise durations, from 30-minute simulations to multi-day events, provides flexibility in fulfilling diverse training requirements. The variety of evaluation measures used in different studies of high-stress simulations, including checklists, quantitative and qualitative surveys, and specialised evaluation tools, reflects the multifaceted nature of disaster preparedness. In addition to the conventional focus on technical abilities, nontechnical factors such as communication skills, teamwork, decision-making, healthcare infrastructure, and operational readiness are also examined. Radestad (40) demonstrates the utility of a simulation model in refining hospital evacuation preparations for mass causality crises. Their system makes it easier to build effective and practical evacuation plans, protocols, and procedures for actual crisis scenarios. This is the first systematic review of its kind, and highlights the effectiveness of full-scale simulation (FSS) in enhancing participants' skills and uncovering hidden safety issues in hospital settings during disaster preparedness exercises. It systematically identifies crucial areas for improvement that are addressed by simulation

Effectiveness of simulations

The included studies systematically revealed crucial areas for disaster response improvement, shedding light on communication breakdowns, equipment deficiencies, and flaws in specific emergency plans. For example, Djalali (2014) highlights the successful implementation of Simple Triage and Rapid Transport (START) in catastrophe scenarios, demonstrating the practical benefits of such simulations for real-world settings. Simulation appears useful not only as an educational aid, but also as a diagnostic tool that identifies weaknesses which might otherwise go undetected.

Identification of gaps, challenges, and limitations

However, it is essential to approach the findings with a nuanced understanding, considering the limitations across various studies. When evaluating the results, potential biases, short sample sizes, and actor-related issues should all be considered. Recognising the diversity in simulation approaches—from high fidelity simulators to functional exercises and in-situ simulations—enhances our understanding of the broad landscape of disaster preparedness research.

Recommendations and implications

These studies go beyond theoretical concerns to provide practical ideas for improving hospital readiness. Suggestions include mandating hospital- wide involvement, appointing dedicated Emergency Management Officers, and utilising simulation- optimisation methodologies. As we work our way through these collective findings, it becomes clear that disaster preparation simulations not only increase education and readiness, but also serve as a platform for continual improvement within the healthcare system.

Discussion

Disaster preparation simulation exercises are critical to assessing and improving the performance of healthcare systems. This systematic review investigates specifically the role of full-scale simulation exercises (FSSE) and identifies gaps to enhance the response of healthcare systems to disasters. Our study emphasises the effectiveness of FSSE as a reliable real-world approach, and highlighted their role in the identification and solution of specific safety issues that are often hidden and unpredictable. The success stories of these simulation methodologies include those of Carmichael et al. (21) and Farahi et al. (28), who demonstrated their

exercises, including communication breakdowns, equipment deficiencies, and flaws in specific emergency plans. Additionally, it evaluates the range of simulation approaches used in disaster preparedness research. Mahdi et al.summarised the extensive process of developing various exercise types, evaluation mechanisms, simulation exercise design models, and guidelines for simulation exercise development (45). Their work underscores the importance of a logical and scientific approach to evaluating disasters in simulation exercises, and touches on the positive impact of simulations and the importance of their evaluation. Meanwhile, the geographical variance of the included studies highlights the dynamic nature of disaster preparedness (45). The inclusion of a wide variety of study designs, such as prospective cross sectional studies, pre-post intervention surveys, qualitative investigations, and mixed methods approaches, reflects the diversisty of organisational complexities, views and attitudes, and provides a comprehensive picture of emergency and disaster readiness. An important point emphasised by this study is the significance of collaboration at all levels and teamwork during disaster response, involving patients, volunteers, healthcare professionals, and administrative personnel. Furthermore, a thorough understanding of the various stakeholders’ roles, expectations, and soft

skills during rapid action response is critical to ensure comprehensive and effective preparedness measures. As with many studies, ours has limitations, foremost of which is the heterogeneity across the studies due to different participant demographic, simulation methodologies, and evaluation criteria.

This systematic review confirms the benefits of full- scale simulation exercises in disaster preparation and improving healthcare system readiness. A wide range of methodological approaches were reviewed, involving different crisis scenarios and training projects. Many areas for improvement were identified, and our evidence supports the notion of enhancement and sustainability of disaster preparedness through simulation and evidence-based practice. Protocol registration: The protocol of this study was registered on the PROSPERO website with reference number CRD42023481418. Funding: No funding was received for this research. Conflict of interest: The authors declare no conflict of interest. Ethical approval: This article does not include any studies by any of the authors involving human participants or animals.

study. Prehosp Disaster Med. 2014 Aug 4;29(5):441–7. 6. World Health Organization Regional Office for the Western Pacific. Hospital and Health Facility Emergency Exercises: Guidance Materials. 2010.