Medication Safety Amid Technological Change: Usability Evaluation to Inform Inpatient Nurses' Electronic Health Record System Transition

Background: Electronic health record (EHR) system transitions are challenging for healthcare organizations. High-volume, safety-critical tasks like barcode medication administration (BCMA) should be evaluated, yet standards for ensuring safety during transition have not been established.

Objective: Identify risks in common and problem-prone medication tasks to inform safe transition between BCMA systems and establish benchmarks for future system changes.

Design: Staff nurses completed simulation-based usability testing in the legacy system (R1) and new system pre- (R2) and post-go-live (R3). Tasks included (1) Hold/Administer, (2) IV Fluids, (3) PRN Pain, (4) Insulin, (5) Downtime/PRN, and (6) Messaging. Audiovisual recordings of task performance were systematically analyzed for time, navigation, and errors. The System Usability Scale measured perceived usability and satisfaction. Post-simulation interviews captured nurses' qualitative comments and perceptions of the systems.

Participants: Fifteen staff nurses completed 2-3-h simulation sessions. Eleven completed both R1 and R2, and seven completed all three rounds. Clinical experience ranged from novice (< 1 year) to experienced (> 10 years). Practice settings included adult and pediatric patient populations in ICU, stepdown, and acute care departments.

Main measures: Task completion rates/times, safety and non-safety-related use errors (interaction difficulties), and user satisfaction.

Key results: Overall success rates remained relatively stable in all tasks except two: IV Fluids task success increased substantially (R1: 17%, R2: 54%, R3: 100%) and Downtime/PRN task success decreased (R1: 92%, R2: 64%, R3: 22%). Among the seven nurses who completed all rounds, overall safety-related errors decreased 53% from R1 to R3 and 50% from R2 to R3, and average task times for successfully completed tasks decreased 22% from R1 to R3 and 38% from R2 to R3.

Conclusions: Usability testing is a reasonable approach to compare different BCMA tasks to anticipate transition problems and establish benchmarks with which to monitor and evaluate system changes going forward.