Background

Emergency Medical Technicians (EMTs) are critical first responders that save many lives. They are usually the first ones to arrive at any situation and are responsible for assessing the scene and taking care of urgent needs. While these individuals are extremely important for society, a lot of training and learning has to be done before someone can be licensed officially as an EMT. EMT training is not only about conceptual knowledge, but also learning how to correctly perform physical, sensory-motor tasks which can be hard during the Covid-19 pandemic when everything is done virtually.

Currently, most of the difficulty and cost comes from pedagogy and assessment. In order to learn how to operate the equipment and conduct a myriad of procedures, potential EMTs have to watch online videos and obtain their hands-on physical training devices in order to practice conceptual and sensory-motor skills. On top of that, when individuals go through licensing exams, it’s extremely hard for remote exam proctors to assess and dissect what the participants are doing within the exam over the internet. Utilizing Augmented Reality for pedagogy and assessment allows for a more hands-on approach for the trainees and also allows proctors to more easily monitor their performance.

Literature Review

Through a literature review, we found a few research papers that focused on the design of AR applications and research into new methods of UI/UX that were beneficial to our project. These papers range from the effectiveness of different types of interfaces for visual communication, 3D gestures, and the benefits of micro-projections.

Rosenthal, Stephanie, et al. explored the effectiveness of micro-projections for task completion purposes. The tasks that were most helped by AR were tasks involving reducing visual context switching by providing to-scale instructions and tasks where the individuals liked having two ways of verifying instructions. However, this technology was detrimental to tasks when they caused visual interference, hand occlusion, and difficulty aligning AR objects to the real world.

Feiner, Steven, et al. describes KARMA, a prototype AR system that explains simple end-user laser printer maintenance using an HMD. The paper itself references other design principles that can aid in visual demonstrations, such as highlighting objects in the user's environment to call attention to them, using arrows and callouts and leader lines to guide the user in the manipulation of these objections, and presenting abstract information near the physical objects that are being tracked. Agrawala, Maneesh, et al. also introduces some design principles to generate cutaway and exploded-view visualizations in cartographic and technical applications, using a general three-stage approach.

These three papers provided valuable design guidelines and principles for me to keep in mind as my team explored best practices for designing an AR application.

Lastly, Delamare, William, et al discusses novel ways for users to learn about various 3D gestures through OctoPocus3D, a guidance system that displays a set of 3D gestures as 3D pipes. Gestures less likely to be recognized are thinner than gestures more likely to be recognized; thus, during the motion of a particular gesture, the guidance system is able to let users learn 3D gestures by controlling their arm to match what is displayed. 

Within our project we hoped to utilize many of these design principles including using AR for tasks involving reducing visual context switching, highlighting objects, using arrows for object manipulation, exploded-view visualizations, and thicker gestures.

Brainstorming

By referencing a CPR procedure on emt-training.org, I was able to determine the most important steps and processes to cover during our AR CPR tutorial. These steps included opening the airway, check for breathing, and circulation.

Mid-Fidelity Mockups

Using a CPR dummy delivered to us by Intuitive Foundation, I was able to take a photo and then composite the AR interface over the image within Figma. I converted the technical steps of the procedure into a more easily readable format with supplemental information and also did some UI/UX work.

High Fidelity Mockups

The following media represent animated wireframes of what the AR tutorial for CPR would look like within our application; they encompass three main steps of CPR - Opening the Airway, Check for Breathing, and Circulation. Everything was made in Cinema4D with some of the models coming from the content browser; X-particles was used mainly within the second step to illustrate airflow. Below is the starting screen of the experience.

Step 1: Opening the Airway

Opening the airway is an important step of CPR because it allows the EMT to position the patient in the best way for them to breath and also removes any possible obstructions of the airway due to body positioning.

Step 2: Check for Breathing

Check for breathing is a crucial step within CPR because it allows the EMT to decide how they should proceed with the procedure; if there is no breath, the EMT should give extra breaths towards the patient to stimulate oxygen flow.

Step 3: Circulation

Circulation is the most prominent step of CPR which consists of checking the pulse, administering compressions so that the EMT can stimulate blood and oxygen flow within the patient to attempt to reinvigorate breathing.

Bridge to Assessment

At the end of the tutorial segment of the application, this animation will play to indicate to the user that they should position their mobile device and start compressions so that we may assess their performance.

Assessment

A prototype was made to assess the compression rate during CPR. After a phone is secured to the mannequin, the phone uses accelerometer data to determine the rate of compression. We analyzed two different approaches to calculate the compression rate.

By applying a discrete fourier transform and a threshold to the accelerometer data of the iphone, my teammates were able to calculate the frequency and depth of the compressions. More detailed information can be found in the video found at the beginning of this page.

Below are a few UI/UX mockups I created with the analyzed accelerometer data to represent what our application’s interface might look like.

Reflection

Since this was a new project conducted from scratch, we had to research the broader project specifications in order to narrow down a few potential approaches before honing down on the most feasible EMT field enhancement for this class; the mannequin was also only delivered to us 14 days before the presentation so we didn't have enough time to take the project to an MVP beyond the proof of concept. We hope that our work will provide a good foundation for future research and work related to this topic.

As a designer, this was the first time I had worked on an instructional AR application and it was a great learning experience for me. Creating the shorter animated gifs were a lot of fun and also challenged me to think of the best ways to visualize 2 dimensional instructions in 3d space. Overall, I’m happy with how everything turned out with the short amount of time we had.