A commissioned usability and user experience study by a leading healthcare machine manufacturer

To begin with, our research team had to undergo an in-depth knowledge transfer with the client's internal stakeholders, a team of engineers, subject matter experts, and technical experts. This included an introduction to the Nuclear Medicine (NM) imaging process, the NM department, technical terminology, and a complete familiarization with the device itself, focusing on the functionality of its operation console. This led us to conduct a detailed study of the existing system through a literature review and observation of the scanner in a simulated setting in order to understand the workflow and touchpoints of the current system. We built a preliminary understanding of the imaging process, became well-versed in the technical processes and terminology involved, and developed a better understanding of the role of the personas involved within the NM department. Our research team, in collaboration with the client's internal stakeholders, arrived at an outline of objectives, which was adopted as a hypothesis.

Hypothesis

There are three key areas that need attention to capitalize on molecular imaging's unique capabilities and elevate a nuclear medicine department to a highly visible and integrated part of the patient's care team. These key areas consist of: 1) staff and patient focus, 2) human factors, and 3) workflow management.

Pre - Field Research Activities

• Secondary research began with competition benchmarking of the nuclear medicine imaging devices and analysis of in-market competitors. This was crucial to the process prior to in-market immersion research observations. This research focused on the technical functionality of the competition's devices, applications, and their imaging system to identify differences in their workflow and the device's technology

• Mapping of user journey of the current Imaging system in the Nuclear Medicine (NM) department involving the two primary personas; NM Physician and NM technologist

• The terminology used to categorize the different processes in the imaging system was relevant to on-field observations as one of the first steps in designing the research tools. The nomenclature was selected through a literature review in collaboration with the client's internal stakeholders

• The insights gathered led us to design and develop primary research activities and tools based on ethnography, in-depth interviews, and heuristic evaluation. This included a participant observation guide for NM technologists, interview questionnaire guides for NM physicians, interview questionnaire guides for NM technologists, a UI observation guide for heuristic evaluation, and observation and on-field data collection protocols envisioning multiple scenarios and personas within the NM department performing major and non-major tasks.

• Preliminary understanding of the layout of the Nuclear Medicine department, that points to a typical PET-CT facility in hospitals and diagnostic imaging centers.

• Identifying, selecting, and managing research recruiting vendors for hospitals, multispecialty hospitals, and diagnostic centers, as well as developing user research assets such as interpreters, discussion guides, observation guides, and legal documents like NDAs and documentation for audio, video recording, and photography permits.

This led us to design a pilot study which was the fundamental phase of the research process to test the feasibility of our research approach and tools prior to the larger field study.

Research Strategy

Our research strategy was tailored with a combination of in-depth participant observation and on-field or simulated contextual observation of the imaging devices within the NM department. These two partnered approaches consistently generated an initial substantive bank of both articulated and latent needs and pain points experienced by the primary personas. Our research team, comprising design researchers and UX designers, had the ability to draw insights from intensely listening to and acutely observing user scenarios, nonverbal communications, pain points, and challenges, with a close eye on the limitations of the client's device and competitor's devices.

Caveat: Our research team had to keep an open mind about the social dynamics and hierarchies of clinical seniority and experiences that could lead to varied data, biased feedback, and in rare cases, a less substantive understanding.

Heuristic Evaluation

• UI observation Strategy was developed using Jakob Nielsen and Rolf Molich’s design principles on-field observations relevant to major and non major tasks scenarios involving primary personas. Heuristic evaluation was conducted of the current application of the PET/CT scanner in a simulated environment.

• Major tasks of the PET-CT scanner were broken down into three categories. These tasks were carried out on the operator console that were analyzed during the heuristics evaluation

  1. Pre scan tasks (maintenance)

  • Tube warm up

  • Fast calibration

  • PET daily quality assurance

  2. Scan tasks

  • Patient information

  • Protocol selection

  • PET tracer activity information

  • Scout

  • CT configuration

  • PET configuration

  • CT scanning

  • PET scanning

  3.  Post scan tasks

  • Fusing images

• Cognitive walkthroughs were conducted with the client’s internal stakeholders, to locate gaps and pain point areas in the flows the research team would test, according to different scenarios that may or may not affect the workflow such as: two or more technologists present on site and how that impacts the workflow, one technologist doing multiple tasks with support staff assisting the primary persona etc.

Nested Workflow Recommendations

• System can provide user the flexibility to plan multiple series together, so that technologists do not have to wait for each acquisition to complete, effectively saving repeated efforts.

• The system can incorporate Dynamic Protocol Selection that can enable the acquisition of brain PET-CT along with whole body (WB) PET-CT, saving 3 -4 mins taken for additional brain scans.

• The hand rest could be made ergonomic to support the patient’s hand without limiting the acquisition of brain images during WB PET CT scan.

• Breath Hold CT scan could be acquired along with WB PET-CT, effectively saving 1.5 mins required for this additional scan

• The process of landmark can be eliminated and it can be replaced with an auto iso centering mechanism. This would improve the user's workflow and accuracy achieved in patient positioning. It would help saving 1 min in the total acquisition time

• The scannable range of PET scan could be increased to WB PET-CT and enable head to acquisition in one scan. This will increase the throughput by saving 10-12 mins required for an additional feet in WB scan

• Reduce frequency of delayed scan to improve throughput via bed related flexibility

• Dose calibrator and the console could be connected to enable auto transfer of dose information to the console, saving the technologist 1 min of time and also aid in discouraging malpractices such as injecting 3 patients at the same time.

• Introduce a system reminder feature that will alert the technologist to inject/scan the next patient when the accurate uptake time has been achieved. This will discouraging

  malpractices such as injecting 3 patients at the same time.

User Experience Implications

• Due to the unorganized presentation and grouping of UI elements, the screen looks cluttered which results in increase time for assimilating information and planning

• The current data input system is counter intuitive making it a high concentration & tedious task

• Technologists are forced to navigate through non essential data fields to input essential patient details

• The current UI of Define Scan Plan has all the features scattered across the entire screen leading to more confusion and cognitive overload to the user

• The current cluttered user interface leads to user memorizing the location of required features rather than identifying them on the screen

• The user requires more time to accomplish the task as they have to search for the features to be used

• Errors in data entry fields leads to an increase in refilling information

• There is no clear navigational flow between the tasks on the CT PET configuration window.

• There are no shortcuts for configuration parameters

• The UI of the filming process lacks customization capabilities

Key Findings and inferences

The biggest impact from this user experience and usability research pertained to the categorization, prioritization, and troubleshooting of different types of problem areas within the PET-CT imaging system and workflow. Inferences gathered from the complexity mapping and user journey mapping, as well as user and patient pain points, led the client's team to identify and address the problem areas.

• For the client, the research findings and implications impacted the design of solutions and the reorganization of the UI to incorporate the development of new features, streamline existing ones, or discontinue the ones that are least used. Recommendations for the improved system now offer the potential to reduce scan times or patient preparation activities

• Recommendations and solutions were broadly categorized under the following:

• Simplification of the process that would lead to reduced time, frequency, complexity and effort

• Automation potential that could incorporate digital interventions within the workflow reducing the scope of human errors

• Learnability

• User Productivity

• Patient Experience

• User Experience & Safety that would enhance the PET/CT imaging process while maintaining the safety of both the patient and the user

Analysis

• Complexity mapping Methodology was designed to map the effort required to accomplish a task with 4 main parameters:

  • Perceptive : Prerequisites required for performing a task. Eg: Receiving patient file to input patient data

  • Cognitive : User has to perform additional tasks or parallel activities such as referring, analyzing, computing or comparing data

  • Action : User has to click on features or buttons on the gantry/ SCIM board/ software or perform a physical task. Eg: mount phantom, press table movement button on the scanner, press confirm on screen, and press “move to scan” on SCIM board

  • Movement : User has to move between different rooms within the NM department, i.e Console Room, Scanner Room, Hot lab and Physician's Cabin

• Transcribing process of the field work and coding of the information based on the parameters defined above

• Workflow mapping across geographies

• Persona Mapping of the primary personas. The users we observed reflected these characteristics on a varying scale and they were fit into two archetypes of an expert and novice

• Pain Points were categorized across geographies from the wholistic lens of the users as well as patients. For example, lack of sensory feedback for the patient in the gantry as they cannot perceive the status of the scan causing restlessness and patient movement during the process. Cluttered and redundant information on the user interface adds to the cognitive load on the user in accessing the relevant information.

• Designed parameters for identifying impact of a recommendation. The following parameters were assessed for rating the impact of a recommendation:

  • Reduce Time

  • Reduce Effort

  • Reduce Frequency

  • Reduce Complexity

  • Reduce Exposure

  • Improve Workflow

  • Improve Throughput

  • Improve Learnability

• Each parameter was given a rating of 1 point. If a recommendation fulfills 1 parameter, it’s impact is rated as 1. If the recommendation fulfills 5 parameters, it is rated as 5. Through this methodology, all the recommendations have been rated from 1 to 8.

• Additionally, the following tags have been used to determine the kind of opportunity area.

  • Excites the User

  • Technology opportunity

  • Training opportunity

  • Catching up with the competition

  • Key differentiator

  • Improves patient experience

Field work

Field work was divided into two phases: research sites pan India and global healthcare sites in three countries (China, Istanbul, and Egypt). Research sites were recruited based on the client's device location and competitor's device sites across the above-mentioned geographies. In this study, we conducted 45 user interviews involving participants operating the client's device and competitor users operating competitor devices. For the client's device in India field research, our team conducted participant observation and in-depth interviews at five centers: Mumbai, Nashik, Mathura, Cuttack, and Satara. We interviewed 13 participants, as a few sites had more than one technologist. For the client's device in China, we conducted research in three sites, in Turkey and Egypt, research was conducted in two sites each. The number of user interviews varied from country to country. For example, in China, support staff was actively assisting the NM technologists due to the high volume of patient traffic, whereas in Turkey and Egypt, they were working independently. In China, we interviewed 12 participants, 6 in Turkey, and 4 in Egypt. At competitor’s site, we observed one site each in India, Egypt and Turkey. Since it is a confidential healthcare device study, it was very difficult to recruit the ideal competitor devices sites. We interviewed 10 competitor user participants.

Field work entailed the following:

• Understanding the current imaging process of PET-CT scanners, including how often NM technologists conduct quality assurance tests, how they perform exam set up, dose preparation, patient intake, patient adjustment on the console, how they conduct a scan, how they manage and resolve issues, what data needs to be stored, what information they need to access periodically, and other steps

• On-field participant observation included task-based observation. Tasks were mapped based on complexity, which was later adopted into a complexity mapping methodology for task analysis. Participants (NM technologists) were closely observed from their first activity of starting the scanner (system start-up), to machine warm-up, lab preparation, patient intake, patient preparation, acquiring the scan, evaluating the scans, creating images, processing the images, composing the films, and ultimately sharing the images with NM physicians or burning the files on a disc. During this process, participants were observed, shadowed, and asked questions by our research team

• Hot lab observations including phantom management and dose preparation were conducted on sites

• Bring up use cases and test hypotheses with the personas according to the conversation

• In-depth interviews were conducted with the primary personas (NM technologists and physicians) at every site to understand their daily activities, pain points, gaps in the system, and what would make their tasks easier.

Pilot Study

In this initial exploratory pilot study, our research team aimed to test the outline of the hypothesis and importantly, the research tools for data collection (observation guides and in-depth interview questionnaires) at two live sites in Bangalore, India. This exercise prepared our team to observe the workflow of the NM department. Participant observation in a live setting of a diagnostic center and a cancer research institute helped the team to build on the foundational knowledge gathered during observation in a simulated environment.

• Observed timed activities of primary personas from daily device preparation to patient preparation, image acquisition (PET-CT scan), and film processing for diagnostic exams. Examined major tasks such as specifics of patient-caregiver encounters, manual document exchange steps, spatial relationships & facility navigations, activity timings & waiting instances, and incremental preparatory steps involved in the overall Nuclear Medicine department

• Understanding the current imaging process of PET-CT scanners, including how NM technologists do exam set up, patient intake, patient adjustment on the console, how they conduct the exam, how they manage and resolve issues, what data needs to be stored, what information they need to access periodically, and other steps

Post the pilot study, we refined the hypothesis and research tools prior to the field study.

NM technologist at the Operating Console at Zhebei Mingzhou Hospital, Huzhou, China

Key Personas