Mobile Conversational AI with Medical Data Intelligence

Virtual Collaborator.

Nursing Assistant

Our application helps clinicians with critical thinking decision-making through innovative data access and communication with our Natural Language Processing assistant Ezra

Overview

The Virtual Collaborator is a patient-centric platform for nurses and hospitalists in the ICU for monitoring sepsis-prone patients and promoting rapid response team decision-making. Using the Virtual Collaborator, Ezra follows a patient from admission to discharge and can extract clinical data from the electronic medical record (EMR) and lab reports. Nurses can ask for specific information or generate graphs like you would any team member.

Ezra can pre-empt a request by providing vital signs or lab results as soon as it has been recorded in the EMR. The HCP team can also request information from Ezra. Lastly, Ezra can keep you and the team aware of using pre-built and personal notifications of vital/lab abnormalities, reminders, and event callouts.

Natural Collaboration

Chat elements should be visual & quick to understand. Communication with Ezra should be clear & straightforward.

Layout Hierarchy

Screen layout should be uncluttered, easily accessible with information clearly presented

Simple Navigation

Users should know how to go back to the home screen or how to reach patients quickly.

Contextual Notifications

We only notify users when necessary to limit alarm and notification fatigue.

  • Early User flows with Clinical Specialist, Designers, Developers and Product Owners

  • Sketches for the Virtual Collaborator Application Patient Cards and Vital Displays

Promoting Mobile Collaboration

 We focus on two major pain points: data access and communication, where clinical staff can have easy access to actionable information. Using Natual Language Processing, we made Ezra, a chatbot who can quickly get hospital data as well as can generate unique data visualizations. We focused on releasing staff from cumbersome computer stations they're forced to return to for any actionable data.

Left to Right: Patient List, Patient Chat Channel, Requesting Custom Graphs, Private Chat

Actionable Information First

One of the many challenges we faced on the project was balancing introducing new workflows with the demanding environment in the ICU. Virtual Collaborator combines traditional data dialogue methods while identifying subtle ways to add new techniques. This helps in two ways by also hinting at the functionality and knowledge base Ezra for the chat.

Each page provides in-depth clinical information from vitals, labs, meds, and orders. We promote actionable information first without losing detail using iconography, color, and passive interactions with data points.

“Up to 49% of clinician time is spent interacting with the EMR.”
— Zanotti-Cavazzoni, S.l
Left to Right: Vitals Expanded, Lab Results, Medication Delivery, Patient Summary

Awareness without disruption

With Sepsis, response time is a critical factor, so keeping them alert with information while balancing the fact nurses are overwhelmed with notification fatigue. We faced this by allowing nurses to develop their own and standardizing the priority.

From the patient list, you can see how active a patient channel is. Nurses can snooze notifications if busy and highly requested in the rapid space. 

Clinicians and care teams would be able to set their own notifications related to specific changes to clinical status. These could be based on published early warning scores or your facility’s protocol.

Building our Design System

Unique for our application. I made our design system to promote the feeling of a clean environment, flexible. The design system was made into three parts: the visual design, which focuses on building a design language promoting the GE message of constancy, and clarity.

Second is the data visualization design system with a focus on actionable information first. Colors are consistent with hospital standards to reduce cognitive load, and allowing for touch navigation for our graphs will promote readability even on data-heavy visualizations.

Lastly is the NLP (Natural Language Processing). Instead of searching the EMR, our chatbot Ezra (working name) brings the information to you. Our responses are designed in both visuals and language to promote actionable information. Reduce cognitive load. At the same time, it is an adaptive structure to handle the 3,000 unique requests and correlating responses.

Design System vr. 2.0 of the Virtual Collaborator

Project Retrospective

We faced many challenges in this project, working with the constraints of regulations on relatively new technologies (NLP, Chatbots, Messaging) and the difficulties of introducing users to them. Through the design process, we created many creative solutions that are not just innovative but valuable. We believe these methods will last for many years, balancing the delicate workflows our users have built. Our challenge was making a product that is official, secure, and improves upon the workflows.

With our app's new workflow, including new technologies for most nurses, user testing was a priority. Over six months, we ran five in-person interview sessions with over 100 medical professionals. Here we would gauge insights into feature desirability and usability. These sessions were an essential way to expose crucial unknowns and encourage new approaches. It also reconfirmed a great excitement in our approach with many interviewees wanting to use the prototype in their teams.

The Virtual Collaborator was shown at HIMMS 2019 with much interest and will be used for testing in hospitals over the next year. Product Site