1. technology and computing
2. software

9
X
Usability Mistakes Your
Team is Probably Making
(And How to Fix Them)
Macadamian’s usability experts answer
frequently-asked healthcare software usability
questions and address common design
process mistakes.
Lorraine Chapman, Melanie Rodney, Didier Thizy
NINETY PERCENT of physicians are concerned about the
usability of Electronic Medical Record systems!1 Too many
clicks! What will make healthcare software usable? Search
for “healthcare software usability” on Google and you will
find thousands of these recent headlines.
The message from physicians and other clinical staff is clear – many of
today’s solutions are perceived as unintuitive, with the potential to disrupt
workflow and slow down their practice. This has introduced a tremendous
opportunity in the market. Organizations that focus on product usability may
gain competitive advantage by showcasing success stories of innovative
design, ease of adoption, and long-term satisfaction.
Yet, most vendors already put careful thought into their software design,
often with help from a partnering physician or advisory board. And as of June
2010, 95% of CCHIT-certified electronic medical records systems received a 4
star usability rating or higher.
Nevertheless, today’s headlines reveal that there is still vast room for
improvement –leaving many organizations wondering what more they can do.
Answering your usability questions &
addressing common errors
Over the past year, Macadamian has presented over 100 seminars and
courses on user-centered design to organizations around North America. At
the end of each session, the same questions always arise, revealing flaws in
the way most organizations research and design their software products.
In this paper, our senior usability specialists tackle 8 of the most commonly
asked usability and design questions. As they dig into each question, they
uncover common design mistakes that may explain this usability “disconnect”
between vendors and healthcare users. Each mistake is addressed with a
simple process improvement recommendation which, if implemented, could
set vendors on a path to developing truly outstanding user experiences.
1. reference: www.healthcareguy.com/2009/04/13/guest-article-what-willmake-healthcare-software-usable/
3
1
X
Mistake # 1
Displaying All Important
Information On “One Screen”
Q
Physicians tell us that they want all of the important
information presented by a software product to be
visible on a single screen. How, though, can we fit all of that
data into one screen without making it cluttered?
A
This is a classic user interface problem. Typically, this
type of observation stems from prior experiences in
which the physician was forced to hunt for information.
Users become very frustrated when they can’t find the information they are looking for. In their mind, the solution to this problem is to have all of the information
available “up front” so that they don’t have to search for it. The question posed
above, however, points out the flaw with this approach. A single UI screen simply
does not have enough space to house all information. Even if it did, this approach
would still not be advisable as it would lead to a very cluttered main page!
Clinical users are really asking for the ability to obtain the information they are
looking for in a quick, non-stressful fashion. You can only help them do this if you
understand the common tasks performed by your users and the terminology that
they use in their work day.
Reserve dashboards for primary tasks only
We frequently see cluttered dashboards / main screens in electronic medical
records and patient portals – particularly when they were designed without
formal user research and usability testing.
5
By modeling your users’ workflow, you may discover that they have a distinct set
of primary and secondary tasks. Users may look for certain information dozens
of times per day, while they may only want to access other information once a
day or once a week. The key is to ensure that the information can be found and
accessed in a logical fashion and displayed at the right time.
In one patient portal project we worked on, we realized that patients using the
system were mainly interested in renewing their prescriptions or sending the
nurse a question. We identified those activities as their primary tasks. Only
occasionally were patients interested in browsing the physician’s progress notes
from previous encounters, so we did not make that feature an important “first
screen” priority.
Secondary tasks require a clear, logical flow
Jeff Belden is a physician and UI designer we work with on the HIMSS usability
task force. He has a popular blog called Too Many Clicks, emphasizing that
physicians and nurses can become especially frustrated with software when they
have the impression that they are “clicking too much” to get to the information
they want.
Research has shown that if the path to the desired information is clear and
obvious, a user will not mind navigating two or three pages “deep” as long as
they know that they are on the right track. It is therefore important to design the
interface in a way that reduces the time that a user must think about where to go
and that moves them through the interface in a logical fashion. If users become
confused about where to look for information or the time it takes to return to
that information, they will become frustrated and their confidence in the solution
will be shaken. A desired page might be just one or two clicks away, but if users
do not see a logical path to that information, they often assume that it is not
available – or simply give up before finding it.
We once worked on a clinical decision support system for nurses. Here, nurses
were using the “Patient Handouts” section multiple times of day because they
needed to provide the handout to each patient upon discharge. We discovered,
though, that nurses would only use the system to look up specific evidencebased nursing monographs once a week or less. Naturally, we designed the
system to allow quick access to the former, and clear, logical access (albeit more
than one “click”) to the latter.
Observing users is the key to distinguishing
primary and secondary tasks
In the same clinical decision support project, we monitored nurses as they navigated through the existing system and we interviewed them about the experience.
Users told us that the system contained a lot of rich and valuable information,
but in observing them it was clear that they were frustrated by the need to weed
through a number of pages, links and resources to find the information they were
looking for.
6
The combined results of the observation and formal interviews led us to a new
design. The product would offer a “Search Links/Resources” section. Nurses
conducting a search on “preventing sepsis”, for example, would be presented
with the resource they were looking for along with other related topics or resources, not unlike the “You Might Also Like…” suggestions provided by modern
e-commerce websites. This paradigm worked for the nurses because the relevant
additional information was visible and only a click away, allowing them to easily
do follow-up searches for the answers they were really looking for.
Often, when we observe people using software we discover “little” things –
such as the need to actually click on each article/journal abstract presented
in a search to determine whether it is really the one they want to read. In the
clinical decision support project, we ended up implementing a fix in which a
short description/abstract of the article or journal would pop up when the user
hovered over the search result. This allowed users to easily skim the page with
their mouse (without having to go back and forth) to determine which article they
actually wanted to read further.
Use the right clinical terminology
At the risk of stating the obvious, it is crucial to make sure that your software
uses the terminology that is expected by your target users. We are always
surprised at the number of solutions that frustrate clinical workers simply
because the information they are looking for is hidden behind an unfamiliar or
confusing term.
For example, we worked on a leading CPOE system that asked users to enter
the name of the “Ordering Provider”. The software vendor believed the term was
clear – “this is the person who is placing the order, typically the originator of the
patient chart”. When we observed and interviewed users however, we discovered
they were confused by wording. They were looking for the term “Attending
Physician” or “Consulting Physician”, reflecting the role of the person on staff
who would be placing the order.
³
The lesson is simple – a software interface’s terminology must match the
language used by users – not the language you think your users would use.
The lesson is simple –
a software interface’s
terminology must
match the language
used by users – not the
language you think
your users would use.”
7
As long as Healthcare IT interfaces continue to frustrate users by offering
unintuitive solutions, users will continue to ask for solutions that show all
information on one screen so that they only have to learn that one screen. By
carefully understanding users’ daily tasks and goals, using the right terminology,
and designing dashboards and logical flows based on primary and secondary
tasks, vendors can stand out with an intuitive experience that offers fast access
to all information.
2
X
Mistake # 2
1
Focusing On Features
Rather Than User Tasks
Q
In our industry, clients are always asking about
features. In a competitive situation, the product with
the longest feature list always seems to be the one selected.
Why should we focus on UI design and usability if our
clients are going to make a decision based on features?
A
This perception is quite common in the Healthcare IT
industry and, in many cases, is reinforced by vendors
that track and rank their competitors by their feature sets.
Usually, companies choose to do things this way because it
is simple – counting up or matching features is reasonably
straightforward.
This sort of analysis, however, has some major drawbacks. First, it implies that
“more is better” – i.e. that a product with more features is superior to a product
with fewer features. There is also another implicit assumption made that each
product feature addresses a desired user goal or task. In many cases, however,
the link between a feature and user goal is either tenuous or non-existent.
Identify goals and tasks
When we talk about usability and an effective UI design, we’re really talking
about understanding the key tasks users want to complete and providing them
with the capability to accomplish those tasks. Remember, users don’t approach a
product with a feature in mind, they approach it with a desired goal or task.
8
For this reason, more isn’t always better when it comes to features. In fact, more
is almost always worse! An interface that offers a large number of features will
become bloated, cluttered and overly complicated. This, in turn, will cause users
to expend more effort than necessary to understand the features available to
them, resulting in excessive training and re-training requests, helpdesk complaints, inconsistent usage across clinical staff, and ultimately frustration and low
satisfaction. Users almost always end up using only a small percentage of the
features that, initially, seemed so important.
Design based on goals and tasks
(YHU\SURGXFWIHDWXUHVKRXOGGLUHFWO\VXSSRUWDQLPSRUWDQWXVHUWDVN2QFH
\RXKDYHPDWFKHGVRIWZDUHIHDWXUHVWRXVHUQHHGV\RXZLOO¿QGWKDWLWLV
PXFKVLPSOHUWRSULRULWL]HWKHVHIHDWXUHVDQGLQFRUSRUDWHWKHPORJLFDOO\
LQWR\RXUXVHULQWHUIDFH
The only way to truly match features with goals and tasks is to study the behavior of your end users: their needs, their goals, their workflow and their workplace
processes. This is possible through direct observation and feedback from users
and stakeholders. Using these techniques, you can organize tasks and subtasks
by order of importance and frequency, and map relationships and dependencies
between tasks through flowcharts and mapping diagrams.
Consider the CCHIT ambulatory certification which has been criticized for
requiring EMR vendors to offer a large number of features. Robert Rowley,
Chief Medical Officer of Practice Fusion, opined: “Each year, new criteria were
added to the CCHIT requirements (never removed), which resulted in ‘feature
bloat’ and became increasingly product- and feature-focused - around 500
items were included in their 2008 certification criteria, and many of them were
specifications apropos legacy client/server software.”
While opinions on the usefulness of CCHIT features vary, the sheer number of
requirements does put an additional burden on Electronic Medical Record vendors to provide simple and usable systems despite the large feature set. Faced
with this reality, vendors who seek the certification should start by identifying the
primary tasks and goals of the institution and specialty that the EMR is targeting
and use this information to organize features into an intuitive user experience.
Validate goals and tasks with users
In one recent project, our client – a leader in the oncology software space – was
convinced that oncologists needed a feature that allowed them to review medical
images from their iPad. We conducted onsite interviews and learned that a mobile device would not help oncologists with this particular task. Reviewing images
is a task that requires undivided attention, a large screen and, in many cases,
access to other resources such as a colleague or written reference. The use of
iPad to review images “on the go” did not fit an important goal or task, and thus
the feature was dropped from the final product.
By identifying users’ tasks and goals rather than a list of features, designers can
prioritize and validate those tasks by testing them with actual users. Ultimately,
this drives the blueprint of an application design, turning even a complex application into an intuitive user experience.
9
3
X
Mistake # 3
1
Directly Mimicking The
Paper-Based World
Q
Clinical users can be very wary of change. They tell
us that they want their software products – especially
electronic medical records (EMR) solutions – to mimic as
closely as possible the actions they already perform as part
of their paper-based workflow. To what degree should our
products match existing paper-based processes?
A
Sometimes, there is a big difference between what
users say and what they actually mean. We have found
that when physicians say that they want their EMR to mimic
their existing paper-based world, they are really saying that
the new solution must support (and not interrupt or slow
down) their current workflow.
When EMR users make a statement like the one in the question above, they are
really saying that they want something that is familiar and doesn’t have a steep
learning curve. Physicians are extremely busy and, in many cases, previous
experiences with poorly developed software have made them wary of new
solutions. All too often, new systems force users to put in more effort without
getting an equivalent return in value.
Ideally, a new software solution should provide additional value while maintaining
or decreasing the amount of effort required. We call that balancing the value-foreffort equation.
10
It is possible to support – and even improve – a clinical user’s workflow using
familiar elements of their paper-based workflow, without copying each and every
aspect of it. Well-designed software interfaces should be introducing efficiencies
and eliminating steps that are unnecessary in the digital world.
Case study: designing an electronic medical
records system for small clinics
In a recent project, we designed an electronic medical records system for small
clinic practices, particularly those that had never used an EMR system and were
planning to make the transition from the paper charting world.
³
When planning the
design of a system for
physicians, nurses and
other clinical users, it
is crucial to design to
their workflow.”
Together with our client, we brought together a team of people with different
backgrounds – clinical, product management, software architects - to participate
in the workshop. An opportunity the participants identified was use of a modern
touch screen and digital ink tablets that make it possible for users to write on
a screen as they would on a piece of paper, facilitating tasks such as drawing
images to facilitate communication between physician and patient, annotating
diagnostic images, and authenticating reports quickly using digital ink.
In addition, we arranged to visit small clinics that fit the “small practice” target
and observe the physicians and nurses as they worked through their paper-based
processes. This led to additional opportunities to incorporate familiarity into the
software workflow, such as:
• Displaying a scrolling pile of chart entries in chronological order, with the most
recent entry on top, replicating the big stack of paper that would have been
put into a chart over months/years of appointments. Users didn’t have to actively “open” a chart item – it was simply there when they pulled out the chart.
• Allowing physicians to “pull multiple charts.” In the paper-based world, physicians and nurses could carry individual charts in their hands from room to
room, so we replicated this in the software rather than taking a “one at a
time” approach.
• Providing free text “comment” fields. We discovered that whenever there was
a bit of white space on a piece of paper, someone would scribble some important information there.
While testing concepts and prototypes with real-world users, it was clear that
tablets and digital ink could not and should not be used to exactly simulate a paper chart. In particular, the use of these technologies can result in loss of precision, illegible handwriting, and error-prone conversion to text.
Moreover, we realized that while there were opportunities to capitalize on the
familiar by incorporating some elements of the paper world, the biggest opportunities were in providing added value to those processes - automatically filling
a patients name and demographics into an electronic form, filtering and sorting
progress notes in a digital chart, and transferring data instantly to the workers
who needed it.
When planning the design of a system for physicians, nurses and other clinical
users, it is crucial to design to their workflow. Rather than copying the paper
world outright, a new software solution must carefully balance familiar elements
and innovative designs, providing an interaction that adds value but still looks
and feels familiar.
11
4
X
Mistake # 4
1
Thinking “Redesign”
Instead Of “Update”
Q
We want to improve the usability of our product but
our users have already become familiar with our
existing UI. Many users are happy and probably wouldn’t
want us to change a thing. How do we improve our UI
without alienating our current users?
A
This is a legitimate concern, but there is an underlying
assumption here that needs to be discussed. When
we have heard physicians say that they are happy with an
existing interface, we’ve found that they are actually saying
“I’ve put a lot of effort into learning how to navigate this UI
and I don’t want to have to start from scratch again!” A good
UI, however, will actually simplify processes for both new
and existing users, and there are a number of best practices
to follow to accomplish this.
“Updating” does not necessarily
mean “redesigning”
In most cases, it is not necessary or advisable to start from scratch and completely redesign a UI. Many existing software products have been in the field for
many years and will always need to maintain aspects of their existing framework.
Instead of contemplating a complete overhaul, think about making specific UI
improvements that will yield the highest business value.
12
Sometimes, a solution simply requires a visual design refresh and this process
may not need to impact the interaction design. In other cases, a vendor may
think that a visual redesign is required when, in fact, the real problem lies in the
solution’s interaction design.
Do you know what specific aspects of your user interface would yield the highest
ROI if you updated them? Often it is the usability and flow of screens that users
interact with on a daily basis to accomplish their primary tasks, like dashboards
and main pages. Other times the issue lies more in the overall “wow factor” of
the application to attract new clients.
The only surefire way to glean this information is to conduct the appropriate user
research, which could be a series of usability tests, interviews, or ethnographic
study. User research should guide the update – not your intuition.
Get the right feedback from the right users
When performing user research on an existing product in order to determine
what aspects of the interface need the most attention, you need to identify
the right users to research. Customer support and IT helpdesk teams can be a
helpful source of data, and they may help you identify some of the problems.
A partnering physician’s opinion can also be a useful data point. But if you rely
exclusively on these data, you are taking a big gamble.
First, you don’t want to rely on one user group or person because they may
not adequately represent the needs of all user groups. It is vital to identify and
conduct research on all user groups. For example, if your solution is already in
the market, gather data from existing users as well as prospective users. If your
solution is used in a hospital, gather data from multiple physicians, nurses, nurse
practitioners, administrative staff, etc.
Second, one of your data sources should be observation of users working with
the solution in the field. Often, what users say they want and what they actually
need are very different. Thus a partnering physician’s opinion can be valuable,
but it should be paired with watching him use the solution in practice.
Build on the familiar
When updating a user interface, be sure to extend it based on what people are
already comfortable with – be it from the current interface or based on an industry standard. Evaluate how well the existing interface matches activities a user
would be familiar with from other software solutions or activities performed in a
typical workday.
For example, if your solution requires physicians to double-click icons while
the other solutions used in his institution require a single click, your existing UI
may be increasing his cognitive load. In other words, he may need to think and
expend more mental effort with your product than they do with others.
Some electronic medical records applications use familiar elements of Microsoft
Outlook for the messaging functionality in the EMR, or elements of Facebook for
the patient portal website. While the degree of success they will have depends
on the details of the implementation, the general idea of building on the familiar
is sound.
13
It’s all about the data
A great UI often is a UI that is unnoticed during use. The interface and user
interaction seem to “disappear” because they are so intuitive and simple. Clinical
users are able to concentrate on the data – not the interface. If people are worried about changing the interface, this might be because users have had to think
too much about their interaction with the product.
Observe and interview the right groups of users. Identify which elements of the
existing interface need an update. Then update (rather than redesign) those elements, building on the familiar. This is the way to let your data shine, ultimately
pleasing both existing and new users.
14
5
X
Mistake # 5
Not Validating Correctly
With Users In The Field
Q
We know that user research and testing are important,
but it is so difficult to get certain users – especially
physicians – to give us their time. How do we get face time
with physicians? How many do we need to interview? And if
we can’t get their time, what alternatives are there?
A
It is very true that physicians can be difficult to reach.
Large organizations typically work with an advisory
board or partnering physician, and even in these cases it can
be difficult to get time to observe and interview physicians in
the field.
We frequently have to recruit users to do interviews and test concepts and prototypes. Below we’ve listed a number of best practices our staff follows in order to
recruit clinical users and maximize the time you have with them.
Encourage physician involvement
We have found that the best way to encourage physicians’ involvement in usercentered design is to demonstrate that their input will improve their work lives
and those of their colleagues.
It is important to show physicians the value of interviews and testing, and explain
the benefits that they will receive – namely, a software product that meets their
needs. Physicians are busy, but they are also human beings. If you demonstrate
to them that their contribution will have a positive impact on their work life, they
will likely be more open to the idea.
15
It’s not just about the physician
Physicians are clearly an important user group, but they may not be the only primary users of your product. Other groups of users who may directly or indirectly
use Healthcare IT software include:
• Nurses and nurse practitioners
• Technicians
• Hospital administrators and executives
• Department managers, middle managers
• Physician-run office workers
• Physician or independent researchers
³
• Patients and patient families
Just-in-time research,
specific questions, and
mini surveys – these
shortcuts are valid
ways to keep iterative
research and design
moving.”
Often, physicians are the ones who request the development of an application
and are the only ones contributing to the requirements. When the software is
released, however, it can fail to deliver an improvement in the institution’s overall
process because it does not deliver what is needed to all those who need it.
Maximize the efficiency of sessions
If your team feels that it is taking up too much of a professional’s time, spend
time preparing concept designs, story boards, or even a jUCMNav process diagram ahead of time. Presenting healthcare professionals with prepared material
during a brief meeting will keep communication clear and simple, confirming your
understanding of their needs.
Remote interviews and testing
There are certainly ways to conduct user research at a distance. Research
techniques using telephones and video conferencing can be used to present
concepts, interview users, and conduct remote usability testing.
Even e-mail can be used to query and interview users on a semi-regular basis. It
can be difficult to get a group of physicians and nurses together on an ongoing
basis, but if these users belong to an e-mail list, designers and developers can
e-mail them specific questions about processes or procedures.
Just-in-time research, specific questions, and mini surveys – these shortcuts are
valid ways to keep iterative research and design moving.
Research by proxy
You can also talk to secondary stakeholders who oversee the process/task in
question, or work closely with the physician, or have the background and experience of the target users but are no longer actively working in the field.
16
For example, we worked on a Hospital Information System by interviewing a hospital’s nurse educators who were providing skills training to staff nurses within
the hospital, but were not active staff or unit nurses.
If you rely on secondary inputs then you also have to understand the risks,
including:
• Biased information
• Missing information, because the users know more about their own needs
• Incorrect information, especially as it relates to a physician or nurse’s goals
and motivations
Interview the right number of users
The number of users that you must interview depends heavily on the type of
usability research you are conducting and the number of distinct user groups
for your product. As a rule of thumb, you can rely on ten behavioral interviews
or observation sessions per user group. Research shows that groups of ten can
uncover more than 90% of all usability issues .
Note here that we are talking specifically about research into user behaviors –
not attitudes or opinions. Behaviors are much less variable than opinions. If you
are researching user views, you will definitely need a sample size larger than 10.
An experienced usability researcher should be involved in order to help assess a
project’s particular situation and needs. Usability researchers are trained to:
• Plan the right “mix” of interviews, design walkthroughs, user testing, etc.
• Ask the right questions using accurate domain terminology
• Analyze user behavior
The goal is to gather the right user feedback quickly and efficiently, and then
interpret that data so it can be fed into the design.
17
6
X
Mistake # 6
1
Rushing Into Mobile Healthcare
Application Development
Q
We know that mobile technology is on the rise, and
we are feeling pressure to release something for
physicians on a mobile platform. How do we decide what to
offer and which features to include on a mobile version of
our product?
A
Not all products lend themselves well to a mobile
application. You need to determine whether there
really is a value proposition for developing a mobile version
of your product. Investigate the types of devices your users
are currently using, their existing work processes, and
determine whether there are tasks that could be performed
on a mobile device to raise their efficiency.
Once you have validated the business case, then it’s time to look into specific
opportunities to innovate with smart phones, tablets, and other portable devices.
Determine the business value
We have worked with many clients who ask themselves: “how do we figure out
how to leverage something new (like an iPhone/iPad/Android app) when the users themselves probably don’t even know what they want?”
The first step is to uncover the business value of your potential mobile application. One way to dig for this information might be to start with a product strategy
workshop,in which you focus on capturing the key business objectives and user
requirements to identify key opportunities in evolving the product value proposi-
18
tion. The workshop could be followed by a concept design and a round of user
testing on that concept.
Determining the business value will allow you to determine what user tasks and
goals your mobile product needs to support. Note there is almost never a case
for having a one-to-one relationship between the capabilities of the desktop
product and its mobile version. User behavior can differ greatly from one platform to another because a user’s goal at a desktop may differ from his or her
goal with a mobile device.
Developing for smart phones
In our experience, physicians and nurses are comfortable using smart phones
like the Apple iPhone. Smart phones are best used for productivity-type tasks in
which time spent per action is short.
Physicians’ effort-to-value ratio is good because practitioners can carry them on
their person, check appointment schedules, send messages to receptionists, and
look up information. Many personally own smart phones, and if their IT solution
allows them to get work done on the go via their phones, this is perceived as a
major advantage.
The rise in demand for patient-facing software such as patient portals, nutrition
and fitness monitors, and other self-care applications suggests there is also a lot
of value in developing smart phone health applications for patients.
Developing for a tablet
Tablet PCs such as the Apple iPad and Intel Mobile Clinical Assistant offer
possibilities beyond smart phone applications. A few examples include:
• Explaining clinical results to a patient at the bedside or in the consultation
room. A tablet can enhance a discussion of test results or possible courses of
treatment with a patient. The physician can sit next to the patient and display
test results, show graphs, look at numbers and access clinical decision support material or other informative websites while explaining the results.
• Looking up resources on the fly. Having a tablet PC at the ready can be just the
thing that physicians need when a drug name slips their mind or they need to
look up options for a differential diagnosis. The ability to hook into the hospital’s infrastructure without having to walk to a computer station can also save
critical steps for nurses.
• Consulting data just before treatment. Before surgery or another type of medical intervention, a surgeon will consult with diagnostic images or other results
as a precautionary measure. With the right infrastructure in place, a tablet can
save the surgeon from making an extra trip to the lab to check this data.
For a detailed investigation of the opportunities and limitations of mobile tablets,
smart phones and other mobile devices, please see Macadamian’s recent paper
11 Disruptive Technologies That Are Changing Healthcare IT – And How Your
Competition Is Using Them.
19
7
X
Mistake # 7
1
Focusing More On Customization
Than Usability
Q
Customers tell us that they need a user interface
that can be completely customized to their practice
and workflow. In the past, however, when we’ve performed
customization work, it has taken a lot of time and effort,
and clients often end up not using the customizations. How
customizable does healthcare software really need to be?
A
Customers often do request a customized or tailored
healthcare solution, or one that they can customize
themselves.
Part of the reason for this is the inherent challenge of healthcare software – every hospital and clinic is unique – each has a set of slightly different processes,
and workers can be frustrated with software that requires them to change these
processes.
However, when we probe users about the need for customization, we also often
find that their request stems more from unhappiness with the current (or past)
design of another software solution they have seen. When users can’t make a
software product work, they frequently conclude that the solution must have
been set up for someone else’s needs. They then request a solution customized
to match their needs.
Despite each customer’s individual differences, there are a number of steps that
a vendor can take to design a solution that closely meets the needs of target
user groups and requires a minimal amount of customization.
20
Know your target specialty
Software teams do not always realize the extent to which requirements depend
on the specialization or generality of the clinical users. A general practitioner,
for example, will not have the same requirements as a physician specializing in
cardiology.
When we walked the exhibit floor at HIMSS10 and investigated the 100+ healthcare products on display, we noticed an overwhelming number of “one size fits
all” software. These solutions will inevitably frustrate any clinical specialist who
was sold on the idea that the product was designed for him.
For example, in a drop-in medical clinic, software records the nature of consultations that take place between physician and patient. The software must also
handle daily patient queues, but precise scheduling is not necessarily required.
On the other hand, a specialist at a teaching hospital requires software that records data from consultations and makes it available to be searched through at a
later date. These physicians are not only interested in treating patients – they are
also interested in evaluating the outcomes for patients with a particular disease
and in detecting trends among the population. In the drop-in clinic, the software
designer can simply write software that provides human readable data. In the
second case, the software must be designed to give access to the fine-grained
data stored in the back end.
User research reduces the need for
customization
³
In some cases, vendors take a customer-by-customer approach and add features
to address individual clients. The problem with this approach is that a vendor
might customize a solution for one particular physician only to learn later that
this physician does not fully represent all users.
The need for
customization
decreases dramatically
with proper usability
and user research.”
The need for customization decreases dramatically with proper usability and user
research. By finding and interviewing a set of representative users, learning how
they think and work and identifying commonalities, you can develop an interface
that successfully addresses the majority of your target users’ needs.
Customization screens must be usable
In some cases, vendors provide a software feature or mechanism for an administrator to customize the solution, or even for the users to customize aspects of
the solution themselves. Often the mechanism itself is not easy to use, and the
customer is either deterred by the lengthy process, the cost of hiring a professional services group to do the work for them, or the difficulty reporting issues
from their customized solution to the IT support.
We recommend that any software tool that allows users to customize a Healthcare IT solution should be given as much usability attention as the solution itself.
In a case study we cited earlier, we developed an electronic medical records
system targeted at small practices that were generally too busy to have the staff
customize the solution themselves, too small to have a dedicated IT administra-
21
tor do the work, and not able to justify the cost of a professional services team.
We spent more time planning the usability of the customization interface – suitable defaults, user and role preferences, flexible workflow options – than many of
the features themselves.
Hospitals, clinics, pharmacies - even individual users - are all unique. Each has
a set of processes and needs that may be slightly (or even wildly) different.
There will always be a need to offer a degree of customization, from a simple
preferences screen to more fine-grained mechanisms. In our experience,
however, organizations that invest time in carefully targeting specific user groups
and designing based on research with those groups ultimately satisfy more users
than organizations which focus on make “everything” customizable.
22
8
X
Mistake # 8
1
Assuming The CCHIT Usability
Rating Measures Outstanding
Usability
Q
Many electronic medical records vendors say that they
received a 5 star usability rating from CCHIT. How
reliable an indicator is this that the system is highly intuitive
and easy to use? Is it even possible to assign a score to
usability?
A
The Certification Commission for Health Information
Technology (CCHIT) offers a usability rating for
ambulatory electronic health record applications that have
passed its certification. The rating is meant to “reflect the
perceived usability of an Ambulatory EHR application as
rated by content experts (i.e. certification jurors)”.
During the certification process, a panel of CCHIT jury members is given a series
of questionnaires and instructed to rate the usability of the product based on a
demonstration by the vendor of the overall screen layout for common screens
(e.g. patient summary, prescribing, results review), the ways in which users
would navigate within a single screen, to other screens, etc. as well as a demonstration of several key usage scenarios. Overall, 30-40 minutes are devoted to
the usability rating process.
Many certified applications have received a 4-star or 5-star rating which, on the
surface, would indicate that their products are very usable. However, it is not a
true indication of outstanding usability.
23
Usability is determined by users
using the system
One issue in the CCHIT rating system is that CCHIT asks vendors to demonstrate
certain scenarios to an evaluator, who scores the product’s usability based on
the demo. User-centered design, however, is all about testing with actual users.
CCHIT jurors may have the correct clinical and software background, but they
are not necessarily the intended users of the system.
User-centered design is also about hands-on testing with users. In the case of
CCHIT, jurors are not actually using it in a hands-on fashion because they are
observing a brief pre-created demo. An analogy we’ve heard is that a reviewer
from Consumer Reports would never rate the driving capabilities of a car by
watching the car manufacturer’s commercial!
Is it possible to calculate a
true usability score?
It would certainly be a worthwhile endeavor to look at quantifying usability to
arrive at repeatable measures for comparison across products, but we believe it
is unlikely that this would result in a single number or “score”. In healthcare software in particular, there are often many different roles and different workflows
involved, and assigning a single number could be very misleading.
Our approach would be to create an in-depth task or workflow-based assessment, however this could involve a lot of effort to determine the parameters and
whether to weight primary uses, secondary uses, and therefore may be of more
theoretical than practical value in the industry.
Several attempts have been made to develop a method for reporting a single usability score. Jeff Sauro has written a paper on the topic, and Tom Tullis and Bill
Albert talk about single usability score metrics in Measuring the User Experience.
Don’t be fooled by electronic medical records sales and marketing literature - a
5-star CCHIT usability rating is not a reliable indication of an outstanding user
experience. In fact, even CCHIT does not intend for the rating to be interpreted
this way. CCHIT is clear that a 5-star rating means that jurors’ perception is the
“key features are discoverable and usable”.
Although determining a true usability score is a very worthwhile goal, we
encourage healthcare software developers to focus primarily on its users. There
is nothing more powerful than user feedback to drive the success of a product
ahead of its competitors. The user-centered design processes outlined in this
paper will help steer any organization towards outstanding software designs that
will delight its target users.
24
9
X
Mistake # 9
1
Waiting Until The Last Minute
To Prepare for Meaningful
Use Stage 2
In June 2011, we were invited to attend the second annual
NIST forum for EHR Usability called “A CommunityBuilding Workshop: Measuring, Evaluating and Improving
the Usability of Electronic Health Records.” NIST, in
collaboration with the ONC, unveiled its initial discussion
points for what it might consider as the “Usability Criteria”
in the upcoming Meaningful Use Stage 2 regulations. We
learned that the government believes that while usability can
be key in increasing product effectiveness, speed, enjoyment,
etc., NIST is going to focus on EHR usability for the
improvement of patient safety.
While the specifics are still forthcoming, vendors have a
window of opportunity today to get ahead of NIST—and
ahead of competitors—by proactively addressing meaningful
use in advance of the early 2013 deadline. Let’s look at what
vendors can do, combining the information NIST has given
so far with fundamental usability best practices.
25
Step 1: Set Usability Goals related to
Patient Safety
These are specific, measurable goals such as “Our EHR must provide a 99%
error-free rate of medication entry”. NIST has given the following examples of
use error categories, each of which might be driving 1 or more goals.
1. patient ID errors
2. mode errors [e.g., dose related]
3. data accuracy errors
4. visibility errors [e.g., tapered dose 80-20mg - 80 shows vs. 20]
5. consistency errors [ e.g., pounds vs. kilos ]
6. recall errors [e.g., 1 time dose]
7. feedback errors [1 tablet vs. 1/4 tablet]
8. data integrity errors [ next vs. finish to enter injection just administered]
Step 2: Identify Target (and Real) Users
The Stage 2 meaningful use criteria will require EMRs to demonstrate that they
have tested their products’ usability with representative users. For hospitals,
representative users will include nurses, and other allied health professionals
in addition to physicians. For smaller clinics, the primary users may just be
physicians, and nurses. Identify and categorize the key users of your product,
then create User Personas to describe and the goals and motivations of each
user group.
Step 3: Identify Critical Tasks
Isolate and describe the primary tasks that users must perform with your product. NIST has said that it will provide example lists of tasks and scenarios tied to
MU criteria, but in general these should be the high frequency and high criticality
tasks of your system. For example:
• Hospital critical task flow: nursing notes and common tasks such as administering medication and taking vitals
• Clinical critical task flow: the sequence of tasks centered around prescription
and drug dosage
Step 4: Benchmark & Track
Focusing on the identified critical tasks, an initial usability test plan should be
written and executed to benchmark the product’s usability status. NIST has
said the testing should be performed with representative users performing the
defined representative tasks, the moderator must have UX/Human Factors and
clinical domain experience, and the tests must be performed in a simulation of
real use environments [e.g., surgery, clinic, nurses desk, etc.]
26
Testing will give your organization quantitative data to measure and track
specific usability metrics, and the ability to formulate very concrete goals.
Step 5: Build An Action Plan for Meeting
Usability Goals
Once your measurable goals and related critical tasks are identified, and
benchmarks are established through initial testing, it’s time to put a plan
together for incorporating user-centered design into your product development
process in order to measurably improve usability for patient safety.
The plan should establish the required investments and timelines to meet the
goals, as well as what is required to achieve them, including related organization
adjustments, such as bringing in qualified HCI personnel.
With this simple plan in place, vendors can already take action to get buy-in
with their internal stakeholders and get their teams systematically identifying
and improving weak points related to patient safety, in preparation for the final
criteria to be announced later in the year.
27
Technology consultation with Macadamian
Macadamian is a global UI design and software innovation studio with significant
sector expertise in healthcare and life sciences. We work with Healthcare IT
vendors to create visually impressive, usable and ultimately successful software
products. Through our usability, design, and software engineering services,
Macadamian can help you transform your idea into a market-ready feature or
product that will stand out from your competition and delight end-users.
We invite you to consult with one of our healthcare usability or technology
experts, to determine what disruptive trends your team could harness to stand
out from the crowd, and how to make them innovative and usable in order to
attract attention from physicians and impress your clinical users.
At Macadamian, our experience and proven ability to work seamlessly with large
Healthcare IT vendors make us the ultimate partner to support your clinical
software development project.
Contact Us
For questions or comments about this white paper, or for more information on a
technology or usability consultation, please contact:
Didier Thizy
Director, Healthcare Solutions
didier@macadamian.com
+ 1 877-779-6336 x136
www.macadamian.com
28