Part 2: Analyzing Usability: How to do it

Welcome back! In my last post, I gave a bit of background about how and why you can do a Usability Audit, based in Nielsen and Molich’s Heuristic Analysis. In this post, you’ll get to watch as I do my own. Let’s get started.

First: Get in touch with your inner crank

Large GIF (640x320).gif

The beauty of the Usability Audit is that you don’t have to be nice! I like to say that I connect with my inner Andy Rooney in doing this - grinding my axe with things that are particularly annoying, wasteful, or inefficient. And yes, it may seem silly to be purposefully indignant, but I think this is a good example of righteous indignation.

Next! Take Screenshots!

I’ll walk you through my process for performing the Usability Audit, and as you’ll see, I’ll share my screenshots. Taking screenshots is the first step I take in approaching a re-design project. I want to capture the interface as it is, so that I can illustrate what I’ve encountered, and then integrate it into my wireframes and flows.

Now, let’s pay my garbage bill

Screen Shot 2017-07-19 at 4.37.19 PM.png

When I ask workshop participants to name a web site or app that frustrates them, often they’ll name one that they have to use in order to pay a bill: sites of utility companies, insurance providers, or financial services institutions. Paying bills is rarely fun - add frustration and difficulty to the mix, and it’s the worst.

My most-frustrating bill to pay is literally garbage: the bill I receive each month from Waste Management. Like 41% of Americans (and 70% of adults under age 35), I’m a renter. Yet, as is the company’s policy, they won’t set up accounts for rental tenants. Thus, this bill doesn’t even have my name on it, but that of my landlady.

So, I have to pay as a guest. As e-commerce usability research shows, forcing users to set up accounts in order to pay increases what is called the “interaction cost” - the cognitive and emotional energy, as well as time, spent by the user.

The paper bill that I get in the mail is covered in reminders that I can pay my bill online. While I have most of my bills set for autopay, this is one that I can’t set up - in fact, when I go to, I can’t set one up, due to the aforementioned circumstance. This is a task I do every month, but the entire experience makes me feel like I’m doing it all wrong.

Screen Shot 2017-07-19 at 4.38.24 PM.png

Likewise, when I go to, the site directs me to set up service, something I’ve already done. This is a relatively recent phenomenon in UX - an emphasis on the “pre-sale”, or onboarding the new customer. To me, this is relatively short-sighted - emphasizing the new customer while leaving the returning user to fend for themselves.

Always a visitor:

Screen Shot 2017-07-19 at 4.37.52 PM (1).png

Thus, when I pay my bill, I have to opt for the one-time payment. And objectively, there’s nothing wrong with this: there are lots of reasons for users to opt not to register. The problem is, accommodating them.

As many have argued, text is a powerful part of the user experience - coercive copy like this contributes to what I’ll call a dark pattern.

Screen Shot 2017-07-19 at 4.38.35 PM.png

When I do find my way to the “Make a One-Time Payment”, I am offered a little bit of useful assistance - the form shows me where to find my Customer ID on the paper bill.

Meanwhile, the UI elements are rather inconsistent. At the bottom of this page, I see this graphic and text for the first time. What does this mean? It doesn’t make me feel much more secure, just confused.

Screen Shot 2017-07-19 at 4.39.38 PM (1).png

Similarly, after I’ve entered my payment information, I see this completion meter for the first time.

Screen Shot 2017-07-19 at 4.41.02 PM.png

Normally, I think completion meters are pretty helpful in showing users where they are in a task, but this is confusing! I’m on step 3, but where were steps 1 and 2? And again, I’m reminded of my “one time” payment status. If I had the option, I would have definitely put this bill on Autopay!

Screen Shot 2017-07-19 at 4.41.13 PM.png

Finally, I have confirmation, and look, I’m on step 4 in the new completion meter.

My Findings:

Screen Shot 2018-05-01 at 4.39.16 PM.png


  1. Visible System Status: 😕

In this category, the findings are mixed. Due to the fact that I can’t create an account, there’s no way to check on account status outside of this process.

Likewise, the completion meters I see here give me some status indicators, but as they say “too little, too late”.

2. Match the Real World: 🤑

In this regard, the experience is… okay. Although the name-on-the-account issue is a fail in this regard, the graphic above showing the Customer ID on the bill is actually helpful in figuring out what my Customer ID is and what numbers to type in.

Likewise, this graphic is helpful (if a bit excessive) in helping one locate the name on one’s credit or debit card.

Screen Shot 2017-07-19 at 4.40.00 PM.png


Overall, it’s a little irksome to see that there is attention to these details - but only when it comes to payment information.

3. User Control and Freedom:🤦

Nielsen notes that users need a “clearly marked ‘emergency exit’”, and that design should support “undo and redo”. Facepalm!

4. Consistency and Standards: 🤷

I’ll give this a Shruggie. Like I pointed out earlier, the completion meters and rather random security icons don’t give me much in this regard.

5. Error Prevention: 🙄

Eyeroll. Because I’m used to this, and have found my own workarounds, I’m not the most reliable source in this regard. I’ve definitely made a lot of errors in trying to get this task done, and I’d hypothesize that new users would as well.

6. Recognition > Recall : 🚷

This is the principle of designing systems that do not rely on end users having to remember   things and make connections on their own. As we see, I’ve had to do both of those things.

7. Flexibility and Efficiency of Use: 🙅‍

Nope, I don’t see this here. I’ve had to create my own workarounds, and have spent considerable time doing so.

8. Minimalist Design:🙌

We've got to give them some credit for this. There aren’t a lot of stray design elements, thankfully!

9. Help Diagnose and Recover from Errors: 😬

Before I reached this point of frustration, I attempted to register for an account to pay online several times, called customer service, and spent a long time on the phone, only to be told. “Nope, sorry!” Grimace.

10. Help and Docs: 🤕

I have consulted the Help and Docs features on the site, and lo and behold, these weren’t particularly useful. Search “renter” or “tenant” in the knowledge base, and you’ll come up empty handed. Ouch.

Conclusion: Get Mad, then Make it Better

Thanks for following along - now it’s your turn!


Part 1: What is Usability? Introducing Heuristic Analysis

What is Usability? Introducing the Usability Audit

Hi y’all! My name is Amelia Abreu, and I’m a design researcher and the founder of UX Night School. I’m so excited to get started with UX Night School’s Tutorial series for Balsamiq! We’ll be working alongside UX Night School’s first online course, Intro to UX Design/Research, and I’m so excited to have you join us.

“What is User Experience, exactly?”

“What do you mean by Design Research?”

I get these questions a lot. After all, naming things is one of the hardest problems in computing, and the concepts that inform UX and design research are complex! These pithy terms don’t really do a great job of encompassing them.

But this brings up a bigger issue - that human-centered design is best introduced in practice. Learning the concepts is one thing, but applying them is much harder, and much more important.

Introducing: Heuristic Analysis (or a Usability Audit)

In the spring of 1990, two Computer Science professors from Denmark named Jakob Nielsen and Rolf Molich presented a paper at the CHI Conference. That is, the Academy of Computing Machinery’s SIG (Special Interest Group) on Computer Human Interaction (often referred to as “Chi”, and pronounced like the greek letter). In it they proposed a heuristic analysis technique for User Interfaces. In short, a technique that they proposed as a credible one for evaluating the usability of interfaces in computing, and ostensibly finding areas to improve and better understand.

What exactly does that mean? They called this a heuristic analysis, they argued, because it could be performed “by simply looking at the interface and passing judgement according to one’s own opinion.” While most of us can see how this might be possible, keep in mind that this initial audience was a group of industry and academic researchers who tend to downplay individual human insight in favor of data collected by more mechanical means. Yet, as humans, we have a sort of capacity that machines don’t - to enact scenarios in context, to have differing ideas, and, most importantly, to reason and feel on a level that machines cannot.

Many of us have worked in jobs alongside folks who put a lot of emphasis on quantitative data, like indicators and metrics, performance data, but who also routinely make decisions based on inconsistent data or “their gut”. Design researchers like to argue that in representing experiences and data visually and through a variety of approaches, we’re better able to have accountability for decisions in product design.

In the following UX Night School posts for Balsamiq, we’ll talk about ways to do this that involve testing and validating design decisions with the end-users of a product. But, in this tutorial, we’ll cover how to conduct a heuristic analysis, or what I’ll less pedantically call a “Usability Audit”, all by yourself. Who would think that something written about the design of digital products in 1990 could still be useful and precise in 2018? This exercise remains more effective and efficient than ever.

You don’t need your boss’s permission (though I always think you should tell your boss when you are learning new things!), a budget, a lab, or any specific expertise. Just an hour and a product (or application, or website) that you’d like to improve.

Is this a bulletproof way to make an application or product completely useable? Nope. But it is a way to get started, which is often the hardest part in improving the user experience of an existing product. You could conduct the usability audit, then present it to your team as a way to propose a more in-depth project, like a series of Usability Tests or User Interviews. If you do client work as a designer or developer, you can perform the audit and offer this as a way to get conversations started.

Pro Tip: Don’t call it a teardown

There is one thing I’d ask you not to do: call this a “teardown”.

I hate the term “teardown”, and I’m made uncomfortable by its use in design practice. I do not enjoy tearing others down, and even the clunkiest user experience was built by other humans, who were most likely doing the best that they could. There is no one way to easily and quickly improve the user experience of a product, and I am suspicious of all who tell me otherwise (and that they alone are the ones to do it.)  Rather, as the rather staggering amount of data from both industry and academia shows - taking a human-centered approach to design and development and bringing people into the process yields the highest rate of success in improving usability, and in yielding a return on investment.

The way we adapt and maintain structures and technologies in everyday life is as interesting and valuable as the way we build them — this is core to my beliefs as a designer, researcher, and observer of technology and humanity.  Let’s do this in a way that is kind to everyone involved.

What I like most about doing a Usability Audit is that regardless of your background, you become the usability expert, with help from this list of criteria, or as they call them, heuristics.

Nielsen’s criteria give you a vocabulary and a place to start. From here, you can identify some next steps for improving usability.

While originally intended to be an exercise done individually with group analysis following, I’ve found that doing this in pairs is often very helpful. Much like pair programming, doing this audit with another person makes you have to think out loud, testing your assumptions.

To get started you’ll need a few things. First, as I mentioned before, about an hour of your time, and a website, app, or other product that you’d personally like to improve the usability of. It can be your company’s website, your bank’s smartphone app, or anything else you’ve used, thought about, or been annoyed by.

Finally, you’ll want to take a look at the criteria itself - here is a PDF download we designed at UX Night School, along with the original paper and Nielsen’s write-up on the Nielsen Norman Group site.

And last, if you aren’t already using Balsamiq on Google Drive, take a look at the documentation and get set up.

Next: I’ll walk you through the process.