How to conduct heuristic inspections for evaluating software usability

Posted on January 1, 2013 by Kevin Matz

Heuristics are “rule-of-thumb” design principles, rules, and characteristics that are stated in broad terms and are often difficult to specify precisely. Assessing whether a product exhibits the qualities embodied in a heuristic is thus a subjective affair.

If you inspect a prototype or product and systematically check whether it adheres to a set of heuristics, you are conducting what is called a heuristic inspection or heuristic evaluation. It is a simple, effective, and inexpensive means of identifying problems and defects and is an excellent first technique to use before moving on to more costly and involved methods such as user observation sessions.

It is usually best when a heuristic evaluation is carried out by an experienced usability specialist, but heuristic evaluations can also be very effectively when they are conducted by a team of individuals with diverse backgrounds (for example, domain experts, developers, and users).

To conduct a heuristic evaluation, you should choose several scenarios for various tasks that a user would perform. As you act out each of the steps of the task flows in the scenarios, consult the list of heuristics, and judge whether the interface conforms to each heuristic (if it is applicable).

Jakob Nielsen introduced the idea of heuristic evaluations, and his 1994 list of ten heuristics, reproduced below, is still the most commonly used set of heuristics today (Nielsen, 1994, p. 30):

Visibility of system status “The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.”
Match between system and the real world “The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.”
User control and freedom “Users often choose system functions by mistake and will need a clearly marked ‘emergency exit’ to leave the unwanted state without having to go through an extended dialog. Supports undo and redo.”
Consistency and standards “Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.”
Error prevention “Even better than a good error message is a careful design that prevents a problem from occurring in the first place.”
Recognition rather than recall “Make objects, actions, and options visible. The user should not have to remember information from one part of the dialog to another. Instructions or use of the system should be visible or easily retrievable whenever appropriate.”
Flexibility and efficiency of use “Accelerators — unseen by the novice user — may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.”
Aesthetic and minimalist design “Dialogs should not contain information that is irrelevant or rarely needed. Every extra unit of information in a dialog competes with the relevant units of information and diminishes their relative visibility.”
Help users recognize, diagnose, and recover from errors “Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.”
Help and documentation “Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.”

An obvious weakness of the heuristic inspection technique is that the inspectors are usually not the actual users. Biases, pre-existing knowledge, incorrect assumptions about how users go about tasks, and the skill or lack of skill of the inspectors are all factors that can skew the results of a heuristic inspection.

Heuristic inspections can also be combined with standards inspections or checklist inspections, where you inspect the interface and verify that it conforms to documents such as style guides, platform standards guides, or specific checklists devised by your project team. This can help ensure conformity and consistency throughout your application.

Posted in Interaction Design, Psychology for UX Design, Uncategorized, Usability, Usability Testing, User-Centered Design | Leave a comment

Focus groups as a usability evaluation technique

Posted on September 11, 2012 by Kevin Matz

A focus group brings together a group of users or other stakeholders to participate in a discussion of preprepared questions, led by a facilitator. A focus group could be used as an usability evaluation technique if the group is shown a demonstration of a product or prototype, and then the group’s impressions and opinions are discussed.

Focus groups might appear to be a convenient, time-saving way to get feedback from eight or ten people in a single session. In practice, however, the technique is not particularly reliable. Watching a demonstration is not the same as having the opportunity to interact with the product hands-on. And group dynamics can vary widely; different groups can come up with completely different conclusions.

Focus group discussions often tend to be dominated by one or two loud and opinionated participants, and the quieter participants often say little and go along with the group consensus. There is also the risk that the facilitator may consciously or unconsciously lead the discussion towards a particular outcome. If you choose to use focus groups, you should use them with caution and be aware of the limitations.

Posted in Product Management, Usability, Usability Testing | Leave a comment

Analytics as a usability evaluation technique

Posted on September 11, 2012 by Kevin Matz

Once your product has been released, understanding how it is actually being used very valuable. Analytics refers to the use of instrumentation to record data on users’ activities, followed by the analysis the collected data to detect trends and patterns. This data can then validate your assumptions as to which functions are being used most frequently and which parts of the product are seldom or never used, and you may be able to identify where users are running into trouble.

Some examples of the type of data that you can collect through analytics include:

  • Pages or screens visited, and time spent on each
  • Functions used, buttons and controls pressed, menu options selected, shortcut keystrokes pressed, etc.
  • Errors and failures
  • Duration of usage sessions

Websites and web apps are well suited to logging and tracking user activities. Many web analytics packages and services can provide additional contextual data such as the user’s geographic location, whether they have visited the site before, and what search terms were used to find the site if the user visited via a search engine.

Desktop and mobile apps can also collect usage data, but because of privacy concerns and regulations, it is important to declare to the user what data you intend to collect, and you must gain the user’s permission before transmitting any usage data.

No matter what type of product you offer, privacy concerns are important and you must ensure that your practices and Terms of Service follow the legal regulations appropriate for your jurisdiction. Tracking abstract usage data such as button presses are generally acceptable, but it is usually considered unacceptable to pry into content the user creates with the product.

Posted in Interaction Design, Usability Testing, User Experience Design | Leave a comment

Quantifying cognitive load and task efficiency

Posted on August 22, 2012 by Kevin Matz

If we wanted to attempt to quantify the cognitive load — i.e., the thinking and effort involved — for performing a particular task, we could write out a list of the actions or operations that a user would have to do to carry out that task under normal circumstances. We could then estimate or assign a score, representing our idea of the effort involved, to each individual action, and then sum up all of the scores to get a total effort score for the task.

The KLM-GOMS model, the Keystroke-Level Model for the Goals, Operators, Methods, and Selection Rules analysis approach (Card, Moran, and Newell, 1983), is one analysis technique based on this idea, but instead of assigning scores representing effort, an estimate of the time required to do each action is estimated instead. The amount of time it takes to complete a task is a good proxy for physical effort, although it does not accurately measure the intensity of mental effort.

Let’s take a very condensed tour of the KLM-GOMS approach.

To accomplish a goal, the user will break the work into tasks, and for each task unit, the user will take a moment to construct a mental representation and choose a strategy or method for carrying out the task. This preparation time is called the “task acquisition” time, and can be very short — perhaps 1 to 3 seconds — for routine tasks, or much longer, perhaps even extending into several minutes, for creative design and composition tasks.

After the task acquisition, the user carries out the task by means of a sequence of actions or operations. The total time taken to carry out the actions is called the “task execution” time. Thus the total time required to complete a task is the sum of the task acquisition and task execution times.

To estimate the task execution time, KLM-GOMS defines basic operations (we assume here that we are dealing with a keyboard-and-mouse system):

Operation Description Suggested average values
K Keystroking Pressing a key or mouse button, including the Shift key and other modifier keys Best typist: 0.08 sec
Good typist: 0.12 sec
Average typist: 0.20 sec
Worst typist: 1.20 sec
P Pointing Moving the mouse pointer to a target on the screen 1.1 sec
H Homing Moving a hand from the keyboard to the mouse or vice-versa 0.40 sec
M Mental operation Preparation 1.35 sec
R System response operation Time taken for the system to respond varies

So to use the mouse to click on a button, we would have a sequence of operations encoded as “HPK”: homing, to move the hand to the mouse; pointing, to point the mouse cursor to the button; and a keystroke, representing the pressing of the mouse button.

In addition to these operators, the KLM-GOMS model also includes a set of heuristic rules governing how the “M” operation, the mental operation, is to be inserted into an encoded sequence. For instance, “M” operations should be placed before most “K” and “P” operations, except for various special cases. So the “HPK” sequence discussed above would become “HMPK”. The rules are fairly arcane and we won’t go into the details here.

As an example, let’s consider the task of finding instances of a search term in a document in a text editor. One possible sequence of actions to accomplish this might be:

  • Click on the “Search” menu
  • Click on the “Find text” item
  • Enter “puppy” as the search term in dialog
  • Click on the “OK” button

This can be encoded using KLM-GOMS and used to formulate an estimate of the average time required as follows:

Action/Operation Encoding Time (s)
Task acquisition (none) 1.5
Click on the “Search” menu H[mouse] 0.40
MP[“Search” menu] 1.35 + 1.1
K[“Search” menu] 0.20
Click on the “Find Text” item MP[“Find Text” item] 1.35 + 1.1
K[“Find Text” item] 0.20
H[keyboard] 0.40
Enter “puppy” as the search term 5K[p u p p y] 5(0.20)
Click on the “OK” button H[mouse] 0.40
MP[OK button] 1.35 + 1.1
K[OK button] 0.20
Total 11.65 s

Of course, we would expect a more skilled user to be able to accomplish the same task in substantially less time by using shortcut keystrokes rather than the mouse, and by typing faster than an “average” user.

There are obviously limitations to this kind of analysis; it provides a general rough estimate only, and it assumes that users know the right sequences of actions to complete a task. It also does not account for errors and mistakes. But when you are designing an interface and considering how to design an interaction, methods such as the KLM-GOMS model give you a way to compare the efficiency of different alternatives, and all other things being equal, the alternative that can be done in the least amount of time is the most convenient to the user, and may involve the least cognitive load.

Posted in Interaction Design, Psychology for UX Design, Usability, User Experience Design | 1 Comment

The impact of hardware devices on software ergonomics

Posted on August 19, 2012 by Kevin Matz

A product that is ergonomic is designed in a way that helps reduces physical discomfort, stress, strain, fatigue, and potential injury during operation. While ergonomics is usually associated with physical products, the design of the a software application’s interface also influences the way the user physically interacts with the hardware device on which the application runs. And ergonomics also extends to the cognitive realm, as we seek to design software that helps people work more productively and comfortably, by reducing the dependence on memorization, for example.

To create an ergonomically sound software application, it is important to first think about the properties and the context of use of the hardware device on which the application will run. For the majority of consumer and business applications, there are currently three main forms of general-purpose personal computing devices:

  • Desktop and laptop computers with a screen, keyboard, and a pointing device such as a mouse or trackpad, are comfortable for users sitting at a desk for a long period of time.
  • Tablet devices with touchscreens have a form factor that is comfortable for sitting and consuming content (reading webpages, watching movies, etc.), but entering information and creating content via touch-screen control is generally not as comfortable and convenient as with a desktop machine.
  • Mobile phones (and similar devices such as portable music players) are usually used for short bursts of activity while on the go.

For more specialized applications, you might have a combination of software and custom-designed, special-purpose hardware. Examples include a machine that sells subway tickets, an automated teller machine, or an industrial thermostat control. If you are a designer for such a product, you may have responsibility for designing the form of the physical interface in addition to the software.

To give you an idea of some of the practical ergonomic aspects that you should keep in mind when designing for different devices, let’s compare desktop computers with touchscreen tablets:

  • Tablet devices with multi-touch touchscreens are pleasant and fun to use from an interaction standpoint because you can interact directly with on-screen elements by touching them with your finger. Desktop machines (as of this writing) generally don’t offer touchscreens, as reaching your arm out to the monitor places strain on the arm and shoulder muscles and would quickly become physically tiring. Desktop setups thus rely on pointing devices such the mouse or trackpads. These pointing devices introduce a level of indirection, however: moving the pointing device moves a cursor on the screen.
  • On desktop systems, there is a pointing device cursor (mouse arrow), whereas touchscreen devices have no such cursor. Some mouse gestures, like hovering the cursor over a control, thus have no counterpart in touchscreen systems. On both desktop and touchscreen systems, however, a text cursor (caret) appears when a text field receives the focus.
  • While a mouse may have multiple buttons, and clicks can be combined with holding down modifier keys (Control/Alt/Command/Shift), touchscreens don’t offer as many options. When you drag your finger across the screen, is it to be interpreted as a scrolling gesture, or an attempt to drag and drop an object on the screen? Cut-and-paste and right-clicking to get a context menu are easy on a desktop machine, but on a tablet, such operations require double-touch or touch-and-hold gestures that are not always evident.
  • Fingers range in size substantially; young children have small, narrow fingertips, whereas some men have very thick, fat fingers. Touchscreen buttons and icons thus must be large enough to accommodate “blunt” presses without triggering other nearby controls. In contrast, the mouse arrow allows pixel-precise pointing, and so buttons and icons can be substantially smaller on desktop applications than on touchscreen devices.
  • When the user is touching something on the screen, the user’s finger and hand will obscure part of the screen, so you have to be careful about what you display and where, so that important information is not hidden. When pressing an on-screen button, the user’s fingertip will obscure the button being pressed. Because button presses don’t always “register”, users seek visual feedback to see that the button press worked, and so you either need to make the buttons large enough so that the animation of the button being depressed is visible, or you should give some other clue when the user retracts the finger to show that the button was pressed (maybe pressing a Next button makes the application navigate to the next screen, which is very clear feedback that the button press was successful). Auditory feedback, like a clicking sound, can also be useful as a cue that the button was pressed successfully.
  • Mobile devices and tablet devices are often held by the user in one hand while standing, and so the user has only the other hand free to operate the touchscreen.

When designing a product, understanding the constraints and limitations, as well as the opportunities, of the hardware devices the software will run on will help you design appropriate and comfortable interactions.

 

Posted in Interaction Design, Product Management, Psychology for UX Design, Uncategorized, Usability, User Experience Design | 1 Comment

Software requirements in a nutshell

Posted on July 30, 2012 by Kevin Matz

Requirements are statements of things that your product must achieve for it to be considered successful. If you are building a customized solution for a client, requirements express the wants and needs of your client. If you are building a product for sale to a wider market, requirements express the aggregate wants and needs of potential customers that will be necessary for the product to be sell enough copies to be economically successful.

  • Functional requirements are the features your product will have, in terms of the functions, actions, and behavior it will support. For example, some functional requirements for a word processor would be that it support bold and italic type, that it allow documents to be printed, and that it allows images to be embedded in documents.
  • Non-functional requirements are performance or quality constraints that are general or “cross-cutting” in nature. Non-functional requirements address areas such as performance, security, stability, capacity, and scalability. For example, an e-commerce website might be required to serve 100,000 users concurrently and provide a response time of 2.0 seconds or less.

The scope of your product and project is defined by the set of requirements that need to be implemented. Without careful management, additional wishes and demands will continually be added to the project’s scope. This phenomenon, known as scope creep, will threaten your ability to deliver on schedule.

Requirements should be stated in concrete, measurable terms whenever possible. For example, rather than stating that “the product must be easy to learn”, which is subjective and unprovable, you might consider a phrasing that lends itself to objective testing, such as “95 percent of users will be able to successfully process a standard passport application after the two-day training session”.

Some requirements are “definitional”: they result from, and form part of, your definition of what the purpose and market positioning of your product is. Other requirements need to be elicited from your users and other stakeholders.

Posted in Product Management, Product Marketing, Project Management, Requirements Engineering | Leave a comment