Settle design debates with data

Make confident decisions
by testing designs with
real users.

Get started now
UsabilityHub

Five-Second Tips: Good Test Instructions #1 – Beware of the context statement

Many of the tests I’ve seen at UsabiltyHub ask participants to put themselves in a specific context for answering questions:

“Imagine that you . . .”
“Pretend that you . . .”

Context statements like these can add realistic “color” to the instructions and ground participants in a situation of relative familiarity. However, they can also confuse or introduce negative bias in your participants before the test has even begun.

Consider the instructions for a test about the design of a poster advertisement:

“Imagine that you are standing on the street and a bus drives past. You see the following advertisement on the back.”

Most (if not all) people can relate to seeing advertisements in real-world contexts, so including a context statement certainly does no harm, and may actually assist in putting the respondent in a better frame of mind for answering test questions.

However, using a context statement can be counter-productive when it represents an unnecessary or unrealistic context:

An unnecessary context adds no value to, or practical assistance with, the participant’s ability to answer test questions. Statements like “Imagine that you are looking at the following web page while browsing” or “Imagine that you are searching for information about mobile plans and you come across this site” will likely do little to help produce useful answers about the design you’re testing, especially when you’re looking for responses about a specific aspect of your design.

An unrealistic context tries to place a participant into a situation to which (s)he cannot relate. Consider this example: “Imagine that you work for a bank and you’re researching software vendors.”  Unless you’re sure that your test sample consists solely of bank employees and/or software purchasers, this type of context statement can put your test at risk for indifference (“I’ll just skip this test”) or hostility (“How can you expect me to answer questions about this?”).

Additionally, context statements rarely work well as instructions in and of themselves. Supplementing a context statement with an indication of test format can help alleviate any potential negative impact of an unrealistic context. “Imagine that you’re researching software vendors for your bank” becomes more accessible (and less easily dismissed) when combined with “You’ll be asked for your opinion about the website’s color scheme.”

Written by

A graduate of Bentley University’s HFID master’s program, Paul Doncaster has spent his career working on highly-complex UX projects within the domains of course technology, legal and intellectual property.

He has written and spoken on many UX topics, including designing for emotional response, online readability, designing for tablet users in the legal domain -- and of course, the Five-Second Test.

LinkedIn: www.linkedin.com/in/pauldoncaster/
Facebook: www.facebook.com/UX5SecondRules
Twitter: @UX5SecondRules

The UX Five Second Rules is available at the Elsevier Publishing Online Store or online at amazon.com.