Five-Second Testing: Taking A Closer Look At First Impressions (Case Study)

Original Source: https://smashingmagazine.com/2023/12/five-second-testing-case-study/

In today’s world of shortening attention spans and omnipresent hustle, wasting even a second could mean losing the chance to earn more time from a person you want to impress. If your interests lie in creating good user experiences, there is a fair chance you have heard of five-second testing.

Five-second testing is an established technique of usability research used by UX researchers, designers, product managers, and in a variety of other professions, such as marketing or business analysis.

In short, you show a picture of whatever you are designing (site, app, pair of socks) to a member of your intended audience for exactly five seconds. Then, you hide the picture and ask the participant a couple of questions. The goal is to learn whether the reaction — the participants’s first impression — is what you wanted to see. Did you get the main message across? Do people remember the company’s name? Sounds like an efficient way to test your product without needing to turn to full-on usability testing, right?

Note: The word “participant” in this article is used to refer to users involved in five-second testing or related usability research methods. The word “user” is used in more general contexts since users form first impressions all the time, not just when you are testing it.

Why is it five seconds exactly, though? Are five seconds some magical moment when everything we see should become clear? And if it does not, does it automatically mean that a user experience is bad? Or are five seconds just the right amount of time for first impressions to brew in the user’s mind so that they’re neither undercooked nor overcooked?

These are some of the questions that we asked ourselves. Not satisfied with the answers written by others who covered the topic before us, we kept drilling and conducted an actual peer-reviewed scientific case study, exploring the hidden truths behind the testing of first impressions. The research paper examines the five-second test and discusses the results.

So strap in and read what science has to say about five-second testing. And then, what the implications are for you so that you can take practical advantage of this new knowledge to develop better first impressions of your services or products. But first, let us delve into what we know about five-second testing and its caveats so that you see the greater picture of the focal points of our investigation.

The Mythos Of Five Seconds And Its Gaps

You may be familiar with the well-known statistic that a website has about ten seconds to communicate its key message to the user. Knowing that waiting only five seconds to ask testing participants about their first impressions may suddenly seem like an odd choice. If indeed visitors of a website have about ten seconds to grasp a message, are five seconds really enough time for users? There is an alleged justification, as we explain below.

Cutting a bit forward, though, the factual basis for it is admittedly a bit of a Wild West if you look for hard data to support it. There is an almost uncanny resemblance to another not wholly scientific five-second rule that says it’s okay to eat food off the ground if it’s within five seconds from when it dropped there.

The five-second testing method has its origins as a simplification of usability testing. The first references to five-second testing point to Christine Perfetti, who coined the term for the method in the mid-2000s. The answer to “Why five seconds exactly?” comes largely from anecdotal evidence in the form of the experience of usability researchers.

The common story is that if something is shown to participants for more than five seconds, their first impressions will start to deviate from the actual user’s genuine initial impressions. The participant’s perspective becomes more analytical and less task-driven. The five-second test lets you avoid overtly speculative feedback that nobody would give you under normal circumstances.

Fair enough, that could potentially be true. But five seconds is still quite a short period of time. Consider how different people can be when it comes to their cognitive abilities (and there is nothing wrong with that). For example, one user’s sharp perception may let them realistically form first impressions in five seconds or faster, but another user may barely have the time to blink, much less absorb any meaningful information–they need a moment to take it in at their own tempo.

The reasoning starts to fall apart a bit more at its seams once you also consider the visual complexity of the stimulus (a.k.a., the picture you show to the participants). The nature of the things you may want to test can range from very simple to very complex. If the stimulus is simple, it is possible to take even less than five seconds for participants to form their initial impressions. Would this mean that there would be the risk of them using the remaining time to get over-exceedingly analytical?

Conversely, there is the question of whether five seconds is enough time to let participants realistically visually scan a more complex stimulus. I can already hear staunch proponents of five-second testing saying that this last discrepancy is actually rightfully intentional. It’s a feature, not a bug, if you will.

After all, if a stimulus is too complex, that is exactly why you conduct five-second testing. It allows us to find out about things like complexity. It can help you find out if participants cannot extract the key information you want to communicate so that you can fix it.

However, we need to consider that not all user interfaces are the landing pages of websites. They serve to support different user tasks, some of which cannot avoid having a certain degree of complexity.

Five-second testing guides typically avoid directly addressing testing of these types of user interfaces by saying that the method has the following limitation: it should not be used to test user interfaces with multiple purposes. If the same stimulus serves for more than one task, it is alleged that you should probably conduct full-fledged usability testing, which is technically correct (the best kind of correct).

Giving up on the five-second testing in the inherently more complex user interfaces, however, also gives up on its advantages for measuring and optimizing first impressions. For instance, the idea that a screenshot or a mockup is all you need to quickly find usability problems and iterate your designs. This is where five-second testing really shines.

Usability testing does not tell you accurately what the actual first impressions are without considerably interrupting the participant. And even then, you would encounter the same problem: At what moment from when the participant is exposed to a design should their first impressions be gauged so that they are genuine?

As we have discussed so far, there are certainly a fair number of question marks surrounding five-second testing. The method still undeniably has a number of merits, as proven by our experience at UXtweak, where we also provide our own Five Second Test tool. A lack of proper research on the topic is what drove us at UXtweak Research to conduct our very own case study.

The Science, Abridged

Essentially, what we sought to investigate in our case study are the relations of a number of key factors that are absolutely crucial for five-second testing:

What are the cognitive abilities of the participant engaging in the five-second test?
How visually complex is the stimulus shown to the participant?
For how long is the stimulus shown to the participant?
What kinds of questions do we ask the participants afterward?
What is the feedback that participants give you?

As you may have noticed, time — that iconic yet controversial five-second threshold — is considered a variable factor. In our experiment, we investigate the differences in feedback between three separate groups of participants who are shown pictures for either five (5) or alternatively two (2) or (10) seconds (so a bit less and a bit more time, respectively). This means that it would not be correct to refer to it as just a five-second test anymore, but rather an N-second test (or a first impression test, if you do not wish to be too pedantic about the number of seconds).

Each participant first passes not just one but two cognitive ability tests. Human minds are multifaceted, and there is not just a single “cognitive ability” metric that would encompass everything that the mind can do. Among standard tests used by psychologists, we picked two that are linked to abilities that can be found as the most relevant to the formation and testing of first impressions:

Perceptual speed: How quickly you pick up visual information.
Working memory: How much information you can mentally process at the same time.

Working memory is the appropriate memory ability to focus on since it operates with information that receives the user’s attention. This distinguishes it from sensory memory (the memory processing information that our senses pick up) and long-term memory, where information is stored persistently for later use.

For the first impression test itself, six website screenshots were used as the stimuli. These screenshots were selected for possessing a broad range of visual complexity, from the simplest with just a few visual elements to the most complex with a number of distinct sections that serve different purposes.

Screenshots of real websites local to Czechia and Slovakia were translated into English, and their logos were replaced with fictional brand names so that, for all intents and purposes, the website screenshots would be authentic yet also unfamiliar to the participants who were recruited in the UK.

Finally, participants were asked to provide feedback by answering practically a complete portfolio of the various types of questions that can be typically asked during a first impression test. Each type of question tests a different aspect of the first impressions that the participants have formed inside their heads:

Attitudinal questions: Rating a perceived quality of the website (e.g., ugly vs. attractive) on a scale from 1 to 7.
Target identification questions: Questions directed at specific elements or aspects of the stimulus.
Memory dump questions: Asking participants to describe everything that they remember about what they saw.

The resulting answers were analyzed both quantitatively (with statistics) and qualitatively (by inspecting the contents of the received answers on an individual level). With it, a number of conclusions can be reached, some expected and some rather surprising.

Now that you have a picture of what our case study was about let’s dive into the actual, interesting implications for developing the first impressions of your product.

Note: If you would like to immerse yourself in further details of how our case study was conducted, you can learn more in our scientific paper.

Takeaways

Statistically, all the variables we experimented with — the time duration of showing pictures to participants, the participants’ innate cognitive abilities, and the visual complexity of pictures — had a significant effect on the first impression answers.

For instance, between the groups that were shown screenshots of websites for two, five, and ten seconds, the number of answers that incorrectly identified what the websites were for dropped as time progressed. Notable is the difference between five and ten seconds. If the participants were really focusing on inconsequential details after five seconds, there should not be differences in recognition of such a key aspect as the website’s entire purpose.

Statistical differences lay the grounds for further observations on how changing the conditions of a test can (or cannot) affect its results:

Attitudes crystallize faster than in five seconds.
In attitudinal questions where participants are asked to rate how they view the picture’s various qualities (e.g., from clear to confusing, from captivating to dull), answers stay relatively consistent, regardless of how much time the participant has or how good their cognitive abilities are. If you are laser-focused on assessing participants’ attitudes about your product and nothing else, you could present pictures for two seconds, or possibly even less, as research done by others on a related topic also implies.
Logos are recognized earlier than in five seconds (with one exception).
The target identification questions where participants are asked to recall the company name from the logo are, on the whole, impacted by time very little. This is to be expected: when viewing a website, our eyes are usually drawn to the top left corner to find out where we have found ourselves. There is an exception to this rule, however.
Among participants with slower perceptual speed, significantly fewer identified the company name correctly at two seconds when compared to five seconds. This establishes five seconds as a more inclusive choice for timing your first impression test if you expect your target audience to have, on average, lower perceptual speed than the general populace and if the primary aim is to test contents of the header, such as logo design or company name identification. Otherwise, two seconds is a safe bet.
Irrelevant nitpicking? Yes, if visual complexity is low.
In some cases, the popular narrative about five seconds being a good viewing time for testing first impressions is indeed true. Particularly for the simplest website screenshots, once five seconds have elapsed, participants start paying attention to minute details (e.g., the girl’s shirt color in the hero image).
Curiously, though, having more time does not mean that participants would write longer or more complex answers. Instead, when participants have ten seconds to view the screenshots, the higher visual complexity of the screenshots is reflected in better-quality answers. Participants stay more on-topic–describing how the site is visually structured or justifying their criticisms of the page’s design. Different viewing times may be optimal in different situations. Especially since…
Low working memory warrants longer viewing time.
When asked to reiterate what they saw in their own words, participants with low and high working memory provided significantly different answers. With low working memory, answers become shorter, less complex, and recall fewer concepts overall. However, when the viewing time is extended to ten seconds, these differences disappear. This implies that the same information is being processed — memory capacity just dictates how fast it can happen.
Without knowing where each participant’s memory ability stands, it is difficult to tell what they would actually recall if we left them to work at their own pace. Consequently, assessment of working memory before testing first impressions (and adjusting viewing time accordingly) should be considered a good practice.
For cognitive powerhouses, five seconds are enough.
A less practical point maybe, but if you are developing an app for people with reasonably high perceptual speed and working memory — be it the mentally gifted, hyperproductive hustle enthusiasts, or caffeine addicts — you could likely show them your screenshots for just two seconds and get similar results as in a five-second test.
Give participants the proper amount of time to form a first impression.
When the visual stimulus is more visually complex in a first impression test, the task of mentally processing it becomes more difficult and time-consuming (just like in any normal scenario). This manifests in test results. Fewer people correctly identify the purpose of a more visually complex website, and they recall fewer elements and aspects of the website.
This could be seen as a bit of a paradox since more complex stimuli mean there is actually more content that participants could potentially remember and comment on, but only if they had the time to absorb the information properly. Data shows that when participants are given ten seconds, the answers do actually normalize, becoming more similar to stimuli of lower visual complexity.

If the purpose of the particular first impression test is not to remove all visual complexity at any cost outright, we would suggest adjusting the viewing time to reflect the visual complexity of the stimulus.

Keep in mind there are still aspects of first impression testing that remain unknown. A reasonable question that you can ask now would be: “Okay, so how exactly do I time my first impression test?” While we can sum up our observations into a conceptual framework of how time can be treated in first-impression test planning, it is not an exact guideline; there may be other interpretations or exceptions.

Take this more as an eye-opener and a call to action. Indeed, in our study, ten seconds yielded more appropriate results for more complex websites than five seconds did. But there is nothing to say that for other websites, the best timing could not be fifteen or even twenty seconds. Even more so, once you also factor in the influence of the cognitive ability of each individual participant.

The key takeaway? When you gauge your audience’s first impressions about something, take a more holistic approach.

Consider your goals for your test. What kind of questions do you want to ask? Use some of the tools that are available to measure the visual complexity of the pictures that you want to present. Give your participants a short working memory test before you start bombarding them with pictures and questions.

Try to adjust the timing in your first impression test to match the situation. To give an analogy, by blindly following a different five-second rule and eating off the floor, you could end up getting sick. Be just as cautious about relying on myths in your usability research methods. This is not to discount five seconds. As we show, it is still good timing for first impression tests in plenty of cases, but it is not the be-all and end-all as far as first impression testing goes. By broadening your perspective, you can do even better.

Resources

“Cognitive abilities and visual complexity impact first impressions in five-second testing”
The paper by UXtweak Research investigates the impact of timing, cognitive abilities, and visual complexity on first impression testing. Read for further details about the experiment, its outcomes, and a discussion of findings.
“How Long Do Users Stay on Web Pages?”
Jakob Nielsen’s article on how long it takes people to leave a website (a fact to consider when testing first impressions).
“Attention web designers: You have 50 milliseconds to make a good first impression!”
The paper by Lindgaard et al. shows that when it comes to attitudes, some first impressions are genuinely formed in a flash.
“The UX Five-Second Rules: Guidelines for User Experience Design’s Simplest Testing Technique”
Paul Doncaster’s book on five-second testing provides a thorough summary of existing knowledge about the method and its applications.
“How to Test a Design Concept for Effectiveness”
Paul Boag’s walkthrough of key concept testing methods, including five-second testing.
“5-Second Tests: Measuring Your Site’s Content Pages”
This article by Christine Perfetti, dating back to 2007, can be seen as the first reference to modern five-second testing.
Aalto Interface Metrics tool for measuring visual complexity of user interfaces
Human Benchmark test of visual working memory
UXtweak’s Five Second Test tool