Carrying out user testing internally: eight obstacles, eight stages, eight pitfalls

You're currently working on a website or mobile app. To ensure its success, you want to ask users about the UX and find out their opinions. You would like to conduct these tests yourself. This is both a desirable approach but also one fraught with potential pitfalls. In this article, we will begin by taking a look at the eight obstacles to overcome when carrying out your own studies yourself, the eight stages you need to adhere to when testing on your own, and the eight errors you need to avoid. We hope these three eights will be useful to you!

Eight obstacles to conducting tests internally

No matter whether you're the creator of a website or the developer of a mobile app, or whether you're on your own or working in a team, there will come a point where you need an external opinion. If you choose to carry out these feedback stages yourself, you'll need to work through each of the following obstacles:

1. The website or app in question is the baby you've been lovingly carrying. And love is blind. In short, you won't always have the necessary distance and perspective when it comes to the feedback stages.

2. You don't have the devices you need to carry out the testing: the world of the cell phone, a device which is now firmly established as an essential piece of everyday technology, is a jungle, with thousands of factors coming into play, such as different OS versions, memory saturation, internet connection quality, etc

3. You aren't testing under real-world conditions. Good quality Wi-Fi, a moment of peace and quiet, sitting behind your desk: this bears little relation to what actually happens in real life.

4. Not having access to the right targets is another commonly encountered obstacle. Self-conducted tests are often carried out with colleagues. Though co-workers can provide you with some initial feedback, they harbor biases and prejudices that your future users do not share.

5. You can end up drowning in ergonomic analysis! Conducting tests results in literally hundreds of problems being brought to the surface, and these will need to be resolved. They will therefore have to be analyzed, categorized, prioritized, etc.

6. A further obstacle: not having enough time. Undertaking out all the various phases of testing can literally become a project in itself, extending over several weeks, or indeed even several months. However, there are methods available that you can use to keep everything progressing at a rapid pace, such as guerrilla testing.

7. You have better – or at least higher priority – things to do. Let's be honest, testing is a long process, one that requires you to set aside time in your schedule. Is your added value really to be found in tests carried out with your colleagues?

8. In a few weeks' time, you'll need to begin doing it again. In the time since you launched your website or app, the requirements and expectations have already changed, with new models of phones and tablets appearing on the market, or perhaps the release of a new OS version, etc.

For a better illustration of these barriers, we recommend taking a look at  the testimonies of our customers, some of whom, such as Bob Emploi, have the skills and resources necessary to carry out tests internally but have instead chosen to conduct them externally.

To illustrate the obstacle presented by the diversity of devices available, here's a visual representation of the different screen sizes used by the main smartphone manufacturers:

Eight stages you need adhere to if, despite everything, you do decide to carry out testing internally

You are now familiar with the reasons why it's a good idea to entrust your user tests to an expert but have decided to conduct them yourself anyway. Excellent decision! So let's now take a look at how you can go about it. Let's begin by listing the stages you'll need to go through in order to optimize your UX and the ergonomy of your website or app:

1. Identify your test objectives: What do you want to find out? Though an essential part of the process, this stage is often neglected.

2. Select your study methodology: There are many ways of conducting tests with customers. You can use our Methodological Assistant to identify the methodology best suited to your purposes.

3. Plan your recruitment: This can be a long stage, and it needs to be properly identified and defined in all cases.

4. Write your test scenarios or interview guide: This involves setting down on paper the introduction and the questions you'll be asking your respondents to reply to.

5. Pre-test your scenarios: Both a project stage and a tip at the same time, the simple act of adjusting your scenarios after first conducting a test run can literally save your entire study!

6. Conduct the tests: Whether it's face to face, remotely with yourself or entirely autonomously, this is where your users express what they think about your website or mobile app. If you have decided to reimburse your respondents, this is the time to present them with their gift voucher or equivalent.

7. Analyze the results: Understanding the feedback provided by your testers is a key stage, and one that requires sufficient time and a rigorous approach. It consists of examining the entire collection of data from all sides, establishing what the recurring issues are, counting the occurrences of each, and finally, summarizing for the purposes of prioritizing.

8. Produce a summary: Finally, you need to collate and organize your analyses so that you can present the methodology employed and the main things learned from the tests, supported with verbatim examples.

How much time does it require? To carry out all these stages with 5–10 respondents, expect the process to take around ten full-time man-days, so that's around 3–6 weeks, depending on the priority you are able to assign to it.

For a more in-depth understanding of the process, we recommend downloading our white paper "Gain in agility by listening to your users". You'll find all these various stages described inside it, along with practical tips and advice to help you conduct your user tests and improve your UX.

Eight errors to avoid when conducting your own self-organized tests

• Testing late: We can always find some reason or other to carry on refining and perfecting our work before presenting it to others. However, the earlier you test, the easier you'll be able to introduce changes, and this doesn't prevent you from re-testing again at a later date. It also means you won't need a designer right away. It's entirely possible to obtain results by showing people a prototype drawn on a sheet of paper. Here's anexample of what software company Intuit did.

• Seeking the opinions of your colleagues: Your colleagues have biases that your users don't (though this doesn't matter if your colleagues are the group targeted by your testing project of course). This is one of the most common pitfalls; though having said that, it is perfectly acceptable to do this for a limited period. It's simply about knowing how to keep a limit on these internal tests and recognizing when to move away from them.

• Conversely, over-focused targeting is also imprecise. Your target is a hypothetical concept, and it is essential to go beyond it in order to not only verify that your value proposition is appropriate for the target you've defined, but also, and potentially, in a broader sense. There are countless examples of services that have achieved success by changing the target.

• Confusing the UX with ergonomy. User tests are often confined to simply improving the user friendliness of the interface, i.e. its ergonomy. Their potential, however, goes way beyond this, even extending as far as enabling you to identify new business opportunities, to better understand the ways in which your project needs to integrate with your business processes, or to quite simply detect functional bugs. In short, user testing is a veritable goldmine for both yourself and your colleagues.

• Conducting more than five interviews in a row: Listening actively to someone in an empathetic manner, without interrupting them and while remaining constantly friendly and good-humored demands lots of energy, and you'll find it difficult to carry on maintaining the same attitude throughout the process if you conduct too many interviews in the same session.

• Altering your interview guide. Avoid making this mistake once you've commenced the testing. The reason for this is that it's essential to use a consistent methodology throughout the entire study if you want to be able to compare the results.

• Not listening to interviewees: This might seem obvious, but active listening is a conscious, specific process. In particular, you need to avoid completing interviewees' sentences for them and responding to them when they ask for confirmation. Active listening consists of encouraging the person but never thinking on their behalf. Though this may seem like a strange approach, by adopting this strategy you will avoid providing the respondent with confirmation that they have either properly or incorrectly understood your project.

• Never doubting: During the analysis stage, it's tempting to call into question the feedback provided by users if it happens to contradict your current hypotheses. It's possible to use various excuses for this, such as "They haven't understood" or "Anyhow, we can't not do anything". Knowing how and when to be doubtful is a skill that develops over time, and it's something that conflicts with what's requested of you in terms of the convictions you hold about your own project.

It’s to respond to those problematics and testing needs that we have created Ferpection, an agile User Research platform. We'd be happy to discuss with you about how we can help you implement researches that you'd rather not do alone.