Participant Experience Discovery Findings
First published January 2018, on Medium
This blog post is the result of a People Thinking project with Nic Price
—
What’s it like to be a user research participant?
Participants have needs when they are in a research interview. The words we use to set the scene can affect the participant’s understanding of what is being asked. The method of gaining consent can affect the participant’s comfort which will impact on how much they may divulge. What is less clear is the impact that the whole journey of taking part in research can have on our chances of getting the best insights.
A hypothesis
Participants have needs when they take part in research, if we meet these needs then we will get good quality insights, however those needs are not being met as well as they could so as result insights aren’t as good as they could be. If we discover what it’s like to be a research participant then we will be able to identify how to better meet those needs.
Additionally user research is still misunderstood, if we promote understanding of what user research is within our project and product teams by communicating from a participants view we will increase overall quality of user research.
What did we do? (a.k.a Methodology)
We had one research question
“What is it like to be a user research participant?”
To answer this question we drew up a research plan that broke that question into four;
What do user research participants need and when?
What’s it like to be a participant in a user research interview?
Are these needs being met today, how and by who?
How might we better meet these needs, and what would happen if we did?
Our participants for this research were people who:
had been participants
recruit participants
conduct user research
People who had been participants
This was further broken down into
Why do you take part in research?
What influences you taking part in research?
How do you feel when you take part in research?
What do you understand to be happening during the research?
Do you ever walk away wishing you’d said something else?
We used these questions to design a discussion guide. We conducted 5 x 30 minute depth interviews over the phone with people who had taken part in user research in the last 3 months. Participants in these interviews received a £25 incentive via bank transfer (handled by the recruiter)
We then took themes from these interviews to design a survey. The survey was administered via survey monkey and emailed to a database of people who have signed up to take part, and have taken part, in User Research. it was open for a week and achieved a response rate of 903. Respondents to the survey were entered into a prize draw for 4 x £25 Amazon voucher (handled by the recruiter).
People who recruit participants
We conducted contextual inquiries with a recruitment organisation.
People who conduct User Research
We conducted 3 semi-structured interviews with people who have conducted user research. This helped form our discussion guide for participants but themes from these interviews did not directly form the findings.
Limitations
There are limitations in the research, all our interviews were conducted remotely and they were all recruited via the same recruiter. The survey was sent to a ready engaged database of people and was conducted digitally. By its very nature, all those that took part had been participants previously (Very meta).
What did we find out?
48% of participants wouldn’t take part in a piece of research over Skype (video call)
Conducting your research over video call will exclude those who don’t have the technical ability in which to use that channel. It’s also seen as a intrusive method as it involves seeing into the participants environment.
If the research topic is sensitive or emotive it maybe that a video call is less success as participants report they can’t express themselves as well or connect with the researcher. This is a familiar finding in other video call research, the theory of social presence describes the need for a good quality communication medium when communicating.
41% of participants wouldn’t take part in home visit research
Similarly to video call interviews, home visits are intrusive but also feel dangerous to the participant inviting a stranger into their home.
But it’s not just the awkwardness of having someone come to your house. We need to be aware of the living arrangements of the person. Do they live in a busy household or shared accommodation therefore having the interview at their home may be unfair on those others in the house.
The non-video call and non-home visit populations overlap, in places.
As can be seen from the reasons for not having a video call or home visit interview there is some overlap in the reasoning. However that doesn’t mean that these populations are the same. There is an overlap, approximately a third of those that said they wouldn’t take part in a Skype interview also said they wouldn’t take part in a house visit interview.
A significant amount participants dislike the focus group method
Focus groups are seen as “competitive, restrictive and shouty”. As a result 15% of participants don’t want to take part in them, but a significant amount of the survey rated it as a method they disliked when compared to other methods.
For those that take part in research to be heard and make a difference say that the focus group is not a method in which they can do this.
The more research you take part in the more positive you are about research methods and environments
We need to be cautious of this due to the limitations in the research, however people are positive about taking part in research. As a result they want to take part in further research. They tell their friends and family about this too which leads to more people taking part in research. From our sample 44% of participants had been referred on by a family/friend/colleague.
We need to be aware too of the blockers to taking part in research:
Logistics
Bad parking was often cited as a pain. Just planning the journey to the research session and the stress that incurs can block people from taking part, or taking part with their whole selves .
Health
Health conditions, physical or mental, create barriers to taking part in research, suffering from agoraphobia — a fear of situations that may make you feel helpless or embarrassed — can prevent a participant from getting to and taking art in research.
Work
Approximately a third of people work 9am — 5pm (the Dolly Parton shift). User Researchers form a large part of this group therefore research is likely to take part between 9am — 5pm which immediately biases against those working that short pattern.
1 in 5 participants think of something relevant to add to the research after taking part.
Most user research sessions last 60 minutes, keeping participants longer would lead to fatigue. However, answering questions that you didn’t expect in a time pressured manner can be difficult. Therefore participants continue to think about the research questions and think of something that they would of wanted to say after the session has ended. They could have provided more to the session with more time or with more preparation.
Linked in with this is a sense of not seeing the value my insights offered. Participants take part in research to make a difference but often don’t see the impact sharing their stories had. It doesn’t satisfy their reason for taking part, neither does it provide closure to the research journey.
Meet Karen and Ron
We created personas to help us when we are designing our research opportunities.
Hello Karen
Hello Ron
So what does this all mean?
No research method is bad, this doesn’t mean you should stop doing Skype interviews. What it does mean is a reliance on a single method of research is bad and will bias your results.
Research doesn’t stop when the camera is off. Participants continue to think about research after the session. How might we capture those thoughts, additionally how might we conduct sessions which are less surprising and allow participants to prepare for them in a way that suits their needs.
Taking part in research is good, share that message to get others taking part in research.
We’ll be sharing shortly what the journey of a participant is like and the tasks they need to complete. In this we’ll explore further the pre-research nerves that participants feel and how that might be mitigated.
There are many questions that are left unanswered, some of those are:
What is the experience when participants are recruited via a different method/recruiter?
How do other user researchers conduct research? And how is this impacted by different sectors?
Why haven’t people taken part in research when offered the opportunity?
