Five cognitive biases that can hugely impact your research

Most people like to believe that they consistently make fair judgements and statements, free of bias and prejudgment. However, as psychological
research has evidenced, time and time again, no one is immune
from cognitive bias. Everyone would benefit from becoming more aware of these biases; but for professional researchers, this is absolutely imperative.

Of course, it is the researcher’s responsibility to be aware of, and (where possible) to mitigate the effect of any known biases, to gather an honest read on feelings and opinions. This is important to achieve good quality feedback, to provide reasonable and gainful orientation and make clear and appropriate client recommendations.

Beyond the opportunity to mitigate or reduce bias effects, we might seek to harness these natural tendencies for the benefits of our research. It is also important to remember that we, as researchers, now live in an age where we are effectively bargaining for people’s time and attention against a variety of competing sources. Hence, where appropriate and ethical, we might further consider opportunities to activate or control these biases, to enhance levels of research commitment and motivation.

There are, quite literally, hundreds of different types of bias one might consider,
so here we will focus on those with the clearest and most direct implications
for effective online research community task design and moderation, and the
ones that might generate the greatest client value.

1. Confirmation Bias

Clients can occasionally arrive with a very preconceived notion of what it takes to explore a particular topic or theme, or of the particular theme that they think needs exploring. This typically involves over-reliance on a single type of research methodology or a narrow focus on a set of questions that might confirm expectations while missing out relevant disconfirming information.

Implications for online qualitative research:

• Use your breadth of experience to make further suggestions and present ideas, and to leverage the versatile opportunities for online research community engagement. This often results in a multi-discipline approach, more capable of delivering against the client’s business and research objectives
• Triangulate methods to maximise validity whilst meeting your clients time and budgetary constraints. Consider the range of community tools and techniques at your disposal (i.e. discussions, surveys, markup tool exercises and diaries). Each particular tool and methodology has its merits, however combining them can provide a result greater than the sum of its individual parts

2. Order Effects

This refers to the impact on the nature and quality of feedback that researchers obtain from participants based on the order in which they introduce activities and questions.

Implications for online qualitative research:

• Consider sequencing online research community activities so that private / individual tasks that are uninfluenced by others (e.g. surveys, polls, unbiased blogs) occur before open, ‘biased’ group discussions.
• Exploring behaviors prior to attitudes, and ask general questions before more specific questions, funneling and drilling down
• Consider opportunities to phase aspects of your enquiries, for instance explore communications that attract and engage attention, prior to the assessment of concepts

3. Reciprocity Bias

This describes the impulse to give when we receive. People feel much more inclined to share information about themselves if you first share information about yourself; if you give them something.

Implications for online qualitative research:

• Trigger your participants’ generosity by sharing with them something about you, your role and the mission / purpose of the community. If you share information via your community, you are not only reciprocating, but also implicitly revealing task and commitment expectations
• Be empathic towards the experience of participants in your research and lead by example. For instance, if video forms a crucial part of your agenda, consider posting a video on
  your community home page to help make your introduction, share expectations and allay
  any potential concerns

4. Social Desirability

This refers to the tendency to over-report socially desirable characteristics or behaviours. In general people want to please others. We all have a natural tendency to discuss (i.e. promote) aspects of our lives and relationships in a positive light.

Implications for online qualitative research:

• ‘House rules’ can help to develop a stimulating discussion environment based on open and honest sharing, but your approach to moderation will also be crucial, challenging and probing when required. Remain objective and avoid over-indulging ‘glowing’ perspectives when moderating
• Try to conceal your expectations of what would be the ‘desirable’ outcome that your client would like to see when framing your enquiries
• Consider the range of online research communities tools available to engage on a private and social basis. You may wish to consider restricting the ability to view other members feedback prior to posting, thus mitigating the effects associated with social desirability bias

5. Consensus

This is the tendency to harmonise in agreement when participants engage in group conversations. Consensus bias is particularly likely to occur in circumstances that are less familiar, where people look to the behaviour of others and in turn judge how they might themselves behave.

Implications for online qualitative research:

• Beyond setting expectations and house rules consider the varying relationships with your client’s product, service, brand, or indeed the category in question. You may wish to consider potential to recruit separately engage ‘segments’ and in your community tasks and activities.
• For instance, if you are interested to explore cooking behaviour and the perceived merits of preparing food using different brands it might make sense to separately engage loyal users of different brands in conversation

In conclusion

While bias is an inevitable part of human cognitive functioning and it makes for
much better research practice if researchers are aware of likely biases and can
think of creative ways to minimise or, indeed, harness them in their favour.
Online research provides a way to address different sources of bias.

Methods can be triangulated in a single online environment, with further
scope to sequence and mix private and social activities, and set tasks as
biased or unbiased.

In addition, online research conducted asynchronously enables relationships
to develop over time. This has inherent risks in terms of generating social
desirability, but it can also make for much better evidence and insight, because reciprocity can be actively encouraged.