• There are no suggestions because the search field is empty.

Product testing and online research communities – how to choose the right methodology

Published 30 Aug 2016 2 minute read

Product Testing

Every researcher worth their salt knows that online research communities are a great way to support product development and carry out successful product testing. After all, who in their right mind wouldn’t want to connect to explore experiences, gain feedback to shape concepts, influence their development and create appropriate sales messages?

When running online communities, it’s not just about getting the right members on board and allowing enough time for different phases, though. The main choice for the researcher is actually whether or not to go with monadic design, where members of the online community experience just one product, or comparative design, where you guessed it, they get to experience two or more different products in order to compare them. So how do you decide which methodology is right for you?

The monadic method

Monadic design is the most popular method used amongst market researchers because it is generally believed to be more realistic. Why, you might ask? Well, to reflect ‘real-life’ situations and circumstances, it’s pretty typical for a consumer to purchase, experience and focus on just one particular product at any one time. Using this approach, researchers can benchmark a product’s performance by having one group of individuals test one product (the new client product, for example) whilst another separate groups tests the alternative, whether it be the existing client product or one of its competitors.

The comparative alternative

Using the comparative approach, community members are placed with two or more different products or product variants for testing at the same time. When using this approach, you should be careful to ensure that all the different products are matched in terms of demographics and any other aspect that might have an influence on its relevance and appeal. For example, if you’re exploring concept expectations and performance for a food product, you should factor in household composition, consumption habits and any brands typically consumed into the community recruitment and research design, as these can all have a significant impact on how the product is received.

Whilst some think it is less realistic, the comparative design approach has a great advantage in that it offers researchers the potential to pick up on finer differences that may exist in the product experience that can’t be identified using the monadic method. This is great if multiple variants of a single product are being considered, (for instance to compare feedback towards different flavours of soup). In fact, comparative designs are typically used for quantitative product rationalisation tests to develop a single  specific aspect of the product mix (e.g. formulations or ingredients in the world of FMCG). 

Product_Testing_Phone (1)
The implications on research design

But what does this all mean when it comes to your research design? Well, the choice of monadic or comparative design can have a number of important implications for the form of questions that might be employed when it comes to designing your research. For example, when it comes to the comparative method, the questions used will need to carefully determine if subsequent product experiences were better worse or similar compared to the first.

Sequential monadic tests can serve as an effective halfway house here. These tests see community members given the first test product to experience and reflect upon during a specific placement period whereby an assessment occurs prior to a second separate placement, where the second product gets placed and in turn assessed. To mitigate order effects in the assessment (e.g. the tendency to benchmark the experience based on the initial product experiences), it’s important to create separate segment groups or ‘cells’ splitting and reversing product exposure. For example, you should create one group segment to assess Product A followed by Product B, whereas another segment will begin with assessing Product B before moving on to Product A.

If you’d like to find out more about how to recruit for online qual, why not download our expert guide? It’s packed full of tips, methods and techniques to help enhance your product testing research using online research communities to ensure you get the best results, every time.

Discover our platform and services


The insight platform for online qual, research communities, digital diaries, ethnography and more.

Services & Support

A range of expert research services and resources to help you deliver your projects with ease, speed and reach.


Human insight with impact; leveraging our academic and industry experts to uncover insight, create impact and make confident decisions.

Conde-Nast (1) (1)

We were amazed at the level of insight we achieved in just a week. Further opened our eyes to new ways of researching and understanding our staff

We helped Conde Nast International define a new global mission and vision statement

van-tay-media-Kab_-4M4I74-unsplash (1)

Further really understood the brief and were extremely proactive. We are now very confident that we’re taking the right products and proposition to market.

We helped this insuretech startup tailor their customer value proposition for the UK market ahead of a planned launch

Keyhouse (1) (1)

Further's expert team pushed us to clarify our assumptions and to think harder about how to communicate the value of our products and services

We helped Keyhouse enter a new market and understand what target users of their case management software needed and how to position their offer

FTH001_Mother_2_children_tablet (1)

Working with Further was a refreshing and eye-opening experience…...the qualityof their output which was excellent.

We helped Unicef generate insights to support the development of a mass market, sustainable fundraising product.

Zwift image (1)
Zwift-logo (1)

Strategically, Further’s insights provided clear and directional answers that will guide us through our next phase of growth

We helped Zwift understand users and non-users needs and wants so they could prioritise their innovation pipeline

Chillys image case study (1)
Chillys-logo (1)

We helped Chilly’s leadership team consider new ways to understand and co-create with their customers’

We helped disruptive pet insurance company Waggel develop customer personas and map out the current and intended customer journey.

What next?

Browse our site, download our resources, request a demo of our platform or speak to one of our experts

Browse our work
Contact Us