Product testing and online research communities – how to choose the right methodology

Every researcher worth their salt knows that online research communities are a great way to support product development and carry out successful product testing. After all, who in their right mind wouldn’t want to connect to explore experiences, gain feedback to shape concepts, influence their development and create appropriate sales messages?

When running online communities, it’s not just about getting the right members on board and allowing enough time for different phases, though. The main choice for the researcher is actually whether or not to go with monadic design, where members of the online community experience just one product, or comparative design, where you guessed it, they get to experience two or more different products in order to compare them. So how do you decide which methodology is right for you?

The monadic method

Monadic design is the most popular method used amongst market researchers because it is generally believed to be more realistic. Why, you might ask? Well, to reflect ‘real-life’ situations and circumstances, it’s pretty typical for a consumer to purchase, experience and focus on just one particular product at any one time. Using this approach, researchers can benchmark a product’s performance by having one group of individuals test one product (the new client product, for example) whilst another separate groups tests the alternative, whether it be the existing client product or one of its competitors.

The comparative alternative

Using the comparative approach, community members are placed with two or more different products or product variants for testing at the same time. When using this approach, you should be careful to ensure that all the different products are matched in terms of demographics and any other aspect that might have an influence on its relevance and appeal. For example, if you’re exploring concept expectations and performance for a food product, you should factor in household composition, consumption habits and any brands typically consumed into the community recruitment and research design, as these can all have a significant impact on how the product is received.

Whilst some think it is less realistic, the comparative design approach has a great advantage in that it offers researchers the potential to pick up on finer differences that may exist in the product experience that can’t be identified using the monadic method. This is great if multiple variants of a single product are being considered, (for instance to compare feedback towards different flavours of soup). In fact, comparative designs are typically used for quantitative product rationalisation tests to develop a single  specific aspect of the product mix (e.g. formulations or ingredients in the world of FMCG). 

The implications on research design

But what does this all mean when it comes to your research design? Well, the choice of monadic or comparative design can have a number of important implications for the form of questions that might be employed when it comes to designing your research. For example, when it comes to the comparative method, the questions used will need to carefully determine if subsequent product experiences were better worse or similar compared to the first.

Sequential monadic tests can serve as an effective halfway house here. These tests see community members given the first test product to experience and reflect upon during a specific placement period whereby an assessment occurs prior to a second separate placement, where the second product gets placed and in turn assessed. To mitigate order effects in the assessment (e.g. the tendency to benchmark the experience based on the initial product experiences), it’s important to create separate segment groups or ‘cells’ splitting and reversing product exposure. For example, you should create one group segment to assess Product A followed by Product B, whereas another segment will begin with assessing Product B before moving on to Product A.

If you’d like to find out more about how to recruit for online qual, why not download our expert guide? It’s packed full of tips, methods and techniques to help enhance your product testing research using online research communities to ensure you get the best results, every time.