Product testing – an extremely important process that will cost your business dearly if you fail to carry it out correctly.
What is product testing and why is it necessary? Generally, product testing is done in the following situations:
- When a company wants to introduce a new product to the market.
- When a company wants to create a new version of an existing product.
- When a company wants to optimise the ingredients/components of an existing product formula or blueprint e.g. trying to use cheaper or more easily available ingredients to cut costs while still achieving the same quality of product.
As our previously covered article on concept testing/idea validation, bringing a new product to the market is usually extremely costly due to the amount of resources required for the production and distribution of a new product to be carried out.
After the concept testing (idea validation) process has ensured that the new product is something that consumers would actually want to buy once it is in the market, we can proceed to the next steps of the innovation journey which are product development and product testing, which together form the key stages of product innovation for companies, i.e. creating something tangible to put in front of consumers and obtain feedback.
Here are some key things to remember when conducting your product tests.
1. Define the action standards for your product
Before product testing with consumers can take place, it’s vital to properly define the goals that you want the product to meet, which are known as action standards. These are the key metrics that will inform your testing process and determine whether or not the product can be considered successful after it has been tested with consumers. A possible example of an action standard would be for the product to get an 80% score on average from consumers. Or, the benchmark could be for consumers to state that the new product is on par or better than an existing product.
The action standards are the first step in setting up a proper product testing and it is important to set this up front, but there are also other things to keep in mind before moving on to the actual product testing phase.
2. Get the right consumers to test your product
When testing a product, it’s important to ensure that you have an appropriate sample of consumers to test the product. Ideally, we would want to have at least 100 respondents to ensure that we have an adequate sample size for the product test and obtain reliable results for whether or not the product is successful amongst consumers.
You must also ensure that the consumers that participate in the product testing are unbiased. Recruiting people that has direct association with your company (e.g. internal employees) to take part in product testing would be unwise as those with ties to the product are likely to be biased in their evaluation. The goal of product testing is to get an accurate picture of what ‘real’ consumers would think of the product, not what the employees of the company think is right. Thus, you can obtain objective, non-biased data that can actually be used to tweak and improve the product.
3. Conduct the test under proper conditions to avoid bias.
One of the most common ways of conducting product tests is known as sequential monadic testing, where a consumer is presented with two products – product A and product B, and asked to try each one, individually at first. Once they have tried both products, they are asked to compare the two.
When conducting such tests, determine your methodology approach. One commonly accepted way of carrying out product testing is through what is known as a “Central Location Test” or CLT. This is when respondents are invited to a specific location for testing, interviewing and data collection. This is done so that the company conducting the test can make sure that the consumers test the product in a controlled environment to prevent external factors from influencing how a potential consumer will experience and evaluate a product. For example, if the testing for a new beverage was done in a shopping mall with consumers that just happened to be walking past, it would be very difficult to get accurate feedback as all of the consumers might be influenced by different factors such as their current level of thirst, or latent aromas in the surroundings, etc. that might be affecting their senses — thus affecting their overall experience of the product. Furthermore, it would be very difficult to ensure that all of those consumers actually take the time to properly enjoy or evaluate the product in the way that would be necessary for a reliable product test.
Environmental variables make conducting product testing at an external location unreliable.
By carrying out a central location test, you will be able to ensure that the testing is done in a controlled environment where the product is always presented to each of the consumers in the same way. This is to make sure that all of the consumers are experiencing the product in the same way as the others. This can include measures such as presenting the product without any branding or coloured packaging e.g. a beverage will always be served in a plain white paper cup so that the consumer’s impression of the product is not influenced by certain colours of imagery, or preparation of the product is done exactly the same way for everyone, e.g. mixing 50ml of water at 70degreesC with 10g of powdered coffee.
4. Conduct the test using standards appropriate for the product
While ensuring that all of the consumers participating in a product test are adhering to the same standards, it is also important to consider the way the consumers are experiencing the product during the test.
Once again, the goal of product testing is to obtain an accurate picture of how consumers will react to the product when they get to experience it. As such, consumers that take part in the product test must be given the opportunity to consume the product in the way that it would normally be consumed when it is being sold on the market.
For example, when it comes to food and beverage products, it wouldn’t be appropriate for the consumer to just take one bite or sip before giving their opinion, as the results for any given product can vary depending on what quantity of it is consumed. The significance of this was proven before in a case known as the Pepsi sip challenge, where consumers were ask to take a single sip of Pepsi and a single sip of Coca-Cola before stating which of the two they preferred. During the sip challenge, it was found that a majority of the consumers preferred Pepsi based on that particular experience, but later studies showed that this did not necessarily mean that the majority of consumers actually preferred drinking Pepsi over Coca-Cola on the whole. The studies found that when it came to comparing two drinks based on a single sip, most people will choose the sweeter of the two beverages, but in practice, many of the same people who chose Pepsi would actually have preferred Coca-Cola if they were made to consume an entire can.
Even the sight of certain colours can affect the consumer’s evaluation of a product due to brand associations.
Therefore, an appropriate portion must always be sampled by the consumer in order to get an accurate result. For beverages, this might mean that the consumer must always consume at least half a cup before giving their opinion on the products. Having the consumer take too much of the product at once can also have a negative impact on the accuracy of the test. In the case of products that have a strong aftertaste such as chocolate, the consumer should not test more than 2 or 3 products in one sitting due to sensory fatigue which affects the way the consumer evaluates the later products.
Finally, the product should always be prepared in the same way before it is presented to the consumer. For example, if you were to test a new type of instant coffee powder, the sample being served to the consumer should always be prepared with the same amount of powder and water, with each one being mixed to the same degree and served at the same temperature.
All of the above measures will help ensure that any feedback collected is based on the actual properties of the product, not the variable conditions that can affect the consumer experience.
5. Let consumers test the product at home if it makes sense to.
Despite taking all of the above measures, there will be certain products for which appropriate testing standard cannot be carried out during a central location test. This is typically the case for products that require time in order for consumers to fully utilise or appreciate, such as skincare products, or a product that cannot practically be tested in a central location, such as laundry detergent.
In these cases, you can conduct what is known as a home placement test or home placement test. This is when the product is given to the consumer for them to bring home and use under a strict set of predetermined guidelines for a set period of time. After that, the consumer can then be interviewed to get their evaluation of the product.
6. Have the right framework for collecting consumer feedback
When it comes to asking consumers for their feedback, there needs to be a proper framework in place so that the feedback can be collected in a structured way that yields meaningful data which isn’t affected by the consumers biases. A good rule of thumb is to ask big questions first to avoid anchoring before going into specific questions (refer to our previous article for an explanation of anchoring and how it can affect a consumer’s evaluation of a product.).
One possible framework you may want to implement is a sensorial framework. This will break down how the consumer experiences the product rather than asking general questions about the product and will include questions related to the senses through which the consumer experiences a product:
- Sight
- Smell
- Touch
- Taste
- Sound
From there, we can conduct a penalty analysis to pinpoint which sensorial attribute impacts liking by using a special rating scale, known as the JAR (just about right) scale – a deliberately polarizing scale of measurement, where both ends are negatives. To use this, we ask the consumer to rate certain sensory attributes of the product from 1-5.
For example, when asking a consumer to rate the sweetness of a product using the JAR scale,
3 would be just right, while 1 equates to not sweet enough and 5 would mean too sweet.
By getting all of the consumers to rate the product’s attribute in this way, we are able to see patterns among the data which inform us of how consumers view the product to help give further improvements needed. For example if all the consumers rate the sweetness of a product as 4 or 5, like that teh tarik at your local mamak stall each morning, then it’s safe to conclude that the sweetness of the product should be toned down so that consumers like it even more, buy more of it, etc.
The best score here in all cases is a 3 – which communicates two things,
1- The sweetness level is good, and
2- I like it.
And this can be repeated for all other key ‘taste’ sensory indicators like milkiness, bitterness, aftertaste, etc.
Conclusion
Much like concept testing, product testing is another step on the innovation journey which must be carried out carefully and precisely if your company is to have any hope of successfully launching a new product that is superior in the eyes of consumers. It pays to be meticulous in your product testing process as you will be able to gain useful information that not only informs you of whether or not your product is favoured by consumers, but also gives you data that can be used to improve your product or even formulate marketing strategies further down the line.
If you would like to find out more about how product testing processes can help your business, please feel free to contact us at: theteam@oppotus.com
