Penalty analyses: diagnostic tools for optimising products

The best product tests should do more than simply determine whether a product has been accepted by consumers. In an ideal scenario, they should also give answers to “why” and “why not” questions and thereby provide practical hints on how to improve the product under investigation. 

Specific analysis methods help answer the following questions: 

• What can or must be changed about the product in order to increase its acceptance or to maintain acceptance in the long run? 
• What exactly should be changed?
• Which aspects are most important?
• Which direction should a product modification take? 

K-Diff analysis is a tool that indirectly measures whether a product leaves consumers feeling disappointed or impressed and willing to make a purchase. The analysis compares buying intentions before and after a product trial.

As opposed to cases when consumers are asked directly whether a product has left them feeling disappointed, the results of K-Diff analyses are completely independent from general findings on the acceptance of a particular concept. This means that it can be clearly determined whether a possibly poor rating was caused by a less successful recipe, for example, or by a generally lower acceptance of the type of product in question. 

Penalty analysis is used to display correlations between diagnostic questions (“just right” scales, e.g. much too strong – somewhat too strong – just right – somewhat too weak – much too weak) and acceptance measurements, such as overall acceptance and intention to buy. 

The specific advantage of this analysis is that it determines both the importance of the individual aspects (e.g. seasoning) as well as how a recipe could potentially be improved. 

Penalty analysis helps answer the following questions:

• How do product weaknesses influence product acceptance? 
• Which product weaknesses devalue the product (=penalty)? In other words, which improvements are absolutely necessary? 
• Which weaknesses are perceived as less serious? In other words, which improvements are not absolutely necessary? 

The analysis is based on the depiction of differences in mean values between satisfied respondents (“just right”) and unsatisfied respondents. The higher the difference in mean value (“mean drop”), the more the product weakness influences product acceptance. 

Critical product features can quickly be identified in a four-field matrix.

Driver analysis can be used to rank the points of criticism in order to show clearly which product features require the most urgent improvements. 

The output is the proportion of criticism that has led to feelings of disappointment with the product (devaluation).