Introduction
Market research studies serve companies as a basis for better decision-making. Whether advertising campaigns for a new product, product designs or brand logos that have to be decided on - the results of a market research study support all strategic plannings of a company.
For this reason, survey results should correspond as closely as possible to reality. If customers like slogan A most in a study of advertising slogans, then this should also correspond to their real opinion.
In order to transfer the results of market research to real consumer behaviour as realistic as possible, the methods used to collect insights must meet certain quality criteria.
This guide explains the most important quality criteria in market research: objectivity, reliability and validity.
I. What are Quality Criteria in Market Research?
The quality of measuring instruments such as questionnaires is determined by certain quality criteria. Whoever commissions a market research study wants to gain insights into the behaviour of consumers. These insights should be as accurate as possible so that meaningful conclusions can be drawn.
However, this only works with reliable survey methods that are precisely suited to the respective project, for example for a design test or a pricing analysis using the Van Westendorp method.
In most cases, a market research survey is based on a questionnaire, which measures exactly what a company wants to know.
In order to gain high-quality data from the survey, certain quality criteria must be met. Objectivity, reliability and validity of a questionnaire are important in market research.
II. Objectivity
In market research, it is particularly important that the results are independent of the conditions under which consumers are surveyed. Only if that is the case, the answers of different consumers can be compared. In general, there are different types of objectivity.
Objectivity during data collection
What target groups answer in a market research survey should not depend on who they are interviewed by or who conducts the survey. The conditions under which the survey takes place should be as standardised as possible. The person who conducts the survey should not have any influence on what the results look like.
Objectivity during evaluation
A questionnaire must be designed in such a way that different market researchers come to the same results when they evaluate the data. This means that they should not have any leeway in the evaluation, but should adhere to pre-defined rules so that the results can be compared at the end. Especially with open-ended questions, it is important to determine exactly how they will be evaluated beforehand. In the case of questions where survey participants only tick the answer options (multiple choice, single choice, etc.), an objective evaluation is usually easy to achieve.In order to check whether a survey has been evaluated objectively, the results obtained by different market researchers can simply be compared. If the results match, the evaluation objectivity is given.
Objectivity during interpretation
When the results of a survey are available, they must be interpreted. What does it mean when a person answers the question: "How do you like this product?" All people involved in the evaluation of a study should come to the same conclusion. It is therefore important to make sure that everyone follows the same rules and does not interpret the answers subjectively.
One advantage of a questionnaire that is carried out with the help of online market research is that the market researcher has no influence on the survey participants while they answer the questionnaire.
If the evaluation is also carried out automatically and digitally, the objectivity of the evaluation is given. For the interpretation of the results, however, a clear procedure must be developed.
III. Reliability
A questionnaire must be designed in such a way that the results obtained can be reproduced again and again. Just as a measuring tape can always measure a person's size exactly and a scale measures exactly how heavy a bag of potatoes is, a questionnaire must also measure exactly.
A high reliability means that if a consumer has completed the questionnaire once and then reworks it later under the same conditions, the results should be exactly the same. In that way, the so-called "retest-reliability" is checked - the extent to which the results of two surveys that were presented to one person at two different times are related.
The basic requirement for a survey to reliably measure what a company wants to know is objectivity. Only if the results of a survey are independent of who conducts, evaluates or interprets a test, the survey can be measured reliably.
Objectivity is a prerequisite for reliability. Reliability is a prerequisite for validity.
IV. Validity
A questionnaire is valid when it measures exactly what a company really wants to know. Objectivity and reliability are the prerequisites for a valid questionnaire. For example, if a company wants to use a survey to find out whether consumers would be willing to buy a certain product at a certain price (e.g. via the Van Westendorp Price Sensitivity Meter), then a questionnaire should also measure exactly that, and not whether consumers are interested in the product at all or generally like it.
There are two important types of validity, internal and external:
Market research questionnaires measure the influence of an independent variable (e.g. the design of a product) on a dependent variable (e.g. the buying intention). If a change in the dependent variable can be attributed solely to the fact that something has changed in the independent variable, then the internal validity of a question is high. As few disruptive factors as possible should be present for internal validity to be given. For example, if a company wants to test the packaging for a yogurt and shows consumers some designs, then the respondents' attitude towards yogurt design should only change because the designs are different - not because a respondent suddenly doesn't feel like answering any more, or is distracted.
External validity, on the other hand, indicates how much the results of a measurement can be generalised and how representative they are. To what extent can the statements made by consumers in surveys be applied to their actual behaviour? And to what extent does this also apply to other people outside the sample that was surveyed? For example, if a company wants to know whether consumers want to buy milk at their next purchase (independent variable), the answer given by the participants (dependent variable) should also reflect reality. What consumers say in surveys should reflect their real behaviour.
Summary
In order to gain high-quality data from the survey, quality criteria must be met. These include objectivity, reliability and validity.
Objectivity: The results of a survey should be independent of the market researchers who carry them out, evaluate and interpret them.
Reliability: A survey should measure reliably. Repeated surveys with the same questionnaire should always produce the same results.
Validity: A questionnaire should ask exactly what you really want to measure with the help of a survey.