Don't Do it Wrong #1 - Always Do Ethno Before Surveys

This is the first in a series of articles designed to help avoid the most common mistakes and failures in commercial research.

Always do Ethno Before Surveys

Ethnography and quantitative surveys are often thought of as polar opposites. Whilst it is true that they approach research in fundamentally different ways, we find that when they are used together they are powerful in generating accurate and effective insight and foresight for our clients. This is especially important with surveys. The purpose of this article is to help you design better and more effective surveys.

In business research, we use surveys to better understand consumer perceptions, attitudes and beliefs towards products and innovations. However, not only are consumers’ perceptions, attitudes and beliefs towards products and innovations shaped by the cultural and social context around them, but the cultural and social context also shapes how consumers interpret and evaluate products and innovations. Very often a lack of engagement with this context in the construction of the survey results in a research instrument that is encoding a problematic set of assumptions in the answers, or at worst is forcing a perspective onto the respondents. Thus, a challenge we face in conducting consumer surveys is ensuring that consumers correctly understand and respond to what is being asked of them in surveys. Good ethnography is essential in conducting effective surveys because it helps the survey designer better understand the humans that will respond to the survey. Ethnographers do this in the following ways:  

What to Ask

Doing ethnography before designing and distributing a survey will help determine what sort of questions are appropriate to ask. People live in different social and cultural contexts, and things that may be very appropriate to ask in one context may be inappropriate or not carry the same meaning in a different context. For example, questions on diet and eating meat in a survey will be received very differently depending on the country the respondents are from, and their religious and personal views. Ethnography is excellent at determining the appropriateness of survey questions because of its emphasis on understanding the meaning people draw from their social and cultural surroundings.    

How to Ask

Once we know what to ask, it is important to understand how to ask. Societies vastly differ in factors that determine how its members communicate and signal competence to each other. For example, there are differences in power distance, self deprecation and humility across cultures and society. Some societies and cultures have high power distance meaning they have a strong reverence for authority. Asking a survey question on a superior’s performance in such a context will elicit a very different response in context where power distance is low. Research has found that consumers satisfaction scores for products differ depending on the country they are from ( Johnson, Herrmann, and Gustafsson, 2002). Good ethnography can help overcome such challenges by helping design survey questions that are phrased in culturally appropriate ways that can allow for effective cross-cultural comparisons. 

Who to Ask 

Ethnography will also help in determining the people from which to collect responses in a survey. Good ethnography gives understanding of how different groups in society experience and draw meaning from different phenomena. This is immensely useful in determining the target sample for a survey. For example, ethnography can inform a survey on sneakers on who to target depending on if the focus is on sneaker culture, price, or usage in sporting activities. 

When to Ask

The social and cultural context individuals are presently in strongly influences how they respond to survey questions on a particular topic. How an individual feels about eating snacks depends on the time of day and how hungry they currently are. How an individual feels about driving a particular car depends on the driving conditions the individual faces in certain situations. Research has shown that how we feel about our jobs, marriage, divorce, and many other significant life events depends on how long we have been experiencing that phenomena (Clarke, et al, 2008). Ethnography is a great tool for identifying which contexts are most salient in determining how people feel about products, services and events. This can then be used to identify the most suitable time and context to require respondents to respond to a survey. 

Conclusion

Analyzing data from a survey that is not informed by ethnography can be extremely confounding. Questions are misinterpreted by survey participants. The quantitative data analyst misunderstands the responses given in the survey. Such surveys lead to an inaccurate picture of the population surveyed and their true consumer preferences. To improve, surveys have to become more human centric! An effective way to do this is always doing ethnography before surveys. 

References

Clark, A. E., Diener, E., Georgellis, Y., & Lucas, R. E. (2008). Lags and leads in life satisfaction: A test of the baseline hypothesis. The Economic Journal, 118(529), F222-F243.

Johnson, M. D., Herrmann, A., & Gustafsson, A. (2002). Comparing customer satisfaction across industries and countries. Journal of Economic Psychology, 23(6), 749-769.