Serious Business: Business Research Needs to Get Real
Business researchers advising companies typically aim to present information in simple ways. Such work can neatly summarize rich and nuanced work and may be educational, entertaining, or even inspiring — or it can be a fancy front for generic, context-devoid work and self-interest. But it is difficult to tell which you have received. Unfortunately, a few consultancies can exploit firms’ need for simplified, actionable insights and lure them to ideas, frameworks, and simple solutions not built on firm research fundamentals. In this article, we explain research quality ideas from academic research and their applicability to business research and offer readers tips for training a discerning eye to spot the difference among potential vendors of such research. Understanding how academics define research quality can help firms understand their vendors’ approaches and learn to ask the right questions. Diving into the topic of business research, interestingly, also raises questions about business and society, which we’ll address at the end.
The Criteria to Help you identify solid research
In 1985, two researchers, Lincoln & Guba outlined four main evaluative criteria for the trustworthiness of qualitative research: credibility, transferability, dependability and confirmability. These criteria can also be useful for understanding business research.
Credibility involves the trustworthiness that the findings are correct. Credible research has clear and understandable methods so readers can understand how researchers reached their conclusions. It also means that there is sufficient quantity of rich data and that data comes from reputable sources, is corroborated with different types of evidence,: for example, multiple types of data such as interviews from experts, observations from the field, and prior research. What this means for business research is that some elements are properly marked as provisional until tested, analytical frameworks are tested for their logical validity, and Interesting one-off anecdotal observations are disclosed as such.
Transferability involves the generalizable applicability of findings to other contexts. Transferability is not a direct concern in dedicated consulting projects that are designed to deliver targeted solutions, but it raises the important topic of context. For such projects, firms can rather ask themselves an opposite question: could this project that is delivered to us be rebranded easily for another client or do the results help us in our unique position, market context, etc.? Clients need to be particularly careful not to fall into client insecurity in the face of vendors — you know your business and your market! The vendor has specific knowledge and abilities, but should respect and build with your internal teams. Additionally, the problems clients face are often multi-faceted and rooted in context; firms can benefit from looking for vendors who take context seriously and do not assume they know more.
Dependability is about whether or not the same conclusions could be reached again, even by the same researchers. Clear and transparent research design and rigorous, step-by-step methods can assuage this concern. From a client perspective, dependability matters to the extent that researchers have followed processes, even if those processes are difficult for clients to understand. Playing a role in the process, or getting a walk-through of the underlying data may help develop confidence that a project is dependable. In part this repeatability is not just about how data was discovered through basic research procedures, but about the logic of the analysis. If someone uses a top-down approach to manage undifferentiated data, it might not be possible to repeat their analysis unless the framework used is identical. You need to know more than a consultant’s conclusion; you need to see how they got there.
Confirmability is about neutrality — have findings been influenced by bias or other interests? Could the research be confirmed or corroborated by other researchers? For business researchers, this relates to whether or not the information has been ‘massaged’ to fit into a storyline a particular advisory group wants to tell. It can also involve not collecting enough different types of data to be sure important information isn’t missing. While vendors inevitably have interests, clients should be able to understand that their vendor has their interests at heart. But if you have a vendor who needs to wow you to grow the relationship, then you might have a project that has a bias to surprising data and big, showy conclusions. Can your vendor avoid this tendency to give you what you need, and not what they think you want?
One important way to create research that is evaluated well, and to address confirmability specifically, is to ensure reflexivity by researchers, including researchers’ awareness about their interaction with the setting they study (Alvesson, 1996; Alvesson & Skoeldberg, 2017; Hammersley & Atkinson, 1995; Humphreys, 2005). Reflexivity is about how researchers reflect and are conscious of the work they are doing, and the questioning of what is taken for granted in one’s beliefs and how those belifs influence organizations (Ripamonti, Galuppo, Gorli, Scaratti & Cunliffe, 2015; Cunliffe, 2014). In a business research setting, of course, research is designed to help firms actively alter their environments by better understanding consumer needs and developing ways to serve them, for example. For firms considering their choice of business research vendors, it may be interesting to ponder the reflexivity among vendors in order to expose underlying questions for discussion. Do vendors and firms carry different assumptions or taken-for-granted ideas about the project?
Zooming out
There are three interrelated, deeper issues involved in the over-simplicity of business research: how the presentation of findings constructs; basic regional differences and context; and rationalization more broadly.
Some simple solutions and presentations construct, not only report. Some popular press business books and typical consulting frameworks present information via easily-digestable formats such as rankings or 2x2 matrices. Such simplified presentations can be beneficial — at a quick glance, decision-makers gain a wealth of information. However, the presentation of data can also disguise or distort the information it attempts to capture or even influence the market. For example, rankings impose relational positions on potentially similar or incomparable entities, such as in the case of city rankings (Kornberger & Carter, 2010). Rankings of schools can change the market by influencing actors to change their institutions in order to climb the rankings (Espeland & Sauder, 2007). Visual rankings in 2x2 matrices, such as Gartner’s ‘magic quadrant’ that positions competing technology companies by their ability to execute and the completeness of their vision, are displayed as pictures that capture the performance of different IT vendors. However, they can also influence those IT firms and are the result of a negotiation of sorts between companies and the ranker (Pollock & D’Adderio, 2012). More broadly, consultants can sometimes try to fit ideas into existing forms, allowing the shape and structure of presentation to shape finding rather than adapting presentation formats to display complex information. While fitting ideas into implicit or explicit templates can seem harmless, it necessarily excludes information and directs focus toward a specific subset of a problem (Hehenberger, Mair & Metz, 2019).
Regional differences. With the exception of Sun Tzu’s The Art of War (not originally meant for business!), most of the books you will find on lists of ‘influential’ business books are written by Americans or British, writing for liberal market settings and free market economic perspectives. Perhaps googling in another language will change that, but nevertheless business schools across the globe teach based on the same fundamental economic principles adherent to a neo-liberal view (principles increasingly called into question as inequality increases and the environment is destroyed[1]). Though business researchers intend to essentially help clients maintain or create profit, not all businesses are the same or in the same contexts and thus, institutional constraints or hold different values. For example, welfare state settings and institutional frameworks in different regions have influence and are influenced by businesses operating there. So-called ‘varieties of capitalism’ (Hall and Soskice, 2003) is criticized heavily in academia for being overly simplified, yet one rarely, if not ever, sees business research take into consideration that a range of institutions inhibit or enable different types of innovation in different welfare structures (ibid).
There are many great ideas in popular press books and articles. However, much of this work Is intentionally written to have wide appeal and in order to have such appeal, context-specific nuance is lost. On a basic level, it fails to address institutional differences between regions when espousing advice. While managers across the globe are surely aware of the unique context in which they operate, the authors of business advice may not be and understanding this gap is useful for critically assessing whether vendors try to force the style and structure of one way of thinking or are more open to nuances.
Rationalization. A further — and deeper — issue involves unpacking the reasons behind oversimplification, which reach back to Max Weber who identified that bureaucracy and rationality characterize modern organizing and ever-increasing concerns with efficiency and more recently, to liberal market ideology propagated in the 1970s. Practices and approaches are indeed simpler when striving for organizational and market efficiency and profit above all else. Such underlying assumptions often underpin the seductively simple solutions presented by business researchers. It is a lot easier, after all, to suggest mass layoffs and charge firms for help restructuring, than to creatively consider new ways to navigate the market adding nuanced layers of value. In today’s business climate, basic assumptions about organizing are increasingly called into question. Vendors who try to use the same tricks and tools from the last 100 years may not be best equipped to help companies navigate a changing world, changing values and new resource constraints.
Yet, organizations do not always adopt new practices or changes brought on in part by consultants because they are actually more efficient (DiMaggio & Powell, 1983). Some consultants, while applying generic frameworks and approaches, spread practices and norms across heterogeneous groups of organizations in different sectors and geographies. For example, one common practice involves comparing aspects of one organization to others in an industry — such as how much an organization spends on R&D — and encouraging firms to adapt by, for example, cutting R&D expenditures. In general, shaping companies to look and act like each other (known as organizational isomorphism; DiMaggio & Powell, 1983) can confer legitimacy (Deephouse, 1996) — but is not necessarily advantageous to goals or profits and can be counterproductive or unnecessary — variation is not necessarily bad (Deephouse, 1999). Actually, so-called mimetic isomorphism (DiMaggio & Powell, 1983) is a response to uncertainty rather than a strategy. Thus, through consultants using approaches that drive similarities, firms can copy one another without actually intending to. Firms may rather seek vendors who help them think about new ideas that can help them truly differentiate. Rather than using tools to help companies homogenize, vendors can use tools that embed engines of inquiry and think with organizations about their own unique future options so they can cut their own path.
Ensuring quality — and defining quality more broadly
Actionable research is contingent on context, filled with history and understandable in fine-grained nuance rather than generic, broad-strokes. Rigorous business research that adheres to academic quality criteria can help. For credible and trustworthy findings, vendors should use rich and high quality data. Business research projects should not be immediately transferable unless specifically requested and should rather be clearly grounded in the client’s own context and situation. The work should be dependable as evinced by clear and rigorous methods. The work is ideally done without bias and influence, but with researcher reflexivity about their own involvement. Overall, deep, contextually-rich insights rather than generic frameworks imposed onto client projects are likely to contribute to credible, dependable and confirmable research. Solid and transparent processes that involve multiple sources can ensure context-relevant and quality work. This may involve new types of data collection, such as ethnography and more inductive approaches to solution design, including — but not limited to — design thinking. It can also involve including academic literature from the social sciences and humanities, and researchers who actively seek to bring the best insights for their clients.
Furthermore, the over-simplification trend in business research is based on its own history and context and understanding this can help companies critically assess who is best equipped to help them navigate the future of business. We argue, it’s time for a new perspective.
Questions to think about when assessing a vendor:
Do they have your interests in mind? Are they more concerned with their own power and position?
Do they want to involve you to leverage your rich experiences and knowledge?
Is their analysis as well considered as their other research processes?
Are they transparent throughout their engagement with you? Can you trace how they got from the beginning to the conclusions? Can they explain their methodology, including steps to gather and analyze the data?
Where do they get their data? Do they consider multiple sources such as representative groups of real consumers, background academic research, and other rigorous work?
Can your vendor defend and argue for their choice of methods and approaches? Can they as individuals think and reflect on what they are doing? Can they clearly explain why their approach is better than some competing alternatives?
Do they try to sell approaches from the last 100 years of business or do they have a fresh perspective?
Dr. Ashley Metz is a Fellow at the Human Futures Studio and an Assistant Professor at Tilburg University
References
Alvesson, M. (1996). Leadership studies: From procedure and abstraction to reflexivity and situation. The Leadership Quarterly, 459–460, 467.
Alvesson, M. & Skoeldberg, K. (2017). Reflexive Methodology: New Vistas for
Qualitative Research. London: SAGE Publications.
Cunliffe, A. L. (2002). Reflexive dialogical practice in management learning. Management Learning, 33, 35–61.
Deephouse, D. (1996). Does Isomorphism Legitimate? The Academy of Management Journal, 39(4): 1024–1039.
Deephouse, D. (1999). To be different, or to be the same? It’s a question (and theory) of strategic balance. Strategic Management Journal, 20: 147–166.
DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48: 147–160.
Espeland, W.N. & Sauder, M. (2007). Rankings and Reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1): 1–40.
Hall, P.A. & Soskice, D. (2003). Varieties of Capitalism: The Institutional Foundations of Comparative Advantage. Oxford: Oxford.
Hammersley, M. and Atkinson, P. (1995). Ethnography: Principles in Practices, (2nd Edition) London: Routledge.
Hehenberger, L., Mair, J., & Metz, A. (in press). The Assembly of a Field Ideology: An Idea- Centric Perspective on Systemic Power in Impact Investing. The Academy of Management Journal. https://journals.aom.org/doi/10.5465/amj.2017.1402
Humphreys, M. (2005). Getting Personal: Reflexivity and Autoethnographic Vignettes. Qualitative Inquiry, 11: 840–860.
Kornberger, M., & Carter, C. (2010). Manufacturing competition: How accounting practices shape strategy making in cities. Accounting, Auditing & Accountability Journal, 23, 325–349.
Lincoln, YS. & Guba, EG. (1985). Naturalistic Inquiry. Newbury Park, CA: Sage Publications.
Pollock, N., & D’Adderio, L. (2012). Give me a two-by-two matrix and I will create the market: Rankings, graphic visualisations and sociomateriality. Accounting, Organizations and Society, 37, 565–586.
Ripamonti, S., Galuppo, L., Gorli, M., Scaratti, G., & Cunliffe, A.L. (2015). Pushing Action Research Toward Reflexive Practice. Journal of Management Inquiry: 1–14.
[1] See, for example: de Saille. S. & Medvecky, F. 2016. Innovation for a steady state: a case for responsible stagnation. Economy and Society. 45(1) 1–23.