RESEARCH ISSUESSHAWNEE VICKERY, Feature Editor, Eli Broad Graduate School of Management, Michigan State University WHAT IS EMPIRICAL RESEARCH?by Jack R. Meredith, University of Cincinnati In the March 1994 issue of DECISION LINE, Joseph Carter wrote a nice article--"The Value of Empirical Research"--pleading for more empirical research in the field of Operations Management. I certainly agree with Joe's desire to see more empirical work conducted and published. Moreover, Joe made some rather intriguing statements along the way that I think are worth exploring further. Interestingly, even though the title of Joe's article included the phrase "empirical research," he never used nor defined this phrase in the article. In fact, the word empirical was only used once, if my fingers are right, and that was in the fifth paragraph in the phrase "empirical investigation." This particular paragraph actually had a number of interesting statements in it that I would like to comment on, in addition to the one about empirical investigation, in the process of exploring the meaning of the phrase "empirical research." For example, the first sentence in that paragraph states: "This is not to suggest that theoretical research is without value." Joe seems to imply here that empirical research is in some sense the opposite of "theoretical research." Since the word "theory" usually means "a coherent group of general propositions used as principles of explanation," isn't all of our research oriented toward generating or proving theory? (An interesting question in itself, to which I will soon return.) Actually, here I believe Joe means theoretical in its lay sense of "not having to do with the practical"; in other words, "abstract" or perhaps even "academic" (in the increasingly common sense of "irrelevant"). Joe's comparison of theoretical with empirical is similar in some ways to the distinction we make between basic and applied research. Thus, empirical in this sense has something to do with practice or application. Then, in the following sentence, Joe continues "all good research has a solid theoretical basis." Well now, this brings up a number of interesting issues. For example, a classic confrontation between the theory builders and the theory testers among us centers on the possibility of conducting truly "grounded research" (Glauser and Strauss 1967): letting a situation or phenomenon reveal its patterns and theory to the researcher, unencumbered by the researchers preconceptions and beliefs. Yet, theory testers would contend that the researcher must have had some theory in mind to even have selected a particular phenomenon for study, as well as which aspects (variables) of the phenomenon to observe. Thus, they would maintain that some pre-existing theory did exist and the researcher was simply "testing" a preconceived theory, not building one. However, I know that when I'm immersed in the fascinating details of a research case site study, what seems to drive me is not some preconceived theory but an irritating inconsistency or nagging intrigue about the situation that I can't seem to ignore. Thus, I seriously question Joe's statement that all good research has a solid theoretical basis. But the issue of theory testing versus theory building deserves further consideration. For example, many researchers believe that research is comprised almost entirely of theory testing and that proactive theory building constitutes only a minor role in the process. Their belief is actually a bit more convoluted than this, however, because they seem to believe that their theory testing research is actually theory building: If I can disprove a theoretical hypothesis, am I not contributing to our understanding and thus helping to evolve a better theory? But where are the hypotheses being disproved supposed to come from? Apparently, from intuition on the researcher's part, or reading and hearing about the phenomenon, and perhaps even some experience. But that experience is not itself considered by most theory-testers to constitute "research," which is why they feel that "empirical" research is weak. And indeed, they certainly do not conduct it in the rigorous, careful fashion that would qualify it as legitimate theory-building research. However, this is how theory building should be done. Personally, I think the most interesting research is concerned with theory building, not theory testing. Thus, good theory building is based not on intuition or hearsay, but on rigorous, careful, practical experience; that is, "empirical research." Moreover, in my view (Meredith 1993), the repeating research cycle of description, explanation, testing, description, etc., in building theory does indeed include experience, and description of the phenomenon of interest through that experience is certainly legitimate research. How can we possibly form hypotheses, frameworks, typologies, or even simple taxonomies without descriptions based on experience? Our journals tend not to publish pure descriptions of interesting (research-wise) phenomena, but I believe that we, as researchers, are poorer and placed at a greater disadvantage in constructing useful theory because of this proclivity. Back to Joe--in the next sentence, he says "all good research should be validated through empirical investigation." Now, there's an interesting word: "validated." According to Webster's, validated means confirmed, substantiated, corroborated or supported on a sound basis, well-grounded, justifiable. Unfortunately, there are about a dozen types of validity that researchers like to talk about: internal, face, statistical, construct, and so on. The two that relate most closely to what Joe is talking about are internal and external. Internal validity concerns the verification of the internal consistency and coherence of the framework, theory, or constructs being created or tested. External validity concerns the ability to apply the theory to situations beyond those used to generate the theory. Confirming that the theory is correct--that is, validating it--should include both these types of validity so that we know the limits of the theory's applicability. These limits are best determined by comparing the theory to reality. That is, to reiterate the cycle of research and return to description through experience, or as Joe states, "empirical investigation." And now we come to that most interesting word in Joe's sentence: "empirical." Unfortunately, the term "empirical research" seems to have become equated by many in OM to survey research based on responses from questionnaires sent to, typically, managers. Actually, this type of research is only a small portion of all the different methods of conducting empirical research: case, field study, grounded research, ethnography, interviews, hermeneutics, field experiments, action research, and so on. According to Webster's, the term empirical means "through experience, observation, or direct sense." Thus, using a manager's perceptions of a phenomenon (such as a committee decision to take some action or the amount of inventory currently in process) would be second source data, rather than primary (i.e., direct sense) data. This is not to say that questionnaire research isn't valuable or useful. Indeed, questionnaire-based research is generally closer to natural reality than our more commonly published artificial reconstructions of object reality. However, the issue of research conducted on empirical data (second source) versus empirical research based on the researcher's own (primary source) experience, observation, or direct sense is an important one worth pursuing. The former is most appropriate for theory-testing research. Although theory-building research can also be based on second source data (often called "data dredging" or "fishing"), it is, I believe, less effective for this purpose. Moreover, the temptation to then turn around and test the newborn theory on the very same data is almost irresistible for most researchers. On the other hand, empirical research based on primary data is equally useful for either theory building or theory testing (although not both at the same time, of course). But given the weakness of data-dredging for theory building, the utility of primary data for this purpose becomes critically important. Moreover, the temptation to try to then reuse the primary data for theory-testing is minimal, another advantage for empirical research. And last, I wanted to comment on a particularly interesting statement in Joe's seventh paragraph: "research findings that never find their way into managerial practice are worthless." This is similar to Galliers and Land's statement (1987, p. 901) that "Information systems is an applied field. Research with no applicability is irrelevant." Similarly, operations management is an applied field, not a pure science. But let's look at Joe's statement a little closer. Some important words in his statement are "never" and "worthless." Never is a long time; must we wait forever to determine what should be published today? Clearly, in order to apply this criterion, someone has to make a personal projection of the likelihood and timing of the future managerial utility of the research. Research that promises to be highly valuable in a year or so would probably pass. Research that appears to hold litttle chance of ever being useful would probably be rejected. Where do we draw the line? But perhaps of even more importance is the term "worthless"-- worthless to whom? I'm afraid I must admit that research by A on a topic that is only of interest to A and B but is reviewed and recommended for publication by B, and vice-versa, is highly worthwhile to two people, A and B, because a continuation of this process will probably bring them tenure. But what about the rest of us? Clearly, the research must be worthwhile to the OM communityþacademics and practitioners alike. Research that is worthwhile to only academics in an applied field like OM is self-serving, so we must make sure that our research is worthwhile to practitioners (and in the relatively near term!) as well. And what type of research holds the most promise to offer this? As Joe stated, and I strongly concur: empirical research! References Galliers, R.D. and Land, F.E. Choosing appropriate information system research methodologies, Communications of the ACM, Nov. 1987, pp. 900-902. Glaser, B.G. and Strauss, A.L. The discovery of grounded theory: Strategies for qualitative research, New York: Aldine de Gruyter, 1967. Meredith, J.R. Theory building through conceptual methods, International Journal of Operations and Production Management, Vol. 13, No. 5, 1993, pp. 3-11.
JACK R. MEREDITH is currently Professor of Operations and Industrial Management at the University of Cincinnati with research interests in technology management and research methodology. He currently serves as editor-in-chief of the Journal of Operations Management. Starting with the 1995 academic year, he has accepted an appointment as the Broyhill Chaired Scholar of Operations at the Babcock Graduate School of Management at Wake Forest University in Winston-Salem, North Carolina.
Dr. Shawnee Vickery |