Introduction

Many theories and much of the research conducted in the field of international business (IB) are closely related to theories originating in macro-foundational (e.g., economics, sociology) and well as micro-foundational (e.g., psychology) disciplines (Buckley, Doh, & Benischke, 2017). Similarly, as documented in the edited volume Research Methods in International Business (Eden, Nielsen, & Verbeke, 2020), IB researchers rely on methodological approaches originating in a wide variety of fields. The reason is that IB research addresses transnational operations, multinational firm strategies, and the behavior of individuals within these settings (i.e., micro-foundations of many of the choices and decisions and their relations to organizational outcomes). Thus, IB research examines not only different levels of analysis, but also interactions among these macro-, meso-, and micro-levels given that it encompasses business (e.g., accounting, finance, management, marketing) as well as other disciplines (e.g., economics, political science, psychology).

Given the diversity regarding disciplines and levels of analysis, contemporary IB research faces two broad challenges, which also serve as opportunities: (a) quicker access to massive amounts of both numerical and text-based data (e.g., firm websites, social media sites), and (b) new software and data-analytic developments. Certainly, these developments are not specific to IB research alone but are also happening in other fields such as management, medicine, and engineering.

Despite its interdisciplinary and multilevel orientation, and the emergence of new challenges and opportunities, we believe that an important constant across IB research domains is that an ounce of methodological prevention is worth a pound of cure (cf. Aguinis & Vandenberg, 2014). Stated differently, although it may be possible to improve a study and its contributions to IB theory by conducting additional analyses, it is difficult, if not impossible, to turn a methodologically inferior study into a methodologically sound one if certain issues were not addressed prior to data collection. In fact, these problems are often the cause of desk-rejection decisions. Accordingly, the specific methodological challenges we address – and associated solutions that constitute features of sound methodology – refer to the pre-data collection stages of research.

Clearly, we are not the first to discuss methodological challenges in IB (e.g., Eden et al., 2020; Nielsen, Eden, et al., 2020a; see Bergh, 2021, for a review). In fact, articles published in Journal of International Business Studies (JIBS) have addressed how to make IB research more reproducible and trustworthy (Aguinis, Cascio, & Ramani, 2017), how to improve the use of and take more advantage of qualitative methods (Birkinshaw, Brannen, & Tung, 2011), and how to gain new insights from the unique features of social network analysis (Cuypers, Ertug, Cantwell, Zaheer, & Kilduff, 2020), among many others. Indeed, most JIBS volumes feature at least one article addressing some methodological matter (Bergh, 2021). In addition, some of the challenges we describe in our article are not necessarily unique to IB research. For example, Shaver (2021) highlighted methodological challenges in strategic management research such as the need to strengthen causal inferences and the use of multi-method approaches for advancing theory, which are issues we address in our article. But, these and other challenges are particularly relevant and, moreover, exacerbated in IB research because of the field’s diversity regarding disciplines and particular interest in developing and testing theory about different levels as well as relations between levels (e.g., individuals and firms, firms and industries, individuals and culture).

Within the framework that an ounce of methodological prevention is worth a pound of cure, our article makes value-added contributions given four features. First, complementing past specific methodological contributions published in JIBS, rather than focusing on a specific approach or technique, we discuss broad challenges that are relevant and applicable for IB theory advancements more generally. Accordingly, we offer insights for a broad range of IB researchers – not just those interested in a particular method or relying on a particular ontological perspective. Second, most of our recommendations and specific courses of action can be implemented concurrently or separately. Thus, best-in-kind IB research and theory advancements will result from the implementation of as many of these sound practices as possible. Third, we offer insights and recommendations on how to achieve best-in-kind IB methodology specifically based on lessons learned from the organizational research literature. Clearly, not all organizational research is directly relevant to IB. However, we point to methodological bridges between IB and organizational research because we believe that making IB’s methodological borders more permeable is likely to result in benefits not only for IB but also for adjacent fields such as organizational behavior, strategic management studies, human resource management, and entrepreneurship (Aguinis & Gabriel, 2022). A benefit of building such bridges is that scholars from fields outside of IB may find it relevant and attractive to submit their research for publication consideration in JIBS and other IB journals, thereby increasing the diversity of perspectives. Finally, the challenges and solutions we offer are particularly relevant and useful for IB research given its heterogeneity regarding substantive domains, levels of analysis, and methodological and ontological orientations and we illustrate this feature using examples from across several IB domains.

Methodological challenges and features of sound IB methodology

We focus on the following five challenges. As a preview, a summary of challenges and recommendations as well as implementation guidelines on how to address these challenges to achieve sound methodology are included in Table 1.

  1. 1.

    Researching an important and relevant issue: How can IB researchers identify and study issues of importance not only for theory but also for practice and policy-making?

  2. 2.

    Making meaningful theoretical progress: How can IB researchers accelerate theoretical progress?

  3. 3.

    Recognizing, anticipating, and resolving dilemmas in research design and execution decisions: How can IB researchers solve dilemmas and tradeoffs when designing and executing empirical research?

  4. 4.

    Integrating quantitative and qualitative research by using mixed methods: How can IB researchers integrate, rather than stack, quantitative and qualitative methods by using mixed methods?

  5. 5.

    Reducing the “distal proxy fallacy” through measurement error management: How can IB researchers reduce the distance between constructs and their proxies when creating measures and use Cronbach’s alpha and other approaches for narrowing this gap?

Table 1 Summary of challenges and recommendations as well as implementation guidelines for achieving sound methodology
Full size table

Researching an important and relevant issue

There is a documented disconnect between the knowledge that IB academics are producing and the knowledge that managers and policy-makers are consuming (Grøgaard, Sartor, & Rademaker, 2022). As noted by Grøgaard et al., (2022: 1512), “it is common to find only a cursory consideration of a study’s practical implications in the discussion section of published IB papers… most published research seems to be primarily intended for an academic audience.” We believe that an important reason for this so-called research-practice gap is that, frankly, our research often does not address important and relevant issues for society. It is very encouraging that JIBS now includes a Societal Impact Advisory Committee (SIAC), whose mission is to support authors, editors, reviewers, and users of IB knowledge in strengthening the discussion of the societal impact of research (Tung, 2023). Specifically, SIAC’s role is to strengthen articles’ (a) practical implications for business or society (i.e., policy or practice) and/or (b) have impact on stakeholders beyond shareholders.

Clearly, not all IB research is supposed to have direct and immediate application and there is a need to produce research that will lead to basic and theoretical knowledge without direct and short-term applicability. But, we also believe that if the bulk of IB research falls into that category, then IB is unlikely to achieve our collective and lofty goal expressed in the mission of the Academy of International Business, which is “dedicated to promoting impactful research, improving business education and practice, and collaborating with leaders in policy and interdisciplinary research” (Academy of International Business, 2022). In fact, there is a close association between the Academy of international Business (AIB) and the World Investment Report (WIR) produced by the United Nations Conference on Trade and Development (UNCTAD), and this is the reason why the annual AIB meetings include a plenary session devoted to the publication and discussion of each year’s WIR.

How can IB researchers identify and study issues of importance for both theory and practice? We offer two features of sound IB methodology to answer this question. First, there is a need to address important questions. For junior researchers, this can be accomplished by becoming exposed to many ideas and topics. One way of doing so is to be involved in as many research projects as possible (beginning in graduate school) and volunteering as a reviewer for conferences. More experienced scholars can volunteer as reviewers for journals and, eventually, as editorial board members. But, exposure to other research and other researchers is only the first step. It is also important to engage in “bridging” activities with the world of IB practice (Grøgaard et al., 2022; Shapiro, Kirkman, & Courtney, 2007). Consider the following three possibilities likely to result in researching important and relevant issues: (a) becoming involved in senior managerial decision-making by serving on boards of directors and leadership positions in global organizations, (b) serving as an instructor in executive education courses, and (c) spending a sabbatical in business practice either as a translator of existing research or researcher on teams including both researchers and practitioners.

Second, consider conducting research with a practical and possibly policy-making end in mind (Aguinis & Vandenberg, 2014). This can be accomplished by adopting a design-science approach as proposed by Simon (1996 [1969]). This approach to doing research involves understanding the present, but also aims to create better futures. For example, in the field of medicine, a design-science approach involves restoring health to a patient suffering from cancer. In the case of IB, as just two examples, a design-science approach may involve producing knowledge to improve the effectiveness of a firm’s global talent management system, or producing knowledge aimed at improving the effectiveness of firms’ entry strategy in a different country. Sound IB methodology based on a design-science approach means studying dependent variables that are of interest to decision-makers (e.g., the attraction and retention of global talent, effective foreign market entry) and independent variables that can be changed by instituting new practices and policies (e.g., improved use of recruiting websites, improved practices regarding cross-cultural management). Although leading to new knowledge, if independent variables are outside of the control of decision-makers and policy-makers, it is unlikely that the resulting knowledge will be of use to them.

Making Meaningful Theoretical Progress

There are important challenges in IB regarding the way we usually test our theories and the overall slow degree of theory advancement. For example, one challenge is the lack of precision in hypothesis testing, given that the majority of hypotheses simply predict that an increase in one variable will covary with an increase or decrease in another. This type of hypothesis is common and two illustrations of hypotheses tested in recently published JIBS articles are the following:1

Hypothesis 1:

The extent of women representation on the board of directors is positively related to corporate social performance.

Hypothesis 2:

An initial public offering has a positive effect on the pace at which small- and medium-sized enterprises establish new foreign subsidiaries.

These are broad and general relations that can often be easily confirmed given a sufficiently large sample size (Edwards & Christian, 2014). As a second illustrative challenge about the need to make meaningful theoretical progress, hypothesizing after results are known (i.e., HARKing) systematically capitalizes on chance. HARKing not only delays, but also derails, theory advancements (Aguinis et al., 2017; Murphy & Aguinis, 2019). For example, cherry-picking involves searching through data involving alternative measures or samples to find the results that offer the strongest possible support for a particular hypothesis. Another particularly pernicious form of HARKing is question trolling, which involves searching through data involving several different constructs and measures to find seemingly notable results worth writing about.

What does sound IB methodology look like if we would like to make meaningful theoretical progress? We offer two complementary courses of action. First, there is a need to develop theories with hypotheses that are more precise in describing the nature of the effect rather than merely stating that an effect or relation will differ from zero or that higher/lower values on variable X are related to higher/lower values on variable Y. In other words, sound IB methodology exposes theories to stringent tests that put them at genuine risk of falsification. To do so, the wording of hypotheses can be such that they include lower and upper bounds for effects (e.g., based on previous research or meta-analyses), non-nil effects, zones of indifference for effects, competitive predictions (i.e., strong inference), and a “good enough” belt for effects. To return to Hypothesis 1 above, a more precise hypothesis leading to improved theory might be the following:

Hypothesis 1a:

The presence of [precise number or proportion] of women on the board results in an increase of corporate social performance of [specific] %.

In addition, it is important for hypotheses to specify the functional form of the predicted relation between variables. For example, the nature of the relation could be linear or curvilinear, in which case the prediction should also include the location of the inflection point. To follow-up on this same example, a more precise version of the same hypothesis would be the following:

Hypothesis 1b:

The presence of [precise number or proportion] of women on the board results in an inflection point such that its impact on corporate social performance becomes [asymptotic or negative].

On a related note, results of hypothesis testing should go beyond simply reporting p values and include effect-size estimates and their confidence intervals, as well as an interpretation of the meaning and significance of the reported effect for theory and practice (Meyer, van Witteloostuijn, & Beugelsdijk, 2017).

Second, sound IB methodology ideally allows for inferences about causal relations. Shaver (2021) highlighted research design choices that can help advance causal identification, noting that a recent trend in the field of strategic management studies is to conduct experiments. Moreover, field experiments are also becoming more popular because they provide opportunities to identify previously unconsidered relations or examine existing relations in new ways (Eden, 2017; Withers & Li, 2021). Research design weaknesses and unclear causal relations are the #3 and #4 most frequent methodological challenges reported by authors of JIBS articles, respectively (Aguinis, Ramani, & Cascio, 2020). But, although randomized experiments are the gold standard for science (Lonati, Quiroga, Zehnder, & Antonakis, 2018), they are difficult, if not impossible, in many IB domains because of the inability to manipulate variables such as internationalization decisions, risk assessments, or buy-or-build decisions for large multinational enterprises. Also, regarding micro-foundations of IB, although some managers may be willing to participate in experiments involving simulations or vignette studies, there are concerns about external validity. Nevertheless, sound IB research can provide stronger evidence about causal relations by incorporating design features that have the potential to turn what otherwise might be a nonexperimental study into a quasi-experiment (Grant & Wall, 2009). As yet another feature of sound IB methodology to address causality, latent class tools such as latent profile analysis can be used to identify groups of firms (or group of individuals, teams, or industries) at different levels of the predictors and subsequently use this distinction as a manipulation to evaluate whether the outcome variables differ in the expected direction between the groups.

We also recommend addressing endogenous threats in research design to anticipate misleading conclusions about causal relations. This topic has received attention in IB research, spanning sampling decisions, experimental design variations, matching models, instrumental variables, 2-stage analytical techniques, and longitudinal data structures (see Li, Ding, Hu, & Wan, 2021; Reeb, Sakakibara, & Mahmood, 2012). We highlight two additional issues. First, all approaches to causal identification have limitations, so there is a need to develop a cumulative body of research that includes a plurality of approaches (as we explain further below). In addition, we call on IB researchers to attend to alternative theoretical mechanisms to help rule out competing explanations. Second, we recommend that IB researchers report quantitative evidence of possible omitted variable bias by implementing the “Impact Threshold of Confounding Variables” (ITCV) procedure (Busenbark, Yoon, Gamache, & Withers, 2022). This test provides insights into how omitted variables can directly bias causal inference. Specifically, it provides information on (a) the minimum correlation an omitted variable would need to have with both an explanatory variable and the dependent variable to invalidate a finding and (b) how many cases (and the overall percentage) of a predictor variable would need to be replaced with a value of zero for the coefficient of interest to become statistically non-significant. If the correlational value in (a) is larger than any others in a study relative to both the independent and dependent variables and if the number (percentage) of cases in (b) needed to become zero is outside the likelihood of chance, then the researcher can conclude that omitted variables pose little risk to the relationship. Overall, ITCV is a sensitivity analysis that provides objective information about the likelihood of omitted variable bias.

Recognizing, Anticipating, and Solving Dilemmas in Research Design and Execution Decisions

The availability of data regarding transnational operations, multinational firm strategies, and the behavior of individuals within these settings (i.e., micro-foundations of many of the choices and decisions and their relations to organizational outcomes) is going through a major transformation. Specifically, records of these topics are now available through “big data” sources that give IB researchers quick access to massive amounts of information. In addition, IB researchers also have access to increasingly sophisticated and new analytical techniques that allow them to parse and glean insights previously unimaginable. These trends, and others, represent significant opportunities for IB researchers to “take advantage of untapped sources of information and to re-analyze currently available data” (Aguinis et al., 2020: 1593). However, these developments also led Nielsen, Eden, et al. (2020: 32) to warn that “[t]echnology is a powerful aid in research, but IB scholars should strive for methodological parsimony.” We agree, noting that as exciting as recent methodological developments may seem, they have implications that IB researchers need to understand and leverage in their research agendas. Indeed, a reminder of tradeoffs in research design choices is warranted.

Clearly, social science research is imperfect. To help understand the origins of the inevitable challenges in a social science such as IB, and how we can possibly address them, it is instructive to review the classic article by McGrath (1981). He identified intractable tradeoffs and how we can learn to live with them. Our discussion in this section provides additional justification for Nielsen et al.'s ( 2020b) observation that “[R]igor requires a holistic approach that integrates multiple methodological elements together in a way that best suits the entire research design….rigor in IB research need[s] to be recalibrated to acknowledge the importance of building in triangulation to strengthen research designs [italics added], a strategy that we found has received little explicit recognition so far” (Nielsen, Welch, et al., 2020: 1492)

Applying what he called the “dilemmatic approach,” McGrath argued that the research process includes a series of interlocking decisions in which scholars try to maximize several conflicting desiderata. Research choices involve a series of mutually incompatible goals, leading McGrath to assert that there is no “one true method or set of methodological choices that will guarantee success…no one ‘best’ strategy or set of choices…all research strategies and methods are seriously flawed” (McGrath, 1981: 179). Several important tradeoffs, or dilemmas, exist. We draw attention to these matters to improve our understanding of the limitations in how knowledge is created and how we can help contribute despite the vulnerabilities in our own work.

First, starting with the most general, there are different types of research designs that can be arranged into four groups including (1) field studies and field experiments, (2) laboratory experiments and experimental simulations, (3) judgment tasks and sample surveys, and (4) formal theory and computer simulations. McGrath then noted that these four can be evaluated relative to two dimensions: obtrusive versus unobtrusive and universal versus particular. Based on this categorization, he identified “three horns” that pertain to each group of methodological strategies: (a) generalizability with respect to populations; (b) precision with respect to measurement, manipulation, and control; and (c) realism of context. These horns represent opposing or conflicting features afflicting empirical research which  a single study cannot maximize all three at the same time. McGrath explained that “the research strategy domain is a three-horned dilemma, and every research strategy either avoids two horns by an uneasy compromise but gets impaled, to the hilt, on the third horn; or grabs the dilemma boldly by one horn, maximizing on it, but at the same time ‘sitting down’ (with some pain) on the other two horns” (McGrath, 1981: 184). The strength of one research strategy is the weakness of another. McGrath opined in a summary, “[i]t is not possible, in principle, to do an unflawed study, or fantasize, if you will, about lying in clover; but be prepared to awake on a bed of horns” (McGrath, 1981: 186). The implication is that knowledge development within a research stream needs a plurality of research strategies to develop insights that span the three horns. If a research stream has developed using the same research strategies, then all the studies suffer from the same general weaknesses. Methods that compensate for each other’s vulnerabilities offer the most promise for advancing theory (Bergh, Boyd, Byron, Gove, & Ketchen, 2022). Researchers might therefore position the strengths of their strategies relative to others that have already appeared within their chosen literature.

Second, research designs also involve dilemmas, especially between information, noise, and uncertainty. In particular, tradeoffs exist between scope (i.e., amount of information in the problem) and precision (i.e., amount of noise in the information). For example, if a particular design reduces uncertainty to help gain precision, such as through controls within an experiment, then it reduces the possible combinations of events and observations that can occur. In turn, there is less information that can be yielded from the variables, meaning that we find out less about our studied relations. In addition, as mentioned earlier, randomized experiments may not be the answer. Although they are useful for untangling relations that may have causal connections, they suffer from important logistical challenges, they do not necessarily address all threats to internal validity, and by the very nature of experimental manipulation they might exaggerate the size of effects. McGrath depicted these tradeoffs as conflicts between standardization to gain precision by reducing noise and generalizability to gain confidence in strength and robustness.

So, IB researchers are left to accept the harsh reality that no single study is perfect and each has built-in flaws. If the scope of our studies is wide and we employ big data sources with apparently little or no constraints, we may be tempted to conclude that we have captured their topic fully. However, we accept noise, suffer a lack of measurement precision, and learn only a little about something very large. Alternatively, if we reduce noise by cutting scope, we learn more about less. The implications are stark: when making design choices, IB researchers face conflicts between information and noise through the tradeoffs between scope and precision. Thus, big data and other technological developments may not be the panacea that some suggest they are.

Overall, McGrath’s insights help us understand research design and execution decisions: the research process “teems” with dilemmas involving two or even three contradictory objectives and there is no single solution that allows all three objectives to be accomplished simultaneously. We cannot conduct flawless research and, therefore, combining a plurality of strategies and designs is the way to overcome or at least lessen the inherent limitations in each. Indeed, McGrath exhorted researchers to consider multiple methods approaches: “The dilemmas can be handled by… bowling them over with multiple methods… multiple designs… multiple strategies, to gain information about the research problems of concern… ‘good research’—using flawed methods well, and in effective combinations—can help us accrue ‘knowledge’ … that are of both theoretical and practical concern” (McGrath, 1981: 209–210).

Integrating Quantitative and Qualitative Research by using Mixed Methods

Based on an examination of empirical studies published in JIBS over its entire 50-year lifetime, Nielsen, Welch, et al. (2020) reported that most articles (86.7%) used quantitative methods. So, although the IB community has been a pioneer in fostering the use of mixed-methods research (Hurmerinta-Peltomäki & Nummela, 2006), this methodological approach is clearly underutilized. We reached this same conclusion after reviewing the references lists of the 35 mixed-methods studies published in JIBS from 2000 to 2019 identified by Nielsen, Welch, et al. (2020). Specifically, we found that only four studies actually included some methodological reference to mixed methods. As noted in the previous section about dilemmas in research decisions, combining research strategies may help overcome inherent and unavoidable limitations and tradeoffs. Moreover, IB methodology can improve its soundness by using mixed methods because of its role in producing responsible research (Tsui & McKiernan, 2022): research that is simultaneously (a) rigorous and credible and (b) relevant and useful for society.

With regards to rigorous and credible research, the main premise of mixed methods is that the use of quantitative and qualitative approaches in combination provides a better understanding of phenomena compared to either approach by itself. But, this combination should not be a simple stacking of separate quantitative and qualitative components. Rather, the key point is to take advantage of the full potential of mixed methods by integration. In fact, integration is the central defining characteristic of mixed methods research. It is this specific and unique feature that prompted Fetters and Freshwater (2015) to conclude that 1 + 1 = 3, reflecting on the idea that conducting mixed methods using a fully integrated approach yields a whole greater than the sum of the individual quantitative and qualitative parts.

An important feature of sound mixed-methods studies is transparency about precisely how integration has been achieved. Specifically, Fetters, Curry and Creswell (2013) described three possible types of integration: (a) study design (i.e., examining integration in three basic mixed methods designs: exploratory sequential, explanatory sequential, and convergent); (b) methods (i.e., considering four approaches: connecting, building, merging, and embedding); and (c) interpretation and reporting (e.g., integration through narrative and data transformation).

Together with recommendations on how to achieve better integration, O’Cathain (2010) provided a set of criteria for evaluating the quality of mixed methods studies, and Fabregues and Molina-Azorin (2017) reviewed the literature about quality in mixed methods and summarized available criteria. For example, some of these criteria include (a) planning quality (i.e., the extent to which the study plan is feasible and the purpose(s) and rationale for using mixed methods are appropriate), (b) design quality (i.e., the extent to which the mixed methods design is described in detail, suitable for the research question(s) and employs quantitative and qualitative methods that complement each other), and (c) methods and data quality (i.e., the extent to which sampling, data collection, analysis and integration of the qualitative and quantitative parts are appropriate, rigorous, and described with transparency) (Creswell & Plano Clark, 2018).

As mentioned above, using mixed methods research can help integrate rigor and relevance. Specifically in IB, Buckley et al. (2017) proposed a redirection of research toward grand challenges, emphasizing methodological implications: the need of phenomena-based research, the use of interdisciplinary research methods, the application of multilevel research, and the use of research involving interactions among business, government, and society in a global environment. Mixed methods are well positioned to address each of these challenges. In addition, involvement of stakeholders (e.g., managers, NGOs, policy-makers) in the research process, as well as the dissemination of research results, is a key aspect for relevant research. A mixed methods approach, through the qualitative part, may facilitate this involvement with the goal of identifying and addressing key research problems and questions regarding grand societal challenges.

Reducing the “Distal Proxy Fallacy” through Measurement Error Management

Challenges about measurement have been reported by 73% of authors of JIBS articles (Aguinis et al., 2020). This result should not be surprising given that some of the most central measures in the field such as the Kogut and Singh’s (1988) cultural distance index have resulted in significant debate and disagreement (Beugelsdijk, Kostova, Kunst, Spadafora, & van Essen, 2018; Cuypers, Ertug, Heugens, Kogut, & Zou, 2018). Other sources of IB measurement challenges include non-equivalence of data sources owing to cross-cultural settings and translations from one language to another. These and other sources of error direct attention to important needs: (a) Critically assessing the fit between theoretical constructs and their empirical proxies, and (b) ensuring the appropriateness of measures. For example, IB researchers often “retrofit” a construct to fit a data source that is available to them, and as a result, may introduce error into their measurement. We offer several recommendations to address these needs.

First, we suggest that researchers pay particularly careful attention to the meaning and definition of constructs. Podsakoff, MacKenzie, and Podsakoff (2016) argued that a lack of conceptual clarity can lead to a cascade of problems, not least of which is measurement error, and provided several recommendations in the form of a four-step process for developing conceptual definitions that can lead to a closer relation between IB constructs and their proxies:

  1. (1)

    Identify potential attributes of the construct and/or collect a representative set of definitions. Such activities encompass searching dictionaries, surveying the literature, interviewing subject-matter experts, consulting focus-groups, and case studies.

  2. (2)

    Organize the potential attributes by theme and identify any necessary and sufficient ones.

  3. (3)

    Develop a preliminary definition of the construct.

  4. (4)

    Refine the definition by introducing greater clarity and parsimony.

Second, we typically use a variety of procedures to assess the validity and reliability of our measures to understand whether they reflect their intended underlying constructs and how consistently they do so. While construct validity in IB research has been examined elsewhere (Aguinis et al., 2020), less attention has been devoted to reliability (e.g., consistency of scores across items in a scale, consistency of variables across databases). We propose that the “distal proxy fallacy” in IB research can be fruitfully addressed by paying more careful attention to how we manage measurement error through recognizing that there is not a “one size fits all” approach to knowing the extent to which measures are reliable (e.g., scores do not fluctuate due to random or measurement error).

More specifically, reliability is typically assessed using Cronbach’s alpha (CA; Cronbach, 1951). In fact, Cho and Kim noted that CA may be “regarded as the best available reliability coefficient because so many researchers use it in practice” (2015: 208). Moreover, CA levels of 0.70 have become institutionalized as evidence to confirm the reliability of measures (Lance, Butts, & Michels, 2006). We used the Business Source Ultimate database with the phrases “Cronbach alpha” and “Journal of International Business Studies” and, not surprisingly, found 87 hits in the past decade alone. However, CA is often “not the most accurate reliability coefficient (i.e., it is overused)” (Cho, 2016: 642). Its accuracy hinges on satisfying several conditions which, if violated, result in “estimates of reliability that are too small, making measures look less reliable than they actually are” (McNeish, 2018: 412). Indeed, several scholars have examined the history, conditions, and assumptions of CA (Cho, 2016; Cho & Kim, 2015; Schmidt & Hunter, 1996) and concluded that it may have more limited application than IB researchers usually recognize. We review these matters to guide future IB research on when it is, and is not appropriate, to select CA, and when we should use a different reliability estimate.

First, CA requires “tau equivalence” among items of a scale, which occurs when items are linearly related and differ only by a constant. In other words, tau equivalence means that all items of a scale contribute equally to the total scale score, which is an assumption likely unmet in most IB measures ranging from cultural distance to innovation and liabilities of outsidership – just to mention a few. Second, scale items should be on a continuous scale and normally distributed – another assumption likely violated in most IB research. For example, the assumption that firm performance is normally distributed is untenable in most industries and especially in the context of international firms. Third, the errors of the terms should not covary (uncorrelated errors) (McNeish, 2018). In addition, the unquestioned used of CA is based on several misconceptions, including that CA was first developed by Cronbach (it was not), CA equals reliability (i.e., it is an imperfect index of one specific source of error only), reliability can be improved by deleting items, CA is deemed sufficient if it is greater than or equal to 0.70, and CA is the best choice among all reliability coefficients (Cho & Kim, 2015). Overall, the use of CA should not be an “unconditional and automatic choice for reliability estimation” (Cho & Kim, 2015: 219). Thus, IB research should not rely on CA without careful justification because the requirements for CA are often difficult to satisfy and the information it yields is often misleading.

Fortunately, IB researchers have several alternatives to CA. Cho and Kim recommended structural equation modeling (SEM)-based reliability estimates, especially when the data include a multiple-factor structure. In addition, McNeish (2018) identified alternatives including omega coefficients and coefficient H. Each of these alternatives offers strengths and weaknesses and can be used based on specific design and measurement conditions. Because SEM may not be suitable for all situations, we focus our attention on reliability estimates other than CA.

Omega coefficients are especially attractive to IB research, in particular because they are best for “congeneric scales…where items vary in how strongly they are related to the construct being measured (i.e., in a factor analysis setting, the loadings would not be assumed to be equal). In other words, where tau equivalence is not assumed” (McNeish, 2018: 416). For example, when examining the reliability of the Kogut–Singh cultural distance index, confidence in a CA value would require that the items meet tau equivalence, otherwise omega becomes the more accurate estimate. Alternatively, the multiple dimensional element may invite Cho and Kim’s SEM reliability-based estimates instead (e.g. Cuypers et al., 2020). Surprisingly, these issues have received less attention than one would expect for such an important and central index used across so many IB domains and theories.

In addition, three variations of omega exist, including omega hierarchical, omega total, and Revelle’s omega total. Each applies to different settings. Coefficient H may be the most applicable to IB research, as it provides an alternative that researchers can use if they are interested in creating a scale with optimally weighted items. This approach, also known as maximal reliability, occurs when “each item contributes different amounts of information to the overall scale score (instead of each item being given the same weight with unit weighting)” (McNeish, 2018: 417).

Concluding remarks

IB researchers rely on theories and methodological approaches originating in a wide variety of micro- and macro-fields given IB’s interdisciplinary and multilevel nature. As a result, methodological challenges faced by IB researchers are quite diverse. However, in spite of the apparent differences, we see a common way to address pervasive methodological challenges, improve methodological soundness, and accelerate theoretical progress: An ounce of methodological prevention is worth a pound of cure. In other words, it is critical to anticipate and address methodological challenges prior to data collection. We described IB challenges and proposed solutions regarding the following issues: (1) researching an important and relevant issue, (2) making meaningful theoretical progress, (3) recognizing and solving dilemmas in research design and execution decisions, (4) integrating quantitative and qualitative research by using mixed methods, and (5) reducing the distal proxy fallacy through measurement error management. We offered specific and actionable recommendations for addressing each of these challenges with the goal of producing sound methodology. Irrespective of substantive domain, all scholars share the ambition to improve the understanding of IB phenomena including transnational operations, multinational firm strategies, and the behavior of individuals within these settings. Accordingly, we offered insights for a broad range of IB researchers – not just those interested in a particular methodological or ontological approach. We hope the concurrent implementation of these recommendations will be useful for authors as well as journal editors and reviewers given our collective and lofty goal of advancing IB theory, its application, and societal impact.

Notes

  1. 1

    Most readers will recognize the customary manner in which these hypotheses are formulated and we therefore do not see the need to “point fingers” by revealing their sources. But, we make this information available upon request.