We use the case of the impact of natural disasters to analyse strengths and weaknesses of meta-analysis in an emerging research field. Macroeconomists have published on this issue since 2002 (we identified 22 studies to date). The results of the studies are contradictory and therefore the need to synthesize the available research is evident. Meta-analysis is a useful method in this field. First, we observe many methodological differences in terms of heterogeneity in the data sources, the samples (country coverage and research period), the econometric specifications and the estimation procedures. We use meta-analysis both to identify the extent of heterogeneity and its potential impact and to find out research needs. Second, in this emerging scientific field the findings are preliminary and often contradictory due to the scientific process of finding out the ‘true’ effect. Meta-analysis can be used to distil this effect that often cannot be observed on the basis of individual studies. Third, as meta-analysis provides a transparent and objective way to synthesize research, this tool is useful in an area that like natural disasters impact is vulnerable to bias due to the ideological or intrinsic motivation of the researcher. An aim of our paper is to show how one can use the identified methodological characteristics to better understand the significance of future findings. Understanding the robustness and importance of new findings is crucial because they influence policy decisions with a potentially long-run impact, especially since both prevention and mitigation require investments over considerable periods of time. The second aim is to find out what are the most important research needs from the perspective of the emerging literature. We identify strengths and weaknesses in terms of coverage and robustness of control variables showing gaps in the literature and highlighting the importance of some rigour in the phase of reporting results and, for example, suggest that it is necessary to include population and institutions more often among the control variables. We also provide a study on the IPCC’s special report Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation in order to contrast the meta-analysis and its findings with an influential traditional review of literature (that also comprises qualitative research). The two methods force two different perspectives on researchers and each perspective helps to uncover other (aspects of) literatures. Our conclusion is that research synthesis would benefit from combining the two approaches. A meta-analysis without a traditional review of the literature is incomplete.
|Series||ISS working papers. General series|