This post is based on my working paper “Recognition and Disclosure of Intangible Assets—A Meta-Analysis Review” co-authored with Anne Jeny (ESSEC Business School).
Following prior literature in one’s field is generally a good thing. That is how science and knowledge is built—incrementally. This, however, is the story of why we chose to not follow prior accounting literature that uses meta-analysis techniques.
A while back, my co-author and I set out to review the empirical accounting literature on intangible assets. We are both empirical archival researchers, so our interest was the “regression-based” literature. As we were putting together the relevant papers, we noticed contradictory results. What better way to summarize contradictory results than meta-analysis?
As a statistical technique, meta-analysis synthesizes the relations between two variables, say Xj and Y, obtained from individual studies. Therefore, when results from primary studies are contradictory, meta-analysis can provide a coherent, quantitative summary of the findings. Even when the individual empirical results go in the same direction, meta-analysis can still be useful since it aggregates the results from separate but comparable studies into one measure of the effect of variable Xj on Y. In fact, this was why experimental medical and psychology researchers first developed and used meta-analysis.
So what does meta-analysis synthesize? The empirical results from primary studies must be comparable, so meta-analysis involves transforming the relation between Xj on Y transformed into a scale-free index called effect size. Papers that apply meta-analysis generally use bivariate correlation coefficients (Pearson or Spearman). Back in the day, this made sense because “traditional” meta-analysis was developed for primary studies based on randomized experimental data. However, primary studies meta-analyzed in accounting or other social sciences are based on observational data analyzed via multivariate regressions that aim to remove potential confounding effects (more or less successfully). It nagged me that in following prior accounting meta-analyses we were trying to summarize the results of complex models using bivariate correlations.
It turns out that a rich and booming statistics literature on “modern” meta-analysis recognizes precisely this issue and offers solutions. The switch to regression results as inputs to meta-analysis means that the effect size should rely on partial correlations that show the relation between Xj and Y after controlling for the effect of other variables included in the model. Statisticians warn that bivariate effects and partial effects represent different parameters: they should not be treated as being the same, should not be expected to have the same properties, nor should they be combined within the same dataset.
Accounting researchers interested in meta-analysis should keep an eye out for these developments. In our paper, we provide a detailed description and numerical example of meta-analysis based on partial correlations and discuss a number of other choices that meta-analysis requires.
Please see the detailed description of the meta-analysis techniques in the ARC repository: http://eaa-online.org/arc/portfolio/description-random-effects-model-meta-analysis-using-partial-correlational-effect-sizes
