This blog entry is based on my closing plenary talk at the Foundation for Audit Research International Conference in the Netherlands in June 2018. The first paper related to this talk appears in Accounting Perspectives in the Fall 2018 issue. I want to thank my research colleagues, Kris Hoang at the University of Alabama, Yi Luo at Queen’s University and Jim Sylph formerly of the International Federation of Accountants and the International Auditing and Assurance Standards Board for their efforts on these projects.
The impact of audit research on audit standard setting has been described as “a productive collaboration” (Bell and Wright1995), with audit researchers having “played, and continue(ing) to play, an important role in the development of auditing practice and auditing standards” (AAA Research Impact Task Force 2009). In contrast, others describe it as “inefficient, pedestrian and increasingly detached from practice” (Parker et al 2011) and having “little relevance”. Having taken part in some of the attempts to transfer knowledge (e.g. research article “practice summaries”, articles in professional accounting journals, establishing academic journals whose goal is to stimulate discussion with practice) and seen them appear to have limited impact, I asked the question “What we might do that would result in more effective knowledge transfer to policymakers?”
The Setting
Standard setters, especially US Public Company Accounting Oversight Board, have felt pressure to show that its standards are based on the best possible evidence and meet the cost-benefit test for new regulations imposed by the US Congress. In the last decade PCAOB proposed rules have varied from citing almost no research to citing numerous research studies including literature syntheses to justify the rule and considerations in its development. One example of next to no research being cited is PCAOB’s revision of its audit risk standard. In contrast, the PCAOB’s proposed rule on disclosure of the engagement partner identity had over 30 research studies and research syntheses referred to in explaining and justifying the standard.
So why one and not the other? It was certainly not due to the often-trotted out excuse that there was “no relevant academic research” on, in this case, audit risk assessment. Indeed, the AICPA/AAA Wildman Medal for research impact on practice was awarded to the audit risk literature synthesis prepared for the PCAOB as part of the Auditing Section project (i.e., Allen et al 2006). Further, the AAA Research Impact Task Force identified audit risk is an exemplar area of the impact of research on practice (p. 419-424). So the quantum of audit research does not seem to map onto policymaker’s use.
Internationally the Monitoring Group (2017), composed of securities and prudential regulators representatives whose mandate is to oversee the International Auditing and Assurance Standards Board, has issued strongly worded calls for the IAASB create more evidence-based standards. The Monitoring Group based this call for evidence-based standard setting on the “theory of better regulation.” They also suggested more evidence to support the auditing standards being proposed due to their belief that the IAASB is captured by the audit profession/IFAC/Big 4. Thus, “in the public interest” the IAASB is said to need more independent evidence and processes to support its standard setting.
Three years ago, based our review of knowledge transfer theory and practices (Hoang, Salterio and Slyph 2018), we concluded that the potential exists for academic audit research to be better support audit standard setting. We recognized that the current reality was one where some knowledge transfer occurred but that it was a “hit or miss” affair with no evidence of effective systematic approaches to transfer being undertaken. We set out to investigate why knowledge transfer approaches taken to date have not been effective (Salterio, Hoang, and Luo, 2018).
The Investigation
We carried out an analysis of how other research based areas transfer research knowledge to policymakers and quickly came upon the fields of evidence-based policymaking and evidence-based management (Rousseau, 2012). The evidence-based policymaking framework originated from the evidence-based medicine movement (EBM), which sought to overcome challenges of using academic research knowledge to create health policy guidelines. Although EBM started in the late 1980’s with a focus on medical (especially drug and diagnostic) interventions, where randomized clinical trials were the ‘gold standard’ evidence, these practices were gradually adapted to wider domains of health policy where more diverse sets of evidence needed to be evaluated and synthesized for transfer to policymakers.
To be clear, “evidence-based” means that the policy makers make an informed decision explicitly including evidence that comes from underlying academic research, in addition to inputs from practitioners, other regulators/guideline producers, and parties that have traditionally been engaged in the policymaking process. Even in the domain of EBM the research knowledge being transferred does not set the policy prescription except in rare areas such as drug safety. In other words, the term “evidence-based” can be better described as “evidence-informed” policymaking as rarely will the research evidence lead to “one right” policy.
In particular, we examined EBM’s knowledge translation framework that facilitates systematizing, analyzing and transferring the broad and always rapidly growing research evidence to the development of practice guidelines (the results of that analysis are documented in our forthcoming article (Hoang et al 2018) and our working paper available on SSRN (Salterio et al 2018). Later we simulated the knowledge transfer process with former IAASB standard setters and staff by recreating an historical standard setting issue on divided responsibility in group audits (we are currently writing up this research in Hoang, Luo and Salterio 2018). Based on our analysis (Hoang, Salterio, and Sylph, 2018, Salterio et al 2018) and our later simulation (Hoang, Luo and Salterio 2018) we believe that evidence informed policymaking is a viable approach to dealing with the complexities associated with transferring the large academic audit literature into forms that policymakers can use. In stating this conclusion we echo Christian Leuz’s conclusions in his PD Leake Lecture (see his blog entry) for the need for evidence-based policymaking in accounting standard setting.
The Practicalities
As Professor Leuz notes, the idea of evidence-informed policymaking sounds simple but the practices and institutions that will allow it to occur need to be developed. Our research suggests four main practices and one type of institution need to evolve to create an ecosystem where knowledge transfer is a routine part of the audit academic’s life and a routine input in the standard setting process. These practices are:
As we have found in our standard setting simulation, all of the practices (items 1 to 4) are possible to develop if done so with some creativity. However, even in our simulation we found that it is easy for researchers to “overpromise” and for standard setters to expect that researchers will give them the “correct” answer.
Finally, there will need to be developed institutions (like in EBM the Cochrane Collaborative) that provides rigorous reviews of the proposed research syntheses (i.e. the synthesis protocol) and the final synthesis before publication and potentially before being provided to policymakers. This institutional endorsement of research synthesis quality would, in time, allow syntheses writing to develop the same status in the academic evaluation process as academic publications. Based on our informal conversations with leaders in the EBM movement, we believe that institutions like medicine’s Cochrane Collaborative, will develop in auditing and accounting. Such institutions will give research synthesis production the academic respectability that it needs in order to have high profile researchers participate in these syntheses.
Conclusion
EBM traces its beginnings to the late 1980’s, became a medical “buzz word” by the early 1990’s, and by the middle of the first decade of this century became accepted practice in health policy. However, just as in health policy debates, we expect that the evidence-based process in auditing will be subject to and interpreted in light of political processes that take place at all policymaking bodies. While the politicization of evidence is an important issue, we are barely at the stage of ensuring that the research evidence we have is made available to those in the policymaking process. Ensuring that high quality audit research evidence in forms assessable to policymakers is our first challenge. It is one that we expect to see many advances in the next decade if audit research is to reach its full potential for impact on society.
Note: This research program is funded by the Canadian Social Sciences and Humanities Research Program (Grant number 430-2015-0155) and supported by the Stephen JR Smith Chair in Accounting and Auditing at the Smith School of Business, Queen’s University. There are no declared conflicts of interests by any researchers involved in this research program.
References:
AAA Research Impact Task Force. 2009. The Impact of Academic Accounting Research on Professional Practice: An Analysis by the AAA Research Impact Task Force. Accounting Horizons. 23(4): 411–456.
R. D. Allen, D. R. Hermanson, T. M. Kozloski, and R. J. Ramsay. 2006. Auditor Risk Assessment: Insights from the Academic Literature. Accounting Horizons. 20(2): 157-177.
T. Bell and A. Wright. 1995. Auditing practice, research, and education: A productive collaboration. New York AICPA.
K. Hoang, S. E. Salterio, and J. Sylph. 2018a. Barriers to Transferring Auditing Research to Standard Setters. Accounting Perspectives. Forthcoming. (early draft available on SSRN)
K. Hoang, Y. Luo and S. E. Salterio. 2018b. Evidence about Evidence-Informed Audit Policymaking: The case of group audits (working title). Research write-up in progress.
Monitoring Group. 2017. Strengthening the governance and oversight of the international audit-related standard-setting boards in the public interest. Available in the International Organizations of Securities Commissions website (https://www.iosco.org/library/pubdocs/pdf/IOSCOPD586.pdf)
L. Parker, J. Guthrie and S. Linacre. 2011. The relationship between academic accounting research and professional practice. Accounting, Auditing & Accountability Journal. 24 (1): 5-14
D. M. Rousseau, editor. 2012. Oxford Handbook of Evidence-Based Management. New York: Oxford University Press.
S. E. Salterio, K. Hoang, and Y. Luo. 2018. Communication is a two-way street: Analyzing approaches undertaken to enhance audit research knowledge transfer to policymakers. Working paper (early draft available on SSRN).
