What drives the use of performance measurement (PM) systems in the public sector? And what are the effects of PM use? In a recent study in Financial Accountability & Management (Van der Kolk, 2022, open access), I address these questions. I do so by reviewing survey papers on public sector PM published in all major accounting outlets, i.e., 3, 4, and 4* journals from the ABS journal list (2018). In this blog, I highlight seven observations from the review.
Fuelled by the new public management movement in the end of the 20th century, and crises in the beginning of the 21st century, performance measurement use and research has gained a strong foothold in practice and academia. In the examined survey studies, some evidence of the benefits of mobilizing PM were listed (e.g. enhanced efficiency, motivation, commitment) as reasons to implement PM systems, while respondents from one UK survey indicated that the use of key performance indicators was the single most successful accounting innovation in their organization.
One goal of the review was to investigate what drives the use of PM in public sector organizations. Three categories of antecedents were identified: 1) context variables (e.g. competition, political constraints), 2) organization variables (e.g. organizational values, financial stress), and 3) individual-level variables (e.g. respondent role, age, tenure, educational background). All reported antecedents and main effects have been visualized in four maps based on the empirical context of the study (map A: Government, map B: Health, map C: Education, map D: Other settings). Map A is included below (scroll down) and shows antecedents and effects of using PM in government settings (for other maps and for the meaning of the various colors and arrows, see table 1 in the original paper, available here).
Another goal of the review was to map what sorts of effects public sector PM generates. Some studies find evidence that the adoption of a PM system impacts performance, in line with prior research. In the review, a distinction is furthermore made between performance-related effects (e.g., a more detailed PM positively impacts performance in a hospital), other organization-level effects (e.g., ease of use of PM impacts the perceived usefulness of PM practices) and individual level effects (e.g., PM subjectivity negatively impacts trust in a supervisor). It is important to note, however, that there are inherent limits to what can be concluded based on cross-sectional survey studies. Aspects that are not measured or expected do not become visible in quantitative survey studies, and other methods such as qualitative field studies should be used to complement this review to gain a more complet understanding of PM and its effects.
Source: Van der Kolk (2022, map A). More maps are available in the review paper.
Looking at the geographical distribution of the empirical work conducted, it is clear that there is a strong bias towards certain geographical areas. For instance: 69% of the papers include evidence from Europe, 22% from Australia, and 9% from North America and Asia. Countries such as The Netherlands and Norway are well represented, while entire continents such as Africa and South America are absent. Given that culture can impact the effectiveness of management control instruments such as PM (cf. Malmi et al., 2020), a call for more research in “non-Western public sector contexts” (Van der Kolk, 2022, p. 723) therefore seems warranted.
Although all 27 accounting journals that are rated 3 or higher in the ABS 2018 journal list were included in the initial search, only 8 journals published survey papers on public sector performance measurement between 1999 and 2018. From these journals, Financial Accountability & Management (17 papers), Accounting, Organizations & Society (4 papers), Management Accounting Research (3 papers) and the European Accounting Review (3 papers) were most strongly represented in the sample. Furthermore, it seems that scholars from the US who study PM publish less often in accounting journals, but rather send their work to outlets in the public management discipline, which may be a reason why also the US is not not strongly represented in this sample.
When comparing this review to extant literature reviews from the discipline of public administration, it struck me that accounting scholars often point to task characteristics (e.g., output measurability, contractibility) to explain the effects of PM in different settings, while in the discipline of public administration this factor received considerably less attention. Comparing the two disciplines, it is (mostly) the accounting discipline that continuously stresses the importance of task characteristics to explain why PM yields different results in different settings.
Based on the reviewed survey papers, a few gaps and directions for future research are identified. Among them are: 1) examining non-linear relations among PM and other variables (given that currently most survey studies only test linear relations while there are reasons to believe that these relations may not be linear in reality), 2) examining combinations of PM and other forms of management control ‘as a package’ (given the various calls and the limited evidence available on this topic in public sector organizations), and 3) studying PM in non-Western public sector settings.
In line with prior research within and outside of the accounting discipline, the findings from this review suggest that there is no easy answer to the question whether PM in the public sector is a blessing or a curse. PM can yield both favorable and unfavorable effects, and this review highlights that PM characteristics such as its perceived fairness, subjectivity and ease of use matter to a great extent for the effects PM can generate. And in line with prior research, much depends on the way how a PM system is used (cf. Van Elten et al., 2021, Gerrish, 2016). PM systems should therefore be handled with care: what you measure isn’t always what you get.
Berend van der Kolk is an associate professor of management accounting at Vrije Universiteit Amsterdam.
References
Gerrish, E. (2016), The Impact of Performance Management on Performance in Public Organizations: A Meta-Analysis. Public Administration Review, 76: 48-66. https://doi.org/10.1111/puar.12433
Malmi, T., Bedford, D. S., Brühl, R., Dergård, J., Hoozée, S., Janschek, O., Willert, J., Ax, C., Bednarek, P., Gosselin, M., Hanzlick, M., Israelsen, P., Johanson, D., Johanson, T., Madsen, D. Ø., Rohde, C., Sandelin, M., Strömsten, T., & Toldbod, T. (2020). Culture and management control interdependence: An analysis of control choices that complement the delegation of authority in Western cultural regions. Accounting, Organizations and Society, 86, [101116]. https://doi.org/10.1016/j.aos.2020.101116
(2022). Performance measurement in the public sector: Mapping 20 years of survey research. Financial Accountability & Management, 38, 703– 729. https://doi.org/10.1111/faam.12345
van Elten, H. J., van der Kolk, B., Sülz, S. (2021). Do different uses of performance measurement systems in hospitals yield different outcomes? Health Care Management Review, 46, 3, 217-226. https://doi.org/10.1097/HMR.0000000000000261
