{"id":4200,"date":"2022-12-15T11:43:05","date_gmt":"2022-12-15T10:43:05","guid":{"rendered":"https:\/\/eaa-online.org\/arc\/?p=4200"},"modified":"2022-12-15T11:43:05","modified_gmt":"2022-12-15T10:43:05","slug":"publication-outcomes-after-presenting-at-the-eaa-annual-congress","status":"publish","type":"post","link":"https:\/\/eaa-online.org\/arc\/blog\/2022\/12\/15\/publication-outcomes-after-presenting-at-the-eaa-annual-congress\/","title":{"rendered":"Publication outcomes after presenting at the EAA Annual Congress"},"content":{"rendered":"

by Edith Leung and Jochen Pierk<\/span><\/p>\n

December 2022<\/span><\/p>\n

One of the most visible activities of the EAA is the Annual Congress, which provides a forum for academics to meet and discuss research. Although the Congress serves many important purposes, such as symposia and networking, a key reason for many academics to attend is to present and receive feedback on ongoing research in one of the scientific sessions. The aim of these sessions is to help authors improve their work-in-progress for eventual publication in an academic journal, yet little is known about eventual publication outcomes following presentation at the Congress.<\/span><\/p>\n

In this blog, we share the results of our analyses of publication outcomes for papers presented at the EAA Annual Congress and identify potential areas of improvement for the scientific presentation sessions at the Congress. We believe these insights are interesting for the academic community at large and provide a useful basis for further discussion about the merits of presenting at the Congress. These data were graciously provided to us by the EAA, but any opinions and views expressed in this article are solely those of the authors and do not represent the official view of the EAA.<\/span><\/p>\n

In the remainder of this paper, we first outline how the EAA sessions and submission process are currently organized. Next, we provide statistics on publication outcomes for differently rated papers and papers in types of sessions. We conclude with several recommendations to further improve the scientific standard at the Congress and give more room to a more diverse set of academics at different stages in their career to attend the Congress.<\/span><\/p>\n

Current Setting and Challenges<\/h1>\n

The Annual Congress organizes two types of sessions in which academics can present their work: the parallel sessions and research fora. In the current setting, scholars are invited to submit their research in the form of a full paper to the EAA Annual Congress, indicating their method of choice and the research area of the paper. Subsequently, the Standing Scientific Committee (SSC) members responsible for a particular research track allocate two reviewers to each submission. Reviewers assign scores between 1 and 6 and are encouraged to provide written feedback that may be useful to the authors of a paper. Based on these reviews and scores based on the evaluation criteria, the SSC of the EAA then decides upon the acceptance or rejection of papers. Usually, papers that receive a score between 2.5 and 4 are accepted for presentation at the Annual Congress in a Research Forum session (RF, 90 minutes for 5 papers), while papers with a score above 4 enter a Parallel Session (PS, 90 minutes for 3 papers) or Parallel Session with Discussant (PSD, 90 minutes for 2 papers, including an assigned discussant).[1]<\/a><\/span><\/p>\n

The implicit assumption of the evaluation criteria is that Research Fora allow for the discussion of viable and interesting research projects at an earlier stage, that need further development to be reasonably submitted to a journal. For example, a score of 2 is described as \u2018initial work on a potentially viable project, but it is not likely to be ready for submission for some time\u2019. However, based on our anecdotal impressions, the public perception of RF sessions seems to be that they often include low-quality, rather than early-stage, papers. A potential reason is that reviewers are not able to distinguish between an early-stage paper and a low-quality paper (both would be assigned a low grade). As such, one goal of our analyses is to determine whether this perception is supported empirically by worse publication outcomes of papers presented in the RF, and we give recommendations to improve the quality of presentation sessions at the EAA.<\/span><\/p>\n

Publication Outcomes of Presented Papers<\/h1>\n

Sample<\/h2>\n

We start by gathering papers published in the top 15 accounting journals (based on 2020 impact factors) and Accounting in Europe.[2]<\/a> In total, these 16 journals published 2,366 papers between 2012 and 2021. While this selection does not include all potential target journals, especially topical journals are missing, we believe that it covers at least a relevant set of journals for most scholars, including the journals of the European Accounting Association.\u00a0<\/span><\/p>\n

We gather the acknowledgments of all these papers from Scopus and scrape the missing acknowledgments from the journal websites. 608 of these publications mention in the acknowledgments that the paper was presented at an EAA conference. We then manually match these 608 publications to the 6,718 EAA conference presentations between 2012 and 2019. In total, we successfully match 418 journal publications to EAA conference presentations. Most of the remaining 190 journal publications that mentioned the EAA in the acknowledgments were presented before 2012, and in a few incidences, we could not find a match.<\/span><\/p>\n

Results<\/h2>\n

Table 1 shows the number of papers published in the 16 accounting journals with prior EAA presentations separately by journal. As one might expect, the European Accounting Review (EAR) includes most publications with prior EAA presentations, which highlights that authors often present their work at the conference before submitting their paper to the EAR.\u00a0<\/span><\/p>\n

Table 1: Publications with prior EAA presentation<\/strong><\/span><\/h1>\n\n\n\n\n\n\n\n\n\n\n\n
Journal<\/span><\/td>\nCount<\/span><\/td>\n <\/td>\nJournal<\/span><\/td>\nCount<\/span><\/td>\n<\/tr>\n
EAR<\/span><\/td>\n75<\/span><\/td>\n <\/td>\nJAE<\/span><\/td>\n23<\/span><\/td>\n<\/tr>\n
CAR<\/span><\/td>\n63<\/span><\/td>\n <\/td>\nTAR<\/span><\/td>\n21<\/span><\/td>\n<\/tr>\n
ABR<\/span><\/td>\n38<\/span><\/td>\n <\/td>\nBAR<\/span><\/td>\n19<\/span><\/td>\n<\/tr>\n
JAR<\/span><\/td>\n30<\/span><\/td>\n <\/td>\nJAPP<\/span><\/td>\n16<\/span><\/td>\n<\/tr>\n
JBFA<\/span><\/td>\n28<\/span><\/td>\n <\/td>\nAIE<\/span><\/td>\n13<\/span><\/td>\n<\/tr>\n
RAST<\/span><\/td>\n28<\/span><\/td>\n <\/td>\nA&F<\/span><\/td>\n9<\/span><\/td>\n<\/tr>\n
AAAJ<\/span><\/td>\n27<\/span><\/td>\n <\/td>\nAH<\/span><\/td>\n1<\/span><\/td>\n<\/tr>\n
AOS<\/span><\/td>\n26<\/span><\/td>\n <\/td>\nIS<\/span><\/td>\n1<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Note: this table documents the number of publications per journal of papers that mention presentation at the EAA Annual Congress between 2012 and 2019. See footnote 2 for explanation of the abbreviated journal names.<\/span><\/p>\n

In Table 2, we document the association between the average reviewer score and the likelihood of publication. We find that the likelihood of publication increases with the reviewer scores, which indicates that reviewers, on average, do a good job of assessing the quality of papers. The reviewer instructions of the EAA include the threshold of an average reviewer score of 4 (called \u2018anchor\u2019 on the conference website<\/a>) to indicate if a paper is \u2018ready\u2019 for submission to a journal such as EAR or AIE. While we do not see a clear-cut threshold in the likelihood of publication in Table 2, publication likelihood for papers with scores 4 and above is much higher than for lower-scoring papers. Especially papers that score below 3 are very rarely published in the selected 16 journals. It is important to note that we do not imply that these papers are necessarily of low quality or are never published (e.g., these papers may be published in journals that are not included in our selection). We also caution that these statistics should not be used to determine if a paper has \u2018potential\u2019 or not, since many factors determine publication outcomes and reviewer scores are to some extent subjective.\u00a0\u00a0\u00a0<\/span><\/p>\n

Nevertheless, only one paper out of the 883 presented papers with a score below 3 was published in the association\u2019s journals EAR and AIE. It is reasonable to assume that the paper quality was often not sufficient for high-quality journals, in line with the conference reviewers’ assessment that these papers are not ready for submission. In unreported analyses, we also do not find that these papers are published with delay. Taken together, these statistics indicate that the quality of low-scoring papers is, on average, too low to warrant publication in these journals.<\/span><\/p>\n

Table 2: Publications by Reviewer Score<\/strong><\/span><\/h1>\n\n\n\n\n\n\n\n\n
SCORE<\/span><\/td>\nPAPER<\/span><\/td>\nPUBLISHED (%)<\/span><\/td>\n <\/td>\nSCORE<\/span><\/td>\nPAPER<\/span><\/td>\nPUBLISHED (%)<\/span><\/td>\n<\/tr>\n
1.5<\/span><\/td>\n3<\/span><\/td>\n0 (0.00)<\/span><\/td>\n\u00a0\u00a0<\/span><\/td>\n4<\/span><\/td>\n1337<\/span><\/td>\n98 (7.33)<\/span><\/td>\n<\/tr>\n
2<\/span><\/td>\n24<\/span><\/td>\n0 (0.00)<\/span><\/td>\n <\/td>\n4.5<\/span><\/td>\n1036<\/span><\/td>\n107 (10.33)<\/span><\/td>\n<\/tr>\n
2.5<\/span><\/td>\n856<\/span><\/td>\n9 (1.05)<\/span><\/td>\n <\/td>\n5<\/span><\/td>\n577<\/span><\/td>\n68 (11.79)<\/span><\/td>\n<\/tr>\n
3<\/span><\/td>\n1203<\/span><\/td>\n29 (2.41)<\/span><\/td>\n <\/td>\n5.5<\/span><\/td>\n245<\/span><\/td>\n41 (16.73)<\/span><\/td>\n<\/tr>\n
3.5<\/span><\/td>\n1387<\/span><\/td>\n53 (3.82)<\/span><\/td>\n <\/td>\n6<\/span><\/td>\n50<\/span><\/td>\n13 (26.00)<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Note: this table presents the count of papers by reviewer score that were presented at the EAA Annual Congress between 2012 and 2019, and the number (percentage) of papers by reviewer score that are eventually published in the journals listed in footnote 3.<\/span><\/p>\n

Recommendations<\/h1>\n

Better distinction between early-stage and developed papers<\/h2>\n

Our analyses suggest that publication likelihood increases with reviewer scores, suggesting that reviewers do a good job of assessing the quality of papers. At the same time, the evaluation criteria and score descriptions provided to reviewers indicate that lower scores should primarily reflect the development stage of the paper. However, low-scoring papers are rarely published in academic journals, rather than published at an equal rate but with a longer delay. Hence, it highlights that the current review system does not allow reviewers to differentiate the development stage of a paper from its underlying quality. To more effectively do so, we suggest adjusting the scoring and acceptance criteria and submission system<\/em><\/strong> to better distinguish early-stage and more developed papers as follows:<\/span><\/p>\n