Human intelligence versus artificial intelligence in classifying economics research articles: exploratory evidence
Jussi T.S. Heikkilä
Journal of Documentation, Vol. ahead-of-print, No. ahead-of-print, pp.-

We compare human intelligence to artificial intelligence (AI) in the choice of appropriate Journal of Economic Literature (JEL) codes for research papers in economics.

We compare the JEL code choices related to articles published in the recent issues of the Journal of Economic Literature and the American Economic Review and compare these to the original JEL code choices of the authors in earlier working paper versions and JEL codes recommended by various generative AI systems (OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini) based on the abstracts of the articles.

There are significant discrepancies and often limited overlap between authors’ choices of JEL codes, editors’ choices as well as the choices by contemporary widely used AI systems. However, the observations suggest that generative AI can augment human intelligence in the micro-task of choosing the JEL codes and, thus, save researchers time.

Rapid development of AI systems makes the findings quickly obsolete.

AI systems may economize on classification costs and (semi-)automate the choice of JEL codes by recommending the most appropriate ones. Future studies may apply the presented approach to analyze whether the JEL code choices between authors, editors and AI systems converge and become more consistent as humans increasingly interact with AI systems.

We assume that the choice of JEL codes is a micro-task in which boundedly rational decision-makers rather satisfice than optimize. This exploratory experiment is among the first to compare human intelligence and generative AI in choosing and justifying the choice of optimal JEL codes.