Exploring the Application of Large Language Models in Social Science

In this episode, our guest Petter Törnberg, an assistant professor in computational social science, delves into the intersection between computational methods and their application in the social sciences. He shares insights on the potential of large language models (LLMs) in this field and highlights key findings from his research.

Computational social science, a burgeoning discipline, encompasses various approaches that blend theory, data, and models. Törnberg emphasizes the significance of LLMs as tools that can perform tasks traditionally believed to be exclusive to humans, such as subjective interpretation.

One study conducted by Törnberg compared the performance of LLMs, human experts, and Mechanical Turkers in classifying political tweets. Surprisingly, the LLM outperformed both human experts and Turkers. Not only did it provide accurate classifications, but it also offered explanations for its decisions, challenging the notion that certain forms of interpretation are beyond the capabilities of machines.

Prompt engineering plays a crucial role in effectively utilizing LLMs. It involves developing prompts that capture the concept under study and iterating them based on the LLM's responses and the expert's own understanding.

LLMs have the potential to reduce the reliance on human annotators in social science research, enabling the analysis of vast amounts of text data. However, controversies and debates surrounding biases and the control of these models by large corporations may arise.

Despite the challenges, LLMs present exciting opportunities for social scientists to explore and understand the social world in new ways. Törnberg emphasizes the importance of critical thinking and awareness of biases while utilizing these powerful tools.

Today, We are joined by Petter Törnberg, an Assistant Professor in Computational Social Science at the University of Amsterdam and a Senior Researcher at the University of Neuchatel. His research is centered on the intersection of computational methods and their applications in social sciences. He joins us to discuss findings from his research papers, ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning, and How to use LLMs for Text Analysis.