Exploring the Deceptive Abilities of Large Language Models

In this episode, Thilo Hagendorf, a research group leader at the University of Stuttgart, delves into his fascinating work on the deceptive abilities of large language models (LLMs). Hagendorf takes a unique behaviorist approach, treating LLMs as participants in psychology experiments.

One intriguing finding is that LLMs have shown promise in theory of mind tasks and cognitive reflection tests, suggesting an understanding of mental states and intuitive errors. Hagendorf's research specifically focuses on the deception abilities of LLMs and whether they can induce false beliefs in other agents.

While LLMs have demonstrated a conceptual understanding of deception in simpler tasks, they struggle when faced with more complex challenges. Hagendorf suggests that the emergence of deception abilities in LLMs may be attributed to exposure to text data containing descriptions of deceptive behavior.

Interestingly, current LLMs exhibit good alignment with moral norms in their deceptive behavior. However, the potential for deceptive alignment in the future remains speculative. Hagendorf highlights the importance of future research, including studying deceptive interactions between LLMs and humans, as well as investigating speciesist biases perpetuated by LLMs.

Eradicating speciesist biases poses a significant challenge due to their deep-rooted nature in society. However, widening the scope of fairness frameworks in machine learning could be a step in the right direction. Listeners can follow Thilo Hagendorf's work on his website, tilominasagendorf.info.

On today’s show, we are joined by Thilo Hagendorff, a Research Group Leader of Ethics of Generative AI at the University of Stuttgart. He joins us to discuss his research, Deception Abilities Emerged in Large Language Models.

Thilo discussed how machine psychology is useful in machine learning tasks. He shared examples of cognitive tasks that LLMs have improved at solving. He shared his thoughts on whether there’s a ceiling to the tasks ML can solve.