I am an AI Scientist + Social Scientist looking to build moral alignment into AI agents. At the moment I am studying this using multi-agent simulations with independent learnering agents, using systems based on Reinforcement Learning (RL) and foundation models (LLMs such as the Gemma2 model), with inspiration from moral philosophy, psychology and game theory.

Currently I’m a final-year PhD candidate at the Machine Intelligence Lab, Department of Computer Science, University College London (UCL). I am funded by the Leverhulme Doctoral Training Programme for the Ecological Study of the Brain. Before my PhD I studied Psychology & Linguistics (@ UCL), conducted research on political psychology using Twitter data (@ University of Cambridge), and worked as an AI/Data Scientist and Behavioural Scientist at two start-ups and a large investment bank.

In my latest work I have developed a methodology for quantifying key moral philosophical frameworks in terms of actions and consequences on an envoronment, used these as intrinsic rewards for training RL agents (see IJCAI’23 paper), studied the emergence of ethical and cooperative behaviours in multi-agent social dilemma environments, exploring population dynamics in morally heterogeneous artificial societies of learning agents (see AIES’24 paper), and applied this framework to fine-tuning LLM agents to play more morally aligned policies in interactive game environments (see pre-print).

News

• [Oct 2024] New preprint out: Moral Alignment for LLM Agents (see arXiv)
• [July 2024] Our paper “Dynamics of Moral Behavior in Heterogeneous Populations of Learning Agents” (see arXiv) has been accepted for The 7th AAAI/ACM Conference on AI, Ethics & Society, 2024 (Santa Clara, California).
• [July 2024] Attending at the Vienna Alignment Workshop and presenting our work at the Unconference which followed :)
• [July 2024] I’m at the Eastern European Machine Learning Summer School in Novi Sad, Serbia!
• [April 2024] I’m at the InterpViz Mechanistic Interpretability hackathon at LISA (The London Initiative for Safe AI)!
• [March 2024] New preprint out: Dynamics of Moral Behavior in Heterogeneous Populations of Learning Agents (see arXiv)
• [December 2023] New preprint out: Learning Machine Morality through Experience and Interaction (see arXiv)
• [November 2023] Gave a (long) student talk at LSE about how AI and Social Science can mutually benefit one another, and interesting questions emerging around aligning LLMs with human values.
• [November 2023] I’m at the Anthropic Hackathon in London!
• [November 2023] Presenting our IJCAI paper at two UCL Conferences - the Computer Science Student Conference (long talk) and the Neuro AI Conference (lightning talk).
• [August 2023] Presented our paper “Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement Learning” at the 32nd International Joint Conference on Artificial Intelligence (IJCAI 2023) (main track) in Macao. Links & materials here.
• [July 2023] I’m at the Cooperative AI Foundation Summer School in London!
• [May 2023] Spent a week volunteering at the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023) in London.

Education

• PhD Computer Science (Leverhulme Doctoral Training Programme for the Ecological Study of the Brain), University College London, 2022-2025 (ONGOING)
• MPhil Biological Science (Psychology), University of Cambridge, 2019-2020. Full-time research, based at the Political Psychology Lab. In my research I studied political and linguistic behaviour on Twitter using Social Network Analysis and Natural Language Processing.
• BSc Psychology & Language Sciences, University College London, 2016-2019.

Previous experience

• Quantitative UX Researcher, JPMorgan (user experience analytics within the Corporate & Investment Bank, Markets business), 2021-2022.
• Data & Behavioural Scientist, Rooster Insurance (car insurance startup which based its pricing model on behavioural data rather than demographcis), 2021.
• Behavioural & AI Scientist, ProdX.ai (productivity startup which was building a product to make digital workers more productive thourgh automated, data-driven & psychology-inspired coaching), 2020.

Other

I’m in the process of changing my last name from Karmannaya to Tennant. If you see different names listed on different publications, don’t get confused :)