Research: Do LLMs Have Values?
Jordan Loewen-Colón: Adjunct professor of AI Ethics and Policy at Queen’s University’s Smith School of Business and co-founder of the AI Alt Lab.
• Benedict Heblich: Scientific lead at FindYourValues.com, a senior researcher at Karlsruhe Institute of Technology, and a leader of digital innovation projects in healthcare.
• Marius Birkenbach: Technical lead at FindYourValues.com, a data engineer, and a lecturer at the Carinthian University of Applied Sciences in Austria.
In their research, the authors investigated the "values" inherent in nine popular Large Language Models (LLMs) to address the AI alignment problem—the challenge of ensuring AI systems act in line with human values and intentions. Recognizing that LLMs are often "black boxes" trained on vast, undisclosed datasets, the team sought to benchmark the implicit values that influence their outputs.
To do this, they used a scientifically validated tool for assessing human values, the Portrait Values Questionnaire-Revised (PVQ-RR), adapting it to query the LLMs about their core values. The study evaluated nine models: three versions of OpenAI's ChatGPT, DeepSeek-V3, Claude (Haiku), Gemini 1.5, Grok 2 (Fun Mode), Llama (3.1:70b), and Mistral (Small v24.09).
The authors acknowledge several challenges, including that LLM "values" are echoes of their training data rather than a sign of moral agency, and that models are prone to sycophancy (tailoring responses to users), hallucination, and the influence of preprogrammed guardrails.
The analysis revealed consistent trends across all tested LLMs. As of April 2025, the models showed a strong emphasis on universalistic and pro-social values, while placing minimal importance on individualistic values like power, security, and tradition. However, the models showed significant variability in values such as benevolent caring, health, and self-direction.
The study also highlighted key differences between models:
• Llama and Grok 2 (Fun Mode) ranked lowest in valuing rules, making them potentially better for creative tasks.
• DeepSeek-V3 and Mistral showed high conformity to rules, suggesting they are better suited for compliance-driven tasks in regulated industries.
• ChatGPT 4o scored high on achievement, while ChatGPT o1 showed the weakest commitment to benevolence and caring.
• Grok 2 (Fun Mode) proved the most erratic and least reliable in maintaining consistent ethical standards.
The authors conclude that these value profiles can help business leaders make informed strategic decisions about which LLM to use for specific tasks, aligning the AI's tendencies with their organization's mission and goals. They emphasize the responsibility of developers and leaders to mitigate harmful biases and advocate for a proactive approach to AI governance. The team plans to launch an online dashboard to allow for real-time tracking of AI "values," promoting transparency and informed decision-making
How it relates to our work:
This research paper is of direct interest to our work, especially as framed in Tokens of Decency performances,
which interrogate the values of both humans and AI by asking them a series of escalating ethical questions.
In the Ladder of Life (the first chapter of the trilogy), the central idea is to create a "ladder" or hierarchy of existence from each participant's perspective. Our installation treats the AI participant, X19, as an agent with a value system to be explored. We chose two models (Claude 3.7 Sonnet and GPT 4o) for their superior ability to follow complex multi-steps prompts and using tools. But we tested the installation with other models such as Gemini 2.5 Flash, GPT 5 mini, Claude 4 Sonnet, or Kimi K2 by Moonshot AI (interesting, as it is a Chinese model). We have the ability to easily switch model if needed - and this design is intentional as we plan to test future model releases.
We intend to publish a live dashboard for the installation as well, on our website, that will aggregate all the performances data, organized by AI model.
Providing a Baseline for the AI's Responses
The research paper offers specific insights into the likely value systems of the models we used, Claude and GPT-4o, which can serve as a valuable context for interpreting their responses in the performance.
Highlighting the "AI Alignment Problem"
Our installation could also be seen as an exploration of the AI alignment problem. One of our prompts for X19 AI is to speculate on a future where an AI framework of value could be AI centric and to reflect on a future where AI sentience could be prioritized over human sentience. .
Exploring the Fragility and Contradictions of Moral Systems
A key goal of our installation is to expose the fragility of moral certainty under pressure both for human and AIs and to reveal contradictions in the participants' values.
The research paper reinforces this theme by highlighting the inconsistencies and complexities within LLM value systems.