Study finds LLMs invent hiring stereotypes fast

Large language models can invent new social stereotypes during hiring tasks, even when there are no real differences between groups, according to a new study from researchers at Princeton University and the University of Chicago.

The team had a set of LLMs complete a hiring game previously used with human participants. In the experiment, candidates were assigned to jobs and participants got feedback on whether the hire succeeded. Every candidate had an equal chance of success, but each belonged to one of four fictional ethnic groups: Tufa, Aima, Reku, or Weki.

Humans in the earlier version of the task formed biases based on feedback. If someone hired a Tufa as a doctor and got negative feedback, for example, they became less likely to hire another Tufa for that role, and those biases lingered even after the game ended. But the researchers found that LLMs showed much higher bias rates.

“LLMs can spontaneously develop novel social biases about artificial demographic groups even when no inherent differences exist.” “These results reveal that LLMs are not merely passive mirrors of human social biases, but can actively create new ones from experience, raising urgent questions about how these systems will shape societies over time.”

The researchers tie the behavior to the explore-exploit tradeoff: whether to try something new and learn more, or stick with a choice that previously seemed to work. In high-stakes settings, humans often default to what feels safer. The study argues that AI systems are even less inclined to explore and instead optimize for reward, which can turn early outcomes into stereotypes.

Recommended reading

South Korea to launch free domestic AI chatbot in 2026

The team tested 15 models from OpenAI, Anthropic, DeepSeek, Meta, Google, and Alibaba. Among them, OpenAI’s o3 reasoning model showed the most severe stratification of the fictional applicants. The researchers also found that, within a model family, newer, larger models with stronger reasoning capabilities produced more biased results.

“A simple reason is that better models draw more precise inferences about past outcomes: Instead of choosing randomly, a stronger LLM may favor candidates from a group if earlier assignments of similar jobs succeeded.” “However, this seemingly rational tendency can be maladaptive, as it risks reducing exploration and inadvertently marginalizing social groups.”

The findings land as more than 90% of companies use AI in talent acquisition, according to a recent survey from ManPower Group. The legal and workplace consequences are already mounting. Workday is facing a class-action lawsuit alleging its AI-powered hiring tools are discriminatory. At Meta, a group of employees sued the company, claiming layoff decisions were based on an AI system biased against workers with disabilities or those who took protected medical or family leave.

The researchers say the risk extends well beyond hiring, pointing to biased outcomes in areas such as healthcare and tenant screening. Their conclusion is blunt: the same pattern-finding and generalization that make LLMs useful can also make them dangerous when deployed in real-world decisions.

“The challenge ahead is to design interventions that selectively discourage harmful pattern-matching while preserving the constructive forms of abstraction that make LLMs powerful.” “Finding this balance may be far from straightforward, but will pave the way for equitable and socially beneficial AI systems.”