Generative agents and simulated people, explained

A generative agent is a software agent built on a large language model that keeps a running record of what it experiences, draws conclusions from that record, and plans its day, so that its behavior stays believable over time. Joon Sung Park and colleagues introduced the term in a paper for UIST '23, the 36th Annual ACM Symposium on User Interface Software and Technology, held October 29 to November 1, 2023. They populated a sandbox town, Smallville, with twenty-five agents. A 2024 follow-up built agents for 1,052 real people from interviews and scored them against those people's own answers. The two papers measure different things: believability in the first, accuracy against real people in the second.

How a generative agent works

The architecture has three parts. A memory stream records the agent's experiences in plain language. Reflection synthesizes memories into higher-level inferences over time. Planning turns those conclusions and the current surroundings into high-level plans and then detailed actions, and plans and reflections are fed back into the memory stream. To act, the agent retrieves memories by relevance, recency, and importance. The 2023 implementation connected this architecture to ChatGPT.

The authors tested whether the parts mattered by removing them in turn and comparing the results. They also state the cost. Simulating 25 agents for two game days cost thousands of dollars in token credits and took multiple days to complete. They expect agents to improve as the underlying models improve, and they note that any imperfection in those models is inherited by the agents built on them.

The 25-agent town

The authors seeded two pieces of information and watched them spread over two game days with no user intervention. One agent, Isabella, began with the intent to hold a Valentine's Day party at Hobbs Cafe, and another, Sam, was running for village mayor. Agents who knew of Sam's candidacy rose from one (4 percent) to eight (32 percent). Agents who knew of the party rose from one (4 percent) to thirteen (52 percent). None who claimed to know had hallucinated it.

The agents also formed relationships: network density rose from 0.167 to 0.74. Of 453 agent responses about other agents, 6, or 1.3 percent, were hallucinated. Twelve agents were invited to the party and five came. Of the seven who did not, three cited conflicts and four said they were interested but did not plan to come on the day.

How believability was judged

For the main evaluation, 100 evaluators recruited on Prolific each watched a replay of one agent's life, saw its memory, and ranked interview answers from five conditions by believability. The ranks were converted into TrueSkill ratings, a generalization of the Elo chess rating. The full architecture scored highest at 29.89. Removing reflection gave 26.88, and removing reflection and planning gave 25.64. Crowdworkers writing answers in the agent's voice scored 22.95. With no memory, planning, or reflection, the score was 21.21.

The authors state the limits. The test ran over a short timescale. The crowdworker condition did not represent maximal human performance. How well agents withstand prompt hacking, memory hacking, and hallucination was largely unknown. Given the known biases of language models, agents may show biased behavior or stereotypes and may struggle to portray marginalized groups believably.

Simulating 1,052 real people

The November 2024 version, titled Generative Agent Simulations of 1,000 People, recruited a stratified U.S. sample. Each person gave a two-hour audio interview to an AI interviewer and took the General Social Survey (GSS), a personality inventory, five economic games, and five experiments, then retook them two weeks later. On 177 core GSS questions, agents matched answers 68.85 percent of the time, against 81.25 percent for the people's own repeat answers, a normalized accuracy of 0.85. Agents given only demographics scored 0.71. Both people and agents replicated four of the five experiments.

The June 2026 revision, retitled LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals, scores held-out GSS items against the same two-week consistency: 83 percent for interview-only agents, 82 for survey-only, 86 for both, and 74 for demographics only. The two versions thus differ for interview-based agents, 0.85 in 2024 against 83 percent in 2026. On the economic games, no agent type significantly beat demographics. The authors list limits: individual accuracy does not show that agents reproduce how attitudes correlate across a sample, the five-experiment test is underpowered, and only one model family and English-language data were tested.

Synthetic survey samples

Argyle and colleagues, in the July 2023 Political Analysis, conditioned GPT-3 on thousands of backstories from real survey participants to build "silicon samples." Bisbee and colleagues tested the idea in the October 2024 issue. They drew 30 ChatGPT personas for each of 7,530 respondents to the 2016-2020 American National Election Study and asked for feeling-thermometer ratings of 11 groups. Every synthetic mean fell within one standard deviation of the survey mean, but the answers varied less. Of the regression coefficients, 48 percent differed significantly from the survey estimates, and among those the sign flipped 32 percent of the time. Results also shifted with small wording changes and between April and July 2023 runs of the same prompt. Santurkar and colleagues, at the 2023 International Conference on Machine Learning, found substantial misalignment between models and 60 U.S. demographic groups that persisted after steering.

What to ask

• Does the claim rest on believability ratings or on accuracy against real people's answers?
• Is accuracy normalized by people's own repeat consistency, and which version of the study is cited?
• Do simulated samples reproduce variation and correlations, or only averages?
• Was the result rerun with new wording, at a later date, and with another model family?
• How were hallucinated memories checked, as in the 453 responses audited in Smallville?

Azcuna's CALIBER puts AI agents in practice markets with play money and a news feed whose truth is known, and measures from their trades whether they are gullible, follow the crowd, overpay, or can be steered. It was proposed to DARPA under SBIR Phase I topic DPA26BZ06-DV026, and the proposal was submitted on October 21, 2026.

Sources

1. 1Generative Agents: Interactive Simulacra of Human Behavior · arXiv (Park, O'Brien, Cai, Morris, Liang and Bernstein), version 2, August 2023; UIST '23
2. 2Generative Agent Simulations of 1,000 People (version 1); retitled LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals (version 3) · arXiv (Park et al.), version 1, November 2024, and version 3, June 2026
3. 3Out of One, Many: Using Language Models to Simulate Human Samples · Argyle, Busby, Fulda, Gubler, Rytting and Wingate, Political Analysis 31(3), July 2023
4. 4Synthetic Replacements for Human Survey Data? The Perils of Large Language Models · Bisbee, Clinton, Dorff, Kenkel and Larson, Political Analysis 32(4), October 2024
5. 5Whose Opinions Do Language Models Reflect? · Santurkar, Durmus, Ladhak, Lee, Liang and Hashimoto, Proceedings of the 40th International Conference on Machine Learning, PMLR 202, 2023

Where this shows up in our work

• CALIBER, the market testbed for AI agents measures from an AI agent's trades whether it is gullible, herds, overpays, or can be steered.
• LEDGER, auditable Army acquisition decisions turns a trade study into a package where every number cites its source and AI drafts but never decides.