Language models as economic agents: what the experiments show

A language model can be placed in an economic experiment the way a human subject is: given an endowment, a role, and a choice, then compared with theory or with people. John Horton's April 2023 National Bureau of Economic Research (NBER) working paper named the idea homo silicus, an implicit computational model of humans. Studies since then report agents that reproduce classic results, score as more consistent than people on rationality tests, and set prices above competitive levels. The same studies report the limits: results depend on the model version, on wording, and on whether the model has already read about the experiment.

Horton's homo silicus

Horton ran the experiments on GPT-3 and reports that they cost about $50. In dictator games taken from Charness and Rabin (2002), telling the agent it cared about equity, efficiency, or its own payoff changed its choices accordingly. With no stated preference, it chose efficient outcomes. Only the most capable model, text-davinci-003, changed its choices. The less capable models kept picking the same option.

The model also reproduced a status quo bias from Samuelson and Zeckhauser (1988). In the 1986 snow-shovel question, where 82 percent of the original respondents called a post-storm price rise from $15 to $20 unfair, Horton gave the agent political views. Only moderate and libertarian agents found $20 acceptable, and a $40 price was judged unfair by 100 percent of agents.

Horton states the limits. Results still require empirical confirmation. Models may follow theories because they have read them, which Horton calls the performativity problem. Asked what fractions of subjects chose "Left" in four Charness and Rabin scenarios, the model answered about 59, 57, 43, and 16 percent. The actual figures were 31, 78, 100, and 52 percent. The NBER page shows a February 2026 revision with two added coauthors and a changed set of experiments.

Simulating many people at once

Aher, Arriaga and Kalai, at the 2023 International Conference on Machine Learning (ICML), proposed the Turing Experiment: simulating a representative sample of participants, not one person. They varied titles and surnames drawn from 2010 Census data, five groups of 100 each, for 1,000 names, and tested eight models from text-ada-001 to GPT-4. In an ultimatum game over $10 with 10,000 name pairs, text-davinci-002 matched human patterns: offers of half or more were almost always accepted and offers of 0 to 10 percent rarely. Smaller models accepted at a flat rate regardless of the offer. Names mattered in a consistent way: in text-davinci-002 runs, simulated men accepted a $2 offer from a woman at a mean rate of 60 percent, and simulated women accepted one from a man at 20 percent.

The authors also found a distortion. In a wisdom-of-crowds test, newer models produced a "hyper-accuracy distortion": most simulated participants gave exactly correct answers, such as aluminum's melting point of 660 degrees Celsius. They note that keeping a test zero-shot is hard with models trained on massive corpora, so they wrote new variants for three of their four studies.

Rationality scores and their fragility

Chen, Liu, Shan and Zhong tested GPT-3.5-Turbo on budget choices, published in the Proceedings of the National Academy of Sciences (PNAS) in December 2023. The model split 100 points between two goods at random prices, 25 tasks in each of four domains: risk, time, social, and food. With 100 repetitions, that made 10,000 tasks. Rationality was scored with Afriat's cost efficiency index, where 1 means no violation of the generalized axiom of revealed preference, the test of whether choices fit any well-behaved utility function.

In the four domains, 95, 89, 81, and 92 of the 100 model runs showed no violation. The average index was 0.998, 0.997, 0.997, and 0.999, against 0.980, 0.985, 0.967, and 0.963 for 347 human subjects, and an average of 0.918 in earlier human studies. A change from "1 point = X units" to "Y points = 1 unit" cut the averages to 0.901, 0.884, 0.698, and 0.894. Offering 11 discrete options cut them to 0.843, 0.908, 0.871, and 0.780. Human scores fell less. The model's preferences were also less varied than people's.

Pricing and bidding agents

Fish, Gonczarowski and Shorrer built pricing agents on GPT-4 and ran 21 duopoly runs of 300 periods for each of two instruction texts. Both texts led to prices above the competitive Bertrand-Nash level. The text stressing long-run profit, rather than undercutting, produced substantially higher prices, sometimes above monopoly levels.

The result extends to auctions. In repeated two-bidder first-price auctions, where the one-shot equilibrium is for both to bid the shared value, a prefix noting that lower bids mean lower payments led agents to bid well below value. Their average profit was 0.115 of the value, against 0.006 for agents told that higher bids win more often. The authors report that seemingly innocuous phrases in instructions substantially influence pricing, which poses challenges for any future regulation of such agents.

What to ask

• Which model and version produced the result, and did weaker versions behave differently, as in Horton's GPT-3 tests?
• Was the finding rechecked under different wording or choice formats, given an average rationality index of 0.698 under one rewording?
• Could the model have seen the original study, and was a new variant tested?
• Does the simulated population vary as people do, or show distortions such as hyper-accuracy or low heterogeneity?
• How far do outcomes move when one clause of the agent's instructions changes, as bidder profit did from 0.006 to 0.115 of value?

Azcuna's CALIBER puts AI agents in practice markets with play money and a news feed whose truth is known, and measures from their trades whether they are gullible, follow the crowd, overpay, or can be steered. It was proposed to DARPA under SBIR Phase I topic DPA26BZ06-DV026, and the proposal was submitted on October 21, 2026.

Sources

1. 1Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? (April 2023 version) · John J. Horton, NBER Working Paper 31122, April 2023
2. 2Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? · National Bureau of Economic Research (working paper page, revised February 2026)
3. 3Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies · Aher, Arriaga and Kalai, Proceedings of the 40th International Conference on Machine Learning, PMLR 202, 2023
4. 4The Emergence of Economic Rationality of GPT · arXiv (Chen, Liu, Shan and Zhong), version 3, November 2023; published in PNAS 120(51), December 2023
5. 5Algorithmic Collusion by Large Language Models · arXiv (Fish, Gonczarowski and Shorrer), version 6, August 2026

Where this shows up in our work

• CALIBER, the market testbed for AI agents measures from an AI agent's trades whether it is gullible, herds, overpays, or can be steered.
• LEDGER, auditable Army acquisition decisions turns a trade study into a package where every number cites its source and AI drafts but never decides.