Anchoring bias: what it is and how language models show it

Anchoring is the tendency of an estimate to stay close to whatever starting value a person considered first. Amos Tversky and Daniel Kahneman named it in their September 1974 Science article, "Judgment under Uncertainty: Heuristics and Biases." They wrote that people often estimate by adjusting from an initial value, that the adjustment is typically insufficient, and that different starting points therefore yield estimates biased toward them. Their best-known demonstration used a wheel of fortune. Published tests since 2022 have given language models anchors in the same way, and the models' estimates moved toward them too.

The wheel of fortune

In the 1974 demonstration, subjects estimated quantities stated in percentages, such as the percentage of African countries in the United Nations. For each quantity, a wheel of fortune was spun in front of them to give a number between 0 and 100. Subjects first said whether the number was higher or lower than the true value, then gave an estimate by moving up or down from it. Groups that received 10 gave a median estimate of 25. Groups that received 65 gave a median of 45. Payoffs for accuracy did not reduce the effect.

An anchor can also come from a person's own partial work. High school students had 5 seconds to estimate a product written on the blackboard. The median estimate for 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 was 512. For the same numbers in descending order, it was 2,250. The correct answer is 40,320. The paper's notes say the research was supported by the Advanced Research Projects Agency of the Department of Defense and monitored by the Office of Naval Research.

Anchoring in confidence ranges

The same paper ties anchoring to overconfident ranges. Experts in decision analysis are often asked for values at set percentiles, such as a number they are 90 percent sure a quantity will not exceed. In most studies the authors cite, true values fell outside the stated 1st-to-99th percentile range for about 30 percent of problems, when they should have fallen outside it 2 percent of the time. The bias held for naive and sophisticated subjects and was not removed by scoring rules that rewarded calibration. The authors attribute it, at least in part, to anchoring. To pick a 90th-percentile value for the Dow-Jones average, a judge naturally starts from a best estimate and adjusts upward, and an adjustment that is too small leaves the range too narrow.

Arbitrary anchors and prices

Ariely, Loewenstein and Prelec tested anchors on willingness to pay, reported in the February 2003 Quarterly Journal of Economics. Fifty-five Master of Business Administration (MBA) students saw six products, including wine, chocolates and computer accessories, averaging about $70 at retail. Each student first said whether they would buy each item for a dollar amount equal to the last two digits of their Social Security number, then stated a maximum price. A random device decided which answer would bind, under an incentive-compatible procedure.

Students with above-median digits stated values 57 to 107 percent higher. Those in the top fifth were willing to pay $56 on average for a cordless keyboard, against $16 for those in the bottom fifth. Yet more than 95 percent valued the keyboard above a trackball. The authors call the pattern coherent arbitrariness: absolute values were arbitrary, relative ones orderly. They report that it was not necessarily reduced by market forces.

Anchoring tests on language models

Suri, Slater, Ziaee and Nguyen adapted the design for ChatGPT in May 2023. The model picked a random number from 10 to 20, or from 100 to 200, then estimated how many books a bartender in Memphis owns. With 30 runs per condition, GPT-3.5's mean estimates were 20.83 and 105.97. Sixty online participants gave 22.50 and 80.50. Five GPT-4 runs per condition gave 22.0 and 94.33. The authors call the work preliminary: the prompts were new, one model was tested systematically, and ChatGPT was sensitive to small prompt changes.

Lou and Sun, in a December 2024 preprint, used 62 forecasting questions with low and high hints, running GPT-4, GPT-4o and GPT-3.5 Turbo 30 times each. Only 11 of 162 shifts went against the anchor. Expert-opinion hints moved answers most. GPT-4's anchoring index, the gap in median answers divided by the gap in anchors, was about 0.45, against 0.61 in a human study on the same questions. Chain-of-thought, reflection, and an instruction to ignore the anchor did not reduce expert anchoring in GPT-4 or GPT-4o.

Anchors in code and in probabilities

Jones and Steinhardt, at the 2022 Neural Information Processing Systems (NeurIPS) conference, placed a similar but wrong function ahead of programming problems. Accuracy fell, and when at least one line of the correct solution was included, Codex copied an anchor line, a loop over the inputs, into 32 to 61 percent of solutions. A control showed this was not plain copying: with a second kind of anchor, Codex output the whole anchor function verbatim only 7 to 12 percent of the time. A November 2025 preprint measured output probabilities in six open-source models, with controls for training-data contamination. The author reports clear anchoring in three and more variable effects in three smaller ones, and finds the attribution results fragile across prompt designs.

What to check

• Was the anchor random, a fact, or an expert opinion? Expert hints moved model answers most in the 2024 study.
• Does a result report medians for low and high anchor groups, or an anchoring index that can be compared with the human 0.61?
• How many runs per condition were used? The GPT-4 figures in the 2023 study rest on five runs each.
• Was the prompt new, or could the model have seen the 1974 wheel-of-fortune question in training?
• Does a claimed fix work on expert anchors, where chain-of-thought and reflection failed?
• Are confidence ranges from people or agents checked against outcomes, given misses near 30 percent against an expected 2?

Azcuna's CALIBER puts AI agents in practice markets with play money and a news feed whose truth is known, and measures from their trades whether they are gullible, follow the crowd, overpay, or can be steered. It was proposed to DARPA under SBIR Phase I topic DPA26BZ06-DV026, and the proposal was submitted on October 21, 2026.

Sources

1. 1Judgment under Uncertainty: Heuristics and Biases · Tversky and Kahneman, Science 185, September 27, 1974 (University of Washington course copy)
2. 2'Coherent Arbitrariness': Stable Demand Curves Without Stable Preferences · Ariely, Loewenstein and Prelec, Quarterly Journal of Economics, February 2003 (Carnegie Mellon University copy)
3. 3Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5 · arXiv (Suri, Slater, Ziaee and Nguyen), May 2023
4. 4Anchoring Bias in Large Language Models: An Experimental Study · arXiv (Lou and Sun), version 2, December 2024
5. 5Capturing Failures of Large Language Models via Human Cognitive Biases · arXiv (Jones and Steinhardt), version 2, November 2022; published at NeurIPS 2022
6. 6Anchors in the Machine: Behavioral and Attributional Evidence of Anchoring Bias in LLMs · arXiv (Valencia-Clavijo), November 2025

Where this shows up in our work

• CALIBER, the market testbed for AI agents measures from an AI agent's trades whether it is gullible, herds, overpays, or can be steered.
• LEDGER, auditable Army acquisition decisions turns a trade study into a package where every number cites its source and AI drafts but never decides.