Prediction markets: what they are and how well they forecast

A prediction market is a market in which people trade contracts whose payoff depends on a future event, so the trading price can be read as a forecast. The University of Iowa's Iowa Electronic Markets (IEM) put it plainly: the price of a contract at any given time is a forecast of the outcome. In May 2008, Kenneth Arrow and 21 coauthors argued in Science that the ability of groups to make predictions is a potent research tool that should be freed of unnecessary government restrictions. In the United States, the Commodity Futures Trading Commission (CFTC) oversees these markets as trading in event contracts. Language-model forecasters are now scored against the same human crowds.

How the Iowa markets work

The IEM is a real-money futures market run by faculty of the Tippie College of Business for research and teaching. Since 1988, students and other traders have invested from $5.00 to $500.00 in contracts tied to elections, economic indicators, earnings, and box office receipts. A March 2003 review by Berg, Forsythe, Nelson and Rietz describes the design. The markets trade 24 hours a day as a continuous double auction. In a vote-share market, a contract typically pays $1 times the candidate's share of the vote. Investments are typically capped at $500 per trader and average less than a tenth of that.

The traders are not a sample of voters. The review describes them as overwhelmingly male, well educated, high income, and young, with an average age close to 30, and they need not be eligible to vote. The market asks traders, with money at stake, who everyone will vote for on election day.

Accuracy against polls

The 2003 review covers 49 markets for 41 elections in 13 countries. For national elections with poll data, the market beat the final polls in 9 of 15 cases on both measures used, the election-eve price and the last week's average price. The average poll error was 1.91 percent, against market errors of 1.49 and 1.58 percent. The authors note that polls are designed to measure current opinion, not to predict outcomes.

A 2008 International Journal of Forecasting article by Berg, Nelson and Rietz tested longer horizons. The authors matched 964 national polls from the 1988 through 2004 presidential elections with the IEM vote-share price on the same day. The market was closer to the final two-party vote split 74 percent of the time, and it significantly outperformed the polls in every election when forecasting more than 100 days ahead. These comparisons come from the University of Iowa researchers behind the IEM.

How the CFTC oversees event contracts

The CFTC's June 12, 2026 proposed rule states the agency's view. Event contracts are commodity derivatives, often with a binary payoff, based on whether an event occurs, and the markets that trade them are commonly called prediction markets. The Commission first designated a prediction market as a designated contract market (DCM) in 2004. A DCM may list a contract by certifying that it complies with the Commodity Exchange Act (CEA) and CFTC rules. Since 2010, CEA section 5c(c)(5)(C) has let the Commission bar event contracts involving terrorism, assassination, war, gaming, or unlawful activity when it finds them contrary to the public interest. The proposal would set factors for that finding. Trading on CFTC-registered prediction markets exceeded $25 billion in 2025, about 0.08 percent of the $31 trillion notional value of regulated futures.

The IEM operates under staff no-action relief instead. The 2026 notice says a 1993 staff letter, superseding a 1992 one, covers small-scale, not-for-profit markets in political and economic indicator contracts and limits trader numbers and how much each trader can risk. The IEM's own FAQ says it is not regulated by the CFTC.

Language-model forecasters against crowds

Forecasts are scored with the Brier score, the squared gap between a stated probability and the outcome of 0 or 1. Lower is better, and a flat 50 percent guess scores .25. Halawi, Zhang, Yueh-Han and Steinhardt built a retrieval-augmented system and tested it on 914 binary questions from five platforms, including the markets Polymarket and Manifold. The test questions opened after the models' knowledge cut-off. The system scored .179 against the crowd's .149. On validation questions, it beat the crowd when crowd forecasts fell between .3 and .7, but trailed when the crowd was confident, likely because the system rarely gave low probabilities.

ForecastBench, presented at the International Conference on Learning Representations (ICLR) 2025, uses only questions about events with no known answer, drawn from nine sources including prediction markets. On a 200-question subset, 39 superforecasters scored 0.096, the general public 0.121, and the best model 0.122. The top models saw the crowd forecast on market questions. The best model without it scored 0.136.

What to check

• Is a market price compared with a poll, a crowd, or the outcome, and at what horizon?
• How large and how representative was the market? The IEM caps most traders at $500.
• Does the venue list contracts as a CFTC-registered DCM, or operate under staff no-action relief?
• Did an AI forecaster's test questions open after the model's training cut-off?
• Did the model see the market price or crowd forecast as an input, as the top ForecastBench models did?
• Is accuracy given as a Brier score with a baseline, such as .25 for a 50 percent guess?

Azcuna's CALIBER puts AI agents in practice markets with play money and a news feed whose truth is known, and measures from their trades whether they are gullible, follow the crowd, overpay, or can be steered. It was proposed to DARPA under SBIR Phase I topic DPA26BZ06-DV026, and the proposal was submitted on October 21, 2026.

Sources

1. 1The promise of prediction markets · Arrow et al., Science 320(5878), May 16, 2008 (CaltechAUTHORS record and abstract)
2. 2What is the IEM? · Iowa Electronic Markets, University of Iowa Tippie College of Business
3. 3Frequently Asked Questions · Iowa Electronic Markets, University of Iowa Tippie College of Business
4. 4Results from a Dozen Years of Election Futures Markets Research · Berg, Forsythe, Nelson and Rietz, March 2003 draft for the Handbook of Experimental Economics Results (Iowa Electronic Markets)
5. 5Prediction Market Accuracy in the Long Run · Berg, Nelson and Rietz, International Journal of Forecasting 24(2), April-June 2008 (Iowa Electronic Markets abstract page)
6. 6Prediction Markets; Public Interest Determinations · Commodity Futures Trading Commission, proposed rule, 91 FR 35806, June 12, 2026
7. 7Approaching Human-Level Forecasting with Language Models · arXiv (Halawi, Zhang, Yueh-Han and Steinhardt), February 2024
8. 8ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities · arXiv (Karger et al.), version 5, February 2025; ICLR 2025

Where this shows up in our work

• CALIBER, the market testbed for AI agents measures from an AI agent's trades whether it is gullible, herds, overpays, or can be steered.
• LEDGER, auditable Army acquisition decisions turns a trade study into a package where every number cites its source and AI drafts but never decides.