Simulating opinion

In 2023, researchers at Brigham Young University published a paper whose title runs the national motto backward: "Out of One, Many" .^[1] They asked GPT-3 to adopt specific demographic personas — a 65-year-old Black woman from Mississippi, a 28-year-old white man from Oregon — to answer survey questions in character, then compared the answers against real survey data. Conditioned carefully on demographics, the model tracked human voting patterns across the 2012, 2016, and 2020 presidential elections. It captured the partisan divide by education. It reproduced regional variation. One model, conditioned a thousand ways, looked from the outside like a thousand respondents.

For a book that has spent thirteen chapters simulating what policies do to households, the question was unavoidable: could the same machinery simulate what households think about those policies? It is also the first question whose ground truth is only partial. A tax computation is right or wrong against the statute. An unemployment forecast meets its first print on a scheduled morning and takes its grade. An opinion has neither; no office certifies what a country believes, and no calendar date resolves it. The closest thing to ground truth is a survey, itself an estimate: sampling error, wording effects, a measurement of opinion rather than a certification of it. The scoreboard the last chapter built cannot operate on this ground, so its discipline has to be carried by hand — commitments registered before results, models tested on questions they cannot have memorized, misses printed beside hits. This chapter is what happened when I carried that discipline into the softest territory in the book, including into a tool of my own, half of which did not survive the grading.

An argument from the start

The field Argyle's paper opened has been an argument from its first year. The optimists had real evidence. Mei and colleagues put GPT-4 through six canonical behavioral-economics games: dictator, ultimatum, trust, and their kin. It reproduced human behavior across all six ^[2] — impressive, and on reflection expected, because those games are among the most thoroughly described experiments in the literature the model trained on. Argyle's voting result has the same character. Decades of polling documented how demographics track the vote so completely that the model absorbed the correlation and replayed it on demand. For voting, partisan identity, and familiar consumer preferences — wherever the mapping from demographics to opinion is strong and well published — a language model generates plausibly human distributions.

The skeptics had evidence just as real. Santurkar and colleagues asked whose opinions these models reflect, and found that a base model's default view represents no real population: it skews toward the liberal, educated, Western profile of its training text, a tilt that fine-tuning with human feedback shifted but did not erase .^[3] Prompting could move the default, sometimes only as far as caricature. Bisbee and colleagues compared ChatGPT's simulated survey responses with real ones and found the silicon version too tidy — less variation than humans show, and regression coefficients that diverged, sometimes sharply, from estimates on human data, which are the numbers social scientists actually use .^[4] And Park and colleagues built the demo that captivated everyone: generative agents, AI characters inhabiting a simulated town, forming plans and relationships nobody scripted .^[5] The town proved nothing about measurement, which made it the cleanest caution in the literature. Fluency is not fidelity.

Both camps were telling the truth about the region they sampled. The practice soon had a name — silicon sampling, using a language model to stand in for human survey respondents — and an operating range: strongest for well-studied populations answering well-studied questions, weakest for underrepresented groups facing novel ones. The optimists tested familiar behaviors on well-covered groups; the skeptics probed the tails. Marketing researchers, who face the question with money on the line, converged on a modest assignment for the tool: pretest the instrument, pilot the study, and field the real thing .^[6]

Who gets to ask

The reason anyone bothers is the cost of asking. Companies and governments spend on the order of $140 billion a year on the insights industry: surveys, panels, focus groups .^[7] A well-designed survey of a thousand respondents costs tens of thousands of dollars and takes weeks to field. Reword one question and you pay again. Cut the results fifty ways and the sample thins exactly where the question gets interesting. Large institutions absorb all this; startups, nonprofits, local governments, and lone researchers often cannot, so their questions go unasked. When it costs $50,000 to learn what people think, only the well-funded ask — information concentrates, and power with it. Cheap, approximate answers would change who gets to pose the question: a first-time candidate testing a message, a neighborhood association gauging support for a development, a graduate student chasing a hunch without a grant. It is the move that turned budget scoring from an institutional privilege into something a citizen can run, pointed this time at the survey.

The market was never going to wait for the methodology to settle. In February 2026 the Stanford researchers behind the generative agents emerged from stealth as Simile, calling itself "the simulation company," with a $100 million Series A led by Index Ventures and an argument that simulation, not prediction alone, is AI's next frontier .^[8] The field's most captivating demo had become a company. Synthetic respondents were going to be sold whatever the literature concluded, and for public-interest use the bar sits where the pitch rarely goes: are the numbers calibrated, is the uncertainty stated, and has anyone measured the operating range against ground truth the model could not have memorized?

A thousand personas

HiveSight is the tool I built to explore synthetic opinion, and its short history compresses the book's method into one product decision: benchmark against ground truth before believing the demo. The naive version fails on contact. Ask a frontier model whether it supports a $15 minimum wage and it returns a careful, hedged essay — the average of everything it has read, which is no individual's answer and no population's distribution. Turning up the sampling temperature adds noise, but human diversity is not noise. People differ systematically, by income and age and geography and a dozen correlated axes, so the repair is to condition the model on those axes. The obvious way to do that was the one the literature was already debating.

The first prototype built personas. From microdata it assembled a demographically detailed individual and asked the model to answer as her: you are a 45-year-old registered nurse in rural Ohio earning $62,000 a year, a weekly churchgoer, a 2020 Republican voter — do you support raising the federal minimum wage to $15 an hour? The nurse is invented, here and in the prototype; invention is what a persona is. Ask a thousand such personas, drawn from real demographic distributions, and the output has the shape of a survey: toplines, crosstabs, gradients by income and age. Watching it run feels like reading a poll of a country that answers in minutes. Benchmarked against known 2024 survey values, persona roleplay landed at roughly 25 points of mean absolute error, on toplines and subgroups alike — enough to put the majority on the wrong side of a 60–40 question.

It demoed like an oracle and scored like a guess.

Cells, not personas

The version of HiveSight that shipped in July 2026 keeps the microdata and retires the actors. Post-stratification is a workhorse of survey statistics: chop the population into demographic cells, estimate opinion within each cell, and reweight the cells to the population's known composition. It is the technique that lets a skewed online panel still call an election, and HiveSight keeps the whole apparatus, swapping a language model into the estimation slot. Roughly 150 cells per geography, drawn from populace, the same calibrated microdata the rest of the stack stands on; for each cell the model is asked not to be a person but to estimate a distribution — across people like this, how do the answers spread? — and the cell-level estimates aggregate under calibrated weights .^[9]

The mechanism explains the scores. A persona asks the model to collapse a group into one imagined individual, and the model obliges with the most probable, most agreeable, most legible version: a stereotype, or a hedge toward the center. A survey never measured an individual; it measures a spread, and the spread is what post-stratification requests, cell by narrow cell. In the prototype, the microdata had been doing the useful work all along — who is in the room, in what proportions — and the roleplay was the part adding error. The intuitive mechanism failed the benchmark; the boring one won.

The registered result

The benchmark that adjudicated all this froze a 63-item anchor bank — 52 attitude questions from the 2024 General Social Survey and 11 from the 2024 Survey of Household Economics and Decisionmaking — under a pre-analysis plan committed before a single answer was elicited: suites, estimator arms, and scoring rules fixed in advance .^[9] One control mattered more than the rest: the elicitation model's training cutoff predates every 2024 target. A model that has read 2024 is not forecasting 2024; it is remembering it.

The registered results, ties intact. The cell estimator scored about 9.2 points of mean absolute error on toplines and 9.8 on subgroups. A simpler contender — asking the model directly for the whole population's distribution, no cells at all — scored about 8.6 and 9.8. On toplines that is a tie, statistically equivalent within the registered ±1.5-point margin, and the paper prints the equivalence test instead of declaring a winner. Where the cells earn their keep is structure. They track the shape of opinion across subgroups better, rank correlation 0.62 against 0.48; they are roughly nine times steadier from run to run; they hold when the questions are reordered. A single national number comes as cheap from the direct question as from the cells. The shape of opinion — who supports, who opposes, and the same answer when you ask twice — is what the cells buy.

The state-level check came back harder. Four items scored across forty-nine states against a different anchor, the Behavioral Risk Factor Surveillance System, a health survey fielded state by state: the cells beat a naive baseline, but a national-constant guess — assume every state looks like the country as a whole — stayed competitive. On those four questions, the model's geographic signal is thin. Cheap national estimates are one thing; trustworthy fifty-state breakdowns are another, and HiveSight has earned the first, not the second.

The tie and the competitive baseline appear in the paper because the registration put them there. A pre-analysis plan is a promise, made before the numbers exist, not to relabel a loss as a win after they arrive; a tool built to be sold would have led with the subgroup victory and left the topline tie unmentioned. One caveat is mine to add: these are my numbers, from a benchmark I designed, scoring a tool I built. The registration constrains what I could shade, and the paper ships with the tool, but no independent team has rerun it. Shipping is not validation. What HiveSight has earned is a defined operating range with a measured error printed on the tin — and that is the product. A demo persuades by looking right; an instrument persuades by telling you how often it is wrong, and on which questions.

What is a tie good for? Suppose a city council is weighing congestion pricing, a $15 charge to drive downtown at peak hours, and a real survey would cost $30,000 to $50,000 and take three weeks. HiveSight can take the city's demographic composition, elicit a distribution for each cell, and return an estimate in minutes: say, 62 percent opposition, heaviest among middle-income commuters who cannot easily switch to transit, lighter among low-income residents who do not drive and high-income residents who shrug at the charge. The percentages are invented; the decision they inform is real. If simulated opposition runs 90 percent in every cell, shelve the idea and save the survey budget. If the split is close and structured, field the real survey, and go in knowing which subgroups to oversample. The estimate is a triage instrument.

Five ways to be systematically wrong

The errors that remain inside the operating range are not random, which is the good news hiding in the bad. Models train mostly on text from Western, educated, industrialized, rich, democratic populations (WEIRD, in the acronym psychologists use), so they simulate Americans better than Nigerians and college graduates better than high-school dropouts. The training data's gaps become systematic error for exactly the groups surveys already undercount.

Models also drift toward central tendencies and sand off the extremes. Real opinion has long tails, full of outliers and contradictions and people who defy their demographics, and silicon samples come out too clean — Bisbee's finding, now with a mechanism attached. A third limit is time. A model renders the world up to its training cutoff and not a day past it, so it cannot answer to the event it never saw — a pandemic, a crash, last week's viral video. The hard cutoff that made the contamination control possible makes the tool blind at the frontier, where opinion actually moves.

Safety tuning adds a fourth distortion. Trained to be agreeable, models hesitate to voice the extreme and unpopular views that some real, demographically matching humans hold: a sanitized public. And the fifth arrives with the prompt itself. Handed a demographic label, a model sometimes returns the caricature instead of the range; prompt it as a rural evangelical and you may get the cartoon.

Systematic is the operative word, because survey research already handles systematic error in its own instruments: correct it with weights learned against known answers. Measure where the model runs hot or cold on historical questions where both the demographics and the real responses are known, then adjust, and report the adjusted number with its measured error. The right output of a calibrated system is "58 percent, give or take eight, given this question type, this population, and our measured error on questions like it," never a bare "58 percent support this." This is the last chapter's forecast calibration pointed at an opinion model — estimate, resolve, score, recalibrate — with the next survey wave standing where the first print stood. The catch is generalization: a correction learned on last year's questions has to hold on next year's, or it is overfitting with extra steps, and the only honest check is grading on held-out items the correction never tuned against. The field is not there yet. The discipline is knowing that and saying so on the label.

The costume of a poll

The same cheapness that democratizes the question arms the people who abuse it. Simulate how a demographic reacts to a message and you can craft the message that moves it; a campaign can A/B test divisive framings by the thousand without exposing one real person, then deploy the winner on all of them. Silicon results wear the costume of real ones — tables, percentages, crosstabs — so a bad actor can pass a simulation off as a poll and manufacture the appearance of public support that no methodology earned. There is a slower failure too: if institutions act on synthetic opinion, and those actions reshape the information environment the next model trains on, the estimates begin to reflect opinions the tool helped create.

The quietest harm falls on dissenters. Because models default to the majority inside every group, the conservative professor, the working-class environmentalist, the minority view within a minority get rounded away — undercounted once by the training data and again by the central tendency. None of this is an argument against building; traditional polling gets manipulated too, and push polls did not wait for language models. But a real poll, however slanted, at least sampled somebody, and a simulation can conjure a consensus from no one and dress it in the same chart. The response available is the one this book keeps arriving at: build in the open, with the model, the cells, the calibration, and the error bars disclosed, and label every output an estimate, never a survey. Approximate knowledge, widely distributed and honestly labeled, can serve a public better than precise knowledge held by whoever can afford a poll. But "clearly labeled" has to be enforced with the same seriousness as the estimate is computed, because the label is all that separates a triage instrument from a counterfeit public.

The third primitive

This chapter's subject is harder than anything else in the book, and the reason is structural. A tax flows through rules; the rules encode exactly and check against an oracle to the dollar. An unemployment forecast names a number that will print on a schedule and grade it. An opinion emerges from a whole life, settles nowhere, and when a model writes "I strongly oppose this," no one is home who opposes anything. Even the anchors the benchmark graded against are surveys — noisy, wording-specific estimates of opinion, not certifications of it. The ground truth here is real but partial, and the discipline has to carry the difference: registration before results, cutoffs before targets, ties printed where they land.

Held to that standard, the tool supports two readings, both true at once. It predicts: given how people like these have answered questions like this, here is the likely distribution — the sample-to-population extrapolation survey research has always performed, with a language model standing in for the sampling weights. And it fabricates: a plausible artifact in the shape of opinion, with no opinion behind it. For deciding whether a survey is worth fielding, the first reading is enough. For anything that touches democratic legitimacy — or any group whose representation is thin and whose cost of being mis-measured is high — the second is the warning label.

Sized honestly, the work in this chapter is the embryo of a third primitive, beside the rules that state and the forecasts that resolve: values elicitation, machinery for hearing what people want. There is a version that plugs into everything this book has built — simulate a policy against the population, then simulate the public's reaction to it, the same household profiles carrying both the budget effect and the opinion, each wearing its error bars. But a distribution of what people think is only an input. What a society does with its opinions — how it counts them, weighs them, protects the dissenters the models round away, and turns the whole contested spread into one choice everyone must live under — has no benchmark at all. That question is the horizon, and the last part of this book walks out to it.