Skip to main content
search

Silicon samples are groups of AI-generated “respondents” that researchers use in place of, or alongside, real people. You give a large language model a short profile, such as a 45-year-old teacher from Ohio who votes Republican, and ask it to answer survey questions as that person would. Repeat that a few thousand times with different profiles, and you have a synthetic survey in minutes, for almost nothing.

The idea sounds too good to be true, and in important ways it is. But it is not nonsense either, and it has become one of the liveliest debates in computational communication. This guide, part of our Computational Communication section, sets out five things the research has established so far.

Key takeaways

  • Silicon samples can reproduce broad, well-known patterns in public opinion, which is why they attracted so much attention.
  • They consistently show too little variation: synthetic people agree with each other more than real people do.
  • Results can shift with prompt wording, model version, and time, which makes them hard to reproduce.
  • They are best used for pretesting and exploring ideas, not for replacing a real sample.

1. Where Silicon Samples Came From

The term comes from a 2023 paper in Political Analysis by Lisa Argyle and colleagues at Brigham Young University. They gave GPT-3 thousands of backstories built from real participants in large US surveys, then compared the model’s answers with what those kinds of people had actually said.

The match was close enough to be startling. The authors called this property “algorithmic fidelity”: the model seemed to have absorbed the fine-grained links between demographics, attitudes, and language that exist in its training data. Around the same time, economist John Horton made a parallel argument that language models could serve as simulated economic agents for cheap pilot experiments.

In this keynote, Argyle reflects on what has been learned since, including the fickle side of these models.

2. They Capture Averages Better Than People

The first wave of excitement was followed by careful replication. A 2024 study led by James Bisbee asked ChatGPT to play respondents from a major US election survey and rate how warmly they felt toward different groups.

The synthetic averages looked similar to the real ones. But the spread was much narrower. Real people with the same demographic profile disagree with each other a great deal. Synthetic people with the same profile mostly gave the same answer. When the team ran standard statistical models on the synthetic data, many relationships between variables came out differently from those in the real survey.

This matters because most research is not about averages. It is about differences: who changes their mind, which groups diverge, what predicts what. A tool that flattens differences can mislead exactly where you most need it.

3. Small Changes Move the Results

The same study found that results changed when prompts were slightly reworded, and that running identical prompts a few months apart gave different answers, because the underlying model had been updated.

For a survey researcher, that is a serious problem. A real sample from last March stays the same forever. A synthetic sample from a commercial model may be impossible to recreate.

Silicon samples strengths and weaknesses: useful for pretesting and ideas, weak on variation and subgroups
Where silicon samples are useful and where they fall short. Summary by Momentum Hub, based on Argyle and colleagues (2023) and Bisbee and colleagues (2024).

4. Some Groups Are Represented Worse Than Others

A language model learns from text, and not everyone writes the same amount of the internet. A Stanford study comparing model opinions with those of US demographic groups found substantial misalignment, with some groups matched much worse than others.

There is also a subtler risk. Research on how models portray identity groups argues that a model asked to speak “as” a member of a group tends to produce what outsiders say about that group, and to flatten the diversity inside it. For communication researchers who study minority audiences, that is close to the opposite of what a good sample should do.

5. The Newest Evidence: Right Answers, Wrong People

The most recent work pushes the critique further. A preprint posted in October 2026 proposes looking at whole respondents, not one question at a time. A real person is a combination of answers, and some combinations are common while others are rare.

Testing six language models on two US social surveys, the authors report that all of them left parts of the real population uncovered: there were kinds of real respondents with no close synthetic match. Under their main settings, the models even did worse than a simple statistical baseline. As a preprint, this has not yet been peer reviewed, but it fits the pattern of earlier findings. Silicon samples can match the answers without matching the people.

A More Promising Variation: Agents Built From Real Interviews

One line of work takes a different approach. Instead of a few demographic labels, a Stanford-led team built AI agents from two-hour interviews with 1,052 real people. Agents built from the interviews reproduced their person’s answers on a major social survey about 83% as accurately as the people reproduced their own answers two weeks later. Agents given only demographic details managed 74%.

That is impressive, and it also shows the price of realism. The quality came from rich data collected from real humans. The more a synthetic respondent resembles a person, the more you needed that person in the first place. This SICSS lecture explores what generative models can and cannot do in experiments.

How to Use Silicon Samples Responsibly

There are sensible uses, as long as you are clear about what you are doing.

  • Use them to pretest question wording and catch confusing or leading items before fieldwork.
  • Use them to brainstorm hypotheses and likely objections, then test those with real people.
  • Never report synthetic results as if they were public opinion.
  • If you publish anything based on them, state the model, version, date, and exact prompts.
  • Compare against real data whenever it exists, and report the spread, not just the average.
  • Be especially careful with small or underrepresented groups.

What This Means for Communicators and Marketers

Vendors now sell “synthetic respondents” and “AI focus groups” for audience and market research. The academic evidence suggests treating them like a clever colleague’s guess: useful for sharpening questions, risky as a basis for decisions. If you are trying to understand a specific audience, for example the habits we described in how Gen Z consumes media, there is still no substitute for asking them.

The same caution applies whenever AI is used to support judgment, a theme we return to in AI and decision making.

Key Terms for Silicon Samples

The debate over silicon samples uses some specialist language. These six terms cover most of it.

  • Persona prompt. The short description, such as age, location, and politics, that tells the model who to answer as.
  • Algorithmic fidelity. The degree to which a model reproduces the real links between people’s backgrounds and their views.
  • Variance. How much answers differ from person to person. Silicon samples tend to show too little of it.
  • Test-retest consistency. How similarly the same person answers the same questions on two occasions. It sets a ceiling for any simulation.
  • Marginals and joint patterns. Marginals are the totals for each question. Joint patterns are the combinations of answers that individual people give.
  • Digital twin. A simulated version of one specific person, usually built from rich data about them.

Most disagreements about silicon samples come down to this: matching marginals is much easier than matching joint patterns.

Quick Questions, Answered

Are silicon samples the same as synthetic data? They are one kind. Synthetic data is a broad term. Silicon samples specifically mean language models answering as simulated people.

Will better models fix the problems? Possibly some of them, but the 2026 preprint found gaps across all six models it tested. Lack of variation appears to be a stubborn feature, not a bug in one version.

Can I use them for a quick pilot? Yes. That is their strongest use. Just do not skip the real study afterwards.

Final Thoughts: A Rehearsal, Not the Performance

Silicon samples are best understood as a rehearsal. They let you practice the survey, find weak questions, and think through what might happen. They are not the audience. The research so far says the same thing in several different ways: models reproduce the outline of public opinion and miss much of the texture. Since texture is what communication research is usually after, real people remain essential. For the broader picture of how AI is changing the way we study and practice communication, see how AI is redefining the rules of communication.

Obada Kraishan

A Computational Social Scientist, Research Scholar, and Software Engineer specializing in machine learning, computational methods, and full-stack development. He leverages these skills to advance research and create innovative web solutions.