- What Topic Modeling Does (and Does Not Do)
- LDA: The Classic Approach
- Structural Topic Models: Adding Context
- BERTopic: Topics From Meaning
- The Interpretation Problem
- Large Language Models Enter the Picture
- How to Validate a Topic Model
- When Topic Modeling Is the Wrong Tool
- A Practical Topic Modeling Workflow
- Key Terms in Topic Modeling
- Quick Questions, Answered
- Final Thoughts: Topics Are a Starting Point
Topic modeling is a family of methods that scan a large collection of documents and group them into themes, without anyone labeling the documents first. Hand it 200,000 news articles, and it will suggest that some are about housing, some about elections, and some about football. It is one of the most used tools in computational communication, and one of the easiest to misuse.
The appeal is obvious. Nobody can read 200,000 articles. But a topic model does not understand your documents, and it will always return topics, whether or not they mean anything. This guide, part of our Computational Communication section, explains the main approaches in plain language and shows how careful researchers check the output.
Key takeaways
- Topic modeling is for discovery: it helps you see what is in a corpus before you know what to look for.
- LDA, the classic method, treats each document as a mix of topics. BERTopic, the popular newer method, clusters documents by meaning.
- The model produces word lists, not meanings. People still have to read, name, and judge every topic.
- Results can shift with settings and random starting points, so validation and transparency are essential.
What Topic Modeling Does (and Does Not Do)
A topic model looks for patterns of words that tend to appear together. If “rent,” “landlord,” “mortgage,” and “tenant” keep showing up in the same documents, the model groups them. The researcher then looks at that group and decides to call it “housing.”
That division of labor is the key point. The algorithm finds statistical regularities. The human supplies the interpretation. A topic model cannot tell you whether coverage was fair, whether a frame was misleading, or whether a theme matters. It gives you a map of the terrain, and maps can be wrong.
LDA: The Classic Approach
Latent Dirichlet Allocation, introduced in 2003, is the method that made topic modeling famous. It assumes every document is a blend of several topics, and every topic is a weighted list of words. A news story might be 60% economy, 30% politics, and 10% something else.
LDA has two well-known limits. First, you must tell it how many topics to find, and there is no single correct number. Second, it only sees word counts, not word order or meaning, so “bank” by a river and “bank” on a high street look identical. It also struggles with very short texts such as social media posts, because a single post contains too few words to reveal a mixture. For communication scholars, a 2018 guide in Communication Methods and Measures remains a careful walkthrough of how to apply LDA responsibly.
Structural Topic Models: Adding Context
Researchers often want to know how topics differ across groups or over time. The structural topic model builds this in. It lets the prevalence of each topic vary with information about the document, such as its date, outlet, or author.
That makes it a natural fit for questions like “did coverage of this issue change after the election?” or “do supporters of different parties give different reasons in open-ended survey answers?” The interpretation burden is the same as with LDA, but the comparisons are built into the model instead of being bolted on afterwards.
BERTopic: Topics From Meaning
BERTopic, released by Maarten Grootendorst in 2022, takes a different route. Instead of counting words first, it converts each document into an embedding, a numerical summary of its meaning produced by a language model. Documents that mean similar things end up close together, even when they use different words.
The pipeline then compresses those embeddings, clusters them, and extracts the words that best distinguish each cluster. Two practical differences follow. BERTopic can decide the number of topics from the data, and by default it assigns each document to a single topic, with unclear documents set aside as outliers. It tends to do better than LDA on short texts, which is why it has become common in social media research. Grootendorst explains the design choices in this interview.

The Interpretation Problem
In 2009, a team of researchers published a paper with a memorable title, “Reading Tea Leaves: How Humans Interpret Topic Models.” They tested whether people could spot a word that did not belong in a topic, and found that models scoring best on standard statistical measures were not always the ones people found most meaningful.
The lesson has held up. A model can fit the data well and still produce topics that are vague, mixed, or driven by quirks such as boilerplate text and repeated signatures. Numbers alone will not tell you which topics deserve trust. Reading will.
Large Language Models Enter the Picture
Language models are now changing topic modeling in two ways. They can write readable labels and summaries for clusters, which saves time. And newer approaches such as TopicGPT ask a language model to propose and assign topics directly in natural language.
These tools make results easier to read, which is useful and also a little dangerous. A fluent label can make a messy cluster look tidy. The checks below matter even more when the output sounds confident. For where this is heading, this conversation on agent-style topic modeling is a good listen.
How to Validate a Topic Model
Good topic modeling studies show their checking. Look for these steps, and use them yourself.
- Read the most representative documents for every topic, not just the top words.
- Run the model several times with different settings and random starts, and keep the topics that stay stable.
- Ask someone outside the project to label a sample of documents, then compare their labels with the model’s.
- Check that topics behave sensibly against things you already know, such as coverage spikes around known events.
- Report how many documents were left as outliers or fell into junk topics.
When Topic Modeling Is the Wrong Tool
Topic modeling is exploratory. If you already know your categories, for example “mentions climate policy” or “attacks an opponent,” a classifier trained on labeled examples or a validated language model prompt will usually be more accurate and easier to defend. We compare those options in our overview of AI and big data in communication research.
It is also a poor fit for very small collections. With a few hundred documents, careful reading beats any algorithm.
A Practical Topic Modeling Workflow
- Write down your question first, so you know what a useful topic would look like.
- Clean the corpus: remove duplicates, boilerplate, and signatures that create fake topics.
- Start with BERTopic for short texts or LDA and structural models for long documents and group comparisons.
- Name topics only after reading at least 20 documents from each.
- Merge or drop topics you cannot explain, and say that you did.
- Confirm your main finding with a second method before you publish or present it.
If privacy is a concern, embeddings can be produced on your own machine with open models, a route we discussed in our guide to open models.
Key Terms in Topic Modeling
Topic modeling papers lean on a small set of terms. Here they are in plain English.
- Topic. In this context, a group of words or documents that tend to occur together. It is a statistical pattern, not a guaranteed theme.
- K. The number of topics a model is asked to find. In LDA the researcher sets it.
- Coherence. A score that estimates how well a topic’s top words belong together. Useful, but not a substitute for reading.
- Outliers. Documents the model could not place confidently. BERTopic sets these aside instead of forcing them into a topic.
- Embedding. A numerical summary of a document’s meaning, used by newer methods to group similar texts.
- Stability. Whether the same topics reappear when you rerun the model with different settings or random starts.
When a topic modeling study reports K, outliers, and stability checks, you can judge its results with much more confidence.
Quick Questions, Answered
How many topics should I ask for? There is no correct number. Try several, compare how interpretable the results are, and choose the level of detail that fits your question.
Is BERTopic always better than LDA? No. It often handles short texts better, but LDA and structural topic models are well understood and work well for long documents and comparisons across groups.
Can I trust topic labels written by AI? Treat them as drafts. Check each label against the documents inside the topic before you rely on it.
Final Thoughts: Topics Are a Starting Point
Topic modeling is at its best as the first step in understanding a large body of text, not the last. It shows you where to look. The findings come from what you do next: reading, comparing, and testing. Used that way, it is one of the most valuable tools in the kit, and a good example of how algorithms and human sensemaking work better together than apart.



