Survey Analytics NLP Pipeline

The problem

Companies run surveys with open-ended questions — "What would you improve about our product?" or "Describe your experience with customer support." The responses are valuable, but when you have thousands of them, no one reads them all. The standard approach is to hire someone to manually tag responses into categories, which is slow, expensive, and inconsistent across taggers.

The client needed a way to automatically identify the main themes in survey responses without predefined categories. The topics should emerge from the data itself, not from assumptions about what people might say.

Approach

I built an NLP pipeline with three stages: embed, reduce, cluster.

First, I used Hugging Face sentence transformers to convert each survey response into a dense embedding vector. These embeddings capture semantic meaning — responses about "long wait times" and "took forever to get help" end up close together in the embedding space even though they share no keywords.

Second, UMAP reduces the high-dimensional embeddings down to a lower-dimensional space while preserving the local neighborhood structure. This step is critical because clustering algorithms struggle in high dimensions — the curse of dimensionality makes distance metrics unreliable.

Third, HDBSCAN identifies clusters in the reduced space. Unlike k-means, HDBSCAN does not require you to specify the number of clusters upfront, and it handles noise well — not every response needs to belong to a topic. Outlier responses get labeled as noise rather than being forced into an ill-fitting cluster.

After clustering, I used the LLM to generate human-readable topic labels by looking at representative responses from each cluster. The final output is a set of topics, each with a label, a count, and example responses.

Results

The pipeline processed several thousand survey responses in minutes and identified coherent topic clusters that matched what manual analysis had found — plus a few themes the manual process had missed because they were less common but still meaningful. The client used the output to prioritize product improvements based on the actual frequency and sentiment of customer feedback.

Reflection

This was my first professional data science project, and the biggest takeaway was that preprocessing matters more than model selection. Cleaning the survey data — handling typos, removing empty responses, normalizing language — had a bigger impact on cluster quality than tuning UMAP or HDBSCAN parameters.

I also learned that HDBSCAN's noise handling is a feature, not a bug. Early on I tried to force every response into a cluster, which produced messy, overlapping topics. Letting the algorithm say "these responses do not clearly belong anywhere" produced much cleaner results. Sometimes the most useful thing a model can do is say "I do not know."