What this automation does
Every comment customers leave, wherever they leave it, ends up tagged in one place, so you can answer what customers complain about most and whether it is getting better.
- Collect. Scheduled pulls bring in new support tickets and conversations from Zendesk or Intercom, survey responses from Typeform, reviews, and sales call notes from HubSpot. Each row keeps its source, date and customer ID.
- Clean. Signatures, quoted email threads, greetings and boilerplate are removed. Obvious personal identifiers are masked before any text leaves your systems.
- Split. A model breaks each comment into separate points. “Love the product but shipping took two weeks and support never answered” becomes three points with their own sentiment.
- Tag. Each point is tagged against your taxonomy (for example Product, Pricing, Shipping, Support, Onboarding, each with subtopics), plus sentiment, severity and whether it mentions a competitor.
- Enrich. Customer IDs are joined to CRM data for plan, revenue and tenure.
- Report. A dashboard shows theme volume, trend, sentiment and revenue at risk by theme. A monthly AI summary describes the biggest movers with example quotes linked to source records.
When it is worth building
The signal is that feedback exists in quantity but nobody reads it systematically. Product decisions get made from the loudest recent anecdote. Support knows customers hate a feature, but cannot prove how many. If you have more than a few hundred comments a month across two or more channels, manual reading stops scaling.
If most of your feedback comes through one survey tool that already has built-in text analysis, use that first. Dedicated feedback analytics platforms are also a reasonable buy if you want a polished UI and many integrations. A custom pipeline fits when you need your own taxonomy, revenue weighting from your CRM, data kept in your own database, and outputs wired into tools such as churn risk alerts.
Architecture
| Layer | What we use | Why |
|---|---|---|
| Sources | Zendesk, Intercom, Typeform, HubSpot, review sites | Built-in n8n nodes for Zendesk and HubSpot and a Typeform trigger; HTTP Request for Intercom conversations and review sites |
| Orchestration | n8n on hourly or nightly schedules | Incremental pulls with checkpoints per source |
| Storage | Supabase (Postgres with pgvector) | Relational data for joins and reports, vectors for similarity search later |
| Splitting and tagging | Claude Haiku 5.5 with structured output | High volume, short inputs, strict schema; cost per comment stays low |
| Monthly synthesis | Claude Sonnet 5.5 | Reads tagged data and representative quotes to write the narrative |
| Taxonomy | A versioned table in Postgres | Changes are deliberate and dated so trends stay honest |
| Reporting | Looker Studio or your BI tool | Filters by theme, source, segment and time |
How to build it
- Draft the taxonomy from real data. Read 200 recent comments by hand with the product and support leads. Agree on 8 to 15 top-level themes with short definitions and examples.
- Build incremental collectors. Pull only new or updated records per source, store a checkpoint, and keep the raw text alongside a cleaned version.
- Mask personal data. Replace emails, phone numbers and order numbers with placeholders before model calls. Keep the mapping inside your database only.
- Split and tag with a strict schema. One call per comment returns an array of points, each with a theme ID from the taxonomy, sentiment, severity and a short quote. Reject any theme ID not in the table.
- Collect proposed themes. Allow the model to return “other” with a suggested label. Review these monthly and promote real patterns into the taxonomy with a new version number.
- Join to CRM and build the dashboard. Report counts, share of total, sentiment and revenue by theme, with drill-down to the original comments.
- Audit monthly. Hand-tag a random sample of 100 points and compare with the model. Fix definitions where disagreement clusters.
The downloadable starter template below covers Zendesk and Typeform collection, masking, splitting and tagging into Supabase.
What breaks in production
- Taxonomy creep. Adding themes every week makes old and new months incomparable. Version the taxonomy and change it on a schedule.
- Volume spikes from one incident. An outage produces hundreds of identical complaints and drowns every other theme. Detect bursts and show them as incidents, separate from ongoing themes.
- Survey bias. Survey responders and support contacts are not your whole customer base. Label every chart by source so people do not mistake ticket volume for overall sentiment.
- Short, ambiguous comments. “Fine” and “meh” carry little meaning. Tag them as low-information instead of forcing a theme.
- Model upgrades shift tags. A new model version can tag differently. Re-run your audit sample before switching and keep the model version on every tagged row.
Timeline and cost
A pipeline with three or four sources, splitting and tagging, CRM enrichment and a dashboard usually takes two to four weeks. Lumeor builds this in the range of $6,000 to $12,000, depending on the number of sources, volume, the depth of the taxonomy work and whether embeddings and similarity search are included. Model usage scales with comment volume, and short comments on a small model keep it modest for most mid-sized companies. Our n8n automations suite case study is a larger example of classifying incoming messages with an LLM and storing them for analytics. Reviews collected by the review response workflow can feed straight into this pipeline.
How to measure success
- Agreement rate between model tags and the monthly human audit sample
- Share of comments tagged to a defined theme rather than “other”
- Time from a new issue appearing in feedback to it being on a team’s backlog
- Product or process changes traced to a reported theme
- Volume and sentiment trend for themes the business acted on