Buffer Bubbles is an exploratory data visualization project for understanding what Buffer users are asking for on the public suggestion boards.
Live app: https://buffer-bubbles.danielubenjamin.com/
Public suggestion boards are useful, but they are not always enough.
A suggestion board is good at showing individual posts. It lets users submit ideas, vote, comment, and track whether something is open, planned, or in progress.
But when you are trying to understand product demand at a higher level, the raw board has a few problems.
Users often ask for the same thing in different ways.
For example:
Add WhatsApp support
Let me schedule messages to WhatsApp groups
Support WhatsApp Business
Post to WhatsApp communitiesThese may be separate posts, but they all point toward the same underlying product theme.
A normal suggestion board shows these as different requests. Buffer Bubbles tries to collapse them into a single category so the demand is easier to see.
The latest request is not always the most important request.
A feature request posted today may have one vote, while an older request may represent a much larger recurring pain point. Looking only at the latest posts makes it easy to miss repeated themes.
Buffer Bubbles tries to make demand visible by grouping requests and showing cluster size, votes, and comments.
A request with many votes may be important, but votes do not tell the whole story.
A stronger signal can come from a combination of:
number of similar requests
+ total votes
+ total comments
+ board/category
+ request statusBuffer Bubbles keeps those signals visible instead of hiding them inside individual posts.
If someone is preparing for a product, engineering, or growth conversation, they need more than a list of links.
They need to be able to say:
I found several independent requests around WhatsApp publishing.
They were not always worded the same way, but they cluster around the same user need.
The theme appears across multiple requests and has visible engagement.That is the gap this project tries to close.
Buffer Bubbles turns public Buffer suggestion data into an interactive feature-demand map.
At a high level, it:
- crawls public Buffer suggestion pages,
- extracts feature request content and metadata,
- cleans and normalizes the request text,
- converts each request into an embedding,
- groups semantically similar requests into clusters,
- scores and summarizes each cluster,
- displays the clusters in an interactive bubble chart.
┌────────────────────────────┐
│ Buffer suggestion boards │
│ suggestions.buffer.com │
└──────────────┬─────────────┘
│
│ crawl pages
▼
┌────────────────────────────┐
│ Crawler │
│ │
│ discovers request URLs │
│ opens request pages │
│ extracts visible content │
└──────────────┬─────────────┘
│
│ raw request records
▼
┌────────────────────────────┐
│ Normalizer │
│ │
│ cleans title/body │
│ keeps votes/comments/status │
│ builds summaries │
└──────────────┬─────────────┘
│
│ clean documents
▼
┌────────────────────────────┐
│ Embedding step │
│ │
│ title + body -> vector │
│ similar meaning -> nearby │
└──────────────┬─────────────┘
│
│ vectors
▼
┌────────────────────────────┐
│ Clustering │
│ │
│ groups similar requests │
│ creates category candidates │
└──────────────┬─────────────┘
│
│ feature clusters
▼
┌────────────────────────────┐
│ Scoring and summaries │
│ │
│ request count │
│ total votes │
│ total comments │
│ representative examples │
└──────────────┬─────────────┘
│
│ JSON/CSV output
▼
┌────────────────────────────┐
│ Interactive UI │
│ │
│ bubble chart │
│ filters │
│ request drill-down │
└────────────────────────────┘.
├── README.md
├── buffer-bubbles-home.png
├── Dockerfile
├── run.sh
├── crawler/
│ ├── pyproject.toml
│ ├── uv.lock
│ ├── rank.py
│ ├── main.py
│ ├── buffer_requests_raw.csv
│ ├── buffer_requests_clustered.csv
│ └── buffer_feature_clusters.json
└── frontend/
├── package.json
├── vite.config.ts
└── src/
├── App.tsx
├── main.tsx
├── index.css
├── data/clusters.json
└── components/ui/Important files:
crawler/rank.py— crawler, clustering, ranking, and output generationrun.sh— runs the crawler and copies JSON into the frontend datasetfrontend/src/App.tsx— the interactive UIfrontend/src/data/clusters.json— the current frontend datasetDockerfile— builds and serves the frontend as a static app
The crawler is implemented in crawler/rank.py.
The source site is treated as a dynamic website rather than simple static HTML. That matters because many modern feedback tools render meaningful content on the client side. A plain HTTP request may only return shell HTML, while the actual posts appear after JavaScript runs.
Because of that, the crawler is designed around browser automation.
It uses:
- Playwright to load and inspect Buffer suggestion pages
- BeautifulSoup as a fallback parser for HTML extraction
- pandas to shape the extracted data into tables
- sentence-transformers to embed request text
- scikit-learn / DBSCAN to group semantically related requests
The crawler starts from three board URLs:
https://suggestions.buffer.com/b/feature-suggestions
https://suggestions.buffer.com/b/new-channel-requests
https://suggestions.buffer.com/b/buffer-api┌────────────────────────────────────────────────────────────┐
│ Crawler │
├────────────────────────────────────────────────────────────┤
│ │
│ 1. Open board page │
│ Example: feature suggestions, new channel requests │
│ │
│ 2. Wait for page content │
│ Let the client-rendered UI load posts │
│ │
│ 3. Scroll through the board │
│ Trigger lazy-loaded results │
│ │
│ 4. Extract request links │
│ Collect individual suggestion URLs │
│ │
│ 5. Visit each request page │
│ Extract title, body, status, votes, comments, board │
│ │
│ 6. Deduplicate by URL │
│ Avoid counting the same suggestion twice │
│ │
└────────────────────────────────────────────────────────────┘The output of the crawler is a list of raw request records.
A request record looks conceptually like this:
{
"board": "feature suggestions",
"url": "https://suggestions.buffer.com/...",
"title": "Support posting to WhatsApp groups",
"body": "It would be helpful if Buffer could...",
"status": "open",
"votes": 91,
"comments": 16
}The crawler does not decide what is important. It only collects the source data and preserves useful metadata for later analysis.
If buffer_requests_raw.csv already exists and FORCE_CRAWL is not set to true, the pipeline skips re-scraping and reconstructs items from the cached CSV via load_cached_items(...).
That makes it easier to iterate on clustering and visualization without crawling the site every time.
The project separates crawling from clustering because those are different responsibilities.
Crawler responsibility:
Get the data
Clustering responsibility:
Understand the data
UI responsibility:
Let a human explore the dataThis separation makes the system easier to debug.
If the UI looks wrong, the first question is:
Is the data wrong?
Is the clustering wrong?
Or is the visualization wrong?Keeping the stages separate makes that easier to answer.
The clustering step tries to answer one question:
Which requests are probably talking about the same underlying product need?It does this using semantic similarity.
Instead of comparing only exact words, it converts each request into a vector representation. Requests with similar meaning should end up close to each other in vector space.
The current code builds a combined_text field from:
title + bodyand then encodes it with:
SentenceTransformer("all-MiniLM-L6-v2")Those embeddings are normalized and clustered using DBSCAN with cosine distance:
eps = 0.22
min_samples = 2
metric = "cosine"┌────────────────────────────┐
│ Clean request text │
│ title + body │
└──────────────┬─────────────┘
│
▼
┌────────────────────────────┐
│ Generate embeddings │
│ one vector per request │
└──────────────┬─────────────┘
│
▼
┌────────────────────────────┐
│ Compare semantic distance │
│ near vectors are similar │
└──────────────┬─────────────┘
│
▼
┌────────────────────────────┐
│ Cluster similar requests │
│ each cluster = one theme │
└──────────────┬─────────────┘
│
▼
┌────────────────────────────┐
│ Generate category labels │
│ from repeated terms │
└──────────────┬─────────────┘
│
▼
┌────────────────────────────┐
│ Score and rank clusters │
│ count + votes + comments │
└────────────────────────────┘DBSCAN can label isolated requests as noise (-1).
This project does not throw those requests away. Instead, each noise point gets a synthetic cluster id so every original request still appears in the final JSON and UI.
Cluster labels are generated heuristically by keyword_label(...).
The code extracts tokens from grouped texts, removes a custom stopword list, counts frequent terms, and joins the top keywords with /.
That is why categories look like this:
ago / post / comments / posts
instagram / posts / ago / post
channel / ago / channels / accountsThese are not polished product labels. They are lightweight summaries meant to help navigation.
The crawler ranks clusters by:
request_counttotal_votestotal_comments
descending.
The UI also computes a simple priority score:
priority_score =
request_count * 3
+ total_votes * 2
+ total_commentsThis is intentionally simple. The point is not to claim final truth, but to make repeated demand easier to compare.
A cluster represents a product theme plus the evidence behind it.
Each cluster record in the JSON includes things like:
cluster_idcategoryrequest_countboardsstatusestotal_votestotal_commentsrepresentative_titlesrepresentative_urlsitems
That means the UI does not just say:
This theme is popularIt can also show the underlying request threads behind that claim.
The UI uses a bubble chart because the main thing we want to see is concentration.
A table can show exact numbers, but it does not immediately show the shape of demand.
A bubble chart makes it easier to see:
- which themes dominate the board,
- which smaller themes are still worth inspecting,
- which clusters deserve manual review,
- how the picture changes when you size by requests, votes, or comments.
The frontend is a Vite + React + TypeScript app centered in frontend/src/App.tsx.
The deployed UI is designed as an exploration tool. It gives you:
- a zoomable bubble canvas for high-level demand themes,
- search for finding a specific topic,
- multi-select filters for board and status,
- metric controls for changing bubble size,
- a detail panel for inspecting the selected cluster,
- representative requests with source links.
User sees filtered cluster landscape
│
▼
Clicks a bubble
│
▼
Reads category details
│
▼
Inspects original feature requests
│
▼
Uses evidence to decide what is worth investigatingfrontend/src/data/clusters.json
|
v
React state in App.tsx
|
+------------------------+
| |
v v
filter/search pipeline selected cluster state
| |
v v
filtered cluster list right-hand detail panel
|
v
D3 force layout + canvas draw
|
v
zoomable bubble explorationAt the time of inspection, the live app included:
- the eyebrow label Feature request intelligence,
- the title Interactive view of aggregated Buffer feature requests,
- a search box for categories, titles, and summaries,
- board filters,
- status filters,
- sizing toggles for requests, votes, and comments,
- overview metric cards,
- a scrollable right-hand detail panel.
The chart is rendered on canvas, not SVG. That matters because the current UI supports:
- many bubbles on screen,
- hover tooltips,
- click hit-testing,
- pan and zoom,
- auto-fit to content bounds,
- label suppression on tiny bubbles.
Representative request cards open the original Buffer suggestion pages in a new tab.
- Python 3.13+ for the crawler
- Node 20+ for the frontend / Docker build path
uvfor the crawler workflowpnpmfor the frontend
From the repo root:
./run.shWhat this does:
repo root
|
+--> run.sh
|
+--> cd crawler
+--> uv run python rank.py
+--> produce buffer_feature_clusters.json
+--> copy to frontend/src/data/clusters.jsoncd crawler
FORCE_CRAWL=true uv run python rank.pyUseful environment variables:
HEADLESS=true|falseMAX_POSTS_PER_BOARD=200FORCE_CRAWL=true|false
cd frontend
pnpm install
pnpm devcd frontend
pnpm buildThe root Dockerfile builds the frontend and serves the compiled static files with serve.
Build and run:
docker build -t buffer-bubbles .
docker run --rm -p 8080:8080 buffer-bubblesThis image packages the frontend only. It does not run the crawler inside the container.
Buffer Bubbles is useful for product exploration, interview preparation, and customer research.
It can help answer:
- What are the biggest repeated requests?
- Which channels are users asking Buffer to support?
- Which workflow issues come up repeatedly?
- Are users asking for more analytics, more scheduling control, or more collaboration features?
- Which themes have enough evidence to justify deeper investigation?
It is especially useful when preparing for conversations where you want to show that you did not just read one or two feature requests, but looked for patterns across many of them.
This project is not a replacement for product judgment.
A large cluster does not automatically mean Buffer should build that feature.
There are still many questions to ask:
- Does the platform API support it?
- Is the request aligned with Buffer's product direction?
- Would it help the right customer segment?
- Would it increase activation, retention, or revenue?
- Is it technically feasible?
- Would it create maintenance or support burden?
- Are there policy, permission, or platform risks?
Buffer Bubbles helps surface demand. It does not decide strategy.
The people who submit suggestions are only a subset of users.
A cluster may show visible public demand, but it does not necessarily represent all customers.
Semantic clustering is useful, but it can still make mistakes.
Some requests may be grouped together even when they are different. Some requests may remain separate even when they should be merged.
The best workflow is:
machine clusters first
human reviews secondVotes can be influenced by age of request, visibility, wording, and how easy it was for users to find the post.
That is why Buffer Bubbles keeps votes as one signal instead of treating votes as the only signal.
Cluster labels are generated from request content. They are meant to help navigation, not replace reading the original requests.
For serious analysis, inspect the representative requests inside each cluster.
- Add manual merge/split controls for clusters
- Add a table view sorted by priority score
- Add trend detection to show which themes are becoming more common
- Add export to Markdown for interview notes
- Add local LLM support for better cluster titles
- Add a confidence score for each cluster
- Track changes over time by running the crawler periodically
- Compare public demand against Buffer's public roadmap or changelog
1. Run the crawler
2. Generate clusters
3. Open the bubble chart
4. Size by request count, votes, or comments
5. Inspect the largest clusters
6. Check representative source links
7. Write down the clearest repeated themes
8. Decide which themes are worth discussing furtherThe goal is not to walk into a conversation with a forced answer.
The goal is to walk in with evidence.
Buffer Bubbles takes a public feature suggestion system and turns it into a structured product-research interface.
Instead of asking:
What are the latest feature requests?It asks:
What are users repeatedly trying to tell us?