PHBench: Predicting Series A Funding from Product Hunt Launch Signals

PHBench is an open benchmark for predicting whether a Product Hunt launch will raise Series A funding within 18 months. It covers 67,292 featured launches across seven years (2019–2025), matched to 528 verified Series A outcomes via Crunchbase domain linking.

Key Results

Model	F₀.₅ (val)	AP (val)	F₀.₅ (test)	AP (test)
Top-3 Ensemble (champion)	0.284	0.126	0.097	0.037
LR Baseline	0.127	0.047	0.045	0.024
Gemini 3 Flash (LLM)	0.129	0.034	—	—
Random	0.000	0.008	—	—

The champion model achieves 4.7× lift over random on the held-out test set (paired bootstrap p < 0.001 vs LR baseline).

Quick Start

1. Get the data

Request access to the dataset on HuggingFace:

👉 huggingface.co/datasets/ihlamury/phbench

2. Install dependencies

pip install -r requirements.txt

3. Engineer features

from phbench.features import engineer_features
import pandas as pd

train = pd.read_csv("phbench_public_train.csv", dtype=str)
X = engineer_features(train)  # 61 features

4. Train a baseline

python phbench/baseline.py

5. Evaluate

from phbench.evaluate import evaluate_submission
import numpy as np

y_true = ...  # binary labels
y_prob = ...  # your predicted probabilities
metrics = evaluate_submission(y_true, y_prob, model_name="My Model")

6. Submit to the leaderboard

Send your predictions CSV to benchmark@vela.partners. We score against private test labels and update the leaderboard at phbench.com.

Repository Structure

phbench/
├── phbench/                  # Core benchmark code
│   ├── features.py           # 61 engineered features
│   ├── evaluate.py           # Official 5-metric evaluation
│   ├── baseline.py           # Train LR, XGB, LGBM, RF baselines
│   ├── llm_vanilla.py        # LLM zero-shot evaluation protocol
│   ├── build_dataset.py      # Merge posts + URLs + ranks
│   ├── join_crunchbase.py    # Join with Crunchbase labels
│   ├── split_dataset.py      # Reproduce train/val/test split
│   └── score_submission.py   # Score submissions
│
├── scripts/                  # Data collection pipeline
│   ├── ph_pipeline.py        # Pull PH posts via GraphQL API
│   ├── resolve_urls_fast.py  # Resolve website URLs
│   ├── ph_enrich_comments.py # Fetch comments
│   └── ph_enrich_ranks.py    # Fetch daily/weekly/monthly ranks
│
├── notebooks/
│   ├── 01_eda.ipynb          # Exploratory data analysis
│   └── 02_getting_started.ipynb
│
├── paper/                    # NeurIPS 2026 submission
│   └── phbench_neurips_2026.tex
│
├── data/
│   ├── benchmark/            # Metadata (data card, leaderboard, Croissant)
│   └── experiments/          # 144-experiment log + LLM results
│
├── models/                   # Champion model configs (JSON)
├── figures/                  # EDA figures
└── sample_data/              # Small illustrative samples (10 rows each)

Evaluation Metrics

Metric	Description
F₀.₅ (primary)	Precision-weighted harmonic mean. Ranks the leaderboard.
AP	Average Precision — threshold-free ranking quality
AUC-ROC	Discriminative performance across full distribution
P@50	Precision in top 50 predictions
P@100	Precision in top 100 predictions

Dataset

• 67,292 featured Product Hunt posts (2019–2025)
• 528 verified Series A raises within 18 months
• 61 engineered features per post
• 0.78% positive rate (1 in 127)
• Splits: train (47,071) / validation (6,753) / test (13,468, labels private)

See data/benchmark/data_card.md for full documentation.

Citation

@article{ihlamur2026phbench,
  title   = {PHBench: A Benchmark for Predicting Startup Series A Funding
              from Product Hunt Launch Signals},
  author  = {Ihlamur, Yagiz and Griffin, Ben and Chen, Rick},
  journal = {arXiv preprint arXiv:2605.02974},
  year    = {2026},
  url     = {https://arxiv.org/abs/2605.02974},
}

License

This work is licensed under CC BY 4.0. The dataset may not be used to target or solicit specific companies or individuals.

Contact

• Website: phbench.com
• Email: benchmark@vela.partners
• Paper: arxiv.org/abs/2605.02974