GRACE: GRounding Attributes for Clustering via External Semantics

This repository contains the source code for the paper:

GRACE: LLM-Grounded Semantic Metric Spaces for Scalable Mixed-Data Clustering

Overview

GRACE introduces LLM world knowledge into heterogeneous attributed data clustering. It converts categorical and numerical attribute values into semantically grounded natural-language descriptions, encodes them into a unified embedding space, and refines the resulting similarity via dual-view neighborhood consistency (DVNC) ,cross-validating semantic affinities against statistical evidence from the raw features. The refined representation is model-agnostic and compatible with diverse clustering algorithms.

Repository Structure

GRACE/
├── main.py                 # Entry point (Dermatology demo)
├── requirements.txt
├── grace/
│   ├── model.py            # Core GRACE and GRACE-A algorithms
│   ├── encoder.py          # InfoNCE fine-tuned semantic encoder
│   └── utils.py            # Utility functions (ACC, REQ, Otsu, etc.)
├── data_loader.py          # Dataset loader (Dermatology demo)
├── data/                   # 20 UCI benchmark datasets
├── descriptions/           # Pre-generated LLM descriptions (JSON)
└── prompts/
    └── prompts.md          # LLM prompts for all 22 datasets

Quick Start

# Environment
conda create -n grace python=3.10 -y && conda activate grace
pip install -r requirements.txt

# Run GRACE on Dermatology
python main.py

# Run GRACE-A (anchor-accelerated)
python main.py --anchor

# GPU encoding
python main.py --device cuda

Programmatic Usage

from grace.encoder import pretrain_encoder, collect_descriptions_for_training
from grace.model import run_grace
from data_loader import load_dermatology, load_descriptions

data = load_dermatology()
descriptions = load_descriptions("Dermatology")

# Hyperparameters: tau=0.05, lr=2e-5, epochs=3, batch_size=32
all_texts = collect_descriptions_for_training(descriptions)
encoder = pretrain_encoder(all_texts)

labels = run_grace(
    features=data["features"],
    feature_names=data["feature_names"],
    descriptions=descriptions,
    encoder=encoder,
    n_clusters=6,
)

Applying GRACE to a New Dataset

1. Prepare the prompt using the four-perspective template in prompts/prompts.md (Core · Indicator · Pattern · Distinction).
2. Query an LLM (GPT, Claude, DeepSeek, or Gemini) with the prompt.
3. Save the output as a JSON file in descriptions/ with keys formatted as attributeName_value.
4. Implement a loader in data_loader.py returning a dict with features, feature_names, true_labels, and dataset_name.

Datasets

Twenty public UCI datasets are used, covering purely categorical and mixed settings.

#	Dataset	Abbr.	(d_c, d_n)	n	K
1	Lenses	LE	(4, 0)	24	3
2	Caesarian Section	CS	(3, 2)	80	2
3	Zoo	ZO	(16, 0)	101	7
4	Autism Adolescent	AA	(18, 2)	104	2
5	Lymphography	LY	(18, 0)	148	4
6	Teaching Assistant	TA	(4, 1)	151	3
7	Amphibians	AM	(12, 3)	189	2
8	Soybean (Large)	SO	(35, 0)	266	15
9	SPECT Heart	SH	(22, 0)	267	2
10	Breast Cancer	BC	(9, 0)	286	2
11	Heart Disease	HD	(8, 5)	303	5
12	Primary Tumor	PT	(17, 0)	339	22
13	Dermatology	DE	(33, 1)	366	6
14	Chronic Kidney Disease	CK	(13, 11)	400	2
15	Congressional Voting	CV	(16, 0)	435	2
16	Statlog Australian	SA	(8, 6)	690	2
17	Car Evaluation	CA	(6, 0)	1,728	4
18	Auction Verification	AV	(7, 1)	2,043	2
19	Obesity Levels	OL	(8, 8)	2,111	7
20	Splice	SP	(60, 0)	3,190	3
21	Mushroom	MU	(21, 0)	8,124	2
22	Adult	AD	(8, 6)	48,842	2

License

This project is licensed under the MIT License - see the LICENSE file for details.