Clustering

Clustering

Overview

Clustering in CRISP-T groups documents or numeric observations to surface latent structure that can be triangulated with qualitative themes and emerging theoretical categories. K-Means is currently implemented (--kmeans). Clustering metadata enables selective retrieval and comparison of grouped narratives for grounded theory development.

Implementation Details

• --kmeans within --ml triggers numeric clustering on corpus.df.
• Cluster labels are added to document metadata when IDs align or when filtering propagates across modalities.
• Visual exploration (e.g., PCA projection) can be generated with --pca and inspected via --print metadata --print pca.

Sense-making and Triangulation

1. Cluster numeric data (e.g., patient metrics).
2. Filter documents by cluster: crisp --inp crisp_input --filters cluster=0.
3. Inspect topics within cluster-specific documents; note convergent or divergent thematic content.
4. Form memos comparing cluster-level quantitative traits with qualitative categories.

Business Example

A dataset of customer usage metrics forms clusters where one group exhibits high feature_adoption_rate and low support_tickets. Narratives from this cluster emphasize "intuitive interface". Cluster characteristics and textual themes cohere around a theoretical proposition about usability enhancing adoption.

Medical Example

Patient behavior metrics (sleep, activity) produce a cluster with low sleep_balance and high travel_time. Narratives in this cluster frequently reference "work schedule disruption". This alignment suggests a contextual category explaining fatigue variability.

Evaluating Clusters

• Use inertia or silhouette (future enhancement) for quantitative validation (see Future Plan).
• Examine distribution of qualitative codes across clusters for theoretical saturation.
• Apply semantic search within a cluster to find prototypical documents.

Limitations

• K-Means assumes spherical clusters and may misrepresent complex manifolds—Ball Mapper (TDABM) offers complementary shape analysis.
• High dimensionality without reduction may produce unstable cluster assignments.

Best Practices

• Run PCA before clustering to mitigate noise.
• Integrate cluster IDs into document metadata to streamline thematic comparison.
• Treat outlier clusters as prompts for focused qualitative re-coding.

References

• Mettler et al. (2025) — Reflective guidance on computational textual methods.
• Rudkin & Dlotko (2024) — Structural analysis via Ball Mapper.

See Also

• Predictive Methods
• Topological Data Analysis
• Human Sense-making and triangulation