NLP based techniques

NLP based techniques

Overview

CRISP-T includes a suite of NLP capabilities to support qualitative analysis: sentiment analysis (VADER), extractive summarization, topic modeling (LDA), sentence-level scoring, and semantic search (via ChromaDB). These techniques facilitate grounded coding, constant comparison, and targeted theoretical sampling.

Implemented Functions

• --sentiment: VADER sentiment scores (document/sentence-level with --sentence).
• --summary: Extractive summarization to condense key points.
• --nlp: Composite switch to run text analyses together.
• --semantic, --semantic-chunks: Similarity-based retrieval for documents and chunks (via crispt).

Triangulation

• Compare sentiment polarity with numeric outcomes (e.g., negative sentiment narratives vs. churn probability).
• Use summaries to rapidly review cluster-specific documents.
• Apply chunk-level semantic search to validate category definitions with exemplar passages.

Examples

Business

Negative sentiment concentrated in high api_errors_rate accounts aligns with decision tree splits; summaries highlight recurring "integration pain" phrases.

Medical

Sentence-level sentiment peaks in narratives describing "sleep disturbance." Logistic regression confirms sleep_balance relevance to high_fatigue.

Best Practices

• Use --sentence for fine-grained validation of codes.
• Combine with topic assignment to create thematic-sentiment matrices.
• Record query strings and thresholds in memos for reproducibility.

See Also

• Latent Dirichlet Allocation
• Summarization techniques
• Semantic filtering
• Human Sense-making and triangulation