Dissecting Corporate Culture Using Generative AI (Python Code)

Li, Kai, Mai, Feng, Shen, Rui, Yang, Chelsea, and Zhang, Tengfei. "Dissecting Corporate Culture Using Generative AI." The Review of Financial Studies, Volume 39, Issue 1, January 2026, Pages 253–296, https://doi.org/10.1093/rfs/hhaf081

This package contains example Python code and data for extracting and analyzing corporate culture from analyst reports using generative AI.

Directory Structure 📁

code/                          # Processing modules
├── preprocessing/             # Phase 1: Text processing and segmentation
│   ├── segment_and_embed.py  # Text segmentation and embedding generation
│   ├── compute_boilerplate_prob.py # Boilerplate detection
│   ├── pdf2text.py           # PDF text extraction utilities
│   └── SegMethods.py          # Text segmentation algorithms
│
├── culture_identification/    # Phase 2: Culture segment identification  
│   ├── train_culture_classifier.py # Model training
│   ├── search_culture_segments.py  # Culture segment detection
│   ├── gpt_filter_culture.py # GPT-based filtering
│   ├── solr/                 # Optional Apache Solr integration
│   ├── cls_heads.py          # Classification model components
│   └── config.py             # Configuration settings
│
├── information_extraction/    # Phase 3: Information extraction and analysis
│   ├── extract_culture.py    # Two-round GPT extraction pipeline
│   ├── normalize_culture.py   # Normalization and standardization
│   ├── visualize_culture.py   # Sankey diagram visualization
│   ├── context_utils.py       # RAG context utilities
│   └── config.py             # Configuration settings
│
├── report_funcs.py           # Database access utilities
└── gpt_funcs.py              # GPT processing utilities

data/                         # Data storage
├── raw/analyst_reports/      # Example input analyst reports (PDF files + metadata)
├── database/                 # SQLite databases (reports, embeddings, segment boundaries)
└── processed/                # Intermediate and final results
    ├── text/                 # Processed text files
    ├── xml/                  # XML processing outputs
    ├── boilerplate/          # Boilerplate detection results
    ├── culture_segments/     # Identified culture segments
    └── culture_extraction/   # Final extraction results and visualizations
        ├── Normalized_Results/  # Standardized results
        └── visualizations/      # Generated charts and diagrams

models/                       # Trained models and weights
└── culture_cls_model/        # Culture classification model
    ├── checkpoints/          # Training checkpoints
    └── final_model_weights/  # Final trained weights

Pipeline Overview & Execution

The package implements three sequential processing phases. You can run the full pipeline or start at any phase if previous outputs exist.

Phase 1: Text Preprocessing ✂️

Location: code/preprocessing/
Purpose: Convert raw analyst reports into segmented, embedded text ready for analysis.

Execution:

cd code/preprocessing
python pdf2text.py run_full_pipeline
python segment_and_embed.py convert_emb_direct
python segment_and_embed.py convert_seg_boundary_direct
python compute_boilerplate_prob.py run_boilerplate_pipeline

Inputs: Raw analyst reports (PDF files) in data/raw/analyst_reports/

Outputs:

• data/database/reports.db - Processed text database
• data/database/embeddings.db - Sentence embeddings
• data/database/seg_boundaries.db - Segmentation boundaries
• data/processed/boilerplate/prob.csv - Boilerplate probabilities

Phase 2: Culture Identification 🔍

Location: code/culture_identification/
Purpose: Identify segments containing corporate culture discussions.

Execution:

cd ../culture_identification

# Stage 1: Keyword search with Word Sense Disambiguation
# Default search:
python search_culture_segments.py run_stage1_pipeline
# OR Optional Solr search (requires setup, see code/culture_identification/README.md):
# python search_culture_segments.py run_stage1_pipeline --engine=solr

# Stage 2: Classifier training
python train_culture_classifier.py run_stage2_pipeline

# Stage 3: GPT filtering and final combination
python gpt_filter_culture.py run_stage3_pipeline

Inputs: Processed text and embeddings from Phase 1

Outputs:

• data/processed/culture_segments/all_culture_segments.csv - Final culture segments
• data/processed/culture_segments/culture_segments_stage1.csv - Initial model predictions
• data/processed/culture_segments/high_probability_segments_gpt_filtered.csv - GPT-filtered segments
• data/processed/culture_segments/training_data.csv - Training data for model

Phase 3: Information Extraction 🗂️

Location: code/information_extraction/
Purpose: Extract structured culture information using GPT and generate analysis.

Execution:

cd ../information_extraction
python extract_culture.py run_extraction_pipeline
python normalize_culture.py run_normalization_pipeline
python visualize_culture.py

Inputs: Culture segments from Phase 2

Outputs:

• data/processed/culture_extraction/combined_extraction_results.csv - Raw extraction results
• data/processed/culture_extraction/round1_results.csv - Direct GPT processing results
• data/processed/culture_extraction/round2_results.csv - RAG-enhanced results
• data/processed/culture_extraction/extracted_causal_relationships.csv - Exploded causal relationships
• data/processed/culture_extraction/Normalized_Results/normalized_results.csv - Final normalized results
• data/processed/culture_extraction/Normalized_Results/causes_normalized.csv - Normalized causes
• data/processed/culture_extraction/Normalized_Results/consequences_normalized.csv - Normalized consequences
• data/processed/culture_extraction/visualizations/sankey_diagram_top_2.png - Cause-effect visualization

Phase-specific Documentation 📚

For detailed instructions on each phase:

• Phase 1: See code/preprocessing/README.md
• Phase 2: See code/culture_identification/README.md
• Phase 3: See code/information_extraction/README.md

Configuration ⚙️

• Install required Python dependencies:

pip install -r requirements.txt

• GPT API: Set OPENAI_API_KEY environment variable. See OpenAI documentation for more information.
• OpenRouter (Alternative):
  1. Set OPENROUTER_API_KEY environment variable.
  2. Update BASE_URL in code/culture_identification/config.py and code/information_extraction/config.py to "https://openrouter.ai/api/v1".
  3. Update GPT_MODEL to your desired model (e.g., "google/gemini-2.0-flash-001").
• Model Settings: Configure GPT model and parameters in phase-specific config.py files.
• File Paths: All paths are configured relative to the package root directory.

Output Files 📤

Final Results:

• data/processed/culture_extraction/Normalized_Results/normalized_results.csv - Complete normalized culture analysis
• data/processed/culture_extraction/Normalized_Results/causes_normalized.csv - Standardized causes
• data/processed/culture_extraction/Normalized_Results/consequences_normalized.csv - Standardized consequences
• data/processed/culture_extraction/visualizations/sankey_diagram_top_2.png - Cause-effect visualization

Key Intermediate Files:

• data/processed/culture_segments/all_culture_segments.csv - Identified culture segments
• data/processed/culture_extraction/combined_extraction_results.csv - Raw GPT extraction results
• data/processed/culture_extraction/extracted_causal_relationships.csv - Exploded causal relationships
• data/database/reports.db - Processed text database
• data/database/embeddings.db - Sentence embeddings
• data/processed/boilerplate/prob.csv - Boilerplate probabilities