A document classification system that assigns multiple hierarchical labels to documents using active learning to minimize labeling costs while maximizing model performance.
This system is designed for knowledge management scenarios where documents need to be tagged with multiple, potentially hierarchical labels. For example, legal documents might be tagged with jurisdiction, practice area, document type, and key issues simultaneously.
The system combines traditional NLP techniques with modern active learning strategies to build an effective classifier while minimizing the human labeling effort required.
- Multi-Label Classification: Assign multiple labels to each document using classifier chains and label powerset approaches
- Hierarchical Label Support: Respect label taxonomy constraints during classification
- Active Learning: Minimize labeling costs through uncertainty sampling and query-by-committee
- Semi-Supervised Learning: Leverage unlabeled document corpora to improve model performance
- Domain-Aware Feature Engineering: TF-IDF with sublinear scaling and custom n-gram extraction for domain terminology
- TF-IDF vectorization with sublinear term frequency scaling
- Custom n-gram extraction optimized for domain-specific terminology
- Document preprocessing pipeline with configurable tokenization
- Classifier chains to capture label dependencies
- Label powerset with feature hashing for tractable multi-label handling
- Hierarchical classification respecting parent-child label relationships
- Uncertainty sampling to select most informative samples
- Query-by-committee with ensemble disagreement metrics
- Batch mode active learning for efficient annotation sessions
multi-label-doc-classifier/
├── src/
│ ├── features/ # Feature extraction and preprocessing
│ ├── models/ # Classification models
│ ├── active_learning/ # Active learning strategies
│ ├── evaluation/ # Metrics and evaluation
│ └── utils/ # Utility functions
├── data/
│ ├── raw/ # Raw document data
│ ├── processed/ # Preprocessed features
│ └── labels/ # Label taxonomies and annotations
├── notebooks/ # Exploratory analysis and experiments
├── tests/ # Unit and integration tests
├── docs/ # Documentation
└── configs/ # Configuration files
- Python 3.9+
- scikit-learn
- numpy
- pandas
- scipy
See requirements.txt for complete dependencies.
git clone https://github.com/Sakeeb91/multi-label-doc-classifier.git
cd multi-label-doc-classifier
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txtfrom src.features.vectorizer import DocumentVectorizer
from src.models.classifier_chain import MultiLabelClassifierChain
from src.active_learning.uncertainty import UncertaintySampler
# Initialize components
vectorizer = DocumentVectorizer(max_features=5000, ngram_range=(1, 3))
classifier = MultiLabelClassifierChain(base_estimator='logistic_regression')
sampler = UncertaintySampler(strategy='entropy')
# Train with initial labeled data
X_train = vectorizer.fit_transform(documents)
classifier.fit(X_train, labels)
# Get next samples to label
samples_to_label = sampler.select_samples(classifier, unlabeled_pool, n_samples=10)See the docs/ directory for detailed documentation:
MIT License
Contributions are welcome. Please read the contributing guidelines before submitting pull requests.